public class CommandStreams
extends java.lang.Object
e.g., run a network monitor command (like Tiger Shark) and ingest its output.
Modifier and Type | Method and Description |
---|---|
static Supplier<java.util.List<java.lang.String>> |
commandReaderList(java.lang.ProcessBuilder cmd)
Create a
Supplier<List<String>> to ingest a command's output. |
static Consumer<java.lang.String> |
commandWriter(java.lang.ProcessBuilder cmd)
Create a
Consumer<String> to write UTF8 string data to a command's input. |
static Supplier<java.lang.String> |
endlessCommandReader(java.lang.ProcessBuilder cmd)
Create an endless
Supplier<String> for ingesting a long running command's output. |
static TStream<java.lang.String> |
generate(Topology topology,
java.lang.ProcessBuilder cmd)
Create an endless
TStream<String> from a long running command's output. |
static TStream<java.util.List<java.lang.String>> |
periodicSource(Topology topology,
java.lang.ProcessBuilder cmd,
long period,
java.util.concurrent.TimeUnit units)
Create a
TStream<String> from a periodically run command's output. |
static TSink<java.lang.String> |
sink(TStream<java.lang.String> stream,
java.lang.ProcessBuilder cmd)
Sink a
TStream<String> to a command's input. |
static java.util.List<java.lang.String> |
tokenize(java.lang.String cmdString)
Tokenize the specified
cmdString in the exact same manner as
done by Runtime.exec(String) . |
public static java.util.List<java.lang.String> tokenize(java.lang.String cmdString)
cmdString
in the exact same manner as
done by Runtime.exec(String)
.
This function provides a convenience for creating a ProcessBuilder
for use by the other CommandStreams methods.
Sample use:
ProcessBuilder cmd = new ProcessBuilder(tokenize("sh someShellCmd.sh and args"));
TStream<String> stream = CommandStreams.generate(topology, cmd);
cmdString
- a command stringpublic static TStream<java.lang.String> generate(Topology topology, java.lang.ProcessBuilder cmd)
TStream<String>
from a long running command's output.
The supplied cmd
is used to start the command.
A tuple is created for each UTF8 line read from the command's
output
.
The tuples contain output from stderr if cmd is configured to
redirect stderr to stdout
.
The command is restarted if a read from the command's output stream
returns EOF or an error.
This is a convenience function equivalent to
topology.generate(endlessCommandReader(cmd))
.
Sample use: create a stream of tuples for the output from a continuously running and restartable command:
ProcessBuilder cmd = new ProcessBuilder("myCommand");
TStream<String> cmdOutput = CommandStreams.generate(topology, cmd);
cmdOutput.print();
topology
- the topology to add the source stream tocmd
- the ProcessBuilder
to start the commandTStream<String>
endlessCommandReader(ProcessBuilder)
,
tokenize(String)
public static TStream<java.util.List<java.lang.String>> periodicSource(Topology topology, java.lang.ProcessBuilder cmd, long period, java.util.concurrent.TimeUnit units)
TStream<String>
from a periodically run command's output.
The supplied cmd
is used to start the command
at the specified period
.
The command's UTF8 output
is read until EOF
and a List<String>
tuple is created containing the collected output.
The tuples contain output from stderr if the cmd is configured to
redirect stderr to stdout
.
This is a convenience function equivalent to
topology.poll(commandReaderList(cmd), period, units)
.
Sample use: create a stream of tuples containing the output from a periodically run command:
ProcessBuilder cmd = new ProcessBuilder("date");
TStream<List<String>> cmdOutput =
CommandStreams.periodicSource(topology, cmd, 2, TimeUnit.SECONDS);
cmdOutput.print();
topology
- the topology to add the source stream tocmd
- the ProcessBuilder
to start the commandperiod
- the period to run the command and collect its outputunits
- TimeUnit for period
TStream<List<String>>
commandReaderList(ProcessBuilder)
,
tokenize(String)
public static TSink<java.lang.String> sink(TStream<java.lang.String> stream, java.lang.ProcessBuilder cmd)
TStream<String>
to a command's input.
The supplied cmd
is used to start the command.
Each tuple is written as UTF8 and flushed to the command's input
.
The command is restarted if a write encounters an error.
While each write is followed by a flush() that only helps to reduce the time it takes to notice that cmd has failed and restart it. Supposedly "successfully written and flushed" values are not guaranteed to have been received by a cmd across restarts.
This is a convenience function equivalent to
stream.sink(commandWriter(cmd))
Sample use: write a stream of tuples to the input of a command:
TStream<String> stream = topology.strings("one", "two", "three");
ProcessBuilder cmd = new ProcessBuilder("cat").redirectOutput(new File("/dev/stdout"));
CommandStreams.sink(stream, cmd);
stream
- the stream to sinkcmd
- the ProcessBuilder
to start the commandTSink
commandWriter(ProcessBuilder)
,
tokenize(String)
public static Supplier<java.lang.String> endlessCommandReader(java.lang.ProcessBuilder cmd)
Supplier<String>
for ingesting a long running command's output.
This method is particularly helpful in creating a sensor or source connector class that hides the fact that it uses a command, enabling it to be used like any other sensor/connector.
For example:
// ingest the sensor data
TStream<MySensorData> stream = topology.generate(new MySensor());
// MySensor class
class MySensor implements Supplier<MySensorData> {
private String[] cmd = new String[] {"mySensorCmd", "arg1"};
private Supplier<String> commandReader =
CommandStreams.endlessCommandReader(new ProcessBuilder(cmd));
// implement Supplier<MySensorData>.get()
public MySensorData get() {
// get the next line from the cmd and create a MySensorData tuple from it
return createMySensorData(commandReader.get());
}
}
The supplied cmd
is used to start the command.
A call to Supplier.get()
reads the next UTF8 line from the command's
output
.
The returned strings contain output from stderr if the cmd is configured to
redirect stderr to stdput
.
The command is restarted if a read from the command's output stream
returns EOF or an error.
cmd
- the ProcessBuilder
to start the commandSupplier<String>
generate(Topology, ProcessBuilder)
,
tokenize(String)
public static Supplier<java.util.List<java.lang.String>> commandReaderList(java.lang.ProcessBuilder cmd)
Supplier<List<String>>
to ingest a command's output.
This method is particularly helpful in creating a sensor or source connector class that hides the fact that it uses a command, enabling it to be used like any other sensor/connector.
For example:
// ingest the sensor data
TStream<MySensorData> stream = topology.periodicSource(new MySensor());
// MySensor class
class MySensor implements Supplier<MySensorData> {
private String[] cmd = new String[] {"mySensorCmd", "arg1"};
private Supplier<List<String>> commandReader =
CommandStreams.commandReaderList(new ProcessBuilder(cmd));
// implement Supplier<MySensorData>.get()
public MySensorData get() {
// get the cmd output and create a MySensorData tuple from it
return createMySensorData(commandReader.get());
}
}
The supplied cmd
is used to start the command.
A call to Supplier.get()
reads the command's UTF8
input stream
until an EOF or error
and returns a List<String>
of the collected input.
The tuples contain output from stderr if the cmd is configured to
redirect stderr to stdout
.
cmd
- the ProcessBuilder
to start the commandSupplier<List<String>>
for the commandperiodicSource(Topology, ProcessBuilder, long, TimeUnit)
,
tokenize(String)
public static Consumer<java.lang.String> commandWriter(java.lang.ProcessBuilder cmd)
Consumer<String>
to write UTF8 string data to a command's input.
This method is particularly helpful in creating a sink connector that hides the fact that it uses a command, enabling it to be used like a native connector.
For example:
// sink a stream to my connector
TStream<MySensorData> stream = ...;
stream.sink(new MySinkConnector());
// MySinkConnector class
class MySinkConnector implements Consumer<MySensorData> {
private String[] cmd = new String[] {"mySinkCmd", "arg1"};
private Consumer<String> commandWriter =
CommandStreams.commandWriter(new ProcessBuilder(cmd));
// implement Consumer<MySensorData>.accept()
public void accept(MySensorData data) {
// convert the data to a string and write it to the cmd
commandWriter.accept(convertMySensorData(data));
}
}
The supplied cmd
is used to start the command.
Each call to accept(String)
writes a
UTF8 string to the command's input
.
Each write is followed by a flush.
The command is restarted if a write encounters an error.
While each write is followed by a flush() that only helps to reduce the time it takes to notice that cmd has failed and restart it. Supposedly "successfully written and flushed" values are not guaranteed to have been received by a cmd across restarts.
cmd
- the ProcessBuilder
to start the commandConsumer<String>
for the commandsink(TStream, ProcessBuilder)
,
tokenize(String)
Copyright © 2016 The Apache Software Foundation. All Rights Reserved - bbe71fa-20161201-1641