Edgent Source Streams
Introduction
One of the first things you probably want to do is to bring data into your Edgent applications. The task is to create a source stream that contains the data that is to be processed and analyzed by Edgent.Edgent provides a number of connectors providing source streams, such as IoTF, MQTT, Kafka, Files, HTTP etc, however for a specific device or application you may need to develop your own source streams.
Source Streams
To simplify the discussion we will describe these in terms of reading sensors, though each approach can apply to non-sensor data, such as polling an HTTP server for information.Thus the goal is to produce a source stream where each tuple on the stream represents a reading from a sensor (or multiple sensors if the tuple object contains a sensor identifier).
Source streams are created using functions and there are three styles of bringing data into Edgent.
- Polling - periodically polling of a sensor's value.
- Blocking - a request is made to fetch data from a sensor
- Events - an event is created (by a non-Edgent framework) when the sensor changes
Polling Sources
To poll a sensor periodically an application provides a non-blocking Supplier function that reads the sensor's value and passes it to Topology.poll().For example imagine a library provides a static method to read an engine's oil temperature.
double oilTemp = EngineInfo.getOilTemperature();
To declare a stream in a topology that will result in the sensor being read every 100ms: an application uses a lambda expression as the function and passes it to
poll()
:
TStream<Double> oilTemps = topology.poll(() -> EngineInfo.getOilTemperature(), 100, TimeUnit.MILLISECONDS);
At runtime this results in a stream containing a new tuple approximately every 100ms
set to the current value of the oil temperature (as a Double object).
A JSON source stream can be created that contains the sensor identifier in addition to the sensor reading. This allows multiple sensors to be present on the same stream (for example by merging a stream containing oil temperatures every 100ms with one containing coolant temperature polled every ten seconds), for partitioned downstream processing.
Here's an example of representing the same oil temperature as a JSON object:
TStream<JsonObject> oilTemps = topology.poll(() ->
{
JsonObject j = new JsonObject();
j.addProperty("id", "oilTemp");
j.addProperty("value", EngineInfo.getOilTemperature());
return j;
}, 100, TimeUnit.MILLISECONDS);
Blocking Sources
Some sensors may be of the blocking style, a method is called that blocks until a new reading is available. Similar to the polling style an application provides a Supplier function that reads the sensor's value. The difference here is that the method can block waiting for a reading to become available. When a reading is available the method returns and the value is put on the returned stream.
The function is passed to
Topology.generate()
which at runtime will result in a dedicated thread that loops calling get()
.
Thus every time the function has a reading available it returns it which places the returned value
on the stream and then is called again to wait for the next available reading.
With this example the function is similar to the polling example, the only difference is the
call to get the sensor reading blocks :
TStream<JsonObject> drillDepths = topology.generate(() ->
{
JsonObject j = new JsonObject();
j.addProperty("id", "drillDepth");
// blocks until the drill has completed a
// requested move and returns its current depth.
j.addProperty("value", Drill.getDepth());
return j;
});
Topology.source() is another method
that supports a blocking source. This time the Supplier
function passed in is called once
to fetch the object that will be iterated over. The iterable object may be finite or infinite. In either
case the next()
method may block waiting to obtain the next value to place on the stream.
Event Sources
This is a typical style for sensors where a framework exists such that an application can register interest in receiving events or notifications when a sensor changes its value or state. This is typically performed by the application registering a callback or listener object that is called by the framework when the sensor changes.
Topology.events() is the method to create a source stream from events using a listener function. The Edgent application passes a Consumer function that will be called when the application starts.The application's consumer function is passed a reference to a different
eventSubmitter
function provided by Edgent, this function (also a Consumer
) is used by the callback
to place tuples onto the stream (it submits the event to the stream).
Thus the application's function has to:
- Register a callback with the sensor framework that will be called when the sensor changes.
- Have the callback convert the sensor change event information into the tuple type it wants to send on the stream, for example a JSON object. In some cases the raw event object is used as the stream's tuple.
- Execute the function provided by Edgent calling
eventSubmitter.accept(tuple)
to place the tuple onto the stream.
Here's an example of creating a stream of sensor events in Android.
SensorManager mSensorManager = (SensorManager) getSystemService(Context.SENSOR_SERVICE);
// Passes a Consumer that registers a SensorChangeEvents that
// puts the SensorEvent onto the stream using eventSubmitter
// supplied by Edgent.
// Note for clarity the application function is implemented as a lambda expression
TStream<SensorEventt> tempSensor = topology.events(eventSubmitter ->
mSensorManager.registerListener(
new SensorChangeEvents(eventSubmitter),
Sensor.TYPE_AMBIENT_TEMPERATURE,
SensorManager.SENSOR_DELAY_NORMAL);
// ....
/**
* Sensor event listener that submits sensor
* change events as tuples using a Consumer.
*/
public class SensorChangeEvents implements SensorEventListener {
private final Consumer<SensorEvent> eventSubmitter;
public SensorChangeEvents(Consumer<SensorEvent> eventSubmitter) {
this.eventSubmitter = eventSubmitter;
}
@Override
public void onSensorChanged(SensorEvent event) {
// Submit the event directly to the stream
eventSubmitter.accept(event);
}
@Override
public void onAccuracyChanged(Sensor sensor, int accuracy) {
}
}