Edgent Source Streams

Introduction

One of the first things you probably want to do is to bring data into your Edgent applications. The task is to create a source stream that contains the data that is to be processed and analyzed by Edgent.
Edgent provides a number of connectors providing source streams, such as IoTF, MQTT, Kafka, Files, HTTP etc, however for a specific device or application you may need to develop your own source streams.

Source Streams

To simplify the discussion we will describe these in terms of reading sensors, though each approach can apply to non-sensor data, such as polling an HTTP server for information.

Thus the goal is to produce a source stream where each tuple on the stream represents a reading from a sensor (or multiple sensors if the tuple object contains a sensor identifier).

Source streams are created using functions and there are three styles of bringing data into Edgent.

  1. Polling - periodically polling of a sensor's value.
  2. Blocking - a request is made to fetch data from a sensor
  3. Events - an event is created (by a non-Edgent framework) when the sensor changes

Polling Sources

To poll a sensor periodically an application provides a non-blocking Supplier function that reads the sensor's value and passes it to Topology.poll().

For example imagine a library provides a static method to read an engine's oil temperature.


    double oilTemp = EngineInfo.getOilTemperature();


To declare a stream in a topology that will result in the sensor being read every 100ms: an application uses a lambda expression as the function and passes it to poll():

    TStream<Double> oilTemps = topology.poll(() -> EngineInfo.getOilTemperature(), 100, TimeUnit.MILLISECONDS);

At runtime this results in a stream containing a new tuple approximately every 100ms set to the current value of the oil temperature (as a Double object).
A JSON source stream can be created that contains the sensor identifier in addition to the sensor reading. This allows multiple sensors to be present on the same stream (for example by merging a stream containing oil temperatures every 100ms with one containing coolant temperature polled every ten seconds), for partitioned downstream processing.
Here's an example of representing the same oil temperature as a JSON object:

    TStream<JsonObject> oilTemps = topology.poll(() ->
       {
          JsonObject j = new JsonObject();
          j.addProperty("id", "oilTemp");
          j.addProperty("value", EngineInfo.getOilTemperature());
          return j;
       }, 100, TimeUnit.MILLISECONDS);

Blocking Sources

Some sensors may be of the blocking style, a method is called that blocks until a new reading is available. Similar to the polling style an application provides a Supplier function that reads the sensor's value. The difference here is that the method can block waiting for a reading to become available. When a reading is available the method returns and the value is put on the returned stream.

The function is passed to Topology.generate() which at runtime will result in a dedicated thread that loops calling get(). Thus every time the function has a reading available it returns it which places the returned value on the stream and then is called again to wait for the next available reading.
With this example the function is similar to the polling example, the only difference is the call to get the sensor reading blocks :


    TStream<JsonObject> drillDepths = topology.generate(() ->
       {
          JsonObject j = new JsonObject();
          j.addProperty("id", "drillDepth");
          // blocks until the drill has completed a
          // requested move and returns its current depth.
          j.addProperty("value", Drill.getDepth());
          return j;
       });

Topology.source() is another method that supports a blocking source. This time the Supplier function passed in is called once to fetch the object that will be iterated over. The iterable object may be finite or infinite. In either case the next() method may block waiting to obtain the next value to place on the stream.

Event Sources

This is a typical style for sensors where a framework exists such that an application can register interest in receiving events or notifications when a sensor changes its value or state. This is typically performed by the application registering a callback or listener object that is called by the framework when the sensor changes.

Topology.events() is the method to create a source stream from events using a listener function. The Edgent application passes a Consumer function that will be called when the application starts.
The application's consumer function is passed a reference to a different eventSubmitter function provided by Edgent, this function (also a Consumer) is used by the callback to place tuples onto the stream (it submits the event to the stream).
Thus the application's function has to:

Here's an example of creating a stream of sensor events in Android.


   SensorManager mSensorManager = (SensorManager) getSystemService(Context.SENSOR_SERVICE);

   // Passes a Consumer that registers a SensorChangeEvents that 
   // puts the SensorEvent onto the stream using eventSubmitter
   // supplied by Edgent.
   
   // Note for clarity the application function is implemented as a lambda expression

   TStream<SensorEventt> tempSensor = topology.events(eventSubmitter ->
      mSensorManager.registerListener(
           new SensorChangeEvents(eventSubmitter),
           Sensor.TYPE_AMBIENT_TEMPERATURE,
           SensorManager.SENSOR_DELAY_NORMAL);

// ....

/**
 * Sensor event listener that submits sensor
 * change events as tuples using a Consumer.
 */
public class SensorChangeEvents implements SensorEventListener {
    private final Consumer<SensorEvent> eventSubmitter;
    
    public SensorChangeEvents(Consumer<SensorEvent> eventSubmitter) {
        this.eventSubmitter = eventSubmitter;
    }
    @Override
    public void onSensorChanged(SensorEvent event) {
        // Submit the event directly to the stream
        eventSubmitter.accept(event);
    }

    @Override
    public void onAccuracyChanged(Sensor sensor, int accuracy) {
    }
}