public class JsonAnalytics
extends java.lang.Object
Constructor and Description |
---|
JsonAnalytics() |
Modifier and Type | Method and Description |
---|---|
static <K extends com.google.gson.JsonElement> |
aggregate(TWindow<com.google.gson.JsonObject,K> window,
java.lang.String resultPartitionProperty,
java.lang.String valueProperty,
JsonUnivariateAggregate... aggregates)
Aggregate against a single
Numeric variable contained in an JSON object. |
static <K extends com.google.gson.JsonElement> |
aggregate(TWindow<com.google.gson.JsonObject,K> window,
java.lang.String resultPartitionProperty,
java.lang.String resultProperty,
ToDoubleFunction<com.google.gson.JsonObject> valueGetter,
JsonUnivariateAggregate... aggregates)
Aggregate against a single
Numeric variable contained in an JSON object. |
static <K extends com.google.gson.JsonElement> |
aggregateList(java.lang.String resultPartitionProperty,
java.lang.String resultProperty,
ToDoubleFunction<com.google.gson.JsonObject> valueGetter,
JsonUnivariateAggregate... aggregates)
Create a Function that aggregates against a single
Numeric
variable contained in an JSON object. |
static com.google.gson.JsonElement |
getMvAggregate(com.google.gson.JsonObject jo,
java.lang.String resultProperty,
java.lang.String variableName,
JsonUnivariateAggregate aggregate)
Get the value of an aggregate computed by a multi-variable aggregation.
|
static boolean |
hasMvAggregate(com.google.gson.JsonObject jo,
java.lang.String resultProperty,
java.lang.String variableName,
JsonUnivariateAggregate aggregate)
Check if an aggregation result from a multi-variable aggregation
is present.
|
static org.apache.commons.math3.util.Pair<java.lang.String,JsonUnivariateAggregate[]> |
mkAggregationSpec(java.lang.String variableName,
JsonUnivariateAggregate... aggregates)
Create an aggregation specification.
|
static <K extends com.google.gson.JsonElement> |
mvAggregate(TWindow<com.google.gson.JsonObject,K> window,
java.lang.String resultPartitionKeyProperty,
java.lang.String resultProperty,
java.util.List<org.apache.commons.math3.util.Pair<java.lang.String,JsonUnivariateAggregate[]>> aggregateSpecs)
Aggregate against multiple
Numeric variables contained in an JSON object. |
static <K extends com.google.gson.JsonElement> |
mvAggregateList(java.lang.String resultPartitionKeyProperty,
java.lang.String resultProperty,
java.util.List<org.apache.commons.math3.util.Pair<java.lang.String,JsonUnivariateAggregate[]>> aggregateSpecs)
Create a Function that aggregates multiple
Numeric
variables contained in an JSON object. |
public static <K extends com.google.gson.JsonElement> TStream<com.google.gson.JsonObject> aggregate(TWindow<com.google.gson.JsonObject,K> window, java.lang.String resultPartitionProperty, java.lang.String valueProperty, JsonUnivariateAggregate... aggregates)
Numeric
variable contained in an JSON object.
The returned stream contains a tuple for each execution performed against a window partition.
The tuple is a JsonObject
containing:
K
as a property with key resultPartitionProperty
. JsonObject
as a property with key valueProperty
.
This results object contains the results of all aggregations defined by aggregates
against
double
property with key valueProperty
.
JsonUnivariateAggregate
declares how it represents its aggregation in this result
object.
For example if the window contains these three tuples (pseudo JSON) for
partition 3:
{id=3,reading=2.0}, {id=3,reading=2.6}, {id=3,reading=1.8}
the resulting aggregation for the stream returned by:
aggregate(window, "id", "reading", Statistic.MIN, Statistic.MAX)
would contain this tuple with the maximum and minimum values in the reading
JSON object:
{id=3, reading={MIN=1.8, MAX=1.8}}
K
- Partition typewindow
- Window to aggregate over.resultPartitionProperty
- Property to store the partition key in tuples on the returned stream.valueProperty
- JSON property containing the value to aggregate.aggregates
- Which aggregations to be performed.public static <K extends com.google.gson.JsonElement> TStream<com.google.gson.JsonObject> aggregate(TWindow<com.google.gson.JsonObject,K> window, java.lang.String resultPartitionProperty, java.lang.String resultProperty, ToDoubleFunction<com.google.gson.JsonObject> valueGetter, JsonUnivariateAggregate... aggregates)
Numeric
variable contained in an JSON object.
The returned stream contains a tuple for each execution performed against a window partition.
The tuple is a JsonObject
containing:
K
as a property with key resultPartitionProperty
. JsonObject
as a property with key resultProperty
.
This results object contains the results of all aggregations defined by aggregates
against
value returned by valueGetter
.
JsonUnivariateAggregate
declares how it represents its aggregation in this result
object.
K
- Partition typewindow
- Window to aggregate over.resultPartitionProperty
- Property to store the partition key in tuples on the returned stream.resultProperty
- Property to store the aggregations in tuples on the returned stream.valueGetter
- How to obtain the single variable from input tuples.aggregates
- Which aggregations to be performed.public static <K extends com.google.gson.JsonElement> BiFunction<java.util.List<com.google.gson.JsonObject>,K,com.google.gson.JsonObject> aggregateList(java.lang.String resultPartitionProperty, java.lang.String resultProperty, ToDoubleFunction<com.google.gson.JsonObject> valueGetter, JsonUnivariateAggregate... aggregates)
Numeric
variable contained in an JSON object.
Calling apply(List<JsonObject>)
on the returned BiFunction
returns a JsonObject
containing:
K
as a property with key resultPartitionProperty
. JsonObject
as a property with key valueProperty
.
This results object contains the results of all aggregations defined by aggregates
against the value returned by valueGetter
.
JsonUnivariateAggregate
declares how it represents its aggregation in this result
object.
For example if the list contains these three tuples (pseudo JSON) for
partition 3:
{id=3,reading=2.0}, {id=3,reading=2.6}, {id=3,reading=1.8}
the resulting aggregation for the JsonObject returned by:
aggregateList("id", "reading", Statistic.MIN, Statistic.MAX).apply(list, 3)
would be this tuple with the maximum and minimum values in the reading
JSON object:
{id=3, reading={MIN=1.8, MAX=1.8}}
K
- Partition typeresultPartitionProperty
- Property to store the partition key in tuples on the returned stream.resultProperty
- Property to store the aggregations in the returned JsonObject.valueGetter
- How to obtain the single variable from input tuples.aggregates
- Which aggregations to be performed.public static <K extends com.google.gson.JsonElement> TStream<com.google.gson.JsonObject> mvAggregate(TWindow<com.google.gson.JsonObject,K> window, java.lang.String resultPartitionKeyProperty, java.lang.String resultProperty, java.util.List<org.apache.commons.math3.util.Pair<java.lang.String,JsonUnivariateAggregate[]>> aggregateSpecs)
Numeric
variables contained in an JSON object.
This is a multi-variable analog of aggregate()
See mvAggregateList()
for
a description of the aggregation processing and result stream.
Sample use:
// Ingest the data. The JsonObject tuples have properties:
// "id" - the partitionKey
// "tx" - a numeric data variable
// "rx" - a numeric data variable
TStream<JsonObject> ingestData = ...
// Define the tuple variables and their aggregations to compute
List<Pair<String, JsonUnivariateAggregate[]>> aggSpecs = new ArrayList<>();
aggSpecs.add(mkAggregationSpec("tx", Statistics.MIN, Statistics.MAX));
aggSpecs.add(mkAggregationSpec("rx", Statistics.MEAN));
// Create the window over which to aggregate
TWindow<JsonObject, JsonElement> window =
ingestData.last(5, TimeUnit.SECONDS, jo -> jo.get("id"));
// Create a stream with the aggregations. The result tuples have properties:
// "id" - the partitionKey
// "aggResults" - the aggregation results
TStream<JsonObject> aggResults =
mvAggregate(window, "id", "aggResults", aggSpecs);
// Create a stream of JsonObject tuples with just the average "rx"
TStream<JsonObject> avgRx = aggResults.map(
jo -> {
JsonObject result = new JsonObject();
result.add("id", jo.get("id"))
result.add("avgRx", getMvAggregate(jo, "aggResults", "Rx", Statistic.MEAN);
return result;
});
window
- the window to compute aggregations overresultPartitionKeyProperty
- name of the partition key property in the resultresultProperty
- name of the aggregation results property in the resultaggregateSpecs
- see mkAggregationSpec()
mvAggregateList()
,
mkAggregationSpec()
,
getMvAggregate()
public static org.apache.commons.math3.util.Pair<java.lang.String,JsonUnivariateAggregate[]> mkAggregationSpec(java.lang.String variableName, JsonUnivariateAggregate... aggregates)
The aggregation specification specifies a variable name and the aggregates to compute on it.
The specification can be use with mkAggregateList()
variableName
- the name of a Numeric
data variable in a JSON objectaggregates
- the aggregates to compute for the variablepublic static <K extends com.google.gson.JsonElement> BiFunction<java.util.List<com.google.gson.JsonObject>,K,com.google.gson.JsonObject> mvAggregateList(java.lang.String resultPartitionKeyProperty, java.lang.String resultProperty, java.util.List<org.apache.commons.math3.util.Pair<java.lang.String,JsonUnivariateAggregate[]>> aggregateSpecs)
Numeric
variables contained in an JSON object.
This is a multi-variable analog of aggregateList()
The overall multi-variable aggregation result is a JSON object with properties:
resultPartionKeyProperty
whose value is the tuple's partition key
resultProperty
whose value is a JSON object containing
a property for each variable aggregation. The property names
correspond to the variable names from the aggregateSpecs
and the values are the aggregation results for the variable.
The aggregation results for a variable are a JSON object
having a property for each aggregation name and its value.
For example if the list contains these three tuples (pseudo JSON) for
partition 3:
{id=3,tx=2.0,rx=1.0,...}, {id=3,tx=2.6,rx=2.0,...}, {id=3,tx=1.8,rx=3.0,...}
the resulting aggregation JsonObject returned is:
{id=3, aggData={tx={MIN=1.8, MAX=2.6}, rx={MEAN=2.0}}}
for the invocation:
mvAggregateList("id", "aggData", aggSpecs).apply(list, 3))
where aggSpecs
is:
aggSpecs.add(mkAggregationSpec("tx", Statistics.MIN, Statistics.MAX));
aggSpecs.add(mkAggregationSpec("rx", Statistics.MEAN));
getMvAggregate()
can be used to extract individual aggregate values from the result.
K
- Partition Key as a JsonElementresultPartitionKeyProperty
- name of the partition key property in the resultresultProperty
- name of the aggregation results property in the resultaggregateSpecs
- see mkAggregationSpec()
mkAggregationSpec()
,
getMvAggregate()
public static com.google.gson.JsonElement getMvAggregate(com.google.gson.JsonObject jo, java.lang.String resultProperty, java.lang.String variableName, JsonUnivariateAggregate aggregate)
This convenience method can be used to extract information from a JSON object
created by mvAggregationList()
or mvAggregate()
Sample use:
...
TStream<JsonObject> aggData = mvAggregate(window, "id", "aggResults", aggSpecs);
// Create a stream of JsonObject tuples with just the average "tx"
TStream<JsonObject> avgTx = aggResults.map(
jo -> {
JsonObject result = new JsonObject();
result.add(partitionKeyName, jo.get(partitionKeyName))
result.add("avgTx", getMvAggregate(jo, "aggResults", "tx", Statistic.MEAN);
return result;
});
jo
- a JSON object created by mvAggregationList
resultProperty
- the corresponding value passed to mvAggragateList
variableName
- the data variable of interest in the multivariable aggregatesaggregate
- the variable's aggregate of interestjava.lang.RuntimeException
- if the aggregate isn't present in the resulthasAggregate()
,
mvAggregate()
,
mvAggregateList()
public static boolean hasMvAggregate(com.google.gson.JsonObject jo, java.lang.String resultProperty, java.lang.String variableName, JsonUnivariateAggregate aggregate)
jo
- a JSON object created by mvAggregationList
resultProperty
- the corresponding value passed to mvAggragateList
variableName
- the data variable of interest in the multivariable aggregatesaggregate
- the variable's aggregate of interestgetMvAggregate()
Copyright © 2016 The Apache Software Foundation. All Rights Reserved - bbe71fa-20161201-1641