public class MapReducePlan extends Plan
Plan.MRContainerKeyComparator
cache, conf, counter_container, counter_key, max_input_files, temporary_paths, value_container
Constructor and Description |
---|
MapReducePlan() |
Modifier and Type | Method and Description |
---|---|
static MRData |
aggregate(Tree acc_fnc,
Tree zero,
DataSet S)
The Aggregate physical operator
|
static DataSet |
closure(Function loop,
DataSet init,
int max_num)
The closure physical operator.
|
static long |
outputRecords(org.apache.hadoop.mapreduce.Job job)
find the number of records in the hadoop MapReduce job output
|
static DataSet |
repeat(Function loop,
DataSet init,
int max_num)
The repeat physical operator.
|
static void |
setupSplits(DataSet[] dsv,
org.apache.hadoop.conf.Configuration conf)
Set hadoop map min and max split size based on number of requested nodes
|
static void |
setupSplits(DataSet ds,
org.apache.hadoop.conf.Configuration conf)
Set hadoop map min and max split size based on number of requested nodes
|
, binarySource, binarySource, clean, collect, collect, distribute_compiled_arguments, fileCache, functional_argument, generator, generator, getCache, merge, merge, new_path, parsedSource, parsedSource, print_stream, setCache, size
public static final long outputRecords(org.apache.hadoop.mapreduce.Job job) throws Exception
Exception
public static void setupSplits(DataSet[] dsv, org.apache.hadoop.conf.Configuration conf) throws IOException
IOException
public static void setupSplits(DataSet ds, org.apache.hadoop.conf.Configuration conf) throws IOException
IOException
public static final MRData aggregate(Tree acc_fnc, Tree zero, DataSet S) throws Exception
acc_fnc
- the accumulator function from (T,T) to Tzero
- the zero element of type TS
- the dataset that contains the bag of values {T}Exception
public static final DataSet repeat(Function loop, DataSet init, int max_num)
loop
- the function from DataSet to DataSet to be repeatedinit
- the initial input DataSet for the loopmax_num
- max number of repetitionspublic static final DataSet closure(Function loop, DataSet init, int max_num)
loop
- the function from DataSet to DataSet to be repeatedinit
- the initial input DataSet for the loopmax_num
- max number of repetitionsCopyright © 2013-2015 The Apache Software Foundation. All Rights Reserved.