org.apache.pig.backend.hadoop.executionengine.mapReduceLayer
Class PigMapReduce.Reduce
java.lang.Object
org.apache.hadoop.mapreduce.Reducer<PigNullableWritable,NullableTuple,PigNullableWritable,org.apache.hadoop.io.Writable>
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce.Reduce
- Direct Known Subclasses:
- PigMapReduce.ReduceWithComparator
- Enclosing class:
- PigMapReduce
public static class PigMapReduce.Reduce
- extends org.apache.hadoop.mapreduce.Reducer<PigNullableWritable,NullableTuple,PigNullableWritable,org.apache.hadoop.io.Writable>
Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.Reducer |
org.apache.hadoop.mapreduce.Reducer.Context |
Method Summary |
protected void |
cleanup(org.apache.hadoop.mapreduce.Reducer.Context context)
Will be called once all the intermediate keys and values are
processed. |
boolean |
processOnePackageOutput(org.apache.hadoop.mapreduce.Reducer.Context oc)
|
protected void |
reduce(PigNullableWritable key,
Iterable<NullableTuple> tupIter,
org.apache.hadoop.mapreduce.Reducer.Context context)
The reduce function which packages the key and List<Tuple>
into key, Bag<Tuple> after converting Hadoop type key into Pig type. |
protected void |
runPipeline(PhysicalOperator leaf)
|
protected void |
setup(org.apache.hadoop.mapreduce.Reducer.Context context)
Configures the Reduce plan, the POPackage operator
and the reporter thread |
Methods inherited from class org.apache.hadoop.mapreduce.Reducer |
run |
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
log
protected final org.apache.commons.logging.Log log
rp
protected PhysicalPlan rp
stores
protected List<POStore> stores
pack
protected POPackage pack
outputCollector
protected org.apache.hadoop.mapreduce.Reducer.Context outputCollector
errorInReduce
protected boolean errorInReduce
initialized
protected volatile boolean initialized
PigMapReduce.Reduce
public PigMapReduce.Reduce()
setup
protected void setup(org.apache.hadoop.mapreduce.Reducer.Context context)
throws IOException,
InterruptedException
- Configures the Reduce plan, the POPackage operator
and the reporter thread
- Overrides:
setup
in class org.apache.hadoop.mapreduce.Reducer<PigNullableWritable,NullableTuple,PigNullableWritable,org.apache.hadoop.io.Writable>
- Throws:
IOException
InterruptedException
reduce
protected void reduce(PigNullableWritable key,
Iterable<NullableTuple> tupIter,
org.apache.hadoop.mapreduce.Reducer.Context context)
throws IOException,
InterruptedException
- The reduce function which packages the key and List<Tuple>
into key, Bag<Tuple> after converting Hadoop type key into Pig type.
The package result is either collected as is, if the reduce plan is
empty or after passing through the reduce plan.
- Overrides:
reduce
in class org.apache.hadoop.mapreduce.Reducer<PigNullableWritable,NullableTuple,PigNullableWritable,org.apache.hadoop.io.Writable>
- Throws:
IOException
InterruptedException
processOnePackageOutput
public boolean processOnePackageOutput(org.apache.hadoop.mapreduce.Reducer.Context oc)
throws IOException,
InterruptedException
- Throws:
IOException
InterruptedException
runPipeline
protected void runPipeline(PhysicalOperator leaf)
throws InterruptedException,
IOException
- Parameters:
leaf
-
- Throws:
InterruptedException
IOException
cleanup
protected void cleanup(org.apache.hadoop.mapreduce.Reducer.Context context)
throws IOException,
InterruptedException
- Will be called once all the intermediate keys and values are
processed. So right place to stop the reporter thread.
- Overrides:
cleanup
in class org.apache.hadoop.mapreduce.Reducer<PigNullableWritable,NullableTuple,PigNullableWritable,org.apache.hadoop.io.Writable>
- Throws:
IOException
InterruptedException
Copyright © ${year} The Apache Software Foundation