org.apache.pig.backend.hadoop.executionengine.mapReduceLayer
Class PigMapReduce.Reduce

java.lang.Object
  extended by org.apache.hadoop.mapred.MapReduceBase
      extended by org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce.Reduce
All Implemented Interfaces:
Closeable, org.apache.hadoop.mapred.JobConfigurable, org.apache.hadoop.mapred.Reducer<PigNullableWritable,NullableTuple,PigNullableWritable,org.apache.hadoop.io.Writable>
Direct Known Subclasses:
PigMapReduce.ReduceWithComparator
Enclosing class:
PigMapReduce

public static class PigMapReduce.Reduce
extends org.apache.hadoop.mapred.MapReduceBase
implements org.apache.hadoop.mapred.Reducer<PigNullableWritable,NullableTuple,PigNullableWritable,org.apache.hadoop.io.Writable>


Field Summary
protected  boolean errorInReduce
           
protected  boolean initialized
           
protected  org.apache.commons.logging.Log log
           
protected  org.apache.hadoop.mapred.OutputCollector<PigNullableWritable,org.apache.hadoop.io.Writable> outputCollector
           
protected  POPackage pack
           
protected  PhysicalPlan rp
           
protected  List<POStore> stores
           
 
Constructor Summary
PigMapReduce.Reduce()
           
 
Method Summary
 void close()
          Will be called once all the intermediate keys and values are processed.
 void configure(org.apache.hadoop.mapred.JobConf jConf)
          Configures the Reduce plan, the POPackage operator and the reporter thread
 boolean processOnePackageOutput(org.apache.hadoop.mapred.OutputCollector<PigNullableWritable,org.apache.hadoop.io.Writable> oc)
           
 void reduce(PigNullableWritable key, Iterator<NullableTuple> tupIter, org.apache.hadoop.mapred.OutputCollector<PigNullableWritable,org.apache.hadoop.io.Writable> oc, org.apache.hadoop.mapred.Reporter reporter)
          The reduce function which packages the key and List<Tuple> into key, Bag<Tuple> after converting Hadoop type key into Pig type.
protected  void runPipeline(PhysicalOperator leaf)
           
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

log

protected final org.apache.commons.logging.Log log

rp

protected PhysicalPlan rp

stores

protected List<POStore> stores

pack

protected POPackage pack

outputCollector

protected org.apache.hadoop.mapred.OutputCollector<PigNullableWritable,org.apache.hadoop.io.Writable> outputCollector

errorInReduce

protected boolean errorInReduce

initialized

protected volatile boolean initialized
Constructor Detail

PigMapReduce.Reduce

public PigMapReduce.Reduce()
Method Detail

configure

public void configure(org.apache.hadoop.mapred.JobConf jConf)
Configures the Reduce plan, the POPackage operator and the reporter thread

Specified by:
configure in interface org.apache.hadoop.mapred.JobConfigurable
Overrides:
configure in class org.apache.hadoop.mapred.MapReduceBase

reduce

public void reduce(PigNullableWritable key,
                   Iterator<NullableTuple> tupIter,
                   org.apache.hadoop.mapred.OutputCollector<PigNullableWritable,org.apache.hadoop.io.Writable> oc,
                   org.apache.hadoop.mapred.Reporter reporter)
            throws IOException
The reduce function which packages the key and List<Tuple> into key, Bag<Tuple> after converting Hadoop type key into Pig type. The package result is either collected as is, if the reduce plan is empty or after passing through the reduce plan.

Specified by:
reduce in interface org.apache.hadoop.mapred.Reducer<PigNullableWritable,NullableTuple,PigNullableWritable,org.apache.hadoop.io.Writable>
Throws:
IOException

processOnePackageOutput

public boolean processOnePackageOutput(org.apache.hadoop.mapred.OutputCollector<PigNullableWritable,org.apache.hadoop.io.Writable> oc)
                                throws IOException
Throws:
IOException

runPipeline

protected void runPipeline(PhysicalOperator leaf)
                    throws ExecException,
                           IOException
Parameters:
leaf -
Throws:
ExecException
IOException

close

public void close()
           throws IOException
Will be called once all the intermediate keys and values are processed. So right place to stop the reporter thread.

Specified by:
close in interface Closeable
Overrides:
close in class org.apache.hadoop.mapred.MapReduceBase
Throws:
IOException


Copyright © ${year} The Apache Software Foundation