org.apache.pig.backend.hadoop.executionengine.mapReduceLayer
Class PigMapReduce.ReduceWithComparator
java.lang.Object
org.apache.hadoop.mapred.MapReduceBase
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce.Reduce
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce.ReduceWithComparator
- All Implemented Interfaces:
- Closeable, org.apache.hadoop.mapred.JobConfigurable, org.apache.hadoop.mapred.Reducer<PigNullableWritable,NullableTuple,PigNullableWritable,org.apache.hadoop.io.Writable>
- Enclosing class:
- PigMapReduce
public static class PigMapReduce.ReduceWithComparator
- extends PigMapReduce.Reduce
This "specialized" reduce class is ONLY to be used in pig queries with
order by a udf. A UDF used for comparison in the order by expects
to be handed tuples. Hence a specialized map class (PigMapReduce.MapWithComparator)
ensures that the "key" used in the order by is wrapped into a tuple (if it
isn't already a tuple). This reduce class unwraps this tuple in the case where
the map had wrapped into a tuple and handes the "unwrapped" key to the POPackage
for processing
Method Summary |
void |
configure(org.apache.hadoop.mapred.JobConf jConf)
Configures the Reduce plan, the POPackage operator
and the reporter thread |
void |
reduce(PigNullableWritable key,
Iterator<NullableTuple> tupIter,
org.apache.hadoop.mapred.OutputCollector<PigNullableWritable,org.apache.hadoop.io.Writable> oc,
org.apache.hadoop.mapred.Reporter reporter)
The reduce function which packages the key and List<Tuple>
into key, Bag<Tuple> after converting Hadoop type key into Pig type. |
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
PigMapReduce.ReduceWithComparator
public PigMapReduce.ReduceWithComparator()
configure
public void configure(org.apache.hadoop.mapred.JobConf jConf)
- Configures the Reduce plan, the POPackage operator
and the reporter thread
- Specified by:
configure
in interface org.apache.hadoop.mapred.JobConfigurable
- Overrides:
configure
in class PigMapReduce.Reduce
reduce
public void reduce(PigNullableWritable key,
Iterator<NullableTuple> tupIter,
org.apache.hadoop.mapred.OutputCollector<PigNullableWritable,org.apache.hadoop.io.Writable> oc,
org.apache.hadoop.mapred.Reporter reporter)
throws IOException
- The reduce function which packages the key and List<Tuple>
into key, Bag<Tuple> after converting Hadoop type key into Pig type.
The package result is either collected as is, if the reduce plan is
empty or after passing through the reduce plan.
- Specified by:
reduce
in interface org.apache.hadoop.mapred.Reducer<PigNullableWritable,NullableTuple,PigNullableWritable,org.apache.hadoop.io.Writable>
- Overrides:
reduce
in class PigMapReduce.Reduce
- Throws:
IOException
Copyright © ${year} The Apache Software Foundation