org.apache.pig.backend.hadoop.executionengine.mapReduceLayer
Class PigCombiner.Combine
java.lang.Object
org.apache.hadoop.mapred.MapReduceBase
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigCombiner.Combine
- All Implemented Interfaces:
- Closeable, org.apache.hadoop.mapred.JobConfigurable, org.apache.hadoop.mapred.Reducer<PigNullableWritable,NullableTuple,PigNullableWritable,org.apache.hadoop.io.Writable>
- Enclosing class:
- PigCombiner
public static class PigCombiner.Combine
- extends org.apache.hadoop.mapred.MapReduceBase
- implements org.apache.hadoop.mapred.Reducer<PigNullableWritable,NullableTuple,PigNullableWritable,org.apache.hadoop.io.Writable>
Method Summary |
void |
close()
Will be called once all the intermediate keys and values are
processed. |
void |
configure(org.apache.hadoop.mapred.JobConf jConf)
Configures the Reduce plan, the POPackage operator
and the reporter thread |
byte |
getKeyType()
|
boolean |
processOnePackageOutput(org.apache.hadoop.mapred.OutputCollector<PigNullableWritable,org.apache.hadoop.io.Writable> oc)
|
void |
reduce(PigNullableWritable key,
Iterator<NullableTuple> tupIter,
org.apache.hadoop.mapred.OutputCollector<PigNullableWritable,org.apache.hadoop.io.Writable> oc,
org.apache.hadoop.mapred.Reporter reporter)
The reduce function which packages the key and List <Tuple>
into key, Bag<Tuple> after converting Hadoop type key into Pig type. |
void |
setKeyType(byte keyType)
|
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
PigCombiner.Combine
public PigCombiner.Combine()
configure
public void configure(org.apache.hadoop.mapred.JobConf jConf)
- Configures the Reduce plan, the POPackage operator
and the reporter thread
- Specified by:
configure
in interface org.apache.hadoop.mapred.JobConfigurable
- Overrides:
configure
in class org.apache.hadoop.mapred.MapReduceBase
reduce
public void reduce(PigNullableWritable key,
Iterator<NullableTuple> tupIter,
org.apache.hadoop.mapred.OutputCollector<PigNullableWritable,org.apache.hadoop.io.Writable> oc,
org.apache.hadoop.mapred.Reporter reporter)
throws IOException
- The reduce function which packages the key and List <Tuple>
into key, Bag<Tuple> after converting Hadoop type key into Pig type.
The package result is either collected as is, if the reduce plan is
empty or after passing through the reduce plan.
- Specified by:
reduce
in interface org.apache.hadoop.mapred.Reducer<PigNullableWritable,NullableTuple,PigNullableWritable,org.apache.hadoop.io.Writable>
- Throws:
IOException
processOnePackageOutput
public boolean processOnePackageOutput(org.apache.hadoop.mapred.OutputCollector<PigNullableWritable,org.apache.hadoop.io.Writable> oc)
throws IOException
- Throws:
IOException
close
public void close()
throws IOException
- Will be called once all the intermediate keys and values are
processed. So right place to stop the reporter thread.
- Specified by:
close
in interface Closeable
- Overrides:
close
in class org.apache.hadoop.mapred.MapReduceBase
- Throws:
IOException
getKeyType
public byte getKeyType()
- Returns:
- the keyType
setKeyType
public void setKeyType(byte keyType)
- Parameters:
keyType
- the keyType to set
Copyright © ${year} The Apache Software Foundation