org.apache.pig.backend.hadoop.executionengine.mapReduceLayer
Class PigMapReduce
java.lang.Object
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapReduce
public class PigMapReduce
- extends Object
This class is the static Mapper & Reducer classes that
are used by Pig to execute Pig Map Reduce jobs. Since
there is a reduce phase, the leaf is bound to be a
POLocalRearrange. So the map phase has to separate the
key and tuple and collect it into the output
collector.
The shuffle and sort phase sorts these keys & tuples
and creates key, List<Tuple> and passes the key and
iterator to the list. The deserialized POPackage operator
is used to package the key, List<Tuple> into pigKey,
Bag<Tuple> where pigKey is of the appropriate pig type and
then the result of the package is attached to the reduce
plan which is executed if its not empty. Either the result
of the reduce plan or the package res is collected into
the output collector.
The index of the tuple (that is, which bag it should be placed in by the
package) is packed into the key. This is done so that hadoop sorts the
keys in order of index for join.
Field Summary |
static org.apache.hadoop.conf.Configuration |
sJobConf
Deprecated. Use UDFContext instead in the following way to get
the job's Configuration :
UdfContext.getUdfContext().getJobConf() |
static org.apache.hadoop.mapreduce.JobContext |
sJobContext
|
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
sJobContext
public static org.apache.hadoop.mapreduce.JobContext sJobContext
sJobConf
@Deprecated
public static org.apache.hadoop.conf.Configuration sJobConf
- Deprecated. Use
UDFContext
instead in the following way to get
the job's Configuration
:
UdfContext.getUdfContext().getJobConf()
PigMapReduce
public PigMapReduce()
Copyright © ${year} The Apache Software Foundation