org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners
Class SkewedPartitioner

java.lang.Object
  extended by org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.partitioners.SkewedPartitioner
All Implemented Interfaces:
org.apache.hadoop.mapred.JobConfigurable, org.apache.hadoop.mapred.Partitioner<PigNullableWritable,org.apache.hadoop.io.Writable>

public class SkewedPartitioner
extends Object
implements org.apache.hadoop.mapred.Partitioner<PigNullableWritable,org.apache.hadoop.io.Writable>

This class is used by skewed join. For the partitioned table, the skewedpartitioner reads the key distribution data from the sampler file and returns the reducer index in a round robin fashion. For ex: if the key distribution file contains (k1, 5, 3) as an entry, reducers from 5 to 3 are returned in a round robin manner.


Constructor Summary
SkewedPartitioner()
           
 
Method Summary
 void configure(org.apache.hadoop.mapred.JobConf job)
           
 int getPartition(PigNullableWritable wrappedKey, org.apache.hadoop.io.Writable value, int numPartitions)
           
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

SkewedPartitioner

public SkewedPartitioner()
Method Detail

getPartition

public int getPartition(PigNullableWritable wrappedKey,
                        org.apache.hadoop.io.Writable value,
                        int numPartitions)
Specified by:
getPartition in interface org.apache.hadoop.mapred.Partitioner<PigNullableWritable,org.apache.hadoop.io.Writable>

configure

public void configure(org.apache.hadoop.mapred.JobConf job)
Specified by:
configure in interface org.apache.hadoop.mapred.JobConfigurable


Copyright © ${year} The Apache Software Foundation