org.apache.hadoop.hbase.mapreduce.hadoopbackport
Class TotalOrderPartitioner<K extends org.apache.hadoop.io.WritableComparable<?>,V>
java.lang.Object
org.apache.hadoop.mapreduce.Partitioner<K,V>
org.apache.hadoop.hbase.mapreduce.hadoopbackport.TotalOrderPartitioner<K,V>
- All Implemented Interfaces:
- org.apache.hadoop.conf.Configurable
public class TotalOrderPartitioner<K extends org.apache.hadoop.io.WritableComparable<?>,V>
- extends org.apache.hadoop.mapreduce.Partitioner<K,V>
- implements org.apache.hadoop.conf.Configurable
Partitioner effecting a total order by reading split points from
an externally generated source.
This is an identical copy of o.a.h.mapreduce.lib.partition.TotalOrderPartitioner
from Hadoop trunk at r910774.
Method Summary |
org.apache.hadoop.conf.Configuration |
getConf()
|
int |
getPartition(K key,
V value,
int numPartitions)
|
static String |
getPartitionFile(org.apache.hadoop.conf.Configuration conf)
Get the path to the SequenceFile storing the sorted partition keyset. |
void |
setConf(org.apache.hadoop.conf.Configuration conf)
Read in the partition file and build indexing data structures. |
static void |
setPartitionFile(org.apache.hadoop.conf.Configuration conf,
org.apache.hadoop.fs.Path p)
Set the path to the SequenceFile storing the sorted partition keyset. |
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
DEFAULT_PATH
public static final String DEFAULT_PATH
- See Also:
- Constant Field Values
PARTITIONER_PATH
public static final String PARTITIONER_PATH
- See Also:
- Constant Field Values
MAX_TRIE_DEPTH
public static final String MAX_TRIE_DEPTH
- See Also:
- Constant Field Values
NATURAL_ORDER
public static final String NATURAL_ORDER
- See Also:
- Constant Field Values
TotalOrderPartitioner
public TotalOrderPartitioner()
setConf
public void setConf(org.apache.hadoop.conf.Configuration conf)
- Read in the partition file and build indexing data structures.
If the keytype is
BinaryComparable
and
total.order.partitioner.natural.order is not false, a trie
of the first total.order.partitioner.max.trie.depth(2) + 1 bytes
will be built. Otherwise, keys will be located using a binary search of
the partition keyset using the RawComparator
defined for this job. The input file must be sorted with the same
comparator and contain JobContext.getNumReduceTasks()
- 1 keys.
- Specified by:
setConf
in interface org.apache.hadoop.conf.Configurable
getConf
public org.apache.hadoop.conf.Configuration getConf()
- Specified by:
getConf
in interface org.apache.hadoop.conf.Configurable
getPartition
public int getPartition(K key,
V value,
int numPartitions)
- Specified by:
getPartition
in class org.apache.hadoop.mapreduce.Partitioner<K extends org.apache.hadoop.io.WritableComparable<?>,V>
setPartitionFile
public static void setPartitionFile(org.apache.hadoop.conf.Configuration conf,
org.apache.hadoop.fs.Path p)
- Set the path to the SequenceFile storing the sorted partition keyset.
It must be the case that for R reduces, there are R-1
keys in the SequenceFile.
getPartitionFile
public static String getPartitionFile(org.apache.hadoop.conf.Configuration conf)
- Get the path to the SequenceFile storing the sorted partition keyset.
- See Also:
setPartitionFile(Configuration, Path)
Copyright © 2011 The Apache Software Foundation. All Rights Reserved.