org.apache.hadoop.hbase.mapred
Class GroupingTableMap
java.lang.Object
org.apache.hadoop.mapred.MapReduceBase
org.apache.hadoop.hbase.mapred.TableMap<ImmutableBytesWritable,RowResult>
org.apache.hadoop.hbase.mapred.GroupingTableMap
- All Implemented Interfaces:
- Closeable, org.apache.hadoop.mapred.JobConfigurable, org.apache.hadoop.mapred.Mapper<ImmutableBytesWritable,RowResult,ImmutableBytesWritable,RowResult>
public class GroupingTableMap
- extends TableMap<ImmutableBytesWritable,RowResult>
Extract grouping columns from input record
Field Summary |
static String |
GROUP_COLUMNS
JobConf parameter to specify the columns used to produce the key passed to
collect from the map phase |
protected byte[][] |
m_columns
|
Methods inherited from class org.apache.hadoop.hbase.mapred.TableMap |
initJob |
Methods inherited from class org.apache.hadoop.mapred.MapReduceBase |
close |
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
GROUP_COLUMNS
public static final String GROUP_COLUMNS
- JobConf parameter to specify the columns used to produce the key passed to
collect from the map phase
- See Also:
- Constant Field Values
m_columns
protected byte[][] m_columns
GroupingTableMap
public GroupingTableMap()
initJob
public static void initJob(String table,
String columns,
String groupColumns,
Class<? extends TableMap> mapper,
org.apache.hadoop.mapred.JobConf job)
- Use this before submitting a TableMap job. It will appropriately set up the
JobConf.
- Parameters:
table
- table to be processedcolumns
- space separated list of columns to fetchgroupColumns
- space separated list of columns used to form the key
used in collectmapper
- map classjob
- job configuration object
configure
public void configure(org.apache.hadoop.mapred.JobConf job)
- Specified by:
configure
in interface org.apache.hadoop.mapred.JobConfigurable
- Overrides:
configure
in class org.apache.hadoop.mapred.MapReduceBase
map
public void map(ImmutableBytesWritable key,
RowResult value,
org.apache.hadoop.mapred.OutputCollector<ImmutableBytesWritable,RowResult> output,
org.apache.hadoop.mapred.Reporter reporter)
throws IOException
- Extract the grouping columns from value to construct a new key.
Pass the new key and value to reduce.
If any of the grouping columns are not found in the value, the record is skipped.
- Specified by:
map
in interface org.apache.hadoop.mapred.Mapper<ImmutableBytesWritable,RowResult,ImmutableBytesWritable,RowResult>
- Specified by:
map
in class TableMap<ImmutableBytesWritable,RowResult>
- Throws:
IOException
extractKeyValues
protected byte[][] extractKeyValues(RowResult r)
- Extract columns values from the current record. This method returns
null if any of the columns are not found.
Override this method if you want to deal with nulls differently.
- Parameters:
r
-
- Returns:
- array of byte values
createGroupKey
protected ImmutableBytesWritable createGroupKey(byte[][] vals)
- Create a key by concatenating multiple column values.
Override this function in order to produce different types of keys.
- Parameters:
vals
-
- Returns:
- key generated by concatenating multiple column values
Copyright © 2008 The Apache Software Foundation