org.apache.hadoop.hbase.mapred
Class TableMapReduceUtil

java.lang.Object
  extended by org.apache.hadoop.hbase.mapred.TableMapReduceUtil

public class TableMapReduceUtil
extends Object

Utility for TableMap and TableReduce


Constructor Summary
TableMapReduceUtil()
           
 
Method Summary
static void initTableMapJob(String table, String columns, Class<? extends TableMap> mapper, Class<? extends org.apache.hadoop.io.WritableComparable> outputKeyClass, Class<? extends org.apache.hadoop.io.Writable> outputValueClass, org.apache.hadoop.mapred.JobConf job)
          Use this before submitting a TableMap job.
static void initTableReduceJob(String table, Class<? extends TableReduce> reducer, org.apache.hadoop.mapred.JobConf job)
          Use this before submitting a TableReduce job.
static void initTableReduceJob(String table, Class<? extends TableReduce> reducer, org.apache.hadoop.mapred.JobConf job, Class partitioner)
          Use this before submitting a TableReduce job.
 void limitNumMapTasks(String table, org.apache.hadoop.mapred.JobConf job)
          Ensures that the given number of map tasks for the given job configuration does not exceed the number of regions for the given table.
 void limitNumReduceTasks(String table, org.apache.hadoop.mapred.JobConf job)
          Ensures that the given number of reduce tasks for the given job configuration does not exceed the number of regions for the given table.
 void setNumMapTasks(String table, org.apache.hadoop.mapred.JobConf job)
          Sets the number of map tasks for the given job configuration to the number of regions the given table has.
 void setNumReduceTasks(String table, org.apache.hadoop.mapred.JobConf job)
          Sets the number of reduce tasks for the given job configuration to the number of regions the given table has.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

TableMapReduceUtil

public TableMapReduceUtil()
Method Detail

initTableMapJob

public static void initTableMapJob(String table,
                                   String columns,
                                   Class<? extends TableMap> mapper,
                                   Class<? extends org.apache.hadoop.io.WritableComparable> outputKeyClass,
                                   Class<? extends org.apache.hadoop.io.Writable> outputValueClass,
                                   org.apache.hadoop.mapred.JobConf job)
Use this before submitting a TableMap job. It will appropriately set up the JobConf.

Parameters:
table - The table name to read from.
columns - The columns to scan.
mapper - The mapper class to use.
outputKeyClass - The class of the output key.
outputValueClass - The class of the output value.
job - The current job configuration to adjust.

initTableReduceJob

public static void initTableReduceJob(String table,
                                      Class<? extends TableReduce> reducer,
                                      org.apache.hadoop.mapred.JobConf job)
                               throws IOException
Use this before submitting a TableReduce job. It will appropriately set up the JobConf.

Parameters:
table - The output table.
reducer - The reducer class to use.
job - The current job configuration to adjust.
Throws:
IOException - When determining the region count fails.

initTableReduceJob

public static void initTableReduceJob(String table,
                                      Class<? extends TableReduce> reducer,
                                      org.apache.hadoop.mapred.JobConf job,
                                      Class partitioner)
                               throws IOException
Use this before submitting a TableReduce job. It will appropriately set up the JobConf.

Parameters:
table - The output table.
reducer - The reducer class to use.
job - The current job configuration to adjust.
partitioner - Partitioner to use. Pass null to use default partitioner.
Throws:
IOException - When determining the region count fails.

limitNumReduceTasks

public void limitNumReduceTasks(String table,
                                org.apache.hadoop.mapred.JobConf job)
                         throws IOException
Ensures that the given number of reduce tasks for the given job configuration does not exceed the number of regions for the given table.

Parameters:
table - The table to get the region count for.
job - The current job configuration to adjust.
Throws:
IOException - When retrieving the table details fails.

limitNumMapTasks

public void limitNumMapTasks(String table,
                             org.apache.hadoop.mapred.JobConf job)
                      throws IOException
Ensures that the given number of map tasks for the given job configuration does not exceed the number of regions for the given table.

Parameters:
table - The table to get the region count for.
job - The current job configuration to adjust.
Throws:
IOException - When retrieving the table details fails.

setNumReduceTasks

public void setNumReduceTasks(String table,
                              org.apache.hadoop.mapred.JobConf job)
                       throws IOException
Sets the number of reduce tasks for the given job configuration to the number of regions the given table has.

Parameters:
table - The table to get the region count for.
job - The current job configuration to adjust.
Throws:
IOException - When retrieving the table details fails.

setNumMapTasks

public void setNumMapTasks(String table,
                           org.apache.hadoop.mapred.JobConf job)
                    throws IOException
Sets the number of map tasks for the given job configuration to the number of regions the given table has.

Parameters:
table - The table to get the region count for.
job - The current job configuration to adjust.
Throws:
IOException - When retrieving the table details fails.


Copyright © 2008 The Apache Software Foundation