org.apache.hadoop.hbase.mapreduce
Class HFileOutputFormat
java.lang.Object
org.apache.hadoop.mapreduce.OutputFormat<K,V>
org.apache.hadoop.mapreduce.lib.output.FileOutputFormat<ImmutableBytesWritable,KeyValue>
org.apache.hadoop.hbase.mapreduce.HFileOutputFormat
public class HFileOutputFormat
- extends org.apache.hadoop.mapreduce.lib.output.FileOutputFormat<ImmutableBytesWritable,KeyValue>
Writes HFiles. Passed KeyValues must arrive in order.
Currently, can only write files to a single column family at a
time. Multiple column families requires coordinating keys cross family.
Writes current time as the sequence id for the file. Sets the major compacted
attribute on created hfiles.
- See Also:
KeyValueSortReducer
Methods inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat |
checkOutputSpecs, getCompressOutput, getDefaultWorkFile, getOutputCommitter, getOutputCompressorClass, getOutputPath, getPathForWorkFile, getUniqueFile, getWorkOutputPath, setCompressOutput, setOutputCompressorClass, setOutputPath |
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
HFileOutputFormat
public HFileOutputFormat()
getRecordWriter
public org.apache.hadoop.mapreduce.RecordWriter<ImmutableBytesWritable,KeyValue> getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
throws IOException,
InterruptedException
- Specified by:
getRecordWriter
in class org.apache.hadoop.mapreduce.lib.output.FileOutputFormat<ImmutableBytesWritable,KeyValue>
- Throws:
IOException
InterruptedException
configureIncrementalLoad
public static void configureIncrementalLoad(org.apache.hadoop.mapreduce.Job job,
HTable table)
throws IOException
- Configure a MapReduce Job to perform an incremental load into the given
table. This
- Inspects the table to configure a total order partitioner
- Uploads the partitions file to the cluster and adds it to the DistributedCache
- Sets the number of reduce tasks to match the current number of regions
- Sets the output key/value class to match HFileOutputFormat's requirements
- Sets the reducer up to perform the appropriate sorting (either KeyValueSortReducer or
PutSortReducer)
The user should be sure to set the map output value class to either KeyValue or Put before
running this function.
- Throws:
IOException
Copyright © 2011 The Apache Software Foundation. All Rights Reserved.