|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectorg.apache.hadoop.mapreduce.OutputFormat<org.apache.hadoop.io.WritableComparable<?>,HCatRecord>
org.apache.hcatalog.mapreduce.HCatBaseOutputFormat
org.apache.hcatalog.mapreduce.HCatOutputFormat
public class HCatOutputFormat
The OutputFormat to use to write data to HCat. The key value is ignored and and should be given as null. The value is the HCatRecord to write.
Field Summary | |
---|---|
protected static java.lang.String |
DYNTEMP_DIR_NAME
|
protected static java.lang.String |
TEMP_DIR_NAME
The directory under which data is initially written for a non partitioned table |
Constructor Summary | |
---|---|
HCatOutputFormat()
|
Method Summary | |
---|---|
org.apache.hadoop.mapreduce.OutputCommitter |
getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
Get the output committer for this output format. |
org.apache.hadoop.mapreduce.RecordWriter<org.apache.hadoop.io.WritableComparable<?>,HCatRecord> |
getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context)
Get the record writer for the job. |
static void |
prepareOutputLocation(HCatOutputStorageDriver osd,
org.apache.hadoop.mapreduce.TaskAttemptContext context)
Any initialization of file paths, set permissions and group on freshly created files This is called at RecordWriter instantiation time which can be at write-time for a dynamic partitioning usecase |
static void |
setOutput(org.apache.hadoop.mapreduce.Job job,
HCatTableInfo outputInfo)
Set the info about the output to write for the Job. |
static void |
setSchema(org.apache.hadoop.mapreduce.Job job,
HCatSchema schema)
Set the schema for the data being written out to the partition. |
Methods inherited from class org.apache.hcatalog.mapreduce.HCatBaseOutputFormat |
---|
checkOutputSpecs, getJobInfo, getOutputDriverInstance, getOutputFormat, getTableSchema, setPartDetails |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
---|
protected static final java.lang.String TEMP_DIR_NAME
protected static final java.lang.String DYNTEMP_DIR_NAME
Constructor Detail |
---|
public HCatOutputFormat()
Method Detail |
---|
public static void setOutput(org.apache.hadoop.mapreduce.Job job, HCatTableInfo outputInfo) throws java.io.IOException
job
- the job objectoutputInfo
- the table output info
java.io.IOException
- the exception in communicating with the metadata serverpublic static void setSchema(org.apache.hadoop.mapreduce.Job job, HCatSchema schema) throws java.io.IOException
job
- the job objectschema
- the schema for the data
java.io.IOException
public org.apache.hadoop.mapreduce.RecordWriter<org.apache.hadoop.io.WritableComparable<?>,HCatRecord> getRecordWriter(org.apache.hadoop.mapreduce.TaskAttemptContext context) throws java.io.IOException, java.lang.InterruptedException
getRecordWriter
in class org.apache.hadoop.mapreduce.OutputFormat<org.apache.hadoop.io.WritableComparable<?>,HCatRecord>
context
- the information about the current task.
java.io.IOException
java.lang.InterruptedException
public org.apache.hadoop.mapreduce.OutputCommitter getOutputCommitter(org.apache.hadoop.mapreduce.TaskAttemptContext context) throws java.io.IOException, java.lang.InterruptedException
getOutputCommitter
in class org.apache.hadoop.mapreduce.OutputFormat<org.apache.hadoop.io.WritableComparable<?>,HCatRecord>
context
- the task context
java.io.IOException
java.lang.InterruptedException
public static void prepareOutputLocation(HCatOutputStorageDriver osd, org.apache.hadoop.mapreduce.TaskAttemptContext context) throws java.io.IOException
context
-
java.io.IOException
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |