org.apache.hcatalog.mapreduce
Class HCatBaseOutputFormat

java.lang.Object
  extended by org.apache.hadoop.mapreduce.OutputFormat<org.apache.hadoop.io.WritableComparable<?>,HCatRecord>
      extended by org.apache.hcatalog.mapreduce.HCatBaseOutputFormat
Direct Known Subclasses:
HCatEximOutputFormat, HCatOutputFormat

public abstract class HCatBaseOutputFormat
extends org.apache.hadoop.mapreduce.OutputFormat<org.apache.hadoop.io.WritableComparable<?>,HCatRecord>


Constructor Summary
HCatBaseOutputFormat()
           
 
Method Summary
 void checkOutputSpecs(org.apache.hadoop.mapreduce.JobContext context)
          Check for validity of the output-specification for the job.
static org.apache.hcatalog.mapreduce.OutputJobInfo getJobInfo(org.apache.hadoop.mapreduce.JobContext jobContext)
          Gets the HCatOuputJobInfo object by reading the Configuration and deserializing the string.
protected static HCatOutputStorageDriver getOutputDriverInstance(org.apache.hadoop.mapreduce.JobContext context, org.apache.hcatalog.mapreduce.OutputJobInfo jobInfo, java.util.Map<java.lang.String,java.lang.String> fullPartSpec)
          Gets the output storage driver instance, with allowing specification of partvals from which it picks the dynamic partvals
protected  org.apache.hadoop.mapreduce.OutputFormat<? super org.apache.hadoop.io.WritableComparable<?>,? super org.apache.hadoop.io.Writable> getOutputFormat(org.apache.hadoop.mapreduce.JobContext context)
          Gets the output format instance.
static HCatSchema getTableSchema(org.apache.hadoop.mapreduce.JobContext context)
          Gets the table schema for the table specified in the HCatOutputFormat.setOutput call on the specified job context.
protected static void setPartDetails(org.apache.hcatalog.mapreduce.OutputJobInfo jobInfo, HCatSchema schema, java.util.Map<java.lang.String,java.lang.String> partMap)
           
 
Methods inherited from class org.apache.hadoop.mapreduce.OutputFormat
getOutputCommitter, getRecordWriter
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

HCatBaseOutputFormat

public HCatBaseOutputFormat()
Method Detail

getTableSchema

public static HCatSchema getTableSchema(org.apache.hadoop.mapreduce.JobContext context)
                                 throws java.io.IOException
Gets the table schema for the table specified in the HCatOutputFormat.setOutput call on the specified job context.

Parameters:
context - the context
Returns:
the table schema
Throws:
java.io.IOException - if HCatOutputFromat.setOutput has not been called for the passed context

checkOutputSpecs

public void checkOutputSpecs(org.apache.hadoop.mapreduce.JobContext context)
                      throws java.io.IOException,
                             java.lang.InterruptedException
Check for validity of the output-specification for the job.

Specified by:
checkOutputSpecs in class org.apache.hadoop.mapreduce.OutputFormat<org.apache.hadoop.io.WritableComparable<?>,HCatRecord>
Parameters:
context - information about the job
Throws:
java.io.IOException - when output should not be attempted
java.lang.InterruptedException

getOutputFormat

protected org.apache.hadoop.mapreduce.OutputFormat<? super org.apache.hadoop.io.WritableComparable<?>,? super org.apache.hadoop.io.Writable> getOutputFormat(org.apache.hadoop.mapreduce.JobContext context)
                                                                                                                                                      throws java.io.IOException
Gets the output format instance.

Parameters:
context - the job context
Returns:
the output format instance
Throws:
java.io.IOException

getJobInfo

public static org.apache.hcatalog.mapreduce.OutputJobInfo getJobInfo(org.apache.hadoop.mapreduce.JobContext jobContext)
                                                              throws java.io.IOException
Gets the HCatOuputJobInfo object by reading the Configuration and deserializing the string. If JobInfo is not present in the configuration, throws an exception since that means HCatOutputFormat.setOutput has not been called.

Parameters:
jobContext - the job context
Returns:
the OutputJobInfo object
Throws:
java.io.IOException - the IO exception

getOutputDriverInstance

protected static HCatOutputStorageDriver getOutputDriverInstance(org.apache.hadoop.mapreduce.JobContext context,
                                                                 org.apache.hcatalog.mapreduce.OutputJobInfo jobInfo,
                                                                 java.util.Map<java.lang.String,java.lang.String> fullPartSpec)
                                                          throws java.io.IOException
Gets the output storage driver instance, with allowing specification of partvals from which it picks the dynamic partvals

Parameters:
context - the job context
jobInfo - the output job info
Returns:
the output driver instance
Throws:
java.io.IOException

setPartDetails

protected static void setPartDetails(org.apache.hcatalog.mapreduce.OutputJobInfo jobInfo,
                                     HCatSchema schema,
                                     java.util.Map<java.lang.String,java.lang.String> partMap)
                              throws HCatException,
                                     java.io.IOException
Throws:
HCatException
java.io.IOException