org.apache.pig.backend.hadoop.executionengine.mapReduceLayer
Class MapReducePOStoreImpl

java.lang.Object
  extended by org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStoreImpl
      extended by org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReducePOStoreImpl

public class MapReducePOStoreImpl
extends POStoreImpl

This class is used to have a POStore write to DFS via a output collector/record writer. It sets up a modified job configuration to force a write to a specific subdirectory of the main output directory. This is done so that multiple output directories can be used in the same job. Since the hadoop framework requires a reporter to be available to create the record writer the main function (createStoreFunc) has to be called from within a map or reduce function.


Field Summary
static String PIG_STORE_CONFIG
           
 
Constructor Summary
MapReducePOStoreImpl(org.apache.hadoop.mapred.JobConf job)
           
 
Method Summary
 void cleanUp()
          To perform cleanup when there is an error.
 StoreFunc createStoreFunc(FileSpec sFile, Schema schema)
          Set up the storer
 void setReporter(org.apache.hadoop.mapred.Reporter reporter)
           
 void tearDown()
          At the end of processing, the outputstream is closed using this method
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

PIG_STORE_CONFIG

public static final String PIG_STORE_CONFIG
See Also:
Constant Field Values
Constructor Detail

MapReducePOStoreImpl

public MapReducePOStoreImpl(org.apache.hadoop.mapred.JobConf job)
Method Detail

setReporter

public void setReporter(org.apache.hadoop.mapred.Reporter reporter)

createStoreFunc

public StoreFunc createStoreFunc(FileSpec sFile,
                                 Schema schema)
                          throws IOException
Description copied from class: POStoreImpl
Set up the storer

Specified by:
createStoreFunc in class POStoreImpl
Parameters:
sFile - - The file the store should write to
Throws:
IOException

tearDown

public void tearDown()
              throws IOException
Description copied from class: POStoreImpl
At the end of processing, the outputstream is closed using this method

Overrides:
tearDown in class POStoreImpl
Throws:
IOException

cleanUp

public void cleanUp()
             throws IOException
Description copied from class: POStoreImpl
To perform cleanup when there is an error. Uses the FileLocalizer method which only deletes the file but not the dirs created with it.

Overrides:
cleanUp in class POStoreImpl
Throws:
IOException


Copyright © ${year} The Apache Software Foundation