org.apache.pig.backend.hadoop.executionengine.mapReduceLayer
Class MapReducePOStoreImpl
java.lang.Object
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POStoreImpl
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReducePOStoreImpl
public class MapReducePOStoreImpl
- extends POStoreImpl
This class is used to have a POStore write to DFS via a output
collector/record writer. It sets up a modified job configuration to
force a write to a specific subdirectory of the main output
directory. This is done so that multiple output directories can be
used in the same job. Since the hadoop framework requires a
reporter to be available to create the record writer the main
function (createStoreFunc) has to be called from within a map or
reduce function.
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
PIG_STORE_CONFIG
public static final String PIG_STORE_CONFIG
- See Also:
- Constant Field Values
MapReducePOStoreImpl
public MapReducePOStoreImpl(org.apache.hadoop.mapred.JobConf job)
setReporter
public void setReporter(org.apache.hadoop.mapred.Reporter reporter)
createStoreFunc
public StoreFunc createStoreFunc(FileSpec sFile,
Schema schema,
SortInfo sortInfo)
throws IOException
- Description copied from class:
POStoreImpl
- Set up the storer
- Specified by:
createStoreFunc
in class POStoreImpl
- Parameters:
sFile
- - The file the store should write toschema
- - the schema of the datasortInfo
- - sortInfo for any sort columns
- Throws:
IOException
tearDown
public void tearDown()
throws IOException
- Description copied from class:
POStoreImpl
- At the end of processing, the outputstream is closed
using this method
- Overrides:
tearDown
in class POStoreImpl
- Throws:
IOException
cleanUp
public void cleanUp()
throws IOException
- Description copied from class:
POStoreImpl
- To perform cleanup when there is an error.
Uses the FileLocalizer method which only
deletes the file but not the dirs created
with it.
- Overrides:
cleanUp
in class POStoreImpl
- Throws:
IOException
Copyright © ${year} The Apache Software Foundation