org.apache.pig
Interface CommittableStoreFunc
- All Superinterfaces:
- StoreFunc
public interface CommittableStoreFunc
- extends StoreFunc
A storefunc which has an extra commit() method which is called
when all mappers (when the storefunc is part of map) or reducers (when the
storefunc is part of reduce) are finished. Currently this will allow storefuncs
to do any cleanup/finalizing activities knowing that all the maps/reducers
have finished - one such use case is for zebra storage to build an index
for sorted files once all writes are done.
Method Summary |
void |
commit(org.apache.hadoop.conf.Configuration conf)
This method is called when all mappers (when the storefunc is part of
map) or reducers (when the storefunc is part of reduce) are finished. |
commit
void commit(org.apache.hadoop.conf.Configuration conf)
throws IOException
- This method is called when all mappers (when the storefunc is part of
map) or reducers (when the storefunc is part of reduce) are finished.
This allows the storeFunc to do any global commit actions - only called
when all mappers/reducers successfully complete.
If the StoreFunc needs to get hold of StoreConfig object for the store
it can call
MapRedUtil.getStoreConfig(org.apache.hadoop.mapred.JobConf)
where
conf is the Configuration object passed in the commit() call.
- Parameters:
conf
- Configuration object for the job
- Throws:
IOException
Copyright © ${year} The Apache Software Foundation