org.apache.falcon.replication
Class CustomReplicator
java.lang.Object
org.apache.hadoop.conf.Configured
org.apache.hadoop.tools.DistCp
org.apache.falcon.replication.CustomReplicator
- All Implemented Interfaces:
- org.apache.hadoop.util.Tool
public class CustomReplicator
- extends org.apache.hadoop.tools.DistCp
A custom implementation of DistCp that overrides the behavior of CopyListing
interface to copy FileOutputCommitter.SUCCEEDED_FILE_NAME last so downstream apps
depending on data availability will work correctly.
Fields inherited from class org.apache.hadoop.tools.DistCp |
inputOptions, metaFolder, rand |
Constructor Summary |
CustomReplicator(Configuration configuration,
org.apache.hadoop.tools.DistCpOptions inputOptions)
Public Constructor. |
Methods inherited from class org.apache.hadoop.tools.DistCp |
createJob, execute, getFileListingPath, main, run |
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
CustomReplicator
public CustomReplicator(Configuration configuration,
org.apache.hadoop.tools.DistCpOptions inputOptions)
throws Exception
- Public Constructor. Creates DistCp object with specified input-parameters.
(E.g. source-paths, target-location, etc.)
- Parameters:
inputOptions:
- Options (indicating source-paths, target-location.)configuration:
- The Hadoop configuration against which the Copy-mapper must run.
- Throws:
Exception,
- on failure.
Exception
createInputFileListing
protected Path createInputFileListing(Job job)
throws IOException
- Overrides:
createInputFileListing
in class org.apache.hadoop.tools.DistCp
Copyright © 2013-2014 Apache Software Foundation. All Rights Reserved.