org.apache.falcon.replication
Class CustomReplicator

java.lang.Object
  extended by org.apache.hadoop.conf.Configured
      extended by org.apache.hadoop.tools.DistCp
          extended by org.apache.falcon.replication.CustomReplicator
All Implemented Interfaces:
org.apache.hadoop.util.Tool

public class CustomReplicator
extends org.apache.hadoop.tools.DistCp

A custom implementation of DistCp that overrides the behavior of CopyListing interface to copy FileOutputCommitter.SUCCEEDED_FILE_NAME last so downstream apps depending on data availability will work correctly.


Field Summary
 
Fields inherited from class org.apache.hadoop.tools.DistCp
inputOptions, metaFolder, rand
 
Constructor Summary
CustomReplicator(Configuration configuration, org.apache.hadoop.tools.DistCpOptions inputOptions)
          Public Constructor.
 
Method Summary
protected  Path createInputFileListing(Job job)
           
 
Methods inherited from class org.apache.hadoop.tools.DistCp
createJob, execute, getFileListingPath, main, run
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

CustomReplicator

public CustomReplicator(Configuration configuration,
                        org.apache.hadoop.tools.DistCpOptions inputOptions)
                 throws Exception
Public Constructor. Creates DistCp object with specified input-parameters. (E.g. source-paths, target-location, etc.)

Parameters:
inputOptions: - Options (indicating source-paths, target-location.)
configuration: - The Hadoop configuration against which the Copy-mapper must run.
Throws:
Exception, - on failure.
Exception
Method Detail

createInputFileListing

protected Path createInputFileListing(Job job)
                               throws IOException
Overrides:
createInputFileListing in class org.apache.hadoop.tools.DistCp


Copyright © 2013-2014 Apache Software Foundation. All Rights Reserved.