org.apache.hadoop.tools
Class DistCp
java.lang.Object
org.apache.hadoop.tools.DistCp
- All Implemented Interfaces:
- org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool
public class DistCp
- extends Object
- implements org.apache.hadoop.util.Tool
A Map-reduce program to recursively copy directories between
different file-systems.
Field Summary |
static org.apache.commons.logging.Log |
LOG
|
Constructor Summary |
DistCp(org.apache.hadoop.conf.Configuration conf)
|
Method Summary |
static void |
copy(org.apache.hadoop.conf.Configuration conf,
String srcPath,
String destPath,
org.apache.hadoop.fs.Path logPath,
boolean srcAsList,
boolean ignoreReadFailures)
Deprecated. |
org.apache.hadoop.conf.Configuration |
getConf()
|
static String |
getRandomId()
|
static void |
main(String[] args)
|
int |
run(String[] args)
This is the main driver for recursively copying directories
across file systems. |
void |
setConf(org.apache.hadoop.conf.Configuration conf)
|
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
LOG
public static final org.apache.commons.logging.Log LOG
DistCp
public DistCp(org.apache.hadoop.conf.Configuration conf)
setConf
public void setConf(org.apache.hadoop.conf.Configuration conf)
- Specified by:
setConf
in interface org.apache.hadoop.conf.Configurable
getConf
public org.apache.hadoop.conf.Configuration getConf()
- Specified by:
getConf
in interface org.apache.hadoop.conf.Configurable
copy
@Deprecated
public static void copy(org.apache.hadoop.conf.Configuration conf,
String srcPath,
String destPath,
org.apache.hadoop.fs.Path logPath,
boolean srcAsList,
boolean ignoreReadFailures)
throws IOException
- Deprecated.
- Throws:
IOException
run
public int run(String[] args)
- This is the main driver for recursively copying directories
across file systems. It takes at least two cmdline parameters. A source
URL and a destination URL. It then essentially does an "ls -lR" on the
source URL, and writes the output in a round-robin manner to all the map
input files. The mapper actually copies the files allotted to it. The
reduce is empty.
- Specified by:
run
in interface org.apache.hadoop.util.Tool
main
public static void main(String[] args)
throws Exception
- Throws:
Exception
getRandomId
public static String getRandomId()
Copyright © 2009 The Apache Software Foundation