org.apache.hadoop.util
Class CopyFiles

java.lang.Object
  extended by org.apache.hadoop.util.CopyFiles
All Implemented Interfaces:
Configurable, Tool

public class CopyFiles
extends Object
implements Tool

A Map-reduce program to recursively copy directories between different file-systems.


Nested Class Summary
static class CopyFiles.FSCopyFilesMapper
          FSCopyFilesMapper: The mapper for copying files between FileSystems.
 
Constructor Summary
CopyFiles()
          Deprecated. 
CopyFiles(Configuration conf)
           
 
Method Summary
static void copy(Configuration conf, List<Path> srcPaths, Path destPath, Path logPath, EnumSet<org.apache.hadoop.util.CopyFiles.cpOpts> flags)
          Driver to copy srcPath to destPath depending on required protocol.
static void copy(Configuration conf, String srcPath, String destPath, Path logPath, boolean srcAsList, boolean ignoreReadFailures)
          Deprecated. 
 Configuration getConf()
          Return the configuration used by this object.
static void main(String[] args)
           
static Path makeRelative(Path root, Path absPath)
          Make a path relative with respect to a root path.
 int run(String[] args)
          This is the main driver for recursively copying directories across file systems.
 void setConf(Configuration conf)
          Set the configuration to be used by this object.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

CopyFiles

@Deprecated
public CopyFiles()
Deprecated. 


CopyFiles

public CopyFiles(Configuration conf)
Method Detail

setConf

public void setConf(Configuration conf)
Description copied from interface: Configurable
Set the configuration to be used by this object.

Specified by:
setConf in interface Configurable

getConf

public Configuration getConf()
Description copied from interface: Configurable
Return the configuration used by this object.

Specified by:
getConf in interface Configurable

copy

@Deprecated
public static void copy(Configuration conf,
                                   String srcPath,
                                   String destPath,
                                   Path logPath,
                                   boolean srcAsList,
                                   boolean ignoreReadFailures)
                 throws IOException
Deprecated. 

Throws:
IOException

copy

public static void copy(Configuration conf,
                        List<Path> srcPaths,
                        Path destPath,
                        Path logPath,
                        EnumSet<org.apache.hadoop.util.CopyFiles.cpOpts> flags)
                 throws IOException
Driver to copy srcPath to destPath depending on required protocol.

Parameters:
srcPaths - list of source paths
destPath - Destination path
logPath - Log output directory
flags - Command-line flags
Throws:
IOException

run

public int run(String[] args)
        throws Exception
This is the main driver for recursively copying directories across file systems. It takes at least two cmdline parameters. A source URL and a destination URL. It then essentially does an "ls -lR" on the source URL, and writes the output in a round-robin manner to all the map input files. The mapper actually copies the files allotted to it. The reduce is empty.

Specified by:
run in interface Tool
Parameters:
args - command specific arguments.
Returns:
exit code.
Throws:
Exception

main

public static void main(String[] args)
                 throws Exception
Throws:
Exception

makeRelative

public static Path makeRelative(Path root,
                                Path absPath)
Make a path relative with respect to a root path. absPath is always assumed to descend from root. Otherwise returned path is null.



Copyright © 2006 The Apache Software Foundation