org.apache.hadoop.fs
Class LocalFileSystem

java.lang.Object
  extended by org.apache.hadoop.conf.Configured
      extended by org.apache.hadoop.fs.FileSystem
          extended by org.apache.hadoop.fs.LocalFileSystem
All Implemented Interfaces:
Configurable

public class LocalFileSystem
extends FileSystem

Implement the FileSystem API for the native filesystem.

Author:
Mike Cafarella

Field Summary
 
Fields inherited from class org.apache.hadoop.fs.FileSystem
LOG
 
Constructor Summary
LocalFileSystem(Configuration conf)
          Construct a local filesystem client.
 
Method Summary
 void close()
          No more filesystem operations are needed.
 void completeLocalInput(File localFile)
          Called when we're all done writing to the target.
 void completeLocalOutput(File fsWorkingFile, File tmpLocalFile)
          Called when we're all done writing to the target.
 void copyFromLocalFile(File src, File dst)
          The src file is on the local disk.
 void copyToLocalFile(File src, File dst)
          The src file is under FS2, and the dst is on the local disk.
 FSOutputStream createRaw(File f, boolean overwrite)
          Opens an OutputStream at the indicated File.
 boolean deleteRaw(File f)
          Deletes File
 boolean exists(File f)
          Check if exists
 long getBlockSize()
          Return the number of bytes that large input files should be optimally be split into to minimize i/o time.
 String[][] getFileCacheHints(File f, long start, long len)
          Return 1x1 'localhost' cell if the file exists.
 long getLength(File f)
          The number of bytes in a file.
 String getName()
          Returns a name for this filesystem, suitable to pass to FileSystem.getNamed(String,Configuration).
 File getWorkingDirectory()
          Get the current working directory for the given file system
 boolean isAbsolute(File f)
          True iff the named path is absolute.
 boolean isDirectory(File f)
          True iff the named path is a directory.
 File[] listFilesRaw(File f)
          List files in a directory.
 void lock(File f, boolean shared)
          Obtain a lock on the given File
 void mkdirs(File f)
          Make the given file and all non-existent parents into directories.
 void moveFromLocalFile(File src, File dst)
          The src file is on the local disk.
 FSInputStream openRaw(File f)
          Opens an InputStream for the indicated File, whether local or via DFS.
 void release(File f)
          Release the lock
 boolean renameRaw(File src, File dst)
          Renames File src to File dst.
 void reportChecksumFailure(File f, FSInputStream in, long start, long length, int crc)
          Moves files to a bad file directory on the same device, so that their storage will not be reused.
 void setWorkingDirectory(File new_dir)
          Set the working directory to the given directory.
 File startLocalInput(File fsInputFile, File tmpLocalFile)
          Returns a local File that the user can read from.
 File startLocalOutput(File fsOutputFile, File tmpLocalFile)
          Returns a local File that the user can write output to.
 String toString()
           
 
Methods inherited from class org.apache.hadoop.fs.FileSystem
create, create, createNewFile, delete, get, getChecksumFile, getNamed, isChecksumFile, isFile, listFiles, listFiles, open, open, parseArgs, rename
 
Methods inherited from class org.apache.hadoop.conf.Configured
getConf, setConf
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Constructor Detail

LocalFileSystem

public LocalFileSystem(Configuration conf)
                throws IOException
Construct a local filesystem client.

Throws:
IOException
Method Detail

getFileCacheHints

public String[][] getFileCacheHints(File f,
                                    long start,
                                    long len)
                             throws IOException
Return 1x1 'localhost' cell if the file exists. Return null if otherwise.

Specified by:
getFileCacheHints in class FileSystem
Throws:
IOException

getName

public String getName()
Description copied from class: FileSystem
Returns a name for this filesystem, suitable to pass to FileSystem.getNamed(String,Configuration).

Specified by:
getName in class FileSystem

openRaw

public FSInputStream openRaw(File f)
                      throws IOException
Description copied from class: FileSystem
Opens an InputStream for the indicated File, whether local or via DFS.

Specified by:
openRaw in class FileSystem
Throws:
IOException

createRaw

public FSOutputStream createRaw(File f,
                                boolean overwrite)
                         throws IOException
Description copied from class: FileSystem
Opens an OutputStream at the indicated File.

Specified by:
createRaw in class FileSystem
Parameters:
f - the file name to open
overwrite - if a file with this name already exists, then if true, the file will be overwritten, and if false an error will be thrown.
Throws:
IOException

renameRaw

public boolean renameRaw(File src,
                         File dst)
                  throws IOException
Description copied from class: FileSystem
Renames File src to File dst. Can take place on local fs or remote DFS.

Specified by:
renameRaw in class FileSystem
Throws:
IOException

deleteRaw

public boolean deleteRaw(File f)
                  throws IOException
Description copied from class: FileSystem
Deletes File

Specified by:
deleteRaw in class FileSystem
Throws:
IOException

exists

public boolean exists(File f)
               throws IOException
Description copied from class: FileSystem
Check if exists

Specified by:
exists in class FileSystem
Throws:
IOException

isDirectory

public boolean isDirectory(File f)
                    throws IOException
Description copied from class: FileSystem
True iff the named path is a directory.

Specified by:
isDirectory in class FileSystem
Throws:
IOException

isAbsolute

public boolean isAbsolute(File f)
Description copied from class: FileSystem
True iff the named path is absolute.

Specified by:
isAbsolute in class FileSystem

getLength

public long getLength(File f)
               throws IOException
Description copied from class: FileSystem
The number of bytes in a file.

Specified by:
getLength in class FileSystem
Throws:
IOException

listFilesRaw

public File[] listFilesRaw(File f)
                    throws IOException
Description copied from class: FileSystem
List files in a directory.

Specified by:
listFilesRaw in class FileSystem
Throws:
IOException

mkdirs

public void mkdirs(File f)
            throws IOException
Description copied from class: FileSystem
Make the given file and all non-existent parents into directories.

Specified by:
mkdirs in class FileSystem
Throws:
IOException

setWorkingDirectory

public void setWorkingDirectory(File new_dir)
Set the working directory to the given directory. Sets both a local variable and the system property. Note that the system property is only used if the application explictly calls java.io.File.getAbsolutePath().

Specified by:
setWorkingDirectory in class FileSystem

getWorkingDirectory

public File getWorkingDirectory()
Description copied from class: FileSystem
Get the current working directory for the given file system

Specified by:
getWorkingDirectory in class FileSystem
Returns:
the directory pathname

lock

public void lock(File f,
                 boolean shared)
          throws IOException
Description copied from class: FileSystem
Obtain a lock on the given File

Specified by:
lock in class FileSystem
Throws:
IOException

release

public void release(File f)
             throws IOException
Description copied from class: FileSystem
Release the lock

Specified by:
release in class FileSystem
Throws:
IOException

moveFromLocalFile

public void moveFromLocalFile(File src,
                              File dst)
                       throws IOException
Description copied from class: FileSystem
The src file is on the local disk. Add it to FS at the given dst name, removing the source afterwards.

Specified by:
moveFromLocalFile in class FileSystem
Throws:
IOException

copyFromLocalFile

public void copyFromLocalFile(File src,
                              File dst)
                       throws IOException
Description copied from class: FileSystem
The src file is on the local disk. Add it to FS at the given dst name and the source is kept intact afterwards

Specified by:
copyFromLocalFile in class FileSystem
Throws:
IOException

copyToLocalFile

public void copyToLocalFile(File src,
                            File dst)
                     throws IOException
Description copied from class: FileSystem
The src file is under FS2, and the dst is on the local disk. Copy it from FS control to the local dst name.

Specified by:
copyToLocalFile in class FileSystem
Throws:
IOException

startLocalOutput

public File startLocalOutput(File fsOutputFile,
                             File tmpLocalFile)
                      throws IOException
Description copied from class: FileSystem
Returns a local File that the user can write output to. The caller provides both the eventual FS target name and the local working file. If the FS is local, we write directly into the target. If the FS is remote, we write into the tmp local area.

Specified by:
startLocalOutput in class FileSystem
Throws:
IOException

completeLocalOutput

public void completeLocalOutput(File fsWorkingFile,
                                File tmpLocalFile)
                         throws IOException
Description copied from class: FileSystem
Called when we're all done writing to the target. A local FS will do nothing, because we've written to exactly the right place. A remote FS will copy the contents of tmpLocalFile to the correct target at fsOutputFile.

Specified by:
completeLocalOutput in class FileSystem
Throws:
IOException

startLocalInput

public File startLocalInput(File fsInputFile,
                            File tmpLocalFile)
                     throws IOException
Description copied from class: FileSystem
Returns a local File that the user can read from. The caller provides both the eventual FS target name and the local working file. If the FS is local, we read directly from the source. If the FS is remote, we write data into the tmp local area.

Specified by:
startLocalInput in class FileSystem
Throws:
IOException

completeLocalInput

public void completeLocalInput(File localFile)
                        throws IOException
Description copied from class: FileSystem
Called when we're all done writing to the target. A local FS will do nothing, because we've written to exactly the right place. A remote FS will copy the contents of tmpLocalFile to the correct target at fsOutputFile.

Specified by:
completeLocalInput in class FileSystem
Throws:
IOException

close

public void close()
           throws IOException
Description copied from class: FileSystem
No more filesystem operations are needed. Will release any held locks.

Specified by:
close in class FileSystem
Throws:
IOException

toString

public String toString()
Overrides:
toString in class Object

reportChecksumFailure

public void reportChecksumFailure(File f,
                                  FSInputStream in,
                                  long start,
                                  long length,
                                  int crc)
Moves files to a bad file directory on the same device, so that their storage will not be reused.

Specified by:
reportChecksumFailure in class FileSystem
Parameters:
f - the file name containing the error
in - the stream open on the file
start - the position of the beginning of the bad data in the file
length - the length of the bad data in the file
crc - the expected CRC32 of the data

getBlockSize

public long getBlockSize()
Description copied from class: FileSystem
Return the number of bytes that large input files should be optimally be split into to minimize i/o time.

Specified by:
getBlockSize in class FileSystem


Copyright © 2006 The Apache Software Foundation