org.apache.hadoop.dfs
Class DistributedFileSystem

java.lang.Object
  extended by org.apache.hadoop.conf.Configured
      extended by org.apache.hadoop.fs.FileSystem
          extended by org.apache.hadoop.dfs.DistributedFileSystem
All Implemented Interfaces:
Configurable

public class DistributedFileSystem
extends FileSystem

Implementation of the abstract FileSystem for the DFS system. This object is the way end-user code interacts with a Hadoop DistributedFileSystem.


Field Summary
 
Fields inherited from class org.apache.hadoop.fs.FileSystem
LOG
 
Constructor Summary
DistributedFileSystem()
           
DistributedFileSystem(InetSocketAddress namenode, Configuration conf)
          Deprecated.  
 
Method Summary
 void close()
          No more filesystem operations are needed.
 FSDataOutputStream create(Path f, boolean overwrite, int bufferSize, short replication, long blockSize, Progressable progress)
          Opens an FSDataOutputStream at the indicated Path with write-progress reporting.
 boolean delete(Path f)
          Get rid of Path f, whether a true file or dir.
 UpgradeStatusReport distributedUpgradeProgress(FSConstants.UpgradeAction action)
           
 boolean exists(Path f)
          Check if exists.
 void finalizeUpgrade()
          Finalize previously upgraded files system state.
 long getContentLength(Path f)
          Return the number of bytes of the given path If f is a file, return the size of the file; If f is a directory, return the size of the directory tree
 DatanodeInfo[] getDataNodeStats()
          Return statistics for each datanode.
 long getDefaultBlockSize()
          Return the number of bytes that large input files should be optimally be split into to minimize i/o time.
 short getDefaultReplication()
          Get the default replication.
 String[][] getFileCacheHints(Path f, long start, long len)
          Return a 2D array of size 1x1 or greater, containing hostnames where portions of the given file can be found.
 FileStatus getFileStatus(Path f)
          Returns the stat information about the file.
 String getName()
          Deprecated.  
 long getRawCapacity()
          Return the total raw capacity of the filesystem, disregarding replication .
 long getRawUsed()
          Return the total raw used space in the filesystem, disregarding replication .
 URI getUri()
          Returns a URI whose scheme and authority identify this FileSystem.
 Path getWorkingDirectory()
          Get the current working directory for the given file system
 void initialize(URI uri, Configuration conf)
          Called after a new FileSystem instance is constructed.
 Path[] listPaths(Path f)
          List files in a directory.
 void lock(Path f, boolean shared)
          Deprecated.  
 void metaSave(String pathname)
           
 boolean mkdirs(Path f)
          Make the given file and all non-existent parents into directories.
 FSDataInputStream open(Path f, int bufferSize)
          Opens an FSDataInputStream at the indicated Path.
 void refreshNodes()
           
 void release(Path f)
          Deprecated.  
 boolean rename(Path src, Path dst)
          Rename files/dirs
 boolean reportChecksumFailure(Path f, FSDataInputStream in, long inPos, FSDataInputStream sums, long sumsPos)
          We need to find the blocks that didn't match.
 boolean setReplication(Path src, short replication)
          Set replication for an existing file.
 boolean setSafeMode(FSConstants.SafeModeAction action)
          Enter, leave or get safe mode.
 void setWorkingDirectory(Path dir)
          Set the current working directory for the given file system.
 String toString()
           
 
Methods inherited from class org.apache.hadoop.fs.FileSystem
checkPath, closeAll, completeLocalOutput, copyFromLocalFile, copyFromLocalFile, copyToLocalFile, copyToLocalFile, create, create, create, create, create, create, create, createNewFile, get, get, getBlockSize, getLength, getLocal, getNamed, getReplication, getUsed, globPaths, globPaths, isDirectory, isFile, listPaths, listPaths, listPaths, makeQualified, moveFromLocalFile, moveToLocalFile, open, parseArgs, startLocalOutput
 
Methods inherited from class org.apache.hadoop.conf.Configured
getConf, setConf
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Constructor Detail

DistributedFileSystem

public DistributedFileSystem()

DistributedFileSystem

public DistributedFileSystem(InetSocketAddress namenode,
                             Configuration conf)
                      throws IOException
Deprecated. 

Throws:
IOException
Method Detail

getName

public String getName()
Deprecated. 

Overrides:
getName in class FileSystem

getUri

public URI getUri()
Description copied from class: FileSystem
Returns a URI whose scheme and authority identify this FileSystem.

Specified by:
getUri in class FileSystem

initialize

public void initialize(URI uri,
                       Configuration conf)
                throws IOException
Description copied from class: FileSystem
Called after a new FileSystem instance is constructed.

Specified by:
initialize in class FileSystem
Parameters:
uri - a uri whose authority section names the host, port, etc. for this FileSystem
conf - the configuration
Throws:
IOException

getWorkingDirectory

public Path getWorkingDirectory()
Description copied from class: FileSystem
Get the current working directory for the given file system

Specified by:
getWorkingDirectory in class FileSystem
Returns:
the directory pathname

getDefaultBlockSize

public long getDefaultBlockSize()
Description copied from class: FileSystem
Return the number of bytes that large input files should be optimally be split into to minimize i/o time.

Overrides:
getDefaultBlockSize in class FileSystem

getDefaultReplication

public short getDefaultReplication()
Description copied from class: FileSystem
Get the default replication.

Overrides:
getDefaultReplication in class FileSystem

setWorkingDirectory

public void setWorkingDirectory(Path dir)
Description copied from class: FileSystem
Set the current working directory for the given file system. All relative paths will be resolved relative to it.

Specified by:
setWorkingDirectory in class FileSystem

getFileCacheHints

public String[][] getFileCacheHints(Path f,
                                    long start,
                                    long len)
                             throws IOException
Description copied from class: FileSystem
Return a 2D array of size 1x1 or greater, containing hostnames where portions of the given file can be found. For a nonexistent file or regions, null will be returned. This call is most helpful with DFS, where it returns hostnames of machines that contain the given file. The FileSystem will simply return an elt containing 'localhost'.

Overrides:
getFileCacheHints in class FileSystem
Throws:
IOException

open

public FSDataInputStream open(Path f,
                              int bufferSize)
                       throws IOException
Description copied from class: FileSystem
Opens an FSDataInputStream at the indicated Path.

Specified by:
open in class FileSystem
Parameters:
f - the file name to open
bufferSize - the size of the buffer to be used.
Throws:
IOException

create

public FSDataOutputStream create(Path f,
                                 boolean overwrite,
                                 int bufferSize,
                                 short replication,
                                 long blockSize,
                                 Progressable progress)
                          throws IOException
Description copied from class: FileSystem
Opens an FSDataOutputStream at the indicated Path with write-progress reporting.

Specified by:
create in class FileSystem
Parameters:
f - the file name to open
overwrite - if a file with this name already exists, then if true, the file will be overwritten, and if false an error will be thrown.
bufferSize - the size of the buffer to be used.
replication - required block replication for the file.
Throws:
IOException

setReplication

public boolean setReplication(Path src,
                              short replication)
                       throws IOException
Description copied from class: FileSystem
Set replication for an existing file.

Overrides:
setReplication in class FileSystem
Parameters:
src - file name
replication - new replication
Returns:
true if successful; false if file does not exist or is a directory
Throws:
IOException

rename

public boolean rename(Path src,
                      Path dst)
               throws IOException
Rename files/dirs

Specified by:
rename in class FileSystem
Throws:
IOException

delete

public boolean delete(Path f)
               throws IOException
Get rid of Path f, whether a true file or dir.

Specified by:
delete in class FileSystem
Throws:
IOException

exists

public boolean exists(Path f)
               throws IOException
Description copied from class: FileSystem
Check if exists.

Specified by:
exists in class FileSystem
Parameters:
f - source file
Throws:
IOException

getContentLength

public long getContentLength(Path f)
                      throws IOException
Description copied from class: FileSystem
Return the number of bytes of the given path If f is a file, return the size of the file; If f is a directory, return the size of the directory tree

Overrides:
getContentLength in class FileSystem
Throws:
IOException

listPaths

public Path[] listPaths(Path f)
                 throws IOException
Description copied from class: FileSystem
List files in a directory.

Specified by:
listPaths in class FileSystem
Throws:
IOException

mkdirs

public boolean mkdirs(Path f)
               throws IOException
Description copied from class: FileSystem
Make the given file and all non-existent parents into directories. Has the semantics of Unix 'mkdir -p'. Existence of the directory hierarchy is not an error.

Specified by:
mkdirs in class FileSystem
Throws:
IOException

lock

@Deprecated
public void lock(Path f,
                            boolean shared)
          throws IOException
Deprecated. 

Description copied from class: FileSystem
Obtain a lock on the given Path

Overrides:
lock in class FileSystem
Throws:
IOException

release

@Deprecated
public void release(Path f)
             throws IOException
Deprecated. 

Description copied from class: FileSystem
Release the lock

Overrides:
release in class FileSystem
Throws:
IOException

close

public void close()
           throws IOException
Description copied from class: FileSystem
No more filesystem operations are needed. Will release any held locks.

Overrides:
close in class FileSystem
Throws:
IOException

toString

public String toString()
Overrides:
toString in class Object

getRawCapacity

public long getRawCapacity()
                    throws IOException
Return the total raw capacity of the filesystem, disregarding replication .

Throws:
IOException

getRawUsed

public long getRawUsed()
                throws IOException
Return the total raw used space in the filesystem, disregarding replication .

Throws:
IOException

getDataNodeStats

public DatanodeInfo[] getDataNodeStats()
                                throws IOException
Return statistics for each datanode.

Throws:
IOException

setSafeMode

public boolean setSafeMode(FSConstants.SafeModeAction action)
                    throws IOException
Enter, leave or get safe mode.

Throws:
IOException
See Also:
ClientProtocol.setSafeMode(FSConstants.SafeModeAction)

refreshNodes

public void refreshNodes()
                  throws IOException
Throws:
IOException

finalizeUpgrade

public void finalizeUpgrade()
                     throws IOException
Finalize previously upgraded files system state.

Throws:
IOException

distributedUpgradeProgress

public UpgradeStatusReport distributedUpgradeProgress(FSConstants.UpgradeAction action)
                                               throws IOException
Throws:
IOException

metaSave

public void metaSave(String pathname)
              throws IOException
Throws:
IOException

reportChecksumFailure

public boolean reportChecksumFailure(Path f,
                                     FSDataInputStream in,
                                     long inPos,
                                     FSDataInputStream sums,
                                     long sumsPos)
We need to find the blocks that didn't match. Likely only one is corrupt but we will report both to the namenode. In the future, we can consider figuring out exactly which block is corrupt.


getFileStatus

public FileStatus getFileStatus(Path f)
                         throws IOException
Returns the stat information about the file.

Specified by:
getFileStatus in class FileSystem
Throws:
IOException


Copyright © 2006 The Apache Software Foundation