org.apache.hadoop.hdfs.server.namenode
Class BackupNode

java.lang.Object
  extended by org.apache.hadoop.hdfs.server.namenode.NameNode
      extended by org.apache.hadoop.hdfs.server.namenode.BackupNode
All Implemented Interfaces:
ClientProtocol, FSConstants, DatanodeProtocol, NamenodeProtocol, NamenodeProtocols, org.apache.hadoop.ipc.VersionedProtocol, org.apache.hadoop.security.authorize.RefreshAuthorizationPolicyProtocol, org.apache.hadoop.security.RefreshUserMappingsProtocol

@InterfaceAudience.Private
public class BackupNode
extends NameNode

BackupNode.

Backup node can play two roles.

  1. HdfsConstants.NamenodeRole.CHECKPOINT node periodically creates checkpoints, that is downloads image and edits from the active node, merges them, and uploads the new image back to the active.
  2. HdfsConstants.NamenodeRole.BACKUP node keeps its namespace in sync with the active node, and periodically creates checkpoints by simply saving the namespace image to local disk(s).


Nested Class Summary
 
Nested classes/interfaces inherited from interface org.apache.hadoop.hdfs.protocol.FSConstants
FSConstants.DatanodeReportType, FSConstants.SafeModeAction, FSConstants.UpgradeAction
 
Field Summary
 
Fields inherited from class org.apache.hadoop.hdfs.server.namenode.NameNode
DEFAULT_PORT, httpAddress, httpServer, LOG, namesystem, nodeRegistration, role, rpcAddress, serviceRPCAddress, serviceRpcServer, stateChangeLog, stopRequested
 
Fields inherited from interface org.apache.hadoop.hdfs.protocol.ClientProtocol
GET_STATS_CAPACITY_IDX, GET_STATS_CORRUPT_BLOCKS_IDX, GET_STATS_MISSING_BLOCKS_IDX, GET_STATS_REMAINING_IDX, GET_STATS_UNDER_REPLICATED_IDX, GET_STATS_USED_IDX, versionID
 
Fields inherited from interface org.apache.hadoop.hdfs.server.protocol.DatanodeProtocol
DISK_ERROR, DNA_ACCESSKEYUPDATE, DNA_FINALIZE, DNA_INVALIDATE, DNA_RECOVERBLOCK, DNA_REGISTER, DNA_SHUTDOWN, DNA_TRANSFER, DNA_UNKNOWN, FATAL_DISK_ERROR, INVALID_BLOCK, NOTIFY, versionID
 
Fields inherited from interface org.apache.hadoop.hdfs.server.protocol.NamenodeProtocol
ACT_CHECKPOINT, ACT_SHUTDOWN, ACT_UNKNOWN, FATAL, JA_CHECKPOINT_TIME, JA_IS_ALIVE, JA_JOURNAL, JA_JSPOOL_START, NOTIFY, versionID
 
Fields inherited from interface org.apache.hadoop.security.authorize.RefreshAuthorizationPolicyProtocol
versionID
 
Fields inherited from interface org.apache.hadoop.security.RefreshUserMappingsProtocol
versionID
 
Fields inherited from interface org.apache.hadoop.hdfs.protocol.FSConstants
BLOCK_INVALIDATE_CHUNK, BLOCKREPORT_INITIAL_DELAY, BLOCKREPORT_INTERVAL, BUFFER_SIZE, DEFAULT_BLOCK_SIZE, DEFAULT_BYTES_PER_CHECKSUM, DEFAULT_DATA_SOCKET_SIZE, DEFAULT_FILE_BUFFER_SIZE, DEFAULT_REPLICATION_FACTOR, DEFAULT_WRITE_PACKET_SIZE, HDFS_URI_SCHEME, HEARTBEAT_INTERVAL, LAYOUT_VERSION, LEASE_HARDLIMIT_PERIOD, LEASE_RECOVER_PERIOD, LEASE_SOFTLIMIT_PERIOD, MAX_PATH_DEPTH, MAX_PATH_LENGTH, MIN_BLOCKS_FOR_WRITE, QUOTA_DONT_SET, QUOTA_RESET, SIZE_OF_INTEGER, SMALL_BUFFER_SIZE
 
Method Summary
 void endCheckpoint(NamenodeRegistration registration, CheckpointSignature sig)
          A request to the active name-node to finalize previously started checkpoint.
 BlocksWithLocations getBlocks(DatanodeInfo datanode, long size)
          Get a list of blocks belonging to datanode whose total size equals size.
protected  InetSocketAddress getHttpServerAddress(org.apache.hadoop.conf.Configuration conf)
           
protected  InetSocketAddress getRpcServerAddress(org.apache.hadoop.conf.Configuration conf)
           
protected  InetSocketAddress getServiceRpcServerAddress(org.apache.hadoop.conf.Configuration conf)
          Given a configuration get the address of the service rpc server If the service rpc is not configured returns null
protected  void initialize(org.apache.hadoop.conf.Configuration conf)
          Initialize name-node.
 void journal(NamenodeRegistration nnReg, int jAction, int length, byte[] args)
          Journal edit records.
protected  void loadNamesystem(org.apache.hadoop.conf.Configuration conf)
           
 NamenodeRegistration register(NamenodeRegistration registration)
          Register a subordinate name-node like backup node.
protected  void setHttpServerAddress(org.apache.hadoop.conf.Configuration conf)
           
protected  void setRpcServerAddress(org.apache.hadoop.conf.Configuration conf)
           
protected  void setRpcServiceServerAddress(org.apache.hadoop.conf.Configuration conf)
          Modifies the configuration passed to contain the service rpc address setting
 boolean setSafeMode(FSConstants.SafeModeAction action)
          Enter, leave or get safe mode.
 NamenodeCommand startCheckpoint(NamenodeRegistration registration)
          A request to the active name-node to start a checkpoint.
 void stop()
          Stop all NameNode threads and wait for all to finish.
 
Methods inherited from class org.apache.hadoop.hdfs.server.namenode.NameNode
abandonBlock, addBlock, append, blockReceived, blockReport, cancelDelegationToken, commitBlockSynchronization, complete, concat, create, createNameNode, createSymlink, delete, delete, distributedUpgradeProgress, errorReport, errorReport, finalizeUpgrade, format, fsync, getAddress, getAddress, getBlockKeys, getBlockLocations, getContentSummary, getDatanodeReport, getDelegationToken, getEditLogSize, getFileInfo, getFileLinkInfo, getFSImage, getFsImageName, getFsImageNameCheckpoint, getHostPortString, getHttpAddress, getInfoServer, getLinkTarget, getListing, getNameNodeAddress, getNameNodeMetrics, getPreferredBlockSize, getProtocolVersion, getRole, getServerDefaults, getServiceAddress, getStats, getUri, isInSafeMode, join, journalSize, listCorruptFileBlocks, main, metaSave, mkdirs, processUpgradeCommand, recoverLease, refreshNodes, refreshServiceAcl, refreshSuperUserGroupsConfiguration, refreshUserToGroupsMappings, registerDatanode, rename, rename, renewDelegationToken, renewLease, reportBadBlocks, restoreFailedStorage, rollEditLog, rollFsImage, saveNamespace, sendHeartbeat, setOwner, setPermission, setQuota, setReplication, setServiceAddress, setTimes, updateBlockForPipeline, updatePipeline, verifyRequest, verifyVersion, versionRequest
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Method Detail

getRpcServerAddress

protected InetSocketAddress getRpcServerAddress(org.apache.hadoop.conf.Configuration conf)
                                         throws IOException
Overrides:
getRpcServerAddress in class NameNode
Throws:
IOException

getServiceRpcServerAddress

protected InetSocketAddress getServiceRpcServerAddress(org.apache.hadoop.conf.Configuration conf)
                                                throws IOException
Description copied from class: NameNode
Given a configuration get the address of the service rpc server If the service rpc is not configured returns null

Overrides:
getServiceRpcServerAddress in class NameNode
Throws:
IOException

setRpcServerAddress

protected void setRpcServerAddress(org.apache.hadoop.conf.Configuration conf)
Overrides:
setRpcServerAddress in class NameNode

setRpcServiceServerAddress

protected void setRpcServiceServerAddress(org.apache.hadoop.conf.Configuration conf)
Description copied from class: NameNode
Modifies the configuration passed to contain the service rpc address setting

Overrides:
setRpcServiceServerAddress in class NameNode

getHttpServerAddress

protected InetSocketAddress getHttpServerAddress(org.apache.hadoop.conf.Configuration conf)
Overrides:
getHttpServerAddress in class NameNode

setHttpServerAddress

protected void setHttpServerAddress(org.apache.hadoop.conf.Configuration conf)
Overrides:
setHttpServerAddress in class NameNode

loadNamesystem

protected void loadNamesystem(org.apache.hadoop.conf.Configuration conf)
                       throws IOException
Overrides:
loadNamesystem in class NameNode
Throws:
IOException

initialize

protected void initialize(org.apache.hadoop.conf.Configuration conf)
                   throws IOException
Description copied from class: NameNode
Initialize name-node.

Overrides:
initialize in class NameNode
Parameters:
conf - the configuration
Throws:
IOException

stop

public void stop()
Description copied from class: NameNode
Stop all NameNode threads and wait for all to finish.

Overrides:
stop in class NameNode

setSafeMode

public boolean setSafeMode(FSConstants.SafeModeAction action)
                    throws IOException
Description copied from interface: ClientProtocol
Enter, leave or get safe mode.

Safe mode is a name node state when it

  1. does not accept changes to name space (read-only), and
  2. does not replicate or delete blocks.

Safe mode is entered automatically at name node startup. Safe mode can also be entered manually using setSafeMode(SafeModeAction.SAFEMODE_GET).

At startup the name node accepts data node reports collecting information about block locations. In order to leave safe mode it needs to collect a configurable percentage called threshold of blocks, which satisfy the minimal replication condition. The minimal replication condition is that each block must have at least dfs.namenode.replication.min replicas. When the threshold is reached the name node extends safe mode for a configurable amount of time to let the remaining data nodes to check in before it will start replicating missing blocks. Then the name node leaves safe mode.

If safe mode is turned on manually using setSafeMode(SafeModeAction.SAFEMODE_ENTER) then the name node stays in safe mode until it is manually turned off using setSafeMode(SafeModeAction.SAFEMODE_LEAVE). Current state of the name node can be verified using setSafeMode(SafeModeAction.SAFEMODE_GET)

Configuration parameters:

dfs.safemode.threshold.pct is the threshold parameter.
dfs.safemode.extension is the safe mode extension parameter.
dfs.namenode.replication.min is the minimal replication parameter.

Special cases:

The name node does not enter safe mode at startup if the threshold is set to 0 or if the name space is empty.
If the threshold is set to 1 then all blocks need to have at least minimal replication.
If the threshold value is greater than 1 then the name node will not be able to turn off safe mode automatically.
Safe mode can always be turned off manually.

Specified by:
setSafeMode in interface ClientProtocol
Overrides:
setSafeMode in class NameNode
Parameters:
action -
  • 0 leave safe mode;
  • 1 enter safe mode;
  • 2 get safe mode state.
Returns:
  • 0 if the safe mode is OFF or
  • 1 if the safe mode is ON.
Throws:
IOException

getBlocks

public BlocksWithLocations getBlocks(DatanodeInfo datanode,
                                     long size)
                              throws IOException
Description copied from interface: NamenodeProtocol
Get a list of blocks belonging to datanode whose total size equals size.

Specified by:
getBlocks in interface NamenodeProtocol
Overrides:
getBlocks in class NameNode
Parameters:
datanode - a data node
size - requested size
Returns:
a list of blocks & their locations
Throws:
IOException
See Also:
Balancer

register

public NamenodeRegistration register(NamenodeRegistration registration)
                              throws IOException
Description copied from interface: NamenodeProtocol
Register a subordinate name-node like backup node.

Specified by:
register in interface NamenodeProtocol
Overrides:
register in class NameNode
Returns:
NamenodeRegistration of the node, which this node has just registered with.
Throws:
IOException

startCheckpoint

public NamenodeCommand startCheckpoint(NamenodeRegistration registration)
                                throws IOException
Description copied from interface: NamenodeProtocol
A request to the active name-node to start a checkpoint. The name-node should decide whether to admit it or reject. The name-node also decides what should be done with the backup node image before and after the checkpoint.

Specified by:
startCheckpoint in interface NamenodeProtocol
Overrides:
startCheckpoint in class NameNode
Parameters:
registration - the requesting node
Returns:
CheckpointCommand if checkpoint is allowed.
Throws:
IOException
See Also:
CheckpointCommand, NamenodeCommand, NamenodeProtocol.ACT_SHUTDOWN

endCheckpoint

public void endCheckpoint(NamenodeRegistration registration,
                          CheckpointSignature sig)
                   throws IOException
Description copied from interface: NamenodeProtocol
A request to the active name-node to finalize previously started checkpoint.

Specified by:
endCheckpoint in interface NamenodeProtocol
Overrides:
endCheckpoint in class NameNode
Parameters:
registration - the requesting node
sig - CheckpointSignature which identifies the checkpoint.
Throws:
IOException

journal

public void journal(NamenodeRegistration nnReg,
                    int jAction,
                    int length,
                    byte[] args)
             throws IOException
Description copied from interface: NamenodeProtocol
Journal edit records. This message is sent by the active name-node to the backup node via EditLogBackupOutputStream in order to synchronize meta-data changes with the backup namespace image.

Specified by:
journal in interface NamenodeProtocol
Overrides:
journal in class NameNode
Parameters:
nnReg - active node registration
jAction - journal action
length - length of the byte array
args - byte array containing serialized journal records
Throws:
IOException


Copyright © 2009 The Apache Software Foundation