org.apache.hadoop.hbase.master
Class AssignmentManager

java.lang.Object
  extended by org.apache.hadoop.hbase.zookeeper.ZooKeeperListener
      extended by org.apache.hadoop.hbase.master.AssignmentManager

@InterfaceAudience.Private
public class AssignmentManager
extends ZooKeeperListener

Manages and performs region assignment.

Monitors ZooKeeper for events related to regions in transition.

Handles existing regions in transition during master failover.


Nested Class Summary
 class AssignmentManager.TimeoutMonitor
          Monitor to check for time outs on region transition operations
 class AssignmentManager.TimerUpdater
          Update timers for all regions in transition going against the server in the serversInUpdatingTimer.
 
Field Summary
static String ALREADY_IN_TRANSITION_WAITTIME
           
static String ASSIGNMENT_TIMEOUT
           
static String ASSIGNMENT_TIMEOUT_MANAGEMENT
           
static int DEFAULT_ALREADY_IN_TRANSITION_WAITTIME
           
static int DEFAULT_ASSIGNMENT_TIMEOUT_DEFAULT
           
static boolean DEFAULT_ASSIGNMENT_TIMEOUT_MANAGEMENT
           
protected  AtomicBoolean failoverCleanupDone
          Indicator that AssignmentManager has recovered the region states so that ServerShutdownHandler can be fully enabled and re-assign regions of dead servers.
static ServerName HBCK_CODE_SERVERNAME
           
protected  Server server
           
static boolean TEST_SKIP_SPLIT_HANDLING
          For testing only! Set to true to skip handling of split.
protected  AssignmentManager.TimeoutMonitor timeoutMonitor
           
 
Fields inherited from class org.apache.hadoop.hbase.zookeeper.ZooKeeperListener
watcher
 
Constructor Summary
AssignmentManager(Server server, ServerManager serverManager, CatalogTracker catalogTracker, LoadBalancer balancer, ExecutorService service, MetricsMaster metricsMaster, TableLockManager tableLockManager)
          Constructs a new assignment manager.
 
Method Summary
 Lock acquireRegionLock(String encodedName)
          To avoid racing with AM, external entities may need to lock a region, for example, when SSH checks what regions to skip re-assigning.
 void addPlan(String encodedName, RegionPlan plan)
          Add a regionPlan for the specified region.
 void addPlans(Map<String,RegionPlan> plans)
          Add a map of region plans.
 void assign(HRegionInfo region, boolean setOfflineInZK)
          Assigns the specified region.
 void assign(HRegionInfo region, boolean setOfflineInZK, boolean forceNewPlan)
          Use care with forceNewPlan.
 void assign(List<HRegionInfo> regions)
          Assigns specified regions round robin, if any.
 void assign(Map<HRegionInfo,ServerName> regions)
          Assigns specified regions retaining assignments, if any.
 void assignMeta()
          Assigns the hbase:meta region.
 void balance(RegionPlan plan)
           
 void deleteClosingOrClosedNode(HRegionInfo region, ServerName sn)
           
 LoadBalancer getBalancer()
           
 int getNumRegionsOpened()
          Used by unit tests.
 RegionPlan getRegionReopenPlan(HRegionInfo hri)
           
 RegionStates getRegionStates()
          This SHOULD not be public.
 Pair<Integer,Integer> getReopenStatus(TableName tableName)
          Used by the client to identify if all regions have the schema updates
 ZKTable getZKTable()
           
 boolean isCarryingMeta(ServerName serverName)
           
 boolean isFailoverCleanupDone()
          Used by ServerShutdownHandler to make sure AssignmentManager has completed the failover cleanup before re-assigning regions of dead servers.
 void nodeChildrenChanged(String path)
          New unassigned node has been created.
 void nodeCreated(String path)
          New unassigned node has been created.
 void nodeDataChanged(String path)
          Existing unassigned node has had data changed.
 void nodeDeleted(String path)
          Called when a node has been deleted
 void offlineDisabledRegion(HRegionInfo regionInfo)
           
protected  String onRegionTransition(ServerName serverName, RegionServerStatusProtos.RegionStateTransition transition)
          Try to update some region states.
 List<HRegionInfo> processServerShutdown(ServerName sn)
          Process shutdown server removing any assignments.
 void regionOffline(HRegionInfo regionInfo)
          Marks the region as offline.
 void registerListener(AssignmentListener listener)
          Add the listener to the notification list.
 void removeClosedRegion(HRegionInfo hri)
          When a region is closed, it should be removed from the regionsToReopen
protected  void setEnabledTable(TableName tableName)
           
 void setRegionsToReopen(List<HRegionInfo> regions)
          Set the list of regions that will be reopened because of an update in table schema
 void shutdown()
          Shutdown the threadpool executor service
 void stop()
           
 void unassign(HRegionInfo region)
          Unassigns the specified region.
 void unassign(HRegionInfo region, boolean force)
           
 void unassign(HRegionInfo region, boolean force, ServerName dest)
          Unassigns the specified region.
 boolean unregisterListener(AssignmentListener listener)
          Remove the listener from the notification list.
 void updateRegionsInTransitionMetrics()
          Set Regions in transitions metrics.
 boolean waitForAssignment(HRegionInfo regionInfo)
          Waits until the specified region has completed assignment.
 void waitOnRegionToClearRegionsInTransition(HRegionInfo hri)
          Wait on region to clear regions-in-transition.
 boolean waitOnRegionToClearRegionsInTransition(HRegionInfo hri, long timeOut)
          Wait on region to clear regions-in-transition or time out
protected  void zkEventWorkersSubmit(org.apache.hadoop.hbase.master.AssignmentManager.RegionRunnable regRunnable)
          Submit a task, ensuring that there is only one task at a time that working on a given region.
 
Methods inherited from class org.apache.hadoop.hbase.zookeeper.ZooKeeperListener
getWatcher
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

HBCK_CODE_SERVERNAME

public static final ServerName HBCK_CODE_SERVERNAME

ASSIGNMENT_TIMEOUT

public static final String ASSIGNMENT_TIMEOUT
See Also:
Constant Field Values

DEFAULT_ASSIGNMENT_TIMEOUT_DEFAULT

public static final int DEFAULT_ASSIGNMENT_TIMEOUT_DEFAULT
See Also:
Constant Field Values

ASSIGNMENT_TIMEOUT_MANAGEMENT

public static final String ASSIGNMENT_TIMEOUT_MANAGEMENT
See Also:
Constant Field Values

DEFAULT_ASSIGNMENT_TIMEOUT_MANAGEMENT

public static final boolean DEFAULT_ASSIGNMENT_TIMEOUT_MANAGEMENT
See Also:
Constant Field Values

ALREADY_IN_TRANSITION_WAITTIME

public static final String ALREADY_IN_TRANSITION_WAITTIME
See Also:
Constant Field Values

DEFAULT_ALREADY_IN_TRANSITION_WAITTIME

public static final int DEFAULT_ALREADY_IN_TRANSITION_WAITTIME
See Also:
Constant Field Values

server

protected final Server server

timeoutMonitor

protected final AssignmentManager.TimeoutMonitor timeoutMonitor

failoverCleanupDone

protected final AtomicBoolean failoverCleanupDone
Indicator that AssignmentManager has recovered the region states so that ServerShutdownHandler can be fully enabled and re-assign regions of dead servers. So that when re-assignment happens, AssignmentManager has proper region states. Protected to ease testing.


TEST_SKIP_SPLIT_HANDLING

public static boolean TEST_SKIP_SPLIT_HANDLING
For testing only! Set to true to skip handling of split.

Constructor Detail

AssignmentManager

public AssignmentManager(Server server,
                         ServerManager serverManager,
                         CatalogTracker catalogTracker,
                         LoadBalancer balancer,
                         ExecutorService service,
                         MetricsMaster metricsMaster,
                         TableLockManager tableLockManager)
                  throws org.apache.zookeeper.KeeperException,
                         IOException
Constructs a new assignment manager.

Parameters:
server -
serverManager -
catalogTracker -
service -
Throws:
org.apache.zookeeper.KeeperException
IOException
Method Detail

registerListener

public void registerListener(AssignmentListener listener)
Add the listener to the notification list.

Parameters:
listener - The AssignmentListener to register

unregisterListener

public boolean unregisterListener(AssignmentListener listener)
Remove the listener from the notification list.

Parameters:
listener - The AssignmentListener to unregister

getZKTable

public ZKTable getZKTable()
Returns:
Instance of ZKTable.

getRegionStates

public RegionStates getRegionStates()
This SHOULD not be public. It is public now because of some unit tests. TODO: make it package private and keep RegionStates in the master package


getRegionReopenPlan

public RegionPlan getRegionReopenPlan(HRegionInfo hri)

addPlan

public void addPlan(String encodedName,
                    RegionPlan plan)
Add a regionPlan for the specified region.

Parameters:
encodedName -
plan -

addPlans

public void addPlans(Map<String,RegionPlan> plans)
Add a map of region plans.


setRegionsToReopen

public void setRegionsToReopen(List<HRegionInfo> regions)
Set the list of regions that will be reopened because of an update in table schema

Parameters:
regions - list of regions that should be tracked for reopen

getReopenStatus

public Pair<Integer,Integer> getReopenStatus(TableName tableName)
                                      throws IOException
Used by the client to identify if all regions have the schema updates

Parameters:
tableName -
Returns:
Pair indicating the status of the alter command
Throws:
IOException

isFailoverCleanupDone

public boolean isFailoverCleanupDone()
Used by ServerShutdownHandler to make sure AssignmentManager has completed the failover cleanup before re-assigning regions of dead servers. So that when re-assignment happens, AssignmentManager has proper region states.


acquireRegionLock

public Lock acquireRegionLock(String encodedName)
To avoid racing with AM, external entities may need to lock a region, for example, when SSH checks what regions to skip re-assigning.


removeClosedRegion

public void removeClosedRegion(HRegionInfo hri)
When a region is closed, it should be removed from the regionsToReopen

Parameters:
hri - HRegionInfo of the region which was closed

nodeCreated

public void nodeCreated(String path)
New unassigned node has been created.

This happens when an RS begins the OPENING or CLOSING of a region by creating an unassigned node.

When this happens we must:

  1. Watch the node for further events
  2. Read and handle the state in the node

Overrides:
nodeCreated in class ZooKeeperListener
Parameters:
path - full path of the new node

nodeDataChanged

public void nodeDataChanged(String path)
Existing unassigned node has had data changed.

This happens when an RS transitions from OFFLINE to OPENING, or between OPENING/OPENED and CLOSING/CLOSED.

When this happens we must:

  1. Watch the node for further events
  2. Read and handle the state in the node

Overrides:
nodeDataChanged in class ZooKeeperListener
Parameters:
path - full path of the updated node

zkEventWorkersSubmit

protected void zkEventWorkersSubmit(org.apache.hadoop.hbase.master.AssignmentManager.RegionRunnable regRunnable)
Submit a task, ensuring that there is only one task at a time that working on a given region. Order is respected.


nodeDeleted

public void nodeDeleted(String path)
Description copied from class: ZooKeeperListener
Called when a node has been deleted

Overrides:
nodeDeleted in class ZooKeeperListener
Parameters:
path - full path of the deleted node

nodeChildrenChanged

public void nodeChildrenChanged(String path)
New unassigned node has been created.

This happens when an RS begins the OPENING, SPLITTING or CLOSING of a region by creating a znode.

When this happens we must:

  1. Watch the node for further children changed events
  2. Watch all new children for changed events

Overrides:
nodeChildrenChanged in class ZooKeeperListener
Parameters:
path - full path of the node whose children have changed

regionOffline

public void regionOffline(HRegionInfo regionInfo)
Marks the region as offline. Removes it from regions in transition and removes in-memory assignment information.

Used when a region has been closed and should remain closed.

Parameters:
regionInfo -

offlineDisabledRegion

public void offlineDisabledRegion(HRegionInfo regionInfo)

assign

public void assign(HRegionInfo region,
                   boolean setOfflineInZK)
Assigns the specified region.

If a RegionPlan is available with a valid destination then it will be used to determine what server region is assigned to. If no RegionPlan is available, region will be assigned to a random available server.

Updates the RegionState and sends the OPEN RPC.

This will only succeed if the region is in transition and in a CLOSED or OFFLINE state or not in transition (in-memory not zk), and of course, the chosen server is up and running (It may have just crashed!). If the in-memory checks pass, the zk node is forced to OFFLINE before assigning.

Parameters:
region - server to be assigned
setOfflineInZK - whether ZK node should be created/transitioned to an OFFLINE state before assigning the region

assign

public void assign(HRegionInfo region,
                   boolean setOfflineInZK,
                   boolean forceNewPlan)
Use care with forceNewPlan. It could cause double assignment.


unassign

public void unassign(HRegionInfo region)
Unassigns the specified region.

Updates the RegionState and sends the CLOSE RPC unless region is being split by regionserver; then the unassign fails (silently) because we presume the region being unassigned no longer exists (its been split out of existence). TODO: What to do if split fails and is rolled back and parent is revivified?

If a RegionPlan is already set, it will remain.

Parameters:
region - server to be unassigned

unassign

public void unassign(HRegionInfo region,
                     boolean force,
                     ServerName dest)
Unassigns the specified region.

Updates the RegionState and sends the CLOSE RPC unless region is being split by regionserver; then the unassign fails (silently) because we presume the region being unassigned no longer exists (its been split out of existence). TODO: What to do if split fails and is rolled back and parent is revivified?

If a RegionPlan is already set, it will remain.

Parameters:
region - server to be unassigned
force - if region should be closed even if already closing

unassign

public void unassign(HRegionInfo region,
                     boolean force)

deleteClosingOrClosedNode

public void deleteClosingOrClosedNode(HRegionInfo region,
                                      ServerName sn)
Parameters:
region - regioninfo of znode to be deleted.

getNumRegionsOpened

public int getNumRegionsOpened()
Used by unit tests. Return the number of regions opened so far in the life of the master. Increases by one every time the master opens a region

Returns:
the counter value of the number of regions opened so far

waitForAssignment

public boolean waitForAssignment(HRegionInfo regionInfo)
                          throws InterruptedException
Waits until the specified region has completed assignment.

If the region is already assigned, returns immediately. Otherwise, method blocks until the region is assigned.

Parameters:
regionInfo - region to wait on assignment for
Throws:
InterruptedException

assignMeta

public void assignMeta()
                throws org.apache.zookeeper.KeeperException
Assigns the hbase:meta region.

Assumes that hbase:meta is currently closed and is not being actively served by any RegionServer.

Forcibly unsets the current meta region location in ZooKeeper and assigns hbase:meta to a random RegionServer.

Throws:
org.apache.zookeeper.KeeperException

assign

public void assign(Map<HRegionInfo,ServerName> regions)
            throws IOException,
                   InterruptedException
Assigns specified regions retaining assignments, if any.

This is a synchronous call and will return once every region has been assigned. If anything fails, an exception is thrown

Throws:
InterruptedException
IOException

assign

public void assign(List<HRegionInfo> regions)
            throws IOException,
                   InterruptedException
Assigns specified regions round robin, if any.

This is a synchronous call and will return once every region has been assigned. If anything fails, an exception is thrown

Throws:
InterruptedException
IOException

updateRegionsInTransitionMetrics

public void updateRegionsInTransitionMetrics()
Set Regions in transitions metrics. This takes an iterator on the RegionInTransition map (CLSM), and is not synchronized. This iterator is not fail fast, which may lead to stale read; but that's better than creating a copy of the map for metrics computation, as this method will be invoked on a frequent interval.


waitOnRegionToClearRegionsInTransition

public void waitOnRegionToClearRegionsInTransition(HRegionInfo hri)
                                            throws IOException,
                                                   InterruptedException
Wait on region to clear regions-in-transition.

Parameters:
hri - Region to wait on.
Throws:
IOException
InterruptedException

waitOnRegionToClearRegionsInTransition

public boolean waitOnRegionToClearRegionsInTransition(HRegionInfo hri,
                                                      long timeOut)
                                               throws InterruptedException
Wait on region to clear regions-in-transition or time out

Parameters:
hri -
timeOut - Milliseconds to wait for current region to be out of transition state.
Returns:
True when a region clears regions-in-transition before timeout otherwise false
Throws:
InterruptedException

isCarryingMeta

public boolean isCarryingMeta(ServerName serverName)

processServerShutdown

public List<HRegionInfo> processServerShutdown(ServerName sn)
Process shutdown server removing any assignments.

Parameters:
sn - Server that went down.
Returns:
list of regions in transition on this server

balance

public void balance(RegionPlan plan)
Parameters:
plan - Plan to execute.

stop

public void stop()

shutdown

public void shutdown()
Shutdown the threadpool executor service


setEnabledTable

protected void setEnabledTable(TableName tableName)

onRegionTransition

protected String onRegionTransition(ServerName serverName,
                                    RegionServerStatusProtos.RegionStateTransition transition)
Try to update some region states. If the state machine prevents such update, an error message is returned to explain the reason. It's expected that in each transition there should have just one region for opening/closing, 3 regions for splitting/merging. These regions should be on the server that requested the change. Region state machine. Only these transitions are expected to be triggered by a region server. On the state transition: (1) Open/Close should be initiated by master (a) Master sets the region to pending_open/pending_close in memory and hbase:meta after sending the request to the region server (b) Region server reports back to the master after open/close is done (either success/failure) (c) If region server has problem to report the status to master, it must be because the master is down or some temporary network issue. Otherwise, the region server should abort since it must be a bug. If the master is not accessible, the region server should keep trying until the server is stopped or till the status is reported to the (new) master (d) If region server dies in the middle of opening/closing a region, SSH picks it up and finishes it (e) If master dies in the middle, the new master recovers the state during initialization from hbase:meta. Region server can report any transition that has not been reported to the previous active master yet (2) Split/merge is initiated by region servers (a) To split a region, a region server sends a request to master to try to set a region to splitting, together with two daughters (to be created) to splitting new. If approved by the master, the splitting can then move ahead (b) To merge two regions, a region server sends a request to master to try to set the new merged region (to be created) to merging_new, together with two regions (to be merged) to merging. If it is ok with the master, the merge can then move ahead (c) Once the splitting/merging is done, the region server reports the status back to the master either success/failure. (d) Other scenarios should be handled similarly as for region open/close


getBalancer

public LoadBalancer getBalancer()
Returns:
Instance of load balancer


Copyright © 2015 The Apache Software Foundation. All rights reserved.