org.apache.hadoop.hbase.master
Class HMaster

java.lang.Object
  extended by org.apache.hadoop.hbase.util.HasThread
      extended by org.apache.hadoop.hbase.master.HMaster
All Implemented Interfaces:
Runnable, Abortable, HMasterInterface, HMasterRegionInterface, VersionedProtocol, MasterServices, Server, Stoppable
Direct Known Subclasses:
HMasterCommandLine.LocalHMaster

public class HMaster
extends HasThread
implements HMasterInterface, HMasterRegionInterface, MasterServices, Server

HMaster is the "master server" for HBase. An HBase cluster has one active master. If many masters are started, all compete. Whichever wins goes on to run the cluster. All others park themselves in their constructor until master or cluster shutdown or until the active master loses its lease in zookeeper. Thereafter, all running master jostle to take over master role.

The Master can be asked shutdown the cluster. See shutdown(). In this case it will tell all regionservers to go down and then wait on them all reporting in that they are down. This master will then shut itself down.

You can also shutdown just this master. Call stopMaster().

See Also:
HMasterInterface, HMasterRegionInterface, Watcher

Field Summary
static String MASTER
           
 
Fields inherited from interface org.apache.hadoop.hbase.ipc.HMasterInterface
VERSION
 
Fields inherited from interface org.apache.hadoop.hbase.ipc.HMasterRegionInterface
VERSION
 
Constructor Summary
HMaster(org.apache.hadoop.conf.Configuration conf)
          Initializes the HMaster.
 
Method Summary
 void abort(String msg, Throwable t)
          Abort the server or client.
 void addColumn(byte[] tableName, HColumnDescriptor column)
          Adds a column to the specified table
 void assign(byte[] regionName)
          Assign a region to a server chosen at random.
 void assign(byte[] regionName, boolean force)
          Deprecated. 
 void assignRegion(HRegionInfo hri)
           
 boolean balance()
          Run the balancer.
 boolean balanceSwitch(boolean b)
          Turn the load balancer on or off.
 void checkTableModifiable(byte[] tableName)
          Check table is modifiable; i.e.
 void clearFromTransition(HRegionInfo hri)
           
static HMaster constructMaster(Class<? extends HMaster> masterClass, org.apache.hadoop.conf.Configuration conf)
          Utility for constructing an instance of the passed HMaster class.
protected  org.apache.hadoop.io.MapWritable createConfigurationSubset()
           
 void createTable(HTableDescriptor hTableDescriptor, byte[][] splitKeys)
          Creates a new table asynchronously.
 void deleteColumn(byte[] tableName, byte[] c)
          Deletes a column from the specified table.
 void deleteTable(byte[] tableName)
          Deletes a table
 void disableTable(byte[] tableName)
          Take table offline
 void enableTable(byte[] tableName)
          Puts the table on-line (only needed if table has been previously taken offline)
 Pair<Integer,Integer> getAlterStatus(byte[] tableName)
          Get the number of regions of the table that have been updated by the alter.
 AssignmentManager getAssignmentManager()
           
 double getAverageLoad()
          Compute the average load across all region servers.
 CatalogTracker getCatalogTracker()
           
 String getClusterId()
           
 ClusterStatus getClusterStatus()
          Return cluster status.
 org.apache.hadoop.conf.Configuration getConfiguration()
          Gets the configuration object for this server.
 MasterCoprocessorHost getCoprocessorHost()
           
 String[] getCoprocessors()
           
 ExecutorService getExecutorService()
           
 HTableDescriptor[] getHTableDescriptors()
          Get all table descriptors
 HTableDescriptor[] getHTableDescriptors(List<String> tableNames)
          Get HTD array for given tables
 InfoServer getInfoServer()
           
static String getLoadedCoprocessors()
          The set of loaded coprocessors is stored in a static set.
 long getMasterActiveTime()
           
 MasterFileSystem getMasterFileSystem()
           
 long getMasterStartTime()
           
 ProtocolSignature getProtocolSignature(String protocol, long version, int clientMethodsHashCode)
          Return protocol version corresponding to protocol interface.
 long getProtocolVersion(String protocol, long clientVersion)
          Return protocol version corresponding to protocol interface.
 MemoryBoundedLogMessageBuffer getRegionServerFatalLogBuffer()
           
 ServerManager getServerManager()
           
 ServerName getServerName()
           
 TableDescriptors getTableDescriptors()
           
 ZooKeeperWatcher getZooKeeper()
          Gets the ZooKeeper instance for this server.
 ZooKeeperWatcher getZooKeeperWatcher()
          Get the ZK wrapper object - needed by master_jsp.java
 boolean isAborted()
          Check if the server or client was aborted.
 boolean isActiveMaster()
          Report whether this master is currently the active master or not.
 boolean isInitialized()
          Report whether this master has completed with its initialization and is ready.
 boolean isMasterRunning()
           
 boolean isServerShutdownHandlerEnabled()
          ServerShutdownHandlerEnabled is set false before completing assignRootAndMeta to prevent processing of ServerShutdownHandler.
 boolean isStopped()
           
static void main(String[] args)
           
 void modifyColumn(byte[] tableName, HColumnDescriptor descriptor)
          Modifies an existing column on the specified table
 void modifyTable(byte[] tableName, HTableDescriptor htd)
          Modify a table's metadata
 void move(byte[] encodedRegionName, byte[] destServerName)
          Move the region r to dest.
 void offline(byte[] regionName)
          Special method, only used by hbck.
 void regionServerReport(byte[] sn, HServerLoad hsl)
           
 org.apache.hadoop.io.MapWritable regionServerStartup(int port, long serverStartCode, long serverCurrentTime)
          Called when a region server first starts.
 void reportRSFatalError(byte[] sn, String errorText)
          Called by a region server to report a fatal error that is causing it to abort.
 void run()
          Main processing loop for the HMaster.
 void setCatalogJanitorEnabled(boolean b)
          Switch for the background CatalogJanitor thread.
 void shutdown()
          Shutdown an HBase cluster.
protected  void splitLogAfterStartup(MasterFileSystem mfs)
          Override to change master's splitLogAfterStartup.
protected  void startCatalogJanitorChore()
          Useful for testing purpose also where we have master restart scenarios.
 void stop(String why)
          Stop this service.
 void stopMaster()
          Stop HBase Master only.
 boolean switchBalancer(boolean b, org.apache.hadoop.hbase.master.HMaster.BalanceSwitchMode mode)
          Assigns balancer switch according to BalanceSwitchMode
 boolean synchronousBalanceSwitch(boolean b)
          Turn the load balancer on or off.
 void unassign(byte[] regionName, boolean force)
          Unassign a region from current hosting regionserver.
 
Methods inherited from class org.apache.hadoop.hbase.util.HasThread
getName, getThread, interrupt, isAlive, isInterrupted, join, join, join, setDaemon, setName, setPriority, setUncaughtExceptionHandler, start
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

MASTER

public static final String MASTER
See Also:
Constant Field Values
Constructor Detail

HMaster

public HMaster(org.apache.hadoop.conf.Configuration conf)
        throws IOException,
               org.apache.zookeeper.KeeperException,
               InterruptedException
Initializes the HMaster. The steps are as follows:

  1. Initialize HMaster RPC and address
  2. Connect to ZooKeeper.

Remaining steps of initialization occur in run() so that they run in their own thread rather than within the context of the constructor.

Throws:
InterruptedException
IOException
org.apache.zookeeper.KeeperException
Method Detail

run

public void run()
Main processing loop for the HMaster.
  1. Block until becoming active master
  2. Finish initialization via finishInitialization(MonitoredTask)
  3. Enter loop until we are stopped
  4. Stop services and perform cleanup once stopped

Specified by:
run in interface Runnable
Specified by:
run in class HasThread

startCatalogJanitorChore

protected void startCatalogJanitorChore()
Useful for testing purpose also where we have master restart scenarios.


splitLogAfterStartup

protected void splitLogAfterStartup(MasterFileSystem mfs)
Override to change master's splitLogAfterStartup. Used testing

Parameters:
mfs -

getProtocolSignature

public ProtocolSignature getProtocolSignature(String protocol,
                                              long version,
                                              int clientMethodsHashCode)
                                       throws IOException
Description copied from interface: VersionedProtocol
Return protocol version corresponding to protocol interface.

Specified by:
getProtocolSignature in interface VersionedProtocol
Parameters:
protocol - The classname of the protocol interface
version - The version of the protocol that the client speaks
clientMethodsHashCode - the hashcode of client protocol methods
Returns:
the server protocol signature containing its version and a list of its supported methods
Throws:
IOException
See Also:
for a default implementation

getProtocolVersion

public long getProtocolVersion(String protocol,
                               long clientVersion)
Description copied from interface: VersionedProtocol
Return protocol version corresponding to protocol interface.

Specified by:
getProtocolVersion in interface VersionedProtocol
Parameters:
protocol - The classname of the protocol interface
clientVersion - The version of the protocol that the client speaks
Returns:
the version that the server will speak

getTableDescriptors

public TableDescriptors getTableDescriptors()
Specified by:
getTableDescriptors in interface MasterServices
Returns:
Return table descriptors implementation.

getInfoServer

public InfoServer getInfoServer()
Returns:
InfoServer object. Maybe null.

getConfiguration

public org.apache.hadoop.conf.Configuration getConfiguration()
Description copied from interface: Server
Gets the configuration object for this server.

Specified by:
getConfiguration in interface Server

getServerManager

public ServerManager getServerManager()
Specified by:
getServerManager in interface MasterServices
Returns:
Master's ServerManager instance.

getExecutorService

public ExecutorService getExecutorService()
Specified by:
getExecutorService in interface MasterServices
Returns:
Master's instance of ExecutorService

getMasterFileSystem

public MasterFileSystem getMasterFileSystem()
Specified by:
getMasterFileSystem in interface MasterServices
Returns:
Master's filesystem MasterFileSystem utility class.

getZooKeeperWatcher

public ZooKeeperWatcher getZooKeeperWatcher()
Get the ZK wrapper object - needed by master_jsp.java

Returns:
the zookeeper wrapper

regionServerStartup

public org.apache.hadoop.io.MapWritable regionServerStartup(int port,
                                                            long serverStartCode,
                                                            long serverCurrentTime)
                                                     throws IOException
Description copied from interface: HMasterRegionInterface
Called when a region server first starts.

Specified by:
regionServerStartup in interface HMasterRegionInterface
Parameters:
port - Port number this regionserver is up on.
serverStartCode - This servers' startcode.
serverCurrentTime - The current time of the region server in ms
Returns:
Configuration for the regionserver to use: e.g. filesystem, hbase rootdir, the hostname to use creating the RegionServer ServerName, etc.
Throws:
IOException - e

createConfigurationSubset

protected org.apache.hadoop.io.MapWritable createConfigurationSubset()
Returns:
Subset of configuration to pass initializing regionservers: e.g. the filesystem to use and root directory to use.

regionServerReport

public void regionServerReport(byte[] sn,
                               HServerLoad hsl)
                        throws IOException
Specified by:
regionServerReport in interface HMasterRegionInterface
Parameters:
sn - ServerName.getVersionedBytes()
hsl - Server load.
Throws:
IOException

reportRSFatalError

public void reportRSFatalError(byte[] sn,
                               String errorText)
Description copied from interface: HMasterRegionInterface
Called by a region server to report a fatal error that is causing it to abort.

Specified by:
reportRSFatalError in interface HMasterRegionInterface
Parameters:
sn - ServerName.getVersionedBytes()
errorText - informative text to expose in the master logs and UI

isMasterRunning

public boolean isMasterRunning()
Specified by:
isMasterRunning in interface HMasterInterface
Returns:
true if master is available

balance

public boolean balance()
Description copied from interface: HMasterInterface
Run the balancer. Will run the balancer and if regions to move, it will go ahead and do the reassignments. Can NOT run for various reasons. Check logs.

Specified by:
balance in interface HMasterInterface
Returns:
True if balancer ran and was able to tell the region servers to unassign all the regions to balance (the re-assignment itself is async), false otherwise.

switchBalancer

public boolean switchBalancer(boolean b,
                              org.apache.hadoop.hbase.master.HMaster.BalanceSwitchMode mode)
Assigns balancer switch according to BalanceSwitchMode

Parameters:
b - new balancer switch
mode - BalanceSwitchMode
Returns:
old balancer switch

synchronousBalanceSwitch

public boolean synchronousBalanceSwitch(boolean b)
Description copied from interface: HMasterInterface
Turn the load balancer on or off. It waits until current balance() call, if outstanding, to return.

Specified by:
synchronousBalanceSwitch in interface HMasterInterface
Parameters:
b - If true, enable balancer. If false, disable balancer.
Returns:
Previous balancer value

balanceSwitch

public boolean balanceSwitch(boolean b)
Description copied from interface: HMasterInterface
Turn the load balancer on or off.

Specified by:
balanceSwitch in interface HMasterInterface
Parameters:
b - If true, enable balancer. If false, disable balancer.
Returns:
Previous balancer value

setCatalogJanitorEnabled

public void setCatalogJanitorEnabled(boolean b)
Switch for the background CatalogJanitor thread. Used for testing. The thread will continue to run. It will just be a noop if disabled.

Parameters:
b - If false, the catalog janitor won't do anything.

move

public void move(byte[] encodedRegionName,
                 byte[] destServerName)
          throws UnknownRegionException
Description copied from interface: HMasterInterface
Move the region r to dest.

Specified by:
move in interface HMasterInterface
Parameters:
encodedRegionName - The encoded region name; i.e. the hash that makes up the region name suffix: e.g. if regionname is TestTable,0094429456,1289497600452.527db22f95c8a9e0116f0cc13c680396., then the encoded region name is: 527db22f95c8a9e0116f0cc13c680396.
destServerName - The servername of the destination regionserver. If passed the empty byte array we'll assign to a random server. A server name is made of host, port and startcode. Here is an example: host187.example.com,60020,1289493121758.
Throws:
UnknownRegionException - Thrown if we can't find a region named encodedRegionName

createTable

public void createTable(HTableDescriptor hTableDescriptor,
                        byte[][] splitKeys)
                 throws IOException
Description copied from interface: HMasterInterface
Creates a new table asynchronously. If splitKeys are specified, then the table will be created with an initial set of multiple regions. If splitKeys is null, the table will be created with a single region.

Specified by:
createTable in interface HMasterInterface
Specified by:
createTable in interface MasterServices
Parameters:
hTableDescriptor - table descriptor
splitKeys - Starting row keys for the initial table regions. If null a single region is created.
Throws:
IOException

deleteTable

public void deleteTable(byte[] tableName)
                 throws IOException
Description copied from interface: HMasterInterface
Deletes a table

Specified by:
deleteTable in interface HMasterInterface
Parameters:
tableName - table to delete
Throws:
IOException - e

getAlterStatus

public Pair<Integer,Integer> getAlterStatus(byte[] tableName)
                                     throws IOException
Get the number of regions of the table that have been updated by the alter.

Specified by:
getAlterStatus in interface HMasterInterface
Returns:
Pair indicating the number of regions updated Pair.getFirst is the regions that are yet to be updated Pair.getSecond is the total number of regions of the table
Throws:
IOException

addColumn

public void addColumn(byte[] tableName,
                      HColumnDescriptor column)
               throws IOException
Description copied from interface: HMasterInterface
Adds a column to the specified table

Specified by:
addColumn in interface HMasterInterface
Parameters:
tableName - table to modify
column - column descriptor
Throws:
IOException - e

modifyColumn

public void modifyColumn(byte[] tableName,
                         HColumnDescriptor descriptor)
                  throws IOException
Description copied from interface: HMasterInterface
Modifies an existing column on the specified table

Specified by:
modifyColumn in interface HMasterInterface
Parameters:
tableName - table name
descriptor - new column descriptor
Throws:
IOException - e

deleteColumn

public void deleteColumn(byte[] tableName,
                         byte[] c)
                  throws IOException
Description copied from interface: HMasterInterface
Deletes a column from the specified table. Table must be disabled.

Specified by:
deleteColumn in interface HMasterInterface
Parameters:
tableName - table to alter
c - column family to remove
Throws:
IOException - e

enableTable

public void enableTable(byte[] tableName)
                 throws IOException
Description copied from interface: HMasterInterface
Puts the table on-line (only needed if table has been previously taken offline)

Specified by:
enableTable in interface HMasterInterface
Parameters:
tableName - table to enable
Throws:
IOException - e

disableTable

public void disableTable(byte[] tableName)
                  throws IOException
Description copied from interface: HMasterInterface
Take table offline

Specified by:
disableTable in interface HMasterInterface
Parameters:
tableName - table to take offline
Throws:
IOException - e

modifyTable

public void modifyTable(byte[] tableName,
                        HTableDescriptor htd)
                 throws IOException
Description copied from interface: HMasterInterface
Modify a table's metadata

Specified by:
modifyTable in interface HMasterInterface
Parameters:
tableName - table to modify
htd - new descriptor for table
Throws:
IOException - e

checkTableModifiable

public void checkTableModifiable(byte[] tableName)
                          throws IOException
Description copied from interface: MasterServices
Check table is modifiable; i.e. exists and is offline.

Specified by:
checkTableModifiable in interface MasterServices
Parameters:
tableName - Name of table to check.
Throws:
IOException

clearFromTransition

public void clearFromTransition(HRegionInfo hri)

getClusterStatus

public ClusterStatus getClusterStatus()
Description copied from interface: HMasterInterface
Return cluster status.

Specified by:
getClusterStatus in interface HMasterInterface
Returns:
cluster status

getClusterId

public String getClusterId()

getLoadedCoprocessors

public static String getLoadedCoprocessors()
The set of loaded coprocessors is stored in a static set. Since it's statically allocated, it does not require that HMaster's cpHost be initialized prior to accessing it.

Returns:
a String representation of the set of names of the loaded coprocessors.

getMasterStartTime

public long getMasterStartTime()
Returns:
timestamp in millis when HMaster was started.

getMasterActiveTime

public long getMasterActiveTime()
Returns:
timestamp in millis when HMaster became the active master.

getCoprocessors

public String[] getCoprocessors()
Returns:
array of coprocessor SimpleNames.

abort

public void abort(String msg,
                  Throwable t)
Description copied from interface: Abortable
Abort the server or client.

Specified by:
abort in interface Abortable
Parameters:
msg - Why we're aborting.
t - Throwable that caused abort. Can be null.

getZooKeeper

public ZooKeeperWatcher getZooKeeper()
Description copied from interface: Server
Gets the ZooKeeper instance for this server.

Specified by:
getZooKeeper in interface Server

getCoprocessorHost

public MasterCoprocessorHost getCoprocessorHost()

getServerName

public ServerName getServerName()
Specified by:
getServerName in interface Server
Returns:
The unique server name for this server.

getCatalogTracker

public CatalogTracker getCatalogTracker()
Specified by:
getCatalogTracker in interface Server
Returns:
Master's instance of CatalogTracker

getAssignmentManager

public AssignmentManager getAssignmentManager()
Specified by:
getAssignmentManager in interface MasterServices
Returns:
Master's instance of the AssignmentManager

getRegionServerFatalLogBuffer

public MemoryBoundedLogMessageBuffer getRegionServerFatalLogBuffer()

shutdown

public void shutdown()
Description copied from interface: HMasterInterface
Shutdown an HBase cluster.

Specified by:
shutdown in interface HMasterInterface

stopMaster

public void stopMaster()
Description copied from interface: HMasterInterface
Stop HBase Master only. Does not shutdown the cluster.

Specified by:
stopMaster in interface HMasterInterface

stop

public void stop(String why)
Description copied from interface: Stoppable
Stop this service.

Specified by:
stop in interface Stoppable
Parameters:
why - Why we're stopping.

isStopped

public boolean isStopped()
Specified by:
isStopped in interface Stoppable
Returns:
True if Stoppable.stop(String) has been closed.

isAborted

public boolean isAborted()
Description copied from interface: Abortable
Check if the server or client was aborted.

Specified by:
isAborted in interface Abortable
Returns:
true if the server or client was aborted, false otherwise

isActiveMaster

public boolean isActiveMaster()
Report whether this master is currently the active master or not. If not active master, we are parked on ZK waiting to become active. This method is used for testing.

Returns:
true if active master, false if not.

isInitialized

public boolean isInitialized()
Report whether this master has completed with its initialization and is ready. If ready, the master is also the active master. A standby master is never ready. This method is used for testing.

Returns:
true if master is ready to go, false if not.

isServerShutdownHandlerEnabled

public boolean isServerShutdownHandlerEnabled()
ServerShutdownHandlerEnabled is set false before completing assignRootAndMeta to prevent processing of ServerShutdownHandler.

Specified by:
isServerShutdownHandlerEnabled in interface MasterServices
Returns:
true if assignRootAndMeta has completed;

assign

@Deprecated
public void assign(byte[] regionName,
                              boolean force)
            throws IOException
Deprecated. 

Description copied from interface: HMasterInterface
Assign a region to a server chosen at random.

Specified by:
assign in interface HMasterInterface
Parameters:
regionName - Region to assign. Will use existing RegionPlan if one found.
force - If true, will force the assignment.
Throws:
IOException

assign

public void assign(byte[] regionName)
            throws IOException
Description copied from interface: HMasterInterface
Assign a region to a server chosen at random.

Specified by:
assign in interface HMasterInterface
Parameters:
regionName - Region to assign. Will use existing RegionPlan if one found.
Throws:
IOException

assignRegion

public void assignRegion(HRegionInfo hri)

unassign

public void unassign(byte[] regionName,
                     boolean force)
              throws IOException
Description copied from interface: HMasterInterface
Unassign a region from current hosting regionserver. Region will then be assigned to a regionserver chosen at random. Region could be reassigned back to the same server. Use HMasterInterface.move(byte[], byte[]) if you want to control the region movement.

Specified by:
unassign in interface HMasterInterface
Parameters:
regionName - Region to unassign. Will clear any existing RegionPlan if one found.
force - If true, force unassign (Will remove region from regions-in-transition too if present as well as from assigned regions -- radical!.If results in double assignment use hbck -fix to resolve.
Throws:
IOException

getHTableDescriptors

public HTableDescriptor[] getHTableDescriptors(List<String> tableNames)
Get HTD array for given tables

Specified by:
getHTableDescriptors in interface HMasterInterface
Parameters:
tableNames -
Returns:
HTableDescriptor[]

getHTableDescriptors

public HTableDescriptor[] getHTableDescriptors()
Get all table descriptors

Specified by:
getHTableDescriptors in interface HMasterInterface
Returns:
All descriptors or null if none.

getAverageLoad

public double getAverageLoad()
Compute the average load across all region servers. Currently, this uses a very naive computation - just uses the number of regions being served, ignoring stats about number of requests.

Returns:
the average load

offline

public void offline(byte[] regionName)
             throws IOException
Special method, only used by hbck.

Specified by:
offline in interface HMasterInterface
Parameters:
regionName - Region to offline. Will clear any existing RegionPlan if one found.
Throws:
IOException

constructMaster

public static HMaster constructMaster(Class<? extends HMaster> masterClass,
                                      org.apache.hadoop.conf.Configuration conf)
Utility for constructing an instance of the passed HMaster class.

Parameters:
masterClass -
conf -
Returns:
HMaster instance.

main

public static void main(String[] args)
                 throws Exception
Throws:
Exception
See Also:
HMasterCommandLine


Copyright © 2012 The Apache Software Foundation. All Rights Reserved.