org.apache.hadoop.hbase.regionserver
Class HRegionServer

java.lang.Object
  extended by org.apache.hadoop.hbase.regionserver.HRegionServer
All Implemented Interfaces:
Runnable, HConstants, HBaseRPCErrorHandler, HBaseRPCProtocolVersion, HRegionInterface, org.apache.hadoop.ipc.VersionedProtocol, org.apache.zookeeper.Watcher
Direct Known Subclasses:
TransactionalRegionServer

public class HRegionServer
extends Object
implements HConstants, HRegionInterface, HBaseRPCErrorHandler, Runnable, org.apache.zookeeper.Watcher

HRegionServer makes a set of HRegions available to clients. It checks in with the HMaster. There are many HRegionServers in a single HBase deployment.


Nested Class Summary
 
Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.HConstants
HConstants.Modify
 
Nested classes/interfaces inherited from interface org.apache.zookeeper.Watcher
org.apache.zookeeper.Watcher.Event
 
Field Summary
protected  boolean abortRequested
           
protected  HBaseConfiguration conf
           
protected  boolean fsOk
           
protected  AtomicBoolean haveRootRegion
           
protected  HLog hlog
           
protected  boolean isOnline
           
protected  ReentrantReadWriteLock lock
           
protected  int numRegionsToReport
           
protected  Map<Integer,HRegion> onlineRegions
           
protected  AtomicBoolean quiesced
           
static String REGIONSERVER
          region server process name
protected  AtomicBoolean safeMode
           
protected  HServerInfo serverInfo
           
protected  AtomicBoolean stopRequested
           
protected  int threadWakeFrequency
           
 
Fields inherited from interface org.apache.hadoop.hbase.HConstants
ALL_VERSIONS, CATALOG_FAMILY, CATALOG_FAMILY_STR, CATALOG_HISTORIAN_FAMILY, CLUSTER_DISTRIBUTED, CLUSTER_IS_DISTRIBUTED, CLUSTER_IS_LOCAL, DEFAULT_HOST, DEFAULT_MASTER_INFOPORT, DEFAULT_MASTER_PORT, DEFAULT_MAX_FILE_SIZE, DEFAULT_REGION_SERVER_CLASS, DEFAULT_REGIONSERVER_INFOPORT, DEFAULT_REGIONSERVER_PORT, DEFAULT_SIZE_RESERVATION_BLOCK, DEFAULT_ZOOKEEPER_PAUSE, DEFAULT_ZOOKEEPER_RETRIES, EMPTY_BYTE_ARRAY, EMPTY_END_ROW, EMPTY_START_ROW, FILE_SYSTEM_VERSION, FOREVER, HBASE_CLIENT_RETRIES_NUMBER_KEY, HBASE_DIR, HREGION_COMPACTIONDIR_NAME, HREGION_LOGDIR_NAME, HREGION_OLDLOGFILE_NAME, IN_MEMORY, LAST_ROW, LATEST_TIMESTAMP, LATEST_TIMESTAMP_BYTES, MAJOR_COMPACTION_PERIOD, MASTER_PORT, MAX_ROW_LENGTH, MAXIMUM_VALUE_LENGTH, META_ROW_DELIMITER, META_TABLE_NAME, NAME, NINES, REGION_SERVER_CLASS, REGION_SERVER_IMPL, REGIONINFO_QUALIFIER, REGIONSERVER_PORT, RETRY_BACKOFF, ROOT_TABLE_NAME, SERVER_QUALIFIER, SPLITA_QUALIFIER, SPLITB_QUALIFIER, STARTCODE_QUALIFIER, THREAD_WAKE_FREQUENCY, UTF8_ENCODING, VERSION_FILE_NAME, VERSIONS, WEEK_IN_SECONDS, ZERO_L, ZEROES, ZOOKEEPER_CONFIG_NAME, ZOOKEEPER_PAUSE, ZOOKEEPER_QUORUM, ZOOKEEPER_RETRIES
 
Fields inherited from interface org.apache.hadoop.hbase.ipc.HBaseRPCProtocolVersion
versionID
 
Constructor Summary
HRegionServer(HBaseConfiguration conf)
          Starts a HRegionServer at the default location
 
Method Summary
 void abort()
          Cause the server to exit without closing the regions it is serving, the log it is using and without notifying the master.
 void addProcessingMessage(HRegionInfo hri)
          Add a MSG_REPORT_PROCESS_OPEN to the outbound queue.
protected  long addRowLock(Integer r, HRegion region)
           
protected  long addScanner(InternalScanner s)
           
 boolean checkAndPut(byte[] regionName, byte[] row, byte[] family, byte[] qualifier, byte[] value, Put put)
          Atomically checks if a row/family/qualifier value match the expectedValue.
protected  boolean checkFileSystem()
          Checks to see if the file system is still accessible.
 boolean checkOOME(Throwable e)
          Take actions on the event of an OutOfMemoryError.
protected  void checkOpen()
          Called to verify that this server is up and running.
 void close(long scannerId)
          Close a scanner
protected  void closeRegion(HRegionInfo hri, boolean reportWhenCompleted)
           
 HServerLoad.RegionLoad createRegionLoad(byte[] regionName)
           
 void delete(byte[] regionName, Delete delete)
          Deletes all the KeyValues that match those found in the Delete object, if their ts <= to the Delete.
 int delete(byte[] regionName, Delete[] deletes)
          Put an array of deletes into the specified region
protected static void doMain(String[] args, Class<? extends HRegionServer> regionServerClass)
          Do class main.
protected  void doMetrics()
           
 boolean exists(byte[] regionName, Get get)
          Perform exists operation.
 Result get(byte[] regionName, Get get)
          Perform Get operation.
 Result getClosestRowBefore(byte[] regionName, byte[] row, byte[] family)
          Return all the data for the row that matches row exactly, or the one that immediately preceeds it.
 HBaseConfiguration getConfiguration()
           
 SortedMap<Long,HRegion> getCopyOfOnlineRegionsSortedBySize()
           
protected  org.apache.hadoop.fs.FileSystem getFileSystem()
           
 FlushRequester getFlushRequester()
           
 long getGlobalMemStoreSize()
          Return the total size of all memstores in every region.
 HServerInfo getHServerInfo()
          Method used when a master is taking the place of another failed one.
 InfoServer getInfoServer()
           
protected  Leases getLeases()
           
protected  org.apache.hadoop.hbase.regionserver.LogRoller getLogRoller()
           
 RegionServerMetrics getMetrics()
           
protected  HRegionInfo[] getMostLoadedRegions()
          Get the top N most loaded regions this server is serving so we can tell the master which regions it can reallocate if we're overloaded.
 HRegion getOnlineRegion(byte[] regionName)
           
 Collection<HRegion> getOnlineRegions()
           
 HRegion[] getOnlineRegionsAsArray()
           
protected  List<HMsg> getOutboundMsgs()
           
 long getProtocolVersion(String protocol, long clientVersion)
           
protected  HRegion getRegion(byte[] regionName)
          Protected utility method for safely obtaining an HRegion handle.
 HRegionInfo getRegionInfo(byte[] regionName)
          Get metainfo about an HRegion
 HRegionInfo[] getRegionsAssignment()
          Method used when a master is taking the place of another failed one.
protected  Set<HRegion> getRegionsToCheck()
           
 AtomicInteger getRequestCount()
           
protected  org.apache.hadoop.fs.Path getRootDir()
           
 SortedSet<HRegionInfo> getSortedOnlineRegionInfos()
           
 ZooKeeperWrapper getZooKeeperWrapper()
           
 long incrementColumnValue(byte[] regionName, byte[] row, byte[] family, byte[] qualifier, long amount, boolean writeToWAL)
          Atomically increments a column value.
protected  void init(org.apache.hadoop.io.MapWritable c)
           
protected  HLog instantiateHLog(org.apache.hadoop.fs.Path logdir)
           
protected  HRegion instantiateRegion(HRegionInfo regionInfo)
           
 boolean isInSafeMode()
           
 boolean isOnline()
          Report the status of the server.
 boolean isStopRequested()
           
 long lockRow(byte[] regionName, byte[] row)
          Opens a remote row lock.
static void main(String[] args)
           
protected  void metrics()
           
 Result next(long scannerId)
          Get the next set of values
 Result[] next(long scannerId, int nbRows)
          Get the next set of values
 long openScanner(byte[] regionName, Scan scan)
          Opens a remote scanner with a RowFilter.
 void process(org.apache.zookeeper.WatchedEvent event)
          We register ourselves as a watcher on the master address ZNode.
 void put(byte[] regionName, Put put)
          Put data into the specified region
 int put(byte[] regionName, Put[] puts)
          Put an array of puts into the specified region
 void run()
          The HRegionServer sticks in this loop until closed.
 void runThread(Thread t, long dfsShutdownWait)
          Run and wait on passed thread in HRS context.
 Thread setHDFSShutdownThreadOnExit(Thread t)
          Set the hdfs shutdown thread to run on exit.
 void stop()
          Sets a flag that will cause all the HRegionServer threads to shut down in an orderly fashion.
 void unlockRow(byte[] regionName, long lockId)
          Releases a remote row lock.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

stopRequested

protected final AtomicBoolean stopRequested

quiesced

protected final AtomicBoolean quiesced

safeMode

protected final AtomicBoolean safeMode

abortRequested

protected volatile boolean abortRequested

fsOk

protected volatile boolean fsOk

serverInfo

protected HServerInfo serverInfo

conf

protected final HBaseConfiguration conf

haveRootRegion

protected final AtomicBoolean haveRootRegion

onlineRegions

protected final Map<Integer,HRegion> onlineRegions

lock

protected final ReentrantReadWriteLock lock

threadWakeFrequency

protected final int threadWakeFrequency

numRegionsToReport

protected final int numRegionsToReport

REGIONSERVER

public static final String REGIONSERVER
region server process name

See Also:
Constant Field Values

hlog

protected volatile HLog hlog

isOnline

protected volatile boolean isOnline
Constructor Detail

HRegionServer

public HRegionServer(HBaseConfiguration conf)
              throws IOException
Starts a HRegionServer at the default location

Parameters:
conf -
Throws:
IOException
Method Detail

process

public void process(org.apache.zookeeper.WatchedEvent event)
We register ourselves as a watcher on the master address ZNode. This is called by ZooKeeper when we get an event on that ZNode. When this method is called it means either our master has died, or a new one has come up. Either way we need to update our knowledge of the master.

Specified by:
process in interface org.apache.zookeeper.Watcher
Parameters:
event - WatchedEvent from ZooKeeper.

getZooKeeperWrapper

public ZooKeeperWrapper getZooKeeperWrapper()
Returns:
ZooKeeperWrapper used by RegionServer.

run

public void run()
The HRegionServer sticks in this loop until closed. It repeatedly checks in with the HMaster, sending heartbeats & reports, and receiving HRegion load/unload instructions.

Specified by:
run in interface Runnable

runThread

public void runThread(Thread t,
                      long dfsShutdownWait)
Run and wait on passed thread in HRS context.

Parameters:
t -
dfsShutdownWait -

setHDFSShutdownThreadOnExit

public Thread setHDFSShutdownThreadOnExit(Thread t)
Set the hdfs shutdown thread to run on exit. Pass null to disable running of the shutdown test. Needed by tests.

Parameters:
t - Thread to run. Pass null to disable tests.
Returns:
Previous occupant of the shutdown thread position.

init

protected void init(org.apache.hadoop.io.MapWritable c)
             throws IOException
Throws:
IOException

createRegionLoad

public HServerLoad.RegionLoad createRegionLoad(byte[] regionName)
Parameters:
regionName -
Returns:
An instance of RegionLoad.
Throws:
IOException

checkOOME

public boolean checkOOME(Throwable e)
Description copied from interface: HBaseRPCErrorHandler
Take actions on the event of an OutOfMemoryError.

Specified by:
checkOOME in interface HBaseRPCErrorHandler
Parameters:
e - the throwable
Returns:
if the server should be shut down

checkFileSystem

protected boolean checkFileSystem()
Checks to see if the file system is still accessible. If not, sets abortRequested and stopRequested

Returns:
false if file system is not available

isOnline

public boolean isOnline()
Report the status of the server. A server is online once all the startup is completed (setting up filesystem, starting service threads, etc.). This method is designed mostly to be useful in tests.

Returns:
true if online, false if not.

instantiateHLog

protected HLog instantiateHLog(org.apache.hadoop.fs.Path logdir)
                        throws IOException
Throws:
IOException

getLogRoller

protected org.apache.hadoop.hbase.regionserver.LogRoller getLogRoller()

doMetrics

protected void doMetrics()

metrics

protected void metrics()

getMetrics

public RegionServerMetrics getMetrics()
Returns:
Region server metrics instance.

stop

public void stop()
Sets a flag that will cause all the HRegionServer threads to shut down in an orderly fashion. Used by unit tests.


abort

public void abort()
Cause the server to exit without closing the regions it is serving, the log it is using and without notifying the master. Used unit testing and on catastrophic events such as HDFS is yanked out from under hbase or we OOME.


instantiateRegion

protected HRegion instantiateRegion(HRegionInfo regionInfo)
                             throws IOException
Throws:
IOException

addProcessingMessage

public void addProcessingMessage(HRegionInfo hri)
Add a MSG_REPORT_PROCESS_OPEN to the outbound queue. This method is called while region is in the queue of regions to process and then while the region is being opened, it is called from the Worker thread that is running the region open.

Parameters:
hri - Region to add the message for

closeRegion

protected void closeRegion(HRegionInfo hri,
                           boolean reportWhenCompleted)
                    throws IOException
Throws:
IOException

getRegionInfo

public HRegionInfo getRegionInfo(byte[] regionName)
                          throws NotServingRegionException
Description copied from interface: HRegionInterface
Get metainfo about an HRegion

Specified by:
getRegionInfo in interface HRegionInterface
Parameters:
regionName - name of the region
Returns:
HRegionInfo object for region
Throws:
NotServingRegionException

getClosestRowBefore

public Result getClosestRowBefore(byte[] regionName,
                                  byte[] row,
                                  byte[] family)
                           throws IOException
Description copied from interface: HRegionInterface
Return all the data for the row that matches row exactly, or the one that immediately preceeds it.

Specified by:
getClosestRowBefore in interface HRegionInterface
Parameters:
regionName - region name
row - row key
family - Column family to look for row in.
Returns:
map of values
Throws:
IOException

get

public Result get(byte[] regionName,
                  Get get)
           throws IOException
Perform Get operation.

Specified by:
get in interface HRegionInterface
Parameters:
regionName - name of region to get from
get - Get operation
Returns:
Result
Throws:
IOException

exists

public boolean exists(byte[] regionName,
                      Get get)
               throws IOException
Description copied from interface: HRegionInterface
Perform exists operation.

Specified by:
exists in interface HRegionInterface
Parameters:
regionName - name of region to get from
get - Get operation describing cell to test
Returns:
true if exists
Throws:
IOException

put

public void put(byte[] regionName,
                Put put)
         throws IOException
Description copied from interface: HRegionInterface
Put data into the specified region

Specified by:
put in interface HRegionInterface
put - the data to be put
Throws:
IOException

put

public int put(byte[] regionName,
               Put[] puts)
        throws IOException
Description copied from interface: HRegionInterface
Put an array of puts into the specified region

Specified by:
put in interface HRegionInterface
Returns:
The number of processed put's. Returns -1 if all Puts processed successfully.
Throws:
IOException

checkAndPut

public boolean checkAndPut(byte[] regionName,
                           byte[] row,
                           byte[] family,
                           byte[] qualifier,
                           byte[] value,
                           Put put)
                    throws IOException
Description copied from interface: HRegionInterface
Atomically checks if a row/family/qualifier value match the expectedValue. If it does, it adds the put.

Specified by:
checkAndPut in interface HRegionInterface
Parameters:
regionName -
row -
family -
qualifier -
value - the expected value
put -
Returns:
true if the new put was execute, false otherwise
Throws:
IOException

openScanner

public long openScanner(byte[] regionName,
                        Scan scan)
                 throws IOException
Description copied from interface: HRegionInterface
Opens a remote scanner with a RowFilter.

Specified by:
openScanner in interface HRegionInterface
Parameters:
regionName - name of region to scan
scan - configured scan object
Returns:
scannerId scanner identifier used in other calls
Throws:
IOException

addScanner

protected long addScanner(InternalScanner s)
                   throws Leases.LeaseStillHeldException
Throws:
Leases.LeaseStillHeldException

next

public Result next(long scannerId)
            throws IOException
Description copied from interface: HRegionInterface
Get the next set of values

Specified by:
next in interface HRegionInterface
Parameters:
scannerId - clientId passed to openScanner
Returns:
map of values; returns null if no results.
Throws:
IOException

next

public Result[] next(long scannerId,
                     int nbRows)
              throws IOException
Description copied from interface: HRegionInterface
Get the next set of values

Specified by:
next in interface HRegionInterface
Parameters:
scannerId - clientId passed to openScanner
nbRows - the number of rows to fetch
Returns:
Array of Results (map of values); array is empty if done with this region and null if we are NOT to go to the next region (happens when a filter rules that the scan is done).
Throws:
IOException

close

public void close(long scannerId)
           throws IOException
Description copied from interface: HRegionInterface
Close a scanner

Specified by:
close in interface HRegionInterface
Parameters:
scannerId - the scanner id returned by openScanner
Throws:
IOException

delete

public void delete(byte[] regionName,
                   Delete delete)
            throws IOException
Description copied from interface: HRegionInterface
Deletes all the KeyValues that match those found in the Delete object, if their ts <= to the Delete. In case of a delete with a specific ts it only deletes that specific KeyValue.

Specified by:
delete in interface HRegionInterface
Throws:
IOException

delete

public int delete(byte[] regionName,
                  Delete[] deletes)
           throws IOException
Description copied from interface: HRegionInterface
Put an array of deletes into the specified region

Specified by:
delete in interface HRegionInterface
Returns:
The number of processed deletes. Returns -1 if all Deletes processed successfully.
Throws:
IOException

lockRow

public long lockRow(byte[] regionName,
                    byte[] row)
             throws IOException
Description copied from interface: HRegionInterface
Opens a remote row lock.

Specified by:
lockRow in interface HRegionInterface
Parameters:
regionName - name of region
row - row to lock
Returns:
lockId lock identifier
Throws:
IOException

addRowLock

protected long addRowLock(Integer r,
                          HRegion region)
                   throws Leases.LeaseStillHeldException
Throws:
Leases.LeaseStillHeldException

unlockRow

public void unlockRow(byte[] regionName,
                      long lockId)
               throws IOException
Description copied from interface: HRegionInterface
Releases a remote row lock.

Specified by:
unlockRow in interface HRegionInterface
lockId - the lock id returned by lockRow
Throws:
IOException

getInfoServer

public InfoServer getInfoServer()
Returns:
the info server

isStopRequested

public boolean isStopRequested()
Returns:
true if a stop has been requested.

isInSafeMode

public boolean isInSafeMode()
Returns:
true if the region server is in safe mode

getConfiguration

public HBaseConfiguration getConfiguration()
Returns:
the configuration

getOnlineRegions

public Collection<HRegion> getOnlineRegions()
Returns:
Immutable list of this servers regions.

getOnlineRegionsAsArray

public HRegion[] getOnlineRegionsAsArray()
Specified by:
getOnlineRegionsAsArray in interface HRegionInterface
Returns:
the regions served by this regionserver

getSortedOnlineRegionInfos

public SortedSet<HRegionInfo> getSortedOnlineRegionInfos()
Returns:
The HRegionInfos from online regions sorted

getCopyOfOnlineRegionsSortedBySize

public SortedMap<Long,HRegion> getCopyOfOnlineRegionsSortedBySize()
Returns:
A new Map of online regions sorted by region size with the first entry being the biggest.

getOnlineRegion

public HRegion getOnlineRegion(byte[] regionName)
Parameters:
regionName -
Returns:
HRegion for the passed regionName or null if named region is not member of the online regions.

getRequestCount

public AtomicInteger getRequestCount()
Returns:
the request count

getFlushRequester

public FlushRequester getFlushRequester()
Returns:
reference to FlushRequester

getRegion

protected HRegion getRegion(byte[] regionName)
                     throws NotServingRegionException
Protected utility method for safely obtaining an HRegion handle.

Parameters:
regionName - Name of online HRegion to return
Returns:
HRegion for regionName
Throws:
NotServingRegionException

getMostLoadedRegions

protected HRegionInfo[] getMostLoadedRegions()
Get the top N most loaded regions this server is serving so we can tell the master which regions it can reallocate if we're overloaded. TODO: actually calculate which regions are most loaded. (Right now, we're just grabbing the first N regions being served regardless of load.)


checkOpen

protected void checkOpen()
                  throws IOException
Called to verify that this server is up and running.

Throws:
IOException

getRegionsToCheck

protected Set<HRegion> getRegionsToCheck()
Returns:
Returns list of non-closed regions hosted on this server. If no regions to check, returns an empty list.

getProtocolVersion

public long getProtocolVersion(String protocol,
                               long clientVersion)
                        throws IOException
Specified by:
getProtocolVersion in interface org.apache.hadoop.ipc.VersionedProtocol
Throws:
IOException

getOutboundMsgs

protected List<HMsg> getOutboundMsgs()
Returns:
Queue to which you can add outbound messages.

getGlobalMemStoreSize

public long getGlobalMemStoreSize()
Return the total size of all memstores in every region.

Returns:
memstore size in bytes

getLeases

protected Leases getLeases()
Returns:
Return the leases.

getRootDir

protected org.apache.hadoop.fs.Path getRootDir()
Returns:
Return the rootDir.

getFileSystem

protected org.apache.hadoop.fs.FileSystem getFileSystem()
Returns:
Return the fs.

doMain

protected static void doMain(String[] args,
                             Class<? extends HRegionServer> regionServerClass)
Do class main.

Parameters:
args -
regionServerClass - HRegionServer to instantiate.

incrementColumnValue

public long incrementColumnValue(byte[] regionName,
                                 byte[] row,
                                 byte[] family,
                                 byte[] qualifier,
                                 long amount,
                                 boolean writeToWAL)
                          throws IOException
Atomically increments a column value. If the column value isn't long-like, this could throw an exception.

Specified by:
incrementColumnValue in interface HRegionInterface
writeToWAL - whether to write the increment to the WAL
Returns:
new incremented column value
Throws:
IOException

getRegionsAssignment

public HRegionInfo[] getRegionsAssignment()
                                   throws IOException
Method used when a master is taking the place of another failed one.

Specified by:
getRegionsAssignment in interface HRegionInterface
Returns:
All regions assigned on this region server
Throws:
IOException

getHServerInfo

public HServerInfo getHServerInfo()
                           throws IOException
Method used when a master is taking the place of another failed one.

Specified by:
getHServerInfo in interface HRegionInterface
Returns:
The HSI
Throws:
IOException

main

public static void main(String[] args)
Parameters:
args -


Copyright © 2009 The Apache Software Foundation