org.apache.hadoop.hbase.master
Class SplitLogManager
java.lang.Object
org.apache.hadoop.hbase.zookeeper.ZooKeeperListener
org.apache.hadoop.hbase.master.SplitLogManager
public class SplitLogManager
- extends ZooKeeperListener
Distributes the task of log splitting to the available region servers.
Coordination happens via zookeeper. For every log file that has to be split a
znode is created under /hbase/splitlog. SplitLogWorkers race to grab a task.
SplitLogManager monitors the task znodes that it creates using the
timeoutMonitor thread. If a task's progress is slow then
resubmit(String, boolean) will take away the task from the owner
SplitLogWorker
and the task will be
upforgrabs again. When the task is done then the task's znode is deleted by
SplitLogManager.
Clients call splitLogDistributed(Path)
to split a region server's
log files. The caller thread waits in this method until all the log files
have been split.
All the zookeeper calls made by this class are asynchronous. This is mainly
to help reduce response time seen by the callers.
There is race in this design between the SplitLogManager and the
SplitLogWorker. SplitLogManager might re-queue a task that has in reality
already been completed by a SplitLogWorker. We rely on the idempotency of
the log splitting task for correctness.
It is also assumed that every log splitting task is unique and once
completed (either with success or with error) it will be not be submitted
again. If a task is resubmitted then there is a risk that old "delete task"
can delete the re-submission.
Constructor Summary |
SplitLogManager(ZooKeeperWatcher zkw,
org.apache.hadoop.conf.Configuration conf,
Stoppable stopper,
String serverName)
Wrapper around SplitLogManager(ZooKeeperWatcher, Configuration,
Stoppable, String, TaskFinisher) that provides a task finisher for
copying recovered edits to their final destination. |
SplitLogManager(ZooKeeperWatcher zkw,
org.apache.hadoop.conf.Configuration conf,
Stoppable stopper,
String serverName,
SplitLogManager.TaskFinisher tf)
Its OK to construct this object even when region-servers are not online. |
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
ignoreZKDeleteForTesting
public boolean ignoreZKDeleteForTesting
SplitLogManager
public SplitLogManager(ZooKeeperWatcher zkw,
org.apache.hadoop.conf.Configuration conf,
Stoppable stopper,
String serverName)
- Wrapper around
SplitLogManager(ZooKeeperWatcher, Configuration,
Stoppable, String, TaskFinisher)
that provides a task finisher for
copying recovered edits to their final destination. The task finisher
has to be robust because it can be arbitrarily restarted or called
multiple times.
- Parameters:
zkw
- conf
- stopper
- serverName
-
SplitLogManager
public SplitLogManager(ZooKeeperWatcher zkw,
org.apache.hadoop.conf.Configuration conf,
Stoppable stopper,
String serverName,
SplitLogManager.TaskFinisher tf)
- Its OK to construct this object even when region-servers are not online. It
does lookup the orphan tasks in zk but it doesn't block waiting for them
to be done.
- Parameters:
zkw
- conf
- stopper
- serverName
- tf
- task finisher
finishInitialization
public void finishInitialization(boolean masterRecovery)
splitLogDistributed
public long splitLogDistributed(org.apache.hadoop.fs.Path logDir)
throws IOException
- Parameters:
logDir
- one region sever hlog dir path in .logs
- Returns:
- cumulative size of the logfiles split
- Throws:
IOException
- if there was an error while splitting any log file
IOException
splitLogDistributed
public long splitLogDistributed(List<org.apache.hadoop.fs.Path> logDirs)
throws IOException
- The caller will block until all the log files of the given region server
have been processed - successfully split or an error is encountered - by an
available worker region server. This method must only be called after the
region servers have been brought online.
- Parameters:
logDirs
-
- Returns:
- cumulative size of the logfiles split
- Throws:
IOException
- if there was an error while splitting any log file
nodeDataChanged
public void nodeDataChanged(String path)
- Description copied from class:
ZooKeeperListener
- Called when an existing node has changed data.
- Overrides:
nodeDataChanged
in class ZooKeeperListener
- Parameters:
path
- full path of the updated node
stop
public void stop()
finishInitialization
public void finishInitialization()
- Completes the initialization
Copyright © 2012 The Apache Software Foundation. All Rights Reserved.