public class SplitLogManager extends ZooKeeperListener
SplitLogWorker
and the task will be
upforgrabs again. When the task is done then the task's znode is deleted by
SplitLogManager.
Clients call splitLogDistributed(Path)
to split a region server's
log files. The caller thread waits in this method until all the log files
have been split.
All the zookeeper calls made by this class are asynchronous. This is mainly
to help reduce response time seen by the callers.
There is race in this design between the SplitLogManager and the
SplitLogWorker. SplitLogManager might re-queue a task that has in reality
already been completed by a SplitLogWorker. We rely on the idempotency of
the log splitting task for correctness.
It is also assumed that every log splitting task is unique and once
completed (either with success or with error) it will be not be submitted
again. If a task is resubmitted then there is a risk that old "delete task"
can delete the re-submission.Modifier and Type | Class and Description |
---|---|
static interface |
SplitLogManager.TaskFinisher
SplitLogManager can use objects implementing this interface to
finish off a partially done task by SplitLogWorker . |
Modifier and Type | Field and Description |
---|---|
boolean |
ignoreZKDeleteForTesting |
watcher
Constructor and Description |
---|
SplitLogManager(ZooKeeperWatcher zkw,
org.apache.hadoop.conf.Configuration conf,
Stoppable stopper,
MasterServices master,
String serverName)
Wrapper around
#SplitLogManager(ZooKeeperWatcher, Configuration,
Stoppable, String, TaskFinisher) that provides a task finisher for
copying recovered edits to their final destination. |
SplitLogManager(ZooKeeperWatcher zkw,
org.apache.hadoop.conf.Configuration conf,
Stoppable stopper,
MasterServices master,
String serverName,
SplitLogManager.TaskFinisher tf)
Its OK to construct this object even when region-servers are not online.
|
Modifier and Type | Method and Description |
---|---|
void |
finishInitialization()
Completes the initialization
|
void |
finishInitialization(boolean masterRecovery) |
void |
nodeDataChanged(String path)
Called when an existing node has changed data.
|
long |
splitLogDistributed(List<org.apache.hadoop.fs.Path> logDirs)
The caller will block until all the log files of the given region server
have been processed - successfully split or an error is encountered - by an
available worker region server.
|
long |
splitLogDistributed(List<org.apache.hadoop.fs.Path> logDirs,
org.apache.hadoop.fs.PathFilter filter)
The caller will block until all the META log files of the given region server
have been processed - successfully split or an error is encountered - by an
available worker region server.
|
long |
splitLogDistributed(org.apache.hadoop.fs.Path logDir) |
void |
stop() |
nodeChildrenChanged, nodeCreated, nodeDeleted
public SplitLogManager(ZooKeeperWatcher zkw, org.apache.hadoop.conf.Configuration conf, Stoppable stopper, MasterServices master, String serverName)
#SplitLogManager(ZooKeeperWatcher, Configuration,
Stoppable, String, TaskFinisher)
that provides a task finisher for
copying recovered edits to their final destination. The task finisher
has to be robust because it can be arbitrarily restarted or called
multiple times.zkw
- conf
- stopper
- serverName
- public SplitLogManager(ZooKeeperWatcher zkw, org.apache.hadoop.conf.Configuration conf, Stoppable stopper, MasterServices master, String serverName, SplitLogManager.TaskFinisher tf)
zkw
- conf
- stopper
- serverName
- tf
- task finisherpublic void finishInitialization(boolean masterRecovery)
public long splitLogDistributed(org.apache.hadoop.fs.Path logDir) throws IOException
logDir
- one region sever hlog dir path in .logsIOException
- if there was an error while splitting any log fileIOException
public long splitLogDistributed(List<org.apache.hadoop.fs.Path> logDirs) throws IOException
logDirs
- IOException
- if there was an error while splitting any log filepublic long splitLogDistributed(List<org.apache.hadoop.fs.Path> logDirs, org.apache.hadoop.fs.PathFilter filter) throws IOException
logDirs
- List of log dirs to splitfilter
- the Path filter to select specific files for consideringIOException
- If there was an error while splitting any log filepublic void nodeDataChanged(String path)
ZooKeeperListener
nodeDataChanged
in class ZooKeeperListener
path
- full path of the updated nodepublic void stop()
public void finishInitialization()
Copyright © 2014 The Apache Software Foundation. All Rights Reserved.