|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectjava.lang.Thread
org.apache.hadoop.hbase.replication.regionserver.ReplicationSource
@InterfaceAudience.Private public class ReplicationSource
Class that handles the source of a replication stream. Currently does not handle more than 1 slave For each slave cluster it selects a random number of peers using a replication ratio. For example, if replication ration = 0.1 and slave cluster has 100 region servers, 10 will be selected.
A stream is considered down when we cannot contact a region server on the peer cluster for more than 55 seconds by default.
Nested Class Summary | |
---|---|
static class |
ReplicationSource.LogsComparator
Comparator used to compare logs together based on their start time |
Nested classes/interfaces inherited from class java.lang.Thread |
---|
Thread.State, Thread.UncaughtExceptionHandler |
Field Summary | |
---|---|
static org.apache.commons.logging.Log |
LOG
|
Fields inherited from class java.lang.Thread |
---|
MAX_PRIORITY, MIN_PRIORITY, NORM_PRIORITY |
Constructor Summary | |
---|---|
ReplicationSource()
|
Method Summary | |
---|---|
void |
enqueueLog(org.apache.hadoop.fs.Path log)
Add a log to the list of logs to replicate |
org.apache.hadoop.fs.Path |
getCurrentPath()
Get the current log that's replicated |
protected boolean |
getNextPath()
Poll for the next path |
String |
getPeerClusterId()
Get the id that the source is replicating to. |
String |
getPeerClusterZnode()
Get the id that the source is replicating to |
String |
getStats()
Get a string representation of the current statistics for this source |
void |
init(org.apache.hadoop.conf.Configuration conf,
org.apache.hadoop.fs.FileSystem fs,
ReplicationSourceManager manager,
ReplicationQueues replicationQueues,
ReplicationPeers replicationPeers,
Stoppable stopper,
String peerClusterZnode,
UUID clusterId)
Instantiation method used by region servers |
protected boolean |
isPeerEnabled()
check whether the peer is enabled or not |
protected boolean |
openReader(int sleepMultiplier)
Open a reader on the current path |
protected boolean |
processEndOfFile()
If the queue isn't empty, switch to the next one Else if this is a recovered queue, it means we're done! Else we'll just continue to try reading the log file |
protected boolean |
readAllEntriesToReplicateOrNextFile(boolean currentWALisBeingWrittenTo)
Read all the entries from the current log files and retain those that need to be replicated. |
protected void |
removeNonReplicableEdits(HLog.Entry entry)
We only want KVs that are scoped other than local |
void |
run()
|
protected void |
shipEdits(boolean currentWALisBeingWrittenTo)
Do the shipping logic |
protected boolean |
sleepForRetries(String msg,
int sleepMultiplier)
Do the sleeping logic |
void |
startup()
Start the replication |
void |
terminate(String reason)
End the replication |
void |
terminate(String reason,
Exception cause)
End the replication |
Methods inherited from class java.lang.Thread |
---|
activeCount, checkAccess, clone, countStackFrames, currentThread, destroy, dumpStack, enumerate, getAllStackTraces, getContextClassLoader, getDefaultUncaughtExceptionHandler, getId, getName, getPriority, getStackTrace, getState, getThreadGroup, getUncaughtExceptionHandler, holdsLock, interrupt, interrupted, isAlive, isDaemon, isInterrupted, join, join, join, resume, setContextClassLoader, setDaemon, setDefaultUncaughtExceptionHandler, setName, setPriority, setUncaughtExceptionHandler, sleep, sleep, start, stop, stop, suspend, toString, yield |
Methods inherited from class java.lang.Object |
---|
equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait |
Field Detail |
---|
public static final org.apache.commons.logging.Log LOG
Constructor Detail |
---|
public ReplicationSource()
Method Detail |
---|
public void init(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, ReplicationSourceManager manager, ReplicationQueues replicationQueues, ReplicationPeers replicationPeers, Stoppable stopper, String peerClusterZnode, UUID clusterId) throws IOException
init
in interface ReplicationSourceInterface
conf
- configuration to usefs
- file system to usemanager
- replication manager to ping tostopper
- the atomic boolean to use to stop the regionserverpeerClusterZnode
- the name of our znode
IOException
public void enqueueLog(org.apache.hadoop.fs.Path log)
ReplicationSourceInterface
enqueueLog
in interface ReplicationSourceInterface
log
- path to the log to replicatepublic void run()
run
in interface Runnable
run
in class Thread
protected boolean readAllEntriesToReplicateOrNextFile(boolean currentWALisBeingWrittenTo) throws IOException
currentWALisBeingWrittenTo
- is the current WAL being written to
IOException
protected boolean getNextPath()
protected boolean openReader(int sleepMultiplier)
sleepMultiplier
- by how many times the default sleeping time is augmented
protected boolean sleepForRetries(String msg, int sleepMultiplier)
msg
- Why we sleepsleepMultiplier
- by how many times the default sleeping time is augmented
sleepMultiplier
is < maxRetriesMultiplier
protected void removeNonReplicableEdits(HLog.Entry entry)
entry
- The entry to check for replicationprotected void shipEdits(boolean currentWALisBeingWrittenTo)
currentWALisBeingWrittenTo
- was the current WAL being (seemingly)
written to when this method was calledprotected boolean isPeerEnabled()
protected boolean processEndOfFile()
public void startup()
ReplicationSourceInterface
startup
in interface ReplicationSourceInterface
public void terminate(String reason)
ReplicationSourceInterface
terminate
in interface ReplicationSourceInterface
reason
- why it's terminatingpublic void terminate(String reason, Exception cause)
ReplicationSourceInterface
terminate
in interface ReplicationSourceInterface
reason
- why it's terminatingcause
- the error that's causing itpublic String getPeerClusterZnode()
ReplicationSourceInterface
getPeerClusterZnode
in interface ReplicationSourceInterface
public String getPeerClusterId()
ReplicationSourceInterface
getPeerClusterId
in interface ReplicationSourceInterface
public org.apache.hadoop.fs.Path getCurrentPath()
ReplicationSourceInterface
getCurrentPath
in interface ReplicationSourceInterface
public String getStats()
ReplicationSourceInterface
getStats
in interface ReplicationSourceInterface
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |