org.apache.hadoop.hbase.replication.regionserver
Class ReplicationSourceManager

java.lang.Object
  extended by org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceManager

public class ReplicationSourceManager
extends Object

This class is responsible to manage all the replication sources. There are two classes of sources:

  • Normal sources are persistent and one per peer cluster
  • Old sources are recovered from a failed region server and our only goal is to finish replicating the HLog queue it had up in ZK
  • When a region server dies, this class uses a watcher to get notified and it tries to grab a lock in order to transfer all the queues in a local old source.


    Nested Class Summary
     class ReplicationSourceManager.OtherRegionServerWatcher
              Watcher used to be notified of the other region server's death in the local cluster.
     class ReplicationSourceManager.PeersWatcher
              Watcher used to follow the creation and deletion of peer clusters.
     
    Constructor Summary
    ReplicationSourceManager(ReplicationZookeeper zkHelper, org.apache.hadoop.conf.Configuration conf, Stoppable stopper, org.apache.hadoop.fs.FileSystem fs, AtomicBoolean replicating, org.apache.hadoop.fs.Path logDir, org.apache.hadoop.fs.Path oldLogDir)
              Creates a replication manager and sets the watch on all the other registered region servers
     
    Method Summary
     ReplicationSourceInterface addSource(String id)
              Add a new normal source to this region server
     void closeRecoveredQueue(ReplicationSourceInterface src)
              Clear the references to the specified old source
     org.apache.hadoop.fs.FileSystem getFs()
              Get the handle on the local file system
    protected  SortedSet<String> getHLogs()
              Get a copy of the hlogs of the first source on this rs
     org.apache.hadoop.fs.Path getLogDir()
              Get the directory where hlogs are stored by their RSs
     org.apache.hadoop.fs.Path getOldLogDir()
              Get the directory where hlogs are archived
     ReplicationSourceInterface getReplicationSource(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs, ReplicationSourceManager manager, Stoppable stopper, AtomicBoolean replicating, String peerClusterId)
              Factory method to create a replication source
     ReplicationZookeeper getRepZkWrapper()
              Get the ZK help of this manager
     List<ReplicationSourceInterface> getSources()
              Get a list of all the normal sources of this rs
     void init()
              Adds a normal source per registered peer cluster and tries to process all old region server hlog queues
     void join()
              Terminate the replication on this region server
     void logPositionAndCleanOldLogs(org.apache.hadoop.fs.Path log, String id, long position, boolean queueRecovered)
              Provide the id of the peer and a log key and this method will figure which hlog it belongs to and will log, for this region server, the current position.
     void removePeer(String id)
              Thie method first deletes all the recovered sources for the specified id, then deletes the normal source (deleting all related data in ZK).
     void transferQueues(String rsZnode)
              Transfer all the queues of the specified to this region server.
     
    Methods inherited from class java.lang.Object
    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
     

    Constructor Detail

    ReplicationSourceManager

    public ReplicationSourceManager(ReplicationZookeeper zkHelper,
                                    org.apache.hadoop.conf.Configuration conf,
                                    Stoppable stopper,
                                    org.apache.hadoop.fs.FileSystem fs,
                                    AtomicBoolean replicating,
                                    org.apache.hadoop.fs.Path logDir,
                                    org.apache.hadoop.fs.Path oldLogDir)
    Creates a replication manager and sets the watch on all the other registered region servers

    Parameters:
    zkHelper - the zk helper for replication
    conf - the configuration to use
    stopper - the stopper object for this region server
    fs - the file system to use
    replicating - the status of the replication on this cluster
    logDir - the directory that contains all hlog directories of live RSs
    oldLogDir - the directory where old logs are archived
    Method Detail

    logPositionAndCleanOldLogs

    public void logPositionAndCleanOldLogs(org.apache.hadoop.fs.Path log,
                                           String id,
                                           long position,
                                           boolean queueRecovered)
    Provide the id of the peer and a log key and this method will figure which hlog it belongs to and will log, for this region server, the current position. It will also clean old logs from the queue.

    Parameters:
    log - Path to the log currently being replicated from replication status in zookeeper. It will also delete older entries.
    id - id of the peer cluster
    position - current location in the log
    queueRecovered - indicates if this queue comes from another region server

    init

    public void init()
              throws IOException
    Adds a normal source per registered peer cluster and tries to process all old region server hlog queues

    Throws:
    IOException

    addSource

    public ReplicationSourceInterface addSource(String id)
                                         throws IOException
    Add a new normal source to this region server

    Parameters:
    id - the id of the peer cluster
    Returns:
    the source that was created
    Throws:
    IOException

    join

    public void join()
    Terminate the replication on this region server


    getHLogs

    protected SortedSet<String> getHLogs()
    Get a copy of the hlogs of the first source on this rs

    Returns:
    a sorted set of hlog names

    getSources

    public List<ReplicationSourceInterface> getSources()
    Get a list of all the normal sources of this rs

    Returns:
    lis of all sources

    getRepZkWrapper

    public ReplicationZookeeper getRepZkWrapper()
    Get the ZK help of this manager

    Returns:
    the helper

    getReplicationSource

    public ReplicationSourceInterface getReplicationSource(org.apache.hadoop.conf.Configuration conf,
                                                           org.apache.hadoop.fs.FileSystem fs,
                                                           ReplicationSourceManager manager,
                                                           Stoppable stopper,
                                                           AtomicBoolean replicating,
                                                           String peerClusterId)
                                                    throws IOException
    Factory method to create a replication source

    Parameters:
    conf - the configuration to use
    fs - the file system to use
    manager - the manager to use
    stopper - the stopper object for this region server
    replicating - the status of the replication on this cluster
    peerClusterId - the id of the peer cluster
    Returns:
    the created source
    Throws:
    IOException

    transferQueues

    public void transferQueues(String rsZnode)
    Transfer all the queues of the specified to this region server. First it tries to grab a lock and if it works it will move the znodes and finally will delete the old znodes. It creates one old source for any type of source of the old rs.

    Parameters:
    rsZnode -

    closeRecoveredQueue

    public void closeRecoveredQueue(ReplicationSourceInterface src)
    Clear the references to the specified old source

    Parameters:
    src - source to clear

    removePeer

    public void removePeer(String id)
    Thie method first deletes all the recovered sources for the specified id, then deletes the normal source (deleting all related data in ZK).

    Parameters:
    id - The id of the peer cluster

    getOldLogDir

    public org.apache.hadoop.fs.Path getOldLogDir()
    Get the directory where hlogs are archived

    Returns:
    the directory where hlogs are archived

    getLogDir

    public org.apache.hadoop.fs.Path getLogDir()
    Get the directory where hlogs are stored by their RSs

    Returns:
    the directory where hlogs are stored by their RSs

    getFs

    public org.apache.hadoop.fs.FileSystem getFs()
    Get the handle on the local file system

    Returns:
    Handle on the local file system


    Copyright © 2011 The Apache Software Foundation. All Rights Reserved.