org.apache.hadoop.hbase.snapshot
Class SnapshotDescriptionUtils

java.lang.Object
  extended by org.apache.hadoop.hbase.snapshot.SnapshotDescriptionUtils

public class SnapshotDescriptionUtils
extends Object

Utility class to help manage SnapshotDesriptions.

Snapshots are laid out on disk like this:

 /hbase/.snapshots
          /.tmp                <---- working directory
          /[snapshot name]     <----- completed snapshot
 
A completed snapshot named 'completed' then looks like (multiple regions, servers, files, etc. signified by '...' on the same directory depth).
 /hbase/.snapshots/completed
                   .snapshotinfo          <--- Description of the snapshot
                   .tableinfo             <--- Copy of the tableinfo
                    /.logs
                        /[server_name]
                            /... [log files]
                         ...
                   /[region name]           <---- All the region's information
                   .regioninfo              <---- Copy of the HRegionInfo
                      /[column family name]
                          /[hfile name]     <--- name of the hfile in the real region
                          ...
                      ...
                    ...
 
Utility methods in this class are useful for getting the correct locations for different parts of the snapshot, as well as moving completed snapshots into place (see completeSnapshot(org.apache.hadoop.hbase.protobuf.generated.HBaseProtos.SnapshotDescription, org.apache.hadoop.fs.Path, org.apache.hadoop.fs.Path, org.apache.hadoop.fs.FileSystem), and writing the HBaseProtos.SnapshotDescription to the working snapshot directory.


Nested Class Summary
static class SnapshotDescriptionUtils.CompletedSnaphotDirectoriesFilter
          Filter that only accepts completed snapshot directories
 
Field Summary
static long DEFAULT_MAX_WAIT_TIME
          By default, wait 60 seconds for a snapshot to complete
static String MASTER_SNAPSHOT_TIMEOUT_MILLIS
           
static long NO_SNAPSHOT_START_TIME_SPECIFIED
          Default value if no start time is specified
static String SNAPSHOT_LAYOUT_CONF_KEY
          Version of the fs layout for new snapshot.
static int SNAPSHOT_LAYOUT_LATEST_FORMAT
           
static String SNAPSHOT_TMP_DIR_NAME
          Temporary directory under the snapshot directory to store in-progress snapshots
static String SNAPSHOTINFO_FILE
          The file contains the snapshot basic information and it is under the directory of a snapshot.
 
Method Summary
static void completeSnapshot(HBaseProtos.SnapshotDescription snapshot, org.apache.hadoop.fs.Path rootdir, org.apache.hadoop.fs.Path workingDir, org.apache.hadoop.fs.FileSystem fs)
          Move the finished snapshot to its final, publicly visible directory - this marks the snapshot as 'complete'.
static org.apache.hadoop.fs.Path getCompletedSnapshotDir(HBaseProtos.SnapshotDescription snapshot, org.apache.hadoop.fs.Path rootDir)
          Get the directory for a specified snapshot.
static org.apache.hadoop.fs.Path getCompletedSnapshotDir(String snapshotName, org.apache.hadoop.fs.Path rootDir)
          Get the directory for a completed snapshot.
static int getDefaultSnapshotLayoutFormat(org.apache.hadoop.conf.Configuration conf)
           
static long getMaxMasterTimeout(org.apache.hadoop.conf.Configuration conf, HBaseProtos.SnapshotDescription.Type type, long defaultMaxWaitTime)
           
static org.apache.hadoop.fs.Path getSnapshotRootDir(org.apache.hadoop.fs.Path rootDir)
          Get the snapshot root directory.
static org.apache.hadoop.fs.Path getSnapshotsDir(org.apache.hadoop.fs.Path rootDir)
           
static org.apache.hadoop.fs.Path getWorkingSnapshotDir(HBaseProtos.SnapshotDescription snapshot, org.apache.hadoop.fs.Path rootDir)
          Get the directory to build a snapshot, before it is finalized
static org.apache.hadoop.fs.Path getWorkingSnapshotDir(org.apache.hadoop.fs.Path rootDir)
          Get the general working directory for snapshots - where they are built, where they are temporarily copied on export, etc.
static org.apache.hadoop.fs.Path getWorkingSnapshotDir(String snapshotName, org.apache.hadoop.fs.Path rootDir)
          Get the directory to build a snapshot, before it is finalized
static HBaseProtos.SnapshotDescription readSnapshotInfo(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path snapshotDir)
          Read in the HBaseProtos.SnapshotDescription stored for the snapshot in the passed directory
static HBaseProtos.SnapshotDescription validate(HBaseProtos.SnapshotDescription snapshot, org.apache.hadoop.conf.Configuration conf)
          Convert the passed snapshot description into a 'full' snapshot description based on default parameters, if none have been supplied.
static void writeSnapshotInfo(HBaseProtos.SnapshotDescription snapshot, org.apache.hadoop.fs.Path workingDir, org.apache.hadoop.fs.FileSystem fs)
          Write the snapshot description into the working directory of a snapshot
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

SNAPSHOT_LAYOUT_CONF_KEY

public static final String SNAPSHOT_LAYOUT_CONF_KEY
Version of the fs layout for new snapshot. Each version of hbase should be able to read older formats.

See Also:
Constant Field Values

SNAPSHOT_LAYOUT_LATEST_FORMAT

public static final int SNAPSHOT_LAYOUT_LATEST_FORMAT
See Also:
Constant Field Values

SNAPSHOTINFO_FILE

public static final String SNAPSHOTINFO_FILE
The file contains the snapshot basic information and it is under the directory of a snapshot.

See Also:
Constant Field Values

SNAPSHOT_TMP_DIR_NAME

public static final String SNAPSHOT_TMP_DIR_NAME
Temporary directory under the snapshot directory to store in-progress snapshots

See Also:
Constant Field Values

NO_SNAPSHOT_START_TIME_SPECIFIED

public static final long NO_SNAPSHOT_START_TIME_SPECIFIED
Default value if no start time is specified

See Also:
Constant Field Values

MASTER_SNAPSHOT_TIMEOUT_MILLIS

public static final String MASTER_SNAPSHOT_TIMEOUT_MILLIS
See Also:
Constant Field Values

DEFAULT_MAX_WAIT_TIME

public static final long DEFAULT_MAX_WAIT_TIME
By default, wait 60 seconds for a snapshot to complete

See Also:
Constant Field Values
Method Detail

getDefaultSnapshotLayoutFormat

public static int getDefaultSnapshotLayoutFormat(org.apache.hadoop.conf.Configuration conf)

getMaxMasterTimeout

public static long getMaxMasterTimeout(org.apache.hadoop.conf.Configuration conf,
                                       HBaseProtos.SnapshotDescription.Type type,
                                       long defaultMaxWaitTime)
Parameters:
conf - Configuration from which to check for the timeout
type - type of snapshot being taken
defaultMaxWaitTime - Default amount of time to wait, if none is in the configuration
Returns:
the max amount of time the master should wait for a snapshot to complete

getSnapshotRootDir

public static org.apache.hadoop.fs.Path getSnapshotRootDir(org.apache.hadoop.fs.Path rootDir)
Get the snapshot root directory. All the snapshots are kept under this directory, i.e. ${hbase.rootdir}/.snapshot

Parameters:
rootDir - hbase root directory
Returns:
the base directory in which all snapshots are kept

getCompletedSnapshotDir

public static org.apache.hadoop.fs.Path getCompletedSnapshotDir(HBaseProtos.SnapshotDescription snapshot,
                                                                org.apache.hadoop.fs.Path rootDir)
Get the directory for a specified snapshot. This directory is a sub-directory of snapshot root directory and all the data files for a snapshot are kept under this directory.

Parameters:
snapshot - snapshot being taken
rootDir - hbase root directory
Returns:
the final directory for the completed snapshot

getCompletedSnapshotDir

public static org.apache.hadoop.fs.Path getCompletedSnapshotDir(String snapshotName,
                                                                org.apache.hadoop.fs.Path rootDir)
Get the directory for a completed snapshot. This directory is a sub-directory of snapshot root directory and all the data files for a snapshot are kept under this directory.

Parameters:
snapshotName - name of the snapshot being taken
rootDir - hbase root directory
Returns:
the final directory for the completed snapshot

getWorkingSnapshotDir

public static org.apache.hadoop.fs.Path getWorkingSnapshotDir(org.apache.hadoop.fs.Path rootDir)
Get the general working directory for snapshots - where they are built, where they are temporarily copied on export, etc.

Parameters:
rootDir - root directory of the HBase installation
Returns:
Path to the snapshot tmp directory, relative to the passed root directory

getWorkingSnapshotDir

public static org.apache.hadoop.fs.Path getWorkingSnapshotDir(HBaseProtos.SnapshotDescription snapshot,
                                                              org.apache.hadoop.fs.Path rootDir)
Get the directory to build a snapshot, before it is finalized

Parameters:
snapshot - snapshot that will be built
rootDir - root directory of the hbase installation
Returns:
Path where one can build a snapshot

getWorkingSnapshotDir

public static org.apache.hadoop.fs.Path getWorkingSnapshotDir(String snapshotName,
                                                              org.apache.hadoop.fs.Path rootDir)
Get the directory to build a snapshot, before it is finalized

Parameters:
snapshotName - name of the snapshot
rootDir - root directory of the hbase installation
Returns:
Path where one can build a snapshot

getSnapshotsDir

public static final org.apache.hadoop.fs.Path getSnapshotsDir(org.apache.hadoop.fs.Path rootDir)
Parameters:
rootDir - hbase root directory
Returns:
the directory for all completed snapshots;

validate

public static HBaseProtos.SnapshotDescription validate(HBaseProtos.SnapshotDescription snapshot,
                                                       org.apache.hadoop.conf.Configuration conf)
                                                throws IllegalArgumentException
Convert the passed snapshot description into a 'full' snapshot description based on default parameters, if none have been supplied. This resolves any 'optional' parameters that aren't supplied to their default values.

Parameters:
snapshot - general snapshot descriptor
conf - Configuration to read configured snapshot defaults if snapshot is not complete
Returns:
a valid snapshot description
Throws:
IllegalArgumentException - if the HBaseProtos.SnapshotDescription is not a complete HBaseProtos.SnapshotDescription.

writeSnapshotInfo

public static void writeSnapshotInfo(HBaseProtos.SnapshotDescription snapshot,
                                     org.apache.hadoop.fs.Path workingDir,
                                     org.apache.hadoop.fs.FileSystem fs)
                              throws IOException
Write the snapshot description into the working directory of a snapshot

Parameters:
snapshot - description of the snapshot being taken
workingDir - working directory of the snapshot
fs - FileSystem on which the snapshot should be taken
Throws:
IOException - if we can't reach the filesystem and the file cannot be cleaned up on failure

readSnapshotInfo

public static HBaseProtos.SnapshotDescription readSnapshotInfo(org.apache.hadoop.fs.FileSystem fs,
                                                               org.apache.hadoop.fs.Path snapshotDir)
                                                        throws CorruptedSnapshotException
Read in the HBaseProtos.SnapshotDescription stored for the snapshot in the passed directory

Parameters:
fs - filesystem where the snapshot was taken
snapshotDir - directory where the snapshot was stored
Returns:
the stored snapshot description
Throws:
CorruptedSnapshotException - if the snapshot cannot be read

completeSnapshot

public static void completeSnapshot(HBaseProtos.SnapshotDescription snapshot,
                                    org.apache.hadoop.fs.Path rootdir,
                                    org.apache.hadoop.fs.Path workingDir,
                                    org.apache.hadoop.fs.FileSystem fs)
                             throws SnapshotCreationException,
                                    IOException
Move the finished snapshot to its final, publicly visible directory - this marks the snapshot as 'complete'.

Parameters:
snapshot - description of the snapshot being tabken
rootdir - root directory of the hbase installation
workingDir - directory where the in progress snapshot was built
fs - FileSystem where the snapshot was built
Throws:
SnapshotCreationException - if the snapshot could not be moved
IOException - the filesystem could not be reached


Copyright © 2015 The Apache Software Foundation. All rights reserved.