public class HLogSplitter extends Object
Modifier and Type | Field and Description |
---|---|
protected org.apache.hadoop.conf.Configuration |
conf |
protected org.apache.hadoop.fs.FileSystem |
fs |
protected org.apache.hadoop.fs.Path |
oldLogDir |
static String |
RECOVERED_EDITS
Name of file that holds recovered edits written by the wal log splitting
code, one per region
|
protected org.apache.hadoop.fs.Path |
rootDir |
protected org.apache.hadoop.fs.Path |
srcDir |
protected AtomicReference<Throwable> |
thrown |
Constructor and Description |
---|
HLogSplitter(org.apache.hadoop.conf.Configuration conf,
org.apache.hadoop.fs.Path rootDir,
org.apache.hadoop.fs.Path srcDir,
org.apache.hadoop.fs.Path oldLogDir,
org.apache.hadoop.fs.FileSystem fs) |
Modifier and Type | Method and Description |
---|---|
static HLogSplitter |
createLogSplitter(org.apache.hadoop.conf.Configuration conf,
org.apache.hadoop.fs.Path rootDir,
org.apache.hadoop.fs.Path srcDir,
org.apache.hadoop.fs.Path oldLogDir,
org.apache.hadoop.fs.FileSystem fs)
Create a new HLogSplitter using the given
Configuration and the
hbase.hlog.splitter.impl property to derived the instance
class to use. |
protected HLog.Writer |
createWriter(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path logfile,
org.apache.hadoop.conf.Configuration conf)
Create a new
HLog.Writer for writing log splits. |
static void |
finishSplitLogFile(org.apache.hadoop.fs.Path rootdir,
org.apache.hadoop.fs.Path oldLogDir,
String logfile,
org.apache.hadoop.conf.Configuration conf) |
static void |
finishSplitLogFile(String logfile,
org.apache.hadoop.conf.Configuration conf)
Completes the work done by splitLogFile by archiving logs
|
protected HLog.Reader |
getReader(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.FileStatus file,
org.apache.hadoop.conf.Configuration conf,
boolean skipErrors)
Create a new
HLog.Reader for reading logs to split. |
protected HLog.Reader |
getReader(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path curLogFile,
org.apache.hadoop.conf.Configuration conf)
Create a new
HLog.Reader for reading logs to split. |
long |
getSize() |
long |
getTime() |
List<org.apache.hadoop.fs.Path> |
splitLog()
Split up a bunch of regionserver commit log files that are no longer being
written to, into new files, one per region for region to replay on startup.
|
List<org.apache.hadoop.fs.Path> |
splitLog(CountDownLatch latch)
Split up a bunch of regionserver commit log files that are no longer being
written to, into new files, one per region for region to replay on startup.
|
boolean |
splitLogFile(org.apache.hadoop.fs.FileStatus logfile,
CancelableProgressable reporter) |
static boolean |
splitLogFile(org.apache.hadoop.fs.Path rootDir,
org.apache.hadoop.fs.FileStatus logfile,
org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.conf.Configuration conf,
CancelableProgressable reporter)
Splits a HLog file into region's recovered-edits directory
|
public static final String RECOVERED_EDITS
protected final org.apache.hadoop.fs.Path rootDir
protected final org.apache.hadoop.fs.Path srcDir
protected final org.apache.hadoop.fs.Path oldLogDir
protected final org.apache.hadoop.fs.FileSystem fs
protected final org.apache.hadoop.conf.Configuration conf
protected AtomicReference<Throwable> thrown
public HLogSplitter(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.Path rootDir, org.apache.hadoop.fs.Path srcDir, org.apache.hadoop.fs.Path oldLogDir, org.apache.hadoop.fs.FileSystem fs)
public static HLogSplitter createLogSplitter(org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.Path rootDir, org.apache.hadoop.fs.Path srcDir, org.apache.hadoop.fs.Path oldLogDir, org.apache.hadoop.fs.FileSystem fs)
Configuration
and the
hbase.hlog.splitter.impl
property to derived the instance
class to use.
conf
- rootDir
- hbase directorysrcDir
- logs directoryoldLogDir
- directory where processed logs are archived tofs
- FileSystempublic List<org.apache.hadoop.fs.Path> splitLog() throws IOException
IOException
- will throw if corrupted hlogs aren't toleratedpublic List<org.apache.hadoop.fs.Path> splitLog(CountDownLatch latch) throws IOException
latch
- IOException
- will throw if corrupted hlogs aren't toleratedpublic long getTime()
public long getSize()
public static boolean splitLogFile(org.apache.hadoop.fs.Path rootDir, org.apache.hadoop.fs.FileStatus logfile, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.conf.Configuration conf, CancelableProgressable reporter) throws IOException
If the log file has N regions then N recovered.edits files will be produced. There is no buffering in this code. Instead it relies on the buffering in the SequenceFileWriter.
rootDir
- logfile
- fs
- conf
- reporter
- IOException
public boolean splitLogFile(org.apache.hadoop.fs.FileStatus logfile, CancelableProgressable reporter) throws IOException
IOException
public static void finishSplitLogFile(String logfile, org.apache.hadoop.conf.Configuration conf) throws IOException
It is invoked by SplitLogManager once it knows that one of the SplitLogWorkers have completed the splitLogFile() part. If the master crashes then this function might get called multiple times.
logfile
- conf
- IOException
public static void finishSplitLogFile(org.apache.hadoop.fs.Path rootdir, org.apache.hadoop.fs.Path oldLogDir, String logfile, org.apache.hadoop.conf.Configuration conf) throws IOException
IOException
protected HLog.Reader getReader(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.FileStatus file, org.apache.hadoop.conf.Configuration conf, boolean skipErrors) throws IOException, org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.CorruptedLogFileException
HLog.Reader
for reading logs to split.fs
- file
- conf
- IOException
CorruptedLogFile
org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.CorruptedLogFileException
protected HLog.Writer createWriter(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path logfile, org.apache.hadoop.conf.Configuration conf) throws IOException
HLog.Writer
for writing log splits.IOException
protected HLog.Reader getReader(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path curLogFile, org.apache.hadoop.conf.Configuration conf) throws IOException
HLog.Reader
for reading logs to split.IOException
Copyright © 2014 The Apache Software Foundation. All Rights Reserved.