org.apache.hadoop.hbase.regionserver.compactions
Class Compactor<T extends Compactor.CellSink>

java.lang.Object
  extended by org.apache.hadoop.hbase.regionserver.compactions.Compactor<T>
Direct Known Subclasses:
AbstractMultiOutputCompactor, DefaultCompactor

@InterfaceAudience.Private
public abstract class Compactor<T extends Compactor.CellSink>
extends Object

A compactor is a compaction algorithm associated a given policy. Base class also contains reusable parts for implementing compactors (what is common and what isn't is evolving).


Nested Class Summary
static interface Compactor.CellSink
          TODO: Replace this with CellOutputStream when StoreFile.Writer uses cells.
protected static interface Compactor.CellSinkFactory<S>
           
protected static class Compactor.FileDetails
          The sole reason this class exists is that java has no ref/out/pointer parameters.
protected static interface Compactor.InternalScannerFactory
           
 
Field Summary
protected  Compression.Algorithm compactionCompression
           
protected  org.apache.hadoop.conf.Configuration conf
           
protected  Compactor.InternalScannerFactory defaultScannerFactory
           
protected  CompactionProgress progress
           
protected  Store store
           
 
Method Summary
protected abstract  void abortWriter(T writer)
           
protected abstract  List<org.apache.hadoop.fs.Path> commitWriter(T writer, Compactor.FileDetails fd, CompactionRequest request)
           
protected  List<org.apache.hadoop.fs.Path> compact(CompactionRequest request, Compactor.InternalScannerFactory scannerFactory, Compactor.CellSinkFactory<T> sinkFactory, CompactionThroughputController throughputController, User user)
           
protected  List<StoreFileScanner> createFileScanners(Collection<StoreFile> filesToCompact, long smallestReadPoint, boolean useDropBehind)
          Creates file scanners for compaction.
protected  InternalScanner createScanner(Store store, List<StoreFileScanner> scanners, long smallestReadPoint, long earliestPutTs, byte[] dropDeletesFromRow, byte[] dropDeletesToRow)
           
protected  InternalScanner createScanner(Store store, List<StoreFileScanner> scanners, ScanType scanType, long smallestReadPoint, long earliestPutTs)
           
protected  StoreFile.Writer createTmpWriter(Compactor.FileDetails fd, boolean shouldDropBehind)
          Creates a writer for a new file in a temporary directory.
protected  Compactor.FileDetails getFileDetails(Collection<StoreFile> filesToCompact, boolean calculatePutTs)
          Extracts some details about the files to compact that are commonly needed by compactors.
 CompactionProgress getProgress()
           
protected  long getSmallestReadPoint()
           
protected  boolean performCompaction(InternalScanner scanner, Compactor.CellSink writer, long smallestReadPoint, CompactionThroughputController throughputController)
          Performs the compaction.
protected  InternalScanner postCreateCoprocScanner(CompactionRequest request, ScanType scanType, InternalScanner scanner, User user)
          Calls coprocessor, if any, to create scanners - after normal scanner creation.
protected  InternalScanner preCreateCoprocScanner(CompactionRequest request, ScanType scanType, long earliestPutTs, List<StoreFileScanner> scanners)
          Calls coprocessor, if any, to create compaction scanner - before normal scanner creation.
protected  InternalScanner preCreateCoprocScanner(CompactionRequest request, ScanType scanType, long earliestPutTs, List<StoreFileScanner> scanners, User user)
           
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

progress

protected volatile CompactionProgress progress

conf

protected final org.apache.hadoop.conf.Configuration conf

store

protected final Store store

compactionCompression

protected Compression.Algorithm compactionCompression

defaultScannerFactory

protected final Compactor.InternalScannerFactory defaultScannerFactory
Method Detail

getProgress

public CompactionProgress getProgress()

getFileDetails

protected Compactor.FileDetails getFileDetails(Collection<StoreFile> filesToCompact,
                                               boolean calculatePutTs)
                                        throws IOException
Extracts some details about the files to compact that are commonly needed by compactors.

Parameters:
filesToCompact - Files.
calculatePutTs - Whether earliest put TS is needed.
Returns:
The result.
Throws:
IOException

createFileScanners

protected List<StoreFileScanner> createFileScanners(Collection<StoreFile> filesToCompact,
                                                    long smallestReadPoint,
                                                    boolean useDropBehind)
                                             throws IOException
Creates file scanners for compaction.

Parameters:
filesToCompact - Files.
Returns:
Scanners.
Throws:
IOException

getSmallestReadPoint

protected long getSmallestReadPoint()

createTmpWriter

protected StoreFile.Writer createTmpWriter(Compactor.FileDetails fd,
                                           boolean shouldDropBehind)
                                    throws IOException
Creates a writer for a new file in a temporary directory.

Parameters:
fd - The file details.
Returns:
Writer for a new StoreFile in the tmp dir.
Throws:
IOException - if creation failed

compact

protected List<org.apache.hadoop.fs.Path> compact(CompactionRequest request,
                                                  Compactor.InternalScannerFactory scannerFactory,
                                                  Compactor.CellSinkFactory<T> sinkFactory,
                                                  CompactionThroughputController throughputController,
                                                  User user)
                                           throws IOException
Throws:
IOException

commitWriter

protected abstract List<org.apache.hadoop.fs.Path> commitWriter(T writer,
                                                                Compactor.FileDetails fd,
                                                                CompactionRequest request)
                                                         throws IOException
Throws:
IOException

abortWriter

protected abstract void abortWriter(T writer)
                             throws IOException
Throws:
IOException

preCreateCoprocScanner

protected InternalScanner preCreateCoprocScanner(CompactionRequest request,
                                                 ScanType scanType,
                                                 long earliestPutTs,
                                                 List<StoreFileScanner> scanners)
                                          throws IOException
Calls coprocessor, if any, to create compaction scanner - before normal scanner creation.

Parameters:
request - Compaction request.
scanType - Scan type.
earliestPutTs - Earliest put ts.
scanners - File scanners for compaction files.
Returns:
Scanner override by coprocessor; null if not overriding.
Throws:
IOException

preCreateCoprocScanner

protected InternalScanner preCreateCoprocScanner(CompactionRequest request,
                                                 ScanType scanType,
                                                 long earliestPutTs,
                                                 List<StoreFileScanner> scanners,
                                                 User user)
                                          throws IOException
Throws:
IOException

postCreateCoprocScanner

protected InternalScanner postCreateCoprocScanner(CompactionRequest request,
                                                  ScanType scanType,
                                                  InternalScanner scanner,
                                                  User user)
                                           throws IOException
Calls coprocessor, if any, to create scanners - after normal scanner creation.

Parameters:
request - Compaction request.
scanType - Scan type.
scanner - The default scanner created for compaction.
Returns:
Scanner scanner to use (usually the default); null if compaction should not proceed.
Throws:
IOException

performCompaction

protected boolean performCompaction(InternalScanner scanner,
                                    Compactor.CellSink writer,
                                    long smallestReadPoint,
                                    CompactionThroughputController throughputController)
                             throws IOException
Performs the compaction.

Parameters:
scanner - Where to read from.
writer - Where to write to.
smallestReadPoint - Smallest read point.
Returns:
Whether compaction ended; false if it was interrupted for some reason.
Throws:
IOException

createScanner

protected InternalScanner createScanner(Store store,
                                        List<StoreFileScanner> scanners,
                                        ScanType scanType,
                                        long smallestReadPoint,
                                        long earliestPutTs)
                                 throws IOException
Parameters:
store - store
scanners - Store file scanners.
scanType - Scan type.
smallestReadPoint - Smallest MVCC read point.
earliestPutTs - Earliest put across all files.
Returns:
A compaction scanner.
Throws:
IOException

createScanner

protected InternalScanner createScanner(Store store,
                                        List<StoreFileScanner> scanners,
                                        long smallestReadPoint,
                                        long earliestPutTs,
                                        byte[] dropDeletesFromRow,
                                        byte[] dropDeletesToRow)
                                 throws IOException
Parameters:
store - The store.
scanners - Store file scanners.
smallestReadPoint - Smallest MVCC read point.
earliestPutTs - Earliest put across all files.
dropDeletesFromRow - Drop deletes starting with this row, inclusive. Can be null.
dropDeletesToRow - Drop deletes ending with this row, exclusive. Can be null.
Returns:
A compaction scanner.
Throws:
IOException


Copyright © 2007–2016 The Apache Software Foundation. All rights reserved.