org.apache.hadoop.hbase.regionserver.compactions
Class DateTieredCompactionPolicy
java.lang.Object
org.apache.hadoop.hbase.regionserver.compactions.CompactionPolicy
org.apache.hadoop.hbase.regionserver.compactions.SortedCompactionPolicy
org.apache.hadoop.hbase.regionserver.compactions.DateTieredCompactionPolicy
@InterfaceAudience.LimitedPrivate(value="Configuration")
public class DateTieredCompactionPolicy
- extends SortedCompactionPolicy
HBASE-15181 This is a simple implementation of date-based tiered compaction similar to
Cassandra's for the following benefits:
<<<<<<< HEAD
1. Improve date-range-based scan by structuring store files in date-based tiered layout.
2. Reduce compaction overhead.
3. Improve TTL efficiency.
Perfect fit for the use cases that:
1. has mostly date-based data write and scan and a focus on the most recent data.
Out-of-order writes are handled gracefully. Time range overlapping among store files is
tolerated and the performance impact is minimized. Configuration can be set at hbase-site
or overridden at per-table or per-column-family level by hbase shell. Design spec is at
=======
- Improve date-range-based scan by structuring store files in date-based tiered layout.
- Reduce compaction overhead.
- Improve TTL efficiency.
Perfect fit for the use cases that:
- has mostly date-based data write and scan and a focus on the most recent data.
Out-of-order writes are handled gracefully. Time range overlapping among store files is tolerated
and the performance impact is minimized. Configuration can be set at hbase-site or overridden at
per-table or per-column-family level by hbase shell. Design spec is at
>>>>>>> 3f749fd... HBASE-15368 Add pluggable window support
https://docs.google.com/document/d/1_AmlNb2N8Us1xICsTeGDLKIqL6T-oHoRLZ323MG_uy8/
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
DateTieredCompactionPolicy
public DateTieredCompactionPolicy(org.apache.hadoop.conf.Configuration conf,
StoreConfigInformation storeConfigInfo)
throws IOException
- Throws:
IOException
needsCompaction
public boolean needsCompaction(Collection<StoreFile> storeFiles,
List<StoreFile> filesCompacting)
- Heuristics for guessing whether we need minor compaction.
- Overrides:
needsCompaction
in class SortedCompactionPolicy
- Parameters:
storeFiles
- files in the store.filesCompacting
- files being scheduled to compact.
- Returns:
- true to schedule a request.
shouldPerformMajorCompaction
public boolean shouldPerformMajorCompaction(Collection<StoreFile> filesToCompact)
throws IOException
- Specified by:
shouldPerformMajorCompaction
in class SortedCompactionPolicy
- Parameters:
filesToCompact
- Files to compact. Can be null.
- Returns:
- True if we should run a major compaction.
- Throws:
IOException
getCompactionRequest
protected CompactionRequest getCompactionRequest(ArrayList<StoreFile> candidateSelection,
boolean tryingMajor,
boolean isUserCompaction,
boolean mayUseOffPeak,
boolean mayBeStuck)
throws IOException
- Specified by:
getCompactionRequest
in class SortedCompactionPolicy
- Throws:
IOException
selectMajorCompaction
public CompactionRequest selectMajorCompaction(ArrayList<StoreFile> candidateSelection)
selectMinorCompaction
public CompactionRequest selectMinorCompaction(ArrayList<StoreFile> candidateSelection,
boolean mayUseOffPeak,
boolean mayBeStuck)
throws IOException
- We receive store files sorted in ascending order by seqId then scan the list of files. If the
current file has a maxTimestamp older than last known maximum, treat this file as it carries
the last known maximum. This way both seqId and timestamp are in the same order. If files carry
the same maxTimestamps, they are ordered by seqId. We then reverse the list so they are ordered
by seqId and maxTimestamp in descending order and build the time windows. All the out-of-order
data into the same compaction windows, guaranteeing contiguous compaction based on sequence id.
- Throws:
IOException
Copyright © 2007–2016 The Apache Software Foundation. All rights reserved.