org.apache.hadoop.hbase.regionserver
Class Store

java.lang.Object
  extended by org.apache.hadoop.hbase.regionserver.metrics.SchemaConfigured
      extended by org.apache.hadoop.hbase.regionserver.Store
All Implemented Interfaces:
HeapSize, SchemaMetrics.SchemaAware

public class Store
extends SchemaConfigured
implements HeapSize

A Store holds a column family in a Region. Its a memstore and a set of zero or more StoreFiles, which stretch backwards over time.

There's no reason to consider append-logging at this level; all logging and locking is handled at the HRegion level. Store just provides services to manage sets of StoreFiles. One of the most important of those services is compaction services where files are aggregated once they pass a configurable threshold.

The only thing having to do with logs that Store needs to deal with is the reconstructionLog. This is a segment of an HRegion's log that might NOT be present upon startup. If the param is NULL, there's nothing to do. If the param is non-NULL, we need to process the log to reconstruct a TreeMap that might not have been written to disk before the process died.

It's assumed that after this constructor returns, the reconstructionLog file will be deleted (by whoever has instantiated the Store).

Locking and transactions are handled at a higher level. This API should not be called directly but by an HRegion manager.


Nested Class Summary
static class Store.ScanInfo
          Immutable information for scans over a store.
 
Field Summary
static long DEEP_OVERHEAD
           
static long FIXED_OVERHEAD
           
protected  MemStore memstore
           
static int NO_PRIORITY
           
static int PRIORITY_USER
           
 
Fields inherited from class org.apache.hadoop.hbase.regionserver.metrics.SchemaConfigured
SCHEMA_CONFIGURED_UNALIGNED_HEAP_SIZE
 
Constructor Summary
protected Store(org.apache.hadoop.fs.Path basedir, HRegion region, HColumnDescriptor family, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.conf.Configuration conf)
          Constructor
 
Method Summary
protected  long add(KeyValue kv)
          Adds a value to the memstore
 boolean canSplit()
           
 void compactRecentForTesting(int N)
          Compact the most recent N files.
 StoreFile.Writer createWriterInTmp(int maxKeyCount, Compression.Algorithm compression, boolean isCompaction)
           
protected  long delete(KeyValue kv)
          Adds a value to the memstore
 void finishRequest(CompactionRequest cr)
           
static int getBytesPerChecksum(org.apache.hadoop.conf.Configuration conf)
          Returns the configured bytesPerChecksum value.
 CacheConfig getCacheConfig()
          Used for tests.
static ChecksumType getChecksumType(org.apache.hadoop.conf.Configuration conf)
          Returns the configured checksum algorithm.
 CompactionProgress getCompactionProgress()
          getter for CompactionProgress object
 int getCompactPriority()
           
 int getCompactPriority(int priority)
           
 KeyValue.KVComparator getComparator()
           
 HFileDataBlockEncoder getDataBlockEncoder()
           
 HColumnDescriptor getFamily()
           
 org.apache.hadoop.fs.Path getHomedir()
          Return the directory in which this store stores its StoreFiles
 HRegion getHRegion()
           
 long getLastCompactSize()
           
static long getLowestTimestamp(List<StoreFile> candidates)
           
 long getMaxMemstoreTS()
           
 int getNumberOfStoreFiles()
           
 Store.ScanInfo getScanInfo()
           
 KeyValueScanner getScanner(Scan scan, NavigableSet<byte[]> targetCols)
          Return a scanner for both the memstore and the HStore files.
protected  List<KeyValueScanner> getScanners(boolean cacheBlocks, boolean isGet, boolean isCompaction, ScanQueryMatcher matcher)
          Get all scanners with no filtering based on TTL (that happens further down the line).
 long getSize()
           
 byte[] getSplitPoint()
          Determines if Store should be split
 org.apache.hadoop.hbase.regionserver.StoreFlusher getStoreFlusher(long cacheFlushId)
           
static org.apache.hadoop.fs.Path getStoreHomedir(org.apache.hadoop.fs.Path tabledir, String encodedName, byte[] family)
           
 long heapSize()
           
 boolean needsCompaction()
          See if there's too much store files in this store
 CompactionRequest requestCompaction()
           
 CompactionRequest requestCompaction(int priority)
           
protected  void rollback(KeyValue kv)
          Removes a kv from the memstore.
 com.google.common.collect.ImmutableList<StoreFile> sortAndClone(List<StoreFile> storeFiles)
           
 String toString()
           
 void triggerMajorCompaction()
           
 long updateColumnValue(byte[] row, byte[] f, byte[] qualifier, long newValue)
          Increments the value for the given row/family/qualifier.
 long upsert(List<KeyValue> kvs)
          Adds or replaces the specified KeyValues.
 
Methods inherited from class org.apache.hadoop.hbase.regionserver.metrics.SchemaConfigured
createUnknown, getColumnFamilyName, getSchemaMetrics, getTableName, isSchemaConfigured, passSchemaMetricsTo, resetSchemaMetricsConf, schemaConfAsJSON, schemaConfigurationChanged
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Field Detail

memstore

protected final MemStore memstore

PRIORITY_USER

public static final int PRIORITY_USER
See Also:
Constant Field Values

NO_PRIORITY

public static final int NO_PRIORITY
See Also:
Constant Field Values

FIXED_OVERHEAD

public static final long FIXED_OVERHEAD

DEEP_OVERHEAD

public static final long DEEP_OVERHEAD
Constructor Detail

Store

protected Store(org.apache.hadoop.fs.Path basedir,
                HRegion region,
                HColumnDescriptor family,
                org.apache.hadoop.fs.FileSystem fs,
                org.apache.hadoop.conf.Configuration conf)
         throws IOException
Constructor

Parameters:
basedir - qualified path under which the region directory lives; generally the table subdirectory
region -
family - HColumnDescriptor for this column
fs - file system object
confParam - configuration object failed. Can be null.
Throws:
IOException
Method Detail

getBytesPerChecksum

public static int getBytesPerChecksum(org.apache.hadoop.conf.Configuration conf)
Returns the configured bytesPerChecksum value.

Parameters:
conf - The configuration
Returns:
The bytesPerChecksum that is set in the configuration

getChecksumType

public static ChecksumType getChecksumType(org.apache.hadoop.conf.Configuration conf)
Returns the configured checksum algorithm.

Parameters:
conf - The configuration
Returns:
The checksum algorithm that is set in the configuration

getFamily

public HColumnDescriptor getFamily()

getMaxMemstoreTS

public long getMaxMemstoreTS()
Returns:
The maximum memstoreTS in all store files.

getStoreHomedir

public static org.apache.hadoop.fs.Path getStoreHomedir(org.apache.hadoop.fs.Path tabledir,
                                                        String encodedName,
                                                        byte[] family)
Parameters:
tabledir -
encodedName - Encoded region name.
family -
Returns:
Path to family/Store home directory.

getHomedir

public org.apache.hadoop.fs.Path getHomedir()
Return the directory in which this store stores its StoreFiles


getDataBlockEncoder

public HFileDataBlockEncoder getDataBlockEncoder()
Returns:
the data block encoder

add

protected long add(KeyValue kv)
Adds a value to the memstore

Parameters:
kv -
Returns:
memstore size delta

delete

protected long delete(KeyValue kv)
Adds a value to the memstore

Parameters:
kv -
Returns:
memstore size delta

rollback

protected void rollback(KeyValue kv)
Removes a kv from the memstore. The KeyValue is removed only if its key & memstoreTS matches the key & memstoreTS value of the kv parameter.

Parameters:
kv -

createWriterInTmp

public StoreFile.Writer createWriterInTmp(int maxKeyCount,
                                          Compression.Algorithm compression,
                                          boolean isCompaction)
                                   throws IOException
Throws:
IOException

getScanners

protected List<KeyValueScanner> getScanners(boolean cacheBlocks,
                                            boolean isGet,
                                            boolean isCompaction,
                                            ScanQueryMatcher matcher)
                                     throws IOException
Get all scanners with no filtering based on TTL (that happens further down the line).

Returns:
all scanners for this store
Throws:
IOException

compactRecentForTesting

public void compactRecentForTesting(int N)
                             throws IOException
Compact the most recent N files. Used in testing.

Throws:
IOException

getLowestTimestamp

public static long getLowestTimestamp(List<StoreFile> candidates)
                               throws IOException
Throws:
IOException

getCompactionProgress

public CompactionProgress getCompactionProgress()
getter for CompactionProgress object

Returns:
CompactionProgress object

requestCompaction

public CompactionRequest requestCompaction()
                                    throws IOException
Throws:
IOException

requestCompaction

public CompactionRequest requestCompaction(int priority)
                                    throws IOException
Throws:
IOException

finishRequest

public void finishRequest(CompactionRequest cr)

sortAndClone

public com.google.common.collect.ImmutableList<StoreFile> sortAndClone(List<StoreFile> storeFiles)

getNumberOfStoreFiles

public int getNumberOfStoreFiles()
Returns:
the number of files in this store

canSplit

public boolean canSplit()

getSplitPoint

public byte[] getSplitPoint()
Determines if Store should be split

Returns:
byte[] if store should be split, null otherwise.

getLastCompactSize

public long getLastCompactSize()
Returns:
aggregate size of all HStores used in the last compaction

getSize

public long getSize()
Returns:
aggregate size of HStore

triggerMajorCompaction

public void triggerMajorCompaction()

getScanner

public KeyValueScanner getScanner(Scan scan,
                                  NavigableSet<byte[]> targetCols)
                           throws IOException
Return a scanner for both the memstore and the HStore files. Assumes we are not in a compaction.

Throws:
IOException

toString

public String toString()
Overrides:
toString in class Object

getCompactPriority

public int getCompactPriority()

getCompactPriority

public int getCompactPriority(int priority)
Parameters:
priority -
Returns:
The priority that this store should have in the compaction queue

getHRegion

public HRegion getHRegion()

updateColumnValue

public long updateColumnValue(byte[] row,
                              byte[] f,
                              byte[] qualifier,
                              long newValue)
                       throws IOException
Increments the value for the given row/family/qualifier. This function will always be seen as atomic by other readers because it only puts a single KV to memstore. Thus no read/write control necessary.

Parameters:
row -
f -
qualifier -
newValue - the new value to set into memstore
Returns:
memstore size delta
Throws:
IOException

upsert

public long upsert(List<KeyValue> kvs)
            throws IOException
Adds or replaces the specified KeyValues.

For each KeyValue specified, if a cell with the same row, family, and qualifier exists in MemStore, it will be replaced. Otherwise, it will just be inserted to MemStore.

This operation is atomic on each KeyValue (row/family/qualifier) but not necessarily atomic across all of them.

Parameters:
kvs -
Returns:
memstore size delta
Throws:
IOException

getStoreFlusher

public org.apache.hadoop.hbase.regionserver.StoreFlusher getStoreFlusher(long cacheFlushId)

needsCompaction

public boolean needsCompaction()
See if there's too much store files in this store

Returns:
true if number of store files is greater than the number defined in minFilesToCompact

getCacheConfig

public CacheConfig getCacheConfig()
Used for tests. Get the cache configuration for this Store.


heapSize

public long heapSize()
Specified by:
heapSize in interface HeapSize
Overrides:
heapSize in class SchemaConfigured
Returns:
Approximate 'exclusive deep size' of implementing object. Includes count of payload and hosting object sizings.

getComparator

public KeyValue.KVComparator getComparator()

getScanInfo

public Store.ScanInfo getScanInfo()


Copyright © 2012 The Apache Software Foundation. All Rights Reserved.