org.apache.hadoop.hbase.io.hfile.bucket
Class BucketCache

java.lang.Object
  extended by org.apache.hadoop.hbase.io.hfile.bucket.BucketCache
All Implemented Interfaces:
Iterable<CachedBlock>, HeapSize, BlockCache

@InterfaceAudience.Private
public class BucketCache
extends Object
implements BlockCache, HeapSize

BucketCache uses BucketAllocator to allocate/free block, and use ramCache and backingMap in order to determine if a given element is in the cache. The bucket cache can use on-heap or off-heap memory ByteBufferIOEngine or in a file FileIOEngine to store/read the block data.

Eviction is via a similar algorithm as used in LruBlockCache

BucketCache can be used as mainly a block cache (see CombinedBlockCache), combined with LruBlockCache to decrease CMS GC and heap fragmentation.

It also can be used as a secondary cache (e.g. using a file on ssd/fusionio to store blocks) to enlarge cache space via LruBlockCache.setVictimCache(org.apache.hadoop.hbase.io.hfile.bucket.BucketCache)


Field Summary
static int DEFAULT_ERROR_TOLERATION_DURATION
           
 
Constructor Summary
BucketCache(String ioEngineName, long capacity, int blockSize, int[] bucketSizes, int writerThreadNum, int writerQLen, String persistencePath)
           
BucketCache(String ioEngineName, long capacity, int blockSize, int[] bucketSizes, int writerThreadNum, int writerQLen, String persistencePath, int ioErrorsTolerationDuration)
           
 
Method Summary
 void cacheBlock(BlockCacheKey cacheKey, Cacheable buf)
          Cache the block with the specified name and buffer.
 void cacheBlock(BlockCacheKey cacheKey, Cacheable cachedItem, boolean inMemory)
          Cache the block with the specified name and buffer.
 void cacheBlockWithWait(BlockCacheKey cacheKey, Cacheable cachedItem, boolean inMemory, boolean wait)
          Cache the block to ramCache
 boolean evictBlock(BlockCacheKey cacheKey)
          Evict block from cache.
 int evictBlocksByHfileName(String hfileName)
          Evicts all blocks for a specific HFile.
 BucketAllocator getAllocator()
           
 Cacheable getBlock(BlockCacheKey key, boolean caching, boolean repeat, boolean updateCacheMetrics)
          Get the buffer of the block with the specified key.
 BlockCache[] getBlockCaches()
           
 long getBlockCount()
          Returns the number of blocks currently cached in the block cache.
 long getCurrentSize()
          Returns the occupied size of the block cache, in bytes.
 long getFreeSize()
          Returns the free size of the block cache, in bytes.
 String getIoEngine()
           
 long getMaxSize()
           
 long getRealCacheSize()
           
 CacheStats getStats()
          Get the statistics for this block cache.
 long heapSize()
           
 Iterator<CachedBlock> iterator()
           
 void logStats()
           
 void shutdown()
          Shutdown the cache.
 long size()
          Returns the total size of the block cache, in bytes.
protected  void startWriterThreads()
          Called by the constructor to start the writer threads.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

DEFAULT_ERROR_TOLERATION_DURATION

public static final int DEFAULT_ERROR_TOLERATION_DURATION
See Also:
Constant Field Values
Constructor Detail

BucketCache

public BucketCache(String ioEngineName,
                   long capacity,
                   int blockSize,
                   int[] bucketSizes,
                   int writerThreadNum,
                   int writerQLen,
                   String persistencePath)
            throws FileNotFoundException,
                   IOException
Throws:
FileNotFoundException
IOException

BucketCache

public BucketCache(String ioEngineName,
                   long capacity,
                   int blockSize,
                   int[] bucketSizes,
                   int writerThreadNum,
                   int writerQLen,
                   String persistencePath,
                   int ioErrorsTolerationDuration)
            throws FileNotFoundException,
                   IOException
Throws:
FileNotFoundException
IOException
Method Detail

startWriterThreads

protected void startWriterThreads()
Called by the constructor to start the writer threads. Used by tests that need to override starting the threads.


getMaxSize

public long getMaxSize()

getIoEngine

public String getIoEngine()

cacheBlock

public void cacheBlock(BlockCacheKey cacheKey,
                       Cacheable buf)
Cache the block with the specified name and buffer.

Specified by:
cacheBlock in interface BlockCache
Parameters:
cacheKey - block's cache key
buf - block buffer

cacheBlock

public void cacheBlock(BlockCacheKey cacheKey,
                       Cacheable cachedItem,
                       boolean inMemory)
Cache the block with the specified name and buffer.

Specified by:
cacheBlock in interface BlockCache
Parameters:
cacheKey - block's cache key
cachedItem - block buffer
inMemory - if block is in-memory

cacheBlockWithWait

public void cacheBlockWithWait(BlockCacheKey cacheKey,
                               Cacheable cachedItem,
                               boolean inMemory,
                               boolean wait)
Cache the block to ramCache

Parameters:
cacheKey - block's cache key
cachedItem - block buffer
inMemory - if block is in-memory
wait - if true, blocking wait when queue is full

getBlock

public Cacheable getBlock(BlockCacheKey key,
                          boolean caching,
                          boolean repeat,
                          boolean updateCacheMetrics)
Get the buffer of the block with the specified key.

Specified by:
getBlock in interface BlockCache
Parameters:
key - block's cache key
caching - true if the caller caches blocks on cache misses
repeat - Whether this is a repeat lookup for the same block
updateCacheMetrics - Whether we should update cache metrics or not
Returns:
buffer of specified cache key, or null if not in cache

evictBlock

public boolean evictBlock(BlockCacheKey cacheKey)
Description copied from interface: BlockCache
Evict block from cache.

Specified by:
evictBlock in interface BlockCache
Parameters:
cacheKey - Block to evict
Returns:
true if block existed and was evicted, false if not

logStats

public void logStats()

getRealCacheSize

public long getRealCacheSize()

shutdown

public void shutdown()
Description copied from interface: BlockCache
Shutdown the cache.

Specified by:
shutdown in interface BlockCache

getStats

public CacheStats getStats()
Description copied from interface: BlockCache
Get the statistics for this block cache.

Specified by:
getStats in interface BlockCache
Returns:
Stats

getAllocator

public BucketAllocator getAllocator()

heapSize

public long heapSize()
Specified by:
heapSize in interface HeapSize
Returns:
Approximate 'exclusive deep size' of implementing object. Includes count of payload and hosting object sizings.

size

public long size()
Description copied from interface: BlockCache
Returns the total size of the block cache, in bytes.

Specified by:
size in interface BlockCache
Returns:
size of cache, in bytes

getFreeSize

public long getFreeSize()
Description copied from interface: BlockCache
Returns the free size of the block cache, in bytes.

Specified by:
getFreeSize in interface BlockCache
Returns:
free space in cache, in bytes

getBlockCount

public long getBlockCount()
Description copied from interface: BlockCache
Returns the number of blocks currently cached in the block cache.

Specified by:
getBlockCount in interface BlockCache
Returns:
number of blocks in the cache

getCurrentSize

public long getCurrentSize()
Description copied from interface: BlockCache
Returns the occupied size of the block cache, in bytes.

Specified by:
getCurrentSize in interface BlockCache
Returns:
occupied space in cache, in bytes

evictBlocksByHfileName

public int evictBlocksByHfileName(String hfileName)
Evicts all blocks for a specific HFile.

This is used for evict-on-close to remove all blocks of a specific HFile.

Specified by:
evictBlocksByHfileName in interface BlockCache
Returns:
the number of blocks evicted

iterator

public Iterator<CachedBlock> iterator()
Specified by:
iterator in interface Iterable<CachedBlock>
Specified by:
iterator in interface BlockCache
Returns:
Iterator over the blocks in the cache.

getBlockCaches

public BlockCache[] getBlockCaches()
Specified by:
getBlockCaches in interface BlockCache
Returns:
The list of sub blockcaches that make up this one; returns null if no sub caches.


Copyright © 2007–2016 The Apache Software Foundation. All rights reserved.