org.apache.hadoop.hbase.util
Class CompoundBloomFilterWriter

java.lang.Object
  extended by org.apache.hadoop.hbase.util.CompoundBloomFilterBase
      extended by org.apache.hadoop.hbase.util.CompoundBloomFilterWriter
All Implemented Interfaces:
InlineBlockWriter, BloomFilterBase, BloomFilterWriter

@InterfaceAudience.Private
public class CompoundBloomFilterWriter
extends CompoundBloomFilterBase
implements BloomFilterWriter, InlineBlockWriter

Adds methods required for writing a compound Bloom filter to the data section of an HFile to the CompoundBloomFilter class.


Field Summary
protected static org.apache.commons.logging.Log LOG
           
 
Fields inherited from class org.apache.hadoop.hbase.util.CompoundBloomFilterBase
comparator, errorRate, hashType, numChunks, totalByteSize, totalKeyCount, totalMaxKeys, VERSION
 
Constructor Summary
CompoundBloomFilterWriter(int chunkByteSizeHint, float errorRate, int hashType, int maxFold, boolean cacheOnWrite, org.apache.hadoop.io.RawComparator<byte[]> comparator)
           
 
Method Summary
 void add(byte[] bloomKey, int keyOffset, int keyLength)
          Adds a Bloom filter key.
 void allocBloom()
          Allocate memory for the bloom filter data.
 void blockWritten(long offset, int onDiskSize, int uncompressedSize)
          Called after a block has been written, and its offset, raw size, and compressed size have been determined.
 void compactBloom()
          Compact the Bloom filter before writing metadata & data to disk.
 boolean getCacheOnWrite()
           
 org.apache.hadoop.io.Writable getDataWriter()
          Get a writable interface into bloom filter data (the actual Bloom bits).
 BlockType getInlineBlockType()
          The type of blocks this block writer produces.
 org.apache.hadoop.io.Writable getMetaWriter()
          Get a writable interface into bloom filter meta data.
 boolean shouldWriteBlock(boolean closing)
          Determines whether there is a new block to be written out.
 void writeInlineBlock(DataOutput out)
          Writes the block to the provided stream.
 
Methods inherited from class org.apache.hadoop.hbase.util.CompoundBloomFilterBase
createBloomKey, getByteSize, getComparator, getKeyCount, getMaxKeys
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 
Methods inherited from interface org.apache.hadoop.hbase.util.BloomFilterBase
createBloomKey, getByteSize, getComparator, getKeyCount, getMaxKeys
 

Field Detail

LOG

protected static final org.apache.commons.logging.Log LOG
Constructor Detail

CompoundBloomFilterWriter

public CompoundBloomFilterWriter(int chunkByteSizeHint,
                                 float errorRate,
                                 int hashType,
                                 int maxFold,
                                 boolean cacheOnWrite,
                                 org.apache.hadoop.io.RawComparator<byte[]> comparator)
Parameters:
chunkByteSizeHint - each chunk's size in bytes. The real chunk size might be different as required by the fold factor.
errorRate - target false positive rate
hashType - hash function type to use
maxFold - maximum degree of folding allowed
Method Detail

shouldWriteBlock

public boolean shouldWriteBlock(boolean closing)
Description copied from interface: InlineBlockWriter
Determines whether there is a new block to be written out.

Specified by:
shouldWriteBlock in interface InlineBlockWriter
Parameters:
closing - whether the file is being closed, in which case we need to write out all available data and not wait to accumulate another block

add

public void add(byte[] bloomKey,
                int keyOffset,
                int keyLength)
Adds a Bloom filter key. This key must be greater than the previous key, as defined by the comparator this compound Bloom filter is configured with. For efficiency, key monotonicity is not checked here. See StoreFile.Writer.append( org.apache.hadoop.hbase.KeyValue) for the details of deduplication.

Specified by:
add in interface BloomFilterWriter
Parameters:
bloomKey - data to be added to the bloom
keyOffset - offset into the data to be added
keyLength - length of the data to be added

writeInlineBlock

public void writeInlineBlock(DataOutput out)
                      throws IOException
Description copied from interface: InlineBlockWriter
Writes the block to the provided stream. Must not write any magic records. Called only if InlineBlockWriter.shouldWriteBlock(boolean) returned true.

Specified by:
writeInlineBlock in interface InlineBlockWriter
Parameters:
out - a stream (usually a compressing stream) to write the block to
Throws:
IOException

blockWritten

public void blockWritten(long offset,
                         int onDiskSize,
                         int uncompressedSize)
Description copied from interface: InlineBlockWriter
Called after a block has been written, and its offset, raw size, and compressed size have been determined. Can be used to add an entry to a block index. If this type of inline blocks needs a block index, the inline block writer is responsible for maintaining it.

Specified by:
blockWritten in interface InlineBlockWriter
Parameters:
offset - the offset of the block in the stream
onDiskSize - the on-disk size of the block
uncompressedSize - the uncompressed size of the block

getInlineBlockType

public BlockType getInlineBlockType()
Description copied from interface: InlineBlockWriter
The type of blocks this block writer produces.

Specified by:
getInlineBlockType in interface InlineBlockWriter

getMetaWriter

public org.apache.hadoop.io.Writable getMetaWriter()
Description copied from interface: BloomFilterWriter
Get a writable interface into bloom filter meta data.

Specified by:
getMetaWriter in interface BloomFilterWriter
Returns:
a writable instance that can be later written to a stream

compactBloom

public void compactBloom()
Description copied from interface: BloomFilterWriter
Compact the Bloom filter before writing metadata & data to disk.

Specified by:
compactBloom in interface BloomFilterWriter

allocBloom

public void allocBloom()
Description copied from interface: BloomFilterWriter
Allocate memory for the bloom filter data.

Specified by:
allocBloom in interface BloomFilterWriter

getDataWriter

public org.apache.hadoop.io.Writable getDataWriter()
Description copied from interface: BloomFilterWriter
Get a writable interface into bloom filter data (the actual Bloom bits). Not used for compound Bloom filters.

Specified by:
getDataWriter in interface BloomFilterWriter
Returns:
a writable instance that can be later written to a stream

getCacheOnWrite

public boolean getCacheOnWrite()
Specified by:
getCacheOnWrite in interface InlineBlockWriter
Returns:
true if inline blocks produced by this writer should be cached


Copyright © 2013 The Apache Software Foundation. All Rights Reserved.