org.apache.hadoop.hbase.io.hfile
Class HFileBlock

java.lang.Object
  extended by org.apache.hadoop.hbase.io.hfile.HFileBlock
All Implemented Interfaces:
HeapSize, Cacheable

public class HFileBlock
extends Object
implements Cacheable

Reading HFile version 1 and 2 blocks, and writing version 2 blocks.

The version 2 block representation in the block cache is the same as above, except that the data section is always uncompressed in the cache.


Nested Class Summary
static class HFileBlock.AbstractFSReader
          A common implementation of some methods of HFileBlock.FSReader and some tools for implementing HFile format version-specific block readers.
static interface HFileBlock.BlockIterator
          An interface allowing to iterate HFileBlocks.
static interface HFileBlock.BlockWritable
          Something that can be written into a block.
static interface HFileBlock.FSReader
          A full-fledged reader with iteration ability.
static class HFileBlock.FSReaderV1
          Reads version 1 blocks from the file system.
static class HFileBlock.FSReaderV2
          Reads version 2 blocks from the filesystem.
static class HFileBlock.Writer
          Unified version 2 HFile block writer.
 
Field Summary
static int BYTE_BUFFER_HEAP_SIZE
           
static byte[] DUMMY_HEADER
          Just an array of bytes of the right size.
static int HEADER_SIZE
          The size of a version 2 HFile block header
 
Constructor Summary
HFileBlock(BlockType blockType, int onDiskSizeWithoutHeader, int uncompressedSizeWithoutHeader, long prevBlockOffset, ByteBuffer buf, boolean fillHeader, long offset)
          Creates a new HFile block from the given fields.
 
Method Summary
 void assumeUncompressed()
          An additional sanity-check in case no compression is being used.
 boolean equals(Object comparison)
           
 void expectType(BlockType expectedType)
           
 BlockType getBlockType()
           
 ByteBuffer getBufferReadOnly()
          Returns the buffer this block stores internally.
 ByteBuffer getBufferWithHeader()
          Returns a byte buffer of this block, including header data, positioned at the beginning of header.
 ByteBuffer getBufferWithoutHeader()
          Returns a buffer that does not include the header.
 DataInputStream getByteStream()
           
 CacheableDeserializer<Cacheable> getDeserializer()
          Returns CacheableDeserializer instance which reconstructs original object from ByteBuffer.
 int getNextBlockOnDiskSizeWithHeader()
           
 long getOffset()
           
 int getOnDiskSizeWithHeader()
           
 int getOnDiskSizeWithoutHeader()
          Returns the size of the compressed part of the block in case compression is used, or the uncompressed size of the data part otherwise.
 long getPrevBlockOffset()
           
 int getSerializedLength()
          Returns the length of the ByteBuffer required to serialized the object.
 int getUncompressedSizeWithoutHeader()
           
 long heapSize()
           
 void readInto(org.apache.hadoop.io.Writable w)
          Deserializes fields of the given writable using the data portion of this block.
static boolean readWithExtra(InputStream in, byte[] buf, int bufOffset, int necessaryLen, int extraLen)
          Read from an input stream.
 void serialize(ByteBuffer destination)
          Serializes its data into destination.
 String toString()
           
 
Methods inherited from class java.lang.Object
clone, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Field Detail

HEADER_SIZE

public static final int HEADER_SIZE
The size of a version 2 HFile block header

See Also:
Constant Field Values

DUMMY_HEADER

public static final byte[] DUMMY_HEADER
Just an array of bytes of the right size.


BYTE_BUFFER_HEAP_SIZE

public static final int BYTE_BUFFER_HEAP_SIZE
Constructor Detail

HFileBlock

public HFileBlock(BlockType blockType,
                  int onDiskSizeWithoutHeader,
                  int uncompressedSizeWithoutHeader,
                  long prevBlockOffset,
                  ByteBuffer buf,
                  boolean fillHeader,
                  long offset)
Creates a new HFile block from the given fields. This constructor is mostly used when the block data has already been read and uncompressed, and is sitting in a byte buffer.

Parameters:
blockType - the type of this block, see BlockType
onDiskSizeWithoutHeader - compressed size of the block if compression is used, otherwise uncompressed size, header size not included
uncompressedSizeWithoutHeader - uncompressed size of the block, header size not included. Equals onDiskSizeWithoutHeader if compression is disabled.
prevBlockOffset - the offset of the previous block in the HFile
buf - block header (HEADER_SIZE bytes) followed by uncompressed data. This
fillHeader - true to fill in the first HEADER_SIZE bytes of the buffer based on the header fields provided
offset - the file offset the block was read from
Method Detail

getBlockType

public BlockType getBlockType()

getOnDiskSizeWithHeader

public int getOnDiskSizeWithHeader()
Returns:
the on-disk size of the block with header size included

getOnDiskSizeWithoutHeader

public int getOnDiskSizeWithoutHeader()
Returns the size of the compressed part of the block in case compression is used, or the uncompressed size of the data part otherwise. Header size is not included.

Returns:
the on-disk size of the data part of the block, header not included

getUncompressedSizeWithoutHeader

public int getUncompressedSizeWithoutHeader()
Returns:
the uncompressed size of the data part of the block, header not included

getPrevBlockOffset

public long getPrevBlockOffset()
Returns:
the offset of the previous block of the same type in the file, or -1 if unknown

getBufferWithoutHeader

public ByteBuffer getBufferWithoutHeader()
Returns a buffer that does not include the header. The array offset points to the start of the block data right after the header. The underlying data array is not copied.

Returns:
the buffer with header skipped

getBufferReadOnly

public ByteBuffer getBufferReadOnly()
Returns the buffer this block stores internally. The clients must not modify the buffer object. This method has to be public because it is used in CompoundBloomFilter to avoid object creation on every Bloom filter lookup, but has to be used with caution.

Returns:
the buffer of this block for read-only operations

getBufferWithHeader

public ByteBuffer getBufferWithHeader()
Returns a byte buffer of this block, including header data, positioned at the beginning of header. The underlying data array is not copied.

Returns:
the byte buffer with header included

readInto

public void readInto(org.apache.hadoop.io.Writable w)
              throws IOException
Deserializes fields of the given writable using the data portion of this block. Does not check that all the block data has been read.

Throws:
IOException

toString

public String toString()
Overrides:
toString in class Object

assumeUncompressed

public void assumeUncompressed()
                        throws IOException
An additional sanity-check in case no compression is being used.

Throws:
IOException

expectType

public void expectType(BlockType expectedType)
                throws IOException
Parameters:
expectedType - the expected type of this block
Throws:
IOException - if this block's type is different than expected

getOffset

public long getOffset()
Returns:
the offset of this block in the file it was read from

getByteStream

public DataInputStream getByteStream()
Returns:
a byte stream reading the data section of this block

heapSize

public long heapSize()
Specified by:
heapSize in interface HeapSize
Returns:
Approximate 'exclusive deep size' of implementing object. Includes count of payload and hosting object sizings.

readWithExtra

public static boolean readWithExtra(InputStream in,
                                    byte[] buf,
                                    int bufOffset,
                                    int necessaryLen,
                                    int extraLen)
                             throws IOException
Read from an input stream. Analogous to IOUtils.readFully(InputStream, byte[], int, int), but specifies a number of "extra" bytes that would be desirable but not absolutely necessary to read.

Parameters:
in - the input stream to read from
buf - the buffer to read into
bufOffset - the destination offset in the buffer
necessaryLen - the number of bytes that are absolutely necessary to read
extraLen - the number of extra bytes that would be nice to read
Returns:
true if succeeded reading the extra bytes
Throws:
IOException - if failed to read the necessary bytes

getNextBlockOnDiskSizeWithHeader

public int getNextBlockOnDiskSizeWithHeader()
Returns:
the on-disk size of the next block (including the header size) that was read by peeking into the next block's header

getSerializedLength

public int getSerializedLength()
Description copied from interface: Cacheable
Returns the length of the ByteBuffer required to serialized the object. If the object cannot be serialized, it should also return 0.

Specified by:
getSerializedLength in interface Cacheable
Returns:
int length in bytes of the serialized form.

serialize

public void serialize(ByteBuffer destination)
Description copied from interface: Cacheable
Serializes its data into destination.

Specified by:
serialize in interface Cacheable

getDeserializer

public CacheableDeserializer<Cacheable> getDeserializer()
Description copied from interface: Cacheable
Returns CacheableDeserializer instance which reconstructs original object from ByteBuffer.

Specified by:
getDeserializer in interface Cacheable
Returns:
CacheableDeserialzer instance.

equals

public boolean equals(Object comparison)
Overrides:
equals in class Object


Copyright © 2012 The Apache Software Foundation. All Rights Reserved.