org.apache.hadoop.hbase.io.encoding
Class DiffKeyDeltaEncoder

java.lang.Object
  extended by org.apache.hadoop.hbase.io.encoding.DiffKeyDeltaEncoder
All Implemented Interfaces:
DataBlockEncoder

public class DiffKeyDeltaEncoder
extends Object

Compress using: - store size of common prefix - save column family once, it is same within HFile - use integer compression for key, value and prefix (7-bit encoding) - use bits to avoid duplication key length, value length and type if it same as previous - store in 3 bits length of timestamp field - allow diff in timestamp instead of actual value Format: - 1 byte: flag - 1-5 bytes: key length (only if FLAG_SAME_KEY_LENGTH is not set in flag) - 1-5 bytes: value length (only if FLAG_SAME_VALUE_LENGTH is not set in flag) - 1-5 bytes: prefix length - ... bytes: rest of the row (if prefix length is small enough) - ... bytes: qualifier (or suffix depending on prefix length) - 1-8 bytes: timestamp or diff - 1 byte: type (only if FLAG_SAME_TYPE is not set in the flag) - ... bytes: value


Nested Class Summary
protected static class BufferedDataBlockEncoder.BufferedEncodedSeeker<STATE extends BufferedDataBlockEncoder.SeekerState>
           
protected static class BufferedDataBlockEncoder.SeekerState
           
protected static class DiffKeyDeltaEncoder.DiffCompressionState
           
protected static class DiffKeyDeltaEncoder.DiffSeekerState
           
 
Nested classes/interfaces inherited from interface org.apache.hadoop.hbase.io.encoding.DataBlockEncoder
DataBlockEncoder.EncodedSeeker
 
Constructor Summary
DiffKeyDeltaEncoder()
           
 
Method Summary
protected  void afterDecodingKeyValue(DataInputStream source, ByteBuffer dest, boolean includesMemstoreTS)
           
protected  void afterEncodingKeyValue(ByteBuffer in, DataOutputStream out, boolean includesMemstoreTS)
           
 void compressKeyValues(DataOutputStream out, ByteBuffer in, boolean includesMemstoreTS)
          Compress KeyValues and write them to output buffer.
 DataBlockEncoder.EncodedSeeker createSeeker(org.apache.hadoop.io.RawComparator<byte[]> comparator, boolean includesMemstoreTS)
          Create a HFileBlock seeker which find KeyValues within a block.
 ByteBuffer getFirstKeyInBlock(ByteBuffer block)
          Return first key in block.
 String toString()
           
 ByteBuffer uncompressKeyValues(DataInputStream source, boolean includesMemstoreTS)
          Uncompress.
 ByteBuffer uncompressKeyValues(DataInputStream source, int allocHeaderLength, int skipLastBytes, boolean includesMemstoreTS)
          Uncompress.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Constructor Detail

DiffKeyDeltaEncoder

public DiffKeyDeltaEncoder()
Method Detail

compressKeyValues

public void compressKeyValues(DataOutputStream out,
                              ByteBuffer in,
                              boolean includesMemstoreTS)
                       throws IOException
Description copied from interface: DataBlockEncoder
Compress KeyValues and write them to output buffer.

Parameters:
out - Where to write compressed data.
in - Source of KeyValue for compression.
includesMemstoreTS - true if including memstore timestamp after every key-value pair
Throws:
IOException - If there is an error writing to output stream.

uncompressKeyValues

public ByteBuffer uncompressKeyValues(DataInputStream source,
                                      int allocHeaderLength,
                                      int skipLastBytes,
                                      boolean includesMemstoreTS)
                               throws IOException
Description copied from interface: DataBlockEncoder
Uncompress.

Parameters:
source - Compressed stream of KeyValues.
allocHeaderLength - allocate this many bytes for the header.
skipLastBytes - Do not copy n last bytes.
includesMemstoreTS - true if including memstore timestamp after every key-value pair
Returns:
Uncompressed block of KeyValues.
Throws:
IOException - If there is an error in source.

getFirstKeyInBlock

public ByteBuffer getFirstKeyInBlock(ByteBuffer block)
Description copied from interface: DataBlockEncoder
Return first key in block. Useful for indexing. Typically does not make a deep copy but returns a buffer wrapping a segment of the actual block's byte array. This is because the first key in block is usually stored unencoded.

Parameters:
block - encoded block we want index, the position will not change
Returns:
First key in block.

toString

public String toString()
Overrides:
toString in class Object

createSeeker

public DataBlockEncoder.EncodedSeeker createSeeker(org.apache.hadoop.io.RawComparator<byte[]> comparator,
                                                   boolean includesMemstoreTS)
Description copied from interface: DataBlockEncoder
Create a HFileBlock seeker which find KeyValues within a block.

Parameters:
comparator - what kind of comparison should be used
includesMemstoreTS - true if including memstore timestamp after every key-value pair
Returns:
A newly created seeker.

uncompressKeyValues

public ByteBuffer uncompressKeyValues(DataInputStream source,
                                      boolean includesMemstoreTS)
                               throws IOException
Description copied from interface: DataBlockEncoder
Uncompress.

Specified by:
uncompressKeyValues in interface DataBlockEncoder
Parameters:
source - Compressed stream of KeyValues.
includesMemstoreTS - true if including memstore timestamp after every key-value pair
Returns:
Uncompressed block of KeyValues.
Throws:
IOException - If there is an error in source.

afterEncodingKeyValue

protected final void afterEncodingKeyValue(ByteBuffer in,
                                           DataOutputStream out,
                                           boolean includesMemstoreTS)

afterDecodingKeyValue

protected final void afterDecodingKeyValue(DataInputStream source,
                                           ByteBuffer dest,
                                           boolean includesMemstoreTS)


Copyright © 2014 The Apache Software Foundation. All Rights Reserved.