public class FastDiffDeltaEncoder extends Object
DiffKeyDeltaEncoder
but supposedly faster.
Compress using:
- store size of common prefix
- save column family once in the first KeyValue
- use integer compression for key, value and prefix (7-bit encoding)
- use bits to avoid duplication key length, value length
and type if it same as previous
- store in 3 bits length of prefix timestamp
with previous KeyValue's timestamp
- one bit which allow to omit value if it is the same
Format:
- 1 byte: flag
- 1-5 bytes: key length (only if FLAG_SAME_KEY_LENGTH is not set in flag)
- 1-5 bytes: value length (only if FLAG_SAME_VALUE_LENGTH is not set in flag)
- 1-5 bytes: prefix length
- ... bytes: rest of the row (if prefix length is small enough)
- ... bytes: qualifier (or suffix depending on prefix length)
- 1-8 bytes: timestamp suffix
- 1 byte: type (only if FLAG_SAME_TYPE is not set in the flag)
- ... bytes: value (only if FLAG_SAME_VALUE is not set in the flag)Modifier and Type | Class and Description |
---|---|
protected static class |
BufferedDataBlockEncoder.BufferedEncodedSeeker<STATE extends BufferedDataBlockEncoder.SeekerState> |
protected static class |
BufferedDataBlockEncoder.SeekerState |
protected static class |
FastDiffDeltaEncoder.FastDiffSeekerState |
DataBlockEncoder.EncodedSeeker
Constructor and Description |
---|
FastDiffDeltaEncoder() |
Modifier and Type | Method and Description |
---|---|
protected void |
afterDecodingKeyValue(DataInputStream source,
ByteBuffer dest,
boolean includesMemstoreTS) |
protected void |
afterEncodingKeyValue(ByteBuffer in,
DataOutputStream out,
boolean includesMemstoreTS) |
void |
compressKeyValues(DataOutputStream out,
ByteBuffer in,
boolean includesMemstoreTS)
Compress KeyValues and write them to output buffer.
|
DataBlockEncoder.EncodedSeeker |
createSeeker(org.apache.hadoop.io.RawComparator<byte[]> comparator,
boolean includesMemstoreTS)
Create a HFileBlock seeker which find KeyValues within a block.
|
ByteBuffer |
getFirstKeyInBlock(ByteBuffer block)
Return first key in block.
|
String |
toString() |
ByteBuffer |
uncompressKeyValues(DataInputStream source,
boolean includesMemstoreTS)
Uncompress.
|
ByteBuffer |
uncompressKeyValues(DataInputStream source,
int allocHeaderLength,
int skipLastBytes,
boolean includesMemstoreTS)
Uncompress.
|
public void compressKeyValues(DataOutputStream out, ByteBuffer in, boolean includesMemstoreTS) throws IOException
DataBlockEncoder
out
- Where to write compressed data.in
- Source of KeyValue for compression.includesMemstoreTS
- true if including memstore timestamp after every
key-value pairIOException
- If there is an error writing to output stream.public ByteBuffer uncompressKeyValues(DataInputStream source, int allocHeaderLength, int skipLastBytes, boolean includesMemstoreTS) throws IOException
DataBlockEncoder
source
- Compressed stream of KeyValues.allocHeaderLength
- allocate this many bytes for the header.skipLastBytes
- Do not copy n last bytes.includesMemstoreTS
- true if including memstore timestamp after every
key-value pairIOException
- If there is an error in source.public ByteBuffer getFirstKeyInBlock(ByteBuffer block)
DataBlockEncoder
block
- encoded block we want index, the position will not changepublic DataBlockEncoder.EncodedSeeker createSeeker(org.apache.hadoop.io.RawComparator<byte[]> comparator, boolean includesMemstoreTS)
DataBlockEncoder
comparator
- what kind of comparison should be usedincludesMemstoreTS
- true if including memstore timestamp after every
key-value pairpublic ByteBuffer uncompressKeyValues(DataInputStream source, boolean includesMemstoreTS) throws IOException
DataBlockEncoder
uncompressKeyValues
in interface DataBlockEncoder
source
- Compressed stream of KeyValues.includesMemstoreTS
- true if including memstore timestamp after every
key-value pairIOException
- If there is an error in source.protected final void afterEncodingKeyValue(ByteBuffer in, DataOutputStream out, boolean includesMemstoreTS)
protected final void afterDecodingKeyValue(DataInputStream source, ByteBuffer dest, boolean includesMemstoreTS)
Copyright © 2014 The Apache Software Foundation. All Rights Reserved.