org.apache.hadoop.hbase.io.hfile
Class HFileWriterV2

java.lang.Object
  extended by org.apache.hadoop.hbase.io.hfile.AbstractHFileWriter
      extended by org.apache.hadoop.hbase.io.hfile.HFileWriterV2
All Implemented Interfaces:
Closeable, HFile.Writer
Direct Known Subclasses:
HFileWriterV3

@InterfaceAudience.Private
public class HFileWriterV2
extends AbstractHFileWriter

Writes HFile format version 2.


Field Summary
protected  HFileBlock.Writer fsBlockWriter
          Unified version 2 block writer
static int KEY_VALUE_VER_WITH_MEMSTORE
          Version for KeyValue which includes memstore timestamp
static byte[] KEY_VALUE_VERSION
          KeyValue version in FileInfo
protected  long lastDataBlockOffset
          The offset of the last data block or 0 if the file is empty.
static byte[] MAX_MEMSTORE_TS_KEY
          Max memstore (mvcc) timestamp in FileInfo
protected  long maxMemstoreTS
           
 
Fields inherited from class org.apache.hadoop.hbase.io.hfile.AbstractHFileWriter
blockEncoder, cacheConf, closeOutputStream, comparator, entryCount, fileInfo, firstKeyInBlock, hFileContext, lastKeyBuffer, lastKeyLength, lastKeyOffset, metaData, metaNames, name, outputStream, path, totalKeyLength, totalUncompressedBytes, totalValueLength
 
Constructor Summary
HFileWriterV2(org.apache.hadoop.conf.Configuration conf, CacheConfig cacheConf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path, org.apache.hadoop.fs.FSDataOutputStream ostream, KeyValue.KVComparator comparator, HFileContext context)
          Constructor that takes a path, creates and closes the output stream.
 
Method Summary
 void addDeleteFamilyBloomFilter(BloomFilterWriter bfw)
          Store delete family Bloom filter in the file, which is only supported in HFile V2.
 void addGeneralBloomFilter(BloomFilterWriter bfw)
          Store general Bloom filter in the file.
 void addInlineBlockWriter(InlineBlockWriter ibw)
          Adds an inline block writer such as a multi-level block index writer or a compound Bloom filter writer.
 void append(byte[] key, byte[] value)
          Add key/value to file.
 void append(byte[] key, byte[] value, byte[] tag)
           
 void append(KeyValue kv)
          Add key/value to file.
protected  void append(long memstoreTS, byte[] key, int koffset, int klength, byte[] value, int voffset, int vlength)
          Add key/value to file.
 void appendMetaBlock(String metaBlockName, org.apache.hadoop.io.Writable content)
          Add a meta block to the end of the file.
protected  void checkBlockBoundary()
          At a block boundary, write all the inline blocks and opens new block.
 void close()
           
protected  void finishInit(org.apache.hadoop.conf.Configuration conf)
          Additional initialization steps
 HFileContext getFileContext()
          Return the file context for the HFile this writer belongs to
protected  int getMajorVersion()
           
protected  int getMinorVersion()
           
protected  void newBlock()
          Ready a new block for writing.
 
Methods inherited from class org.apache.hadoop.hbase.io.hfile.AbstractHFileWriter
appendFileInfo, checkKey, checkValue, compressionByName, createOutputStream, finishClose, finishFileInfo, getPath, toString, writeFileInfo
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait
 

Field Detail

MAX_MEMSTORE_TS_KEY

public static final byte[] MAX_MEMSTORE_TS_KEY
Max memstore (mvcc) timestamp in FileInfo


KEY_VALUE_VERSION

public static final byte[] KEY_VALUE_VERSION
KeyValue version in FileInfo


KEY_VALUE_VER_WITH_MEMSTORE

public static final int KEY_VALUE_VER_WITH_MEMSTORE
Version for KeyValue which includes memstore timestamp

See Also:
Constant Field Values

fsBlockWriter

protected HFileBlock.Writer fsBlockWriter
Unified version 2 block writer


lastDataBlockOffset

protected long lastDataBlockOffset
The offset of the last data block or 0 if the file is empty.


maxMemstoreTS

protected long maxMemstoreTS
Constructor Detail

HFileWriterV2

public HFileWriterV2(org.apache.hadoop.conf.Configuration conf,
                     CacheConfig cacheConf,
                     org.apache.hadoop.fs.FileSystem fs,
                     org.apache.hadoop.fs.Path path,
                     org.apache.hadoop.fs.FSDataOutputStream ostream,
                     KeyValue.KVComparator comparator,
                     HFileContext context)
              throws IOException
Constructor that takes a path, creates and closes the output stream.

Throws:
IOException
Method Detail

finishInit

protected void finishInit(org.apache.hadoop.conf.Configuration conf)
Additional initialization steps


checkBlockBoundary

protected void checkBlockBoundary()
                           throws IOException
At a block boundary, write all the inline blocks and opens new block.

Throws:
IOException

newBlock

protected void newBlock()
                 throws IOException
Ready a new block for writing.

Throws:
IOException

appendMetaBlock

public void appendMetaBlock(String metaBlockName,
                            org.apache.hadoop.io.Writable content)
Add a meta block to the end of the file. Call before close(). Metadata blocks are expensive. Fill one with a bunch of serialized data rather than do a metadata block per metadata instance. If metadata is small, consider adding to file info using AbstractHFileWriter.appendFileInfo(byte[], byte[])

Parameters:
metaBlockName - name of the block
content - will call readFields to get data later (DO NOT REUSE)

append

public void append(KeyValue kv)
            throws IOException
Add key/value to file. Keys must be added in an order that agrees with the Comparator passed on construction.

Parameters:
kv - KeyValue to add. Cannot be empty nor null.
Throws:
IOException

append

public void append(byte[] key,
                   byte[] value)
            throws IOException
Add key/value to file. Keys must be added in an order that agrees with the Comparator passed on construction.

Parameters:
key - Key to add. Cannot be empty nor null.
value - Value to add. Cannot be empty nor null.
Throws:
IOException

append

protected void append(long memstoreTS,
                      byte[] key,
                      int koffset,
                      int klength,
                      byte[] value,
                      int voffset,
                      int vlength)
               throws IOException
Add key/value to file. Keys must be added in an order that agrees with the Comparator passed on construction.

Parameters:
key -
koffset -
klength -
value -
voffset -
vlength -
Throws:
IOException

close

public void close()
           throws IOException
Throws:
IOException

addInlineBlockWriter

public void addInlineBlockWriter(InlineBlockWriter ibw)
Description copied from interface: HFile.Writer
Adds an inline block writer such as a multi-level block index writer or a compound Bloom filter writer.


addGeneralBloomFilter

public void addGeneralBloomFilter(BloomFilterWriter bfw)
Description copied from interface: HFile.Writer
Store general Bloom filter in the file. This does not deal with Bloom filter internals but is necessary, since Bloom filters are stored differently in HFile version 1 and version 2.


addDeleteFamilyBloomFilter

public void addDeleteFamilyBloomFilter(BloomFilterWriter bfw)
Description copied from interface: HFile.Writer
Store delete family Bloom filter in the file, which is only supported in HFile V2.


append

public void append(byte[] key,
                   byte[] value,
                   byte[] tag)
            throws IOException
Throws:
IOException

getMajorVersion

protected int getMajorVersion()

getMinorVersion

protected int getMinorVersion()

getFileContext

public HFileContext getFileContext()
Description copied from interface: HFile.Writer
Return the file context for the HFile this writer belongs to



Copyright © 2007–2015 The Apache Software Foundation. All rights reserved.