public class StoreFile extends SchemaConfigured
StoreFile#WriterBuilder
and append data. Be sure to add any metadata before calling close on the
Writer (Use the appendMetadata convenience methods). On close, a StoreFile
is sitting in the Filesystem. To refer to it, create a StoreFile instance
passing filesystem and path. To read, call createReader()
.
StoreFiles may also reference store files in another Store. The reason for this weird pattern where you use a different instance for the writer and a reader is that we write once but read a lot more.
Modifier and Type | Class and Description |
---|---|
static class |
StoreFile.BloomType |
static class |
StoreFile.Reader
Reader for a StoreFile.
|
static class |
StoreFile.Writer
A StoreFile writer.
|
static class |
StoreFile.WriterBuilder |
Modifier and Type | Field and Description |
---|---|
static byte[] |
BLOOM_FILTER_TYPE_KEY
Bloom filter Type in FileInfo
|
static byte[] |
BULKLOAD_TASK_KEY
Meta key set when store file is a result of a bulk load
|
static byte[] |
BULKLOAD_TIME_KEY |
static int |
DEFAULT_BLOCKSIZE_SMALL |
static byte[] |
DELETE_FAMILY_COUNT
Delete Family Count in FileInfo
|
static byte[] |
EARLIEST_PUT_TS
Key for timestamp of earliest-put in metadata
|
static byte[] |
EXCLUDE_FROM_MINOR_COMPACTION_KEY
Major compaction flag in FileInfo
|
static String |
HFILE_NAME_REGEX
A non-capture group, for hfiles, so that this can be embedded.
|
static byte[] |
MAJOR_COMPACTION_KEY
Major compaction flag in FileInfo
|
static byte[] |
MAX_SEQ_ID_KEY
Max Sequence ID in FileInfo
|
static byte[] |
TIMERANGE_KEY
Key for Timerange information in metadata
|
SCHEMA_CONFIGURED_UNALIGNED_HEAP_SIZE
Constructor and Description |
---|
StoreFile(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path p,
org.apache.hadoop.conf.Configuration conf,
CacheConfig cacheConf,
StoreFile.BloomType cfBloomType,
HFileDataBlockEncoder dataBlockEncoder)
Constructor, loads a reader and it's indices, etc.
|
Modifier and Type | Method and Description |
---|---|
void |
closeReader(boolean evictOnClose) |
StoreFile.Reader |
createReader() |
void |
deleteReader()
Delete this file
|
long |
getBulkLoadTimestamp()
Return the timestamp at which this bulk load file was generated.
|
HDFSBlocksDistribution |
getHDFSBlockDistribution() |
long |
getMaxMemstoreTS() |
static long |
getMaxMemstoreTSInList(Collection<StoreFile> sfs)
Return the largest memstoreTS found across all storefiles in
the given list.
|
long |
getMaxSequenceId() |
static long |
getMaxSequenceIdInList(Collection<StoreFile> sfs)
Return the highest sequence ID found across all storefiles in
the given list.
|
long |
getModificationTimeStamp() |
org.apache.hadoop.fs.Path |
getPath() |
StoreFile.Reader |
getReader() |
static org.apache.hadoop.fs.Path |
getReferredToFile(org.apache.hadoop.fs.Path p) |
static org.apache.hadoop.fs.Path |
getUniqueFile(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path dir) |
static boolean |
isReference(org.apache.hadoop.fs.Path p) |
static boolean |
isReference(String name) |
static org.apache.hadoop.fs.Path |
rename(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path src,
org.apache.hadoop.fs.Path tgt)
Utility to help with rename.
|
void |
setMaxMemstoreTS(long maxMemstoreTS) |
String |
toString() |
String |
toStringDetailed() |
static boolean |
validateStoreFileName(String fileName)
Validate the store file name.
|
createUnknown, getColumnFamilyName, getSchemaMetrics, getTableName, heapSize, isSchemaConfigured, passSchemaMetricsTo, resetSchemaMetricsConf, schemaConfAsJSON, schemaConfigurationChanged
public static final byte[] MAX_SEQ_ID_KEY
public static final byte[] MAJOR_COMPACTION_KEY
public static final byte[] EXCLUDE_FROM_MINOR_COMPACTION_KEY
public static final byte[] BLOOM_FILTER_TYPE_KEY
public static final byte[] DELETE_FAMILY_COUNT
public static final byte[] TIMERANGE_KEY
public static final byte[] EARLIEST_PUT_TS
public static final int DEFAULT_BLOCKSIZE_SMALL
public static final byte[] BULKLOAD_TASK_KEY
public static final byte[] BULKLOAD_TIME_KEY
public static final String HFILE_NAME_REGEX
public StoreFile(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path p, org.apache.hadoop.conf.Configuration conf, CacheConfig cacheConf, StoreFile.BloomType cfBloomType, HFileDataBlockEncoder dataBlockEncoder) throws IOException
fs
- The current file system to use.p
- The path of the file.blockcache
- true
if the block cache is enabled.conf
- The current configuration.cacheConf
- The cache configuration and block cache reference.cfBloomType
- The bloom type to use for this store file as specified
by column family configuration. This may or may not be the same
as the Bloom filter type actually present in the HFile, because
column family configuration might change. If this is
StoreFile.BloomType.NONE
, the existing Bloom filter is ignored.dataBlockEncoder
- data block encoding algorithm.IOException
- When opening the reader fails.public long getMaxMemstoreTS()
public void setMaxMemstoreTS(long maxMemstoreTS)
public org.apache.hadoop.fs.Path getPath()
public static boolean isReference(org.apache.hadoop.fs.Path p)
p
- Path to check.public static boolean isReference(String name)
name
- file name to check.public static org.apache.hadoop.fs.Path getReferredToFile(org.apache.hadoop.fs.Path p)
public long getMaxSequenceId()
public long getModificationTimeStamp()
public static long getMaxMemstoreTSInList(Collection<StoreFile> sfs)
public static long getMaxSequenceIdInList(Collection<StoreFile> sfs)
public long getBulkLoadTimestamp()
public HDFSBlocksDistribution getHDFSBlockDistribution()
public StoreFile.Reader createReader() throws IOException
IOException
public StoreFile.Reader getReader()
createReader()
public void closeReader(boolean evictOnClose) throws IOException
evictOnClose
- whether to evict blocks belonging to this fileIOException
public void deleteReader() throws IOException
IOException
public String toStringDetailed()
public static org.apache.hadoop.fs.Path rename(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path tgt) throws IOException
fs
- src
- tgt
- IOException
public static org.apache.hadoop.fs.Path getUniqueFile(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path dir) throws IOException
fs
- dir
- Directory to create file in.dir
IOException
public static boolean validateStoreFileName(String fileName)
fileName
- name of the file to validateCopyright © 2014 The Apache Software Foundation. All Rights Reserved.