public class IgfsHadoopParameters extends Object
core-site.xml
file
or passed to map-reduce task directly when using Hadoop driver for IGFS file system:
fs.igfs.[name].open.sequential_reads_before_prefetch
- this parameter overrides
the one specified in IgfsConfiguration.getSequentialReadsBeforePrefetch()
IGFS data node configuration property.
fs.igfs.[name].log.enabled
- specifies whether IGFS sampling logger is enabled. If
true
, then all file system operations will be logged to a file.
fs.igfs.[name].log.dir
- specifies log directory where sampling log files should be placed.fs.igfs.[name].log.batch_size
- specifies how many log entries are accumulated in a batch before
it gets flushed to log file. Higher values will imply greater performance, but will increase delay
before record appears in the log file.
fs.igfs.[name].colocated.writes
- specifies whether written files should be colocated on data
node to which client is connected. If true
, file will not be distributed and will be written
to a single data node. Default value is true
.
fs.igfs.prefer.local.writes
- specifies whether file preferably should be written to
local data node if it has enough free space. After some time it can be redistributed across nodes though.
[name]
is file system endpoint which you specify in file system URI authority part. E.g. in
case your file system URI is igfs://127.0.0.1:10500
then name
will be 127.0.0.1:10500
.
Sample configuration that can be placed to core-site.xml
file:
<property> <name>fs.igfs.127.0.0.1:10500.log.enabled</name> <value>true</value> </property> <property> <name>fs.igfs.127.0.0.1:10500.log.dir</name> <value>/home/apache/ignite/log/sampling</value> </property> <property> <name>fs.igfs.127.0.0.1:10500.log.batch_size</name> <value>16</value> </property>Parameters could also be specified per mapreduce job, e.g.
hadoop jar myjarfile.jar MyMapReduceJob -Dfs.igfs.open.sequential_reads_before_prefetch=4If you want to use these parameters in code, then you have to substitute you file system name in it. The easiest way to do that is
String.format(PARAM_IGFS_COLOCATED_WRITES, [name])
.Modifier and Type | Field and Description |
---|---|
static String |
PARAM_IGFS_COLOCATED_WRITES
Parameter name for control over file colocation write mode.
|
static String |
PARAM_IGFS_LOG_BATCH_SIZE
Parameter name for log batch size.
|
static String |
PARAM_IGFS_LOG_DIR
Parameter name for client logger directory.
|
static String |
PARAM_IGFS_LOG_ENABLED
Parameter name for log enabled flag.
|
static String |
PARAM_IGFS_PREFER_LOCAL_WRITES
Parameter name for prefer local writes flag.
|
static String |
PARAM_IGFS_SEQ_READS_BEFORE_PREFETCH
Parameter name for custom sequential reads before prefetch value.
|
Constructor and Description |
---|
IgfsHadoopParameters() |
public static final String PARAM_IGFS_COLOCATED_WRITES
public static final String PARAM_IGFS_SEQ_READS_BEFORE_PREFETCH
public static final String PARAM_IGFS_LOG_DIR
public static final String PARAM_IGFS_LOG_BATCH_SIZE
public static final String PARAM_IGFS_LOG_ENABLED
public static final String PARAM_IGFS_PREFER_LOCAL_WRITES
Follow @ApacheIgnite
Apache Ignite Fabric : ver. 1.0.0-RC1 Release Date : February 16 2015