public interface IgniteFs extends Igfs, IgniteAsyncSupport
ignite.sh
or
ignite.bat
scripts, like this: ignite.sh config/hadoop/default-config.xml
IGFS
can be started as a data node or as a client node. Data node is responsible for
caching data, while client node is responsible for basic file system operations and accessing
data nodes remotely. When used as Hadoop file system, clients nodes usually started together
with job-submitter
or job-scheduler
processes, while data nodes are usually
started together with Hadoop task-tracker
processes.
IGFS
can be integrated with Hadoop
by
plugging in as Hadoop FileSystem
. Refer to
org.apache.ignite.igfs.hadoop.v1.IgfsHadoopFileSystem
or
org.apache.ignite.igfs.hadoop.v2.IgfsHadoopFileSystem
for more information.
NOTE: integration with Hadoop is available only in In-Memory Accelerator For Hadoop
edition.
Modifier and Type | Field and Description |
---|---|
static String |
IGFS_SCHEME
IGFS scheme name.
|
static String |
PROP_PREFER_LOCAL_WRITES
File property: prefer writes to local node.
|
PROP_GROUP_NAME, PROP_PERMISSION, PROP_USER_NAME
Modifier and Type | Method and Description |
---|---|
Collection<IgfsBlockLocation> |
affinity(IgfsPath path,
long start,
long len)
Gets affinity block locations for data blocks of the file, i.e. the nodes, on which the blocks
are stored.
|
Collection<IgfsBlockLocation> |
affinity(IgfsPath path,
long start,
long len,
long maxLen)
Get affinity block locations for data blocks of the file.
|
IgfsOutputStream |
append(IgfsPath path,
boolean create)
Opens an output stream to an existing file for appending data.
|
IgfsOutputStream |
append(IgfsPath path,
int bufSize,
boolean create,
Map<String,String> props)
Opens an output stream to an existing file for appending data.
|
IgfsConfiguration |
configuration()
Gets IGFS configuration.
|
IgfsOutputStream |
create(IgfsPath path,
boolean overwrite)
Creates a file and opens it for writing.
|
IgfsOutputStream |
create(IgfsPath path,
int bufSize,
boolean overwrite,
IgniteUuid affKey,
int replication,
long blockSize,
Map<String,String> props)
Creates a file and opens it for writing.
|
IgfsOutputStream |
create(IgfsPath path,
int bufSize,
boolean overwrite,
int replication,
long blockSize,
Map<String,String> props)
Creates a file and opens it for writing.
|
<T,R> R |
execute(Class<? extends IgfsTask<T,R>> taskCls,
IgfsRecordResolver rslvr,
Collection<IgfsPath> paths,
boolean skipNonExistentFiles,
long maxRangeLen,
T arg)
Executes IGFS task with overridden maximum range length (see
IgfsConfiguration.getMaximumTaskRangeLength() for more information). |
<T,R> R |
execute(Class<? extends IgfsTask<T,R>> taskCls,
IgfsRecordResolver rslvr,
Collection<IgfsPath> paths,
T arg)
Executes IGFS task.
|
<T,R> R |
execute(IgfsTask<T,R> task,
IgfsRecordResolver rslvr,
Collection<IgfsPath> paths,
boolean skipNonExistentFiles,
long maxRangeLen,
T arg)
Executes IGFS task with overridden maximum range length (see
IgfsConfiguration.getMaximumTaskRangeLength() for more information). |
<T,R> R |
execute(IgfsTask<T,R> task,
IgfsRecordResolver rslvr,
Collection<IgfsPath> paths,
T arg)
Executes IGFS task.
|
void |
format()
Formats the file system removing all existing entries from it.
|
IgfsMetrics |
metrics()
Gets metrics snapshot for this file system.
|
String |
name()
Gets IGFS name.
|
IgfsInputStream |
open(IgfsPath path)
Opens a file for reading.
|
IgfsInputStream |
open(IgfsPath path,
int bufSize)
Opens a file for reading.
|
IgfsInputStream |
open(IgfsPath path,
int bufSize,
int seqReadsBeforePrefetch)
Opens a file for reading.
|
void |
resetMetrics()
Resets metrics for this file system.
|
void |
setTimes(IgfsPath path,
long accessTime,
long modificationTime)
Sets last access time and last modification time for a given path.
|
long |
size(IgfsPath path)
Determines size of the file denoted by provided path.
|
IgfsPathSummary |
summary(IgfsPath path)
Gets summary (total number of files, total number of directories and total length)
for a given path.
|
IgniteFs |
withAsync()
Gets component with asynchronous mode enabled.
|
delete, exists, info, listFiles, listPaths, mkdirs, mkdirs, properties, rename, update, usedSpaceSize
future, isAsync
static final String IGFS_SCHEME
static final String PROP_PREFER_LOCAL_WRITES
String name()
null
for default file system.IgfsConfiguration configuration()
IgfsPathSummary summary(IgfsPath path) throws IgniteException
path
- Path to get information for.IgfsFileNotFoundException
- If path is not found.IgniteException
- If failed.IgfsInputStream open(IgfsPath path) throws IgniteException
path
- File path to read.IgniteException
- In case of error.IgfsFileNotFoundException
- If path doesn't exist.IgfsInputStream open(IgfsPath path, int bufSize) throws IgniteException
open
in interface Igfs
path
- File path to read.bufSize
- Read buffer size (bytes) or zero
to use default value.IgniteException
- In case of error.IgfsFileNotFoundException
- If path doesn't exist.IgfsInputStream open(IgfsPath path, int bufSize, int seqReadsBeforePrefetch) throws IgniteException
path
- File path to read.bufSize
- Read buffer size (bytes) or zero
to use default value.seqReadsBeforePrefetch
- Amount of sequential reads before prefetch is started.IgniteException
- In case of error.IgfsFileNotFoundException
- If path doesn't exist.IgfsOutputStream create(IgfsPath path, boolean overwrite) throws IgniteException
create
in interface Igfs
path
- File path to create.overwrite
- Overwrite file if it already exists. Note: you cannot overwrite an existent directory.IgniteException
- In case of error.IgfsOutputStream create(IgfsPath path, int bufSize, boolean overwrite, int replication, long blockSize, @Nullable Map<String,String> props) throws IgniteException
create
in interface Igfs
path
- File path to create.bufSize
- Write buffer size (bytes) or zero
to use default value.overwrite
- Overwrite file if it already exists. Note: you cannot overwrite an existent directory.replication
- Replication factor.blockSize
- Block size.props
- File properties to set.IgniteException
- In case of error.IgfsOutputStream create(IgfsPath path, int bufSize, boolean overwrite, @Nullable IgniteUuid affKey, int replication, long blockSize, @Nullable Map<String,String> props) throws IgniteException
path
- File path to create.bufSize
- Write buffer size (bytes) or zero
to use default value.overwrite
- Overwrite file if it already exists. Note: you cannot overwrite an existent directory.affKey
- Affinity key used to store file blocks. If not null
, the whole file will be
stored on node where affKey
resides.replication
- Replication factor.blockSize
- Block size.props
- File properties to set.IgniteException
- In case of error.IgfsOutputStream append(IgfsPath path, boolean create) throws IgniteException
path
- File path to append.create
- Create file if it doesn't exist yet.IgniteException
- In case of error.IgfsFileNotFoundException
- If path doesn't exist and create flag is false
.IgfsOutputStream append(IgfsPath path, int bufSize, boolean create, @Nullable Map<String,String> props) throws IgniteException
append
in interface Igfs
path
- File path to append.bufSize
- Write buffer size (bytes) or zero
to use default value.create
- Create file if it doesn't exist yet.props
- File properties to set only in case it file was just created.IgniteException
- In case of error.IgfsFileNotFoundException
- If path doesn't exist and create flag is false
.void setTimes(IgfsPath path, long accessTime, long modificationTime) throws IgniteException
null
,
corresponding time will not be changed.path
- Path to update.accessTime
- Optional last access time to set. Value -1
does not update access time.modificationTime
- Optional last modification time to set. Value -1
does not update
modification time.IgfsFileNotFoundException
- If target was not found.IgniteException
- If error occurred.Collection<IgfsBlockLocation> affinity(IgfsPath path, long start, long len) throws IgniteException
path
- File path to get affinity for.start
- Position in the file to start affinity resolution from.len
- Size of data in the file to resolve affinity for.IgniteException
- In case of error.IgfsFileNotFoundException
- If path doesn't exist.Collection<IgfsBlockLocation> affinity(IgfsPath path, long start, long len, long maxLen) throws IgniteException
maxLen
parameter is set and
particular block location length is greater than this value, block locations will be split into smaller
chunks.path
- File path to get affinity for.start
- Position in the file to start affinity resolution from.len
- Size of data in the file to resolve affinity for.maxLen
- Maximum length of a single returned block location length.IgniteException
- In case of error.IgfsFileNotFoundException
- If path doesn't exist.IgfsMetrics metrics() throws IgniteException
IgniteException
- In case of error.void resetMetrics() throws IgniteException
IgniteException
- In case of error.long size(IgfsPath path) throws IgniteException
path
- File system path.IgniteException
- In case of error.@IgniteAsyncSupported void format() throws IgniteException
Supports asynchronous execution (see IgniteAsyncSupport
).
IgniteException
- In case format has failed.@IgniteAsyncSupported <T,R> R execute(IgfsTask<T,R> task, @Nullable IgfsRecordResolver rslvr, Collection<IgfsPath> paths, @Nullable T arg) throws IgniteException
Supports asynchronous execution (see IgniteAsyncSupport
).
task
- Task to execute.rslvr
- Optional resolver to control split boundaries.paths
- Collection of paths to be processed within this task.arg
- Optional task argument.IgniteException
- If execution failed.@IgniteAsyncSupported <T,R> R execute(IgfsTask<T,R> task, @Nullable IgfsRecordResolver rslvr, Collection<IgfsPath> paths, boolean skipNonExistentFiles, long maxRangeLen, @Nullable T arg) throws IgniteException
IgfsConfiguration.getMaximumTaskRangeLength()
for more information).
Supports asynchronous execution (see IgniteAsyncSupport
).
task
- Task to execute.rslvr
- Optional resolver to control split boundaries.paths
- Collection of paths to be processed within this task.skipNonExistentFiles
- Whether to skip non existent files. If set to true
non-existent files will
be ignored. Otherwise an exception will be thrown.maxRangeLen
- Optional maximum range length. If 0
, then by default all consecutive
IGFS blocks will be included.arg
- Optional task argument.IgniteException
- If execution failed.@IgniteAsyncSupported <T,R> R execute(Class<? extends IgfsTask<T,R>> taskCls, @Nullable IgfsRecordResolver rslvr, Collection<IgfsPath> paths, @Nullable T arg) throws IgniteException
Supports asynchronous execution (see IgniteAsyncSupport
).
taskCls
- Task class to execute.rslvr
- Optional resolver to control split boundaries.paths
- Collection of paths to be processed within this task.arg
- Optional task argument.IgniteException
- If execution failed.@IgniteAsyncSupported <T,R> R execute(Class<? extends IgfsTask<T,R>> taskCls, @Nullable IgfsRecordResolver rslvr, Collection<IgfsPath> paths, boolean skipNonExistentFiles, long maxRangeLen, @Nullable T arg) throws IgniteException
IgfsConfiguration.getMaximumTaskRangeLength()
for more information).
Supports asynchronous execution (see IgniteAsyncSupport
).
taskCls
- Task class to execute.rslvr
- Optional resolver to control split boundaries.paths
- Collection of paths to be processed within this task.skipNonExistentFiles
- Whether to skip non existent files. If set to true
non-existent files will
be ignored. Otherwise an exception will be thrown.maxRangeLen
- Maximum range length.arg
- Optional task argument.IgniteException
- If execution failed.IgniteFs withAsync()
withAsync
in interface IgniteAsyncSupport
Follow @ApacheIgnite
Apache Ignite Fabric : ver. 1.0.0-RC1 Release Date : February 16 2015