public interface IgniteDataLoader<K,V> extends AutoCloseable
Note that loader will load data concurrently by multiple internal threads, so the data may get to remote nodes in different order from which it was added to the loader.
Also note that GridDataLoader
is not the only way to load data into cache.
Alternatively you can use IgniteCache.loadCache(IgniteBiPredicate, Object...)
method to load data from underlying data store. You can also use standard
cache put(...)
and putAll(...)
operations as well, but they most
likely will not perform as well as this class for loading data. And finally,
data can be loaded from underlying data store on demand, whenever it is accessed -
for this no explicit data loading step is needed.
IgniteDataLoader
supports the following configuration properties:
perNodeBufferSize(int)
- when entries are added to data loader via
addData(Object, Object)
method, they are not sent to in-memory data grid right
away and are buffered internally for better performance and network utilization.
This setting controls the size of internal per-node buffer before buffered data
is sent to remote node. Default is defined by DFLT_PER_NODE_BUFFER_SIZE
value.
perNodeParallelLoadOperations(int)
- sometimes data may be added
to the data loader via addData(Object, Object)
method faster than it can
be put in cache. In this case, new buffered load messages are sent to remote nodes
before responses from previous ones are received. This could cause unlimited heap
memory utilization growth on local and remote nodes. To control memory utilization,
this setting limits maximum allowed number of parallel buffered load messages that
are being processed on remote nodes. If this number is exceeded, then
addData(Object, Object)
method will block to control memory utilization.
Default is defined by DFLT_MAX_PARALLEL_OPS
value.
autoFlushFrequency(long)
- automatic flush frequency in milliseconds. Essentially,
this is the time after which the loader will make an attempt to submit all data
added so far to remote nodes. Note that there is no guarantee that data will be
delivered after this concrete attempt (e.g., it can fail when topology is
changing), but it won't be lost anyway. Disabled by default (default value is 0
).
allowOverwrite(boolean)
- defines if data loader will assume that there are no other concurrent
updates and allow data loader choose most optimal concurrent implementation.
updater(IgniteDataLoader.Updater)
- defines how cache will be updated with loaded entries.
It allows to provide user-defined custom logic to update the cache in the most effective and flexible way.
deployClass(Class)
- optional deploy class for peer deployment. All classes
loaded by a data loader must be class-loadable from the same class-loader.
Ignite will make the best effort to detect the most suitable class-loader
for data loading. However, in complex cases, where compound or deeply nested
class-loaders are used, it is best to specify a deploy class which can be any
class loaded by the class-loader for given data.
Modifier and Type | Interface and Description |
---|---|
static interface |
IgniteDataLoader.Updater<K,V>
Updates cache with batch of entries.
|
Modifier and Type | Field and Description |
---|---|
static int |
DFLT_MAX_PARALLEL_OPS
Default max concurrent put operations count.
|
static int |
DFLT_PER_NODE_BUFFER_SIZE
Default per node buffer size.
|
Modifier and Type | Method and Description |
---|---|
IgniteFuture<?> |
addData(Collection<? extends Map.Entry<K,V>> entries)
Adds data for loading on remote node.
|
IgniteFuture<?> |
addData(K key,
V val)
Adds data for loading on remote node.
|
IgniteFuture<?> |
addData(Map.Entry<K,V> entry)
Adds data for loading on remote node.
|
IgniteFuture<?> |
addData(Map<K,V> entries)
Adds data for loading on remote node.
|
boolean |
allowOverwrite()
Gets flag value indicating that this data loader assumes that there are no other concurrent updates to the cache.
|
void |
allowOverwrite(boolean allowOverwrite)
Sets flag indicating that this data loader should assume that there are no other concurrent updates to the cache.
|
long |
autoFlushFrequency()
Gets automatic flush frequency.
|
void |
autoFlushFrequency(long autoFlushFreq)
Sets automatic flush frequency.
|
String |
cacheName()
Name of cache to load data to.
|
void |
close()
Closes data loader.
|
void |
close(boolean cancel)
Loads any remaining data and closes this loader.
|
void |
deployClass(Class<?> depCls)
Optional deploy class for peer deployment.
|
void |
flush()
Loads any remaining data, but doesn't close the loader.
|
IgniteFuture<?> |
future()
Gets future for this loading process.
|
int |
perNodeBufferSize()
Gets size of per node key-value pairs buffer.
|
void |
perNodeBufferSize(int bufSize)
Sets size of per node key-value pairs buffer.
|
int |
perNodeParallelLoadOperations()
Gets maximum number of parallel load operations for a single node.
|
void |
perNodeParallelLoadOperations(int parallelOps)
Sets maximum number of parallel load operations for a single node.
|
IgniteFuture<?> |
removeData(K key)
Adds key for removal on remote node.
|
boolean |
skipStore()
Gets flag indicating that write-through behavior should be disabled for data loading.
|
void |
skipStore(boolean skipStore)
Sets flag indicating that write-through behavior should be disabled for data loading.
|
void |
tryFlush()
Makes an attempt to load remaining data.
|
void |
updater(IgniteDataLoader.Updater<K,V> updater)
Sets custom cache updater to this data loader.
|
static final int DFLT_MAX_PARALLEL_OPS
static final int DFLT_PER_NODE_BUFFER_SIZE
String cacheName()
null
for default cache.boolean allowOverwrite()
true
.void allowOverwrite(boolean allowOverwrite) throws IgniteException
updater(IgniteDataLoader.Updater)
method.
Default is true
. When this flag is set, updates will not be propagated to the cache store.allowOverwrite
- Flag value.IgniteException
- If failed.boolean skipStore()
false
.void skipStore(boolean skipStore)
false
.skipStore
- Skip store flag.int perNodeBufferSize()
void perNodeBufferSize(int bufSize)
This method should be called prior to addData(Object, Object)
call.
If not provided, default value is DFLT_PER_NODE_BUFFER_SIZE
.
bufSize
- Per node buffer size.int perNodeParallelLoadOperations()
void perNodeParallelLoadOperations(int parallelOps)
This method should be called prior to addData(Object, Object)
call.
If not provided, default value is DFLT_MAX_PARALLEL_OPS
.
parallelOps
- Maximum number of parallel load operations for a single node.long autoFlushFrequency()
If set to 0
, automatic flush is disabled.
Automatic flush is disabled by default (default value is 0
).
0
if automatic flush is disabled.flush()
void autoFlushFrequency(long autoFlushFreq)
If set to 0
, automatic flush is disabled.
Automatic flush is disabled by default (default value is 0
).
autoFlushFreq
- Flush frequency or 0
to disable automatic flush.flush()
IgniteFuture<?> future()
close(boolean)
completes. By attaching listeners to this future
it is possible to get asynchronous notifications for completion of this
loading process.void deployClass(Class<?> depCls)
depCls
- Any class loaded by the class-loader for given data.void updater(IgniteDataLoader.Updater<K,V> updater)
updater
- Cache updater.IgniteFuture<?> removeData(K key) throws IgniteException, IgniteInterruptedException, IllegalStateException
addData(key, null)
.key
- Key.IgniteException
- If failed to map key to node.IgniteInterruptedException
- If thread has been interrupted.IllegalStateException
- If grid has been concurrently stopped or
close(boolean)
has already been called on loader.IgniteFuture<?> addData(K key, @Nullable V val) throws IgniteException, IgniteInterruptedException, IllegalStateException
Note that loader will load data concurrently by multiple internal threads, so the data may get to remote nodes in different order from which it was added to the loader.
key
- Key.val
- Value or null
if respective entry must be removed from cache.IgniteException
- If failed to map key to node.IgniteInterruptedException
- If thread has been interrupted.IllegalStateException
- If grid has been concurrently stopped or
close(boolean)
has already been called on loader.IgniteFuture<?> addData(Map.Entry<K,V> entry) throws IgniteException, IgniteInterruptedException, IllegalStateException
Note that loader will load data concurrently by multiple internal threads, so the data may get to remote nodes in different order from which it was added to the loader.
entry
- Entry.IgniteException
- If failed to map key to node.IgniteInterruptedException
- If thread has been interrupted.IllegalStateException
- If grid has been concurrently stopped or
close(boolean)
has already been called on loader.IgniteFuture<?> addData(Collection<? extends Map.Entry<K,V>> entries) throws IllegalStateException
Note that loader will load data concurrently by multiple internal threads, so the data may get to remote nodes in different order from which it was added to the loader.
entries
- Collection of entries to be loaded.IllegalStateException
- If grid has been concurrently stopped or
close(boolean)
has already been called on loader.IgniteFuture<?> addData(Map<K,V> entries) throws IllegalStateException
Note that loader will load data concurrently by multiple internal threads, so the data may get to remote nodes in different order from which it was added to the loader.
entries
- Map to be loaded.IllegalStateException
- If grid has been concurrently stopped or
close(boolean)
has already been called on loader.void flush() throws IgniteException, IgniteInterruptedException, IllegalStateException
If another thread is already performing flush, this method will block, wait for
another thread to complete flush and exit. If you don't want to wait in this case,
use tryFlush()
method.
IgniteException
- If failed to map key to node.IgniteInterruptedException
- If thread has been interrupted.IllegalStateException
- If grid has been concurrently stopped or
close(boolean)
has already been called on loader.tryFlush()
void tryFlush() throws IgniteException, IgniteInterruptedException, IllegalStateException
flush()
,
with the difference that it won't wait and will exit immediately.IgniteException
- If failed to map key to node.IgniteInterruptedException
- If thread has been interrupted.IllegalStateException
- If grid has been concurrently stopped or
close(boolean)
has already been called on loader.flush()
void close(boolean cancel) throws IgniteException, IgniteInterruptedException
cancel
- True
to cancel ongoing loading operations.IgniteException
- If failed to map key to node.IgniteInterruptedException
- If thread has been interrupted.void close() throws IgniteException, IgniteInterruptedException
close(false)
method.
The method is invoked automatically on objects managed by the
try-with-resources
statement.
close
in interface AutoCloseable
IgniteException
- If failed to close data loader.IgniteInterruptedException
- If thread has been interrupted.
Follow @ApacheIgnite
Apache Ignite Fabric : ver. 1.0.0-RC1 Release Date : February 16 2015