org.apache.accumulo.examples.wikisearch.iterator
Class ReadAheadIterator

java.lang.Object
  extended by org.apache.accumulo.examples.wikisearch.iterator.ReadAheadIterator
All Implemented Interfaces:
OptionDescriber, SortedKeyValueIterator<Key,Value>

public class ReadAheadIterator
extends Object
implements SortedKeyValueIterator<Key,Value>, OptionDescriber

This iterator takes the source iterator (the one below it in the iterator stack) and puts it in a background thread. The background thread continues processing and fills a queue with the Keys and Values from the source iterator. When seek() is called on this iterator, it pauses the background thread, clears the queue, calls seek() on the source iterator, then resumes the thread filling the queue. Users of this iterator can set the queue size, default is five elements. Users must be aware of the potential for OutOfMemory errors when using this iterator with large queue sizes or large objects. This iterator copies the Key and Value from the source iterator and puts them into the queue. This iterator introduces some parallelism into the server side iterator stack. One use case for this would be when an iterator takes a relatively long time to process each K,V pair and causes the iterators above it to wait. By putting the longer running iterator in a background thread we should be able to achieve greater throughput. NOTE: Experimental!


Nested Class Summary
 
Nested classes/interfaces inherited from interface org.apache.accumulo.core.iterators.OptionDescriber
OptionDescriber.IteratorOptions
 
Field Summary
static String QUEUE_SIZE
           
static String TIMEOUT
           
 
Constructor Summary
  ReadAheadIterator()
           
protected ReadAheadIterator(ReadAheadIterator other, IteratorEnvironment env)
           
 
Method Summary
 SortedKeyValueIterator<Key,Value> deepCopy(IteratorEnvironment env)
          Creates a deep copy of this iterator as though seek had not yet been called.
 OptionDescriber.IteratorOptions describeOptions()
          Gets an iterator options object that contains information needed to configure this iterator.
 Key getTopKey()
          Returns top key.
 Value getTopValue()
          Returns top value.
 boolean hasTop()
          Returns true if the iterator has more elements.
 void init(SortedKeyValueIterator<Key,Value> source, Map<String,String> options, IteratorEnvironment env)
          Initializes the iterator.
 void next()
          Populate the key and value
 void seek(Range range, Collection<ByteSequence> columnFamilies, boolean inclusive)
          Seek to the next matching cell and call next to populate the key and value.
 boolean validateOptions(Map<String,String> options)
          Check to see if an options map contains all options required by an iterator and that the option values are in the expected formats.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

QUEUE_SIZE

public static final String QUEUE_SIZE
See Also:
Constant Field Values

TIMEOUT

public static final String TIMEOUT
See Also:
Constant Field Values
Constructor Detail

ReadAheadIterator

protected ReadAheadIterator(ReadAheadIterator other,
                            IteratorEnvironment env)

ReadAheadIterator

public ReadAheadIterator()
Method Detail

deepCopy

public SortedKeyValueIterator<Key,Value> deepCopy(IteratorEnvironment env)
Description copied from interface: SortedKeyValueIterator
Creates a deep copy of this iterator as though seek had not yet been called. init should be called on an iterator before deepCopy is called. init should not need to be called on the copy that is returned by deepCopy; that is, when necessary init should be called in the deepCopy method on the iterator it returns. The behavior is unspecified if init is called after deepCopy either on the original or the copy.

Specified by:
deepCopy in interface SortedKeyValueIterator<Key,Value>
Parameters:
env - IteratorEnvironment environment in which iterator is being run.
Returns:
SortedKeyValueIterator a copy of this iterator (with the same source and settings).

getTopKey

public Key getTopKey()
Description copied from interface: SortedKeyValueIterator
Returns top key. Can be called 0 or more times without affecting behavior of next() or hasTop(). Note that in minor compaction scope and in non-full major compaction scopes the iterator may see deletion entries. These entries should be preserved by all iterators except ones that are strictly scan-time iterators that will never be configured for the minc or majc scopes. Deletion entries are only removed during full major compactions.

Specified by:
getTopKey in interface SortedKeyValueIterator<Key,Value>
Returns:
K

getTopValue

public Value getTopValue()
Description copied from interface: SortedKeyValueIterator
Returns top value. Can be called 0 or more times without affecting behavior of next() or hasTop().

Specified by:
getTopValue in interface SortedKeyValueIterator<Key,Value>
Returns:
V

hasTop

public boolean hasTop()
Description copied from interface: SortedKeyValueIterator
Returns true if the iterator has more elements.

Specified by:
hasTop in interface SortedKeyValueIterator<Key,Value>
Returns:
true if the iterator has more elements.

init

public void init(SortedKeyValueIterator<Key,Value> source,
                 Map<String,String> options,
                 IteratorEnvironment env)
          throws IOException
Description copied from interface: SortedKeyValueIterator
Initializes the iterator. Data should not be read from the source in this method.

Specified by:
init in interface SortedKeyValueIterator<Key,Value>
Parameters:
source - SortedKeyValueIterator source to read data from.
options - Map map of string option names to option values.
env - IteratorEnvironment environment in which iterator is being run.
Throws:
IOException - unused.

next

public void next()
          throws IOException
Populate the key and value

Specified by:
next in interface SortedKeyValueIterator<Key,Value>
Throws:
IOException - if an I/O error occurs.

seek

public void seek(Range range,
                 Collection<ByteSequence> columnFamilies,
                 boolean inclusive)
          throws IOException
Seek to the next matching cell and call next to populate the key and value.

Specified by:
seek in interface SortedKeyValueIterator<Key,Value>
Parameters:
range - Range of keys to iterate over.
columnFamilies - Collection of column families to include or exclude.
inclusive - boolean that indicates whether to include (true) or exclude (false) column families.
Throws:
IOException - if an I/O error occurs.

describeOptions

public OptionDescriber.IteratorOptions describeOptions()
Description copied from interface: OptionDescriber
Gets an iterator options object that contains information needed to configure this iterator. This object will be used by the accumulo shell to prompt the user to input the appropriate information.

Specified by:
describeOptions in interface OptionDescriber
Returns:
an iterator options object

validateOptions

public boolean validateOptions(Map<String,String> options)
Description copied from interface: OptionDescriber
Check to see if an options map contains all options required by an iterator and that the option values are in the expected formats.

Specified by:
validateOptions in interface OptionDescriber
Parameters:
options - a map of option names to option values
Returns:
true if options are valid, false otherwise


Copyright © 2012 The Apache Software Foundation. All Rights Reserved.