org.apache.hadoop.mapred.join
Class OverrideRecordReader<K extends WritableComparable,V extends Writable>

java.lang.Object
  extended by org.apache.hadoop.mapred.join.CompositeRecordReader<K,V,V>
      extended by org.apache.hadoop.mapred.join.MultiFilterRecordReader<K,V>
          extended by org.apache.hadoop.mapred.join.OverrideRecordReader<K,V>
All Implemented Interfaces:
Configurable

public class OverrideRecordReader<K extends WritableComparable,V extends Writable>
extends MultiFilterRecordReader<K,V>

Prefer the "rightmost" data source for this key. For example, override(S1,S2,S3) will prefer values from S3 over S2, and values from S2 over S1 for all keys emitted from all sources.


Nested Class Summary
 
Nested classes/interfaces inherited from class org.apache.hadoop.mapred.join.MultiFilterRecordReader
MultiFilterRecordReader.MultiFilterDelegationIterator
 
Field Summary
 
Fields inherited from class org.apache.hadoop.mapred.join.CompositeRecordReader
jc, kids
 
Method Summary
 void accept(org.apache.hadoop.mapred.join.CompositeRecordReader.JoinCollector jc, K key)
          While key-value pairs from this RecordReader match the given key, register them with the JoinCollector provided.
protected  V emit(TupleWritable dst)
          Emit the value with the highest position in the tuple.
protected  void fillJoinCollector(K iterkey)
          Instead of filling the JoinCollector with iterators from all data sources, fill only the rightmost for this key.
 boolean hasNext()
          Returns true if the stream is not empty, but provides no guarantee that a call to next(K,V) will succeed.
 int id()
          Return the position in the collector this class occupies.
 K key()
          Return the key this RecordReader would supply on a call to next(K,V)
 void key(K key)
          Clone the key at the head of this RecordReader into the object provided.
 void skip(K key)
          Skip key-value pairs with keys less than or equal to the key provided.
 
Methods inherited from class org.apache.hadoop.mapred.join.MultiFilterRecordReader
combine, createValue, getDelegate, next
 
Methods inherited from class org.apache.hadoop.mapred.join.CompositeRecordReader
accept, add, close, compareTo, createInternalValue, createKey, getComparator, getConf, getPos, getProgress, getRecordReaderQueue, hasNext, id, key, key, setConf, skip
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 
Methods inherited from interface org.apache.hadoop.mapred.RecordReader
close, createKey, getPos, getProgress
 
Methods inherited from interface java.lang.Comparable
compareTo
 

Method Detail

emit

protected V emit(TupleWritable dst)
Emit the value with the highest position in the tuple.

Specified by:
emit in class MultiFilterRecordReader<K extends WritableComparable,V extends Writable>

fillJoinCollector

protected void fillJoinCollector(K iterkey)
                          throws IOException
Instead of filling the JoinCollector with iterators from all data sources, fill only the rightmost for this key. This not only saves space by discarding the other sources, but it also emits the number of key-value pairs in the preferred RecordReader instead of repeating that stream n times, where n is the cardinality of the cross product of the discarded streams for the given key.

Overrides:
fillJoinCollector in class CompositeRecordReader<K extends WritableComparable,V extends Writable,V extends Writable>
Throws:
IOException

id

public int id()
Return the position in the collector this class occupies.


key

public K key()
Return the key this RecordReader would supply on a call to next(K,V)


key

public void key(K key)
         throws IOException
Clone the key at the head of this RecordReader into the object provided.

Throws:
IOException

hasNext

public boolean hasNext()
Returns true if the stream is not empty, but provides no guarantee that a call to next(K,V) will succeed.


skip

public void skip(K key)
          throws IOException
Skip key-value pairs with keys less than or equal to the key provided.

Throws:
IOException

accept

public void accept(org.apache.hadoop.mapred.join.CompositeRecordReader.JoinCollector jc,
                   K key)
            throws IOException
While key-value pairs from this RecordReader match the given key, register them with the JoinCollector provided.

Throws:
IOException


Copyright © 2006 The Apache Software Foundation