org.apache.hadoop.streaming
Class MergerInputFormat

java.lang.Object
  extended by org.apache.hadoop.mapred.InputFormatBase
      extended by org.apache.hadoop.streaming.MergerInputFormat
All Implemented Interfaces:
InputFormat

public class MergerInputFormat
extends InputFormatBase

Eventually will be fed TupleInputFormats. For now will be fed primitive InputFormats.

Author:
Michel Tourn

Field Summary
 
Fields inherited from class org.apache.hadoop.mapred.InputFormatBase
LOG
 
Constructor Summary
MergerInputFormat()
           
 
Method Summary
 boolean[] areValidInputDirectories(FileSystem fileSys, Path[] inputDirs)
          This implementation always returns true.
 RecordReader getRecordReader(InputSplit split, JobConf job, Reporter reporter)
          Construct a RecordReader for a FileSplit.
 InputSplit[] getSplits(JobConf job, int numSplits)
          Delegate to the primary InputFormat.
 
Methods inherited from class org.apache.hadoop.mapred.InputFormatBase
isSplitable, listPaths, setMinSplitSize, validateInput
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

MergerInputFormat

public MergerInputFormat()
Method Detail

areValidInputDirectories

public boolean[] areValidInputDirectories(FileSystem fileSys,
                                          Path[] inputDirs)
                                   throws IOException
This implementation always returns true.

Throws:
IOException

getSplits

public InputSplit[] getSplits(JobConf job,
                              int numSplits)
                       throws IOException
Delegate to the primary InputFormat. Force full-file splits since there's no index to sync secondaries. (and if there was, this index may need to be created for the first time full file at a time... )

Specified by:
getSplits in interface InputFormat
Overrides:
getSplits in class InputFormatBase
Parameters:
job - the job whose input files are to be split
numSplits - the desired number of splits
Returns:
the splits
Throws:
IOException

getRecordReader

public RecordReader getRecordReader(InputSplit split,
                                    JobConf job,
                                    Reporter reporter)
                             throws IOException
Description copied from interface: InputFormat
Construct a RecordReader for a FileSplit.

Specified by:
getRecordReader in interface InputFormat
Specified by:
getRecordReader in class InputFormatBase
Parameters:
split - the InputSplit
job - the job that this split belongs to
Returns:
a RecordReader
Throws:
IOException


Copyright © 2006 The Apache Software Foundation