org.apache.hadoop.streaming
Class MergerInputFormat
java.lang.Object
org.apache.hadoop.mapred.InputFormatBase
org.apache.hadoop.streaming.MergerInputFormat
- All Implemented Interfaces:
- InputFormat
public class MergerInputFormat
- extends InputFormatBase
Eventually will be fed TupleInputFormats.
For now will be fed primitive InputFormats.
- Author:
- Michel Tourn
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
MergerInputFormat
public MergerInputFormat()
areValidInputDirectories
public boolean[] areValidInputDirectories(FileSystem fileSys,
Path[] inputDirs)
throws IOException
- This implementation always returns true.
- Throws:
IOException
getSplits
public InputSplit[] getSplits(JobConf job,
int numSplits)
throws IOException
- Delegate to the primary InputFormat.
Force full-file splits since there's no index to sync secondaries.
(and if there was, this index may need to be created for the first time
full file at a time... )
- Specified by:
getSplits
in interface InputFormat
- Overrides:
getSplits
in class InputFormatBase
- Parameters:
job
- the job whose input files are to be splitnumSplits
- the desired number of splits
- Returns:
- the splits
- Throws:
IOException
getRecordReader
public RecordReader getRecordReader(InputSplit split,
JobConf job,
Reporter reporter)
throws IOException
- Description copied from interface:
InputFormat
- Construct a
RecordReader
for a FileSplit
.
- Specified by:
getRecordReader
in interface InputFormat
- Specified by:
getRecordReader
in class InputFormatBase
- Parameters:
split
- the InputSplit
job
- the job that this split belongs to
- Returns:
- a
RecordReader
- Throws:
IOException
Copyright © 2006 The Apache Software Foundation