public abstract class SparkMRQLFileInputFormat extends org.apache.hadoop.mapred.FileInputFormat<MRContainer,MRContainer> implements MRQLFileInputFormat
Constructor and Description |
---|
SparkMRQLFileInputFormat() |
Modifier and Type | Method and Description |
---|---|
Bag |
collect(DataSet x,
boolean strip)
materialize the entire dataset into a Bag
|
abstract org.apache.hadoop.mapred.RecordReader<MRContainer,MRContainer> |
getRecordReader(org.apache.hadoop.mapred.InputSplit split,
org.apache.hadoop.mapred.JobConf job,
org.apache.hadoop.mapred.Reporter reporter)
record reader for spark
|
Bag |
materialize(org.apache.hadoop.fs.Path file)
materialize the input file into a memory Bag
|
public abstract org.apache.hadoop.mapred.RecordReader<MRContainer,MRContainer> getRecordReader(org.apache.hadoop.mapred.InputSplit split, org.apache.hadoop.mapred.JobConf job, org.apache.hadoop.mapred.Reporter reporter) throws IOException
getRecordReader
in interface org.apache.hadoop.mapred.InputFormat<MRContainer,MRContainer>
getRecordReader
in class org.apache.hadoop.mapred.FileInputFormat<MRContainer,MRContainer>
IOException
public Bag materialize(org.apache.hadoop.fs.Path file) throws IOException
materialize
in interface MRQLFileInputFormat
IOException
public final Bag collect(DataSet x, boolean strip) throws Exception
collect
in interface MRQLFileInputFormat
x
- the DataSet in HDFS to collect values fromstrip
- is not used in MapReduce modeException
Copyright © 2013-2016 The Apache Software Foundation. All Rights Reserved.