|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectorg.apache.pig.builtin.TextLoader
public class TextLoader
This load function simply creates a tuple for each line of text that has a single field that contains the line of text.
Nested Class Summary |
---|
Nested classes/interfaces inherited from interface org.apache.pig.LoadFunc |
---|
LoadFunc.RequiredField, LoadFunc.RequiredFieldList, LoadFunc.RequiredFieldResponse |
Field Summary | |
---|---|
protected PigLineRecordReader |
in
|
Constructor Summary | |
---|---|
TextLoader()
|
Method Summary | |
---|---|
void |
bindTo(String fileName,
BufferedPositionedInputStream in,
long offset,
long end)
Specifies a portion of an InputStream to read tuples. |
DataBag |
bytesToBag(byte[] b)
TextLoader does not support conversion to Bag |
Boolean |
bytesToBoolean(byte[] b)
TextLoader does not support conversion to Boolean. |
String |
bytesToCharArray(byte[] b)
Cast data from bytes to chararray value. |
Double |
bytesToDouble(byte[] b)
TextLoader does not support conversion to Double |
Float |
bytesToFloat(byte[] b)
TextLoader does not support conversion to Float |
Integer |
bytesToInteger(byte[] b)
TextLoader does not support conversion to Integer |
Long |
bytesToLong(byte[] b)
TextLoader does not support conversion to Long |
Map<String,Object> |
bytesToMap(byte[] b)
TextLoader does not support conversion to Map |
Tuple |
bytesToTuple(byte[] b)
TextLoader does not support conversion to Tuple |
Schema |
determineSchema(String fileName,
ExecType execType,
DataStorage storage)
TextLoader does not provide a schema. |
LoadFunc.RequiredFieldResponse |
fieldsToRead(LoadFunc.RequiredFieldList requiredFieldList)
TextLoader doesn't make use of this. |
Tuple |
getNext()
Retrieves the next tuple to be processed. |
byte[] |
toBytes(DataBag bag)
|
byte[] |
toBytes(Double d)
|
byte[] |
toBytes(Float f)
|
byte[] |
toBytes(Integer i)
|
byte[] |
toBytes(Long l)
|
byte[] |
toBytes(Map<String,Object> m)
|
byte[] |
toBytes(String s)
|
byte[] |
toBytes(Tuple t)
|
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
---|
protected PigLineRecordReader in
Constructor Detail |
---|
public TextLoader()
Method Detail |
---|
public void bindTo(String fileName, BufferedPositionedInputStream in, long offset, long end) throws IOException
LoadFunc
A common way of handling slices in the middle of records is to start at the given offset and, if the offset is not zero, skip to the end of the first record (which may be a partial record) before reading tuples. Reading continues until a tuple has been read that ends at an offset past the ending offset.
The load function should not do any buffering on the input stream. Buffering will cause the offsets returned by is.getPos() to be unreliable.
bindTo
in interface LoadFunc
fileName
- the name of the file to be readin
- the stream representing the file to be processed, and which can also provide its position.offset
- the offset to start reading tuples.end
- the ending offset for reading.
IOException
public Tuple getNext() throws IOException
LoadFunc
getNext
in interface LoadFunc
IOException
public Boolean bytesToBoolean(byte[] b) throws IOException
IOException
- if the value cannot be cast.public Integer bytesToInteger(byte[] b) throws IOException
bytesToInteger
in interface LoadFunc
b
- byte array to be cast.
IOException
- if the value cannot be cast.public Long bytesToLong(byte[] b) throws IOException
bytesToLong
in interface LoadFunc
b
- byte array to be cast.
IOException
- if the value cannot be cast.public Float bytesToFloat(byte[] b) throws IOException
bytesToFloat
in interface LoadFunc
b
- byte array to be cast.
IOException
- if the value cannot be cast.public Double bytesToDouble(byte[] b) throws IOException
bytesToDouble
in interface LoadFunc
b
- byte array to be cast.
IOException
- if the value cannot be cast.public String bytesToCharArray(byte[] b) throws IOException
bytesToCharArray
in interface LoadFunc
b
- byte array to be cast.
IOException
- if the value cannot be cast.public Map<String,Object> bytesToMap(byte[] b) throws IOException
bytesToMap
in interface LoadFunc
b
- byte array to be cast.
IOException
- if the value cannot be cast.public Tuple bytesToTuple(byte[] b) throws IOException
bytesToTuple
in interface LoadFunc
b
- byte array to be cast.
IOException
- if the value cannot be cast.public DataBag bytesToBag(byte[] b) throws IOException
bytesToBag
in interface LoadFunc
b
- byte array to be cast.
IOException
- if the value cannot be cast.public LoadFunc.RequiredFieldResponse fieldsToRead(LoadFunc.RequiredFieldList requiredFieldList) throws FrontendException
fieldsToRead
in interface LoadFunc
requiredFieldList
- RequiredFieldList indicating which columns will be needed.
FrontendException
public Schema determineSchema(String fileName, ExecType execType, DataStorage storage) throws IOException
determineSchema
in interface LoadFunc
fileName
- Name of the file to be read.(this will be the same as the filename
in the "load statement of the script)execType
- - execution mode of the pig script - one of ExecType.LOCAL or ExecType.MAPREDUCEstorage
- - the DataStorage object corresponding to the execType
IOException
public byte[] toBytes(DataBag bag) throws IOException
IOException
public byte[] toBytes(String s) throws IOException
IOException
public byte[] toBytes(Double d) throws IOException
IOException
public byte[] toBytes(Float f) throws IOException
IOException
public byte[] toBytes(Integer i) throws IOException
IOException
public byte[] toBytes(Long l) throws IOException
IOException
public byte[] toBytes(Map<String,Object> m) throws IOException
IOException
public byte[] toBytes(Tuple t) throws IOException
IOException
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |