org.apache.pig.impl.logicalLayer
Class LOLoad

java.lang.Object
  extended by org.apache.pig.impl.plan.Operator<LOVisitor>
      extended by org.apache.pig.impl.logicalLayer.LogicalOperator
          extended by org.apache.pig.impl.logicalLayer.RelationalOperator
              extended by org.apache.pig.impl.logicalLayer.LOLoad
All Implemented Interfaces:
Serializable, Cloneable, Comparable<Operator>

public class LOLoad
extends RelationalOperator

See Also:
Serialized Form

Field Summary
 
Fields inherited from class org.apache.pig.impl.logicalLayer.LogicalOperator
mAlias, mIsProjectionMapComputed, mIsSchemaComputed, mPinnedOptions, mPlan, mProjectionMap, mRequestedParallelism, mSchema, mType
 
Fields inherited from class org.apache.pig.impl.plan.Operator
mKey
 
Constructor Summary
LOLoad(LogicalPlan plan, OperatorKey key, FileSpec inputFileSpec, org.apache.hadoop.conf.Configuration conf)
           
 
Method Summary
 org.apache.hadoop.conf.Configuration getConfiguration()
           
 Schema getDeterminedSchema()
           
 Schema getEnforcedSchema()
           
 FileSpec getInputFile()
           
 LoadFunc getLoadFunc()
           
 ProjectionMap getProjectionMap()
          Produce a map describing how this operator modifies its projection.
 List<RequiredFields> getRelevantInputs(int output, int column)
          Get relevant input columns of a particular output column.
 LoadPushDown.RequiredFieldList getRequiredFieldList()
           
 List<RequiredFields> getRequiredFields()
          Get a list of fields that this operator requires.
 Schema getSchema()
          Get a copy of the schema for the output of this operator.
 String getSchemaFile()
           
 byte getType()
          Get the type of this operator.
 String name()
           
 boolean pruneColumns(List<Pair<Integer,Integer>> columns)
           
 LoadPushDown.RequiredFieldResponse pushProjection(LoadPushDown.RequiredFieldList requiredFieldList)
           
 void setAlias(String newAlias)
           
 void setEnforcedSchema(Schema enforcedSchema)
          Set this when user enforces schema
 void setInputFile(FileSpec inputFileSpec)
           
 void setSchema(Schema schema)
          Set the output schema for this operator.
 boolean supportsMultipleInputs()
          Indicates whether this operator supports multiple inputs.
 void visit(LOVisitor v)
          Visit this node with the provided visitor.
 
Methods inherited from class org.apache.pig.impl.logicalLayer.RelationalOperator
insertPlainForEachAfter, pruneColumnInPlan, regenerateProjectionMap, unsetProjectionMap
 
Methods inherited from class org.apache.pig.impl.logicalLayer.LogicalOperator
clone, forceSchema, getAlias, getOperatorKey, getPlan, getRequestedParallelism, isPinnedOption, pinOption, reconcileSchema, regenerateSchema, setCanonicalNames, setPlan, setRequestedParallelism, setSchemaComputed, setType, supportsMultipleOutputs, toString, unsetSchema
 
Methods inherited from class org.apache.pig.impl.plan.Operator
compareTo, equals, hashCode, rewire
 
Methods inherited from class java.lang.Object
finalize, getClass, notify, notifyAll, wait, wait, wait
 

Constructor Detail

LOLoad

public LOLoad(LogicalPlan plan,
              OperatorKey key,
              FileSpec inputFileSpec,
              org.apache.hadoop.conf.Configuration conf)
       throws IOException
Parameters:
plan - LogicalPlan this operator is a part of.
key - OperatorKey for this operator
inputFileSpec - the file to be loaded *
conf - the read-only configuration object
Throws:
IOException
Method Detail

getInputFile

public FileSpec getInputFile()

setInputFile

public void setInputFile(FileSpec inputFileSpec)
                  throws IOException
Throws:
IOException

getSchemaFile

public String getSchemaFile()

getLoadFunc

public LoadFunc getLoadFunc()

name

public String name()
Specified by:
name in class Operator<LOVisitor>

getSchema

public Schema getSchema()
                 throws FrontendException
Description copied from class: LogicalOperator
Get a copy of the schema for the output of this operator.

Specified by:
getSchema in class LogicalOperator
Throws:
FrontendException

setSchema

public void setSchema(Schema schema)
               throws FrontendException
Description copied from class: LogicalOperator
Set the output schema for this operator. If a schema already exists, an attempt will be made to reconcile it with this new schema.

Overrides:
setSchema in class LogicalOperator
Parameters:
schema - Schema to set.
Throws:
FrontendException

supportsMultipleInputs

public boolean supportsMultipleInputs()
Description copied from class: Operator
Indicates whether this operator supports multiple inputs.

Specified by:
supportsMultipleInputs in class Operator<LOVisitor>
Returns:
true if it does, otherwise false.

visit

public void visit(LOVisitor v)
           throws VisitorException
Description copied from class: LogicalOperator
Visit this node with the provided visitor. This should only be called by the visitor class itself, never directly.

Specified by:
visit in class LogicalOperator
Parameters:
v - Visitor to visit with.
Throws:
VisitorException - if the visitor has a problem.

getEnforcedSchema

public Schema getEnforcedSchema()

setEnforcedSchema

public void setEnforcedSchema(Schema enforcedSchema)
Set this when user enforces schema

Parameters:
enforcedSchema -

getType

public byte getType()
Description copied from class: LogicalOperator
Get the type of this operator.

Overrides:
getType in class LogicalOperator

getDeterminedSchema

public Schema getDeterminedSchema()
Returns:
the DeterminedSchema

getProjectionMap

public ProjectionMap getProjectionMap()
Description copied from class: RelationalOperator
Produce a map describing how this operator modifies its projection.

Overrides:
getProjectionMap in class RelationalOperator
Returns:
ProjectionMap null indicates it does not know how the projection changes, for example a join of two inputs where one input does not have a schema.

getRequiredFields

public List<RequiredFields> getRequiredFields()
Description copied from class: RelationalOperator
Get a list of fields that this operator requires. This is not necessarily equivalent to the list of fields the operator projects. For example, a filter will project anything passed to it, but requires only the fields explicitly referenced in its filter expression.

Overrides:
getRequiredFields in class RelationalOperator
Returns:
list of RequiredFields null indicates that the operator does not need any fields from its input.

getRelevantInputs

public List<RequiredFields> getRelevantInputs(int output,
                                              int column)
                                       throws FrontendException
Description copied from class: RelationalOperator
Get relevant input columns of a particular output column. The resulting input columns are necessary components only to the output column. Input columns needed by the entire RelationalOperator thus indirectly contribute to the output columns are not counted. Those are required columns. eg1: A = load 'a' AS (a0, a1, a2); B = filter a by a0=='1'; Relevant input columns for B.$1 is A.a1 because A.a1 direct generate B.$1. A.a0 is needed by the filter operator and it is considered as required fields for the relational operator. eg2: A = load 'a' AS (a0, a1); B = load 'b' AS (b0, b1); C = join A by a0, B by b0; Relevant input columns for C.$0 is A.a0. Relevant input columns for C.$1 is A.a1. eg3: A = load 'a' AS (a0, a1); B = load 'b' AS (b0, b1); C = cogroup A by a0, B by b0; Relevant input columns for C.$0 is A.a0, B.b0. Relevant input columns for C.$1 is A.*. Relevant input columns for C.$2 is B.*. eg4: A = load 'a' AS (a0, a1, a2); B = foreach A generate a1, a0+a2; Relevant input columns for B.$0 is A.a1. Relevant input columns for B.$1 is A.a0 and A.a2. eg5: A = load 'a' AS (a0, a1, a2); B = foreach A generate a1, *; Relevant input columns for B.$0 is A.a1. Relevant input columns for B.$1 is A.a0. Relevant input columns for B.$2 is A.a1. Relevant input columns for B.$3 is A.a2.

Specified by:
getRelevantInputs in class RelationalOperator
Parameters:
output - output index. Only LOSplit have output other than 0 currently
column - output column
Returns:
List of relevant input columns. null if Pig cannot determine relevant inputs or any error occurs
Throws:
FrontendException

pushProjection

public LoadPushDown.RequiredFieldResponse pushProjection(LoadPushDown.RequiredFieldList requiredFieldList)
                                                  throws FrontendException
Throws:
FrontendException

setAlias

public void setAlias(String newAlias)
Overrides:
setAlias in class LogicalOperator

pruneColumns

public boolean pruneColumns(List<Pair<Integer,Integer>> columns)
                     throws FrontendException
Overrides:
pruneColumns in class RelationalOperator
Throws:
FrontendException

getRequiredFieldList

public LoadPushDown.RequiredFieldList getRequiredFieldList()

getConfiguration

public org.apache.hadoop.conf.Configuration getConfiguration()


Copyright © ${year} The Apache Software Foundation