org.apache.pig.backend.local.executionengine
Class LocalPigLauncher
java.lang.Object
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher
org.apache.pig.backend.local.executionengine.LocalPigLauncher
public class LocalPigLauncher
- extends Launcher
Methods inherited from class org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.Launcher |
calculateProgress, computeTimeSpent, getErrorMessages, getFailedFiles, getStackTraceElement, getStats, getSucceededFiles, getTotalHadoopTimeSpent, isComplete, progressOfRunningJob, reset |
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
LocalPigLauncher
public LocalPigLauncher()
explain
public void explain(PhysicalPlan pp,
PigContext pc,
PrintStream ps,
String format,
boolean isVerbose)
throws PlanException,
VisitorException,
IOException
- Description copied from class:
Launcher
- Explain how a pig job will be executed on the underlying
infrastructure.
- Specified by:
explain
in class Launcher
- Parameters:
pp
- PhysicalPlan to explainpc
- PigContext to use for configurationps
- PrintStream to write output on.format
- Format to write inisVerbose
- Amount of information to print
- Throws:
VisitorException
IOException
PlanException
launchPig
public PigStats launchPig(PhysicalPlan php,
String grpName,
PigContext pc)
throws PlanException,
VisitorException,
IOException,
ExecException,
JobCreationException
- Description copied from class:
Launcher
- Method to launch pig for hadoop either for a cluster's
job tracker or for a local job runner. THe only difference
between the two is the job client. Depending on the pig context
the job client will be initialize to one of the two.
Launchers for other frameworks can overide these methods.
Given an input PhysicalPlan, it compiles it
to get a MapReduce Plan. The MapReduce plan which
has multiple MapReduce operators each one of which
has to be run as a map reduce job with dependency
information stored in the plan. It compiles the
MROperPlan into a JobControl object. Each Map Reduce
operator is converted into a Job and added to the JobControl
object. Each Job also has a set of dependent Jobs that
are created using the MROperPlan.
The JobControl object is obtained from the JobControlCompiler
Then a new thread is spawned that submits these jobs
while respecting the dependency information.
The parent thread monitors the submitted jobs' progress and
after it is complete, stops the JobControl thread.
- Specified by:
launchPig
in class Launcher
- Throws:
PlanException
VisitorException
IOException
ExecException
JobCreationException
Copyright © ${year} The Apache Software Foundation