org.apache.hadoop.hive.ql.exec
Class ExecDriver

java.lang.Object
  extended by org.apache.hadoop.hive.ql.exec.Task<mapredWork>
      extended by org.apache.hadoop.hive.ql.exec.ExecDriver
All Implemented Interfaces:
Serializable

public class ExecDriver
extends Task<mapredWork>
implements Serializable

See Also:
Serialized Form

Field Summary
protected  org.apache.hadoop.mapred.JobConf job
           
static HashMap<String,String> runningJobKillURIs
          A list of the currently running jobs spawned in this Hive instance that is used to kill all running jobs in the event of an unexpected shutdown - i.e., the JVM shuts down while there are still jobs running.
 
Fields inherited from class org.apache.hadoop.hive.ql.exec.Task
childTasks, conf, console, db, id, LOG, parentTasks, work
 
Constructor Summary
ExecDriver()
          Constructor when invoked from QL
ExecDriver(mapredWork plan, org.apache.hadoop.mapred.JobConf job, boolean isSilent)
          Constructor/Initialization for invocation as independent utility
 
Method Summary
 int estimateNumberOfReducers(HiveConf hive, org.apache.hadoop.mapred.JobConf job, mapredWork work)
          Estimate the number of reducers needed for this job, based on job input, and configuration parameters.
 int execute()
          Execute a query plan using Hadoop
static String generateCmdLine(HiveConf hconf)
          Given a Hive Configuration object - generate a command line fragment for passing such configuration information to ExecDriver
static String getResourceFiles(org.apache.hadoop.conf.Configuration conf, SessionState.ResourceType t)
           
 long getTotalInputFileSize(org.apache.hadoop.mapred.JobConf job, mapredWork work)
          Calculate the total size of input files.
 boolean hasReduce()
           
 void initialize(HiveConf conf)
          Initialization when invoked from QL
 boolean isMapRedTask()
           
 void jobInfo(org.apache.hadoop.mapred.RunningJob rj)
          from StreamJob.java
 org.apache.hadoop.mapred.RunningJob jobProgress(org.apache.hadoop.mapred.JobClient jc, org.apache.hadoop.mapred.RunningJob rj)
          from StreamJob.java
static void main(String[] args)
           
protected  void setNumberOfReducers()
          Set the number of reducers for the mapred work.
 
Methods inherited from class org.apache.hadoop.hive.ql.exec.Task
addDependentTask, done, fetch, getChildTasks, getId, getParentTasks, getWork, isRunnable, removeDependentTask, setChildTasks, setDone, setId, setParentTasks, setWork
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

job

protected transient org.apache.hadoop.mapred.JobConf job

runningJobKillURIs

public static HashMap<String,String> runningJobKillURIs
A list of the currently running jobs spawned in this Hive instance that is used to kill all running jobs in the event of an unexpected shutdown - i.e., the JVM shuts down while there are still jobs running.

Constructor Detail

ExecDriver

public ExecDriver()
Constructor when invoked from QL


ExecDriver

public ExecDriver(mapredWork plan,
                  org.apache.hadoop.mapred.JobConf job,
                  boolean isSilent)
           throws HiveException
Constructor/Initialization for invocation as independent utility

Throws:
HiveException
Method Detail

getResourceFiles

public static String getResourceFiles(org.apache.hadoop.conf.Configuration conf,
                                      SessionState.ResourceType t)

initialize

public void initialize(HiveConf conf)
Initialization when invoked from QL

Overrides:
initialize in class Task<mapredWork>

jobInfo

public void jobInfo(org.apache.hadoop.mapred.RunningJob rj)
from StreamJob.java


jobProgress

public org.apache.hadoop.mapred.RunningJob jobProgress(org.apache.hadoop.mapred.JobClient jc,
                                                       org.apache.hadoop.mapred.RunningJob rj)
                                                throws IOException
from StreamJob.java

Throws:
IOException

estimateNumberOfReducers

public int estimateNumberOfReducers(HiveConf hive,
                                    org.apache.hadoop.mapred.JobConf job,
                                    mapredWork work)
                             throws IOException
Estimate the number of reducers needed for this job, based on job input, and configuration parameters.

Returns:
the number of reducers.
Throws:
IOException

setNumberOfReducers

protected void setNumberOfReducers()
                            throws IOException
Set the number of reducers for the mapred work.

Throws:
IOException

getTotalInputFileSize

public long getTotalInputFileSize(org.apache.hadoop.mapred.JobConf job,
                                  mapredWork work)
                           throws IOException
Calculate the total size of input files.

Parameters:
job - the hadoop job conf.
Returns:
the total size in bytes.
Throws:
IOException

execute

public int execute()
Execute a query plan using Hadoop

Specified by:
execute in class Task<mapredWork>

main

public static void main(String[] args)
                 throws IOException,
                        HiveException
Throws:
IOException
HiveException

generateCmdLine

public static String generateCmdLine(HiveConf hconf)
Given a Hive Configuration object - generate a command line fragment for passing such configuration information to ExecDriver


isMapRedTask

public boolean isMapRedTask()
Overrides:
isMapRedTask in class Task<mapredWork>

hasReduce

public boolean hasReduce()
Overrides:
hasReduce in class Task<mapredWork>


Copyright © 2009 The Apache Software Foundation