org.apache.hadoop.mapred
Class TaskTracker

java.lang.Object
  extended by org.apache.hadoop.mapred.TaskTracker
All Implemented Interfaces:
Runnable

public class TaskTracker
extends Object
implements Runnable

TaskTracker is a process that starts and tracks MR Tasks in a networked environment. It contacts the JobTracker for Task assignments and reporting results.

Author:
Mike Cafarella

Nested Class Summary
static class TaskTracker.Child
          The main() for child processes.
 
Field Summary
static int FILE_NOT_FOUND
           
static long HEARTBEAT_INTERVAL
           
static Logger LOG
           
static int SUCCESS
           
static long TASKTRACKER_EXPIRY_INTERVAL
           
 
Constructor Summary
TaskTracker(Configuration conf)
          Start with the local machine name, and the default JobTracker
TaskTracker(InetSocketAddress jobTrackAddr, Configuration conf)
          Start with the local machine name, and the addr of the target JobTracker
 
Method Summary
 void close()
          Close down the TaskTracker and all its components.
 void done(String taskid)
          The task is done.
 void fsError(String message)
          A child task had a local filesystem error.
 org.apache.hadoop.mapred.MapOutputFile getFile(String mapTaskId, String reduceTaskId, IntWritable partition)
          Returns the output from the named map task destined for this partition.
 org.apache.hadoop.mapred.InterTrackerProtocol getJobClient()
          The connection to the JobTracker, used by the TaskRunner for locating remote files.
 org.apache.hadoop.mapred.Task getTask(String taskid)
          Called upon startup by the child process, to fetch Task data.
static void main(String[] argv)
          Start the TaskTracker, point toward the indicated JobTracker
 void mapOutputLost(String taskid)
          A completed map task's output has been lost.
 void ping(String taskid)
          Child checking to see if we're alive.
 void progress(String taskid, float progress, String state)
          Called periodically to report Task progress, from 0.0 to 1.0.
 void reportDiagnosticInfo(String taskid, String info)
          Called when the task dies before completion, and we want to report back diagnostic info
 void run()
          The server retry loop.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

LOG

public static final Logger LOG

HEARTBEAT_INTERVAL

public static final long HEARTBEAT_INTERVAL
See Also:
Constant Field Values

TASKTRACKER_EXPIRY_INTERVAL

public static final long TASKTRACKER_EXPIRY_INTERVAL
See Also:
Constant Field Values

SUCCESS

public static final int SUCCESS
See Also:
Constant Field Values

FILE_NOT_FOUND

public static final int FILE_NOT_FOUND
See Also:
Constant Field Values
Constructor Detail

TaskTracker

public TaskTracker(Configuration conf)
            throws IOException
Start with the local machine name, and the default JobTracker

Throws:
IOException

TaskTracker

public TaskTracker(InetSocketAddress jobTrackAddr,
                   Configuration conf)
            throws IOException
Start with the local machine name, and the addr of the target JobTracker

Throws:
IOException
Method Detail

close

public void close()
           throws IOException
Close down the TaskTracker and all its components. We must also shutdown any running tasks or threads, and cleanup disk space. A new TaskTracker within the same process space might be restarted, so everything must be clean.

Throws:
IOException

getJobClient

public org.apache.hadoop.mapred.InterTrackerProtocol getJobClient()
The connection to the JobTracker, used by the TaskRunner for locating remote files.


run

public void run()
The server retry loop. This while-loop attempts to connect to the JobTracker. It only loops when the old TaskTracker has gone bad (its state is stale somehow) and we need to reinitialize everything.

Specified by:
run in interface Runnable

getFile

public org.apache.hadoop.mapred.MapOutputFile getFile(String mapTaskId,
                                                      String reduceTaskId,
                                                      IntWritable partition)
Returns the output from the named map task destined for this partition.


getTask

public org.apache.hadoop.mapred.Task getTask(String taskid)
                                      throws IOException
Called upon startup by the child process, to fetch Task data.

Throws:
IOException

progress

public void progress(String taskid,
                     float progress,
                     String state)
              throws IOException
Called periodically to report Task progress, from 0.0 to 1.0.

Parameters:
taskid - the id of the task
progress - value between zero and one
state - description of task's current state
Throws:
IOException

reportDiagnosticInfo

public void reportDiagnosticInfo(String taskid,
                                 String info)
                          throws IOException
Called when the task dies before completion, and we want to report back diagnostic info

Parameters:
taskid - the id of the task involved
info - the text to report
Throws:
IOException

ping

public void ping(String taskid)
          throws IOException
Child checking to see if we're alive. Normally does nothing.

Throws:
IOException

done

public void done(String taskid)
          throws IOException
The task is done.

Throws:
IOException

fsError

public void fsError(String message)
             throws IOException
A child task had a local filesystem error. Exit, so that no future jobs are accepted.

Throws:
IOException

mapOutputLost

public void mapOutputLost(String taskid)
                   throws IOException
A completed map task's output has been lost.

Throws:
IOException

main

public static void main(String[] argv)
                 throws IOException
Start the TaskTracker, point toward the indicated JobTracker

Throws:
IOException


Copyright © 2006 The Apache Software Foundation