org.apache.hama.bsp
Class BSPMaster

java.lang.Object
  extended by org.apache.hama.bsp.BSPMaster
All Implemented Interfaces:
org.apache.hadoop.ipc.VersionedProtocol, HamaRPCProtocolVersion, JobSubmissionProtocol, MasterProtocol

public class BSPMaster
extends Object
implements JobSubmissionProtocol, MasterProtocol

BSPMaster is responsible to control all the groom servers and to manage bsp jobs.


Nested Class Summary
static class BSPMaster.State
           
 
Field Summary
static long GROOMSERVER_EXPIRY_INTERVAL
           
protected  ConcurrentMap<GroomServerStatus,WorkerProtocol> groomServers
           
static org.apache.commons.logging.Log LOG
           
 
Fields inherited from interface org.apache.hama.ipc.HamaRPCProtocolVersion
versionID
 
Constructor Summary
BSPMaster(HamaConfiguration conf)
          Start the BSPMaster process, listen on the indicated hostname/port
 
Method Summary
 void addJobInProgressListener(org.apache.hama.bsp.JobInProgressListener listener)
          Registers a JobInProgressListener to GroomServerManager.
static BSPMaster constructMaster(Class<? extends BSPMaster> masterClass, org.apache.hadoop.conf.Configuration conf)
           
 Map<String,String> currentGroomServerPeers()
          Current GroomServer Peers.
 BSPMaster.State currentState()
           
 WorkerProtocol findGroomServer(GroomServerStatus status)
          Find WorkerProtocol with corresponded groom server status
 Collection<WorkerProtocol> findGroomServers()
          Find the collection of groom servers.
static InetSocketAddress getAddress(org.apache.hadoop.conf.Configuration conf)
           
 JobStatus[] getAllJobs()
          Get all the jobs submitted.
 ClusterStatus getClusterStatus(boolean detailed)
          Get the current status of the cluster
 String getFilesystemName()
          A BSP system always operates on a single filesystem.
 JobProfile getJobProfile(BSPJobID jobid)
          Grab a handle to a job that is already known to the BSPMaster.
 JobStatus getJobStatus(BSPJobID jobid)
          Grab a handle to a job that is already known to the BSPMaster.
 BSPJobID getNewJobId()
          This method returns new job id.
 long getProtocolVersion(String protocol, long clientVersion)
           
 String getSystemDir()
          Return system directory to which BSP store control files.
 Collection<GroomServerStatus> groomServerStatusKeySet()
          Collection of GroomServerStatus as the key set.
 JobStatus[] jobsToComplete()
          Get the jobs that are not completed and not failed
 void killJob(BSPJobID jobid)
          Kill the indicated job
 boolean killTask(TaskAttemptID taskId, boolean shouldFail)
          Kill indicated task attempt.
 void offerService()
           
 boolean register(GroomServerStatus status)
          A GroomServer registers with its status to BSPMaster when startup, which will update GroomServers cache.
 void removeJobInProgressListener(org.apache.hama.bsp.JobInProgressListener listener)
          Unregisters a JobInProgressListener to GroomServerManager.
 boolean report(Directive directive)
          A GroomServer (periodically) reports task statuses back to the BSPMaster.
 void shutdown()
           
static BSPMaster startMaster(HamaConfiguration conf)
           
static BSPMaster startMaster(HamaConfiguration conf, String identifier)
           
 JobStatus submitJob(BSPJobID jobID, String jobFile)
          Submit a Job for execution.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

LOG

public static final org.apache.commons.logging.Log LOG

GROOMSERVER_EXPIRY_INTERVAL

public static final long GROOMSERVER_EXPIRY_INTERVAL
See Also:
Constant Field Values

groomServers

protected ConcurrentMap<GroomServerStatus,WorkerProtocol> groomServers
Constructor Detail

BSPMaster

public BSPMaster(HamaConfiguration conf)
          throws IOException,
                 InterruptedException
Start the BSPMaster process, listen on the indicated hostname/port

Throws:
IOException
InterruptedException
Method Detail

register

public boolean register(GroomServerStatus status)
                 throws IOException
A GroomServer registers with its status to BSPMaster when startup, which will update GroomServers cache.

Specified by:
register in interface MasterProtocol
Parameters:
status - to be updated in cache.
Returns:
true if registering successfully; false if fail.
Throws:
IOException

report

public boolean report(Directive directive)
               throws IOException
Description copied from interface: MasterProtocol
A GroomServer (periodically) reports task statuses back to the BSPMaster.

Specified by:
report in interface MasterProtocol
Throws:
IOException

startMaster

public static BSPMaster startMaster(HamaConfiguration conf)
                             throws IOException,
                                    InterruptedException
Throws:
IOException
InterruptedException

startMaster

public static BSPMaster startMaster(HamaConfiguration conf,
                                    String identifier)
                             throws IOException,
                                    InterruptedException
Throws:
IOException
InterruptedException

getAddress

public static InetSocketAddress getAddress(org.apache.hadoop.conf.Configuration conf)

offerService

public void offerService()
                  throws InterruptedException,
                         IOException
Throws:
InterruptedException
IOException

getProtocolVersion

public long getProtocolVersion(String protocol,
                               long clientVersion)
                        throws IOException
Specified by:
getProtocolVersion in interface org.apache.hadoop.ipc.VersionedProtocol
Throws:
IOException

getNewJobId

public BSPJobID getNewJobId()
                     throws IOException
This method returns new job id. The returned job id increases sequentially.

Specified by:
getNewJobId in interface JobSubmissionProtocol
Returns:
job id
Throws:
IOException

submitJob

public JobStatus submitJob(BSPJobID jobID,
                           String jobFile)
                    throws IOException
Description copied from interface: JobSubmissionProtocol
Submit a Job for execution. Returns the latest profile for that job. The job files should be submitted in system-dir/jobName.

Specified by:
submitJob in interface JobSubmissionProtocol
Returns:
jobStatus
Throws:
IOException

getClusterStatus

public ClusterStatus getClusterStatus(boolean detailed)
Description copied from interface: JobSubmissionProtocol
Get the current status of the cluster

Specified by:
getClusterStatus in interface JobSubmissionProtocol
Parameters:
detailed - if true then report groom names as well
Returns:
summary of the state of the cluster

findGroomServer

public WorkerProtocol findGroomServer(GroomServerStatus status)
Find WorkerProtocol with corresponded groom server status

Returns:
GroomServerStatus

findGroomServers

public Collection<WorkerProtocol> findGroomServers()
Find the collection of groom servers.

Returns:
Collection of groom servers list.

groomServerStatusKeySet

public Collection<GroomServerStatus> groomServerStatusKeySet()
Collection of GroomServerStatus as the key set.

Returns:
Collection of GroomServerStatus.

addJobInProgressListener

public void addJobInProgressListener(org.apache.hama.bsp.JobInProgressListener listener)
Registers a JobInProgressListener to GroomServerManager. Therefore, adding a JobInProgress will trigger the jobAdded function.


removeJobInProgressListener

public void removeJobInProgressListener(org.apache.hama.bsp.JobInProgressListener listener)
Unregisters a JobInProgressListener to GroomServerManager. Therefore, the remove of a JobInProgress will trigger the jobRemoved action.


currentGroomServerPeers

public Map<String,String> currentGroomServerPeers()
Current GroomServer Peers.

Returns:
GroomName and PeerName(host:port) in pair.

jobsToComplete

public JobStatus[] jobsToComplete()
                           throws IOException
Description copied from interface: JobSubmissionProtocol
Get the jobs that are not completed and not failed

Specified by:
jobsToComplete in interface JobSubmissionProtocol
Returns:
array of JobStatus for the running/to-be-run jobs.
Throws:
IOException

getAllJobs

public JobStatus[] getAllJobs()
                       throws IOException
Description copied from interface: JobSubmissionProtocol
Get all the jobs submitted.

Specified by:
getAllJobs in interface JobSubmissionProtocol
Returns:
array of JobStatus for the submitted jobs
Throws:
IOException

getFilesystemName

public String getFilesystemName()
                         throws IOException
Description copied from interface: JobSubmissionProtocol
A BSP system always operates on a single filesystem. This function returns the fs name. ('local' if the localfs; 'addr:port' if dfs). The client can then copy files into the right locations prior to submitting the job.

Specified by:
getFilesystemName in interface JobSubmissionProtocol
Throws:
IOException

getSystemDir

public String getSystemDir()
Return system directory to which BSP store control files.

Specified by:
getSystemDir in interface JobSubmissionProtocol
Specified by:
getSystemDir in interface MasterProtocol
Returns:
the system directory where job-specific files are to be placed.

getJobProfile

public JobProfile getJobProfile(BSPJobID jobid)
                         throws IOException
Description copied from interface: JobSubmissionProtocol
Grab a handle to a job that is already known to the BSPMaster.

Specified by:
getJobProfile in interface JobSubmissionProtocol
Returns:
Profile of the job, or null if not found.
Throws:
IOException

getJobStatus

public JobStatus getJobStatus(BSPJobID jobid)
                       throws IOException
Description copied from interface: JobSubmissionProtocol
Grab a handle to a job that is already known to the BSPMaster.

Specified by:
getJobStatus in interface JobSubmissionProtocol
Returns:
Status of the job, or null if not found.
Throws:
IOException

killJob

public void killJob(BSPJobID jobid)
             throws IOException
Description copied from interface: JobSubmissionProtocol
Kill the indicated job

Specified by:
killJob in interface JobSubmissionProtocol
Throws:
IOException

killTask

public boolean killTask(TaskAttemptID taskId,
                        boolean shouldFail)
                 throws IOException
Description copied from interface: JobSubmissionProtocol
Kill indicated task attempt.

Specified by:
killTask in interface JobSubmissionProtocol
Parameters:
taskId - the id of the task to kill.
shouldFail - if true the task is failed and added to failed tasks list, otherwise it is just killed, w/o affecting job failure status.
Throws:
IOException

constructMaster

public static BSPMaster constructMaster(Class<? extends BSPMaster> masterClass,
                                        org.apache.hadoop.conf.Configuration conf)

shutdown

public void shutdown()

currentState

public BSPMaster.State currentState()


Copyright © 2011 The Apache Software Foundation