org.apache.hama.bsp
Class BSPMaster

java.lang.Object
  extended by org.apache.hama.bsp.BSPMaster
All Implemented Interfaces:
org.apache.hadoop.ipc.VersionedProtocol, HamaRPCProtocolVersion, JobSubmissionProtocol, MasterProtocol

public class BSPMaster
extends java.lang.Object
implements JobSubmissionProtocol, MasterProtocol

BSPMaster is responsible to control all the groom servers and to manage bsp jobs.


Nested Class Summary
static class BSPMaster.State
          Constants for BSPMaster's status.
 
Field Summary
static long GROOMSERVER_EXPIRY_INTERVAL
           
protected  java.util.concurrent.ConcurrentMap<GroomServerStatus,WorkerProtocol> groomServers
           
static org.apache.commons.logging.Log LOG
           
 
Fields inherited from interface org.apache.hama.ipc.HamaRPCProtocolVersion
versionID
 
Constructor Summary
BSPMaster(HamaConfiguration conf)
          Start the BSPMaster process, listen on the indicated hostname/port
 
Method Summary
 void addJobInProgressListener(org.apache.hama.bsp.JobInProgressListener listener)
          Registers a JobInProgressListener to GroomServerManager.
static BSPMaster constructMaster(java.lang.Class<? extends BSPMaster> masterClass, org.apache.hadoop.conf.Configuration conf)
           
 java.util.Map<java.lang.String,java.lang.String> currentGroomServerPeers()
          Current GroomServer Peers.
 BSPMaster.State currentState()
           
 WorkerProtocol findGroomServer(GroomServerStatus status)
          Find WorkerProtocol with corresponded groom server status
 java.util.Collection<WorkerProtocol> findGroomServers()
          Find the collection of groom servers.
static java.net.InetSocketAddress getAddress(org.apache.hadoop.conf.Configuration conf)
           
 JobStatus[] getAllJobs()
          Get all the jobs submitted.
 java.lang.String getBSPMasterIdentifier()
           
 java.lang.String getBSPMasterName()
           
 ClusterStatus getClusterStatus(boolean detailed)
          Get the current status of the cluster
 java.lang.String getFilesystemName()
          A BSP system always operates on a single filesystem.
 int getHttpPort()
           
 JobProfile getJobProfile(BSPJobID jobid)
          Grab a handle to a job that is already known to the BSPMaster.
 JobStatus getJobStatus(BSPJobID jobid)
          Grab a handle to a job that is already known to the BSPMaster.
 BSPJobID getNewJobId()
          This method returns new job id.
 long getProtocolVersion(java.lang.String protocol, long clientVersion)
           
 long getStartTime()
           
 java.lang.String getSystemDir()
          Return system directory to which BSP store control files.
 java.util.Collection<GroomServerStatus> groomServerStatusKeySet()
          Collection of GroomServerStatus as the key set.
 JobStatus[] jobsToComplete()
          Get the jobs that are not completed and not failed
 void killJob(BSPJobID jobid)
          Kill the indicated job
 boolean killTask(TaskAttemptID taskId, boolean shouldFail)
          Kill indicated task attempt.
 void offerService()
           
 boolean register(GroomServerStatus status)
          A GroomServer registers with its status to BSPMaster when startup, which will update GroomServers cache.
 void removeJobInProgressListener(org.apache.hama.bsp.JobInProgressListener listener)
          Unregisters a JobInProgressListener to GroomServerManager.
 boolean report(Directive directive)
          A GroomServer (periodically) reports task statuses back to the BSPMaster.
 void shutdown()
           
static BSPMaster startMaster(HamaConfiguration conf)
           
static BSPMaster startMaster(HamaConfiguration conf, java.lang.String identifier)
           
 JobStatus submitJob(BSPJobID jobID, java.lang.String jobFile)
          Submit a Job for execution.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

LOG

public static final org.apache.commons.logging.Log LOG

GROOMSERVER_EXPIRY_INTERVAL

public static final long GROOMSERVER_EXPIRY_INTERVAL
See Also:
Constant Field Values

groomServers

protected java.util.concurrent.ConcurrentMap<GroomServerStatus,WorkerProtocol> groomServers
Constructor Detail

BSPMaster

public BSPMaster(HamaConfiguration conf)
          throws java.io.IOException,
                 java.lang.InterruptedException
Start the BSPMaster process, listen on the indicated hostname/port

Throws:
java.io.IOException
java.lang.InterruptedException
Method Detail

register

public boolean register(GroomServerStatus status)
                 throws java.io.IOException
A GroomServer registers with its status to BSPMaster when startup, which will update GroomServers cache.

Specified by:
register in interface MasterProtocol
Parameters:
status - to be updated in cache.
Returns:
true if registering successfully; false if fail.
Throws:
java.io.IOException

report

public boolean report(Directive directive)
               throws java.io.IOException
Description copied from interface: MasterProtocol
A GroomServer (periodically) reports task statuses back to the BSPMaster.

Specified by:
report in interface MasterProtocol
Throws:
java.io.IOException

startMaster

public static BSPMaster startMaster(HamaConfiguration conf)
                             throws java.io.IOException,
                                    java.lang.InterruptedException
Throws:
java.io.IOException
java.lang.InterruptedException

startMaster

public static BSPMaster startMaster(HamaConfiguration conf,
                                    java.lang.String identifier)
                             throws java.io.IOException,
                                    java.lang.InterruptedException
Throws:
java.io.IOException
java.lang.InterruptedException

getAddress

public static java.net.InetSocketAddress getAddress(org.apache.hadoop.conf.Configuration conf)

offerService

public void offerService()
                  throws java.lang.InterruptedException,
                         java.io.IOException
Throws:
java.lang.InterruptedException
java.io.IOException

getProtocolVersion

public long getProtocolVersion(java.lang.String protocol,
                               long clientVersion)
                        throws java.io.IOException
Specified by:
getProtocolVersion in interface org.apache.hadoop.ipc.VersionedProtocol
Throws:
java.io.IOException

getNewJobId

public BSPJobID getNewJobId()
                     throws java.io.IOException
This method returns new job id. The returned job id increases sequentially.

Specified by:
getNewJobId in interface JobSubmissionProtocol
Returns:
job id
Throws:
java.io.IOException

submitJob

public JobStatus submitJob(BSPJobID jobID,
                           java.lang.String jobFile)
                    throws java.io.IOException
Description copied from interface: JobSubmissionProtocol
Submit a Job for execution. Returns the latest profile for that job. The job files should be submitted in system-dir/jobName.

Specified by:
submitJob in interface JobSubmissionProtocol
Returns:
jobStatus
Throws:
java.io.IOException

getClusterStatus

public ClusterStatus getClusterStatus(boolean detailed)
Description copied from interface: JobSubmissionProtocol
Get the current status of the cluster

Specified by:
getClusterStatus in interface JobSubmissionProtocol
Parameters:
detailed - if true then report groom names as well
Returns:
summary of the state of the cluster

findGroomServer

public WorkerProtocol findGroomServer(GroomServerStatus status)
Find WorkerProtocol with corresponded groom server status

Returns:
GroomServerStatus

findGroomServers

public java.util.Collection<WorkerProtocol> findGroomServers()
Find the collection of groom servers.

Returns:
Collection of groom servers list.

groomServerStatusKeySet

public java.util.Collection<GroomServerStatus> groomServerStatusKeySet()
Collection of GroomServerStatus as the key set.

Returns:
Collection of GroomServerStatus.

addJobInProgressListener

public void addJobInProgressListener(org.apache.hama.bsp.JobInProgressListener listener)
Registers a JobInProgressListener to GroomServerManager. Therefore, adding a JobInProgress will trigger the jobAdded function.


removeJobInProgressListener

public void removeJobInProgressListener(org.apache.hama.bsp.JobInProgressListener listener)
Unregisters a JobInProgressListener to GroomServerManager. Therefore, the remove of a JobInProgress will trigger the jobRemoved action.


currentGroomServerPeers

public java.util.Map<java.lang.String,java.lang.String> currentGroomServerPeers()
Current GroomServer Peers.

Returns:
GroomName and PeerName(host:port) in pair.

getBSPMasterName

public java.lang.String getBSPMasterName()

getStartTime

public long getStartTime()

getBSPMasterIdentifier

public java.lang.String getBSPMasterIdentifier()

getHttpPort

public int getHttpPort()

jobsToComplete

public JobStatus[] jobsToComplete()
                           throws java.io.IOException
Description copied from interface: JobSubmissionProtocol
Get the jobs that are not completed and not failed

Specified by:
jobsToComplete in interface JobSubmissionProtocol
Returns:
array of JobStatus for the running/to-be-run jobs.
Throws:
java.io.IOException

getAllJobs

public JobStatus[] getAllJobs()
                       throws java.io.IOException
Description copied from interface: JobSubmissionProtocol
Get all the jobs submitted.

Specified by:
getAllJobs in interface JobSubmissionProtocol
Returns:
array of JobStatus for the submitted jobs
Throws:
java.io.IOException

getFilesystemName

public java.lang.String getFilesystemName()
                                   throws java.io.IOException
Description copied from interface: JobSubmissionProtocol
A BSP system always operates on a single filesystem. This function returns the fs name. ('local' if the localfs; 'addr:port' if dfs). The client can then copy files into the right locations prior to submitting the job.

Specified by:
getFilesystemName in interface JobSubmissionProtocol
Throws:
java.io.IOException

getSystemDir

public java.lang.String getSystemDir()
Return system directory to which BSP store control files.

Specified by:
getSystemDir in interface JobSubmissionProtocol
Specified by:
getSystemDir in interface MasterProtocol
Returns:
the system directory where job-specific files are to be placed.

getJobProfile

public JobProfile getJobProfile(BSPJobID jobid)
                         throws java.io.IOException
Description copied from interface: JobSubmissionProtocol
Grab a handle to a job that is already known to the BSPMaster.

Specified by:
getJobProfile in interface JobSubmissionProtocol
Returns:
Profile of the job, or null if not found.
Throws:
java.io.IOException

getJobStatus

public JobStatus getJobStatus(BSPJobID jobid)
                       throws java.io.IOException
Description copied from interface: JobSubmissionProtocol
Grab a handle to a job that is already known to the BSPMaster.

Specified by:
getJobStatus in interface JobSubmissionProtocol
Returns:
Status of the job, or null if not found.
Throws:
java.io.IOException

killJob

public void killJob(BSPJobID jobid)
             throws java.io.IOException
Description copied from interface: JobSubmissionProtocol
Kill the indicated job

Specified by:
killJob in interface JobSubmissionProtocol
Throws:
java.io.IOException

killTask

public boolean killTask(TaskAttemptID taskId,
                        boolean shouldFail)
                 throws java.io.IOException
Description copied from interface: JobSubmissionProtocol
Kill indicated task attempt.

Specified by:
killTask in interface JobSubmissionProtocol
Parameters:
taskId - the id of the task to kill.
shouldFail - if true the task is failed and added to failed tasks list, otherwise it is just killed, w/o affecting job failure status.
Throws:
java.io.IOException

constructMaster

public static BSPMaster constructMaster(java.lang.Class<? extends BSPMaster> masterClass,
                                        org.apache.hadoop.conf.Configuration conf)

shutdown

public void shutdown()

currentState

public BSPMaster.State currentState()


Copyright © 2008-2011 The Apache Software Foundation. All Rights Reserved.