|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectorg.apache.hadoop.conf.Configured
org.apache.hadoop.mapred.gridmix.Gridmix
public class Gridmix
Driver class for the Gridmix3 benchmark. Gridmix accepts a timestamped stream (trace) of job/task descriptions. For each job in the trace, the client will submit a corresponding, synthetic job to the target cluster at the rate in the original trace. The intent is to provide a benchmark that can be configured and extended to closely match the measured resource profile of actual, production loads.
Field Summary | |
---|---|
static String |
GRIDMIX_OUT_DIR
Output (scratch) directory for submitted jobs. |
static String |
GRIDMIX_QUE_DEP
The depth of the queue of job descriptions. |
static String |
GRIDMIX_SUB_MUL
Multiplier to accelerate or decelerate job submission. |
static String |
GRIDMIX_SUB_THR
Number of submitting threads at the client and upper bound for in-memory split data. |
static String |
GRIDMIX_USR_RSV
Class used to resolve users in the trace to the list of target users on the cluster. |
static org.apache.commons.logging.Log |
LOG
|
Constructor Summary | |
---|---|
Gridmix()
|
Method Summary | |
---|---|
protected org.apache.hadoop.mapred.gridmix.JobFactory |
createJobFactory(org.apache.hadoop.mapred.gridmix.JobSubmitter submitter,
String traceIn,
org.apache.hadoop.fs.Path scratchDir,
org.apache.hadoop.conf.Configuration conf,
CountDownLatch startFlag,
org.apache.hadoop.mapred.gridmix.UserResolver resolver)
|
protected org.apache.hadoop.mapred.gridmix.JobMonitor |
createJobMonitor(Statistics stats)
|
protected JobStoryProducer |
createJobStoryProducer(String traceIn,
org.apache.hadoop.conf.Configuration conf)
Create an appropriate JobStoryProducer object for the
given trace. |
protected org.apache.hadoop.mapred.gridmix.JobSubmitter |
createJobSubmitter(org.apache.hadoop.mapred.gridmix.JobMonitor monitor,
int threads,
int queueDepth,
org.apache.hadoop.mapred.gridmix.FilePool pool,
org.apache.hadoop.mapred.gridmix.UserResolver resolver,
Statistics statistics)
|
org.apache.hadoop.mapred.gridmix.UserResolver |
getCurrentUserResolver()
|
static void |
main(String[] argv)
|
protected void |
printUsage(PrintStream out)
|
int |
run(String[] argv)
|
protected void |
writeInputData(long genbytes,
org.apache.hadoop.fs.Path ioPath)
Write random bytes at the path provided. |
Methods inherited from class org.apache.hadoop.conf.Configured |
---|
getConf, setConf |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Methods inherited from interface org.apache.hadoop.conf.Configurable |
---|
getConf, setConf |
Field Detail |
---|
public static final org.apache.commons.logging.Log LOG
public static final String GRIDMIX_OUT_DIR
public static final String GRIDMIX_SUB_THR
public static final String GRIDMIX_QUE_DEP
public static final String GRIDMIX_SUB_MUL
public static final String GRIDMIX_USR_RSV
Constructor Detail |
---|
public Gridmix()
Method Detail |
---|
protected void writeInputData(long genbytes, org.apache.hadoop.fs.Path ioPath) throws IOException, InterruptedException
IOException
InterruptedException
GenerateData
protected JobStoryProducer createJobStoryProducer(String traceIn, org.apache.hadoop.conf.Configuration conf) throws IOException
JobStoryProducer
object for the
given trace.
traceIn
- the path to the trace file. The special path
"-" denotes the standard input stream.conf
- the configuration to be used.
IOException
- if there was an error.protected org.apache.hadoop.mapred.gridmix.JobMonitor createJobMonitor(Statistics stats) throws IOException
IOException
protected org.apache.hadoop.mapred.gridmix.JobSubmitter createJobSubmitter(org.apache.hadoop.mapred.gridmix.JobMonitor monitor, int threads, int queueDepth, org.apache.hadoop.mapred.gridmix.FilePool pool, org.apache.hadoop.mapred.gridmix.UserResolver resolver, Statistics statistics) throws IOException
IOException
protected org.apache.hadoop.mapred.gridmix.JobFactory createJobFactory(org.apache.hadoop.mapred.gridmix.JobSubmitter submitter, String traceIn, org.apache.hadoop.fs.Path scratchDir, org.apache.hadoop.conf.Configuration conf, CountDownLatch startFlag, org.apache.hadoop.mapred.gridmix.UserResolver resolver) throws IOException
IOException
public org.apache.hadoop.mapred.gridmix.UserResolver getCurrentUserResolver()
public int run(String[] argv) throws IOException, InterruptedException
run
in interface org.apache.hadoop.util.Tool
IOException
InterruptedException
public static void main(String[] argv) throws Exception
Exception
protected void printUsage(PrintStream out)
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |