org.apache.hadoop.examples
Class RandomWriter

java.lang.Object
  extended by org.apache.hadoop.util.ToolBase
      extended by org.apache.hadoop.examples.RandomWriter
All Implemented Interfaces:
Configurable, Tool

public class RandomWriter
extends ToolBase

This program uses map/reduce to just run a distributed job where there is no interaction between the tasks and each task write a large unsorted random binary sequence file of BytesWritable. In order for this program to generate data for terasort with 10-byte keys and 90-byte values, have the following config:

<?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <configuration> <property> <name>test.randomwrite.min_key</name> <value>10</value> </property> <property> <name>test.randomwrite.max_key</name> <value>10</value> </property> <property> <name>test.randomwrite.min_value</name> <value>90</value> </property> <property> <name>test.randomwrite.max_value</name> <value>90</value> </property> <property> <name>test.randomwrite.total_bytes</name> <value>1099511627776</value> </property> </configuration> Equivalently, RandomWriter also supports all the above options and ones supported by ToolBase via the command-line.


Field Summary
 
Fields inherited from class org.apache.hadoop.util.ToolBase
conf
 
Constructor Summary
RandomWriter()
           
 
Method Summary
static void main(String[] args)
           
 int run(String[] args)
          This is the main routine for launching a distributed random write job.
 
Methods inherited from class org.apache.hadoop.util.ToolBase
doMain, getConf, setConf
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

RandomWriter

public RandomWriter()
Method Detail

run

public int run(String[] args)
        throws Exception
This is the main routine for launching a distributed random write job. It runs 10 maps/node and each node writes 1 gig of data to a DFS file. The reduce doesn't do anything.

Parameters:
args - command specific arguments
Returns:
exit code
Throws:
IOException
Exception

main

public static void main(String[] args)
                 throws Exception
Throws:
Exception


Copyright © 2006 The Apache Software Foundation