org.apache.hadoop.examples
Class NNBench

java.lang.Object
  extended by org.apache.hadoop.mapred.MapReduceBase
      extended by org.apache.hadoop.examples.NNBench
All Implemented Interfaces:
Closeable, JobConfigurable, Reducer

public class NNBench
extends MapReduceBase
implements Reducer

This program uses map/reduce to run a distributed job where there is no interaction between the tasks. Each task creates a configurable number of files. Each file has a configurable number of bytes written to it, then it is closed, re-opened, and read from, and re-closed. This program functions as a stress-test and benchmark for namenode, especially when the number of bytes written to each file is small.

Author:
Milind Bhandarkar

Nested Class Summary
static class NNBench.Map
           
 
Constructor Summary
NNBench()
           
 
Method Summary
static void main(String[] args)
          This is the main routine for launching a distributed namenode stress test.
 void reduce(WritableComparable key, Iterator values, OutputCollector output, Reporter reporter)
          Combines values for a given key.
 
Methods inherited from class org.apache.hadoop.mapred.MapReduceBase
close, configure
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 
Methods inherited from interface org.apache.hadoop.mapred.JobConfigurable
configure
 
Methods inherited from interface org.apache.hadoop.io.Closeable
close
 

Constructor Detail

NNBench

public NNBench()
Method Detail

reduce

public void reduce(WritableComparable key,
                   Iterator values,
                   OutputCollector output,
                   Reporter reporter)
            throws IOException
Description copied from interface: Reducer
Combines values for a given key. Output values must be of the same type as input values. Input keys must not be altered. Typically all values are combined into zero or one value. Output pairs are collected with calls to OutputCollector.collect(WritableComparable,Writable).

Specified by:
reduce in interface Reducer
Parameters:
key - the key
values - the values to combine
output - to collect combined values
Throws:
IOException

main

public static void main(String[] args)
                 throws IOException
This is the main routine for launching a distributed namenode stress test. It runs 10 maps/node. The reduce doesn't do anything.

Throws:
IOException


Copyright © 2006 The Apache Software Foundation