org.apache.mahout.math.stats.entropy
Class SpecificConditionalEntropyReducer
java.lang.Object
org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.Text,VarIntWritable,org.apache.hadoop.io.Text,org.apache.hadoop.io.DoubleWritable>
org.apache.mahout.math.stats.entropy.SpecificConditionalEntropyReducer
public final class SpecificConditionalEntropyReducer
- extends org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.Text,VarIntWritable,org.apache.hadoop.io.Text,org.apache.hadoop.io.DoubleWritable>
Does the weighted conditional entropy calculation with
H(values|key) = p(key) * sum_i(p(values_i|key) * log_2(p(values_i|key)))
= p(key) * (log(|key|) - sum_i(values_i * log_2(values_i)) / |key|)
= (sum * log_2(sum) - sum_i(values_i * log_2(values_i))/n WITH sum = sum_i(values_i)
= (sum * log(sum) - sum_i(values_i * log(values_i)) / (n * log(2))
Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.Reducer |
org.apache.hadoop.mapreduce.Reducer.Context |
Method Summary |
protected void |
reduce(org.apache.hadoop.io.Text key,
Iterable<VarIntWritable> values,
org.apache.hadoop.mapreduce.Reducer.Context context)
|
protected void |
setup(org.apache.hadoop.mapreduce.Reducer.Context context)
|
Methods inherited from class org.apache.hadoop.mapreduce.Reducer |
cleanup, run |
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
SpecificConditionalEntropyReducer
public SpecificConditionalEntropyReducer()
setup
protected void setup(org.apache.hadoop.mapreduce.Reducer.Context context)
throws IOException,
InterruptedException
- Overrides:
setup
in class org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.Text,VarIntWritable,org.apache.hadoop.io.Text,org.apache.hadoop.io.DoubleWritable>
- Throws:
IOException
InterruptedException
reduce
protected void reduce(org.apache.hadoop.io.Text key,
Iterable<VarIntWritable> values,
org.apache.hadoop.mapreduce.Reducer.Context context)
throws IOException,
InterruptedException
- Overrides:
reduce
in class org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.Text,VarIntWritable,org.apache.hadoop.io.Text,org.apache.hadoop.io.DoubleWritable>
- Throws:
IOException
InterruptedException
Copyright © 2008-2012 The Apache Software Foundation. All Rights Reserved.