org.apache.blur.mapreduce.lib
Class DefaultBlurReducer
java.lang.Object
org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.Writable,BlurMutate,org.apache.hadoop.io.Writable,BlurMutate>
org.apache.blur.mapreduce.lib.DefaultBlurReducer
public class DefaultBlurReducer
- extends org.apache.hadoop.mapreduce.Reducer<org.apache.hadoop.io.Writable,BlurMutate,org.apache.hadoop.io.Writable,BlurMutate>
This class is to be used in conjunction with BlurOutputFormat
.
Here is a basic example of how to use both the BlurOutputFormat
and
the DefaultBlurReducer
together to build indexes.
Once this job has successfully completed the indexes will be imported by the
running shard servers and be placed online. This is a polling mechicism in
the shard servers and by default they poll every 10 seconds.
Job job = new Job(conf, "blur index");
job.setJarByClass(BlurOutputFormatTest.class);
job.setMapperClass(CsvBlurMapper.class);
job.setReducerClass(DefaultBlurReducer.class);
job.setNumReduceTasks(1);
job.setInputFormatClass(TrackingTextInputFormat.class);
job.setOutputKeyClass(Text.class);
job.setOutputValueClass(BlurMutate.class);
job.setOutputFormatClass(BlurOutputFormat.class);
FileInputFormat.addInputPath(job, new Path(TEST_ROOT_DIR + "/in"));
CsvBlurMapper.addColumns(job, "cf1", "col"); TableDescriptor
tableDescriptor = new TableDescriptor();
tableDescriptor.setShardCount(1)
;tableDescriptor.setAnalyzerDefinition(new
AnalyzerDefinition());tableDescriptor.setTableUri(new Path(TEST_ROOT_DIR
+ "/out").toString());BlurOutputFormat.setTableDescriptor(job,
tableDescriptor);
Nested classes/interfaces inherited from class org.apache.hadoop.mapreduce.Reducer |
org.apache.hadoop.mapreduce.Reducer.Context |
Methods inherited from class org.apache.hadoop.mapreduce.Reducer |
run |
DefaultBlurReducer
public DefaultBlurReducer()
Copyright © 2012-2014 The Apache Software Foundation. All Rights Reserved.