org.apache.blur.utils
Class TableShardCountCollapser
java.lang.Object
org.apache.hadoop.conf.Configured
org.apache.blur.utils.TableShardCountCollapser
- All Implemented Interfaces:
- org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool
public class TableShardCountCollapser
- extends org.apache.hadoop.conf.Configured
- implements org.apache.hadoop.util.Tool
This class is used to reduce the total number of shards of a table. The main
use would be if during an indexing job the number of reducers were increased
to make indexing faster, but the total number of shards in the table needed
to be smaller. This utility safely collapses indexes together thus reducing
the total number of shards in the table.
For example if you wanted to run 1024 reducers but only wanted to run 128
shards in a table. After the bulk map reducer job finishes, this utility
could be executed:
TableShardCountCollapser 128
The result would be 128 shards in the table path.
Methods inherited from class org.apache.hadoop.conf.Configured |
getConf, setConf |
Methods inherited from interface org.apache.hadoop.conf.Configurable |
getConf, setConf |
TableShardCountCollapser
public TableShardCountCollapser()
main
public static void main(String[] args)
throws Exception
- Throws:
Exception
run
public int run(String[] args)
throws Exception
- Specified by:
run
in interface org.apache.hadoop.util.Tool
- Throws:
Exception
validateCount
public boolean validateCount(int count)
throws IOException
- Throws:
IOException
setTablePath
public void setTablePath(org.apache.hadoop.fs.Path path)
getCollapsePossibilities
public List<Integer> getCollapsePossibilities()
throws IOException
- Throws:
IOException
collapseShardsTo
public void collapseShardsTo(int newShardCount)
throws IOException
- Throws:
IOException
Copyright © 2012-2014 The Apache Software Foundation. All Rights Reserved.