org.apache.blur.utils
Class TableShardCountCollapser

java.lang.Object
  extended by org.apache.hadoop.conf.Configured
      extended by org.apache.blur.utils.TableShardCountCollapser
All Implemented Interfaces:
org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool

public class TableShardCountCollapser
extends org.apache.hadoop.conf.Configured
implements org.apache.hadoop.util.Tool

This class is used to reduce the total number of shards of a table. The main use would be if during an indexing job the number of reducers were increased to make indexing faster, but the total number of shards in the table needed to be smaller. This utility safely collapses indexes together thus reducing the total number of shards in the table. For example if you wanted to run 1024 reducers but only wanted to run 128 shards in a table. After the bulk map reducer job finishes, this utility could be executed: TableShardCountCollapser 128 The result would be 128 shards in the table path.


Constructor Summary
TableShardCountCollapser()
           
 
Method Summary
 void collapseShardsTo(int newShardCount)
           
 List<Integer> getCollapsePossibilities()
           
static void main(String[] args)
           
 int run(String[] args)
           
 void setTablePath(org.apache.hadoop.fs.Path path)
           
 boolean validateCount(int count)
           
 
Methods inherited from class org.apache.hadoop.conf.Configured
getConf, setConf
 
Methods inherited from class java.lang.Object
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 
Methods inherited from interface org.apache.hadoop.conf.Configurable
getConf, setConf
 

Constructor Detail

TableShardCountCollapser

public TableShardCountCollapser()
Method Detail

main

public static void main(String[] args)
                 throws Exception
Throws:
Exception

run

public int run(String[] args)
        throws Exception
Specified by:
run in interface org.apache.hadoop.util.Tool
Throws:
Exception

validateCount

public boolean validateCount(int count)
                      throws IOException
Throws:
IOException

setTablePath

public void setTablePath(org.apache.hadoop.fs.Path path)

getCollapsePossibilities

public List<Integer> getCollapsePossibilities()
                                       throws IOException
Throws:
IOException

collapseShardsTo

public void collapseShardsTo(int newShardCount)
                      throws IOException
Throws:
IOException


Copyright © 2012-2014 The Apache Software Foundation. All Rights Reserved.