org.apache.hadoop.hbase.master.balancer
Class StochasticLoadBalancer

java.lang.Object
  extended by org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer
      extended by org.apache.hadoop.hbase.master.balancer.StochasticLoadBalancer
All Implemented Interfaces:
org.apache.hadoop.conf.Configurable, LoadBalancer

@InterfaceAudience.Private
public class StochasticLoadBalancer
extends BaseLoadBalancer

This is a best effort load balancer. Given a Cost function F(C) => x It will randomly try and mutate the cluster to Cprime. If F(Cprime) < F(C) then the new cluster state becomes the plan. It includes costs functions to compute the cost of:

Every cost function returns a number between 0 and 1 inclusive; where 0 is the lowest cost best solution, and 1 is the highest possible cost and the worst solution. The computed costs are scaled by their respective multipliers:

In addition to the above configurations, the balancer can be tuned by the following configuration values:

This balancer is best used with hbase.master.loadbalance.bytable set to false so that the balancer gets the full picture of all loads on the cluster.


Nested Class Summary
static class StochasticLoadBalancer.CostFromRegionLoadFunction
          Base class the allows writing costs functions from rolling average of some number from RegionLoad.
static class StochasticLoadBalancer.CostFunction
          Base class of StochasticLoadBalancer's Cost Functions.
static class StochasticLoadBalancer.LoadPicker
           
static class StochasticLoadBalancer.LocalityCostFunction
          Compute a cost of a potential cluster configuration based upon where StoreFiles are located.
static class StochasticLoadBalancer.MemstoreSizeCostFunction
          Compute the cost of total memstore size.
static class StochasticLoadBalancer.MoveCostFunction
          Given the starting state of the regions and a potential ending state compute cost based upon the number of regions that have moved.
static class StochasticLoadBalancer.ReadRequestCostFunction
          Compute the cost of total number of read requests The more unbalanced the higher the computed cost will be.
static class StochasticLoadBalancer.RegionCountSkewCostFunction
          Compute the cost of a potential cluster state from skew in number of regions on a cluster.
static class StochasticLoadBalancer.StoreFileCostFunction
          Compute the cost of total open storefiles size.
static class StochasticLoadBalancer.TableSkewCostFunction
          Compute the cost of a potential cluster configuration based upon how evenly distributed tables are.
static class StochasticLoadBalancer.WriteRequestCostFunction
          Compute the cost of total number of write requests.
 
Nested classes/interfaces inherited from class org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer
BaseLoadBalancer.Cluster
 
Field Summary
 
Fields inherited from class org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer
services
 
Constructor Summary
StochasticLoadBalancer()
           
 
Method Summary
 List<RegionPlan> balanceCluster(Map<ServerName,List<HRegionInfo>> clusterState)
          Given the cluster state this will try and approach an optimal balance.
protected  double computeCost(BaseLoadBalancer.Cluster cluster, double previousCost)
          This is the main cost function.
 void setClusterStatus(ClusterStatus st)
          Set the current cluster status.
 void setConf(org.apache.hadoop.conf.Configuration conf)
           
 void setMasterServices(MasterServices masterServices)
          Set the master service.
 
Methods inherited from class org.apache.hadoop.hbase.master.balancer.BaseLoadBalancer
getConf, immediateAssignment, needsBalance, randomAssignment, retainAssignment, roundRobinAssignment
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

StochasticLoadBalancer

public StochasticLoadBalancer()
Method Detail

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)
Specified by:
setConf in interface org.apache.hadoop.conf.Configurable
Overrides:
setConf in class BaseLoadBalancer

setClusterStatus

public void setClusterStatus(ClusterStatus st)
Description copied from interface: LoadBalancer
Set the current cluster status. This allows a LoadBalancer to map host name to a server

Specified by:
setClusterStatus in interface LoadBalancer
Overrides:
setClusterStatus in class BaseLoadBalancer

setMasterServices

public void setMasterServices(MasterServices masterServices)
Description copied from interface: LoadBalancer
Set the master service.

Specified by:
setMasterServices in interface LoadBalancer
Overrides:
setMasterServices in class BaseLoadBalancer

balanceCluster

public List<RegionPlan> balanceCluster(Map<ServerName,List<HRegionInfo>> clusterState)
Given the cluster state this will try and approach an optimal balance. This should always approach the optimal state given enough steps.

Returns:
List of plans

computeCost

protected double computeCost(BaseLoadBalancer.Cluster cluster,
                             double previousCost)
This is the main cost function. It will compute a cost associated with a proposed cluster state. All different costs will be combined with their multipliers to produce a double cost.

Parameters:
cluster - The state of the cluster
previousCost - the previous cost. This is used as an early out.
Returns:
a double of a cost associated with the proposed cluster state. This cost is an aggregate of all individual cost functions.


Copyright © 2013 The Apache Software Foundation. All Rights Reserved.