org.apache.hadoop.hive.ql.optimizer
Class MapJoinProcessor

java.lang.Object
  extended by org.apache.hadoop.hive.ql.optimizer.MapJoinProcessor
All Implemented Interfaces:
Transform

public class MapJoinProcessor
extends Object
implements Transform

Implementation of one of the rule-based map join optimization. User passes hints to specify map-joins and during this optimization, all user specified map joins are converted to MapJoins - the reduce sink operator above the join are converted to map sink operators. In future, once statistics are implemented, this transformation can also be done based on costs.


Nested Class Summary
static class MapJoinProcessor.CurrentMapJoin
          CurrentMapJoin.
static class MapJoinProcessor.Default
          Default.
static class MapJoinProcessor.MapJoinDefault
          MapJoinDefault.
static class MapJoinProcessor.MapJoinFS
          MapJoinFS.
static class MapJoinProcessor.MapJoinWalkerCtx
          MapJoinWalkerCtx.
 
Constructor Summary
MapJoinProcessor()
          empty constructor.
 
Method Summary
static void checkMapJoin(int mapJoinPos, JoinCondDesc[] condns)
           
static MapJoinOperator convertMapJoin(LinkedHashMap<Operator<? extends Serializable>,OpParseContext> opParseCtxMap, JoinOperator op, QBJoinTree joinTree, int mapJoinPos, boolean noCheckOuterJoin)
          convert a regular join to a a map-side join.
 MapJoinOperator generateMapJoinOperator(ParseContext pctx, JoinOperator op, QBJoinTree joinTree, int mapJoinPos)
           
static String genMapJoinOpAndLocalWork(MapredWork newWork, JoinOperator op, int mapJoinPos)
           
static NodeProcessor getCurrentMapJoin()
           
static NodeProcessor getDefault()
           
static NodeProcessor getMapJoinDefault()
           
static NodeProcessor getMapJoinFS()
           
static HashSet<Integer> getSmallTableOnlySet(JoinCondDesc[] condns)
           
 ParseContext transform(ParseContext pactx)
          Transform the query tree.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Constructor Detail

MapJoinProcessor

public MapJoinProcessor()
empty constructor.

Method Detail

genMapJoinOpAndLocalWork

public static String genMapJoinOpAndLocalWork(MapredWork newWork,
                                              JoinOperator op,
                                              int mapJoinPos)
                                       throws SemanticException
Throws:
SemanticException

convertMapJoin

public static MapJoinOperator convertMapJoin(LinkedHashMap<Operator<? extends Serializable>,OpParseContext> opParseCtxMap,
                                             JoinOperator op,
                                             QBJoinTree joinTree,
                                             int mapJoinPos,
                                             boolean noCheckOuterJoin)
                                      throws SemanticException
convert a regular join to a a map-side join.

Parameters:
op - join operator
qbJoin - qb join tree
mapJoinPos - position of the source to be read as part of map-reduce framework. All other sources are cached in memory
Throws:
SemanticException

generateMapJoinOperator

public MapJoinOperator generateMapJoinOperator(ParseContext pctx,
                                               JoinOperator op,
                                               QBJoinTree joinTree,
                                               int mapJoinPos)
                                        throws SemanticException
Throws:
SemanticException

getSmallTableOnlySet

public static HashSet<Integer> getSmallTableOnlySet(JoinCondDesc[] condns)

checkMapJoin

public static void checkMapJoin(int mapJoinPos,
                                JoinCondDesc[] condns)
                         throws SemanticException
Throws:
SemanticException

transform

public ParseContext transform(ParseContext pactx)
                       throws SemanticException
Transform the query tree. For each join, check if it is a map-side join (user specified). If yes, convert it to a map-side join.

Specified by:
transform in interface Transform
Parameters:
pactx - current parse context
Returns:
ParseContext
Throws:
SemanticException

getMapJoinFS

public static NodeProcessor getMapJoinFS()

getMapJoinDefault

public static NodeProcessor getMapJoinDefault()

getDefault

public static NodeProcessor getDefault()

getCurrentMapJoin

public static NodeProcessor getCurrentMapJoin()


Copyright © 2011 The Apache Software Foundation