|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectorg.apache.hadoop.hive.ql.optimizer.GenMapRedUtils
public class GenMapRedUtils
General utility common functions for the Processor to convert operator into map-reduce tasks
Constructor Summary | |
---|---|
GenMapRedUtils()
|
Method Summary | |
---|---|
static mapredWork |
getMapRedWork()
create a new plan and return |
static void |
initMapJoinPlan(Operator<? extends Serializable> op,
GenMRProcContext opProcCtx,
boolean readInputMapJoin,
boolean readInputUnion,
boolean setReducer,
int pos)
Initialize the current plan by adding it to root tasks |
static void |
initPlan(ReduceSinkOperator op,
GenMRProcContext opProcCtx)
Initialize the current plan by adding it to root tasks |
static void |
initUnionPlan(GenMRProcContext opProcCtx,
Task<? extends Serializable> currTask,
boolean local)
|
static void |
initUnionPlan(ReduceSinkOperator op,
GenMRProcContext opProcCtx)
Initialize the current union plan. |
static void |
joinPlan(Operator<? extends Serializable> op,
Task<? extends Serializable> oldTask,
Task<? extends Serializable> task,
GenMRProcContext opProcCtx,
int pos,
boolean split,
boolean readMapJoinData,
boolean readUnionData)
Merge the current task with the task for the current reducer |
static void |
mergeMapJoinUnion(UnionOperator union,
GenMRProcContext ctx,
int pos)
|
static void |
setKeyAndValueDesc(mapredWork plan,
Operator<? extends Serializable> topOp)
set key and value descriptor |
static void |
setTaskPlan(String alias_id,
Operator<? extends Serializable> topOp,
mapredWork plan,
boolean local,
GenMRProcContext opProcCtx)
set the current task in the mapredWork |
static void |
setTaskPlan(String path,
String alias,
Operator<? extends Serializable> topOp,
mapredWork plan,
boolean local,
tableDesc tt_desc)
set the current task in the mapredWork |
static void |
splitPlan(ReduceSinkOperator op,
GenMRProcContext opProcCtx)
Split the current plan by creating a temporary destination |
static void |
splitTasks(Operator<? extends Serializable> op,
Task<? extends Serializable> parentTask,
Task<? extends Serializable> childTask,
GenMRProcContext opProcCtx,
boolean setReducer,
boolean local,
int posn)
|
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Constructor Detail |
---|
public GenMapRedUtils()
Method Detail |
---|
public static void initPlan(ReduceSinkOperator op, GenMRProcContext opProcCtx) throws SemanticException
op
- the reduce sink operator encounteredopProcCtx
- processing context
SemanticException
public static void initMapJoinPlan(Operator<? extends Serializable> op, GenMRProcContext opProcCtx, boolean readInputMapJoin, boolean readInputUnion, boolean setReducer, int pos) throws SemanticException
op
- the map join operator encounteredopProcCtx
- processing contextpos
- position of the parent
SemanticException
public static void initUnionPlan(ReduceSinkOperator op, GenMRProcContext opProcCtx) throws SemanticException
op
- the reduce sink operator encounteredopProcCtx
- processing context
SemanticException
public static void initUnionPlan(GenMRProcContext opProcCtx, Task<? extends Serializable> currTask, boolean local)
public static void joinPlan(Operator<? extends Serializable> op, Task<? extends Serializable> oldTask, Task<? extends Serializable> task, GenMRProcContext opProcCtx, int pos, boolean split, boolean readMapJoinData, boolean readUnionData) throws SemanticException
op
- operator being processedoldTask
- the old task for the current reducertask
- the current task for the current reduceropProcCtx
- processing contextpos
- position of the parent in the stack
SemanticException
public static void splitPlan(ReduceSinkOperator op, GenMRProcContext opProcCtx) throws SemanticException
op
- the reduce sink operator encounteredopProcCtx
- processing context
SemanticException
public static void setTaskPlan(String alias_id, Operator<? extends Serializable> topOp, mapredWork plan, boolean local, GenMRProcContext opProcCtx) throws SemanticException
alias_id
- current aliastopOp
- the top operator of the stackplan
- current planlocal
- whether you need to add to map-reduce or local workopProcCtx
- processing context
SemanticException
public static void setTaskPlan(String path, String alias, Operator<? extends Serializable> topOp, mapredWork plan, boolean local, tableDesc tt_desc) throws SemanticException
alias
- current aliastopOp
- the top operator of the stackplan
- current planlocal
- whether you need to add to map-reduce or local worktt_desc
- table descriptor
SemanticException
public static void setKeyAndValueDesc(mapredWork plan, Operator<? extends Serializable> topOp)
plan
- current plantopOp
- current top operator in the pathpublic static mapredWork getMapRedWork()
public static void splitTasks(Operator<? extends Serializable> op, Task<? extends Serializable> parentTask, Task<? extends Serializable> childTask, GenMRProcContext opProcCtx, boolean setReducer, boolean local, int posn) throws SemanticException
SemanticException
public static void mergeMapJoinUnion(UnionOperator union, GenMRProcContext ctx, int pos) throws SemanticException
SemanticException
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |