org.apache.hadoop.hive.ql.exec
Class FileSinkOperator

java.lang.Object
  extended by org.apache.hadoop.hive.ql.exec.Operator<T>
      extended by org.apache.hadoop.hive.ql.exec.TerminalOperator<fileSinkDesc>
          extended by org.apache.hadoop.hive.ql.exec.FileSinkOperator
All Implemented Interfaces:
Serializable, Node

public class FileSinkOperator
extends TerminalOperator<fileSinkDesc>
implements Serializable

File Sink operator implementation

See Also:
Serialized Form

Nested Class Summary
static interface FileSinkOperator.RecordWriter
           
static class FileSinkOperator.TableIdEnum
           
 
Nested classes/interfaces inherited from class org.apache.hadoop.hive.ql.exec.Operator
Operator.OperatorFunc, Operator.State
 
Field Summary
protected  boolean autoDelete
           
protected  org.apache.hadoop.io.BytesWritable commonKey
           
protected  org.apache.hadoop.fs.Path finalPath
           
protected  org.apache.hadoop.fs.FileSystem fs
           
protected  org.apache.hadoop.fs.Path outPath
           
protected  FileSinkOperator.RecordWriter outWriter
           
protected  Serializer serializer
           
protected  FileSinkOperator.TableIdEnum tabIdEnum
           
 
Fields inherited from class org.apache.hadoop.hive.ql.exec.Operator
alias, childOperators, conf, done, gWork, id, joinAlias, LOG, out, parentOperators, reporter, statsMap
 
Constructor Summary
FileSinkOperator()
           
 
Method Summary
 void close(boolean abort)
           
 String getName()
          Implements the getName function for the Node Interface.
 void initialize(org.apache.hadoop.conf.Configuration hconf, org.apache.hadoop.mapred.Reporter reporter)
           
 void jobClose(org.apache.hadoop.conf.Configuration hconf, boolean success)
          Unlike other operator interfaces which are called from map or reduce task, jobClose is called from the jobclient side once the job has completed
 void process(Object row, ObjectInspector rowInspector)
           
static void renameOrMoveFiles(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path src, org.apache.hadoop.fs.Path dst)
          Rename src to dst, or in the case dst already exists, move files in src to dst.
 
Methods inherited from class org.apache.hadoop.hive.ql.exec.Operator
dump, endGroup, forward, getChildOperators, getChildren, getConf, getDone, getParentOperators, getSchema, getStats, logStats, preorderMap, resetStats, setAlias, setChildOperators, setConf, setDone, setId, setJoinAlias, setMapredWork, setOutputCollector, setParentOperators, setSchema, startGroup
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

outWriter

protected transient FileSinkOperator.RecordWriter outWriter

fs

protected transient org.apache.hadoop.fs.FileSystem fs

outPath

protected transient org.apache.hadoop.fs.Path outPath

finalPath

protected transient org.apache.hadoop.fs.Path finalPath

serializer

protected transient Serializer serializer

commonKey

protected transient org.apache.hadoop.io.BytesWritable commonKey

tabIdEnum

protected transient FileSinkOperator.TableIdEnum tabIdEnum

autoDelete

protected transient boolean autoDelete
Constructor Detail

FileSinkOperator

public FileSinkOperator()
Method Detail

close

public void close(boolean abort)
           throws HiveException
Overrides:
close in class Operator<fileSinkDesc>
Throws:
HiveException

initialize

public void initialize(org.apache.hadoop.conf.Configuration hconf,
                       org.apache.hadoop.mapred.Reporter reporter)
                throws HiveException
Overrides:
initialize in class Operator<fileSinkDesc>
Throws:
HiveException

process

public void process(Object row,
                    ObjectInspector rowInspector)
             throws HiveException
Specified by:
process in class Operator<fileSinkDesc>
Throws:
HiveException

getName

public String getName()
Description copied from class: Operator
Implements the getName function for the Node Interface.

Specified by:
getName in interface Node
Overrides:
getName in class Operator<fileSinkDesc>
Returns:
the name of the operator

jobClose

public void jobClose(org.apache.hadoop.conf.Configuration hconf,
                     boolean success)
              throws HiveException
Description copied from class: Operator
Unlike other operator interfaces which are called from map or reduce task, jobClose is called from the jobclient side once the job has completed

Overrides:
jobClose in class Operator<fileSinkDesc>
Parameters:
hconf - Configuration with with which job was submitted
success - whether the job was completed successfully or not
Throws:
HiveException

renameOrMoveFiles

public static void renameOrMoveFiles(org.apache.hadoop.fs.FileSystem fs,
                                     org.apache.hadoop.fs.Path src,
                                     org.apache.hadoop.fs.Path dst)
                              throws IOException
Rename src to dst, or in the case dst already exists, move files in src to dst. If there is an existing file with the same name, the new file's name will be appended with "_1", "_2", etc.

Parameters:
fs - the FileSystem where src and dst are on.
src - the src directory
dst - the target directory
Throws:
IOException


Copyright © 2009 The Apache Software Foundation