|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectorg.apache.hadoop.hive.ql.io.HiveFileFormatUtils
public class HiveFileFormatUtils
An util class for various Hive file format tasks. registerOutputFormatSubstitute(Class, Class) getOutputFormatSubstitute(Class) are added for backward compatibility. They return the newly added HiveOutputFormat for the older ones.
Field Summary | |
---|---|
static String |
READ_COLUMN_IDS_CONF_STR
|
Constructor Summary | |
---|---|
HiveFileFormatUtils()
|
Method Summary | |
---|---|
static boolean |
checkInputFormat(org.apache.hadoop.fs.FileSystem fs,
HiveConf conf,
Class<? extends org.apache.hadoop.mapred.InputFormat> inputFormatCls,
ArrayList<org.apache.hadoop.fs.FileStatus> files)
checks if files are in same format as the given input format |
static Class<? extends InputFormatChecker> |
getInputFormatChecker(Class<?> inputFormat)
get an InputFormatChecker for a file format. |
static org.apache.hadoop.fs.Path |
getOutputFormatFinalPath(org.apache.hadoop.fs.Path parent,
org.apache.hadoop.mapred.JobConf jc,
HiveOutputFormat<?,?> hiveOutputFormat,
boolean isCompressed,
org.apache.hadoop.fs.Path defaultFinalPath)
get the final output path of a given FileOutputFormat. |
static Class<? extends HiveOutputFormat> |
getOutputFormatSubstitute(Class<?> origin)
get a OutputFormat's substitute HiveOutputFormat |
static ArrayList<Integer> |
getReadColumnIDs(org.apache.hadoop.conf.Configuration conf)
Returns an array of column ids(start from zero) which is set in the given parameter conf. |
static void |
registerInputFormatChecker(Class<? extends org.apache.hadoop.mapred.InputFormat> format,
Class<? extends InputFormatChecker> checker)
register an InputFormatChecker for a given InputFormat |
static void |
registerOutputFormatSubstitute(Class<? extends org.apache.hadoop.mapred.OutputFormat> origin,
Class<? extends HiveOutputFormat> substitute)
register a substitute |
static void |
setFullyReadColumns(org.apache.hadoop.conf.Configuration conf)
Clears the read column ids set in the conf, and will read all columns. |
static void |
setReadColumnIDs(org.apache.hadoop.conf.Configuration conf,
ArrayList<Integer> ids)
Sets read columns' ids(start from zero) for RCFile's Reader. |
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
---|
public static String READ_COLUMN_IDS_CONF_STR
Constructor Detail |
---|
public HiveFileFormatUtils()
Method Detail |
---|
public static void registerOutputFormatSubstitute(Class<? extends org.apache.hadoop.mapred.OutputFormat> origin, Class<? extends HiveOutputFormat> substitute)
origin
- the class that need to be substitutedsubstitute
- public static Class<? extends HiveOutputFormat> getOutputFormatSubstitute(Class<?> origin)
public static org.apache.hadoop.fs.Path getOutputFormatFinalPath(org.apache.hadoop.fs.Path parent, org.apache.hadoop.mapred.JobConf jc, HiveOutputFormat<?,?> hiveOutputFormat, boolean isCompressed, org.apache.hadoop.fs.Path defaultFinalPath) throws IOException
parent
- parent dir of the expected final output pathjc
- job configuration
IOException
public static void registerInputFormatChecker(Class<? extends org.apache.hadoop.mapred.InputFormat> format, Class<? extends InputFormatChecker> checker)
format
- the class that need to be substitutedchecker
- public static Class<? extends InputFormatChecker> getInputFormatChecker(Class<?> inputFormat)
public static boolean checkInputFormat(org.apache.hadoop.fs.FileSystem fs, HiveConf conf, Class<? extends org.apache.hadoop.mapred.InputFormat> inputFormatCls, ArrayList<org.apache.hadoop.fs.FileStatus> files) throws HiveException
HiveException
public static void setReadColumnIDs(org.apache.hadoop.conf.Configuration conf, ArrayList<Integer> ids)
public static ArrayList<Integer> getReadColumnIDs(org.apache.hadoop.conf.Configuration conf)
public static void setFullyReadColumns(org.apache.hadoop.conf.Configuration conf)
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |