org.apache.hadoop.hive.ql.metadata
Class Hive

java.lang.Object
  extended by org.apache.hadoop.hive.ql.metadata.Hive

public class Hive
extends Object

The Hive class contains information about this instance of Hive. An instance of Hive represents a set of data in a file system (usually HDFS) organized for easy query processing


Method Summary
 void alterPartition(String tblName, Partition newPart)
          Updates the existing table metadata with the new metadata.
 void alterTable(String tblName, Table newTbl)
          Updates the existing table metadata with the new metadata.
static void closeCurrent()
           
protected static void copyFiles(org.apache.hadoop.fs.Path srcf, org.apache.hadoop.fs.Path destf, org.apache.hadoop.fs.FileSystem fs)
           
protected  boolean createDatabase(String name, String locationUri)
           
 Partition createPartition(Table tbl, Map<String,String> partSpec)
          Creates a partition.
 Partition createPartition(Table tbl, Map<String,String> partSpec, org.apache.hadoop.fs.Path location)
          Creates a partition
 void createTable(String tableName, List<String> columns, List<String> partCols, Class<? extends org.apache.hadoop.mapred.InputFormat> fileInputFormat, Class<?> fileOutputFormat)
          Creates a table metdata and the directory for the table data
 void createTable(String tableName, List<String> columns, List<String> partCols, Class<? extends org.apache.hadoop.mapred.InputFormat> fileInputFormat, Class<?> fileOutputFormat, int bucketCount, List<String> bucketCols)
          Creates a table metdata and the directory for the table data
 void createTable(Table tbl)
          Creates the table with the give objects
 void createTable(Table tbl, boolean ifNotExists)
          Creates the table with the give objects
protected  boolean dropDatabase(String name)
           
 boolean dropPartition(String db_name, String tbl_name, List<String> part_vals, boolean deleteData)
           
 void dropTable(String dbName, String tableName)
          Drops table along with the data in it.
 void dropTable(String dbName, String tableName, boolean deleteData, boolean ignoreUnknownTab)
          Drops the table.
static Hive get()
           
static Hive get(HiveConf c)
          Gets hive object for the current thread.
static Hive get(HiveConf c, boolean needsRefresh)
          get a connection to metastore.
 List<String> getAllTables()
           
 HiveConf getConf()
           
static List<FieldSchema> getFieldsFromDeserializer(String name, Deserializer serde)
           
 Partition getPartition(Table tbl, Map<String,String> partSpec, boolean forceCreate)
          Returns partition metadata
 List<String> getPartitionNames(String dbName, String tblName, short max)
           
 List<Partition> getPartitions(Table tbl)
          get all the partitions that the table has
 Table getTable(String dbName, String tableName)
          Returns metadata of the table.
 Table getTable(String dbName, String tableName, boolean throwException)
          Returns metadata of the table
 List<String> getTablesByPattern(String tablePattern)
          returns all existing tables from default database which match the given pattern.
 List<String> getTablesForDb(String database, String tablePattern)
          returns all existing tables from the given database which match the given pattern.
 void loadPartition(org.apache.hadoop.fs.Path loadPath, String tableName, AbstractMap<String,String> partSpec, boolean replace, org.apache.hadoop.fs.Path tmpDirPath)
          Load a directory into a Hive Table Partition - Alters existing content of the partition with the contents of loadPath.
 void loadTable(org.apache.hadoop.fs.Path loadPath, String tableName, boolean replace, org.apache.hadoop.fs.Path tmpDirPath)
          Load a directory into a Hive Table.
protected static void replaceFiles(org.apache.hadoop.fs.Path srcf, org.apache.hadoop.fs.Path destf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path tmppath)
          Replaces files in the partition with new data set specifed by srcf.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Method Detail

get

public static Hive get(HiveConf c)
                throws HiveException
Gets hive object for the current thread. If one is not initialized then a new one is created If the new configuration is different in metadata conf vars then a new one is created.

Parameters:
c - new Hive Configuration
Returns:
Hive object for current thread
Throws:
HiveException

get

public static Hive get(HiveConf c,
                       boolean needsRefresh)
                throws HiveException
get a connection to metastore. see get(HiveConf) function for comments

Parameters:
c - new conf
needsRefresh - if true then creates a new one
Returns:
The connection to the metastore
Throws:
HiveException

get

public static Hive get()
                throws HiveException
Throws:
HiveException

closeCurrent

public static void closeCurrent()

createTable

public void createTable(String tableName,
                        List<String> columns,
                        List<String> partCols,
                        Class<? extends org.apache.hadoop.mapred.InputFormat> fileInputFormat,
                        Class<?> fileOutputFormat)
                 throws HiveException
Creates a table metdata and the directory for the table data

Parameters:
tableName - name of the table
columns - list of fields of the table
partCols - partition keys of the table
fileInputFormat - Class of the input format of the table data file
fileOutputFormat - Class of the output format of the table data file
Throws:
HiveException - thrown if the args are invalid or if the metadata or the data directory couldn't be created

createTable

public void createTable(String tableName,
                        List<String> columns,
                        List<String> partCols,
                        Class<? extends org.apache.hadoop.mapred.InputFormat> fileInputFormat,
                        Class<?> fileOutputFormat,
                        int bucketCount,
                        List<String> bucketCols)
                 throws HiveException
Creates a table metdata and the directory for the table data

Parameters:
tableName - name of the table
columns - list of fields of the table
partCols - partition keys of the table
fileInputFormat - Class of the input format of the table data file
fileOutputFormat - Class of the output format of the table data file
bucketCount - number of buckets that each partition (or the table itself) should be divided into
Throws:
HiveException - thrown if the args are invalid or if the metadata or the data directory couldn't be created

alterTable

public void alterTable(String tblName,
                       Table newTbl)
                throws InvalidOperationException,
                       HiveException
Updates the existing table metadata with the new metadata.

Parameters:
tblName - name of the existing table
newTbl - new name of the table. could be the old name
Throws:
InvalidOperationException - if the changes in metadata is not acceptable
org.apache.thrift.TException
HiveException

alterPartition

public void alterPartition(String tblName,
                           Partition newPart)
                    throws InvalidOperationException,
                           HiveException
Updates the existing table metadata with the new metadata.

Parameters:
tblName - name of the existing table
newTbl - new name of the table. could be the old name
Throws:
InvalidOperationException - if the changes in metadata is not acceptable
org.apache.thrift.TException
HiveException

createTable

public void createTable(Table tbl)
                 throws HiveException
Creates the table with the give objects

Parameters:
tbl - a table object
Throws:
HiveException

createTable

public void createTable(Table tbl,
                        boolean ifNotExists)
                 throws HiveException
Creates the table with the give objects

Parameters:
tbl - a table object
ifNotExists - if true, ignore AlreadyExistsException
Throws:
HiveException

dropTable

public void dropTable(String dbName,
                      String tableName)
               throws HiveException
Drops table along with the data in it. If the table doesn't exist then it is a no-op

Parameters:
dbName - database where the table lives
tableName - table to drop
Throws:
HiveException - thrown if the drop fails

dropTable

public void dropTable(String dbName,
                      String tableName,
                      boolean deleteData,
                      boolean ignoreUnknownTab)
               throws HiveException
Drops the table.

Parameters:
tableName -
deleteData - deletes the underlying data along with metadata
ignoreUnknownTab - an exception if thrown if this is falser and table doesn't exist
Throws:
HiveException

getConf

public HiveConf getConf()

getTable

public Table getTable(String dbName,
                      String tableName)
               throws HiveException
Returns metadata of the table.

Parameters:
dbName - the name of the database
tableName - the name of the table
Returns:
the table
Throws:
HiveException - if there's an internal error or if the table doesn't exist

getTable

public Table getTable(String dbName,
                      String tableName,
                      boolean throwException)
               throws HiveException
Returns metadata of the table

Parameters:
dbName - the name of the database
tableName - the name of the table
throwException - controls whether an exception is thrown or a returns a null
Returns:
the table or if throwException is false a null value.
Throws:
HiveException

getAllTables

public List<String> getAllTables()
                          throws HiveException
Throws:
HiveException

getTablesByPattern

public List<String> getTablesByPattern(String tablePattern)
                                throws HiveException
returns all existing tables from default database which match the given pattern. The matching occurs as per Java regular expressions

Parameters:
tablePattern - java re pattern
Returns:
list of table names
Throws:
HiveException

getTablesForDb

public List<String> getTablesForDb(String database,
                                   String tablePattern)
                            throws HiveException
returns all existing tables from the given database which match the given pattern. The matching occurs as per Java regular expressions

Parameters:
database - the database name
tablePattern - java re pattern
Returns:
list of table names
Throws:
HiveException

createDatabase

protected boolean createDatabase(String name,
                                 String locationUri)
                          throws AlreadyExistsException,
                                 MetaException,
                                 org.apache.thrift.TException
Parameters:
name -
locationUri -
Returns:
true or false
Throws:
AlreadyExistsException
MetaException
org.apache.thrift.TException
See Also:
HiveMetaStoreClient.createDatabase(java.lang.String, java.lang.String)

dropDatabase

protected boolean dropDatabase(String name)
                        throws MetaException,
                               org.apache.thrift.TException
Parameters:
name -
Returns:
true or false
Throws:
MetaException
org.apache.thrift.TException
See Also:
HiveMetaStoreClient.dropDatabase(java.lang.String)

loadPartition

public void loadPartition(org.apache.hadoop.fs.Path loadPath,
                          String tableName,
                          AbstractMap<String,String> partSpec,
                          boolean replace,
                          org.apache.hadoop.fs.Path tmpDirPath)
                   throws HiveException
Load a directory into a Hive Table Partition - Alters existing content of the partition with the contents of loadPath. - If he partition does not exist - one is created - files in loadPath are moved into Hive. But the directory itself is not removed.

Parameters:
loadPath - Directory containing files to load into Table
tableName - name of table to be loaded.
partSpec - defines which partition needs to be loaded
replace - if true - replace files in the partition, otherwise add files to the partition
tmpDirPath - The temporary directory.
Throws:
HiveException

loadTable

public void loadTable(org.apache.hadoop.fs.Path loadPath,
                      String tableName,
                      boolean replace,
                      org.apache.hadoop.fs.Path tmpDirPath)
               throws HiveException
Load a directory into a Hive Table. - Alters existing content of table with the contents of loadPath. - If table does not exist - an exception is thrown - files in loadPath are moved into Hive. But the directory itself is not removed.

Parameters:
loadPath - Directory containing files to load into Table
tableName - name of table to be loaded.
replace - if true - replace files in the table, otherwise add files to table
tmpDirPath - The temporary directory.
Throws:
HiveException

createPartition

public Partition createPartition(Table tbl,
                                 Map<String,String> partSpec)
                          throws HiveException
Creates a partition.

Parameters:
tbl - table for which partition needs to be created
partSpec - partition keys and their values
Returns:
created partition object
Throws:
HiveException - if table doesn't exist or partition already exists

createPartition

public Partition createPartition(Table tbl,
                                 Map<String,String> partSpec,
                                 org.apache.hadoop.fs.Path location)
                          throws HiveException
Creates a partition

Parameters:
tbl - table for which partition needs to be created
partSpec - partition keys and their values
location - location of this partition
Returns:
created partition object
Throws:
HiveException - if table doesn't exist or partition already exists

getPartition

public Partition getPartition(Table tbl,
                              Map<String,String> partSpec,
                              boolean forceCreate)
                       throws HiveException
Returns partition metadata

Parameters:
tbl - the partition's table
partSpec - partition keys and values
forceCreate - if this is true and partition doesn't exist then a partition is created
Returns:
result partition object or null if there is no partition
Throws:
HiveException

dropPartition

public boolean dropPartition(String db_name,
                             String tbl_name,
                             List<String> part_vals,
                             boolean deleteData)
                      throws HiveException
Throws:
HiveException

getPartitionNames

public List<String> getPartitionNames(String dbName,
                                      String tblName,
                                      short max)
                               throws HiveException
Throws:
HiveException

getPartitions

public List<Partition> getPartitions(Table tbl)
                              throws HiveException
get all the partitions that the table has

Parameters:
tbl - object for which partition is needed
Returns:
list of partition objects
Throws:
HiveException

copyFiles

protected static void copyFiles(org.apache.hadoop.fs.Path srcf,
                                org.apache.hadoop.fs.Path destf,
                                org.apache.hadoop.fs.FileSystem fs)
                         throws HiveException
Throws:
HiveException

replaceFiles

protected static void replaceFiles(org.apache.hadoop.fs.Path srcf,
                                   org.apache.hadoop.fs.Path destf,
                                   org.apache.hadoop.fs.FileSystem fs,
                                   org.apache.hadoop.fs.Path tmppath)
                            throws HiveException
Replaces files in the partition with new data set specifed by srcf. Works by moving files

Parameters:
srcf - Files to be moved. Leaf Directories or Globbed File Paths
destf - The directory where the final data needs to go
fs - The filesystem handle
tmppath - Temporary directory
Throws:
HiveException

getFieldsFromDeserializer

public static List<FieldSchema> getFieldsFromDeserializer(String name,
                                                          Deserializer serde)
                                                   throws HiveException
Throws:
HiveException


Copyright © 2009 The Apache Software Foundation