org.apache.hadoop.hive.ql.metadata
Class Hive

java.lang.Object
  extended by org.apache.hadoop.hive.ql.metadata.Hive

public class Hive
extends Object

The Hive class contains information about this instance of Hive. An instance of Hive represents a set of data in a file system (usually HDFS) organized for easy query processing


Method Summary
 void alterPartition(String tblName, Partition newPart)
          Updates the existing table metadata with the new metadata.
 void alterTable(String tblName, Table newTbl)
          Updates the existing table metadata with the new metadata.
static void closeCurrent()
           
protected static void copyFiles(org.apache.hadoop.fs.Path srcf, org.apache.hadoop.fs.Path destf, org.apache.hadoop.fs.FileSystem fs)
           
 void createDatabase(Database db)
          Create a Database.
 void createDatabase(Database db, boolean ifNotExist)
          Create a database
 Partition createPartition(Table tbl, Map<String,String> partSpec)
          Creates a partition.
 Partition createPartition(Table tbl, Map<String,String> partSpec, org.apache.hadoop.fs.Path location)
          Creates a partition
 void createTable(String tableName, List<String> columns, List<String> partCols, Class<? extends org.apache.hadoop.mapred.InputFormat> fileInputFormat, Class<?> fileOutputFormat)
          Creates a table metdata and the directory for the table data
 void createTable(String tableName, List<String> columns, List<String> partCols, Class<? extends org.apache.hadoop.mapred.InputFormat> fileInputFormat, Class<?> fileOutputFormat, int bucketCount, List<String> bucketCols)
          Creates a table metdata and the directory for the table data
 void createTable(Table tbl)
          Creates the table with the give objects
 void createTable(Table tbl, boolean ifNotExists)
          Creates the table with the give objects
 boolean databaseExists(String dbName)
          Query metadata to see if a database with the given name already exists.
 void dropDatabase(String name)
          Drop a database.
 void dropDatabase(String name, boolean deleteData, boolean ignoreUnknownDb)
          Drop a database
 boolean dropPartition(String db_name, String tbl_name, List<String> part_vals, boolean deleteData)
           
 void dropTable(String tableName)
          Drops table along with the data in it.
 void dropTable(String dbName, String tableName)
          Drops table along with the data in it.
 void dropTable(String dbName, String tableName, boolean deleteData, boolean ignoreUnknownTab)
          Drops the table.
static Hive get()
           
static Hive get(HiveConf c)
          Gets hive object for the current thread.
static Hive get(HiveConf c, boolean needsRefresh)
          get a connection to metastore.
 List<String> getAllDatabases()
          Get all existing database names.
 List<String> getAllTables()
          Get all table names for the current database.
 List<String> getAllTables(String dbName)
          Get all table names for the specified database.
 HiveConf getConf()
           
 String getCurrentDatabase()
          Get the name of the current database
 List<String> getDatabasesByPattern(String databasePattern)
          Get all existing databases that match the given pattern.
static List<FieldSchema> getFieldsFromDeserializer(String name, Deserializer serde)
           
 Partition getPartition(Table tbl, Map<String,String> partSpec, boolean forceCreate)
          Returns partition metadata
 List<String> getPartitionNames(String dbName, String tblName, Map<String,String> partSpec, short max)
           
 List<String> getPartitionNames(String dbName, String tblName, short max)
           
 List<Partition> getPartitions(Table tbl)
          get all the partitions that the table has
 List<Partition> getPartitions(Table tbl, Map<String,String> partialPartSpec)
          get all the partitions of the table that matches the given partial specification.
 Table getTable(String tableName)
          Returns metadata for the table named tableName in the current database.
 Table getTable(String dbName, String tableName)
          Returns metadata of the table
 Table getTable(String dbName, String tableName, boolean throwException)
          Returns metadata of the table
 List<String> getTablesByPattern(String tablePattern)
          Returns all existing tables from default database which match the given pattern.
 List<String> getTablesByPattern(String dbName, String tablePattern)
          Returns all existing tables from the specified database which match the given pattern.
 List<String> getTablesForDb(String database, String tablePattern)
          Returns all existing tables from the given database which match the given pattern.
 ArrayList<LinkedHashMap<String,String>> loadDynamicPartitions(org.apache.hadoop.fs.Path loadPath, String tableName, Map<String,String> partSpec, boolean replace, org.apache.hadoop.fs.Path tmpDirPath, int numDP)
          Given a source directory name of the load path, load all dynamically generated partitions into the specified table and return a list of strings that represent the dynamic partition paths.
 void loadPartition(org.apache.hadoop.fs.Path loadPath, String tableName, Map<String,String> partSpec, boolean replace, org.apache.hadoop.fs.Path tmpDirPath)
          Load a directory into a Hive Table Partition - Alters existing content of the partition with the contents of loadPath.
 void loadTable(org.apache.hadoop.fs.Path loadPath, String tableName, boolean replace, org.apache.hadoop.fs.Path tmpDirPath)
          Load a directory into a Hive Table.
protected static void replaceFiles(org.apache.hadoop.fs.Path srcf, org.apache.hadoop.fs.Path destf, org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path tmppath)
          Replaces files in the partition with new data set specifed by srcf.
 void setCurrentDatabase(String currentDatabase)
          Set the name of the current database
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Method Detail

get

public static Hive get(HiveConf c)
                throws HiveException
Gets hive object for the current thread. If one is not initialized then a new one is created If the new configuration is different in metadata conf vars then a new one is created.

Parameters:
c - new Hive Configuration
Returns:
Hive object for current thread
Throws:
HiveException

get

public static Hive get(HiveConf c,
                       boolean needsRefresh)
                throws HiveException
get a connection to metastore. see get(HiveConf) function for comments

Parameters:
c - new conf
needsRefresh - if true then creates a new one
Returns:
The connection to the metastore
Throws:
HiveException

get

public static Hive get()
                throws HiveException
Throws:
HiveException

closeCurrent

public static void closeCurrent()

createDatabase

public void createDatabase(Database db,
                           boolean ifNotExist)
                    throws AlreadyExistsException,
                           HiveException
Create a database

Parameters:
db -
ifNotExist - if true, will ignore AlreadyExistsException exception
Throws:
AlreadyExistsException
HiveException

createDatabase

public void createDatabase(Database db)
                    throws AlreadyExistsException,
                           HiveException
Create a Database. Raise an error if a database with the same name already exists.

Parameters:
db -
Throws:
AlreadyExistsException
HiveException

dropDatabase

public void dropDatabase(String name)
                  throws HiveException,
                         NoSuchObjectException
Drop a database.

Parameters:
name -
Throws:
NoSuchObjectException
HiveException
See Also:
HiveMetaStoreClient.dropDatabase(java.lang.String)

dropDatabase

public void dropDatabase(String name,
                         boolean deleteData,
                         boolean ignoreUnknownDb)
                  throws HiveException,
                         NoSuchObjectException
Drop a database

Parameters:
name -
deleteData -
ignoreUnknownDb - if true, will ignore NoSuchObjectException
Throws:
HiveException
NoSuchObjectException

createTable

public void createTable(String tableName,
                        List<String> columns,
                        List<String> partCols,
                        Class<? extends org.apache.hadoop.mapred.InputFormat> fileInputFormat,
                        Class<?> fileOutputFormat)
                 throws HiveException
Creates a table metdata and the directory for the table data

Parameters:
tableName - name of the table
columns - list of fields of the table
partCols - partition keys of the table
fileInputFormat - Class of the input format of the table data file
fileOutputFormat - Class of the output format of the table data file
Throws:
HiveException - thrown if the args are invalid or if the metadata or the data directory couldn't be created

createTable

public void createTable(String tableName,
                        List<String> columns,
                        List<String> partCols,
                        Class<? extends org.apache.hadoop.mapred.InputFormat> fileInputFormat,
                        Class<?> fileOutputFormat,
                        int bucketCount,
                        List<String> bucketCols)
                 throws HiveException
Creates a table metdata and the directory for the table data

Parameters:
tableName - name of the table
columns - list of fields of the table
partCols - partition keys of the table
fileInputFormat - Class of the input format of the table data file
fileOutputFormat - Class of the output format of the table data file
bucketCount - number of buckets that each partition (or the table itself) should be divided into
Throws:
HiveException - thrown if the args are invalid or if the metadata or the data directory couldn't be created

alterTable

public void alterTable(String tblName,
                       Table newTbl)
                throws InvalidOperationException,
                       HiveException
Updates the existing table metadata with the new metadata.

Parameters:
tblName - name of the existing table
newTbl - new name of the table. could be the old name
Throws:
InvalidOperationException - if the changes in metadata is not acceptable
org.apache.thrift.TException
HiveException

alterPartition

public void alterPartition(String tblName,
                           Partition newPart)
                    throws InvalidOperationException,
                           HiveException
Updates the existing table metadata with the new metadata.

Parameters:
tblName - name of the existing table
newTbl - new name of the table. could be the old name
Throws:
InvalidOperationException - if the changes in metadata is not acceptable
org.apache.thrift.TException
HiveException

createTable

public void createTable(Table tbl)
                 throws HiveException
Creates the table with the give objects

Parameters:
tbl - a table object
Throws:
HiveException

createTable

public void createTable(Table tbl,
                        boolean ifNotExists)
                 throws HiveException
Creates the table with the give objects

Parameters:
tbl - a table object
ifNotExists - if true, ignore AlreadyExistsException
Throws:
HiveException

dropTable

public void dropTable(String tableName)
               throws HiveException
Drops table along with the data in it. If the table doesn't exist then it is a no-op

Parameters:
dbName - database where the table lives
tableName - table to drop
Throws:
HiveException - thrown if the drop fails

dropTable

public void dropTable(String dbName,
                      String tableName)
               throws HiveException
Drops table along with the data in it. If the table doesn't exist then it is a no-op

Parameters:
dbName - database where the table lives
tableName - table to drop
dbName - database where the table lives
tableName - table to drop
Throws:
HiveException - thrown if the drop fails Drops table along with the data in it. If the table doesn't exist then it is a no-op
HiveException - thrown if the drop fails

dropTable

public void dropTable(String dbName,
                      String tableName,
                      boolean deleteData,
                      boolean ignoreUnknownTab)
               throws HiveException
Drops the table.

Parameters:
tableName -
deleteData - deletes the underlying data along with metadata
ignoreUnknownTab - an exception if thrown if this is falser and table doesn't exist
Throws:
HiveException

getConf

public HiveConf getConf()

getTable

public Table getTable(String tableName)
               throws HiveException
Returns metadata for the table named tableName in the current database.

Parameters:
tableName - the name of the table
Returns:
Throws:
HiveException - if there's an internal error or if the table doesn't exist

getTable

public Table getTable(String dbName,
                      String tableName)
               throws HiveException
Returns metadata of the table

Parameters:
dbName - the name of the database
tableName - the name of the table
Returns:
the table
Throws:
HiveException - if there's an internal error or if the table doesn't exist

getTable

public Table getTable(String dbName,
                      String tableName,
                      boolean throwException)
               throws HiveException
Returns metadata of the table

Parameters:
dbName - the name of the database
tableName - the name of the table
throwException - controls whether an exception is thrown or a returns a null
Returns:
the table or if throwException is false a null value.
Throws:
HiveException

getAllTables

public List<String> getAllTables()
                          throws HiveException
Get all table names for the current database.

Returns:
List of table names
Throws:
HiveException

getAllTables

public List<String> getAllTables(String dbName)
                          throws HiveException
Get all table names for the specified database.

Parameters:
dbName -
Returns:
List of table names
Throws:
HiveException

getTablesByPattern

public List<String> getTablesByPattern(String tablePattern)
                                throws HiveException
Returns all existing tables from default database which match the given pattern. The matching occurs as per Java regular expressions

Parameters:
tablePattern - java re pattern
Returns:
list of table names
Throws:
HiveException

getTablesByPattern

public List<String> getTablesByPattern(String dbName,
                                       String tablePattern)
                                throws HiveException
Returns all existing tables from the specified database which match the given pattern. The matching occurs as per Java regular expressions.

Parameters:
dbName -
tablePattern -
Returns:
list of table names
Throws:
HiveException

getTablesForDb

public List<String> getTablesForDb(String database,
                                   String tablePattern)
                            throws HiveException
Returns all existing tables from the given database which match the given pattern. The matching occurs as per Java regular expressions

Parameters:
database - the database name
tablePattern - java re pattern
Returns:
list of table names
Throws:
HiveException

getAllDatabases

public List<String> getAllDatabases()
                             throws HiveException
Get all existing database names.

Returns:
List of database names.
Throws:
HiveException

getDatabasesByPattern

public List<String> getDatabasesByPattern(String databasePattern)
                                   throws HiveException
Get all existing databases that match the given pattern. The matching occurs as per Java regular expressions

Parameters:
databasePattern - java re pattern
Returns:
list of database names
Throws:
HiveException

databaseExists

public boolean databaseExists(String dbName)
                       throws HiveException
Query metadata to see if a database with the given name already exists.

Parameters:
dbName -
Returns:
true if a database with the given name already exists, false if does not exist.
Throws:
HiveException

loadPartition

public void loadPartition(org.apache.hadoop.fs.Path loadPath,
                          String tableName,
                          Map<String,String> partSpec,
                          boolean replace,
                          org.apache.hadoop.fs.Path tmpDirPath)
                   throws HiveException
Load a directory into a Hive Table Partition - Alters existing content of the partition with the contents of loadPath. - If he partition does not exist - one is created - files in loadPath are moved into Hive. But the directory itself is not removed.

Parameters:
loadPath - Directory containing files to load into Table
tableName - name of table to be loaded.
partSpec - defines which partition needs to be loaded
replace - if true - replace files in the partition, otherwise add files to the partition
tmpDirPath - The temporary directory.
Throws:
HiveException

loadDynamicPartitions

public ArrayList<LinkedHashMap<String,String>> loadDynamicPartitions(org.apache.hadoop.fs.Path loadPath,
                                                                     String tableName,
                                                                     Map<String,String> partSpec,
                                                                     boolean replace,
                                                                     org.apache.hadoop.fs.Path tmpDirPath,
                                                                     int numDP)
                                                              throws HiveException
Given a source directory name of the load path, load all dynamically generated partitions into the specified table and return a list of strings that represent the dynamic partition paths.

Parameters:
loadPath -
tableName -
partSpec -
replace -
tmpDirPath -
numSp: - number of static partitions in the partition spec
Returns:
Throws:
HiveException

loadTable

public void loadTable(org.apache.hadoop.fs.Path loadPath,
                      String tableName,
                      boolean replace,
                      org.apache.hadoop.fs.Path tmpDirPath)
               throws HiveException
Load a directory into a Hive Table. - Alters existing content of table with the contents of loadPath. - If table does not exist - an exception is thrown - files in loadPath are moved into Hive. But the directory itself is not removed.

Parameters:
loadPath - Directory containing files to load into Table
tableName - name of table to be loaded.
replace - if true - replace files in the table, otherwise add files to table
tmpDirPath - The temporary directory.
Throws:
HiveException

createPartition

public Partition createPartition(Table tbl,
                                 Map<String,String> partSpec)
                          throws HiveException
Creates a partition.

Parameters:
tbl - table for which partition needs to be created
partSpec - partition keys and their values
Returns:
created partition object
Throws:
HiveException - if table doesn't exist or partition already exists

createPartition

public Partition createPartition(Table tbl,
                                 Map<String,String> partSpec,
                                 org.apache.hadoop.fs.Path location)
                          throws HiveException
Creates a partition

Parameters:
tbl - table for which partition needs to be created
partSpec - partition keys and their values
location - location of this partition
Returns:
created partition object
Throws:
HiveException - if table doesn't exist or partition already exists

getPartition

public Partition getPartition(Table tbl,
                              Map<String,String> partSpec,
                              boolean forceCreate)
                       throws HiveException
Returns partition metadata

Parameters:
tbl - the partition's table
partSpec - partition keys and values
forceCreate - if this is true and partition doesn't exist then a partition is created
Returns:
result partition object or null if there is no partition
Throws:
HiveException

dropPartition

public boolean dropPartition(String db_name,
                             String tbl_name,
                             List<String> part_vals,
                             boolean deleteData)
                      throws HiveException
Throws:
HiveException

getPartitionNames

public List<String> getPartitionNames(String dbName,
                                      String tblName,
                                      short max)
                               throws HiveException
Throws:
HiveException

getPartitionNames

public List<String> getPartitionNames(String dbName,
                                      String tblName,
                                      Map<String,String> partSpec,
                                      short max)
                               throws HiveException
Throws:
HiveException

getPartitions

public List<Partition> getPartitions(Table tbl)
                              throws HiveException
get all the partitions that the table has

Parameters:
tbl - object for which partition is needed
Returns:
list of partition objects
Throws:
HiveException

getPartitions

public List<Partition> getPartitions(Table tbl,
                                     Map<String,String> partialPartSpec)
                              throws HiveException
get all the partitions of the table that matches the given partial specification. partition columns whose value is can be anything should be an empty string.

Parameters:
tbl - object for which partition is needed. Must be partitioned.
Returns:
list of partition objects
Throws:
HiveException

getCurrentDatabase

public String getCurrentDatabase()
Get the name of the current database

Returns:

setCurrentDatabase

public void setCurrentDatabase(String currentDatabase)
Set the name of the current database

Parameters:
currentDatabase -

copyFiles

protected static void copyFiles(org.apache.hadoop.fs.Path srcf,
                                org.apache.hadoop.fs.Path destf,
                                org.apache.hadoop.fs.FileSystem fs)
                         throws HiveException
Throws:
HiveException

replaceFiles

protected static void replaceFiles(org.apache.hadoop.fs.Path srcf,
                                   org.apache.hadoop.fs.Path destf,
                                   org.apache.hadoop.fs.FileSystem fs,
                                   org.apache.hadoop.fs.Path tmppath)
                            throws HiveException
Replaces files in the partition with new data set specifed by srcf. Works by moving files

Parameters:
srcf - Files to be moved. Leaf Directories or Globbed File Paths
destf - The directory where the final data needs to go
fs - The filesystem handle
tmppath - Temporary directory
Throws:
HiveException

getFieldsFromDeserializer

public static List<FieldSchema> getFieldsFromDeserializer(String name,
                                                          Deserializer serde)
                                                   throws HiveException
Throws:
HiveException


Copyright © 2010 The Apache Software Foundation