Hive configuration
-
Upload
shashwat2010 -
Category
Documents
-
view
215 -
download
0
Transcript of Hive configuration
-
7/28/2019 Hive configuration
1/5
Hive configuration
Shashwat ShriparvHive with Hadoop configuration document
-
7/28/2019 Hive configuration
2/5
Requirements
Java 1.6 Hadoop 0.20.x.
Installing Hive from a Stable Release
Start by downloading the most recent stable release of Hive from one of theApache download mirrors :
http://hive.apache.org/releases.html
We can directly download from site or issue command through terminsl to
download as follows :
wget http://ftp.wayne.edu/apache/hive/stable/hive-0.8.1.tar.gz
After downloading complete issue following command :
tar -xzf hive-0.8.1.tar.gz
open hive-0.8.1/conf/hive-env.sh
http://hive.apache.org/releases.htmlhttp://ftp.wayne.edu/apache/hive/stable/hive-0.8.1.tar.gzhttp://hive.apache.org/releases.htmlhttp://ftp.wayne.edu/apache/hive/stable/hive-0.8.1.tar.gz -
7/28/2019 Hive configuration
3/5
# Set HADOOP_HOME to point to a specific hadoop install directoryexport HADOOP_HOME=/home/shashwat/Hadoop/hadoop-0.20.205export HADOOP_CONF_DIR=/home/shashwat/Hadoop/hadoop-0.20.205/conf
# Hive Configuration Directory can be controlled by:export HIVE_CONF_DIR=/home/shashwat/Hadoop/hive-0.7.1/conf
# Folder containing extra ibraries required for hive compilation/execution can becontrolled by:export HIVE_AUX_JARS_PATH=/home/shashwat/Hadoop/hadoop-0.20.205/lib:/home/shashwat/Hadoop/hbase-
0.90.4/lib:/home/shashwat/Hadoop/hive-0.7.1/lib:/home/shashwat/Hadoop/hbase-0.90.4/hbase-0.90.4.jar
-
7/28/2019 Hive configuration
4/5
Then open hive-site.xml, in this file you will find a property calledhive.aux.jars.path as follows :
hive.aux.jars.pathfile:///home/shashwat/Hadoop/hive-0.7.1/lib/hive-hbase-handler-0.7.1.jar,file:///home/shashwat/Hadoop/hive-0.7.1/lib/hbase-0.90.4.jar,file:///home/shashwat/Hadoop/hive-0.7.1/lib/zookeeper-3.3.1.jar
Version compatibility with hadoop and hive :
Hive
Version
Hadoop
VersionConfiguration Parameters
0.4 0.18 --hadoop-version 0.18
0.5 0.20--hadoop-version 0.20 --hive-versions 0.5
--ami-version 1.0
0.5 and 0.7 0.20 --hadoop-version 0.20 --hive-versions 0.5,0.7
-
7/28/2019 Hive configuration
5/5
Hive
Version
Hadoop
VersionConfiguration Parameters
--ami-version 1.0
0.7 0.20--hadoop-version 0.20 --hive-versions 0.7
--ami-version 1.0
0.7.1 0.20
--hadoop-version 0.20 --hive-versions 0.7.1
--ami-version 1.0
0.7.1 0.20.205--hadoop-version 0.20 --hive-versions 0.7.1
--ami-version 2.0
0.7.1.1 0.20.205--hadoop-version 0.20.205 --hive-versions
0.7.1.1 --ami-version 2.0
0.7.1.2 0.20.205--hadoop-version 0.20.205 --hive-versions
0.7.1.2 --ami-version 2.0
0.7.1.3 0.20.205--hadoop-version 0.20.205 --hive-versions
0.7.1.3 --ami-version 2.0
0.7.1.4 0.20.205--hadoop-version 0.20.205 --hive-versions
0.7.1.4 --ami-version 2.0
After configuration complete issue these commmands from terminal :
bin/hive - -service hiveserver : it will start the thrift server :
then open new terminal and issue commmand
bin/hive : this will give you hive terminal where you can execute commandslike
create tableshow tables etc.
More configuration options
Hive Configuration Variables used to interact with Hadoop
Variable Name Description Default Value
hadoop.bin.path
The location of hadoop scriptwhich is used to submit jobs tohadoop when submitting through aseparate jvm.
$HADOOP_HOME/bin/hadoop
hadoop.config.dirThe location of the configurationdirectory of the hadoop installation
$HADOOP_HOME/conf