Hive单节点安装使用(1.00) 2015-02-14 12:00

前提条件

本教程要求已经提前安装好如下服务:
1、Hadoop 2.x
2、MySQL

环境规划

Hive将安装在如下节点上:
ctrl节点

Hive将以独立的Hadoop的OS用户运行,OS用户名为hadoop。

安装

  • 下载、安装

将HIVE解压到/opt/hive目录,将该目录的属主设置为hadoop用户。

1
2
3
4
5
cd /opt
wget http://apache.fayea.com/hive/stable/apache-hive-1.0.0-bin.tar.gz
tar -zxvf apache-hive-1.0.0-bin.tar.gz
mv apache-hive-1.0.0-bin hive
chown -R hadoop:hadoop hive
  • 将hive的bin目录加入PATH变量中

  • 配置文件conf/hbase-env.sh

1
2
3
su - hadoop
cd /opt/hive/conf
mv hive-env.sh.template hive-env.sh

内容如下:

1
2
3
export HADOOP_HOME=/opt/hadoop
export HIVE_HOME=/opt/hive
export HIVE_CONF_DIR=/opt/hive/conf
  • 配置文件hive-site.xml

创建配置文件hive-site.xml:

1
2
su - hadoop
mv hive-default.xml.template hive-site.xml

配置Hive相关目录

  • 配置项说明
1
2
3
4
hive.metastore.warehouse.dir:hive数据在HDFS中的目录   
hive.exec.scratchdir:hive在HDFS中的临时目录   
hive.exec.local.scratchdir:hive的本地临时目录,/tmp/hive  
hive.downloaded.resources.dir: hive下载的本地临时目录, /tmp/hive
  • 在HDFS中创建相关目录:
1
2
3
su - hadoop
hdfs dfs -mkdir /user/hive/
hdfs dfs -mkdir /user/hive/warehouse
  • hive-site.xml
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
<property>
    <name>hive.metastore.warehouse.dir</name>
    <value>/user/hive/warehouse</value>
    <description>location of default database for the warehouse</description>
</property>
<property>
    <name>hive.exec.scratchdir</name>
    <value>/tmp/hive</value>
    <description>HDFS root scratch dir for Hive jobs which gets 
    created with write all (733) 
    permission. For each connecting user, an HDFS scratch dir: 
    ${hive.exec.scratchdir}/< username> is created, with 
    ${hive.scratch.dir.permission}.</description>
</property>
<property>
    <name>hive.exec.local.scratchdir</name>
    <value>/tmp/hive</value>
    <description>Local scratch space for Hive jobs</description>
</property>
<property>
    <name>hive.downloaded.resources.dir</name>
    <value>/tmp/hive</value>
    <description>Temporary local directory for added resources 
    in the remote file system.</description>
</property>

配置Metadata的数据库

  • 配置连接MySQL数据库(hive-site.xml)
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
<property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://localhost:3306/hive?createDatabaseIfNotExist=true</value>
    <description>JDBC connect string for a JDBC metastore</description>
</property>
<property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
    <description>Driver class name for a JDBC metastore</description>
</property>
<property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>hive</value>
    <description>username to use against metastore database</description>
</property>
<property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>hive</value>
    <description>password to use against metastore database</description>
</property>
  • 在MySQL上创建hive数据库
1
2
3
CREATE USER 'hive'@'%' IDENTIFIED BY 'hive';
GRANT ALL PRIVILEGES ON *.* TO 'hive'@'%' WITH GRANT OPTION;
flush privileges;
  • 下载Java连接MySQL的jar文件

将mysql-connector-java-5.1.22-bin.jar上传到/opt/hive/lib/

运行

  • 登录&启动Hive(shell)
1
2
3
hive
> show tables
> CREATE TABLE t_hive (a int, b int, c int) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t';
Tags: #Hive    Post on Hadoop