Apache Hive Install

Overview

Hive is a MapReduce wrapper that allows you to query data stored in Hadoop using SQL.

Installation Steps
Metastore Schema Initialization
Running Hive

Installation Steps

MySQL Installation

mysql-server

sudo apt install mysql-server

MySQL Connector Installation

Download the mysql-connector version compatible with your installed MySQL from mysql-connector-official-download-site.

wget https://downloads.mysql.com/archives/get/p/3/file/mysql-connector-java_8.0.30-1ubuntu22.04_all.deb

dpkg -x mysql-connector-java_8.0.30-1ubuntu22.04_all.deb ./
cp usr/share/java/mysql-connector-java-8.0.30.jar /usr/local/hive/lib/

Hive Installation

Find and download the appropriate Hive version from https://www.apache.org/dyn/closer.cgi/hive/.

wget https://dlcdn.apache.org/hive/hive-3.1.3/apache-hive-3.1.3-bin.tar.gz
tar -zxvf apache-hive-3.1.3-bin.tar.gz
cd -R apache-hive-3.1.3-bin /usr/local/hive

Editing ~/.bashrc

Add the HIVE_HOME environment variable.

export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
export HADOOP_HOME=/usr/local/hadoop
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export YARN_HOME=$HADOOP_HOME
export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
export YARN_CONF_DIR=$HADOOP_HOME/etc/hadoop
export HADOOP_YARN_USER=${HADOOP_YARN_USER:-yarn}
export HIVE_HOME=/usr/local/hive
PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$JAVA_HOME/bin:$HIVE_HOME/bin

Editing hive-env.sh

Set the Hadoop home directory below.

HADOOP_HOME=/usr/local/hadoop

Editing hive-site.xml

Since the hive-site.xml file does not exist initially, copy the hive-default.xml.template file to create hive-site.xml.

Add the following content.

hive-site.xml

 <property>
    <name>system:java.io.tmpdir</name>
    <value>/tmp/hive/java</value>
  </property>
  <property>
    <name>system:user.name</name>
    <value>${user.name}</value>
  </property>

Modify the following content.

hive-site.xml

  <property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
    <description>Driver class name for a JDBC metastore</description>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://ubuntu01:3306/hive?createDatabaseIfNotExist=true</value>
    <description>
      JDBC connect string for a JDBC metastore.
      To use SSL to encrypt/authenticate the connection, provide database-specific SSL flag in the connection URL.
      For example, jdbc:postgresql://myhost/db?ssl=true for postgres database.
    </description>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>my_mysql_password</value>
    <description>password to use against metastore database</description>
  </property>
   <property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>my_mysql_username</value>
    <description>Username to use against metastore database</description>
  </property>

Metastore Schema Initialization

schematool -initSchema -dbType mysql -verbose

Running Hive

/usr/local/hive/bin/hive