Sqoop 1.4.4 Installation & Configuration

Apache Sqoop is a tool designed for efficiently transferring bulk data between Apache Hadoop and relational databases. Sqoop is used to import data from RDBMS to Hadoop Distributed File System or  Hadoop eco-systems like Hive and HBase. Similarly, Sqoop can also be used to extract data from Hadoop to relational databases like oracle, Mysql etc.

Prerequisite:

• Hadoop
• MySql Database 

Download Sqoop 1.4.4 using below command on /usr/local directory.

$sudo wget http://archive.apache.org/dist/sqoop/1.4.4/ sqoop-1.4.4.bin__hadoop-2.0.4-alpha.tar.gz /usr/local

Unpack sqoop-1.4.4.bin__hadoop-2.0.4-alpha.tar.gz file

$ cd /usr/local $ sudo tar -xvf sqoop-1.4.4.bin__hadoop-2.0.4-alpha.tar.gz

Rename sqoop-1.4.4.bin__hadoop-2.0.4-alpha to sqoop

$ sudo mv sqoop-1.4.4.bin__hadoop-2.0.4-alpha sqoop

Setting up environment for Sqoop

Edit ~/.bashrc file for set up the Pig environment by appending the following lines.

$ sudo nano ~/.bashrc

export SQOOP_HOME=/usr/local/sqoop export PATH=$SQOOP_HOME/bin:$PATH

Reload the configuration file ~/.bashrc with the following command.

# source ~/.bashrc

Sqoop can be connected to various types of databases. It uses JDBC to connect to them. JDBC driver for each of databases is needed by sqoop to connect to them. In this setup we are handling with MySQL connector to connect MySQL DB.

Download below jar files in /usr/local/sqoop path

mysql-connector-java-5.1.25.jar sqljdbc4.jar

Copy above jar files from /usr/local/sqoop to /usr/local/sqoop/lib/ folder

$sudo cp /usr/local/sqoop/mysql-connector-java-5.1.28.jar /usr/local/sqoop/lib/ $sudo cp /usr/local/sqoop/sqljdbc4.jar /usr/local/sqoop/lib/

Change the ownership and permissions of the directory /usr/local/sqoop

$ sudo chown -R hdfs:hdfs /usr/local/sqoop $ sudo chmod -R 755 /usr/local/sqoop

Check the version of sqoop.

$ sqoop version Sqoop 1.4.4 git commit id 050a2015514533bc25f3134a33401470ee9353ad Compiled by vasanthkumar on Mon Jul 22 20:06:06 IST 2013