Pig 0.14 Installation & Configuration

Apache Pig is a platform which is used to analyze larger sets of data representing them as data flows. Apache Pig is a high-level platform for creating programs that run on Apache Hadoop. Pig's language layer currently consists of a textual language called Pig Latin.

Prerequisites

Java
Hadoop

Download Pig Jar file

Download Pig 0.14 using below command on /usr/local directory.

$sudo wget http://archive.apache.org/dist/pig/pig-0.14.0/pig-0.14.0.tar.gz /usr/local

Unpack pig-0.14.0.tar.gz file

$ cd /usr/local $ sudo tar -xvf pig-0.14.0.tar.gz

Rename pig-0.14.0 folder to pig

$ sudo mv pig-0.14.0 pig

Setting up environment for Pig

Edit ~/.bashrc file for set up the Hive environment by appending the following lines

$ sudo nano ~/.bashrc

export PIG_HOME=/usr/local/pig export PATH=$PIG_HOME/bin:$PATH

Reload the configuration file ~/.bashrc with the following command.

$ source ~/.bashrc

Change the ownership and permissions of the folger /usr/local/pig

$ sudo chown -R hdfs:hdfs /usr/local/pig $ sudo chmod -R 755 /usr/local/pig

Check the version

Check the version of pig.

$ pig --version Apache Pig version 0.14.0 (r1640057) compiled Nov 16 2014, 18:02:05

Apache Pig Execution Modes

You can run Apache Pig in two modes, namely, Local Mode and MapReduce Mode.
Local Mode

$ pig -x local Grunt>

MapReduce Mode

$ pig -x mapreduce Grunt>

Read More

Java Installation & Configuration

Java is a widely used programming language expressly designed for use in the distributed environment of the internet. Java can be used to create complete applications that may run on a single computer or be distributed among servers and clients in a network.

Downloading Oracle Java JDK

Download Java (JDK) source tarball files for your system architecture by going to official (http://www.oracle.com/technetwork/java/javase/downloads) page and kept on path/opt/ directory.

JDK: Java Development Kit. Includes a complete JRE plus tools for developing, debugging, and monitoring Java applications.

Server JRE: Java Runtime Environment. For deploying Java applications on servers. Includes tools for JVM monitoring and tools commonly required for server applications.

In this tutorial we will be installing the JDK Java SE Development Kit 8 x64 bits. Accept the license and copy the download link into your clipboard. Remember to choose the right tar.gz (64 or 32 bits).

jdk-8u131-linux-i586.tar.gz   [For 32-bit System] jdk-8u131-linux-x64.tar.gz    [For 64-bit System]

Note: Always get the latest version from Oracle's website

Installing Java

Unpack the compressed jdk (/opt/jdk-8u131-linux-x64.tar.gz) file by using this command

# cd /opt # tar –xvzf jdk-8u131-linux-x64.tar.gz

Rename jdk-8u131 directory to jdk in /opt directory by using give command

# mv jdk-8u131 /opt/jdk

Setting up Java Environment Variables

First we need to set environment variable for java. Edit ~/.bashrc file

# nano ~/.bashrc

Append following values at end of file and save the file.

export JAVA_HOME=/opt/jdk export PATH=$JAVA_HOME/bin:$PATH

Reload the configuration file ~/.bashrc with the following command.

# source ~/.bashrc

Verify the Java version

# java -version

java version "1.8.0_131" Java(TM) SE Runtime Environment (build 1.8.0_131-b13) Java HotSpot(TM) 64-Bit Server VM (build 25.131-b13, mixed mode)

Read More

Installation and Configuration guide

Installation and Configuration guide for Hadoop Eco-system and other Big-Data Technologies

👉How to Install Java 8 on CentOS
👉How to Install MySql on CentOS
👉How to Install Hadoop 2.6 with Single Node on CentOS
👉How to Install Hive 0.14 with Single Node on CentOS
👉How to Install Pig 0.14 with Single Node on CentOS
👉How to Install Sqoop 1.4.4 with Single Node on CentOS
👉How to Install Flume 1.4.0 with Single Node on CentOS
👉How to Install HBase 0.98.4 with Single Node on CentOS
👉How to Install Drill 1.10.0 with Single Node on CentOS
👉How to Install Spark 1.6.1 with Single Node on CentOS
👉How to Install Zookeeper 3.4.6 with Single Node on CentOS
👉How to Install NiFi 1.3.0 with Single Node on CentOS

Read More

MongoDB Installation & Configuration on Ubuntu 64 Bit

MongoDB is Document-oriented database system. It is part of the NoSQL family of database systems. Instead of storing data in tables as is done in a "classical" relational database, MongoDB stores structured data as JSON-like documents.

  

Prerequisites

1. Java

2. Linux 64 Bit OS (Ubuntu/Debian)

Installing MongoDB

Import the public key used by the package management system.

$sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 –recv 0C49F3730359A14518585931BC711F9BA15703C6

Create the /etc/apt/sources.list.d/mongodb-org-3.4.list list file using the command appropriate for your version of Ubuntu.

Ubuntu 12.04: $sudo echo "deb [ arch=amd64 ] http://repo.mongodb.org/apt/ubuntu precise/mongodb-org/3.4 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.4.list Ubuntu 14.04: $sudo echo "deb [ arch=amd64 ] http://repo.mongodb.org/apt/ubuntu trusty/mongodb-org/3.4 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.4.list Ubuntu 16.04: $sudo echo "deb [ arch=amd64,arm64 ] http://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/3.4 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.4.list.

Reload local package database

$sudo apt-get update

Reload Install the MongoDB packages. (Install the latest stable version of MongoDB).

$sudo apt-get install -y mongodb-org

Start MongoDB

$sudo service mongod start

Verify that MongoDB has started successfully

$sudo service mongod status

Check below file successfully created or not.

$cd /var/log/mongodb/ $ls –l mongod.log $cd /etc $ls -l mongod.conf

Stop Mongodb service and modify mongod.conf file in case of need to add Ip address. (Default bindIp: 127.0.0.1)

$sudo service mongod stop $sudo nano /etc/mongod.conf

storage:   dbPath: /var/lib/mongodb   journal:   enabled: true systemLog:   destination: file   logAppend: true   path: /var/log/mongodb/mongod.log net:   port: 27017   bindIp: 127.0.0.1,localhost

Change the read & write permission of data directory

$sudo chmod 776 /var/lib/mongodb/*

Check Mongodb start with both IP addresses.

$sudo service mongod start $netstat -nlp | grep 27017 tcp        0      0 localhost:27017      0.0.0.0:* LISTEN      - tcp        0      0 127.0.0.1:27017         0.0.0.0:* LISTEN      -

Enter into Mongodb shell with IP address.

$sudo mongo localhost:27017 MongoDB shell version v3.4.3 connecting to: localhost:27017 MongoDB server version: 3.4.3 >

Read More