Step 1: Verifying JAVA Installation
You really want to have Java introduced on your framework prior to introducing Sqoop. Allow us to check Java establishment utilizing the accompanying command −
Assuming Java is now introduced on your framework, you get to see the accompanying reaction −
$ java –version
On the off chance that Java isn't introduced on your framework, then, at that point, follow the means given below.
java version "1.7.0_71"
SE Runtime Environment (build 1.7.0_71-b13>
Client VM (build 25.0-b02, mixed mode>
Follow the basic advances given below to introduce Java on your framework.
Download Java (JDK
by visiting the accompanying connection Java install Then jdk-7u71-linux-x64.tar.gz will be downloaded onto your framework.
For the most part, you can observe the downloaded Java record in the Downloads envelope. Confirm it and concentrate the jdk-7u71-linux-x64.gz record utilizing the accompanying commands.
$ cd Downloads/
$ tar zxf jdk-7u71-linux-x64.gz
To make Java accessible to every one of the users, you need to move it to the area "/usr/nearby/". Open root, and type the accompanying command.
# mv jdk1.7.0_71 /usr/local/java
# exitStep IV:
For setting up PATH and JAVA_HOME factors, add the accompanying orders to ~/.bashrc document.
Presently apply every one of the progressions into this running framework.
$ source ~/.bashrc
Utilize the accompanying command to arrange Java choices −
# alternatives --install /usr/bin/java java usr/local/java/bin/java 2
# alternatives --install /usr/bin/javac javac usr/local/java/bin/javac 2
# alternatives --install /usr/bin/jar jar usr/local/java/bin/jar 2
# alternatives --set java usr/local/java/bin/java
# alternatives --set javac usr/local/java/bin/javac
# alternatives --set jar usr/local/java/bin/jar
Step 2: Verifying Hadoop Installation
Hadoop should be introduced on your framework prior to introducing Sqoop. Allow us to check the Hadoop establishment utilizing the accompanying command −
On the off chance that Hadoop is now introduced on your framework, you will get the accompanying reaction −
$ hadoop version
On the off chance that Hadoop isn't introduced on your framework, then, at that point, continue with the accompanying steps −
Subversion https://svn.apache.org/repos/asf/hadoop/common -r 1529768
Compiled by hortonmu on 2013-10-07T06:28Z
Compiled with protoc 2.5.0
Download and extricate Hadoop 2.4.1 from Apache Software Foundation utilizing the accompanying commands.
# cd /usr/local
# wget http://apache.claz.org/hadoop/common/hadoop-2.4.1/
# tar xzf hadoop-2.4.1.tar.gz
# mv hadoop-2.4.1/* to hadoop/
Installing Hadoop in Pseudo Distributed Mode
Follow the means given below to introduce Hadoop 2.4.1 in pseudo-dispersed mode.
Step 1: Setting up Hadoop You can set Hadoop climate factors by attaching the accompanying orders to ~/.bashrc document.
Presently, apply every one of the progressions into the present running framework.
Step 2: Hadoop Configuration
$ source ~/.bashrc
You can observe all the Hadoop setup records in the area "$HADOOP_HOME/and so on/hadoop". You want to roll out reasonable improvements in those setup documents as per your Hadoop framework.
To faster Hadoop programs utilizing java, you need to reset the java environment factors in hadoop-env.shhadoop-env.sh record by supplanting JAVA_HOME esteem with the area of java in your framework.
$ cd $HADOOP_HOME/etc/hadoop
Given below is the list of documents that you want to alter to arrange Hadoop.
The core-site.xml document contains data, for example, the port number utilized for Hadoop occurrence, memory distributed for the record framework, memory limit for putting away the information, and the size of Read/Write buffers. Open the core-site.xml and add the accompanying properties in the middle the configuration and configuration labels.
The hdfs-site.xml record contains data like the worth of replication information, namenode way, and datanode way of your nearby document frameworks. It implies where you need to store the Hadoop framework. Allow us to expect the accompanying information.
Open this record and add the accompanying properties in the middle the <configuration>, </configuration>labels in this document.
dfs.replication (data replication value>
= 1 (In the following path /hadoop/ is the user name. hadoopinfra/hdfs/namenode is the directory created by hdfs file system.>
namenode path = //home/hadoop/hadoopinfra/hdfs/namenode (hadoopinfra/hdfs/datanode is the directory created by hdfs file system.>
datanode path = //home/hadoop/hadoopinfra/hdfs/datanode
This document is utilized to design yarn into Hadoop. Open the yarn-site.xml record and add the accompanying properties in the middle the <configuration>, </configuration> labels in this document.
This record is utilized to indicate which MapReduce structure we are utilizing. Naturally, Hadoop contains a format of yarn-site.xml. As a matter of first importance, you want to duplicate the document from mapred-site.xml.template to mapred-site.xml record utilizing the accompanying order.
Open mapred-site.xml document and add the accompanying properties in the middle the <configuration>, </configuration> labels in this record.
$ cp mapred-site.xml.template mapred-site.xml
Verifying Hadoop Installation
Step 1 - Name Node Setup
Set up the namenode utilizing the order "hdfs namenode - design" as follows.
Step 2 - Verifying Hadoop dfs
$ cd ~
$ hdfs namenode -format
The accompanying order is utilized to start dfs. Executing this order will begin your Hadoop record framework.
Step 3 - Verifying Yarn Script
The accompanying command is utilized to begin the yarn script. Executing this order will begin your yarn daemons.
Step 4 - Accessing Hadoop on Browser
Verifying Yarn Script
The default port number to get to Hadoop is 50070. Utilize the accompanying URL to get Hadoop administrations on your program.
Step 5 - Verify All Applications for Cluster
The default port number to get to all utilizations of cluster is 8088. Utilize the accompanying url to visit this help.
Step 3:Downloading Sqoop
We can download the most recent adaptation of Sqoop from the accompanying connectionSqoop-download. For this instructional exercise, we are utilizing variant 1.4.5, that is to say, sqoop-1.4.5.bin__hadoop-2.0.4-alpha.tar.gz.
Step 4:Installing Sqoop
The accompanying orders are utilized to remove the Sqoop tar ball and move it to "/usr/lib/sqoop" index.
$tar -xvf sqoop-1.4.4.bin__hadoop-2.0.4-alpha.tar.gz
# mv sqoop-1.4.4.bin__hadoop-2.0.4-alpha /usr/lib/sqoop
Step 5:Configuring bashrc
You need to set up the Sqoop environment by annexing the accompanying lines to ~/.bashrc document −
The accompanying order is utilized to execute ~/.bashrc document.
export SQOOP_HOME=/usr/lib/sqoop export PATH=$PATH:$SQOOP_HOME/bin
$ source ~/.bashrc
Step 6:Configuring Sqoop
To design Sqoop with Hadoop, you really want to alter the sqoop-env.sh record, which is put in the $SQOOP_HOME/conf index. As a matter of first importance, Redirect to Sqoop config index and duplicate the format document utilizing the accompanying order −
Open sqoop-env.sh and alter the accompanying lines −
$ cd $SQOOP_HOME/conf
$ mv sqoop-env-template.sh sqoop-env.sh
Step 7: Download and Configure mysql-connector-java
We can download mysql-connector-java-5.1.30.tar.gz record from the accompanying connection.
The accompanying orders are utilized to extricate mysql-connector-java tarball and move mysql-connector-java-5.1.30-bin.jar to/usr/lib/sqoop/lib registry.
$ tar -zxf mysql-connector-java-5.1.30.tar.gz
# cd mysql-connector-java-5.1.30
# mv mysql-connector-java-5.1.30-bin.jar /usr/lib/sqoop/lib
Step 8: Verifying Sqoop
The accompanying command is utilized to check the Sqoop adaptation..
$ cd $SQOOP_HOME/bin