Installation of Hadoop 2.6.0 in Ubuntu 14.04 (Single Node Setup)

Installation steps for Hadoop
In this article we will learn how to install hadoop 2.6.0 on ubuntu 14.04, Similar way you can install Hadoop 2.x version on any Linux system.

1. Install Java
Hadoop is developed on java, so Java is very important.

kb@kb:~$ sudo apt-get update
kb@kb:~$ sudo apt-get install openjdk-7-jre
kb@kb:~$ sudo apt-get install openjdk-7-jdk
kb@kb:~$ java -version
2. Configure ssh
kb@kb:~$ ssh-keygen -t rsa -P “”
kb@kb:~$ cat $HOME/.ssh/rsa.pub >> $HOME/.ssh/authorized_keys
kb@kb:~$ ssh localhost
3. Download Hadoop
To download hadoop Click here
4. Extract and move hadoop-2.6.0 folder to /usr/local/
kb@kb:~$ mv hadoop-2.6.0 /usr/local
5.Configure .bashrc file
kb@kb:~$ gedit ~/.bashrc
export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.60-2.4.3.0.fc20.x86_64
export HADOOP_HOME=/usr/local/hadoop-2.6.0
export PATH=$PATH:$HADOOP_HOME/bin
export PATH=$PATH:$HADOOP_HOME/sbin
export HADOOP_MAPRED_HOME=$HADOOP_HOME
export HADOOP_COMMON_HOME=$HADOOP_HOME
export HADOOP_HDFS_HOME=$HADOOP_HOME
export YARN_HOME=$HADOOP_HOME
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export HADOOP_OPTS="-Djava.library.path=$HADOOP_HOME/lib"
Update .bashrc file to apply changes
kb@kb:~$ source ~/.bashrc
Configure the following files
a) Modify hadoop-env.sh

kb@kb:/usr/local/hadoop-2.6.0/etc/hadoop$ gedit hadoop-env.sh

(Find JAVA_HOME and adjust path according to your java installation path)

export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.60-2.4.3.0.fc20.x86_64

b) Modify core-site.xml

kb@kb:/usr/local/hadoop-2.6.0/etc/hadoop$ gedit core-site.xml
<configuration>
 <property>
  <name>fs.default.name</name>
  <value>hdfs://localhost:9000</value>
 </property>
</configuration>

c) Modify mapred-site.xml

kb@kb:/usr/local/hadoop-2.6.0/etc/hadoop$ cp mapred-site.xml.template mapred-site.xml

kb@kb:/usr/local/hadoop-2.6.0/etc/hadoop$ gedit mapred-site.xml
<configuration>
  <property>
    <name>mapreduce.framework.name</name>
    <value>yarn</value>
  </property>
</configuration>

d) Modify hdfs-site.xml

kb@kb:/usr/local/hadoop-2.6.0/etc/hadoop$ gedit hdfs-site.xml
<configuration>
  <property>
    <name>dfs.replication</name>
    <value>1</value>
  </property>
  <property>
    <name>dfs.namenode.name.dir</name>
    <value>file:/usr/local/hadoop_tmp/hdfs/namenode</value>
  </property>
  <property>
    <name>dfs.datanode.data.dir</name>
    <value>file:/usr/local/hadoop_tmp/hdfs/datanode</value>
  </property>
</configuration>

e) Modify yarn-site.xml

kb@kb:/usr/local/hadoop-2.6.0/etc/hadoop$ gedit yarn-site.xml
<configuration>
  <property>
     <name>yarn.nodemanager.aux-services</name>
     <value>mapreduce_shuffle</value>
  </property>
  <property>
    <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
    <value>org.apache.hadoop.mapred.ShuffleHandler</value>
  </property>
</configuration>
8. Format namenode for first time.
kb@kb:/usr/local/hadoop-2.6.0/etc/hadoop$ cd /home/kb
kb@kb:~$ hadoop namenode -format
9. Start hadoop
kb@kb:~$ start-all.sh
10. Check proper installation of hadoop
kb@kb:~$ jps
Namenode
Datanode
NodeMananger
ResourceMananger
SecondaryNamenode
11. Stop hadoop
kb@kb:~$ stop-all.sh
12. Hadoop Web Interface
Start the Hadoop again and see its Web UI:

NameNode Information : http://localhost:50070 

Secondary NameNode Information : http://localhost:50090 

DataNode : http://localhost:50070/dfshealth.html#tab-datanode 

Logs : http://localhost:50070/logs/ 

Hey, finally Hadoop is successfully installed and working. Now you can go ahead for Hello world of Hadoop i.e. Wordcount program. Hopes you enjoyed easy way to install Hadoop.

1 comments:

Post a Comment