hadoop error coming when starting hadoop - hadoop

Hi i can't resolve my problem when running hadoop with start-all.sh
rochdi#127:~$ start-all.sh
/usr/local/hadoop/bin/hadoop-daemon.sh: line 62: [: localhost: integer
expression expected
starting namenode, logging to
/usr/local/hadoop/libexec/../logs/hadoop-rochdi-namenode-127.0.0.1
localhost: /usr/local/hadoop/bin/hadoop-daemon.sh: line 62: [:
localhost: integer expression expected
localhost: starting datanode, logging to
/usr/local/hadoop/libexec/../logs/hadoop-rochdi-datanode-127.0.0.1
localhost: /usr/local/hadoop/bin/hadoop-daemon.sh: line 62: [:
localhost: integer expression expected
localhost: starting secondarynamenode, logging to
/usr/local/hadoop/libexec/../logs/hadoop-rochdi-secondarynamenode-127.0.0.1
/usr/local/hadoop/bin/hadoop-daemon.sh: line 62: [: localhost: integer
expression expected
starting jobtracker, logging to
/usr/local/hadoop/libexec/../logs/hadoop-rochdi-jobtracker-127.0.0.1
localhost: /usr/local/hadoop/bin/hadoop-daemon.sh: line 62: [:
localhost: integer expression expected
localhost: starting tasktracker, logging to
/usr/local/hadoop/libexec/../logs/hadoop-rochdi-tasktracker-127.0.0.1
localhost: Erreur : impossible de trouver ou charger la classe
principale localhost
path:
rochdi#127:~$ echo "$PATH"
/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/usr/local/hadoop/bin:/usr/local/hadoop/lib
before coming error i change hostname file as:
127.0.0.1 localhost
127.0.1.1 ubuntu.local ubuntu
and i configured my bashrc file as
export HADOOP_PREFIX=/usr/local/hadoop
export PATH=$PATH:$HADOOP_PREFIX/bin
export JAVA_HOME=/usr/lib/jvm/java-7-oracle
and jps command
rochdi#127:~$ jps
3427 Jps
help me please

i resolve the problem i just change my hostname but and all nodes start but when i stop them i have this message:
rochdi#acer:~$ jps
4605 NameNode
5084 SecondaryNameNode
5171 JobTracker
5460 Jps
5410 TaskTracker
rochdi#acer:~$ stop-all.sh
stopping jobtracker
localhost: no tasktracker to stop
stopping namenode
localhost: no datanode to stop
localhost: stopping secondarynamenode

after extract hadoop tar file
open ~/bashrc file and add following at the end of file
export HADOOP_HOME=/usr/local/hadoop
export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib/native
export PATH=$PATH:$HADOOP_HOME/sbin:$HADOOP_HOME/bin
export HADOOP_INSTALL=$HADOOP_HOME
then,
edit file $HADOOP_HOME/etc/hadoop/core-site.xml add following config then start hadoop
<configuration>
<property>
<name>fs.default.name </name>
<value> hdfs://localhost:9000 </value>
</property>
</configuration>
still problem the use this link click here

Check your hosts file and *-site.xml files for host names. This error occurs when the hostnames not defined properly.

Use server IP address instead of using localhost in core-site.xml and check your entries in etc/hosts and slaves file.

The OP posted that the main error was fixed by changing the hostname (answered Dec 6 '13 at 14:19). It suggests issues with the file /etc/hosts and the file 'slaves' in the master. Remember that each host name in the cluster must match the values in those files. When an xml is wrongly configured, it normally throw up Connectivity issues between the ports of the services.
From the message "no ${SERVICE} to stop" most probably the previous start-all.sh leaved the java processes orphaned. The solutions is to stop manually each process, e.g.
$ kill -9 4605
and then execute again the start-all.sh command
It's important to mention that this is an old question, and currently we have the version 2 and 3 of Hadoop. And I strongly recommend using on of the latest version.

Related

Failed to retrieve data from /webhdfs/v1/?op=LISTSTATUS: Server Error

vijay#ubuntu:~$ start-all.sh
WARNING: Attempting to start all Apache Hadoop daemons as vijay in 10 seconds.
WARNING: This is not a recommended production deployment configuration.
WARNING: Use CTRL-C to abort.
Starting namenodes on [localhost]
localhost: namenode is running as process 22733. Stop it first and ensure /tmp/hadoop-vijay-namenode.pid file is empty before retry.
Starting datanodes
localhost: datanode is running as process 22866. Stop it first and ensure /tmp/hadoop-vijay-datanode.pid file is empty before retry.
Starting secondary namenodes [ubuntu]
ubuntu: secondarynamenode is running as process 23072. Stop it first and ensure /tmp/hadoop-vijay-secondarynamenode.pid file is empty before retry.
Starting resourcemanager
Starting nodemanagers
vijay#ubuntu:~$ jps
23072 SecondaryNameNode
22866 DataNode
22733 NameNode
24447 Jps
enter image description here
I am facing hadoop web console error
Currently installed java version "19.0.1" 2022-10-18 and Hadoop 3.3.4

Error in loading hadoop distributed file system

I installed Hadoop-3.3.4 in Ubuntu-20. I wrote the command for starting hadoop, i.e.
samar#pc:~$ $HADOOP_HOME/sbin/start-all.sh
Then it showed the output as.
WARNING: Attempting to start all Apache Hadoop daemons as samar in 10 seconds.
WARNING: This is not a recommended production deployment configuration.
WARNING: Use CTRL-C to abort.
Starting namenodes on [localhost]
Starting datanodes
Starting secondary namenodes [pc]
Starting resourcemanager
Starting nodemanagers
But when I tried to access the HDFS with the command
samar#pc:~$ hdfs dfs -ls
It gave a message as:
ls: Call From pc/127.0.1.1 to localhost:9000 failed on connection exception:
java.net.ConnectException: Connection refused; For more details see:
http://wiki.apache.org/hadoop/ConnectionRefused
and the output of jps was:
10485 Jps
10101 NodeManager
9946 ResourceManager
9739 SecondaryNameNode
9533 DataNode
Namenode did not start successfully (9000 is namenodes services port)
Are there more logs?

Everything else starting on Hadoop pseudo-distributed except namenode

I have Hadoop 2.9.0 on Ubuntu at
/usr/local/hadoop
But when I try start-dfs.sh
No error is shown while starting namenode
But when I type jps, only
10900 SecondaryNameNode
11047 Jps
10696 DataNode
Seams to have started, not namenode
Things tried:
=> Removed temp files and formatted namenode hadoop namenode -format
terminal:
blaze#blazian:/tmp$ start-dfs.sh
Starting namenodes on [localhost]
blaze#localhost's password:
localhost: starting namenode, logging to /usr/local/hadoop/logs/hadoop-blaze-namenode-blazian.out
blaze#localhost's password:
localhost: starting datanode, logging to /usr/local/hadoop/logs/hadoop-blaze-datanode-blazian.out
Starting secondary namenodes [0.0.0.0]
blaze#0.0.0.0's password:
0.0.0.0: starting secondarynamenode, logging to /usr/local/hadoop/logs/hadoop-blaze-secondarynamenode-blazian.out
blaze#blazian:/tmp$ jps
10900 SecondaryNameNode
11047 Jps
10696 DataNode
You don't have a SSH setup with localhost. Please follow these steps and you'll be able to run the namenode.
Go to your system terminal and type:
cd(It will redirect you to ~)
ssh-keygen(hit enter three times and it will create a .ssh directory in ~)
cat id_rsa.pub >> authorized_keys(it will make sure that your localhost is the trusted source and give permission to make
passwordless ssh.)
Then simply run start-all.sh and you're all set.

How to start hadoop without asking local machine password?

amtex#amtex-desktop:~$ start-all.sh
This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
Starting namenodes on [localhost]
amtex#localhost's password:
localhost: starting namenode, logging to /home/amtex/Documents/installed/hadoop/logs/hadoop-amtex-namenode-amtex-desktop.out
amtex#localhost's password:
localhost: starting datanode, logging to /home/amtex/Documents/installed/hadoop/logs/hadoop-amtex-datanode-amtex-desktop.out
Starting secondary namenodes [0.0.0.0]
amtex#0.0.0.0's password:
0.0.0.0: starting secondarynamenode, logging to /home/amtex/Documents/installed/hadoop/logs/hadoop-amtex-secondarynamenode-amtex-desktop.out
starting yarn daemons
starting resourcemanager, logging to /home/amtex/Documents/installed/hadoop/logs/yarn-amtex-resourcemanager-amtex-desktop.out
amtex#localhost's password:
localhost: starting nodemanager, logging to /home/amtex/Documents/installed/hadoop/logs/yarn-amtex-nodemanager-amtex-desktop.out
amtex#amtex-desktop:~$ jps
2404 Startup
18244 DataNode
18580 ResourceManager
18101 NameNode
18889 NodeManager
18425 SecondaryNameNode
18924 Jps
You need set password less login between machine
Below link has the step by step procedure to setup ssh password less login
http://www.tecmint.com/ssh-passwordless-login-using-ssh-keygen-in-5-easy-steps/
Hope this helps!!!...
Based on my research i follow these steps to avoid the above problem
step 1: ssh-keygen -t rsa -P ""
step 2: cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys
and now i started hadoop
amtex#amtex-desktop:~$ start-all.sh
This script is Deprecated. Instead use start-dfs.sh and start-yarn.sh
Starting namenodes on [localhost]
localhost: starting namenode, logging to /home/amtex/Documents/installed/hadoop/logs/hadoop-amtex-namenode-amtex-desktop.out
localhost: starting datanode, logging to /home/amtex/Documents/installed/hadoop/logs/hadoop-amtex-datanode-amtex-desktop.out
Starting secondary namenodes [0.0.0.0]
0.0.0.0: starting secondarynamenode, logging to /home/amtex/Documents/installed/hadoop/logs/hadoop-amtex-secondarynamenode-amtex-desktop.out
starting yarn daemons
starting resourcemanager, logging to /home/amtex/Documents/installed/hadoop/logs/yarn-amtex-resourcemanager-amtex-desktop.out
localhost: starting nodemanager, logging to /home/amtex/Documents/installed/hadoop/logs/yarn-amtex-nodemanager-amtex-desktop.out
amtex#amtex-desktop:~$ start-master.sh
starting org.apache.spark.deploy.master.Master, logging to /home/amtex/Documents/installed/spark/logs/spark-amtex-org.apache.spark.deploy.master.Master-1-amtex-desktop.out
amtex#amtex-desktop:~$ start-slaves.sh
localhost: starting org.apache.spark.deploy.worker.Worker, logging to /home/amtex/Documents/installed/spark/logs/spark-amtex-org.apache.spark.deploy.worker.Worker-1-amtex-desktop.out
amtex#amtex-desktop:~$ jps
21523 Jps
2404 Startup
21029 NodeManager
20581 DataNode
20439 NameNode
20760 SecondaryNameNode
21353 Master
21466 Worker
20911 ResourceManager

error when i start job in hadoop

j just want to install hadoop single node mode in ubuntu i formatted hdfs succesfully but when i start the job with start-all.sh I am getting error message as
rochdi#127:~$ start-all.sh
/usr/local/hadoop/bin/hadoop-daemon.sh: line 62: [: localhost: integer expression expected
starting namenode, logging to /usr/local/hadoop/libexec/../logs/hadoop-rochdi-namenode-127.0.0.1
localhost: /usr/local/hadoop/bin/hadoop-daemon.sh: line 62: [: localhost: integer expression expected
localhost: starting datanode, logging to /usr/local/hadoop/libexec/../logs/hadoop-rochdi-datanode-127.0.0.1
localhost: /usr/local/hadoop/bin/hadoop-daemon.sh: line 62: [: localhost: integer expression expected
localhost: starting secondarynamenode, logging to /usr/local/hadoop/libexec/../logs/hadoop-rochdi-secondarynamenode-127.0.0.1
/usr/local/hadoop/bin/hadoop-daemon.sh: line 62: [: localhost: integer expression expected
starting jobtracker, logging to /usr/local/hadoop/libexec/../logs/hadoop-rochdi-jobtracker-127.0.0.1
localhost: /usr/local/hadoop/bin/hadoop-daemon.sh: line 62: [: localhost: integer expression expected
localhost: starting tasktracker, logging to /usr/local/hadoop/libexec/../logs/hadoop-rochdi-tasktracker-127.0.0.1
localhost: Erreur : impossible de trouver ou charger la classe principale localhost
and this is my bashrc config file
export HADOOP_PREFIX=/usr/local/hadoop
export PATH=$PATH:$HADOOP_PREFIX/bin
export JAVA_HOME=/usr/lib/jvm/java-7-oracle
and when i use jps command:
> rochdi#127:~$ jps
3427 Jps
help me please

Resources