I installed hadoop in ubuntu, when I run the following command:
sudo su hduser
cd /usr/local/hadoop/etc/hadoop
sudo rm -R /app/*
sudo rm -R /tmp/*
hadoop namenode -format
start-dfs.sh && start-yarn.sh
jps
~$ jps
4722 Jps
4281 ResourceManager
3971 SecondaryNameNode
4426 NodeManager
3603 NameNode
*datanode is starting but not showing in jps. I don't know how to fix this issue and when i run the commands hadoop-daemon.sh stop datanode or stop-all.sh, the datanode is not stopping.
Related
I'm trying to run Hadoop on a single-node in a pseudo-distributed mode.
I'm using Ubuntu 20.04 on WSL and have Java 8.
When I run:
start-dfs.sh
start-yarn.sh
And then run:
jps
My system outputs the following:
1829 SecondaryNameNode
2549 Jps
1612 DataNode
2188 NodeManager
2045 ResourceManager
Why isn't it showing "NameNode"?
I've already tried deleting the tmp files with:
rm -Rf <tmp dir>
Then formatted the namenode:
bin/hdfs namenode -format
And yet the same output appears when I run jps
What am I doing wrong?
I have Hadoop 2.9.0 on Ubuntu at
/usr/local/hadoop
But when I try start-dfs.sh
No error is shown while starting namenode
But when I type jps, only
10900 SecondaryNameNode
11047 Jps
10696 DataNode
Seams to have started, not namenode
Things tried:
=> Removed temp files and formatted namenode hadoop namenode -format
terminal:
blaze#blazian:/tmp$ start-dfs.sh
Starting namenodes on [localhost]
blaze#localhost's password:
localhost: starting namenode, logging to /usr/local/hadoop/logs/hadoop-blaze-namenode-blazian.out
blaze#localhost's password:
localhost: starting datanode, logging to /usr/local/hadoop/logs/hadoop-blaze-datanode-blazian.out
Starting secondary namenodes [0.0.0.0]
blaze#0.0.0.0's password:
0.0.0.0: starting secondarynamenode, logging to /usr/local/hadoop/logs/hadoop-blaze-secondarynamenode-blazian.out
blaze#blazian:/tmp$ jps
10900 SecondaryNameNode
11047 Jps
10696 DataNode
You don't have a SSH setup with localhost. Please follow these steps and you'll be able to run the namenode.
Go to your system terminal and type:
cd(It will redirect you to ~)
ssh-keygen(hit enter three times and it will create a .ssh directory in ~)
cat id_rsa.pub >> authorized_keys(it will make sure that your localhost is the trusted source and give permission to make
passwordless ssh.)
Then simply run start-all.sh and you're all set.
when i give the command
for service in /etc/init.d/hadoop*
>do
>sudo $service stop
>done
its stops all the service
and when i give
for service in /etc/init.d/hadoop-hdfs-*
>do
>sudo $service stop
>done
its stops all the service
it sometimes start datanode and sometimes namenode
eg:
21270 NameNode
21422 Jps
21374 SecondaryNameNode
2624 HMaster
or
11070 DataNode
11422 Jps
11554 SecondaryNameNode
2554 HMaster
same thing happens for jobtracker and tasktracker
I tried formating the namenode but it didnt help
I also changing the path of localhost in
core-site.xml from 8020 to 50020
and also in mapred-site.xml from 8021 to 50020
this time it shows NameNode, DataNode, JobTracker,Tasktracker using jps
but when i check the browser localhost:50070 and localhost:50030
it refers to 8020 instead of 50020.
why is this happening ?
please help
Run the following script from terminal to stop the running hadoop daemons.
> $HADOOP_INSTALL/hadoop/bin/stop-all.sh
Run the following script from terminal to start the hadoop daemons.
$HADOOP_INSTALL/hadoop/bin/start-all.sh
What is this 7214 when I run the jps command.
17134 SecondaryNameNode
16934 NameNode
**7214**
17401 Jps
17249 ResourceManager
17024 DataNode
17365 NodeManager
I am running Apache Hadoop version 1.0.4.
I followed a tutorial here: http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/ with some tweaks to set up Hadoop.
The data node starts when I use the script start-dfs.sh but dies after some time (less then a second).
hduser#mudit-Studio-1535:/usr/local/hadoop/bin$ jps
25672 NameNode
26276 SecondaryNameNode
25970 DataNode
26328 Jps
hduser#mudit-Studio-1535:/usr/local/hadoop/bin$ jps
25672 NameNode
26276 SecondaryNameNode
26360 Jps
Similar problem is encountered when I use start-all.sh
I tried formatting the namenode by ./hadoop namenode -format. The output is
Still I am getting the same problem.