I'm trying to run Hadoop on a single-node in a pseudo-distributed mode.
I'm using Ubuntu 20.04 on WSL and have Java 8.
When I run:
start-dfs.sh
start-yarn.sh
And then run:
jps
My system outputs the following:
1829 SecondaryNameNode
2549 Jps
1612 DataNode
2188 NodeManager
2045 ResourceManager
Why isn't it showing "NameNode"?
I've already tried deleting the tmp files with:
rm -Rf <tmp dir>
Then formatted the namenode:
bin/hdfs namenode -format
And yet the same output appears when I run jps
What am I doing wrong?
Related
I installed hadoop in ubuntu, when I run the following command:
sudo su hduser
cd /usr/local/hadoop/etc/hadoop
sudo rm -R /app/*
sudo rm -R /tmp/*
hadoop namenode -format
start-dfs.sh && start-yarn.sh
jps
~$ jps
4722 Jps
4281 ResourceManager
3971 SecondaryNameNode
4426 NodeManager
3603 NameNode
*datanode is starting but not showing in jps. I don't know how to fix this issue and when i run the commands hadoop-daemon.sh stop datanode or stop-all.sh, the datanode is not stopping.
I followed this tutorial and tried to setup a multinode hadoop cluster on centOS. After doing all the configurations and running start-dfs.sh and start-yarn.sh, this is what jps outputs:
Master
26121 ResourceManager
25964 SecondaryNameNode
25759 NameNode
25738 Jps
Slave 1
19082 Jps
17826 NodeManager
Slave 2
17857 Jps
16650 NodeManager
Data node is not started on slaves.
Can anyone suggest what is wrong with this setup?
when i give the command
for service in /etc/init.d/hadoop*
>do
>sudo $service stop
>done
its stops all the service
and when i give
for service in /etc/init.d/hadoop-hdfs-*
>do
>sudo $service stop
>done
its stops all the service
it sometimes start datanode and sometimes namenode
eg:
21270 NameNode
21422 Jps
21374 SecondaryNameNode
2624 HMaster
or
11070 DataNode
11422 Jps
11554 SecondaryNameNode
2554 HMaster
same thing happens for jobtracker and tasktracker
I tried formating the namenode but it didnt help
I also changing the path of localhost in
core-site.xml from 8020 to 50020
and also in mapred-site.xml from 8021 to 50020
this time it shows NameNode, DataNode, JobTracker,Tasktracker using jps
but when i check the browser localhost:50070 and localhost:50030
it refers to 8020 instead of 50020.
why is this happening ?
please help
Run the following script from terminal to stop the running hadoop daemons.
> $HADOOP_INSTALL/hadoop/bin/stop-all.sh
Run the following script from terminal to start the hadoop daemons.
$HADOOP_INSTALL/hadoop/bin/start-all.sh
I have configured the Hadoop 1.0.4 and started the following without any issue:
1. $ start-dfs.sh : -Works fine
2. $ start-mapred.sh : - Work fine
3. $ jps (Output is below)
Out put:
rahul#rahul-Inspiron-N4010:/usr/local/hadoop-1.0.4/bin$ jps
6964 DataNode
7147 SecondaryNameNode
6808 NameNode
7836 Jps
7254 JobTracker
7418 TaskTracker
But facing issue: While issuing command
rahul#rahul-Inspiron-N4010:/usr/local/hadoop-1.0.4/bin$ hadoop -mkdir /user
Getting following error
Unrecognized option: -mkdir
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
I applied the patch : HDFS-1943.patch but not use full
Should be: hdfs dfs -mkdir /user
Consult documentation at http://hadoop.apache.org/docs/r1.0.4/file_system_shell.html
fs option is missing
hadoop fs -mkdir /user
I am running Apache Hadoop version 1.0.4.
I followed a tutorial here: http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/ with some tweaks to set up Hadoop.
The data node starts when I use the script start-dfs.sh but dies after some time (less then a second).
hduser#mudit-Studio-1535:/usr/local/hadoop/bin$ jps
25672 NameNode
26276 SecondaryNameNode
25970 DataNode
26328 Jps
hduser#mudit-Studio-1535:/usr/local/hadoop/bin$ jps
25672 NameNode
26276 SecondaryNameNode
26360 Jps
Similar problem is encountered when I use start-all.sh
I tried formatting the namenode by ./hadoop namenode -format. The output is
Still I am getting the same problem.