I followed this tutorial and tried to setup a multinode hadoop cluster on centOS. After doing all the configurations and running start-dfs.sh and start-yarn.sh, this is what jps outputs:
Master
26121 ResourceManager
25964 SecondaryNameNode
25759 NameNode
25738 Jps
Slave 1
19082 Jps
17826 NodeManager
Slave 2
17857 Jps
16650 NodeManager
Data node is not started on slaves.
Can anyone suggest what is wrong with this setup?
Related
my question might be duplicated, i had searched similar question here but it didn't solve my problem.
so i'm new in hadoop, now i'm setting up multinode cluster which is 1 master and 2 slaves. when i run jps command on master node, my terminal shows this
3250 - DataNode
3090 - NameNode
4099 - jps
3498 - SecondaryNameNode
and when i run jps command on slaves node, my terminal shows this
3896 - DataNode
4684 - jps
4111 - SecondaryNameNode
according to this tutorial link, my master node would have the this output
jps
namenode
secondary namenode
resource manager
and my slaves node would have this
jps
NodeManager
DataNode
so on master node there is no resource manager and on slaves node there is no node manager
edit
when i start start-dfs.sh command, the output of my terminal shows this
Starting namenodes on [HadoopMaster]
starting datanodes
starting secondary namenodes [farhan-master]
and when i start start-yarn.sh command, the output of my terminal shows this
starting resourcemanager
starting nodemanager
how do i solve this problem? thanks in advance
We have a Hadoop setup with 2 Master nodes and 1 Slave node.
We have configured the Hadoop cluster. After configuring, when we executed "jps" command, we are getting following output on my Master Node:
13405 NameNode
14614 Jps
13860 ResourceManager
13650 DataNode
14083 NodeManage
On my second Master Node, output is:
9698 Jps
9234 DataNode
9022 NameNode
9450 NodeManager
On my Data Node, output is:
21681 NodeManager
21461 DataNode
21878 Jps
I feel my secondary node is not running. Please tell me this is right or wrong. If its wrong, what should be the status of my node? Please answer me as soon as possible.
You can check status of node by running below command
hdfs haadmin -getServiceState
What is this 7214 when I run the jps command.
17134 SecondaryNameNode
16934 NameNode
**7214**
17401 Jps
17249 ResourceManager
17024 DataNode
17365 NodeManager
It seems I have no jobtracker with Hadoop 2.2.0. JPS does not show it, there is no one listening on port 50030, and there are no logs about the jobtracker inside the logs folder. Is this because of YARN? How can I configure and start the job tracker?
If you are using YARN framework, there is no jobtracker in it. Its functionality is split and replaced by ResourceManager and ApplicationMaster. Here is expected jps prinout while running YARN
$jps
18509 Jps
17107 NameNode
17170 DataNode
17252 ResourceManager
17309 NodeManager
17626 JobHistoryServer
I am running Apache Hadoop version 1.0.4.
I followed a tutorial here: http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/ with some tweaks to set up Hadoop.
The data node starts when I use the script start-dfs.sh but dies after some time (less then a second).
hduser#mudit-Studio-1535:/usr/local/hadoop/bin$ jps
25672 NameNode
26276 SecondaryNameNode
25970 DataNode
26328 Jps
hduser#mudit-Studio-1535:/usr/local/hadoop/bin$ jps
25672 NameNode
26276 SecondaryNameNode
26360 Jps
Similar problem is encountered when I use start-all.sh
I tried formatting the namenode by ./hadoop namenode -format. The output is
Still I am getting the same problem.