It seems I have no jobtracker with Hadoop 2.2.0. JPS does not show it, there is no one listening on port 50030, and there are no logs about the jobtracker inside the logs folder. Is this because of YARN? How can I configure and start the job tracker?
If you are using YARN framework, there is no jobtracker in it. Its functionality is split and replaced by ResourceManager and ApplicationMaster. Here is expected jps prinout while running YARN
$jps
18509 Jps
17107 NameNode
17170 DataNode
17252 ResourceManager
17309 NodeManager
17626 JobHistoryServer
Related
While giving Command jps :
it is showing only
some _pid jps
not showing services others likes
Namenode
Datanode
ResourceManager
etc
Have you started the service. If services is not started it will not show anything. To start the service use start-all.sh, than check jps
I followed this tutorial and tried to setup a multinode hadoop cluster on centOS. After doing all the configurations and running start-dfs.sh and start-yarn.sh, this is what jps outputs:
Master
26121 ResourceManager
25964 SecondaryNameNode
25759 NameNode
25738 Jps
Slave 1
19082 Jps
17826 NodeManager
Slave 2
17857 Jps
16650 NodeManager
Data node is not started on slaves.
Can anyone suggest what is wrong with this setup?
when i give the command
for service in /etc/init.d/hadoop*
>do
>sudo $service stop
>done
its stops all the service
and when i give
for service in /etc/init.d/hadoop-hdfs-*
>do
>sudo $service stop
>done
its stops all the service
it sometimes start datanode and sometimes namenode
eg:
21270 NameNode
21422 Jps
21374 SecondaryNameNode
2624 HMaster
or
11070 DataNode
11422 Jps
11554 SecondaryNameNode
2554 HMaster
same thing happens for jobtracker and tasktracker
I tried formating the namenode but it didnt help
I also changing the path of localhost in
core-site.xml from 8020 to 50020
and also in mapred-site.xml from 8021 to 50020
this time it shows NameNode, DataNode, JobTracker,Tasktracker using jps
but when i check the browser localhost:50070 and localhost:50030
it refers to 8020 instead of 50020.
why is this happening ?
please help
Run the following script from terminal to stop the running hadoop daemons.
> $HADOOP_INSTALL/hadoop/bin/stop-all.sh
Run the following script from terminal to start the hadoop daemons.
$HADOOP_INSTALL/hadoop/bin/start-all.sh
I have hbase-0.94.10 version .. all my setting works fine
Here is my JPS command output
11804 HRegionServer
10784 TaskTracker
12401 HQuorumPeer
10539 SecondaryNameNode
10201 NameNode
10369 DataNode
12669 JPS
10619 JobTracker
11624 HMaster
But after 30 sec. passed and if i type JPS command again then output was -
11804 HRegionServer
10784 TaskTracker
12401 HQuorumPeer
10539 SecondaryNameNode
10201 NameNode
10369 DataNode
12669 JPS
10619 JobTracker
Looks like HMaster is missing and because of this i am not able to access hbase master-status on web browser
Anyhelp please ..
Thanks
One of the issue could be stale zookeeper data. Try to remove zookeeper data and restart HBase.
Go to the logs.
Run HBase at debug level and try starting the master. Search for Exception.
I am running Apache Hadoop version 1.0.4.
I followed a tutorial here: http://www.michael-noll.com/tutorials/running-hadoop-on-ubuntu-linux-single-node-cluster/ with some tweaks to set up Hadoop.
The data node starts when I use the script start-dfs.sh but dies after some time (less then a second).
hduser#mudit-Studio-1535:/usr/local/hadoop/bin$ jps
25672 NameNode
26276 SecondaryNameNode
25970 DataNode
26328 Jps
hduser#mudit-Studio-1535:/usr/local/hadoop/bin$ jps
25672 NameNode
26276 SecondaryNameNode
26360 Jps
Similar problem is encountered when I use start-all.sh
I tried formatting the namenode by ./hadoop namenode -format. The output is
Still I am getting the same problem.