How do I restart the task trackers and job tracker using CDH4 from the command line?
I tried following given script but got error
[root#adc1210765 bin]# ./stop-mapred.sh
/usr/lib/hadoop-0.20-mapreduce/bin/../conf no jobtracker to stop cat:
/usr/lib/hadoop-0.20-mapreduce/bin/../conf/slaves: No such file or
directory
I want to restart all instance of tasktracker running at my cluster nodes
You must do this on each of the task trackers
sudo service hadoop-0.20-mapreduce-tasktracker restart
And on the job tracker
sudo service hadoop-0.20-mapreduce-jobtracker restart
You can also use stop and start in place of restart. Might have to change your hadoop version number.
http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/latest/CDH4-Installation-Guide/cdh4ig_topic_11_3.html?scroll=topic_11_3
You can also try starting the JobTracker Daemon by
/etc/init.d/hadoop-0.20-mapreduce-jobtracker start
and Task Tracker by
/etc/init.d/hadoop-0.20-mapreduce-tasktracker start
Ensure the version number is appropiate.
Related
I have a hadoop cluster 2.7.4 version. Due to some reason, I have to restart my cluster. I need job IDs of those jobs that were executed on cluster before cluster reboot. Command mapred -list provide currently running of waiting jobs details only
You can see a list of all jobs on the Yarn Resource Manager Web UI.
In your browser go to http://ResourceManagerIPAdress:8088/
This is how the history looks on the Yarn cluster I am currently testing on (and I restarted the services several times):
See more info here
I have a Hadoop cluster running Cloudera's CDH3, Apache Hadoop's 0.20.2 equivalent. I want to restart the job-tracker as there are some jobs which are not getting killed. I tried killing them from the command line, the command executes successfully, but the jobs are still in Job Cleanup: Pending status. Anyways I want to restart the job-tracker and see if that cleanup the jobs. I know the command to restart the job-tracker, but I am not sure if I need to put the name-node in safe-mode before I restart the job-tracker.
You can try to kill the unwanted jobs using hadoop job -kill <Job-ID> and check for command status echo "$?". If that doesn't work, Restart is the only option.
Hadoop Jobtracker and namenodes are independent components, No need to execute namenode safenode before Jobtracker restart. You can restart Jobtracker process alone.(tasktracker if required)
Job tracker and Task tracker don't sow up when ran the start-all.sh command in ububtu for hadoop
I do get the rest of the processes while i run the "JPS" command in unix.
Not sure why i am not being shown with the job tracker and task tracker.Have been following couple of links and couldn't get my prob sorted.
Steps done :
-Multiple times formatted the namenode
-Multiple time deleted and recreated the tmp folder with appropriate permissions.
What could be the issue ?
Any suggestions would really help me as i am struggling in setting up hadoop on my laptop.I am new to it though.
Try starting jobtracker and tasktracker separately.
From your hadoop HOME directory run
. bin/../libexec/hadoop-config.sh
Then from hadoop BIN directory run
hadoop-daemon.sh --config $HADOOP_CONF_DIR start jobtracker
hadoop-daemon.sh --config $HADOOP_CONF_DIR start tasktracker
You must have been using hadoop 2.x version where jobtracker is replaced with YARN resource manager. Using jps(jdk is needed) you can check whether resouce manager is running. If it is running then the default url for it is (host-name):8088. You can check your nodes,jobs also configuration there.If not running then start them with sbin/start-yarn.sh.
I followed the tutorialshttp://hadoop.apache.org/docs/r2.4.1/hadoop-project-dist/hadoop-common/SingleCluster.html and installed hadoop 2.4.1 as pseudo distributed cluster. I created a ubuntu VM using OracleVM and installed hadoop as mentioned in the link. It was setup fine and able to run the examples. However the job tracker URL is not working. :50030 gives page not found. I also tried netstat on the server and there is no process waiting on 50030 port . Do i need to start any other service ? What are the possible reasons ?
You need to execute this:
$HADOOP_HOME/sbin/mr-jobhistory-daemon.sh start historyserver
Or JobTracker won't start.
(In my case, $HADOOP_HOME is in /usr/local/hadoop)
Check the value of mapred.job.tracker.http.address in mapred-site.xml
If the port is different, use that.
Also check if jobtracker is running. Check the jobtracker logs.
You need to enter the following command
http://localhost:50030/
Job Tracker web UI.
I have been running some benchmarks and i am new to hadoop and hdfs. I have got the setup and things running and they were working fine. But now i am faced with this issue, jps on the master shows
1. secondary name node
2. job tracker
but not the name node and task tracker.
similarly jps on the slave nodes shows only name node, but task tracker is not running.
I usually run the job as the user and not root, but mistakenly i ran it as root and then when i exited and ran the job as user, i found the job doesn't start. then with jps i found the task tracker is not running.
I am new to hdfs, and not sure how to debug and solve this, it would be great if you can give some pointers/help on this one, i did try google and couldnt find relevant answers.
Edit: I tried clearing tmp files, killing obsolete java process and restarting. still i get the same issue.
Thanks.
Kill all java process, after stopping the cluster
remove /tmp hadoop pids
verify file permission errors, but looking at hadoop/logs/*.log file in name node and data node, this gave me useful info in debugging the issue.
this link was helpful,
http://felixtechnique.blogspot.com/2010/09/no-namenode-to-stop-no-tasktracker-to.html