I have to run multiple spark job one by one in a sequence, So I am writing a shell script. One way I can do is to check success file in output folder for job status, but i wanna know that is there any other way to check the status of spark-submit job using unix script, where I am running my jobs.
You can use command
yarn application -status <APPLICATIOM ID>
where <APPLICATIOM ID> is your application ID and check for line like:
State : RUNNING
This will give you the status of your application
To check the list of application, run via yarn you can use command
yarn application --list
You can add also -appTypes to limit the listing based on the application type
Related
I need to get the id of a specific hadoop job.
In my case, I lunch a sqoop commande remotely and I went to verify the job status with this commande :
hadoop job -status job_id | grep -w 'state'
I can get this information from the GUI but i went to do something
can any one help me !!!
You can use the Yarn REST apis, via your browser or curl from the command line. It will list all the currently running and previously running jobs, including sqoop and the mapreduce jobs that sqoop generates and executes. Use the UI first, if you have it up and running just point your browser to http:<host>:8088/cluster (not sure if the port is the same on all hadoop distributions. I believe 8088 is the default on apache). Alternatively you can use yarn commands directly, e.g, yarn application -list.
I'd like to list the nodes on which the containers are running for a particular MR job.
I only have the application_id.
Is it possible to do it with Hadoop REST API and/or through command line?
This can be done using the yarn command.
Run yarn applicationattempt -list <Application Id> to get an app attempt id
Run yarn container -list <Application Attempt Id> to get the container ids
Run yarn container -status <Container Id> to get the host for any particular container.
If you want this in a bash script or want to get every host for an application with a large number of containers you will probably want to parse out the attempt/container id and host, but this is at least a start.
You can find them using Resource manager UI. Find your application by ID among the existing applications and click on the link with ID you have. You will see your application stats. Fint the tracking URL and click on the link 'History'. There you'll be able to find the tasks in your map operation and recude optration. You can open each task and see the information, to which node it was assigned for, nubmer of attempts, logs for each task and attempts and lots of other usefull information.
For getting the information about the container status from command line you can use yarn container -status command from bash
I'm working by CDH 5.1 now. It starts normal Hadoop job by YARN but hive still works with mapred. Sometimes a big query will hang for a long time and I want to kill it.
I can find this big job by JobTracker web console while it didn't provide a button to kill it.
Another way is killing by command line. However, I couldn't find any job running by command line.
I have tried 2 commands:
yarn application -list
mapred job -list
How to kill big query like this?
You can get the Job ID from Hive CLI when you run a job or from the Web UI. You can also list the job IDs using the application ID from resource manager. Ideally, you should get everything from
mapred job -list
or
hadoop job -list
Using the Job ID you can kill it by using the below command.
hadoop job -kill <job_id>
Another alternative would be to kill the application using
yarn application -kill <application_id>
I am trying to submit an example map reduce oozie job and all the properties are configured properly with regards to the path and name node and job-tracker port etc. I validated the workflow.xml too . when I deploy the job I get a job id and when I check the status I see a status KILLED and the details basically say that
/var/tmp/oozie/oozie-oozi7188507762062318929.dir/map-reduce-launcher.jar does not exist.
In order to resolve this error, just crate hdfs folders and give appropriate permissions to them.
http://kadirsert.blogspot.com.tr/2014/03/oozie-says-jar-does-not-exist.html
Local file system (no HDFS) should have '/var/tmp/oozie' directory.
If the directory doesn't exist, create the directory and restart the Oozie server. Then there comes a lot of files under /var/tmp/oozie including *-launcher.jar files.
'/var/tmp/oozie' is the value of -Djava.io.tmpdir variable in Oozie server start-up command line. You can check the value using 'ps -ef | grep oozie' where the Oozie server is running.
I know how to run a bash script on Jenkins. However, if I use qsub to submit the bash script to OGE system, how does Jenkins know that my job terminates or not?
You can use "-sync y" on your qsub to cause the qsub to wait until the job(s) are finished.
Jenkins allows for you to submit the results of a build using a web based API. I currently use this to monitor a job remotely for a grid at my orginization. If you have the ability to perform a web post to the jenkins server you could use the below script to accomplish the job.
#!/bin/bash
MESSAGE="Some message about job success"
RUNTIME="Some calculation to estimate runtime"
USERNAME="userNameForJenkinsLogin"
PASSWORD="passwordForJenkinsLogin"
JENKINS_HOST="URLToJenkins"
TEST_NAME="Name of Test"
curl -i -X post -d "<run><log>$MESSAGE</log><result>0</result><duration>$RUNTIME</duration></run>" http://$USERNAME:$PASSWORD#$JENKINS_HOST/jenkins/job/$TEST_NAME/postBuildResult
The Jenkins SGE Cloud plugin submits builds to the Sun Grid Engine (SGE) batch scheduler. Both the open source version of SGE and the commercial Univa Grid Engine (UGE) are supported.
This plugin adds a new type of build step Run job on SGE that submits batch jobs to SGE. The build step monitors the job status and periodically appends the progress to the build's Console Output. Should the build fail, errors and the exit status of the job also appear. If the job is terminated in Jenkins, it is also terminated in SGE.
Builds are submitted to SGE by a new type of cloud, SGE Cloud. The cloud is given a label like any other slave. When a job with a matching label is run, SGE Cloud submits the build to SGE.