We get multiple user request to hold run or kill the jobs running in our autosys server
I want to crete a script which will take input as joobname and what send event they want as input parameter and then trigger the respective job as per requirement .
Can it be done.Please help
Sounds like the answer is the killallcommand (Examples)
Yes it is possible.
You need to create a script with the following information.
Either you can store all the job names which you want to kill via script in text file which will be parm file and then read this file in the script. A script will be having an logic to read the file which will be having job names and while running the script you can pass the ICE_TYPE ( it can be ON_ICE/OFF_ICE/KILLJOB/....or any event)
A script will be having following main code so if you say created script autosysICE.sh then you need to run the script in the following manner.
Shell > Wrapper autosysICE.sh
Following is the important line we need to embed into the script which will take argument as KILLJOB and will replace it with the $ICE_TYPE.
sendevent -E $ICE_TYPE -J JOB_NAME
The rest you can do with the help of array read the job names from file and go through it
Code Snippet ( Note : I haven't tested this but this is the logic )
open(DATA,"<autosysParm.txt") or die "Can't open data";
#jobList= <DATA>;
close(DATA);
foreach my $jobName (#jobList) {
sendevent -E $ICE_TYPE -J $jobName
}
Let me know if you still having an issue.
Thanks!
Related
I am trying to create an auto-rerun mechanism by implementing some code into sasbatch script after sascommand will finish. General idea is to:
locate a log of sas process and an id of the flow containing current job,
check if the log contains particular ORA-xxxxx errors which we know that solution for them is just rerun of the process,
if so, then trigger jrerun class from LSF Platform Command Line Interface,
exit sasbatch passing $rc to LSF
The idea was implemented as:
#define used paths
log_dir=/path/to/sas_logs_directory
out_log=/path/to/auto-rerun_log.txt
out_log2=/path/to/lsf_rerun_log.txt
if [ -n "${LSB_JOBNAME}"]; then
if [ ! -f "$out_log"]; then
touch $out_log
fi
#get flow runtime attributes
IFS-: read -r flow_id username flow_name job_name <<< "${LSB_JOBNAME}"
#find log of the current process
log_path=$(ls -t $log_dir/*.log | xargs grep -li "job:\s*$job_name" | grep -i "/$flow_name_" | head -1)
#set path to txt file containing lines which represents ORA errors we look for
conf_path-/path/to/error_list
#analyse process' log line by line
while read -r line;
do
#if error is found in log then try to rerun flow
if grep -q "$line" $log_path; then
(nohup /path/to/rerun_script.sh $flow_id >$out_log2 2>&1) &
disown
break
fi
done < $conf_path
fi
While rerun_script is the script which calls jrerun class after sleep command - in order to let parent script exit $rc in the meanwhile. It looks like:
sleep 10
/some/lsf/path/jrerun
Problem is that job is running for the all time. In LSF history I can see that jrerun was called before job exited.
Furthermore in $out_log2 I can see message: <flow_id> has no starting or exit points.
Do anyone have an idea how I can pass return code to LSF before jrerun calling? Or maybe some simplier way to perform autorerun of SAS jobs in Platform LSF?
I am using SAS 9.4 and Platform Process Manager 9.1
Or maybe some simplier way to perform autorerun of SAS jobs in Platform LSF?
I'm not knowledgeable about the SAS part. But on the LSF side there's at least a couple of ways to requeue the job.
If you have control of the job script, you can use special process exit value to automatically requeue the job.
https://www.ibm.com/support/knowledgecenter/en/SSWRJV_10.1.0/lsf_admin/job_requeue_about.html
If you have control outside of the job script, you can use brequeue -r to requeue a running job.
https://www.ibm.com/support/knowledgecenter/en/SSWRJV_10.1.0/lsf_command_ref/brequeue.1.html
Good Luck
I managed to get this working by using two additional configuration files. When my grep returnes 1 I add found flow_id to flow_list.txt configuration file and modify especially made trigger_file.txt.
I scheduled additional flow execute_rerun in LSF which is triggered after file trigger_file.txt is modified. The execute_rerun flow reads flow_list.txt configuration file line by line and calls jrerun method on each flow.
I managed to achieve an automatic rerun of the flows which fails due to particular errors.
I am trying to learn about making simple bash scripts to do things on my computer just because I want to learn because I think it is interesting (and I can also think of uses down the track).
I am trying to write a script that assigns variables that will call another script depending on what I type. I have managed to call another script from a variable using the below:
#!/bin/bash
Echo hello please choose you next step
VBA="/Users/zap/VBA.sh"
$VBA
but now I want to be able to call one script or another depending on user input, and I have tried to make the script below, so that if when I type VBA in the "read" section it runs one script and if I type in VBB in it runs a different script. But it seems that this does not work how do I need to change the syntax to make the out put run with the script VBA or VBB?
#!/bin/bash
Echo hello please choose you next step
VBA="/Users/zap/VBA.sh"
VBB="/Users/zap/VBB.sh"
read IPT
NXT="$"$IPT""
echo $NXT
If I can make this work I will turn this into a simple script that runs sudo shutdown and then asks me if I want to shut down immediately (I think -h) or restart (I think -r).
There are a few options.
1) You could use an if statement and explicitly run the command you want:
echo Do you want to reboot (Y/N)?
read IPT
if [ "$IPT" = "Y" ] ; then
shutdown -h now
else
shutdown -r now
fi
2) you can try to force the user to type the command. It could be dangerous if you sudo
since the user could type something you don't want to run.
echo do you want to run VBA or VBB?
read IPT
"/Users/zap/${IPT}.sh"
I have a script lying into a Unix server which looks like this:
mainScript.sh
#some stuff here
emailScript.sh $PARAM_1 $PARAM_2
#some other stuff here
As you can see, mainScript.sh is calling another script called emailScript.sh.
The emailScript.sh is supposed to perform a query via sqlplus, then parse the results and return them via email if any.
The interesting part of the code in emailScript.sh is this:
DB_SERVER=$1
USERNAME=$2
PASSWORD=$3
EVENT_DATE=$4
LIST_AUTHORIZED_USERS=$5
ENVID=$6
INTERESTED_PARTY=$7
RAW_LIST=$(echo "select distinct M_OS_USER from MX_USER_CONNECTION_DBF where M_EVENT_DATE >= to_date('$EVENT_DATE','DD-MM-YYYY') and M_OS_USER is not null and M_OS_USER not in $LIST_AUTHORIZED_USERS;" | sqlplus -s $USERNAME/$PASSWORD#$DB_SERVER)
As you can see, all I do is just creating the variable RAW_LIST executing a query with sqlplus.
The problem is the following:
If I call the script mainScript.sh via command line (PuTTy / KiTTy), the sqlplus command works fine and returns something.
If I call the script mainScript.sh via an external job (a ssh connection opened on the server via a Jenkins job), the sqlplus returns nothing and takes 0 seconds, meaning it doesn't even try to execute itself.
In order to debug, I've printed all the variables, the query itself in order to check if something wasn't properly set: everything is correctly set.
It really seems that the command sqlplus is not recognized, or something like this.
Would you please have any idea on how I can debug this? Where should I look the issue?
You need to consider few things here. While you are running the script, from which directory location you are executing the script? And while you are executing the script from your external application from which directory location it is executing the script. Better use full path to the script like /path/to/the/script/script.sh or use cd /path/to/the/script/ command to go to the script directory first and execute the script. Also check execute permission for your application. You as an user might have permission to execute the script or sql command but your application does not have that permission. Check the user id for your application and add that into the proper group.
Hopefully this is not a dublicate and also not just a problem of our cluster's configuration...
I am submitting a job array to a cluster using qsub with the following command:
qsub -q QUEUE -N JOBNAME -t 1:10 -e ${ERRFILE}_$SGE_TASK_ID /path/to/script.sh
where
ERRFILE=/home/USER/somedir/errors.
The idea is to specify an error file (also analogously the output file) that also contains the task ID from within the job array.
So far I have learned that the line
#$ -e ${ERRFILE}_$SGE_TASK_ID
inside the script.sh, does not work, because it is a comment and not evaluated by bash. My first line does not work however because $SGE_TASK_ID is only set AFTER the job is submitted.
I read here that escaping the evaluation of $SGE_TASK_ID (in that link it's PBS' $PBS_JOBID, but a similar problem) should work, but when I tried
qsub -q QUEUE -N JOBNAME -t 1:10 -e ${ERRFILE}_\$SGE_TASK_ID /path/to/script.sh
it did not work as expected.
Am I missing something obvious? Is it possible to use $SGE_TASK_ID in the name of an error file (the automatic naming of error files does that, but I want to specify the directory and if possible the name, too)?
Some additional remarks:
I am using the -cwd option for qsub inside script.sh, but that is NOT where I want my error files to be stored.
I have next to no control over how the cluster works and no root access (wouldn't know what I could need it for in this context but anyway...).
Apparently our cluster does not use PBS.
Yes my scripts are all executable and where applicable started with #!/bin/bash (I also specified the use of bash with the -S /bin/bash option for qsub).
There seems to be a solution here, but I am not quite sure how that works and it also appears to be using PBS. If that answer DOES apply to my question and I misunderstood it, please let me know.
I would appreciate any hint into the right direction.
Thank You!
I didn't know this either, but it looks like Grid Engine has something called "pseudo environment variables" like $TASK_ID for this purpose. This should work:
qsub -q QUEUE -N JOBNAME -t 1:10 -e ${ERRFILE}_\$TASK_ID /path/to/script.sh
From the man page:
-e [[hostname]:]path,...
...
If the pathname contains certain pseudo
environment variables, their value will be expanded at
runtime of the job and will be used to constitute the
standard error stream path name. The following pseudo
environment variables are supported currently:
$HOME home directory on execution machine
$USER user ID of job owner
$JOB_ID current job ID
$JOB_NAME current job name (see -N option)
$HOSTNAME name of the execution host
$TASK_ID array job task index number
I have jenkins build console output of a job that i want to use in execute shell of the same job. The output is
POST Response Code: 200
Response:
{"status":"success","results":{"job_id":"57","ip":null,"hostname":null}}
I want to use the status variable to print whether is is success or failure. How can I do that.
Im using echo "$status" its not working
I think you want to break the job based on the value right?
You can sed the response:
echo '{"status":"success","results":{"job_id":"57","ip":null,"hostname":null}}'
\ | sed -r 's|.+status":"(.+?)","r.*|\1|p'
Assign it to a variable and do whatever you want.
Otherwise you can use the Text-finder Plugin. You assign the response of your action to a temporary file and let the plugin check for a phrase.