I am reaching to SME on the subject after trying to figure out the solution of below scenario.
I have a very specific requirement:
1) Say I have a schedule called SCH1, having 3 jobs JOB1, JOB2 and JOB3 like:
Schedule S1
JOB1
JOB2
FOLLOWS JOB1
JOB3
FOLLOWS JOB2
END
2) The JOB1 is calling a shell script.
3) Now if JOB1 shell script returns a code 0 – it is successful and execute JOB2 and JOB3.
4) If shell script returns code is anything other than 0, Still the JOB1 should not abend; but all other job JOB2 and JOB3 should be skipped (or completed without run); and schedule should be marked successful.
I got below link online, which suggest that this should be possible. But when I refer to syntax for Job/Schedule definition; I am not able to figure out that how it should be written.
In the job syntax, it looks like that below can do the trick, but not so sure.
[rccondsucc "Success Condition"]
[recovery
{stop | continue | rerun}
[after [workstation#]jobname]
[abendprompt “text”] ]
Conditional Dependency:
http://www-01.ibm.com/support/knowledgecenter/SSGSPN_9.1.0/com.ibm.tivoli.itws.doc_9.1/zos/src_man/eqqr1conddepevalex.htm?lang=en
Job syntax:
http://www-01.ibm.com/support/knowledgecenter/SSGSPN_9.1.0/com.ibm.tivoli.itws.doc_9.1/distr/src_ref/awsrgjobdefn.htm?lang=en
Schedule syntax:
http://www-01.ibm.com/support/knowledgecenter/SSGSPN_9.1.0/com.ibm.tivoli.itws.doc_9.1/distr/src_ref/awsrgjsdefn.htm?lang=en
I would say this is not possible.
With RCCONDSUCC you can fine tune the status of a job (SUCC or ABEND) depending on it's return code. This just means you are not limited to RC = 0 = SUCC or RC > 0 = ABEND.
With RECOVERY you can specify which action should be taken only when the JOB is in ABEND.
Here you want to force a JOB to SUCC and specify a RECOVERY....
To skip Job2 and Job3 with the help of Conditional dependencies Job1 must be in status "E". But you can add a Job4 to application that only runs if Job1 aborted (also defined with CondDep). This job completes Job1 (or whole application).
It works - I demonstrated just this example on customer meeting :-)
It is also possible that Job1 completes and jobs Job2 and Job3 are skipped if these jobs have CondDep to Job1: Job1 must be in status "C" AND Job1 must end with RC<>0.
Since 9.3 FP1 this can be implemented with built-in conditional dependencies
Assuming you want JOB3 to wait if JOB2 abends, using conditional dependencies your scenario can be implemented like this
SCHEDULE SCH1
:
JOB1
DOCOMMAND "myscript"
STREAMLOGON twsuser
SUCCOUTPUTCOND GO "RC=0"
SUCCOUTPUTCOND NOGO "RC!=0"
JOB2
FOLLOWS JOB1 IF GO
JOB3
FOLLOWS JOB2
FOLLOWS JOB1 IF GO
END
JOB1 has 2 SUCCOUTPUTCOND defined: GO if RC=0 and NOGO if RC!=0.
Both are SUCCOUTPUTCOND, meaning that JOB1 will go in SUCC in both cases.
If JOB1 ends with RC=0, the GO condition is true and the FOLLOWS JOB1 IF GO dependencies are satisfied, so JOB2 and JOB3 can be run (JOB3 will wait JOB2 since it has also a FOLLOWS JOB2)
If JOB1 ends with a RC that is not 0, GO is false and NOGO is true. Since at least on SUCCOUTPUTCOND is true, JOB1 completes in SUCC, but the FOLLOWS JOB1 IF GO are not satisfied and JOB2 and JOB3 would be set to SUPPR status and will not run.
Related
When using go-cron to work, multiple services are started at the same time, and multiple cron tasks will be executed at the same time? What can be done to ensure that the cron task is only executed once?
my cron expression is [0 2 */1 * *]
You can configure the SingletonMode for your task to prevents a new job from starting if the prior job has not yet completed:
s := gocron.NewScheduler(time.UTC)
_, _ = s.Every(1).Second().SingletonMode().Do(task)
https://pkg.go.dev/github.com/go-co-op/gocron#Scheduler.SingletonMode
Or enable the SingletonModeAll to prevent new jobs from starting if the prior instance of the particular job has not yet completed:
s := gocron.NewScheduler(time.UTC)
s.SingletonModeAll()
_, _ = s.Every(1).Second().Do(task)
https://pkg.go.dev/github.com/go-co-op/gocron#Scheduler.SingletonModeAll
In Azure pipeline yaml file, when defining multiple jobs in a single stage, one can specify dependencies between them. One can also specify the conditions under which each job runs.
Code #1
jobs:
- job: A
steps:
- script: echo hello
- job: B
dependsOn: A
condition: and(succeeded(), eq(variables['build.sourceBranch'], 'refs/heads/master'))
steps:
- script: echo this only runs for master
Code #2
jobs:
- job: A
steps:
- script: "echo ##vso[task.setvariable variable=skipsubsequent;isOutput=true]false"
name: printvar
- job: B
condition: and(succeeded(), ne(dependencies.A.outputs['printvar.skipsubsequent'], 'true'))
dependsOn: A
steps:
- script: echo hello from B
Question:
Code #1 & #2 above have different orders of the dependency and condition. Does the order matters? If so, what's matter? (what's the difference between different orders)
Discuss 1 and 2 separately.
Code #1:
Since there is no data connection between job1 and job2, data connection here refers to variable sharing and etc.
So, for #1, there's no matters on order. Here you can ignore the dependsOn specified while you have no special requirements on the execution order between job A and job B.
BUT, there's one key thing you need pay attention is, the actual running order will be changed randomly when you do not specify the dependsOn. For example, most of time, they will respect with the order job A, job B. Occasionally, they will randomly run as job B, job A.
Code #2:
This must make the dependsOn specified. Because your job B is using the output variable which created/generated at job A. Since our system allow same variables name exists in different jobs, you must specify the dependsOn so that the system can know the job B should find the variable skipsubsequent from job A not others. Only this key words specified, the variables which generated in job A can be exposed and available to next jobs.
So, the nutshell is once there is any data connection between jobs, e.g variable pass, you must specify dependsOn to make the jobs has connection with each other.
I am trying to trigger a particular job in CI after either of the 2 conditions
trigger by another job in the same pipeline
OR
changes: somefile.txt
My CI is as described
job1:
stage: build
script:
- echo "JOb1"
- curl -X POST -F token=2342344444 -F "variables[TRIGGER_JOB]=job1" -F ref=master https://main.gitlab.myconmpanyxyz.com/api/v4/projects/1234/trigger/pipeline
only:
changes:
- job1.md
job2: # This does not RUN as expected because of the TRIGGER_JOB set to job1
stage: test
script:
- echo "Job2"
rules:
- if: $TRIGGER_JOB =="job2"
job3: # this RUNS as expected because of VARIABLE TRIGGER_JOB
stage: test
script:
- echo "Job3"
rules:
- if: $TRIGGER_JOB =="job1"
job4: # this also RUNS, but this should not be the expected behavior
stage: test
script:
- echo “job4“
rules:
- if: $TRIGGER_JOB == "xyz"
- changes:
- job4.md
After job1 finishes it also needs to call job4 and not any other jobs (job2 in this case). So I am using CURL to call the job itself. If there are any better ways of calling a specific job in the same CI, also please let me know.
I have already seen this stack-overflow page, but it does not help because my job needs to be triggered by either of 2 conditions which is not allowed bit gitlab-ci.
I need job4 to be called by either of the 2 conditions - if the TRIGGER_JOB=="job1" or if there are any changes in job4.md file.
Currently job4 runs if changes are made in job4.md file, however it also runs if the job1 is triggered. But afaik this should not be the expected behavior.
docs. Can anyone please give me some leads how to create this kind of design.
Your solution was almost correct, but the changes keyword with only or except does only work, if the pipeline is triggered by a push or a merge_request event. This is defined in the variable CI_PIPELINE_SOURCE. When you trigger the pipeline by calling the API, the variable CI_PIPELINE_SOURCE contains the value trigger and therefore only:changes returns always true, which triggers job1 again and ends in an endless loop. You can add a simple except rule to your job1 to prevent that:
job1:
stage: build
script:
- echo "JOb1"
- curl -X POST -F token=2342344444 -F "variables[TRIGGER_JOB]=job1" -F ref=master https://main.gitlab.myconmpanyxyz.com/api/v4/projects/1234/trigger/pipeline
only:
changes:
- job1.md
except:
variables:
- $CI_PIPELINE_SOURCE == "trigger"
You can find more information on only/except:changes in the documentation.
I have below piece of code :
Converter.delay.convert("some params")
Now I want this job to be run for max 1 minute. If exceeded,delayed job should raise the exception.
I tried setting up
Delayed::Worker.max_run_time = 1.minute
but it seems it sets a timeout on the worker , not on the job.
Converter class is defined in RAILS_ROOT/lib/my_converter.rb
Timeout in the job itself
require 'timeout'
class Converter
def self.convert(params)
Timeout.timeout(60) do
# your processing
end
end
end
Delayed::Worker.max_run_time=1.minute
Its the max time on each task given to worker. The execution of any task takes more than specified we get exception raised as .
execution expired (Delayed::Worker.max_run_time is only 1 minutes)
The worker continue to run and process next tasks.
I'm doing some experiments on a computing cluster. My algorithm has two steps. The first one writes its outputs to some files which will be used by the second step. The dependecies are 1 to n meaning one step2 programs needs the output of n step1 program. I'm not sure what to do neither waist cluster resources nor keep the head node busy. My current solution is:
submit script (this runs on the head node)
for different params, p:
run step 1 with p
sleep some time based on the an estimate of how much step 1 takes
for different params, q:
run step 2 with q
step 2 algorithm (this runs on the computing nodes)
while files are not ready:
sleep a few minutes
do the step 2
Is there any better way to do this?
SGE provides both job dependencies and array jobs for that. You can submit your phase 1 computations an array job and then submit the phase 2 computation as a dependent job using the qsub -hold_jid <phase 1 job ID|name> .... This will make the phase 2 job wait until all the phase 1 computations have finished and then it will be released and dispatched. The phase 1 computations will run in parallel as long as there are enough slots in the cluster.
In a submission script it might be useful to specifiy holds by job name and name each array job in a unique way. E.g.
mkdir experiment_1; cd experiment_1
qsub -N phase1_001 -t 1-100 ./phase1
qsub -hold_jid phase1_001 -N phase2_001 ./phase2 q1
cd ..
mkdir experiment_2; cd experiment_2
qsub -N phase1_002 -t 1-42 ./phase1 parameter_file
qsub -hold_jid phase1_002 -N phase2_002 ./phase2 q2
cd ..
This will schedule 100 executions of the phase1 script as the array job phase1_001 and another 42 executions as the array job phase1_002. If there are 142 slots on the cluster, all 142 executions will run in parallel. Then one execution of the phase2 script will be dispatched after all tasks in the phase1_001 job have finished and one execution will be dispatched after all tasks in the phase1_002 job have finished. Again those can run in parallel.
Each taks in the array job will receive a unique $SGE_TASK_ID value ranging from 1 to 100 for the tasks in job phase1_001 and from 1 to 42 for the tasks in job phase1_002. From it you can compute the p parameter.