How can I run more complex job dependencies via Rundeck - bash

Question

I'm currently testing Rundeck for some PoCs I'm working on.
I currently only have it running on a single machine via docker, so I don't have access to multiple nodes.
I want to simulate the following job dependencies:
JobA is needed for JobB and JobC. But JobB and JobC must run in parallel.
JobC must run if the current local time is 9:00am. Doesn't matter if Job A is finished/started yet or not.
All this should be able to easily scale and expand up to several hundreds of Jobs.
Can some of you guys possibly help me?
I have tried several configurations including the Plug-in Job State. I somehow can't get this to work. The best I can do is run all the jobs either sequential or parallel.

Answer 1

The best (and easiest) way to achieve that is to use the ruleset strategy (only available on PagerDuty Process Automation On Prem, formerly "Rundeck Enterprise") calling your jobs using the job reference step.
So, on the community version, the "workaround" is to call the jobs from a Parent Job via Rundeck API, which basically is about scripting the custom workflow behavior, e.g:
JobA
- defaultTab: nodes
description: ''
executionEnabled: true
id: b156b3ed-fde6-4fdc-af81-1ee919edc4ff
loglevel: INFO
name: JobA
nodeFilterEditable: false
plugins:
ExecutionLifecycle: null
scheduleEnabled: true
sequence:
commands:
- exec: whoami
keepgoing: false
strategy: node-first
uuid: b156b3ed-fde6-4fdc-af81-1ee919edc4ff
JobB
- defaultTab: nodes
description: ''
executionEnabled: true
id: 4c46c795-20c9-47a8-aa2e-e72f183b150f
loglevel: INFO
name: JobB
nodeFilterEditable: false
plugins:
ExecutionLifecycle: null
scheduleEnabled: true
sequence:
commands:
- exec: whoami
keepgoing: false
strategy: node-first
uuid: 4c46c795-20c9-47a8-aa2e-e72f183b150f
JobC (runs every day at 9.00 AM, no matter the JobA execution).
- defaultTab: nodes
description: ''
executionEnabled: true
id: ce45c4d1-1350-407b-8001-2d90daf49eaa
loglevel: INFO
name: JobC
nodeFilterEditable: false
plugins:
ExecutionLifecycle: null
schedule:
month: '*'
time:
hour: '09'
minute: '00'
seconds: '0'
weekday:
day: '*'
year: '*'
scheduleEnabled: true
sequence:
commands:
- exec: whoami
keepgoing: false
strategy: node-first
uuid: ce45c4d1-1350-407b-8001-2d90daf49eaa
ParentJob (all the behavior logic is inside a bash script, needs jq tool to work)
- defaultTab: nodes
description: ''
executionEnabled: true
id: 6e67020c-7293-4674-8d56-5db811ae5745
loglevel: INFO
name: ParentJob
nodeFilterEditable: false
plugins:
ExecutionLifecycle: null
scheduleEnabled: true
sequence:
commands:
- description: A friendly starting message
exec: echo "starting..."
- description: Execute JobA and Get the Status
fileExtension: .sh
interpreterArgsQuoted: false
plugins:
LogFilter:
- config:
invalidKeyPattern: \s|\$|\{|\}|\\
logData: 'true'
regex: ^(job_a_exec_id)\s*=\s*(.+)$
type: key-value-data
script: "# execute JobA\nexec_id=$(curl -s -X POST \\\n \"http://localhost:4440/api/41/job/b156b3ed-fde6-4fdc-af81-1ee919edc4ff/run\"\
\ \\\n --header \"Accept: application/json\" \\\n --header \"X-Rundeck-Auth-Token:\
\ 45hNfblPiF6C1l9CfJeG37l08oAgs0Gd\" | jq .id)\n \necho \"job_a_exec_id=$exec_id\""
scriptInterpreter: /bin/bash
- description: Time to execute the Job
exec: sleep 2
- description: 'Get the execution Status '
fileExtension: .sh
interpreterArgsQuoted: false
plugins:
LogFilter:
- config:
invalidKeyPattern: \s|\$|\{|\}|\\
logData: 'false'
regex: ^(job_a_exec_status)\s*=\s*(.+)$
type: key-value-data
script: "exec_status=$(curl -s -X GET \\\n 'http://localhost:4440/api/41/execution/#data.job_a_exec_id#'\
\ \\\n --header 'Accept: application/json' \\\n --header 'X-Rundeck-Auth-Token:\
\ 45hNfblPiF6C1l9CfJeG37l08oAgs0Gd' | jq -r .status)\n \necho \"job_a_exec_status=$exec_status\""
scriptInterpreter: /bin/bash
- description: 'Run JobB and JobC pararelly, depending on JobA Execution Status'
fileExtension: .sh
interpreterArgsQuoted: false
script: "# Now execute JobB and JobC depending on the JobA final status (dependency)\n\
\nif [ #data.job_a_exec_status# = \"succeeded\" ]; then\n echo \"Job A execition\
\ is OK, running JobB and JobC pararelly...\"\n \n # JobB \n curl -X POST\
\ \\\n \"http://localhost:4440/api/41/job/4c46c795-20c9-47a8-aa2e-e72f183b150f/run\"\
\ \\\n --header \"Accept: application/json\" \\\n --header \"X-Rundeck-Auth-Token:\
\ 45hNfblPiF6C1l9CfJeG37l08oAgs0Gd\" &\n \n # JobC\n curl -X POST \\\n\
\ \"http://localhost:4440/api/41/job/ce45c4d1-1350-407b-8001-2d90daf49eaa/run\"\
\ \\\n --header \"Accept: application/json\" \\\n --header \"X-Rundeck-Auth-Token:\
\ 45hNfblPiF6C1l9CfJeG37l08oAgs0Gd\" &\nelse\n echo \"Job A has failed, dependencey\
\ failed\"\nfi"
scriptInterpreter: /bin/bash
- description: A friendly ending message
exec: echo "done."
keepgoing: false
strategy: node-first
uuid: 6e67020c-7293-4674-8d56-5db811ae5745
Parent Job's JobB/JobC launching script:
# Now execute JobB and JobC depending on the JobA final status (dependency)
if [ #data.job_a_exec_status# = "succeeded" ]; then
echo "Job A execition is OK, running JobB and JobC pararelly..."
# JobB
curl -X POST \
"http://localhost:4440/api/41/job/4c46c795-20c9-47a8-aa2e-e72f183b150f/run" \
--header "Accept: application/json" \
--header "X-Rundeck-Auth-Token: 45hNfblPiF6C1l9CfJeG37l08oAgs0Gd" &
# JobC
curl -X POST \
"http://localhost:4440/api/41/job/ce45c4d1-1350-407b-8001-2d90daf49eaa/run" \
--header "Accept: application/json" \
--header "X-Rundeck-Auth-Token: 45hNfblPiF6C1l9CfJeG37l08oAgs0Gd" &
else
echo "Job A has failed, dependencey failed"
fi