I'm trying to set up a GitLab pipeline, so that certain exit_codes are okay for a script I'm executing.
I have tried both shell and a ruby script, but both seem to have the same behaviour.
test_job:
stage: build
image: ruby:3.0
script:
- chmod +x ci/wrap.sh
- ./ci/wrap.sh
allow_failure:
exit_codes:
- 64
As you can see, I am just executing the script and nothing more, my expectation would be, that the last script executed is used a the exit status for the job.
In the script I'm only calling exit 64, which should be a "allowed failure" in that case, the pipeline log however says that the job failed because of exit code 1:
How do I get GitLab to accept the exit code of this (or a ruby) script as the job exit code?
I found a way to fix this problem. Apparently Gitlab Runner uses the -e flag, which means that any non-zero exit code will cancel the job. This can be updated by using set +e, but then you still need to capture the actual exit code for the job.
Using $? in two different lines of the configuration does not work, because Gitlab does echo calls in-between them.
So the exit code needs to be captured directly, example:
script:
- set +e
- ruby "ci/example.rb" || EXIT_CODE=$?
- exit $EXIT_CODE
Here's my trick for turning off early failure and checking the result code later:
script:
- set +e +o pipefail
- python missing_module.py 2>&1 | grep found; result=$?
- set -e -o pipefail
- "[ $result == 0 ]"
This turns off early exit and runs a command that we consider to have an acceptable failure if "found" is in the error text. It then turns early exit back on and tests whether the exit code we saved was good or not.
Related
I have a freestyle Jenkins job that has a simple bash script for a build.
In the bash script, there is a for loop which sometimes returns a non-zero return code.
Thing is, Jenkins quits immediately when it happens. However, if during the run it gets a non-zero return code, the job needs to continue, and in the end mark the job as failed. (In other words, don't stop on failure but show the job failed when it's finished). (that's why I can't just append || true)
Is it possible to do? Thanks ahead!
If you set the bash option -e, (ie: #!/bin/bash -e) then the shell script will exit upon error. This is typically desired to avoid lots of testing for failures and still catching them, so many scripts have that configuration. Also, typically, this is NOT applicable inside loops if it-then-else constructs, so you should investigate what inner command you are invoking and what it is doing .. it may have an exit $? that is propagating.
If running a Jenkins execute shell step, you can add the first line #!/bin/bash +e and that may override the fail. Otherwise, try:
#!/bin/bash
FAILURE=false
set +e
for ENTRY in 1 2 3 4 5 .. N
do
command ${ENTRY}
[[ $? == 0 ]] || FAILURE=true
done
set -e
[[ ${FAILURE} == 'true' ]] && return 1
return 0
I am trying to run parallel commands on Azure using the bash script.
The script is working correctly when all are successful.
When one of them fails, the other processes are properly killed but the step is marked as successful. What am I missing?
- job: main
pool:
vmImageL 'ubuntu-latest'
steps:
# some steps before
- bash: |
PIDS=()
my_command_1 &
PIDS+=($!)
my_command_2 &
PIDS+=($!)
my_command_3 &
PIDS+=($!)
for pid in "${PIDS[#]}"; do
wait $pid
done
# some steps after
The same script used in CircleCI and on GitHub Actions is working.
Explicitly checking exit status is likely a safer bet. Although "set -eu" may work for the immediate use case, it can have unexpected behavior:
https://mywiki.wooledge.org/BashPitfalls#set_-euo_pipefail
Explicit check:
for pid in "${PIDS[#]}"; do
wait $pid
if [ $? -ne 0 ]; then
exit 1
fi
done
Found the issue.
By adding set -eu at the beginning of the bash script block, we ensure the step will return the error code when any of the commands fail.
I do not know whether here is a problem in one of my steps regarding an github-workflow-job.
The step looks like this:
- name: check_tf_code
id: tf_check
run: |
terraform fmt -check
echo '::set-output name=bash_return_Value::$?'
and the error output for this step in the log is:
Error: Process completed with exit code 3
What could be the problem here? Is GitHub not able to make a bash evaluation for $? to get the
info whether the last (terraform) command was successful or not?
Run steps are run with set -eo pipefail (see docs); this means that if any command has a non-zero exit status, the job aborts, including when the command is part of a pipeline.
In your case, this means that if terraform fmt -check has a non-zero exit status, the next line is never run.
Then, your output would be literally $? because Bash doesn't expand anything in single quotes. You'd have to use
echo "::set-output name=bash_return_Value::$?"
but you already know that when this line is hit, the value of $? is 0.
If you want to do something in case a command fails, you could do
if ! terraform fmt -check; then
# Do something
fi
because if conditions don't trigger the set -e behaviour.
Running this command inside a .gitlab-ci.yml:
task:
script:
- yes | true
- yes | someOtherCommandWhichNeedsYOrN
Returns:
$ yes | true
ERROR: Job failed: exit status 1
Any clues, ideas why this happens or how to debug this?
Setup:
Gitlab runner in a docker
If running with set -o pipefail, a failure at any stage in a shell pipeline will cause the entire pipeline to be considered failed. When yes tries to write to stdout but the program whose stdin that stdout is connected to is not reading, this will cause an EPIPE signal -- thus, an expected failure message, which the shell will usually ignore (in favor of treating only the last component of a pipeline as important for purposes of that pipeline's exit status).
Turn this off for the remainder of your current script with set +o pipefail
Explicitly ignore a single failure: { yes || :; } | true
Can't comment yet.
I would extract the script in to a file and run that file from the pipeline with some debug stuff in it and see if you can reproduce it.
Make sure you make it to to, and not past, the line in question.
I try the following to get some more info maybe?
( set -x ; yes | true ; echo status: "${PIPESTATUS[#]}" )
See if you have some weird chars in the file or some weird modes set.
Make sure you are in the right shell, true can be built in so worth checking.
Good luck.
Without using set -e, the script runs as expected, with all results correctly generated.
After adding set -e, it exits after this command:
./NameOfATool > result.txt
When I wrap set +e and set -e around that command, then the script terminates as expected.
Why would it exit, or what might be wrong with the command?
p.s. NameOfAToolis an executable compiled from C code. When I manually type that command, it runs ok without giving an error.
set -e will cause the script to exit if any command returns a non-zero exit status. (Well, there are a bunch of exceptions, but that's the general rule.) So, ./NameOfATool apparently returns a non-zero exit status. This might mean that it actually thinks there's an error, or it might mean that the program was poorly written and doesn't report an appropriate exit status for success, or it might mean that it uses special exit-status values to report specific things (much like the standard utility diff, which returns 0 for "same", 1 for "different", and 2 for "error").
Try set +e in your trap:
set -e;
trap 'x=$?; set +e; echo Hello; false; echo World; exit 22;' ERR
echo Testing
false
echo Never See This
Omit the set +e and you don't see the "World" as the non-zero exit code in the trap exits before the trap is completed.
As #ruakh said, this indicates that the tool is exiting with a nonzero (=error) status. You can prevent this from exiting the script by putting it in a compound command that always succeeds:
./NameOfATool > result.txt || true
If the tool exits with a nonzero status, it runs true, which succeeds; hence, the entire compound command is considered to have succeeded. If the command's exit status is significant (i.e. you need to be able to tell if it exited with status 0, 1, or 2), you can either record it for later use:
./NameOfATool > result.txt && toolStatus=0 || toolStatus=$?
...or use the status directly:
if ./NameOfATool > result.txt; then
# do things appropriate for exit status = 0
else
toolStatus=$?
# do things appropriate for exit status != 0
fi