OK, so I've been asked to allow a "safe word" in our Git commit comments so that our CI build will skip if the developer wants to just commit without building. So, I set up a Conditional Step (multiple) with Execute shell in the Run? field. I'm running the following command to see if the safe word is in the commit message and if it is, "Don't run" the build and if it's not, to run the build. If I put a header in this shell (i.e. #!/bin/bash), I get an error. Without it, my script stops after setting the RUN variable. The IF..THEN..ELSE loop doesn't even begin to run. Here's my script..
RUN=$(git --no-pager log -1 --pretty=online:"%s" --grep "keyword")
if [ "$RUN" != "" ];
then exit 0
else
exit 666
fi
The output when the "safe word" is not present looks like this:
++ git --no-pager log -1 --pretty=online:%s --grep blech
+ RUN=
Run condition [Execute Shell] preventing perform for step [BuilderChain]
Finished: SUCCESS
With the "safe word" present, the output is this:
++ git --no-pager log -1 --pretty=online:%s --grep initializer
+ RUN='online:CVirtualBroker - Tweak construction to use initializer lists -
Run condition [Execute Shell] preventing perform for step [BuilderChain]
Finished: SUCCESS
Notice that the RUN variable gets set, then the script stops. What gives?
AJ
you have a semicolon at the end of If statement
if [ "$RUN" != "" ];
which make it stop...
The command after is exit 0 so I assume that your process stops
Related
I do not know whether here is a problem in one of my steps regarding an github-workflow-job.
The step looks like this:
- name: check_tf_code
id: tf_check
run: |
terraform fmt -check
echo '::set-output name=bash_return_Value::$?'
and the error output for this step in the log is:
Error: Process completed with exit code 3
What could be the problem here? Is GitHub not able to make a bash evaluation for $? to get the
info whether the last (terraform) command was successful or not?
Run steps are run with set -eo pipefail (see docs); this means that if any command has a non-zero exit status, the job aborts, including when the command is part of a pipeline.
In your case, this means that if terraform fmt -check has a non-zero exit status, the next line is never run.
Then, your output would be literally $? because Bash doesn't expand anything in single quotes. You'd have to use
echo "::set-output name=bash_return_Value::$?"
but you already know that when this line is hit, the value of $? is 0.
If you want to do something in case a command fails, you could do
if ! terraform fmt -check; then
# Do something
fi
because if conditions don't trigger the set -e behaviour.
I have this bash script (actually a part of https://github.com/ddollar/heroku-buildpack-multi/blob/master/bin/compile with echo I've added myself):
echo "[DEBUG] chmod done"
framework=$($dir/bin/detect $1)
echo "[DEBUG] $framework done"
And I see in the log:
[DEBUG] chmod done
Staging failed: Buildpack compilation step failed
And I do not see the second echo in the logs at all.
I do not know much bash unfortunately. Could anybody explain to me in what case the first echo performs and second do not? I always thought that both echo should always work no matter whether the second line succeeds or not.
It's not visible in your question, but clicking on your link, it says in the third line
set -e
This means to stop processing the script immediately whenever an error occurs. Comment that line, and the script should run through and also print the second echo statement.
Note that I didn't inspect what the script actually does and I cannot tell you if commenting out set -e is actually good advice or not.
From man set:
−e: When this option is on, when any command fails (for any of the reasons listed
in Section 2.8.1, Consequences of Shell Errors or by returning an exit status
greater than zero), the shell immediately shall exit with the following excep‐
tions:
1. The failure of any individual command in a multi-command pipeline shall not
cause the shell to exit. Only the failure of the pipeline itself shall be
considered.
2. The −e setting shall be ignored when executing the compound list following
the while, until, if, or elif reserved word, a pipeline beginning with the
! reserved word, or any command of an AND-OR list other than the last.
3. If the exit status of a compound command other than a subshell command was
the result of a failure while −e was being ignored, then −e shall not apply
to this command.
This requirement applies to the shell environment and each subshell environment
separately. For example, in:
set -e; (false; echo one) | cat; echo two
the false command causes the subshell to exit without executing echo one; how‐
ever, echo two is executed because the exit status of the pipeline (false; echo
one) | cat is zero.
I would like to setup a error handling on my shell script where is my invocation of script fail with error, I should be able to stop the executin and flow should nt go on next line.
Like In my main script, I am making call to below script
sh /usr/oracle/StopServer.sh
if this script fail with error, my next script on this main file should not execute.. pls help.
You can check the return value of the command execution, one way to do this is:
sh /usr/oracle/StopServer.sh
if [ $? -ne 0 ]; then
# exit or take action
fi
it should do the trick
Here you go.
sh /usr/oracle/StopServer.sh && sh my_next_line_that_only_happens_if_Stop_server_exits_without_error
For more information, the && operator in bash (and most languages) exhibits McCarthy evaluation which is basically just lazy evaluation for boolean conditionals. what this means is that for an and (&&) then the second term of the and expression will only be evaluated if the first part is true (because otherwise the result is garuanteed to be false. Similarly if we did A || B (or) B would only be executed if A were false, which means it returned with an exit code of 0 and thus failed. If you had a program that you wanted to execute if the program exited normally and another that you want to execute if it exited with a failed state (I'm going to call them normal and fail) then you could execute them like so:
sh condition.sh && sh normal.sh || sh fail.sh
EDIT:
if [ sh condition.sh ]; then
# do whatever
else
#this is what to do if it failed
fi
EDIT #2:
If you want to see what happens try running this:
if [ ls -badoption ] ; then
echo "passed "
else
echo "failed"
fi
# result follows
# zsh: parse error: condition expected: ls
# failed
it will fail as there is a bad option and failed will be run, if you had just put ls it would have echoed passed, this is where you can either exit or run the script depending on what path you take.
I am writing a script that executes around 10 back-end processes in sequence, depending on if the previous process was executed without any errors.
Now let's assume the scenario, in which lets say 5th process failed and script came out. But I want to code it in a way such that, when next time user runs it(after removing the error because of which script exited last time), he should be able to run from 5th process onwards and not again from 1st process.
To be more specific, assume following is the script:
Script Starts
Process1
if [ $? -eq 0 ] then
Process2
if [ $? -eq 0 ] then
Process3
if [ $? -eq 0 ] then
..
..
..
..
if [ $? -eq 0 ] then
Process10
else
exit
So here the script will exit anytime if any one of the process fails to complete with status 0. So again, if process5 fails, and user corrects the problem and restarts script, the script should start with process5 again and not process1 or at least there should be an option to user if he wants to resume the script or start it back from beginning i.e. process1.
What all possible ways we can code this kind of script, also please bear in mind, I am not allowed to use a temporary db, where I can store the status of each process.
I need to code in sh (shell script) in unix.
A simple solution would be to write stamp files:
#/bin/sh
set -e # Automatically abort if any simple command fails
if ! test -f cmd1-stamp; cmd1; fi
touch cmd1-stamp
if ! test -f cmd2-stamp; cmd2; fi
touch cmd2-stamp
When the script executes, if cmd1-stamp exists, cmd1 is not executed. Otherwise, cmd1 is executed. The script will abort if it fails. Note that it is very tempting to write test -f cmd1-stamp || cmd1, and this seems to work ( in bash ) but the shell specs state that the shell shall abort if the simple command that fails is not a part of an AND or OR list, and I suspect this is (yet another) instance of bash not conforming to the spec. (Although it doesn't seem to specify that the shell shall not abort if the failing command is part of an AND or OR list.)
I have a Hudson job that periodically merges changes from an upstream bazaar repository.
Currently, when there are no changes upstream, Hudson reports this job as a failure because the bzr commit command returns with an error. My script looks like something this:
bzr branch lp:~lorinh/project/my-local-branch
cd my-local-branch
REV_UPSTREAM=`bzr version-info lp:project --custom --template="{revno}"`
bzr merge lp:project
bzr commit -m "merged upstream version ${REV_UPSTREAM}"
./run_tests.sh
bzr push lp:~lorinh/project/my-local-branch
If there are no changes to merge, the Hudson console output looks something like this:
+ bzr branch lp:~lorinh/project/my-local-branch
Branched 807 revision(s).
+ bzr merge lp:project
Nothing to do.
+ bzr commit -m merged upstream version 733
Committing to: /var/lib/hudson/jobs/merge-upstream/workspace/myproject/
aborting commit write group: PointlessCommit(No changes to commit)
bzr: ERROR: No changes to commit. Use --unchanged to commit anyhow.
Sending e-mails to: me#example.com
Finished: FAILURE
The problem is that I don't want Hudson to report this as a failure. How do I modify my commands so the script terminates at the failed commit, but it isn't interpreted by Hudson as an error? I tried changing the commit command to:
bzr commit -m "merged upstream version ${REV_UPSTREAM}" || exit
But that didn't work.
(Note: I realize I could use Hudson's "Poll SCM" instead of "Build periodically". However, with bazaar, if somebody does a push with local commits that were done before the most recent modifications, then Hudson won't detect a change to the repository.)
You were very close! Here's the corrected version of what you were trying:
bzr commit -m "merged upstream version ${REV_UPSTREAM}" || exit 0
This now does what you asked for, but isn't perfect. I'll get to that later.
Note the tiny important change from your version - we are now being explicit that we should exit with code 0 (success), if the bzr command does not do so. In your version, exit (with no argument) will terminate your script but return the exit code of the last command executed - in this case the bzr commit.
More about exit
How do we find out about this behaviour of exit? The exit command is a shell built-in - to find documentation on it we use the help command:
help exit
Which on my machine tells me:
exit: exit [n]
Exit the shell.
Exits the shell with a status of N. If N is omitted, the exit status
is that of the last command executed.
Here's a decent tutorial on exit and exit codes in the bash shell
Hudson and exit codes
Hudson follows this common convention of interpreting exit code 0 as success, and any other code as failure. It will flag your build as a failure if the build script it executes exits with a non-zero code.
Why your script is stopping after the bzr commit
If, as you say, you have the following and your script is stopping after the bzr commit...
bzr commit -m "merged upstream version ${REV_UPSTREAM}"
./run_tests.sh
... I suspect your script has an instruction such as set -e or is being invoked with something like bash -e build_script.sh
Either of these tells the shell to exit immediately if a command exits with a non-zero status, and to pass along that same "failure" exit code. (There are subtleties - see footnote 1).
Disabling exit-on-error
While this behaviour of exiting on error is extremely useful, sometimes we'd like to disable it temporarily. You've found one way, in
bzr commit whatever || true
We can also disable the error-checking with set +e.
Here's a pattern you may find useful. In it we will:
Disable exit-on-error (with set +e)
run the command which may error bzr commit whatever
capture its exit code ($?) for later inspection
Re-enable exit-on-error (with set -e)
Test and act upon the exit code of any commands
Let's implement that. Again we'll exit 0 (success) if the bzr command failed.
set +e
bzr commit whatever
commit_status=$?
set -e
if [[ "$commit_status" != "0" ]]; then
echo "bzr commit finds nothing to do. Build will stop, with success"
exit 0
fi
echo "On we go with the rest of the build script..."
Note that we bracket as little as possible with set +e / set -e. If we have typos in our script in that section, they won't stop the script and there'll be chaos. Read the section "Avoiding set -e" in the post "Insufficiently known POSIX shell features" for more ideas.
What's wrong with foo || exit 0 ?
As I mentioned earlier, there's a problem with our first proposed solution. We've said that when bzr commit is non-zero (i.e. it doesn't commit normally) we'll always stop and indicate success. This will happen even if bzr commit fails for some other reason (and with some other non-zero exit code): perhaps you've made a typo in your command invocation, or bzr can't connect to the repo.
In at least some of these cases, you'd probably want the build to be flagged as a failure so you can do something about it.
Towards a better solution
We want to be specific about which non-zero exit codes we expect from bzr, and what we'll do about each.
If you look back at the set +e / set -e pattern above, it shouldn't be difficult to expand the conditional logic (if) above into something that can deal with a number of specific exit codes from bzr, and with a catch-all for unanticipated exit codes which (I suggest) fails the build and reports the offending code and command.
To find out the exit codes for any command, read the docs or run the command and then run echo $? as the next command. $? holds the exit code of the previous command.
Footnote 1: The exit-on-error behaviour switched with set -e has some subtleties you'll need to read up on, concerning behaviour when commands are in pipelines, conditional statements and other constructs.
given that bzr doesn't seem to emit a correct exit code (based on you bzr ... || exit example), one solution is to capture the output of bzr and then scan for ERROR or other.
bzr commit -m "merged upstream version ${REV_UPSTREAM}" 2>&1 | tee /tmp/bzr_tmp.$$
case $( < /tmp/bzr_tmp.$$ ) in
*ERROR* )
printf "error running bzr, found error msg = $(< /tmp/bzr_tmp.$$)\n"
exit 1
;;
* )
: # everything_OK this case target
# just to document the default action of 'nothing' ;-)
;;
esac
A slightly easier case target regex, based on your sample output, would be *FAILURE ) ....
the $( < file ) is a newer feature of the shell that you can think of as $(cat file), but is more efficient in use of process resources, as it doesn't need to launch a new process (cat), to dump a file.
I hope this helps.