Is there a generic way in a bash script to "try" something but continue if it fails? The analogue in other languages would be wrapping it in a try/catch and ignoring the exception.
Specifically I am trying to source an optional satellite script file:
. $OPTIONAL_PATH
But when executing this, if $OPTIONAL_PATH doesn't exist, the whole script screeches to a halt.
I realize I could check to see if the file exists before sourcing it, but I'm curious if there is a generic reusable mechanism I can use that will ignore the error without halting.
Update: Apparently this is not normal behavior. I'm not sure why this is happening. I'm not explicitly calling set -e anywhere ($- is hB), yet it halts on the error. Here is the output I see:
./script.sh: line 36: projects/mobile.sh: No such file or directory
I added an echo "test" immediately after the source line, but it never prints, so it's not anything after that line that is exiting. I am running Mac OS 10.9.
Update 2: Nevermind, it was indeed shebanged as #!/bin/sh instead of #!/bin/bash. Thanks for the informative answer, Kaz.
Failed commands do not abort the script unless you explicitly configure that mode with set -e.
With regard to Bash's dot command, things are tricky. If we invoke bash as /bin/sh then it bails the script if the . command does not find the file. If we invoke bash as /bin/bash then it doesn't fail!
$ cat source.sh
#!/bin/sh
. nonexistent
echo here
$ ./source.sh
./source.sh: 3: .: nonexistent: not found
$ ed source.sh
35
1s/sh/bash/
wq
37
$ ./source.sh
./source.sh: line 3: nonexistent: No such file or directory
here
It does respond to set -e; if we have #!/bin/bash, and use set -e, then the echo is not reached. So one solution is to invoke bash this way.
If you want to keep the script maximally portable, it looks like you have to do the test.
The behavior of the dot command aborting the script is required by POSIX. Search for the "dot" keyword here. Quote:
If no readable file is found, a non-interactive shell shall abort; an interactive shell shall write a diagnostic message to standard error, but this condition shall not be considered a syntax error.
Arguably, this is the right thing to do, because dot is used for including pieces of the script. How can the script continue when a whole chunk of it has not been found?
Otherwise arguably, this is braindamaged behavior inconsistent with the treatment of other commands, and so Bash makes it consistent in its non-POSIX-conforming mode. If programmers want a command to fail, they can use set -e.
I tend to agree with Bash. The POSIX behavior is actually more broken than initially meets the eye, because this also doesn't work the way you want:
if . nonexistent ; then
echo loaded
fi
Even if the command is tested, it still aborts the script when it bails.
Thank GNU-deness we have alternative utilities, with source code.
You have several options:
Make sure set -e wasn't used, or turn it off with set +e. Your bash script should not exit by default simply because the . command failed.
Test that the file exists prior to sourcing.
[ -f "$OPTIONAL_PATH" ] && . "$OPTIONAL_PATH"
This option is complicated by the fact that if $OPTIONAL_PATH does not contain
any slashes, . will still try to find the file in your path.
If you want to keep set -e on, "hide" the failure like this:
. "$OPTIONAL_PATH" || true
Even if the source fails, the exit status of the command list as a whole will be 0, due to the || true.
(Much of this is covered [better] by Kaz's answer, especially the references to the POSIX standard, but I wasn't sure when or if he would undelete his answer.)
This is not the default behavior. Did you set -e or use #!/bin/bash -e anywhere in your script, to make it automatically exit on failure?
If so, you can use
. $OPTIONAL_PATH || true
to continue anyways.
Related
consider script here:
set -e
make && make install
echo "SHOULD NOT BE HERE"
I expect that if make fails, the script will be aborted, but it's not:
make: *** No targets specified and no makefile found. Stop.
SHOULD NOT BE HERE
But, if I changed it like this:
set -e
make
make install
echo "SHOULD NOT BE HERE"
It works as expected:
make: *** No targets specified and no makefile found. Stop.
Why this happens?
Due to make && make install is commonly used in my build script, how should I use it correctly?
And please DO NOT link this question to Using set -e / set +e in bash with functions, it's not the same question.
Quoting from the answer to the question you linked:
Quoting sh(1) from FreeBSD, which explains this better than bash's man page:
-e errexit
Exit immediately if any untested command fails in non-interactive
mode. The exit status of a command is considered to be explicitly
tested if the command is part of the list used to control an if,
elif, while, or until; if the command is the left hand operand of
an “&&” or “||” operator; or if the command is a pipeline preceded
by the ! operator. If a shell function is executed and its exit
status is explicitly tested, all commands of the function are con‐
sidered to be tested as well.
errexit exits only if an untested command fails. Using && (or ||) will means that bash considers the command to the left of && to be explicitly tested (which in turn means that it will not be handled by errexit).
See also here (specifically the part about list constructs).
As far a I know, there is no way to achieve what you would like by setting bash options.
How to correctly pass to the script and substitute a variable that is already defined there?
My script test.sh:
#!/bin/bash
TARGETARCH=amd64
echo $1
When I enter:
bash test.sh https://example/$TARGETARCH
I want to see
https://example/amd64
but I actually see
https://example/
What am I doing wrong?
The first problem with the original approach is that the $TARGETARCH is removed by your calling shell before your script is ever invoked. To prevent that, you need to use quotes:
./yourscript 'https://example.com/$TARGETARCH'
The second problem is that parameter expansions only happen in code, not in data. This is, from a security perspective, a Very Good Thing -- if data were silently treated as code it would be impossible to write secure scripts handling untrusted data -- but it does mean you need to do some more work. The easy thing, in this case, is to export your variable and use GNU envsubst, as long as your operating system provides it:
#!/bin/bash
export TARGETARCH=amd64
substitutedValue=$(envsubst <<<"$1")
echo "Original value was: $1"
echo "Substituted value is: $substitutedValue"
See the above running in an online sandbox at https://replit.com/#CharlesDuffy2/EcstaticAfraidComputeranimation#replit.nix
Note the use of yourscript instead of test.sh here -- using .sh file extensions, especially for bash scripts as opposed to sh scripts, is an antipattern; the essay at https://www.talisman.org/~erlkonig/documents/commandname-extensions-considered-harmful/ has been linked by the #bash IRC channel on this topic for over a decade.
For similar reasons, changing bash yourscript to ./yourscript lets the #!/usr/bin/env bash line select an interpreter, so you aren't repeating the "bash" name in multiple places, leading to the risk of those places getting out of sync with each other.
When posting this question originally, I totally misworded it, obtaining another, reasonable but different question, which was correctly answered here.
The following is the correct version of the question I originally wanted to ask.
In one of my Bash scripts, there's a point where I have a variable SCRIPT which contains the /path/to/an/exe which, when executed, outputs a line to be executed.
What my script ultimately needs to do, is executing that line to be executed. Therefore the last line of the script is
$($SCRIPT)
so that $SCRIPT is expanded to /path/to/an/exe, and $(/path/to/an/exe) executes the executable and gives back the line to be executed, which is then executed.
However, running shellcheck on the script generates this error:
In setscreens.sh line 7:
$($SCRIPT)
^--------^ SC2091: Remove surrounding $() to avoid executing output.
For more information:
https://www.shellcheck.net/wiki/SC2091 -- Remove surrounding $() to avoid e...
Is there a way I can rewrite that $($SCRIPT) in a more appropriate way? eval does not seem to be of much help here.
If the script outputs a shell command line to execute, the correct way to do that is:
eval "$("$SCRIPT")"
$($SCRIPT) would only happen to work if the command can be completely evaluated using nothing but word splitting and pathname expansion, which is generally a rare situation. If the program instead outputs e.g. grep "Hello World" or cmd > file.txt then you will need eval or equivalent.
You can make it simple by setting the command to be executed as a positional argument in your shell and execute it from the command line
set -- "$SCRIPT"
and now run the result that is obtained by expansion of SCRIPT, by doing below on command-line.
"$#"
This works in case your output from SCRIPT contains multiple words e.g. custom flags that needs to be run. Since this is run in your current interactive shell, ensure the command to be run is not vulnerable to code injection. You could take one step of caution and run your command within a sub-shell, to not let your parent environment be affected by doing ( "$#" ; )
Or use shellcheck disable=SCnnnn to disable the warning and take the occasion to comment on the explicit intention, rather than evade the detection by cloaking behind an intermediate variable or arguments array.
#!/usr/bin/env bash
# shellcheck disable=SC2091 # Intentional execution of the output
"$("$SCRIPT")"
By disabling shellcheck with a comment, it clarifies the intent and tells the questionable code is not an error, but an informed implementation design choice.
you can do it in 2 steps
command_from_SCRIPT=$($SCRIPT)
$command_from_SCRIPT
and it's clean in shellcheck
I am trying to run the next code on bash. It is suppose to work but it does not.
Can you help me to fix it? I am starting with programming.
The code is this:
for i in {1:5}
do
cd path-to-folder-number/"$i"0/
echo path-to-folder-number/"$i"0/
done
EXAMPLE
I want to iterate over folders which have numbers (10,20..50), and so it changes directory from "path-to-folder-number/10/" to "path-to-folder-number/20/" ..etc
I replace : with .. but it is not working yet. When the script is applied i get:
can't cd to path-to-folder-number/{1..5}0/
I think there are three problems here: you're using the wrong shell, the wrong syntax for a range, and if you solved those problems you may also have trouble with successive cds not doing what you want.
The shell problem is that you're running the script with sh instead of bash. On some systems sh is actually bash (but running in POSIX compatibility mode, with some advanced features turned off), but I think on your system it's a more basic shell that doesn't have any of the bash extensions.
The best way to control which shell a script runs with is to add a "shebang" line at the beginning that says what interpreter to run it with. For bash, that'd be either #!/bin/bash or #!/usr/bin/env bash. Then run the script by either placing it in a directory that's in your $PATH, or explicitly giving the path to the script (e.g. with ./scriptname if you're in the same directory it's in). Do not run it with sh scriptname, because that'll override the shebang and use the basic shell, and it won't work.
(BTW, the name "shebang" comes from the "#!" characters the line starts with -- the "#" character is sometimes called "sharp", and "!" is sometimes called "bang", so it's "sharp-bang", which gets abbreviated to "shebang".)
In bash, the correct syntax for a range in a brace expansion is {1..5}, not {1:5}. Note that brace expansions are a bash extension, which is why getting the right shell matters.
Finally, a problem you haven't actually run into yet, but may when you get the first two problems fixed: you cd to path-to-folder-number/10/, and then path-to-folder-number/20/, etc. You are not cding back to the original directory in between, so if the path-to-folder-number is relative (i.e. doesn't start with "/"), it's going to try to cd to path-to-folder-number/10/path-to-folder-number/20/path-to-folder-number/30/path-to-folder-number/40/path-to-folder-number/50/.
IMO using cd in scripts is generally a bad idea, because there are a number of things that can go wrong. It's easy to lose track of where the script is going to be at which point. If any cd fails for any reason, then the rest of the script will be running in the wrong place. And if you have any files specified by relative paths, those paths become invalid as soon as you cd someplace other than the original directory.
It's much less fragile to just use explicit paths to refer to file locations within the script. So, for example, instead of cd "path-to-folder-number/${i}0/"; ls, use ls "path-to-folder-number/${i}0/".
For up ranges the syntax is:
for i in {1..5}
do
cd path-to-folder-number/"$i"0/
echo $i
done
So replace the : with ..
To get exactly what you want you can use this:
for i in 10 {20..50}
do
echo $i
done
You can also use seq :
for i in $(seq 10 10 50); do
cd path-to-folder-number/$i/
echo path-to-folder-number/$i/
done
I have a Jenkins job in which I want to read a file from a directory using the shell and pass that file in ant test step.
Say the file I want to read is /home/xxx/y.txt. The name of the file always changes but there will be only single file with .txt extension at any given point in that directory.
So, I am trying to pass that file in the "Execute Shell" build action as ant -Dfile=/home/xxx/*.txt but the build is "unable to read the file".
The shell won't expand -Dfile=/home/xxx/*.txt into -Dfile=/home/xxx/y.txt because -Dfile=/home/xxx/y.txt is not a file. However, the shell will expand /home/xxx/*.txt into /home/xxx/y.txt. You can get the result you want using command substitution:
ant -Dfile=`echo /home/xxx/*.txt`
To protect against whitespace in the file path, you can use double quotes around the backticks:
ant -Dfile="`echo /home/xxx/*.txt`"
General tip: If you are having trouble with a shell script running in a Jenkins job, try enabling command tracing and view the console output to help debug. Command tracing can be enabled in one of two ways (take your pick):
Pass -x as an option to the shebang at the beginning of the script. For example, replace #!/bin/sh with #!/bin/sh -x. All commands will be output on standard error before they are executed.
Place set -x somewhere in your script. Commands after this line will be traced.
Consider:
set -- /home/xxx/*.txt
{ [ "$#" -eq 1 ] && [ -e "$1" ]; } || {
echo "ERROR: There should be exactly one file matching /home/xxx/*.txt" >&2
exit 1
}
ant -Dfile="$1"
This has several advantages:
You're actually detecting the unexpected cases instead of letting it passed unnoticed when (not if) an impossible thing happens.
Everything is happening in a single shell -- there's no subshell performance impact.
Your filenames aren't being mangled at all -- all the odd corner cases (ie. names with literal backslashes, which echo is allowed by POSIX to mangle) are fully supported.
It's fully compliant with any POSIX shell.
There's also a caveat:
set -- /home/xxx/*.txt overrides "$#", the argument vector, in the current context. If you need to refer to arguments as "$1", "$2", etc. in the outside script, you might put this code inside a function.
file_name=(`/home/xxx/*.txt`)
ant -Dfile=${file_name}