Going through the Bash reference guide, trying to understand nullglob and bash in general. But when I try:
$shopt -q nullglob
I get no return. If nullglob is off, shouldn't I get a 0 return? I don't understand -q option
From the description of the -q option in the Bash
Suppresses normal output; the return status indicates whether the optname is set or unset.
The variable $? contains the status of the last command.
shopt -q nullglob
echo $?
That will print 0 if it's set, 1 if it's not set.
But it's more useful in an if:
if shopt -q nullglob
then echo nullglob is set
else echo nullglob is not set
fi
Related
Given the following contrived code:
#!/usr/bin/env bash
set -Eeuo pipefail
shopt -s inherit_errexit
echo 'before'
mapfile -t tuples < <(exit 1)
# ^ what option do I need to enable so the error exit code from this is not ignored
echo 'after'
Which produces:
before
after
Is there a set or shopt option that can be turned on such that <(exit 1) will cause the caller to inherit the failure, and thus preventing after from being executed? Such as what inherit_errexit and pipefail do in other contexts.
In bash 4.4 or later, process substitutions will set $!, which means you can wait on that process to get its exit status.
#!/usr/bin/env bash
set -Eeuo pipefail
shopt -s inherit_errexit
echo 'before'
mapfile -t tuples < <(exit 1)
wait $!
echo 'after'
mapfile itself (in general) won't have a non-zero status, because it's perfectly happy read what, if anything, the process substitution produces.
You can assign a variable with the output of the command. The variable assignment propagates errors from the command substitution.
t=$(exit 1)
echo 'after'
mapfile -t tuples <<<"$t"
If you have Bash 4.2 or later, since you are already setting errexit and pipefail, you can avoid the problem by using:
...
shopt -s lastpipe
exit 1 | mapfile -t tuples
shopt -s lastpipe causes the last command in a pipeline to be run in the current shell. See how does shopt -s lastpipe affect bash script behavior?. In this case it means that a tuples value read by mapfile can be accessed later in the code.
Trying to open a bash shell with limited command capability.
Tried command line options like -r restriction but doesn't give intended result. Also tried shopt & unset commands.
bash --noprofile --noediting --verbose --version --init-file test.sh
unset ls
shopt -u -o history
Start a bash shell with only few built-in commands. For example cd, ls, cat only.
The usage of such a shell would be for read only purposes for Directory Navigation, Listing & File viewing purpose
You can take the list of all builtins and declare functions with the same name.
I did it like this:
File bash_limited.sh:
#!/bin/bash
export PATH=
eval "$(
echo '
:
.
[
alias
bg
bind
break
builtin
caller
cd
command
compgen
complete
compopt
continue
declare
dirs
disown
echo
enable
eval
exec
exit
export
fc
fg
getopts
hash
help
history
jobs
kill
let
local
logout
mapfile
popd
printf
pushd
pwd
read
readarray
readonly
return
set
shift
shopt
source
test
times
trap
type
typeset
ulimit
umask
unalias
unset
wait
' |
while IFS= read -r line; do
case "$line" in
''|ls|cat|cd|return|printf) continue; ;;
esac
printf "%s\n" "function $line () { /bin/printf -- 'bash: $line: Command not found.\n' >&2; return 127; }"
done
echo 'function ls() { /bin/ls "$#"; }'
echo 'function cat() { /bin/cat "$#"; }'
)" ## eval
Then I open a new shell and do:
$ source bash_limited.sh
after that it's just:
$ .
bash: .: Command not found.
$ :
bash: :: Command not found.
$ source
bash: source: Command not found.
$ declare
bash: declare: Command not found.
You can also use some chroot techniques with some other PATH restriction and it will be hard to get out.
Problem
The execution of a Bash script fails with the following error message when the 'extglob' option is set inside a subshell:
/tmp/foo.sh: line 7: syntax error near unexpected token `('
#!/usr/bin/env bash
set -euo pipefail
(
shopt -s extglob
for f in ?(.)!(|+(.)|vendor); do
echo "$f"
done
)
It fails in the same manner inside a function:
#!/usr/bin/env bash
set -euo pipefail
list_no_vendor () {
shopt -s extglob
for f in ?(.)!(|+(.)|vendor); do
echo "$f"
done
}
list_no_vendor
Investigation
In both cases, the script executes successfully when the option is set globally, outside of the subshell or function.
Surprisingly, when set locally, the 'extglob' option appears to be effectively enabled in both the subshell and function:
#!/usr/bin/env bash
set -euo pipefail
(
shopt -s extglob
echo 'In the subshell:' "$(shopt extglob)"
)
list_no_vendor () {
shopt -s extglob
echo 'In the function:' "$(shopt extglob)"
}
echo 'In the main shell:' "$(shopt extglob)"
list_no_vendor
Output:
In the subshell: extglob on
In the main shell: extglob off
In the function: extglob on
This makes the syntax error extremely puzzling to me.
Workaround
Passing a heredoc to the bash command works.
#!/usr/bin/env bash
set -euo pipefail
bash <<'EOF'
shopt -s extglob
echo 'In the child:' "$(shopt extglob)"
EOF
echo 'In the parent:' "$(shopt extglob)"
Output:
In the child: extglob on
In the parent: extglob off
However I would be curious to understand the gist of the problem here.
extglob is a flag used by the parser. Functions, compound commands, &c. are parsed in entirety ahead of execution. Thus, extglob must be set before that content is parsed; setting it at execution time but after parse time does not have any effect for previously-parsed content.
This is also why you can't run shopt -s extglob; ls !(*.txt) as a one-liner (when extglob is previously unset), but must have a newline between the two commands.
Not as an example of acceptable practice, but as an example demonstrating the behavior, consider the following:
#!/usr/bin/env bash
(
shopt -s extglob
# Parse of eval'd code is deferred, so this succeeds
eval '
for f in ?(.)!(|+(.)|vendor); do
echo "$f"
done
'
)
No such error takes place here, because parsing of the content passed to eval happens only after the shopt -s extglob was executed, rather than when the block of code to be run in a subshell is parsed.
I'm writing a shell script that loops over some values and run a long command line for each value. I'd like to print out these commands along the way, just like make does when running a makefile. I know I could just "echo" all commands before running them, but it feels inelegant. So I'm looking at set -x and similar mechanisms instead :
#!/bin/sh
for value in a long list of values
do
set -v
touch $value # imagine a complicated invocation here
set +v
done
My problem is: at each iteration, not only is the interresting line printed out, but also the set +x line as well. Is it somehow possible to prevent that ? If not, what workaround do you recommend ?
PS: the MWE above uses sh, but I also have bash and zsh installed in case that helps.
Sandbox it in a subshell:
(set -x; do_thing_you_want_traced)
Of course, changes to variables or the environment made in that subshell will be lost.
If you REALLY care about this, you could also use a DEBUG trap (using set -T to cause it to be inherited by functions) to implement your own set -x equivalent.
For instance, if using bash:
trap_fn() {
[[ $DEBUG && $BASH_COMMAND != "unset DEBUG" ]] && \
printf "[%s:%s] %s\n" "$BASH_SOURCE" "$LINENO" "$BASH_COMMAND"
return 0 # do not block execution in extdebug mode
}
trap trap_fn DEBUG
DEBUG=1
# ...do something you want traced...
unset DEBUG
That said, emitting BASH_COMMAND (as a DEBUG trap can do) is not fully equivalent of set -x; for instance, it does not show post-expansion values.
You want to try using a single-line xtrace:
function xtrace() {
# Print the line as if xtrace was turned on, using perl to filter out
# the extra colon character and the following "set +x" line.
(
set -x
# Colon is a no-op in bash, so nothing will execute.
: "$#"
set +x
) 2>&1 | perl -ne 's/^[+] :/+/ and print' 1>&2
# Execute the original line unmolested
"$#"
}
The original command executes in the same shell under an identity transformation. Just prior to running, you get a non-recursive xtrace of the arguments. This allows you to xtrace the commands you care about without spamming stederr with duplicate copies of every "echo" command.
# Example
for value in $long_list; do
computed_value=$(echo "$value" | sed 's/.../...')
xtrace some_command -x -y -z $value $computed_value ...
done
Next command disables 'xtrace' option:
$ set +o xtrace
I thought of
set -x >/dev/null 2>1; echo 1; echo 2; set +x >/dev/null 2>&1
but got
+ echo 1
1
+ echo 2
2
+ 1> /dev/null 2>& 1
I'm surprised by these results. .... But
set -x ; echo 1; echo 2; set +x
+ echo 1
1
+ echo 2
2
looks to meet your requirement.
I saw similar results when I put each statement on its only line (excepting the set +x)
IHTH.
$ cat test.sh
set -eu
echo "`wc -l < $DNE`"
echo should not get here
$ /bin/bash test.sh
test.sh: line 2: DNE: unbound variable
should not get here
I'm running bash version 4.1.2. Is there a way to make sure all such usage of unbound variables in subshells cause the script to exit without having to modify each call involving a subshell?
The better solution to ensure variable sanitisation
#!/usr/bin/env bash
set -eu
if [[ ${1-} ]]; then
DNE=$1
else
echo "ERROR: Please enter a valid filename" 1>&2
exit 1
fi
By putting a hyphen in to the variable name inside curly braces like this allows bash to sanely handle the variable being undefined. I also highly recommend looking at the Google shell style guide, it's a great reference https://google.github.io/styleguide/shell.xml
[[ -z ${variable-} ]] \
&& echo "ERROR: Unset variable \${variable}" \
&& exit 1 \
|| echo "INFO: Using variable (${variable})"
Use a temporary variable so as to let test.sh process know about the failure of wc. You can change it to:
#!/bin/bash
set -eu
out=$(wc -l < $DNE)
echo $out
echo should not get here
Now, you won't see the should not get here if wc fails.