Bash script throws syntax errors when the 'extglob' option is set inside a subshell or function - bash

Problem
The execution of a Bash script fails with the following error message when the 'extglob' option is set inside a subshell:
/tmp/foo.sh: line 7: syntax error near unexpected token `('
#!/usr/bin/env bash
set -euo pipefail
(
shopt -s extglob
for f in ?(.)!(|+(.)|vendor); do
echo "$f"
done
)
It fails in the same manner inside a function:
#!/usr/bin/env bash
set -euo pipefail
list_no_vendor () {
shopt -s extglob
for f in ?(.)!(|+(.)|vendor); do
echo "$f"
done
}
list_no_vendor
Investigation
In both cases, the script executes successfully when the option is set globally, outside of the subshell or function.
Surprisingly, when set locally, the 'extglob' option appears to be effectively enabled in both the subshell and function:
#!/usr/bin/env bash
set -euo pipefail
(
shopt -s extglob
echo 'In the subshell:' "$(shopt extglob)"
)
list_no_vendor () {
shopt -s extglob
echo 'In the function:' "$(shopt extglob)"
}
echo 'In the main shell:' "$(shopt extglob)"
list_no_vendor
Output:
In the subshell: extglob on
In the main shell: extglob off
In the function: extglob on
This makes the syntax error extremely puzzling to me.
Workaround
Passing a heredoc to the bash command works.
#!/usr/bin/env bash
set -euo pipefail
bash <<'EOF'
shopt -s extglob
echo 'In the child:' "$(shopt extglob)"
EOF
echo 'In the parent:' "$(shopt extglob)"
Output:
In the child: extglob on
In the parent: extglob off
However I would be curious to understand the gist of the problem here.

extglob is a flag used by the parser. Functions, compound commands, &c. are parsed in entirety ahead of execution. Thus, extglob must be set before that content is parsed; setting it at execution time but after parse time does not have any effect for previously-parsed content.
This is also why you can't run shopt -s extglob; ls !(*.txt) as a one-liner (when extglob is previously unset), but must have a newline between the two commands.
Not as an example of acceptable practice, but as an example demonstrating the behavior, consider the following:
#!/usr/bin/env bash
(
shopt -s extglob
# Parse of eval'd code is deferred, so this succeeds
eval '
for f in ?(.)!(|+(.)|vendor); do
echo "$f"
done
'
)
No such error takes place here, because parsing of the content passed to eval happens only after the shopt -s extglob was executed, rather than when the block of code to be run in a subshell is parsed.

Related

How to have bash inherit failures from stdin subshells

Given the following contrived code:
#!/usr/bin/env bash
set -Eeuo pipefail
shopt -s inherit_errexit
echo 'before'
mapfile -t tuples < <(exit 1)
# ^ what option do I need to enable so the error exit code from this is not ignored
echo 'after'
Which produces:
before
after
Is there a set or shopt option that can be turned on such that <(exit 1) will cause the caller to inherit the failure, and thus preventing after from being executed? Such as what inherit_errexit and pipefail do in other contexts.
In bash 4.4 or later, process substitutions will set $!, which means you can wait on that process to get its exit status.
#!/usr/bin/env bash
set -Eeuo pipefail
shopt -s inherit_errexit
echo 'before'
mapfile -t tuples < <(exit 1)
wait $!
echo 'after'
mapfile itself (in general) won't have a non-zero status, because it's perfectly happy read what, if anything, the process substitution produces.
You can assign a variable with the output of the command. The variable assignment propagates errors from the command substitution.
t=$(exit 1)
echo 'after'
mapfile -t tuples <<<"$t"
If you have Bash 4.2 or later, since you are already setting errexit and pipefail, you can avoid the problem by using:
...
shopt -s lastpipe
exit 1 | mapfile -t tuples
shopt -s lastpipe causes the last command in a pipeline to be run in the current shell. See how does shopt -s lastpipe affect bash script behavior?. In this case it means that a tuples value read by mapfile can be accessed later in the code.

Bash with few commands, whitelisting few built-in commands

Trying to open a bash shell with limited command capability.
Tried command line options like -r restriction but doesn't give intended result. Also tried shopt & unset commands.
bash --noprofile --noediting --verbose --version --init-file test.sh
unset ls
shopt -u -o history
Start a bash shell with only few built-in commands. For example cd, ls, cat only.
The usage of such a shell would be for read only purposes for Directory Navigation, Listing & File viewing purpose
You can take the list of all builtins and declare functions with the same name.
I did it like this:
File bash_limited.sh:
#!/bin/bash
export PATH=
eval "$(
echo '
:
.
[
alias
bg
bind
break
builtin
caller
cd
command
compgen
complete
compopt
continue
declare
dirs
disown
echo
enable
eval
exec
exit
export
fc
fg
getopts
hash
help
history
jobs
kill
let
local
logout
mapfile
popd
printf
pushd
pwd
read
readarray
readonly
return
set
shift
shopt
source
test
times
trap
type
typeset
ulimit
umask
unalias
unset
wait
' |
while IFS= read -r line; do
case "$line" in
''|ls|cat|cd|return|printf) continue; ;;
esac
printf "%s\n" "function $line () { /bin/printf -- 'bash: $line: Command not found.\n' >&2; return 127; }"
done
echo 'function ls() { /bin/ls "$#"; }'
echo 'function cat() { /bin/cat "$#"; }'
)" ## eval
Then I open a new shell and do:
$ source bash_limited.sh
after that it's just:
$ .
bash: .: Command not found.
$ :
bash: :: Command not found.
$ source
bash: source: Command not found.
$ declare
bash: declare: Command not found.
You can also use some chroot techniques with some other PATH restriction and it will be hard to get out.

shopt -q in terminal no return

Going through the Bash reference guide, trying to understand nullglob and bash in general. But when I try:
$shopt -q nullglob
I get no return. If nullglob is off, shouldn't I get a 0 return? I don't understand -q option
From the description of the -q option in the Bash
Suppresses normal output; the return status indicates whether the optname is set or unset.
The variable $? contains the status of the last command.
shopt -q nullglob
echo $?
That will print 0 if it's set, 1 if it's not set.
But it's more useful in an if:
if shopt -q nullglob
then echo nullglob is set
else echo nullglob is not set
fi

Why won't my version of bash accept function keyword?

My version of bash is:
bash --version
GNU bash, version 4.2.45(1)-release (x86_64-pc-linux-gnu)
If I do the prototype function
#!/bin/bash
function f()
{
echo "hello" $1
}
f "world"
I get Syntax error: "(" unexpected
Why is that?
Output of shopt is:
autocd off
cdable_vars off
cdspell off
checkhash off
checkjobs off
checkwinsize on
cmdhist on
compat31 off
compat32 off
compat40 off
compat41 off
direxpand off
dirspell off
dotglob off
execfail off
expand_aliases on
extdebug off
extglob on
extquote on
failglob off
force_fignore on
globstar off
gnu_errfmt off
histappend on
histreedit off
histverify off
hostcomplete off
huponexit off
interactive_comments on
lastpipe off
lithist off
login_shell off
mailwarn off
no_empty_cmd_completion off
nocaseglob off
nocasematch off
nullglob off
progcomp on
promptvars on
restricted_shell off
shift_verbose off
sourcepath on
xpg_echo off
Your version of bash does accept the function keyword. The problem is that you're not running your script under bash.
I get the same error if I run the script with dash foo.bash or sh foo.bash (/bin/sh is a symlink to dash on my system).
dash doesn't recognize the function syntax.
The #!/bin/bash line causes your script to be interpreted by bash -- but only if you invoke it directly, not if you feed it to some other shell.
Rather than invoking a shell and passing your script name to it as an argument:
sh foo.bash
just invoke your script directly:
foo.bash (or ./foo.bash)

How to escape extended pathname expansion patterns in quoted expressions?

In addition to the basic *, ? and [...] patterns, the Bash shell provides extended pattern matching operators like !(pattern-list) ("match all except one of the given patterns"). The extglob shell option needs to be set to use them. An example:
~$ mkdir test ; cd test ; touch file1 file2 file3
~/test$ echo *
file1 file2 file3
~/test$ shopt -s extglob # make sure extglob is set
~/test$ echo !(file2)
file1 file3
If I pass a shell expression to a program which executes it in a sub-shell, the operator causes an error. Here's a test which runs a sub-shell directly (here I'm executing from another directory to make sure expansion doesn't happen prematurely):
~/test$ cd ..
~$ bash -c "cd test ; echo *"
file1 file2 file3
~$ bash -c "cd test ; echo !(file2)" # expected output: file1 file3
bash: -c: line 0: syntax error near unexpected token `('
bash: -c: line 0: `cd test ; echo !(file2)'
I've tried all kinds of escaping, but nothing I've come up with has worked correctly. I also suspected extglob is not set in a sub-shell, but that's not the case:
~$ bash -c "shopt -s extglob ; cd test ; echo !(file2)"
bash: -c: line 0: syntax error near unexpected token `('
bash: -c: line 0: `cd test ; echo !(file2)'
Any solution appreciated!
bash parses each line before executing it, so "shopt -s extglob" won't have taken effect when bash is validating the globbing pattern syntax. The option can't be enabled on the same line. That's why the "bash -O extglob -c 'xyz'" solution (from Randy Proctor) works and is required.
$ bash -O extglob -c 'echo !(file2)'
file1 file3
Here's another way, if you want to avoid eval and you need to be able to turn extglob on and off within the subshell. Just put your pattern in a variable:
bash -c 'shopt -s extglob; cd test; patt="!(file2)"; echo $patt; shopt -u extglob; echo $patt'
gives this output:
file1 file3
!(file2)
demonstrating that extglob was set and unset. If the first echo had quotes around the $patt, it would just spit out the pattern like the second echo (which probably should have quotes).
Well, I don't have any real experince with extglob, but I can get it to work by wrapping the echo in an eval:
$ bash -c 'shopt -s extglob ; cd test ; eval "echo !(file2)"'
file1 file3

Resources