Somehow my bash rc scripts (which are hundreds of lines long and non-trivial to review) are breaking !! and I'm not sure how to quickly debug it. I know this because when I run bash with no user config it works fine. How can I debug this quickly, or does anything come to mind? I'm not running this in a script -- I'm typing from the command line.
$ !!
!!: command not found
$ echo $-
himBs
$ bash --version
GNU bash, version 4.4.20(1)-release (x86_64-pc-linux-gnu)
$ shopt
autocd off
cdable_vars off
cdspell off
checkhash off
checkjobs off
checkwinsize off
cmdhist on
compat31 off
compat32 off
compat40 off
compat41 off
compat42 off
compat43 off
complete_fullquote on
direxpand off
dirspell off
dotglob off
execfail off
expand_aliases on
extdebug off
extglob on
extquote on
failglob off
force_fignore on
globasciiranges off
globstar off
gnu_errfmt off
histappend off
histreedit off
histverify off
hostcomplete off
huponexit off
inherit_errexit off
interactive_comments on
lastpipe off
lithist off
login_shell on
mailwarn off
no_empty_cmd_completion off
nocaseglob off
nocasematch off
nullglob off
progcomp on
promptvars on
restricted_shell off
shift_verbose off
sourcepath on
xpg_echo off
Related
In bash, I can use a negative wildcard to glob all files in a directory that don't match some pattern, for example:
echo src/main/webapp/!(WEB-INF)
This works fine.
However, if I try to use exactly the same wildcard with bash -c to pass the command as an argument to a new bash shell, I get a syntax error:
$ bash -c 'echo src/main/webapp/!(WEB-INF)'
bash: -c: line 0: syntax error near unexpected token `('
bash: -c: line 0: `echo src/main/webapp/!(WEB-INF)'
Note that if I use a different glob, like bash -c 'echo src/main/webapp/*' it works as expected.
Why doesn't bash accept the same negative glob with -c as it does when run normally, and how can I get it to accept this negative glob?
That's because !(..) is a extended glob pattern that is turned on by default in your interactive bash shell, but in an explicit sub-shell launched with -c, the option is turned off. You can see that
$ shopt | grep extglob
extglob on
$ bash -c 'shopt | grep extglob'
extglob off
One way to turn on the option explicitly in command line would be to use the -O flag followed by the option to be turned on
$ bash -O extglob -c 'shopt | grep extglob'
extglob on
See extglob on Greg's Wiki for the list of extended glob patterns supported and The Shopt Builtin for a list of the extended shell options and which ones are enabled by default.
It happens the feature at stake is only enabled by default in an interactive shell. In bash, this is controlled by the extglob option:
extglob
If set, the extended pattern matching features described above (see Pattern Matching) are enabled.
To confirm this, you can run for example:
$ bash -c 'shopt -p | grep extglob'
shopt -u extglob
$ bash -i -c 'shopt -p | grep extglob'
shopt -s extglob
Problem
The execution of a Bash script fails with the following error message when the 'extglob' option is set inside a subshell:
/tmp/foo.sh: line 7: syntax error near unexpected token `('
#!/usr/bin/env bash
set -euo pipefail
(
shopt -s extglob
for f in ?(.)!(|+(.)|vendor); do
echo "$f"
done
)
It fails in the same manner inside a function:
#!/usr/bin/env bash
set -euo pipefail
list_no_vendor () {
shopt -s extglob
for f in ?(.)!(|+(.)|vendor); do
echo "$f"
done
}
list_no_vendor
Investigation
In both cases, the script executes successfully when the option is set globally, outside of the subshell or function.
Surprisingly, when set locally, the 'extglob' option appears to be effectively enabled in both the subshell and function:
#!/usr/bin/env bash
set -euo pipefail
(
shopt -s extglob
echo 'In the subshell:' "$(shopt extglob)"
)
list_no_vendor () {
shopt -s extglob
echo 'In the function:' "$(shopt extglob)"
}
echo 'In the main shell:' "$(shopt extglob)"
list_no_vendor
Output:
In the subshell: extglob on
In the main shell: extglob off
In the function: extglob on
This makes the syntax error extremely puzzling to me.
Workaround
Passing a heredoc to the bash command works.
#!/usr/bin/env bash
set -euo pipefail
bash <<'EOF'
shopt -s extglob
echo 'In the child:' "$(shopt extglob)"
EOF
echo 'In the parent:' "$(shopt extglob)"
Output:
In the child: extglob on
In the parent: extglob off
However I would be curious to understand the gist of the problem here.
extglob is a flag used by the parser. Functions, compound commands, &c. are parsed in entirety ahead of execution. Thus, extglob must be set before that content is parsed; setting it at execution time but after parse time does not have any effect for previously-parsed content.
This is also why you can't run shopt -s extglob; ls !(*.txt) as a one-liner (when extglob is previously unset), but must have a newline between the two commands.
Not as an example of acceptable practice, but as an example demonstrating the behavior, consider the following:
#!/usr/bin/env bash
(
shopt -s extglob
# Parse of eval'd code is deferred, so this succeeds
eval '
for f in ?(.)!(|+(.)|vendor); do
echo "$f"
done
'
)
No such error takes place here, because parsing of the content passed to eval happens only after the shopt -s extglob was executed, rather than when the block of code to be run in a subshell is parsed.
Going through the Bash reference guide, trying to understand nullglob and bash in general. But when I try:
$shopt -q nullglob
I get no return. If nullglob is off, shouldn't I get a 0 return? I don't understand -q option
From the description of the -q option in the Bash
Suppresses normal output; the return status indicates whether the optname is set or unset.
The variable $? contains the status of the last command.
shopt -q nullglob
echo $?
That will print 0 if it's set, 1 if it's not set.
But it's more useful in an if:
if shopt -q nullglob
then echo nullglob is set
else echo nullglob is not set
fi
My version of bash is:
bash --version
GNU bash, version 4.2.45(1)-release (x86_64-pc-linux-gnu)
If I do the prototype function
#!/bin/bash
function f()
{
echo "hello" $1
}
f "world"
I get Syntax error: "(" unexpected
Why is that?
Output of shopt is:
autocd off
cdable_vars off
cdspell off
checkhash off
checkjobs off
checkwinsize on
cmdhist on
compat31 off
compat32 off
compat40 off
compat41 off
direxpand off
dirspell off
dotglob off
execfail off
expand_aliases on
extdebug off
extglob on
extquote on
failglob off
force_fignore on
globstar off
gnu_errfmt off
histappend on
histreedit off
histverify off
hostcomplete off
huponexit off
interactive_comments on
lastpipe off
lithist off
login_shell off
mailwarn off
no_empty_cmd_completion off
nocaseglob off
nocasematch off
nullglob off
progcomp on
promptvars on
restricted_shell off
shift_verbose off
sourcepath on
xpg_echo off
Your version of bash does accept the function keyword. The problem is that you're not running your script under bash.
I get the same error if I run the script with dash foo.bash or sh foo.bash (/bin/sh is a symlink to dash on my system).
dash doesn't recognize the function syntax.
The #!/bin/bash line causes your script to be interpreted by bash -- but only if you invoke it directly, not if you feed it to some other shell.
Rather than invoking a shell and passing your script name to it as an argument:
sh foo.bash
just invoke your script directly:
foo.bash (or ./foo.bash)
I don't understand what the shopt xpg_echo changes if it is activated or deactivated.
In the manual:
xpg_echo
If set, the echo builtin expands backslash-escape
sequences by default.
I tried to activate/deactivate xpg_echo, but echo has the same behaviour.
It determines if echo will process escape-sequences like \n:
$ shopt -u xpg_echo # Disable xpg_echo
$ echo "Hello\nworld"
Hello\nworld
$ shopt -s xpg_echo # Enable xpg_echo
$ echo "Hello\nworld"
Hello
world