In addition to the basic *, ? and [...] patterns, the Bash shell provides extended pattern matching operators like !(pattern-list) ("match all except one of the given patterns"). The extglob shell option needs to be set to use them. An example:
~$ mkdir test ; cd test ; touch file1 file2 file3
~/test$ echo *
file1 file2 file3
~/test$ shopt -s extglob # make sure extglob is set
~/test$ echo !(file2)
file1 file3
If I pass a shell expression to a program which executes it in a sub-shell, the operator causes an error. Here's a test which runs a sub-shell directly (here I'm executing from another directory to make sure expansion doesn't happen prematurely):
~/test$ cd ..
~$ bash -c "cd test ; echo *"
file1 file2 file3
~$ bash -c "cd test ; echo !(file2)" # expected output: file1 file3
bash: -c: line 0: syntax error near unexpected token `('
bash: -c: line 0: `cd test ; echo !(file2)'
I've tried all kinds of escaping, but nothing I've come up with has worked correctly. I also suspected extglob is not set in a sub-shell, but that's not the case:
~$ bash -c "shopt -s extglob ; cd test ; echo !(file2)"
bash: -c: line 0: syntax error near unexpected token `('
bash: -c: line 0: `cd test ; echo !(file2)'
Any solution appreciated!
bash parses each line before executing it, so "shopt -s extglob" won't have taken effect when bash is validating the globbing pattern syntax. The option can't be enabled on the same line. That's why the "bash -O extglob -c 'xyz'" solution (from Randy Proctor) works and is required.
$ bash -O extglob -c 'echo !(file2)'
file1 file3
Here's another way, if you want to avoid eval and you need to be able to turn extglob on and off within the subshell. Just put your pattern in a variable:
bash -c 'shopt -s extglob; cd test; patt="!(file2)"; echo $patt; shopt -u extglob; echo $patt'
gives this output:
file1 file3
!(file2)
demonstrating that extglob was set and unset. If the first echo had quotes around the $patt, it would just spit out the pattern like the second echo (which probably should have quotes).
Well, I don't have any real experince with extglob, but I can get it to work by wrapping the echo in an eval:
$ bash -c 'shopt -s extglob ; cd test ; eval "echo !(file2)"'
file1 file3
Related
In bash, I can use a negative wildcard to glob all files in a directory that don't match some pattern, for example:
echo src/main/webapp/!(WEB-INF)
This works fine.
However, if I try to use exactly the same wildcard with bash -c to pass the command as an argument to a new bash shell, I get a syntax error:
$ bash -c 'echo src/main/webapp/!(WEB-INF)'
bash: -c: line 0: syntax error near unexpected token `('
bash: -c: line 0: `echo src/main/webapp/!(WEB-INF)'
Note that if I use a different glob, like bash -c 'echo src/main/webapp/*' it works as expected.
Why doesn't bash accept the same negative glob with -c as it does when run normally, and how can I get it to accept this negative glob?
That's because !(..) is a extended glob pattern that is turned on by default in your interactive bash shell, but in an explicit sub-shell launched with -c, the option is turned off. You can see that
$ shopt | grep extglob
extglob on
$ bash -c 'shopt | grep extglob'
extglob off
One way to turn on the option explicitly in command line would be to use the -O flag followed by the option to be turned on
$ bash -O extglob -c 'shopt | grep extglob'
extglob on
See extglob on Greg's Wiki for the list of extended glob patterns supported and The Shopt Builtin for a list of the extended shell options and which ones are enabled by default.
It happens the feature at stake is only enabled by default in an interactive shell. In bash, this is controlled by the extglob option:
extglob
If set, the extended pattern matching features described above (see Pattern Matching) are enabled.
To confirm this, you can run for example:
$ bash -c 'shopt -p | grep extglob'
shopt -u extglob
$ bash -i -c 'shopt -p | grep extglob'
shopt -s extglob
Problem
The execution of a Bash script fails with the following error message when the 'extglob' option is set inside a subshell:
/tmp/foo.sh: line 7: syntax error near unexpected token `('
#!/usr/bin/env bash
set -euo pipefail
(
shopt -s extglob
for f in ?(.)!(|+(.)|vendor); do
echo "$f"
done
)
It fails in the same manner inside a function:
#!/usr/bin/env bash
set -euo pipefail
list_no_vendor () {
shopt -s extglob
for f in ?(.)!(|+(.)|vendor); do
echo "$f"
done
}
list_no_vendor
Investigation
In both cases, the script executes successfully when the option is set globally, outside of the subshell or function.
Surprisingly, when set locally, the 'extglob' option appears to be effectively enabled in both the subshell and function:
#!/usr/bin/env bash
set -euo pipefail
(
shopt -s extglob
echo 'In the subshell:' "$(shopt extglob)"
)
list_no_vendor () {
shopt -s extglob
echo 'In the function:' "$(shopt extglob)"
}
echo 'In the main shell:' "$(shopt extglob)"
list_no_vendor
Output:
In the subshell: extglob on
In the main shell: extglob off
In the function: extglob on
This makes the syntax error extremely puzzling to me.
Workaround
Passing a heredoc to the bash command works.
#!/usr/bin/env bash
set -euo pipefail
bash <<'EOF'
shopt -s extglob
echo 'In the child:' "$(shopt extglob)"
EOF
echo 'In the parent:' "$(shopt extglob)"
Output:
In the child: extglob on
In the parent: extglob off
However I would be curious to understand the gist of the problem here.
extglob is a flag used by the parser. Functions, compound commands, &c. are parsed in entirety ahead of execution. Thus, extglob must be set before that content is parsed; setting it at execution time but after parse time does not have any effect for previously-parsed content.
This is also why you can't run shopt -s extglob; ls !(*.txt) as a one-liner (when extglob is previously unset), but must have a newline between the two commands.
Not as an example of acceptable practice, but as an example demonstrating the behavior, consider the following:
#!/usr/bin/env bash
(
shopt -s extglob
# Parse of eval'd code is deferred, so this succeeds
eval '
for f in ?(.)!(|+(.)|vendor); do
echo "$f"
done
'
)
No such error takes place here, because parsing of the content passed to eval happens only after the shopt -s extglob was executed, rather than when the block of code to be run in a subshell is parsed.
I am trying to run a command with a variable which holds another command that suppresses warning messages of the jar. However, it is not working as expected and I can't figure out what I am doing wrong.
TEST=${TEST:-2> /dev/null}
java -jar ~/bin/aw.jar ${Test}
Redirection is performed only if it is unquoted and present in the command line literally rather than originating from any kind of expansion:
$ ls
$ echo hello >out
$ ls
out
$ cat out
hello
$ rm *
$ echo hello '>out'
hello >out
$ ls
$ x='>out'
$ echo hello $x
hello >out
$ ls
In order to interpret redirection operator coming from a quoted string or an expansion you must execute the command through eval (note, however, that this may result in undesired expansions in other parts of your command):
$ ls
$ x='>out'
$ eval echo hello $x
$ ls
out
$ cat out
hello
In user's console I have bash:
$ echo $SHELL
/bin/bash
$ bash --version
GNU bash, version 4.2.46(1)-release (x86_64-redhat-linux-gnu)
I have code in file test.sh:
$ cat test.sh
aaa='---aa-aa---'
echo "${aaa}"
echo 'does not work...'
# trim "-"
echo ${aaa/+(-)}
echo ${aaa%%+(-)}
echo 'works for one symbol...'
echo ${aaa%-}
echo ${aaa/-}
The last two rows work fine but previous ones.
$ bash test.sh
---aa-aa---
does not work...
---aa-aa---
---aa-aa---
works for one symbol...
---aa-aa--
--aa-aa---
In the same time, if you would try to make this console it works:
$ aaa='---aa-aa---'
$ echo ${aaa/+(-)}
aa-aa---
$ echo ${aaa%%+(-)}
---aa-aa
So, why it doesn't work in a script?
You seem to have shopt -s extglob enabled in your interactive shell, which turns on extended globbing. This is not the default behavior, and needs to be explicitly enabled in your script. See extended pattern matching in the bash hackers wiki for details.
I'm running gnu-parallel on a command that works fine when run from a bash shell but returns an error when parallel executes it with bash using the -c flag. I assume this has to do with the special globbing expression I'm using.
ls !(*site*).mol2
This returns successfully.
With the flag enabled the command fails
/bin/bash -c 'ls !(*site*).mol2'
/bin/bash: -c: line 0: syntax error near unexpected token `('
The manual only specifies that -c calls for bash to read the arguments for a string, am I missing something?
Edit:
I should add I need this to run from a gnu-parallel string, so the end resultant command must be runnable by /bin/bash -c "Some Command"
You should try the following code :
bash <<EOF
shopt -s extglob
ls !(*site*).mol2
EOF
Explanation :
when you run bash -c, you create a subshell, and shopt settings are not inherited.
EDIT
If you really need a one liner :
bash -O extglob -c 'ls !(*site*).mol2'
See this thread