Why ./script.sh ? throws No match. ./script.sh is running fine.
script.sh
#!/bin/sh
echo "Hello World"
? is a glob character on UNIX. By default, in POSIX shells, a glob that matches no files at all will evaluate to itself; however, many shells have the option to modify this behavior and either pass no arguments in this case or make it an error.
If you want to pass this (or any other string which can be interpreted as a glob) literally, quote it:
./script.sh '?'
If you didn't use quotes, consider what the following would do:
touch a b c
./script.sh ? ## this is the same as running: ./script.sh a b c
That said -- the behavior of your outer shell (exiting when no matches exist, rather than defaulting to pass the non-matching glob expression as a literal) is non-default. If this shell is bash, you can modify it with:
shopt -u failglob
Note, however, that this doesn't really fix your problem, but only masks it when your current directory has no single-character filenames. The only proper fix is to correct your usage to quote and escape values properly.
Related
I'm trying to exclude multiple directories when using grep as in the following command
grep -r --exclude-dir={folder1, folder2} 'foo'
However, an error is raised grep: foo: No such file or directory. Maybe I'm doing something wrong with --exclude-dir option since the command below works as expected
grep -r 'foo'
How can I use --exclude-dir option correctly? Thanks in advance.
The --exclude-dir flag of GNU grep takes a glob expression as an argument. The glob expression with more than items then becomes a brace expansion sequence which is expanded by the shell.
The expansion involves words separated by a comma character and doesn't like spaces between the words. So ideally it should have been
--exclude-dir={folder1,folder2}
You can see this as a simple brace expansion in your shell by running
echo {a,b} # produces 'a b'
echo {a, b} # this doesn't undergo expansion by shell
echo --exclude-dir={folder1, folder2}
--exclude-dir={folder1, folder2}
so, your original command becomes
grep -r '--exclude-dir={folder1,' 'folder2}' foo
i.e. the exclude-dir takes a unexpanded glob expansion string as {folder1,' and 'folder2}' becomes the content that you are trying to search for, leaving foo as an unwanted extra argument, which the argparser of grep doesn't like throwing a command line parse error.
Remember brace expansion is a feature of the shell (e.g. bash), and not grep. In shells that don't support the feature, putting directories between {..} will be treated literally and might not work desirably.
In my understanding of command substitution this should work, but it doesn't, can you explain me why and how to do something like this.
Why does this work:
cd ..; echo 123 # output is "123", after changing directories
...when this doesn't:
cd $(echo "..; echo 123") # error message is "cd: too many arguments"
Command substitution results (like expansion of variables) do not go through all parsing phases; they only go through word-splitting[1] and glob expansion[2], and even those happen only when the expansion itself is unquoted.
That means that your result is identical to:
cd "..;" "echo" "123"
...the semicolon is treated as literal text to be passed to the cd command, not shell syntax.
This is a feature, not a bug: If command substitution results were parsed as syntax, writing secure shell scripts handling untrusted data would be basically impossible.
[1] dividing the results into "words" on characters in IFS -- or, by default, whitespace.
[2] looking at whether each resulting word can be treated as a filename-matching pattern, and, if so, matching them against the local filesystem.
If I want to match a file called "-f" or "-r" I might do something like
test.sh -?
And if I want to send the literal '-?' to a program as an argument I might do something like:
test.sh -\?
If no such file "-f" or "-r" or anything like it exists, then what should my shell do with
test.sh -?
Should it tell me that no file matches this pattern?
In bash, the default is to treat an unmatched pattern literally. If the nullglob option is set, an unmatched pattern "evaporates"; it is removed from command, not even expanding to the empty string.
In zsh, an unmatched pattern produces an error by default. Setting the nomatch option causes an unmatched pattern to be treated literally, and zsh also supports a nullglob option which causes unmatched patterns to disappear. There is also a cshnullglob option which acts like nullglob, but requires at least one pattern in a command to match, or an error is produced.
Note that POSIX specifies that if the pattern contains an invalid bracket expression or does not match any existing filenames or pathnames, the pattern string shall be left unchanged in sh.
ash, dash, ksh, bash and zsh all behave this way when invoked as sh.
POSIX specifies that if the pattern contains an invalid bracket expression or does not match any existing filenames or pathnames, the pattern string shall be left unchanged in sh.
ash, dash, ksh, bash and zsh all behave this way when invoked as sh.
You seem to be looking for the nullglob option, at least with Bash:
shopt -s nullglob
Without the nullglob option, an unmatched pattern is passed as its literal self to your program: the shell will pass -? to the script if there isn't a file that matches. With the nullglob option, unmatched patterns are replaced with nothing at all.
If no such pattern exists, the shell, by default, just returns the pattern your gave, including whatever * or ? characters you used. To determine whether the file actually exists, test it. Thus, inside your script, use:
[ -f "$1" ] || echo "no such file exists"
I found a weird behavior that I don't know how to workaround.
$ var1=*
$ echo $var1
Audiobooks Downloads Desktop (etc.)
$ ls $var1
Audiobooks:
Downloads:
(etc)
All seems OK. At declaration, the variable gets expanded and everything else works. But see this:
$ var2=~/rpmbuild/{SRPMS,RPMS/*}/enki-*.rpm
$ echo $var2
/home/yajo/rpmbuild/{SRPMS,RPMS/*}/enki-*.rpm
$ ls $var2
ls: no se puede acceder a /home/yajo/rpmbuild/{SRPMS,RPMS/*}/enki-*.rpm: No existe el fichero o el directorio
$ ls /home/yajo/rpmbuild/{SRPMS,RPMS/*}/enki-*.rpm
/home/yajo/rpmbuild/RPMS/noarch/enki-12.10.3-1.fc18.noarch.rpm /home/yajo/rpmbuild/SRPMS/enki-12.10.3-1.fc18.src.rpm
/home/yajo/rpmbuild/RPMS/noarch/enki-12.10.3-1.fc19.noarch.rpm /home/yajo/rpmbuild/SRPMS/enki-12.10.3-1.fc19.src.rpm
This time, at declaration only ~ gets expanded, which produces that I cannot pass it as an argument to ls. However, passing the same string literally produces the expected results.
Questions are:
Why sometimes expand and sometimes not?
How to mimic the behavior of $var1 with $var2?
Thanks.
Extra notes:
I tried the same with double and single quotes, but with the same bad results.
The order in which the shell parses various aspects of the command line is not obvious, and it matters for things like this.
First, the wildcards aren't expanded at declaration, they're expanded after the variable value is substituted (note: in these examples I'll pretend I have your filesystem):
$ var1=*
$ echo "$var1" # double-quotes prevent additional parsing of the variable's value
*
$ echo $var1 # without double-quotes, variable value undergoes wildcard expansion and word splitting
Audiobooks:
Downloads:
(etc)
BTW, the ~ is expanded at declaration, confusing things even further:
$ var2=~
$ echo "$var2" # again, double-quotes let me see what's actually in the variable
/home/yajo
The problem with ~/rpmbuild/{SRPMS,RPMS/*}/enki-*.rpm1 is that while the shell does wildcard expansion (*) on the value after substitution, it doesn't do brace expansion ({SRPMS,RPMS/*}), so it's actually looking for directory names with braces and commas in the name... and not finding any.
The best way to handle this is generally to store the file list as an array; if you do this right, everything gets expanded at declaration:
$ var2=(~/rpmbuild/{SRPMS,RPMS/*}/enki-*.rpm)
$ echo "${var2[#]}" # This is the proper way to expand an array into a word list
/home/yajo/rpmbuild/RPMS/noarch/enki-12.10.3-1.fc18.noarch.rpm etc...
Note that arrays are a bash extension, and will not work in plain POSIX shells. So be sure to start your script with #!/bin/bash, not #!/bin/sh.
Everybody says eval is evil, and you should use $() as a replacement. But I've run into a situation where the unquoting isn't handled the same inside $().
Background is that I've been burned too often by file paths with spaces in them, and so like to quote all such paths. More paranoia about wanting to know where all my executables are coming from. Even more paranoid, not trusting myself, and so like being able to display the created commands I'm about to run.
Below I try variations on using eval vs. $(), and whether the command name is quoted (cuz it could contain spaces)
BIN_LS="/bin/ls"
thefile="arf"
thecmd="\"${BIN_LS}\" -ld -- \"${thefile}\""
echo -e "\n Running command '${thecmd}'"
$($thecmd)
Running command '"/bin/ls" -ld -- "arf"'
./foo.sh: line 8: "/bin/ls": No such file or directory
echo -e "\n Eval'ing command '${thecmd}'"
eval $thecmd
Eval'ing command '"/bin/ls" -ld -- "arf"'
/bin/ls: cannot access arf: No such file or directory
thecmd="${BIN_LS} -ld -- \"${thefile}\""
echo -e "\n Running command '${thecmd}'"
$($thecmd)
Running command '/bin/ls -ld -- "arf"'
/bin/ls: cannot access "arf": No such file or directory
echo -e "\n Eval'ing command '${thecmd}'"
eval $thecmd
Eval'ing command '/bin/ls -ld -- "arf"'
/bin/ls: cannot access arf: No such file or directory
$("/bin/ls" -ld -- "${thefile}")
/bin/ls: cannot access arf: No such file or directory
So... this is confusing. A quoted command path is valid everywhere except inside a $() construct? A shorter, more direct example:
$ c="\"/bin/ls\" arf"
$ $($c)
-bash: "/bin/ls": No such file or directory
$ eval $c
/bin/ls: cannot access arf: No such file or directory
$ $("/bin/ls" arf)
/bin/ls: cannot access arf: No such file or directory
$ "/bin/ls" arf
/bin/ls: cannot access arf: No such file or directory
How does one explain the simple $($c) case?
The use of " to quote words is part of your interaction with Bash. When you type
$ "/bin/ls" arf
at the prompt, or in a script, you're telling Bash that the command consists of the words /bin/ls and arf, and the double-quotes are really emphasizing that /bin/ls is a single word.
When you type
$ eval '"/bin/ls" arf'
you're telling Bash that the command consists of the words eval and "/bin/ls" arf. Since the purpose of eval is to pretend that its argument is an actual human-input command, this is equivalent to running
$ "/bin/ls" arf
and the " gets processed just like at the prompt.
Note that this pretense is specific to eval; Bash doesn't usually go out of its way to pretend that something was an actual human-typed command.
When you type
$ c='"/bin/ls" arf'
$ $c
the $c gets substituted, and then undergoes word splitting (see ยง3.5.7 "Word Splitting" in the Bash Reference Manual), so the words of the command are "/bin/ls" (note the double-quotes!) and arf. Needless to say, this doesn't work. (It's also not very safe, since in addition to word-splitting, $c also undergoes filename-expansion and whatnot. Generally your parameter-expansions should always be in double-quotes, and if they can't be, then you should rewrite your code so they can be. Unquoted parameter-expansions are asking for trouble.)
When you type
$ c='"/bin/ls" arf'
$ $($c)
this is the same as before, except that now you're also trying to use the output of the nonworking command as a new command. Needless to say, that doesn't cause the nonworking command to suddenly work.
As Ignacio Vazquez-Abrams says in his answer, the right solution is to use an array, and handle the quoting properly:
$ c=("/bin/ls" arf)
$ "${c[#]}"
which sets c to an array with two elements, /bin/ls and arf, and uses those two elements as the word of a command.
With the fact that it doesn't make sense in the first place. Use an array instead.
$ c=("/bin/ls" arf)
$ "${c[#]}"
/bin/ls: cannot access arf: No such file or directory
From the man page for bash, regarding eval:
eval [arg ...]:
The args are read and concatenated together into a single command.
This command is then read and executed by the shell, and its exit
status is returned as the value of eval.
When c is defined as "\"/bin/ls\" arf", the outer quotes will cause the entire thing to be processed as the first argument to eval, which is expected to be a command or program. You need to pass your eval arguments in such a way that the target command and its arguments are listed separately.
The $(...) construct behaves differently than eval because it is not a command that takes arguments. It can process the entire command at once instead of processing arguments one at a time.
A note on your original premise: The main reason that people say that eval is evil was because it is commonly used by scripts to execute a user-provided string as a shell command. While handy at times, this is a major security problem (there's typically no practical way to safety-check the string before executing it). The security problem doesn't apply if you are using eval on hard-coded strings inside your script, as you are doing. However, it's typically easier and cleaner to use $(...) or `...` inside of scripts for command substitution, leaving no real use case left for eval.
Using set -vx helps us understand how bash process the command string.
As seen in the picture, "command" works cause quotes will be stripped when processing. However, when $c(quoted twice) is used, only the outside single quotes are removed. eval can process the string as the argument and outside quotes are removed step by step.
It is probably just related to how bash semanticallly process the string and quotes.
Bash does have many weird behaviours about quotes processing:
Bash inserting quotes into string before execution
How do you stop bash from stripping quotes when running a variable as a command?
Bash stripping quotes - how to preserve quotes