The following is a simple test case for what I want to illustrate.
In bash,
# define the function f
f () { ls $args; }
# Runs the command `ls`
f
# Runs the fommand `ls -a`
args="-a"
f
# Runs the command `ls -a -l`
args="-a -l"
f
But in zsh
# define the function f
f () { ls $args }
# Runs the command `ls`
f
# Runs the fommand `ls -a`
args="-a"
f
# I expect it to run `ls -a -l`, instead it gives me an error
args="-a -l"
f
The last line in the zsh on above, gives me the following error
ls: invalid option -- ' '
Try `ls --help' for more information.
I think zsh is executing
ls "-a -l"
which is when I get the same error. So, how do I get bash's behavior here?
I'm not sure if I'm clear, let me know if there is something you want to know.
The difference is that (by default) zsh does not do word splitting for unquoted parameter expansions.
You can enable “normal” word splitting by setting the SH_WORD_SPLIT option or by using the = flag on an individual expansion:
ls ${=args}
or
setopt SH_WORD_SPLIT
ls $args
If your target shells support arrays (ksh, bash, zsh), then you may be better off using an array:
args=(-a -l)
ls "${args[#]}"
From the zsh FAQ:
2.1: Differences from sh and ksh
The classic difference is word splitting, discussed in question 3.1; this catches out very many beginning zsh users.
3.1: Why does $var where var="foo bar" not do what I expect? is the FAQ that covers this question.
From the zsh Manual:
14.3 Parameter Expansion
Note in particular the fact that words of unquoted parameters are not automatically split on whitespace unless the option SH_WORD_SPLIT is set; see references to this option below for more details. This is an important difference from other shells.
SH_WORD_SPLIT
Causes field splitting to be performed on unquoted parameter expansions.
Related
The following is a simple test case for what I want to illustrate.
In bash,
# define the function f
f () { ls $args; }
# Runs the command `ls`
f
# Runs the fommand `ls -a`
args="-a"
f
# Runs the command `ls -a -l`
args="-a -l"
f
But in zsh
# define the function f
f () { ls $args }
# Runs the command `ls`
f
# Runs the fommand `ls -a`
args="-a"
f
# I expect it to run `ls -a -l`, instead it gives me an error
args="-a -l"
f
The last line in the zsh on above, gives me the following error
ls: invalid option -- ' '
Try `ls --help' for more information.
I think zsh is executing
ls "-a -l"
which is when I get the same error. So, how do I get bash's behavior here?
I'm not sure if I'm clear, let me know if there is something you want to know.
The difference is that (by default) zsh does not do word splitting for unquoted parameter expansions.
You can enable “normal” word splitting by setting the SH_WORD_SPLIT option or by using the = flag on an individual expansion:
ls ${=args}
or
setopt SH_WORD_SPLIT
ls $args
If your target shells support arrays (ksh, bash, zsh), then you may be better off using an array:
args=(-a -l)
ls "${args[#]}"
From the zsh FAQ:
2.1: Differences from sh and ksh
The classic difference is word splitting, discussed in question 3.1; this catches out very many beginning zsh users.
3.1: Why does $var where var="foo bar" not do what I expect? is the FAQ that covers this question.
From the zsh Manual:
14.3 Parameter Expansion
Note in particular the fact that words of unquoted parameters are not automatically split on whitespace unless the option SH_WORD_SPLIT is set; see references to this option below for more details. This is an important difference from other shells.
SH_WORD_SPLIT
Causes field splitting to be performed on unquoted parameter expansions.
The following is a simple test case for what I want to illustrate.
In bash,
# define the function f
f () { ls $args; }
# Runs the command `ls`
f
# Runs the fommand `ls -a`
args="-a"
f
# Runs the command `ls -a -l`
args="-a -l"
f
But in zsh
# define the function f
f () { ls $args }
# Runs the command `ls`
f
# Runs the fommand `ls -a`
args="-a"
f
# I expect it to run `ls -a -l`, instead it gives me an error
args="-a -l"
f
The last line in the zsh on above, gives me the following error
ls: invalid option -- ' '
Try `ls --help' for more information.
I think zsh is executing
ls "-a -l"
which is when I get the same error. So, how do I get bash's behavior here?
I'm not sure if I'm clear, let me know if there is something you want to know.
The difference is that (by default) zsh does not do word splitting for unquoted parameter expansions.
You can enable “normal” word splitting by setting the SH_WORD_SPLIT option or by using the = flag on an individual expansion:
ls ${=args}
or
setopt SH_WORD_SPLIT
ls $args
If your target shells support arrays (ksh, bash, zsh), then you may be better off using an array:
args=(-a -l)
ls "${args[#]}"
From the zsh FAQ:
2.1: Differences from sh and ksh
The classic difference is word splitting, discussed in question 3.1; this catches out very many beginning zsh users.
3.1: Why does $var where var="foo bar" not do what I expect? is the FAQ that covers this question.
From the zsh Manual:
14.3 Parameter Expansion
Note in particular the fact that words of unquoted parameters are not automatically split on whitespace unless the option SH_WORD_SPLIT is set; see references to this option below for more details. This is an important difference from other shells.
SH_WORD_SPLIT
Causes field splitting to be performed on unquoted parameter expansions.
I want to run this cmd line script
$ script.sh lib/* ../test_git_thing
I want it to process all the files in the /lib folder.
FILES=$1
for f in $FILES
do
echo "Processing $f file..."
done
Currently it only prints the first file. If I use $#, it gives me all the files, but also the last param which I don't want. Any thoughts?
The argument list is being expanded at the command line when you invoke "script.sh lib/*" your script is being called with all the files in lib/ as args. Since you only reference $1 in your script, it's only printing the first file. You need to escape the wildcard on the command line so it's passed to your script to perform the globbing.
As correctly noted, lib/* on the command line is being expanded into all files in lib. To prevent expansion, you have 2 options. (1) quote your input:
$ script.sh 'lib/*' ../test_git_thing
Or (2), turn file globbing off. However, the option set -f will disable pathname expansion within the shell, but it will disable all pathname expansion (setting it within the script doesn't help as expansion is done by the shell before passing arguments to your script). In your case, it is probably better to quote the input or pass the first arguments as a directory name, and add the expansion in the script:
DIR=$1
for f in "$DIR"/*
In bash and ksh you can iterate through all arguments except the last like this:
for f in "${#:1:$#-1}"; do
echo "$f"
done
In zsh, you can do something similar:
for f in $#[1,${#}-1]; do
echo "$f"
done
$# is the number of arguments and ${#:start:length} is substring/subsequence notation in bash and ksh, while $#[start,end] is subsequence in zsh. In all cases, the subscript expressions are evaluated as arithmetic expressions, which is why $#-1 works. (In zsh, you need ${#}-1 because $#- is interpreted as "the length of $-".)
In all three shells, you can use the ${x:start:length} syntax with a scalar variable, to extract a substring; in bash and ksh, you can use ${a[#]:start:length} with an array to extract a subsequence of values.
This answers the question as given, without using non-POSIX features, and without workarounds such as disabling globbing.
You can find the last argument using a loop, and then exclude that when processing the list of files. In this example, $d is the directory name, while $f has the same meaning as in the original answer:
#!/bin/sh
if [ $# != 0 ]
then
for d in "$#"; do :; done
if [ -d "$d" ]
then
for f in "$#"
do
if [ "x$f" != "x$d" ]
then
echo "Processing $f file..."
fi
done
fi
fi
Additionally, it would be a good idea to also test if "$f" is a file, since it is common for shells to pass the wildcard character through the argument list if no match is found.
Suppose you have the following command stored in a variable:
COMMAND='echo hello'
What's the difference between
$ eval "$COMMAND"
hello
$ bash -c "$COMMAND"
hello
$ $COMMAND
hello
? Why is the last version almost never used if it is shorter and (as far as I can see) does exactly the same thing?
The third form is not at all like the other two -- but to understand why, we need to go into the order of operations when bash in interpreting a command, and look at which of those are followed when each method is in use.
Bash Parsing Stages
Quote Processing
Splitting Into Commands
Special Operator Parsing
Expansions
Word Splitting
Globbing
Execution
Using eval "$string"
eval "$string" follows all the above steps starting from #1. Thus:
Literal quotes within the string become syntactic quotes
Special operators such as >() are processed
Expansions such as $foo are honored
Results of those expansions are split on characters into whitespace into separate words
Those words are expanded as globs if they parse as same and have available matches, and finally the command is executed.
Using sh -c "$string"
...performs the same as eval does, but in a new shell launched as a separate process; thus, changes to variable state, current directory, etc. will expire when this new process exits. (Note, too, that that new shell may be a different interpreter supporting a different language; ie. sh -c "foo" will not support the same syntax that bash, ksh, zsh, etc. do).
Using $string
...starts at step 5, "Word Splitting".
What does this mean?
Quotes are not honored.
printf '%s\n' "two words" will thus parse as printf %s\n "two words", as opposed to the usual/expected behavior of printf %s\n two words (with the quotes being consumed by the shell).
Splitting into multiple commands (on ;s, &s, or similar) does not take place.
Thus:
s='echo foo && echo bar'
$s
...will emit the following output:
foo && echo bar
...instead of the following, which would otherwise be expected:
foo
bar
Special operators and expansions are not honored.
No $(foo), no $foo, no <(foo), etc.
Redirections are not honored.
>foo or 2>&1 is just another word created by string-splitting, rather than a shell directive.
$ bash -c "$COMMAND"
This version starts up a new bash interpreter, runs the command, and then exits, returning control to the original shell. You don't need to be running bash at all in the first place to do this, you can start a bash interpreter from tcsh, for example. You might also do this from a bash script to start with a fresh environment or to keep from polluting your current environment.
EDIT:
As #CharlesDuffy points out starting a new bash shell in this way will clear shell variables but environment variables will be inherited by the spawned shell process.
Using eval causes the shell to parse your command twice. In the example you gave, executing $COMMAND directly or doing an eval are equivalent, but have a look at the answer here to get a more thorough idea of what eval is good (or bad) for.
There are at least times when they are different. Consider the following:
$ cmd="echo \$var"
$ var=hello
$ $cmd
$var
$ eval $cmd
hello
$ bash -c "$cmd"
$ var=world bash -c "$cmd"
world
which shows the different points at which variable expansion is performed. It's even more clear if we do set -x first
$ set -x
$ $cmd
+ echo '$var'
$var
$ eval $cmd
+ eval echo '$var'
++ echo hello
hello
$ bash -c "$cmd"
+ bash -c 'echo $var'
$ var=world bash -c "$cmd"
+ var=world
+ bash -c 'echo $var'
world
We can see here much of what Charles Duffy talks about in his excellent answer. For example, attempting to execute the variable directly prints $var because parameter expansion and those earlier steps had already been done, and so we don't get the value of var, as we do with eval.
The bash -c option only inherits exported variables from the parent shell, and since I didn't export var it's not available to the new shell.
The following is a simple test case for what I want to illustrate.
In bash,
# define the function f
f () { ls $args; }
# Runs the command `ls`
f
# Runs the fommand `ls -a`
args="-a"
f
# Runs the command `ls -a -l`
args="-a -l"
f
But in zsh
# define the function f
f () { ls $args }
# Runs the command `ls`
f
# Runs the fommand `ls -a`
args="-a"
f
# I expect it to run `ls -a -l`, instead it gives me an error
args="-a -l"
f
The last line in the zsh on above, gives me the following error
ls: invalid option -- ' '
Try `ls --help' for more information.
I think zsh is executing
ls "-a -l"
which is when I get the same error. So, how do I get bash's behavior here?
I'm not sure if I'm clear, let me know if there is something you want to know.
The difference is that (by default) zsh does not do word splitting for unquoted parameter expansions.
You can enable “normal” word splitting by setting the SH_WORD_SPLIT option or by using the = flag on an individual expansion:
ls ${=args}
or
setopt SH_WORD_SPLIT
ls $args
If your target shells support arrays (ksh, bash, zsh), then you may be better off using an array:
args=(-a -l)
ls "${args[#]}"
From the zsh FAQ:
2.1: Differences from sh and ksh
The classic difference is word splitting, discussed in question 3.1; this catches out very many beginning zsh users.
3.1: Why does $var where var="foo bar" not do what I expect? is the FAQ that covers this question.
From the zsh Manual:
14.3 Parameter Expansion
Note in particular the fact that words of unquoted parameters are not automatically split on whitespace unless the option SH_WORD_SPLIT is set; see references to this option below for more details. This is an important difference from other shells.
SH_WORD_SPLIT
Causes field splitting to be performed on unquoted parameter expansions.