This is driving me absolutely crazy:
$ a="/"
echo $a # note empty output line below
$ var="/home/vivek/foo/bar"
$ echo $var
home vivek foo bar
What's going on in my bash shell on OS X?
I've tried this on my other Mac.. and it works perfectly!
tl;dr:
Reset the special $IFS variable to its default - IFS=$' \t\n' - or, preferably, double-quote your variable reference (echo "$var") to print the value as-is.
You're referencing $var unquoted, which means that its value is subject to word splitting (one of the many expansions that Bash applies to unquoted tokens).
Word splitting happens by any of the characters defined in the built-in $IFS variable (the Internal Field Separator), which defaults to $' \t\n' (space, tab, newline).
In your case, $IFS contains / (possibly among other chars.), which means that /home/vivek/foo/bar is split into separate arguments home, vivek, foo, bar, which are then passed to echo.
echo, when given multiple arguments, prints them separated with a space, which is what you're seeing.
(Similarly, / as the value of $var is interpreted as just a separator, with no fields, which means that no arguments are passed to echo, which just prints a newline).
There are 2 lessons here:
Only temporarily change $IFS; restore the previous value once you're done with the custom value.
Generally, double-quote all variable references to ensure that their values are preserved as-is; only use unquoted variable references if you explicitly want shell expansions applied to their values.
Related
I've noticed that to print the value of the $IFS variable in the shell I have to do something like:
$ printf '%s\nYour IFS is: %q' 'Hello' "$IFS"
Hello
Your IFS is: $' \t\n'
My question is why do I need to pass the IFS in that special way? For example, why (or why wouldn't) it be possible to do:
$ echo $IFS -- some parameter that prints special characters?
$ printf "$IFS" or $ printf '$IFS' -- why wouldn't either of these work?
Why $ printf "%q" $IFS and $ printf "%q" '$IFS' don't show this properly but $ printf "%q" "$IFS" does?
$ echo $IFS -- some parameter that prints special characters?
echo doesn't have such a parameter
$ printf "$IFS" or $ printf '$IFS' -- why wouldn't either of these work?
The first does interpolation of the string just like echo does and prints the IFS string just like it is - which by default is a bunch of whitespaces.
The second does not do interpolation and obviously prints $IFS.
printf "%q" $IFS
The variable has already been expanded into whitespaces that are eaten up by the shell so nothing is passed as a second parameter to printf and therefore %q has no input to work with.
printf "%q" '$IFS'
The string $IFS is passed as a parameter to %q which just adds escape characters to it.
You are running into three basic problems:
Unquoted variable references (as in echo $IFS and printf "%q" $IFS) undergo word splitting. Basically, any whitespace in the variable's value is treated as a separator between "words" (which get passed as arguments to the command). But "whitespace" is defined as the characters in $IFS (that's what it's for!). Therefore, the entire variable's value gets treated as just spacing, not actual content, and effectively vanishes!
This is one of the many examples of why you should put double-quotes around variable references.
Single-quoted strings (as in printf '$IFS' and printf "%q" '$IFS') don't undergo variable expansion at all. In these cases, $IFS is just a literal string, not a variable reference.
Finally, the default characters in $IFS aren't particularly visible on screen. printf "$IFS" actually does print them correctly (unless you put "%" or "\" in IFS for some reason), but you can't really see them. By default, the first character in $IFS is a space, so when that prints the cursor moves to the second column, but nothing visible appears. The second character is a tab, so the cursor moves even further over, but again nothing is actually visible. Then the last character is a newline, so the cursor moves to the beginning of the next line... and again, nothing visible appears.
This is why printf %q is needed -- the %q format converts the not-really-visible characters in $IFS into a visible, readable representation.
If you don't quote the variable, word-splitting is done after the variable is expanded. Word splitting treats all the characters in IFS as word delimiters; multiple of them in a row between words are collapsed into a single space, and if the output is entirely delimiters, nothing at all is output.
You need to use quotes around the variable to output it literally.
Then you need to use the %q format operator to output it as escape sequences so you can see what the individual characters are. Otherwise you wouldn't be able to tell that the second character is a TAB, you would just see a bunch of spaces on the screen. And the newline would just go to the next line.
I seem to be able to create environment variables that execute commands; like this:
$ cat ./src
FOO="echo"
$ . ./src
$ echo $FOO
echo
$ $FOO hello
hello
$
Is there a way I can modify that environment variable so that it prefixes the setting of another environment variable before the command? I.e. is there a way to work around the following problem?
$ cat ./src
FOO="MY_DIR=/tmp echo"
$ . ./src
$ echo $FOO
MY_DIR=/tmp echo
$ $FOO hello
-bash: MY_DIR=/tmp: No such file or directory
$
I.e. what I'd like to happen is to have an environment variable that does the equivalent of the following manually typed in the shell:
$ MY_DIR=/tmp echo hello
hello
$
...similar to how sans envvar-prefix, $FOO effectively had the same effect as typing echo at the shell.
/tmp/ exists of course, btw:
$ ls -ld /tmp/
drwxrwxrwt. 25 root root 500 May 19 11:35 /tmp/
$
Update:
I have a constraint that "FOO" must be invoked like $FOO hello and not FOO hello. So unfortunately a function like in #John Kugelman's (current) answer can't be a solution, even if it's more proper.
It's best to put data into variables, code into functions. Functions are more natural, expressive, and flexible than variables holding code. They look just like any other command but can take arbitrary actions, including but not limited to prepending commands and variable assignments.
foo() {
MY_DIR=/tmp echo "$#"
}
foo hello
Here "$#" is a placeholder for the arguments passed to foo().
I have a constraint that "FOO" must be invoked like $FOO hello and not FOO hello.
That constraint is impossible, I'm afraid.
I am curious about the mechanics of what's going on here: i.e. why can you make an environment variable that's sort of "aliased" to a command (I know true aliasing is something else), but that mechanism doesn't accommodate the seemingly small change to prefix "stuff" to the command?
Bash expands commands in several passes in a fixed, prescribed order. Very early on it splits the command into words and then marks the variable assignments with invisible flags. It expands $variable references in a later pass. It doesn't look at the results to see if they look like additional variable expansions. The equal signs are effectively ignored.
If you want to know the nitty gritty details, open up the Bash man page. It's incredibly long and the details are scattered throughout. Let me pull out the key sections and some choice quotes to help you digest it:
Shell Grammar, Simple Commands
A simple command is a sequence of optional variable assignments followed by blank-separated words and redirections, and terminated by a control operator.
Simple Command Expansion
When a simple command is executed, the shell performs the following expansions, assignments, and redirections, from left to right.
The words that the parser has marked as variable assignments (those preceding the command name) and redirections are saved for later processing.
The words that are not variable assignments or redirections are expanded. If any words remain after expansion, the first word is taken to be the name of the command and the remaining words are the arguments.
...
If no command name results, the variable assignments affect the current shell environment. Otherwise, the variables are added to the environment of the executed command and do not affect the current shell environment.
Expansion
Expansion is performed on the command line after it has been split into words. There are seven kinds of expansion performed: brace expansion, tilde expansion, parameter and variable expansion, command substitution, arithmetic expansion, word splitting, and pathname expansion.
The order of expansions is: brace expansion, tilde expansion, parameter, variable and arithmetic expansion and command substitution (done in a left-to-right fashion), word splitting, and pathname expansion.
Expansion, Parameter Expansion
The $ character introduces parameter expansion, command substitution, or arithmetic expansion.
Assignments are marked in step 1 and variables (AKA parameters) are expanded in step 4.
The only things that happen after variable expansion are:
Word splitting. A variable can expand to multiple words if it contains whitespace. (Or to be more precise, if it contains any of the characters in the inter-field separator variable $IFS.)
Pathname expansion. Also known as globbing, or wildcards. If a variable contains *, ?, or [ they'll be expanded to the names of matching files, if there are any.
Quote removal. This pass happens after variable expansion, but it specifically does not apply to the results of any previous expansion step. So quotes the user typed are removed, but quotes that were the results of a substitution are retained.
Neither word splitting nor pathname expansion are what you need, so that's why it's not possible to store an assignment in a variable.
Given a variable VAR, that may contain an empty string, is it possible to make bash always interpret both
ls $VAR
ls "$VAR"
as "command that is given one argument"?
For bash, the IFS variable allows determining how the unquoted variable expansion in the first command is interpreted, defaulting to "Split the string at newlines, tabs and spaces, and treat each one as separate argument."
Things can be made a bit more convenient by settings IFS=''. But even then, if $VAR expands to am empty string, it will not be treated as the empty string but as absence of an argument.
This behavior actually makes sense for other values of IFS: By default unquoted expansion is treated as "list of whitespace-separated tokens", with the empty string being the empty list.
But is there a feature, to make $VAR and "$VAR" truely equivalent, at least for IFS=''?
I have a simple question but I wonder what is the difference between ${varname} and $varname ?
I use both but I don't see any difference which could tell me when to use one or the other.
Using {} in variable names helps get rid of ambiguity while performing variable expansion.
Consider two variables var and varname. Lets see you wanted to append the string name to the variable var. You can't say $varname because that would result in the expansion of the variable varname. However, saying ${var}name would help you achieve the desired result.
$ var="This is var variable."
$ varname="This is varname variable."
$ echo $varname
This is varname variable.
$ echo ${var}name
This is var variable.name
Braces are also required when accessing any element of an array.
$ a=( foo bar baz ) # Declare an array
$ echo $a[0] # Accessing first element -- INCORRECT
foo[0]
$ echo ${a[0]} # Accessing first element -- CORRECT
foo
Quoting from info bash:
Any element of an array may be referenced using ${name[subscript]}.
The braces are required to avoid conflicts with pathname expansion.
They are the same in a basic case, but using ${varname} gives more control and ability to work with the variable. It also skips edge cases in which it can create confusion. And finally, it enables variable expansion as described in Shell Parameter Expansion:
The ‘$’ character introduces parameter expansion, command
substitution, or arithmetic expansion. The parameter name or symbol to
be expanded may be enclosed in braces, which are optional but serve to
protect the variable to be expanded from characters immediately
following it which could be interpreted as part of the name.
When braces are used, the matching ending brace is the first ‘}’ not
escaped by a backslash or within a quoted string, and not within an
embedded arithmetic expansion, command substitution, or parameter
expansion.
The basic form of parameter expansion is ${parameter}. The value of
parameter is substituted. The braces are required when parameter is a
positional parameter with more than one digit, or when parameter is
followed by a character that is not to be interpreted as part of its
name.
Let's see a basic example. Here, the use of ${} allows us to do something that a simple $ does not. Consider we want to write $myvar + "blabla"::
$ myvar=23
$ echo $myvar
23
$ echo $myvarblabla
<--- the variable $myvarblabla doesn't exist!
$ echo ${myvar}blabla
23blabla
The distinction becomes important when something follows the variable:
text="House"
plural="${text}s"
Without the braces, the shell would see texts as variable name which wouldn't work.
The braces are also necessary when you use the extended syntax to specify defaults (${name-default}), display errors when undefined (${name?error}), or pattern substitution (see this article for other patterns; it's for BASH but most work for KSH as well)
> echo $name-default
-default
> echo ${name-default}
default
Related:
Parameter Substitution in Korn-/POSIX-Shell
What does $* mean in bash scripting?
I tried to search on google for it, but I found only about $0, $1 and so on.
From the man page:
* Expands to the positional parameters, starting from one. When the expansion occurs within double quotes, it expands to a single
word with the value of each parameter separated by the first character of the IFS special variable. That is, "$*" is equivalent
to "$1c$2c...", where c is the first character of the value of the IFS variable. If IFS is unset, the parameters are separated
by spaces. If IFS is null, the parameters are joined without intervening separators.
So it is equivalent to all the positional parameters, with slightly different semantics depending on whether or not it is in quotes.
See this page:
http://tldp.org/LDP/abs/html/internalvariables.html#IFSEMPTY
The behavior of $* and $# when $IFS is empty depends
+ on which Bash or sh version being run.
It is therefore inadvisable to depend on this "feature" in a script.
It's all the arguments passed to the script, except split by word. You almost always want to use "$#" instead. And it's all in the bash(1) man page.
Its the list of arguments supplied on the command line to the script .$0 will be the script name.
It's a space separated string of all arguments. For example, if $1 is "hello" and $2 is "world", then $* is "hello world". (Unless $IFS is set; then it's an $IFS separated string.)
You can use symbolhound search engine to find codes that google will not look for.
For your query click here
If you see $ in prefix with anything , it means its a variable. The value of the variable is used.
Example:
count=100
echo $count
echo "Count Value = $count"
Output of the above script:
100
Count Value = 100
As an independent command it doesn't have any significance in bash scripting.
But, as per usage in commands, it's used to indicate common operation on files / folders with some common traits.
and with grep used to represent zero or more common traits in a command.