I have a shell script (let's call it produce.sh) which outputs data in the following form, but does not save it to a file:
FOO=value
BAR=value2
ZAP=value3
I'd like to use these values as environment variables in a shell script. I'm currently doing this using the following shell code:
export $(./produce.sh)
This works great, except when the values to the right of the = contain spaces. For instance:
FOO=split value
I've tried two different approaches in produce.sh:
Wrapping the values in quotes (FOO="split value")
Escaping whitespace with backslashes (FOO=split\ value)
Both of these do not work: if I inspect the environment variables, FOO contains "split in the first example and split\ in the second.
How can I get export to handle this correctly?
The f parameter expansion flag in zsh will split an input on newlines, so this should handle input values with whitespace:
export ${(f)"$(./produce.sh)"}
What's happening
The output from produce.sh:
key-value pairs.
each kv pair is on its own line.
key is separated from the value by =.
syntax is shell-like, but not exactly shell syntax, so,
white space and some other characters special to the shell are allowed in the value.
The parts of the substitution:
produce.sh : generates the key-value output, for example: N=V1\nP=V 2\n.
$(...) : command substitution. It is replaced with the output, minus trailing newlines: N=V1\nP=V 2.
"..." : the quotes ensure that the prior result is treated as a single term for the next step.
${(f)...} : expands that single term into multiple scalar values, splitting on newlines due to the (f) flag. The result is effectively 'N=V1' 'P=V 2'.
export : assigns and exports each parameter, acting like export 'N=V1' 'P=V 2'.
Another option
The substitution below adds some other cryptic zsh-isms to create an associative array. This avoids adding arbitrary variables to the consuming shell's
environment:
% ./s2.sh
A=A
B=
C=C C
% typeset -A vals=("${(#s:=:)${(f)"$(./s2.sh)"}}")
% print ${vals[A]}
A
% print ${vals[C]}
C C
A minor tradeoff - this will not work if the value contains an equals, e.g.
D=D=D.
FOO=split value
Does not set a variable to a value containing a space. It temporarily sets FOO to split and runs the command value in an environment where FOO has been set. The overall effect is similar to
(export FOO=split; value)
You can experiment with this by doing a
FOO=split printenv
which will run printenv and show in its output the value of FOO.
To set a variable to a value containing space, you need to escape the space, for instance with
FOO=split\ value
or by using quotes.
For processing this file, you could use eval:
eval export $(./produce.sh)
Of course you should this only if you have full control over what produce.sh is writing to stdout (i.e. don't let any tainted data sneak in), because eval is a potential security hole.
Alternatively, you could modify produce.sh to already produce the export prefix in front of each line, and then do just a
eval $(./produce.sh)
Related
How can I add a path with a space in a Bash variable in .bashrc? I want to store some variables in .bashrc for paths and I encountered a path with a space in it.
I tried to add it between ' ' or use the escape character \, but it didn't help:
games=/run/media/mohamedRadwan/games\ moves # this doesn't work
games='/run/media/mohamedRadwan/games moves' # or this
games="/run/media/mohamedRadwan/games moves" # or this
... when I run:
mount $games
... it throws an error indicating that it's only trying to mount /run/media/mohamedRadwan/games.
But when I run echo $games, it shows the full value, /run/media/mohamedRadwan/games moves.
How can I solve this?
mount /dev/sda9 "$games"
As mentioned, always quote variable dereferences. Otherwise, the shell confuses the spaces in the variable's value as spaces separating multiple values.
When variable contains spaces, variable expansion and then word splitting will result to many arguments, echo command will display all arguments but other program or function may handle arguments another way.
Surrounding variable with double quotes will prevent arguments to be splitted
printf "'%s'\n" $games
printf "'%s'\n" "$games"
For each of two examples below I'll try to explain what result I expected and what I got instead. I'm hoping for you to help me understand why I was wrong.
1)
VAR1=VAR2
$VAR1=FOO
result: -bash: VAR2=FOO: command not found
In the second line, $VAR1 gets expanded to VAR2, but why does Bash interpret the resulting VAR2=FOO as a command name rather than a variable assignment?
2)
'VAR=FOO'
result: -bash: VAR=FOO: command not found
Why do the quotes make Bash treat the variable assignment as a command name?
Could you please describe, step by step, how Bash processes my two examples?
How best to indirectly assign variables is adequately answered in other Q&A entries in this knowledgebase. Among those:
Indirect variable assignment in bash
Saving function output into a variable named in an argument
If that's what you actually intend to ask, then this question should be closed as a duplicate. I'm going to make a contrary assumption and focus on the literal question -- why your other approaches failed -- below.
What does the POSIX sh language specify as a valid assignment? Why does $var1=foo or 'var=foo' fail?
Background: On the POSIX sh specification
The POSIX shell command language specification is very specific about what constitutes an assignment, as quoted below:
4.21 Variable Assignment
In the shell command language, a word consisting of the following parts:
varname=value
When used in a context where assignment is defined to occur and at no other time, the value (representing a word or field) shall be assigned as the value of the variable denoted by varname.
The varname and value parts shall meet the requirements for a name and a word, respectively, except that they are delimited by the embedded unquoted equals-sign, in addition to other delimiters.
Also, from section 2.9.1, on Simple Commands, with emphasis added:
The words that are recognized as variable assignments or redirections according to Shell Grammar Rules are saved for processing in steps 3 and 4.
The words that are not variable assignments or redirections shall be expanded. If any fields remain following their expansion, the first field shall be considered the command name and remaining fields are the arguments for the command.
Redirections shall be performed as described in Redirection.
Each variable assignment shall be expanded for tilde expansion, parameter expansion, command substitution, arithmetic expansion, and quote removal prior to assigning the value.
Also, from the grammar:
If all the characters preceding '=' form a valid name (see the Base Definitions volume of IEEE Std 1003.1-2001, Section 3.230, Name), the token ASSIGNMENT_WORD shall be returned. (Quoted characters cannot participate in forming a valid name.)
Note from this:
The command must be recognized as an assignment at the very beginning of the parsing sequence, before any expansions (or quote removal!) have taken place.
The name must be a valid name. Literal quotes are not part of a valid variable name.
The equals sign must be unquoted. In your second example, the entire string was quoted.
Assignments are recognized before tilde expansion, parameter expansion, command substitution, etc.
Why $var1=foo fails to act as an assignment
As given in the grammar, all characters before the = in an assignment must be valid characters within a variable name for an assignment to be recognized. $ is not a valid character in a name. Because assignments are recognized in step 1 of simple command processing, before expansion takes place, the literal text $var1, not the value of that variable, is used for this matching.
Why 'var=foo' fails to act as an assignment
First, all characters before the = must be valid in variable names, and ' is not valid in a variable name.
Second, an assignment is only recognized if the = is not quoted.
1)
VAR1=VAR2
$VAR1=FOO
You want to use a variable name contained in a variable for the assignment. Bash syntax does not allow this. However, there is an easy workaround :
VAR1=VAR2
declare "$VAR1"=FOO
It works with local and export too.
2)
By using single quotes (double quotes would yield the same result), you are telling Bash that what is inside is a string and to treat it as a single entity. Since it is the first item on the line, Bash tries to find an alias, or shell builtin, or an executable file in its PATH, that would be named VAR=FOO. Not finding it, it tells you there is no such command.
An assignment is not a normal command. To perform an assignment contained in a quote, you would need to use eval, like so :
eval "$VAR1=FOO" # But please don't do that in real life
Most experienced bash programmers would probably tell you to avoid eval, as it has serious drawbacks, and I am giving it as an example just to recommend against its use : while in the example above it would not involve any security risk or error potential because the value of VAR1 is known and safe, there are many cases where an arbitrary (i.e. user-supplied) value could cause a crash or unexpected behavior. Quoting inside an eval statement is also more difficult and reduces readability.
You declare VAR2 earlier in the program, right?
If you are trying to assign the value of VAR2 to VAR1, then you need to make sure and use $ in front of VAR2, like so:
VAR1=$VAR2
That will set the value of VAR2 equal to VAR1, because when you utilize the $, you are saying that value that is stored in the variable. Otherwise it doesn't recognize it as a variable.
Basically, a variable that doesn't have a $ in front of it will be interpreted as a command. Any word will. That's why we have the $ to clarify "hey this is a variable".
Am trying to print a bunch of strings in a script (in zsh) and it doesn't seem to work. The code would work if I place the array in a variable and use it instead. Any ideas why this doesn't work otherwise?
for string in (some random strings to print) ; echo $string
The default form of the for command in zsh does not use parentheses (if there are any they are not interpreted as part of the for statement):
for string in some random strings to show
do
echo _$string
done
This results in the following output:
_some
_random
_strings
_to
_show
So, echo _$string was run for each word after in. The list ends with the newline.
It is possible to write the whole statement in a single line:
for string in some random strings to show; do echo _$string; done
As usual when putting multiple shell commands in the same line, newlines just need to be replaced by ;. The exception here is the newline after do; while zsh allows a ; to be placed after do, it is usually not done, and in bash it would be a syntax error.
There are also several short forms available for for, all of which are equivalent to the default form above and produce the same output:
for single commands (to be exact: single pipelines or multiple pipelines linked with && or ||, where a pipeline can also be just a single command), there are two options:
the default form, just without do or done:
for string in some random strings to show ; echo _$string
without in but with parentheses, also without do or done
for string (some random strings to show) ; echo _$string
for a list of commands (like in the default form), foreach instead of for, no in, with parentheses and terminated by end:
foreach string (some random strings to show) echo _$string ; end
In your case, you mixed the two short forms for single commands. Due to the presence of in, zsh did not take the parentheses as a syntactic element of the for command. Instead they are interpreted as a glob qualifier. Aside from the fact that you did not intend any filename expansions, this fails for two reasons:
there is no pattern (with or without actual globs) before the glob qualifier. So any matching filename would have to exactly match an empty string, which is just not possible
but mainly "some random strings to print" is not a valid glob qualifier. You probably get an error like "zsh: unknown file attribute: i" (at least with zsh 5.0.5, it may depend on the zsh version).
Check the zsh forloop documentation:
for x (1 2 3); do echo $x; done
for x in 1 2 3; do echo $x; done
You are probably trying to do this:
for string in some random strings to print ;do
echo $string
done
can someone explain me with this code
data=$(date +"%Y-%m-%dS%H:%M:%S")
name="/home/cft/"$data"_test.tar"
touch $name
works, creating a new .tar file but this code doesn't work
data=$(date +"%Y-%m-%dS%H:%M:%S")
name= "/home/cft/"$data"_test.tar"
touch $name
and gives me this error: no such file or directory?
why the space between = and inverted commas creates this error?
Shell allows you to provide per-command environment overrides by prefixing the command with one or more variable assignments.
name= "/home/cft/"$data"_test.tar"
asks the shell to run the program named /home/cft/2013-10-08S12:00:00_test.tar (for example) with the value of name set to the empty string in its environment.
(In your case, the error occurs because the named tar file either doesn't exist or, if it does, is not an executable file.)
A variable assignment is identified by having no whitespace after the equal sign.
(name = whatever, of course, is simply a command called name with two string arguments, = and whatever.)
You can't have whitespace between the equal sign and the definition.
http://www.tldp.org/LDP/abs/html/varassignment.html
There is no theory behind this. It's just a decision the language designers made, and which the parser enforces.
In BASH (and other Bourne type shells like zsh and Kornshell), the equal sign cannot have spaces around it when setting variables.
Good:
$ foo="bar"
Bad:
$ foo= "bar"
$ foo = "bar"
There's no real reason that would prevent spaces from being used. Other programming languages have no problems with this. It's just the syntax of the shell itself.
The reason might be related to the original Bourne shell parsing where the shell would break up a command line based upon whitespace. That would make foo=bar a single parameter instead of two or three (depending if you have white space on both sides or just one side of the equal sign). The shell could see the = sign, and know this parameter is an assignment.
The shell parameter parsing is very primitive in many ways. Whitespace is very important. The shell has to be small and fast in order to be responsive. That means stripping down unessential things like complex line parsing.
Inverted commas I believe you mean quotation marks. Double quotes are used to override the breaking out of parameters over white space:
Bad:
$ foo=this is a test
bash: is: command not found
Good:
$ foo="this is a test"
Double quotes allow interpolation. Single quotes don't:
$ foo="bar"
$ echo "The value of foo is $foo"
The value of foo is bar
$ echo 'The value of foo is $foo'
The value of foo is $foo.
If you start out with single quotes, you can put double quotes inside. If you have single quotes, you can put double quotes inside.
$ foo="bar"
$ echo "The value of foo is '$foo'"
The value of foo is 'bar'
$ echo 'The value of foo is "$foo"'
The value of foo is "$foo"
This means you didn't have to unquote $data. However, you would have to put curly braces around it because underscores are legal characters in variable names. Thus, you want to make sure that the shell understand that the variable is $data and not $data_backup:
name="/home/cft/${data}_test.tar"
A contrived example... given
FOO="/foo/bar/baz"
this works (in bash)
BAR=$(basename $FOO) # result is BAR="baz"
BAZ=${BAR:0:1} # result is BAZ="b"
this doesn't
BAZ=${$(basename $FOO):0:1} # result is bad substitution
My question is which rule causes this [subshell substitution] to evaluate incorrectly? And what is the correct way, if any, to do this in 1 hop?
First off, note that when you say this:
BAR=$(basename $FOO) # result is BAR="baz"
BAZ=${BAR:0:1} # result is BAZ="b"
the first bit in the construct for BAZ is BAR and not the value that you want to take the first character of. So even if bash allowed variable names to contain arbitrary characters your result in the second expression wouldn't be what you want.
However, as to the rule that's preventing this, allow me to quote from the bash man page:
DEFINITIONS
The following definitions are used throughout the rest of this docu‐
ment.
blank A space or tab.
word A sequence of characters considered as a single unit by the
shell. Also known as a token.
name A word consisting only of alphanumeric characters and under‐
scores, and beginning with an alphabetic character or an under‐
score. Also referred to as an identifier.
Then a bit later:
PARAMETERS
A parameter is an entity that stores values. It can be a name, a num‐
ber, or one of the special characters listed below under Special Param‐
eters. A variable is a parameter denoted by a name. A variable has a
value and zero or more attributes. Attributes are assigned using the
declare builtin command (see declare below in SHELL BUILTIN COMMANDS).
And later when it defines the syntax you're asking about:
${parameter:offset:length}
Substring Expansion. Expands to up to length characters of
parameter starting at the character specified by offset.
So the rules as articulated in the manpage say that the ${foo:x:y} construct must have a parameter as the first part, and that a parameter can only be a name, a number, or one of the few special parameter characters. $(basename $FOO) is not one of the allowed possibilities for a parameter.
As for a way to do this in one assignment, use a pipe to other commands as mentioned in other responses.
Modified forms of parameter substitution such as ${parameter#word} can only modify a parameter, not an arbitrary word.
In this case, you might pipe the output of basename to a dd command, like
BAR=$(basename -- "$FOO" | dd bs=1 count=1 2>/dev/null)
(If you want a higher count, increase count and not bs, otherwise you may get fewer bytes than requested.)
In the general case, there is no way to do things like this in one assignment.
It fails because ${BAR:0:1} is a variable expansion. Bash expects to see a variable name after ${, not a value.
I'm not aware of a way to do it in a single expression.
As others have said, the first parameter of ${} needs to be a variable name. But you can use another subshell to approximate what you're trying to do.
Instead of:
BAZ=${$(basename $FOO):0:1} # result is bad substitution
Use:
BAZ=$(_TMP=$(basename $FOO); echo ${_TMP:0:1}) # this works
A contrived solution for your contrived example:
BAZ=$(expr $(basename $FOO) : '\(.\)')
as in
$ FOO=/abc/def/ghi/jkl
$ BAZ=$(expr $(basename $FOO) : '\(.\)')
$ echo $BAZ
j
${string:0:1},string must be a variable name
for example:
FOO="/foo/bar/baz"
baz="foo"
BAZ=eval echo '${'"$(basename $FOO)"':0:1}'
echo $BAZ
the result is 'f'