I've been trying to figure out what is the purpose of brackets in the bash environment variables. For example, in the below actual example of code, why are some of the definitions using a {} around the PATH, for example, export ...=.../${PATH}. Note also that some of the definitions are different: some use {$ECLIPSE_DIR} with the $ within the brackets; some use ${PATH} with the $ outside of the brackets, and some omit brackets altogether. This code generally works, although sometimes errors like the one shown at the bottom are shown (they appear to be transient), and I'm not sure why such errors only show up sometimes and not others.
What are the common practices concerning ways to include bash environment variables, when should brackets be used, and what is the difference between putting the $ inside and outside of brackets? Also, why do some lines have an "export" before the variable name, and some do not? What is the difference here?
# ECLIPSE
ECLIPSE_DIR=$HOME/eclipse
PATH=${PATH}:{$ECLIPSE_DIR}
# ANT
ANT_HOME=/usr/bin/ant
PATH=${ANT_HOME}/bin:${PATH}
export ANT_HOME PATH
# GRADLE
export GRADLE_HOME=/usr/local/gradle
export PATH=$GRADLE_HOME/bin:$PATH</code>
-bash: export: `/usr/bin/ant/bin:/usr/local/bin:{/Users/me/eclipse}:/usr/bin/scala-2.9.0.1/bin:/usr/local/mysql/bin:/usr/local/bin:{/Users/me/eclipse}': not a valid identifier
The braces are usually used for clarity, but a practical use is breaking up text from variable names. Say I had the following:
$ word="ello"
$ echo "h$word"
hello
$ echo "y$wordw" # bash tries to find the var wordw, and replaces with a blank
y
$ echo "y${word}w"
yellow
Variable names are automatically separated by most punctuation (notably . or /).
echo "$word/$word.$word"
ello/ello.ello
Looking at that error you presented, {$ECLIPSE_DIR} gets the variable expanded and then surrounded with literal open and close braces. I think the solution should be changing it to ${ECLIPSE_DIR}
In response to the export question, export is used to make a variable accessible to the shell that called this script. Any variable set up in a script does not exist once the script is finished unless it is exported. Hence, if you want your PATH to change after running that script, export PATH will have to be called before the script is over.
Braces are used with bash variables to disambiguate between variables. For example, consider this:
VAR=this
echo $VAR_and_that
echo ${VAR}_and_that
The first echo prints nothing, since bash thinks you are trying to echo out the var this_and_that which of course doesn't exist. The second echo doesn't have this problem and outputs this_and_that, since bash knows to expand out the VAR variable due to the braces.
Related
I have the following bash script:
$ echo $(dotnet run --project Updater)
UPDATE_NEEDED='0' MD5_SUM="7e3ad68397421276a205ac5810063e0a"
$ export UPDATE_NEEDED='0' MD5_SUM="7e3ad68397421276a205ac5810063e0a"
$ echo $UPDATE_NEEDED
0
$ export $(dotnet run --project Updater)
$ echo $UPDATE_NEEDED
'0'
Why is it $UPDATE_NEEDED is 0 on the 3rd command, but '0' on the 5th command?
What would I need to do to get it to simply set 0? Using UPDATE_NEEDED=0 instead is not an option, as some of the other variables may contain a space (And I'd like to optimistically quote them to have it properly parse spaces).
Also, this is a bit of a XY problem. If anyone knows an easier way to export multiple variables from an executable that can be used later on in the bash script, that could also be useful.
To expand on the answer by Glenn:
When you write something like export UPDATE_NEEDED='0' in Bash code, this is 100% identical to export UPDATE_NEEDED=0. The quotes are used by Bash to parse the command expression, but they are then discarded immediately. Their only purpose is to prevent word splitting and to avoid having to escape special characters. In the same vein, the code fragment 'foo bar' is exactly identical to foo\ bar as far as Bash is concerned: both lead to space being treated as literal rather than as a word splitter.
Conversely, parameter expansion and command substitution follows different rules, and preserves literal quotes.
When you use eval, the command line arguments passed to eval are treated as if they were Bash code, and thus follow the same rules of expansion as regular Bash code, which leads to the same result as (1).
Apparently that Updater project is doing the equivalent of
echo "UPDATE_NEEDED=\'0\' MD5_SUM=\"7e3ad68397421276a205ac5810063e0a\""
It's explicitly outputting the quotes.
When you do export UPDATE_NEEDED='0' MD5_SUM="7e3ad68397421276a205ac5810063e0a",
bash will eventually remove the quotes before actually setting the variables.
I agree with #pynexj, eval is warranted here, although additional quoting is recommended:
eval export "$(dotnet ...)"
I have a script that outputs a list of env vars like:
THING=one
ANOTHER_THING=two
A_PATH="path/to a/directory"
When I try to export these env vars as export `./script`, the A_PATH env var exports as path/to.
If I export the list as plain text like:
export THING=one ANOTHER_THING=two A_PATH="path/to a/directory", it works just fine.
I'm stumped as to why bash treats the white space in the A_PATH differently in these two cases. I've tried various attempts at escaping the whitespace and I've even tried exporting line by line, but in every case it sees the whitespace as a delimiter rather than as a part of the path string.
why bash treats the white space in the A_PATH differently in these two cases
The result of command substitution `...` undergoes word splitting. Do not use ` backticks - use $(....) instead. Check your scripts with shellcheck.net .
Spaces inside string around double quotes are literally preserved. See quoting.
If the file has proper correct shell syntax, and it's meant to be sourced and support shell-ish execution, see https://unix.stackexchange.com/questions/614568/is-is-possible-to-export-all-variables-obtained-from-sourcing-a-file . If the file contains = separated variable name and optionally quoted string with custom syntax, write a parser for the file for that syntax - see ex. Parsing variables from config file in Bash for a stub to get started.
Without the actual code, it is difficult to understand what is going on. But one workaround that might solve your problem is to output the export command and evaluate it.
Example (script.sh):
#!/bin/bash
echo "export THING=one"
echo "export ANOTHER_THING=two"
echo "export A_PATH='path/to a/directory'"
eval $(./script.sh)
In the bash shell, I'm using git-bash.exe, how do I access the Windows 10 ProgramFiles(x86) environment variable?
If I execute printenv I see it in the output with the casing noted but attempts to access it using echo $ProgramFiles(x86), echo $ProgramFiles\(x86\) and echo $"ProgramFiles(x86)" do not work.
I am able to access the non-x86 version of that environment variable without any issue using echo $PROGRAMFILES and do relevant colon removal and backslash to forward replacements necessary to use it in PATH environment variable updates, e.g. PATH=$PATH:"/${PROGRAMFILES//:\\//} (x86)/Some App Path With Spaces/" followed by echo $PATH and printenv PATH that confirms the desired result. The issue is that I'd rather not have to compose the ProgramFiles(x86) environment variable versus being able to use it directly in updates to the PATH environment variable.
Along these same lines when trying to use the Windows APPDATA [ = C:\Users<username>\AppData\Roaming ] environment variable in updates to PATH environment variable I need to be able to replace not only the initial colon & backslash but also the subsequent backslashes with forward slashes. Using echo ${APPDATA//:\\//} produces C/Users\<username>\AppData\Roaming and I'm not aware of how to get the bash environment variable character matching and substitution syntax to cover both cases in order to produce the required C/Users/<username>/AppData/Roaming necessary for use in updates to PATH environment variable.
Note: there's a flaw in the process described below. In particular, if some environment variable is set to a multi-line value where one of the value lines matches the sed expression, you'll capture that line as well. To avoid this, if you have a Python available, you could use:
python -c 'import os; print(os.getenv("FOO(BAR)"))'
for instance. This will print None if the variable is not set, so you might want to make it fancier: e.g., supply a default value, or use sys.exit(1) if the variable is not set, for instance. (But if you have a Python interpreter available, you might consider writing in Python rather than directly in bash.)
Unix shell (sh, bash, etc) variable names—including environment variables—are constrained to character sets that exclude parentheses. In particular, "$FOO(BAR)" always parses as a reference to variable $FOO, followed by (BAR) as a separate word. This holds even with braceed expansion forms, where the separate word (BAR) is syntactically invalid:
bash$ echo "${FOO(BAR)}"
bash: ${FOO(BAR)}: bad substitution
Nonetheless, it is possible to set such variables, and access them, using other programs. For instance, using Python I set FOO(BAR) to hello:
>>> import os
>>> os.environ["FOO(BAR)"] = "hello"
>>> import subprocess
>>> subprocess.call("bash")
bash$
This bash instance cannot directly access the variable, but env prints all the variables:
bash$ env | grep FOO
FOO(BAR)=hello
If you have env (you probably do) and sed, you can combine them to extract arbitrary variables:
bash$ setting="$(env | sed -n 's/^FOO(BAR)=//p')"
bash$ echo "$setting"
hello
So assuming that Windows Bash doesn't have any special case to work around this particular clumsiness better, this same trick should work for "ProgramFiles(x86)".
Substituting multiple backslashes with forward slashes
You're mostly there: the problem is that your pattern looks specifically for :\ but the strings have multiple \s without colons. Your best bet is probably to have a program or function that actually understands Windows paths, as they don't necessarily have drive letters at the front (see https://learn.microsoft.com/en-us/dotnet/standard/io/file-path-formats). But this pattern works for all-backslash:
bash$ v='a\b\c'
bash$ echo ${v//\\/\/}
a/b/c
The double slash means "substitute all occurrences". The pattern is then \\, which matches one backslash. The next slash introduces the replacement string, which is \/, which means one forward slash. (This can also be written as just / but I find that harder to read, oddly enough.)
Of course this does not replace the colon in C:, so we need one more substitution. You can't do that in one ${...} expansion, so the trick is to add another one:
bash$ v='C:\a\b\c'
bash$ echo ${v//\\/\/}
C:/a/b/c
bash$ v1="${v//\\//}"; echo ${v1/:/}
C/a/b/c
Put this inside a shell function, which you can eventually make smart enough to handle all valid paths, and that way you can use local to keep the variable name v1 from leaking.
Regarding APPDATA: The cygpath program can convert pathnames between Windows, Unix and "Mixed" conventions. Both Cygwin and Git for Windows come with this tool. Example:
$ echo "$APPDATA"
C:\Users\me\AppDataRoaming\
$ cygpath -u "$APPDATA"
/c/Users/me/AppData/Roaming
$ cygpath -m "$APPDATA"
C:/Users/me/AppData/Roaming
$ cygpath -w "$APPDATA"
C:\Users\me\AppData\Roaming
The "mixed" format is quite usefull because even most windows programs and Git for Windows can handle that format directly.
Assigning the output of cygpath to a variable works like this (note the quotes!):
$ XAPP=$(cygpath "$APPDATA")
$ echo "$XAPP"
$ cd "$XAPP"
I'm using bash and have a file called x.config that contains the following:
MY_VAR=Something1
ANOTHER=Something2
To load these as environment variables I just use source:
$ source x.config
But this doesn't work if MY_VAR is called MY-VAR:
MY-VAR=Something1
ANOTHER=Something2
If I do the same thing I get:
x.config:1: command not found: MY-VAR=Something1
I've tried escaping - and a lot of other things but I'm stuck. Does anyone know a workaround for this?
A pure bash workaround that might work for you is to re-run the script using env to set the environment. Add this to the beginning of your script.
if [[ ! -v myscript_env_set ]]; then
export myscript_env_set=1
readarray -t newenv < x.config
exec env "${newenv[#]}" "$0" "$#"
fi
# rest of the script here
This assumes that x.config doesn't contain anything except variable assignments. If myscript_env_set is not in the current environment, put it there so that the next invocation skips this block. Then read the assignments into an array to pass to env. Using exec replaces the current process with another invocation of the script, but with the desired variables in the environment.
A dash (-) in an environment variable is not portable, and as you noticed, will cause a lot of problems. You can't set these from bash. Fix the application you want to invoke.
That being said, if you can't change the target app, you can do this from python:
#!/usr/bin/python
import os
with open('x.config') as f:
for line in f:
name, value = line.strip().split('=')
os.environ[name] = value
os.system('/path/to/your/app')
This is a very simplistic config reader, and for a more complex syntax you might want to use ConfigParser.
I have a shell script that I'm trying to write to a file using multiple variables, but one of them is being ignored.
#!/bin/bash
dir=/folder
name=bob
date=`date +%Y`
command > $dir/$name_$date.ext
The $name is being ignored. How can I fix this?
Have you noticed that the _ was "ignored" as well? That's a big hint.
If you use set -u, you'll see the following:
-bash: name_: unbound variable
The way bash parses it, the underscore is part of the variable name.
There are several ways to fix the problem.
The cleanest is the ${var} construct which separate the variable name from its surroundings.
You can also use quotation in various ways to force the right parsing, e.g.: "$dir/$name""_$date.ext"
And in case your variables might contain spaces (now, or in the future) use quotation for words.
command >"$dir/${name}_$date.ext"
command >"${dir}/${name}_${date}.ext"
Both these are fine, just pick one style and stick to it.