I've connected to a container inside my K8s cluster using:
kubectl -n <namespace> exec -ti <pod> -- bash
Next I'm looking at the content of files within the /proc virtual folder. So for looking at the environment variables defined for process with id 1, I do cat /proc/1/environ.
The problem is that the output is:
var1=valuevar2=valuevar3=value
instead of:
var1=value
var2=value
var3=value
The same thing happens for cat /proc/1/cmdline, as the output is
commandarg1arg2
instead of
command arg1 arg2
What's causing the newline and spaces to be omitted from the output, and how can it be corrected?
The pieces of the environment can not be separated by Newlines, because a Newline character could be part of the environment string itself. Imagine an environment variable with name X, where its value is AB\nY=Z. You could not distinguish it from the case that you have two variables, X and Y.
The heart of the environment managements are the system calls getenv and setenv, which are for instance present in the C language. The value of the environment variable is a C-string, i.e. a sequence of bytes terminated by a NUL-byte. In fact, NUL is the only value which is not permitted inside the value of an environment variable.
Hence what your cat shows you is the environment definitions, separated by a NUL-byte.
Related
In the bash shell, I'm using git-bash.exe, how do I access the Windows 10 ProgramFiles(x86) environment variable?
If I execute printenv I see it in the output with the casing noted but attempts to access it using echo $ProgramFiles(x86), echo $ProgramFiles\(x86\) and echo $"ProgramFiles(x86)" do not work.
I am able to access the non-x86 version of that environment variable without any issue using echo $PROGRAMFILES and do relevant colon removal and backslash to forward replacements necessary to use it in PATH environment variable updates, e.g. PATH=$PATH:"/${PROGRAMFILES//:\\//} (x86)/Some App Path With Spaces/" followed by echo $PATH and printenv PATH that confirms the desired result. The issue is that I'd rather not have to compose the ProgramFiles(x86) environment variable versus being able to use it directly in updates to the PATH environment variable.
Along these same lines when trying to use the Windows APPDATA [ = C:\Users<username>\AppData\Roaming ] environment variable in updates to PATH environment variable I need to be able to replace not only the initial colon & backslash but also the subsequent backslashes with forward slashes. Using echo ${APPDATA//:\\//} produces C/Users\<username>\AppData\Roaming and I'm not aware of how to get the bash environment variable character matching and substitution syntax to cover both cases in order to produce the required C/Users/<username>/AppData/Roaming necessary for use in updates to PATH environment variable.
Note: there's a flaw in the process described below. In particular, if some environment variable is set to a multi-line value where one of the value lines matches the sed expression, you'll capture that line as well. To avoid this, if you have a Python available, you could use:
python -c 'import os; print(os.getenv("FOO(BAR)"))'
for instance. This will print None if the variable is not set, so you might want to make it fancier: e.g., supply a default value, or use sys.exit(1) if the variable is not set, for instance. (But if you have a Python interpreter available, you might consider writing in Python rather than directly in bash.)
Unix shell (sh, bash, etc) variable names—including environment variables—are constrained to character sets that exclude parentheses. In particular, "$FOO(BAR)" always parses as a reference to variable $FOO, followed by (BAR) as a separate word. This holds even with braceed expansion forms, where the separate word (BAR) is syntactically invalid:
bash$ echo "${FOO(BAR)}"
bash: ${FOO(BAR)}: bad substitution
Nonetheless, it is possible to set such variables, and access them, using other programs. For instance, using Python I set FOO(BAR) to hello:
>>> import os
>>> os.environ["FOO(BAR)"] = "hello"
>>> import subprocess
>>> subprocess.call("bash")
bash$
This bash instance cannot directly access the variable, but env prints all the variables:
bash$ env | grep FOO
FOO(BAR)=hello
If you have env (you probably do) and sed, you can combine them to extract arbitrary variables:
bash$ setting="$(env | sed -n 's/^FOO(BAR)=//p')"
bash$ echo "$setting"
hello
So assuming that Windows Bash doesn't have any special case to work around this particular clumsiness better, this same trick should work for "ProgramFiles(x86)".
Substituting multiple backslashes with forward slashes
You're mostly there: the problem is that your pattern looks specifically for :\ but the strings have multiple \s without colons. Your best bet is probably to have a program or function that actually understands Windows paths, as they don't necessarily have drive letters at the front (see https://learn.microsoft.com/en-us/dotnet/standard/io/file-path-formats). But this pattern works for all-backslash:
bash$ v='a\b\c'
bash$ echo ${v//\\/\/}
a/b/c
The double slash means "substitute all occurrences". The pattern is then \\, which matches one backslash. The next slash introduces the replacement string, which is \/, which means one forward slash. (This can also be written as just / but I find that harder to read, oddly enough.)
Of course this does not replace the colon in C:, so we need one more substitution. You can't do that in one ${...} expansion, so the trick is to add another one:
bash$ v='C:\a\b\c'
bash$ echo ${v//\\/\/}
C:/a/b/c
bash$ v1="${v//\\//}"; echo ${v1/:/}
C/a/b/c
Put this inside a shell function, which you can eventually make smart enough to handle all valid paths, and that way you can use local to keep the variable name v1 from leaking.
Regarding APPDATA: The cygpath program can convert pathnames between Windows, Unix and "Mixed" conventions. Both Cygwin and Git for Windows come with this tool. Example:
$ echo "$APPDATA"
C:\Users\me\AppDataRoaming\
$ cygpath -u "$APPDATA"
/c/Users/me/AppData/Roaming
$ cygpath -m "$APPDATA"
C:/Users/me/AppData/Roaming
$ cygpath -w "$APPDATA"
C:\Users\me\AppData\Roaming
The "mixed" format is quite usefull because even most windows programs and Git for Windows can handle that format directly.
Assigning the output of cygpath to a variable works like this (note the quotes!):
$ XAPP=$(cygpath "$APPDATA")
$ echo "$XAPP"
$ cd "$XAPP"
I have in my environment some variables that have an invalid identifier. When I try to unset them, I get the following error:
$ unset A-B
bash: unset: `A-B': not a valid identifier
How to unset them?
Related to the answer by Honza P:
According to man bash the environment variables have the form name=value where name consists only of alphanumeric characters and underscores, and begins with an alphabetic character or an underscore. So the name A-B is an invalid variable identifier for Bash, and cannot be used or accessed because it has internal validations (enforced strongly in late Bash versions because of vulnerabilities in Bash itself). The only solution is starting another secondary sub-shell that not have the offending names using the utility env to erasing them before launching Bash:
env -u 'A-B' bash
Note that the single quotes aren't needed, but to me is more readable that way as indicates the text inside is a string, not other command.
If you have more variables simply list them separated by spaces, or if you want to run only an script without the variables you can use for example:
env -u 'A-B' 'OTHER-VAR' 'bad-name' bash -c 'myscript.sh arg1 arg2'
The subshell will run myscript.sh with the arguments arg1 and arg2, and exit to the current Bash shell.
Consult the man page of 'env': man env
Not the best solution I guess but it worked in my case.
I tried an example by bishop env -u "foo-bar=baz", then env -u "foo-bar" but this approach still left some garbage in variables e.g. _=foo-bar. So I used unset _ and then it is gone.
I'm using a bash shell. I have some values from my env that I want to extract and assign those values to a variable. I also need to split it at the = sign. What's the best utility for this using bash?
I basically need to create a string with these values stripping out the delimter as well:
echo -e "This is my $KEY and my profile is $PROFILE"
This is my Ru2cXJbgWQ0wdtKBGbS5/nVQvGo and my profile is foobar
Result:
-bash-4.1$ echo -e "This is my $KEY"
This is my
-bash-4.1$ env | grep KEY
KEY=Ru2cXJbgWQ0wdtKBGbS5/nVQvGo
Example:
$ env | grep KEY
KEY=Ru2cXJbgWQ0wdtKBGbS5/nVQvGo
$ env | grep DEFAULT_PROFILE
DEFAULT_PROFILE=foobar
To complement Anthony Geoghegan's helpful answer, which shows that Bash automatically represents environment variables as shell variables[1]:
In short: the fact that env | grep KEY returns KEY=Ru2cXJbgWQ0wdtKBGbS5/nVQvGo does NOT necessarily mean that an environment variable named KEY exists.
It is conceivable that the grep matches you're getting are matches of other environment variables' values or even the last line of multi-line variable name. Thanks, chepner.
While environment variable and values are typically single-line, they don't have to be. An example is Bash's own ability to export functions, whose typically multi-line definitions are exported as the values of specially named environment variables.
To verify that a given (Bash) shell variable is indeed based on an environment variable, use:
declare -p KEY # example with variable name 'KEY'
If KEY is indeed an environment variable, you'll see (using your example):
declare -x KEY="Ru2cXJbgWQ0wdtKBGbS5/nVQvGo"
Note the -x, which indicates an exported variable, which is synonymous with being an environment variable.
By contrast:
If KEY is a shell variable that isn't also an environment variable, the -x will be missing.
if KEY is neither a shell variable nor an environment variable, you'll see an error message.
anubhava, in a comment on the question, demonstrates a shell-independent way to test if an environment variable is defined:
printenv KEY # prints value of env. var. 'KEY'
An exit code of 0 tells you that the variable exists (even if it is has no value).
Note that printenv is an external utility that is not POSIX-compliant; it does, however, come with both Linux and BSD/OSX.
[1] Note that by using Bash's own string expansion (interpolation of double-quoted strings), what gets expanded is not only environment variable references, but also shell-only variable references. GNU utility envsubst, by contrast, allows you to restrict expansion to environment variables.
When Bash starts, each environment variable that it inherits is automatically created as a shell variable aka parameter. From the Bash man page:
When a program is invoked it is given an array of strings called the environment. This is a list of name-value pairs, of the form name=value.
The shell provides several ways to manipulate the environment. On invocation, the shell scans its own environment and creates a parameter for each name found, automatically marking it for export to child processes.
If KEY and PROFILE are environment variables, running
echo -e "This is my $KEY and my profile is $PROFILE"
should print the desired response.
What is the difference between:
prompt$ TSAN_OPTIONS="suppressions=/somewhere/file" ./myprogram
and
prompt$ TSAN_OPTIONS="suppressions=/somewhere/file"
prompt$ ./myprogram
The thread-sanitizer library gives the first case as how to get their library (used within myprogram) to read the file given in options. I read it, and assumed it was supposed to be two separate lines, so ran it as the second case.
The library doesn't use the file in the second case, where the environment variable and the program execution are on separate lines.
What's the difference?
Bonus question: How does the first case even run without error? Shouldn't there have to be a ; or && between them? The answer to this question likely answers my first...
The format VAR=value command sets the variable VAR to have the value value in the environment of the command command. The spec section covering this is the Simple Commands. Specifically:
Otherwise, the variable assignments shall be exported for the execution environment of the command and shall not affect the current execution environment except as a side-effect of the expansions performed in step 4.
The format VAR=value; command sets the shell variable VAR in the current shell and then runs command as a child process. The child process doesn't know anything about the variables set in the shell process.
The mechanism by which a process exports (hint hint) a variable to be seen by child processes is by setting them in its environment before running the child process. The shell built-in which does this is export. This is why you often see export VAR=value and VAR=value; export VAR.
The syntax you are discussing is a short-form for something akin to:
VAR=value
export VAR
command
unset -v VAR
only without using the current process environment at all.
To complement Etan Reisner's helpful answer:
It's important to distinguish between shell variables and environment variables:
Note: The following applies to all POSIX-compatible shells; bash-specific extensions are marked as such.
A shell variable is a shell-specific construct that is limited to the shell that defines it (with the exception of subshells, which get their own copies of the current shell's variables),
whereas an environment variable is inherited by any child process created by the current process (shell), whether that child process is itself a shell or not.
Note that all-uppercase variable names should only be used for environment variables.
Either way, a child process only ever inherits copies of variables, whose modification (by the child) does not affect the parent.
All environment variables are also shell variables (the shell ensures that),
but the inverse is NOT true: shell variables are NOT environment variables, unless explicitly designated or inherited as such - this designation is called exporting.
note that the off-by-default -a shell option (set with set -a, or passed to the shell itself as a command-line option) can be used to auto-export all shell variables.
Thus,
any variables you create implicitly by assignment - e.g., TSAN_OPTIONS="suppressions=/somewhere/file" - are ONLY shell variables, but NOT ALSO environment variables,
EXCEPT - perhaps confusingly - when prepended directly to a command - e.g. TSAN_OPTIONS="suppressions=/somewhere/file" ./myprogram - in which case they are ONLY environment variables, only in effect for THAT COMMAND.
This is what Etan's answer describes.
Shell variables become environment variables as well under the following circumstances:
based on environment variables that the shell itself inherited, such as $HOME
shell variables created explicitly with export varName[=value] or, in bash, also with declare -x varName[=value]
by contrast, in bash, using declare without -x, or using local in a function, creates mere shell variables
shell variables created implicitly while the off-by-default -a shell option is in effect (with limited exceptions)
Once a shell variable is marked as exported - i.e., marked as an environment variable - any subsequent changes to the shell variable update the environment variable as well; e.g.:
export TSAN_OPTIONS # creates shell variable *and* corresponding environment variable
# ...
TSAN_OPTIONS="suppressions=/somewhere/file" # updates *both* the shell and env. var.
export -p prints all environment variables
unset [-v] MYVAR undefines shell variable $MYVAR and also removes it as an environment variable, if applicable.
in bash:
You can "unexport" a given variable without also undefining it as a shell variable with export -n MYVAR - this removes MYVAR from the environment, but retains its current value as a shell variable.
declare -p MYVAR prints variable $MYVAR's current value along with its attributes; if the output starts with declare -x, $MYVAR is exported (is an environment variable)
This question already has answers here:
What does the 'export' command do?
(3 answers)
Closed 7 years ago.
When I customize my environment, I add PATH=$PATH:$My-own-Path in file .bash_profile.
The tutorials tell me I should use this one: export PATH=$PATH:$My-own-Path
So, what is the difference?
To answer your exact specific question, in this particular case, there isn't any difference. Why?
Somewhere in the initialization process, the variable PATH has already been exported. A change in the variable's value which is already exported does not need another export; this is automatic. The processes fired hereafter will get the new value.
export makes the environment variable available to child processes
From man bash:
... The export and declare -x commands allow parameters and functions to be added to and deleted from the environment. If the value of a parameter in the environment is modified, the new value becomes part of the environment, replacing the old.
Also from man bash:
export [-fn] [name[=word]] ...
export -p
The supplied names are marked for automatic export to the environment of subsequently executed commands. If the -f option is given, the names refer to functions. If no names are given, or if the -p option is supplied, a list of names of all exported variables is printed. The -n option causes the export property to be removed from each name. If a variable name is followed by =word, the value of the variable is set to word. export returns an exit status of 0 unless an invalid option is encountered, one of the names is not a valid shell variable name, or -f is supplied with a name that is not a function.
Exported variables are available to other programs. Non-exported variables are not.
Example:
$ myVar=Foo # Create local
$ env | grep '^myVar='
$ export myVar # Export myVar to child process
$ env | grep '^myVar='
Foo
If you want to read more about this, check out export (GNU Bash manual).
Also, please note that non-exported variables will be available to subshells run with (...) and other similar notations:
$ thereVar=Bar
$ (echo $thereVar; echo $myVar; $myVar=testing; echo $myVar)
Bar
Foo
Testing
$echo $myVar
Foo
The subshell cannot affect variables in the parent shell.
For more information on subshells, please reference:
Command Grouping
Command Execution Environment
Every process has an area of memory called the environment block. In the environment block are environment variables. These look like ordinary variables, for example x=42.
In most shells (C shell is an exception) you move an ordinary variable into the environment block using export. That command can also create an environment variable without going through an intermediate stage. If the variable is already in the environment block then export will have no effect.
So why? When a new process is created, the default action is to copy various "core information" from parent to child. These include the current directory, the umask, the file descriptor table, the uid and gid, and the environment block.
Note that the child only gets a copy of the parent's environment block. The variable is not shared and cannot be passed back to the parent (except by using some other inter-process communication mechanism).
You can override this default behaviour using the env program, but this is rarely required.
So, if we set an environment variable in a shell script using export then all our child processes we create, when we call other programs, will get a copy of them. Some variable names are well-known and have a special meaning, and the PATH environment variable is probably the most important of those.
The PATH environment variable is used to find programs on UNIX/Linux. Directories in PATH are searched in left-right order each time we need to load a program. Bash also caches executable paths in a hash (KornShell calls them "tracked aliases").