I'm using a bash shell. I have some values from my env that I want to extract and assign those values to a variable. I also need to split it at the = sign. What's the best utility for this using bash?
I basically need to create a string with these values stripping out the delimter as well:
echo -e "This is my $KEY and my profile is $PROFILE"
This is my Ru2cXJbgWQ0wdtKBGbS5/nVQvGo and my profile is foobar
Result:
-bash-4.1$ echo -e "This is my $KEY"
This is my
-bash-4.1$ env | grep KEY
KEY=Ru2cXJbgWQ0wdtKBGbS5/nVQvGo
Example:
$ env | grep KEY
KEY=Ru2cXJbgWQ0wdtKBGbS5/nVQvGo
$ env | grep DEFAULT_PROFILE
DEFAULT_PROFILE=foobar
To complement Anthony Geoghegan's helpful answer, which shows that Bash automatically represents environment variables as shell variables[1]:
In short: the fact that env | grep KEY returns KEY=Ru2cXJbgWQ0wdtKBGbS5/nVQvGo does NOT necessarily mean that an environment variable named KEY exists.
It is conceivable that the grep matches you're getting are matches of other environment variables' values or even the last line of multi-line variable name. Thanks, chepner.
While environment variable and values are typically single-line, they don't have to be. An example is Bash's own ability to export functions, whose typically multi-line definitions are exported as the values of specially named environment variables.
To verify that a given (Bash) shell variable is indeed based on an environment variable, use:
declare -p KEY # example with variable name 'KEY'
If KEY is indeed an environment variable, you'll see (using your example):
declare -x KEY="Ru2cXJbgWQ0wdtKBGbS5/nVQvGo"
Note the -x, which indicates an exported variable, which is synonymous with being an environment variable.
By contrast:
If KEY is a shell variable that isn't also an environment variable, the -x will be missing.
if KEY is neither a shell variable nor an environment variable, you'll see an error message.
anubhava, in a comment on the question, demonstrates a shell-independent way to test if an environment variable is defined:
printenv KEY # prints value of env. var. 'KEY'
An exit code of 0 tells you that the variable exists (even if it is has no value).
Note that printenv is an external utility that is not POSIX-compliant; it does, however, come with both Linux and BSD/OSX.
[1] Note that by using Bash's own string expansion (interpolation of double-quoted strings), what gets expanded is not only environment variable references, but also shell-only variable references. GNU utility envsubst, by contrast, allows you to restrict expansion to environment variables.
When Bash starts, each environment variable that it inherits is automatically created as a shell variable aka parameter. From the Bash man page:
When a program is invoked it is given an array of strings called the environment. This is a list of name-value pairs, of the form name=value.
The shell provides several ways to manipulate the environment. On invocation, the shell scans its own environment and creates a parameter for each name found, automatically marking it for export to child processes.
If KEY and PROFILE are environment variables, running
echo -e "This is my $KEY and my profile is $PROFILE"
should print the desired response.
Related
I recently got a book on shell scripting in bash. It states to list all the Environment Variables using the printenv command. I've noticed though that this doesn't list all variables, for example $PWD or $REPLY or $SSH_TTY. Is their a complete list I can reference for all of these Environment Variables and their Functions?
Within a shell like bash there are two types of variables; environment variables (Wikipedia) and shell variables. There are a number of predefined shell variables.
You can use the export built-in to "promote" a shell variable to an environment variable, which has the effect of making that variable available to any subprocesses launched from the shell.
As the name implies, printenv only reports the process' environment variables. Variables like PWD or REPLY are shell variables, and thus aren't displayed. As suggested in the comments, invoking set with no arguments will print all variables (environment and shell) available in your current session.
To display a list of the Environment Variables you can use
set
#if you want to see it nicely you can pipe it to more like this
set | more
From what I've read, is it correct to assume that the only thing export does is make the variable visible to child processes?
What would be a scenario where you would want to make a variable only visible to the scope it was initialized in, and what would be a scenario in which you would want a variable available to all child scopes?
In general, you only need to export a variable that another process will look for in its environment. How do you know which variables those are? You have to read their documentation.
Whether or not a variable is marked for export makes no difference to the current shell.
Let's construct a demonstration.
$ printf 'echo "foo=$foo"\n' > script
$ bash script
foo=
$ foo=3
$ bash script
foo=
$ export foo
bash script
foo=3
The first time and second time you run script, foo is undefined in its environment because its parent process (the current shell) did not export foo. The third time it is called, the parent adds foo to the script's initial environment because foo was exported.
In response to your comment, the term "environment" has a very precise meaning here. All processes, not just shells, receive an array of strings from its parent on startup, referred to as its environment. There are no particular semantics associated with these strings; it's up to the receiving program how to interpret them.
The shell, for example, ignores strings in its environment that do not have the form name=value, where name is a valid shell identifier. For each such string, the shell defines a shell variable with the given name and value, and marks its export attribute. This is what we mean by an environment variable. You can also "promote" a regular shell variable at any time to an environment variable using the export command, but this doesn't affect the meaning of the variable in the current process.
When any process creates a new process, a copy of its environment is given to the new process. The shell additionally creates name=value strings to pass on from each of its environment variables.
What is the difference between:
prompt$ TSAN_OPTIONS="suppressions=/somewhere/file" ./myprogram
and
prompt$ TSAN_OPTIONS="suppressions=/somewhere/file"
prompt$ ./myprogram
The thread-sanitizer library gives the first case as how to get their library (used within myprogram) to read the file given in options. I read it, and assumed it was supposed to be two separate lines, so ran it as the second case.
The library doesn't use the file in the second case, where the environment variable and the program execution are on separate lines.
What's the difference?
Bonus question: How does the first case even run without error? Shouldn't there have to be a ; or && between them? The answer to this question likely answers my first...
The format VAR=value command sets the variable VAR to have the value value in the environment of the command command. The spec section covering this is the Simple Commands. Specifically:
Otherwise, the variable assignments shall be exported for the execution environment of the command and shall not affect the current execution environment except as a side-effect of the expansions performed in step 4.
The format VAR=value; command sets the shell variable VAR in the current shell and then runs command as a child process. The child process doesn't know anything about the variables set in the shell process.
The mechanism by which a process exports (hint hint) a variable to be seen by child processes is by setting them in its environment before running the child process. The shell built-in which does this is export. This is why you often see export VAR=value and VAR=value; export VAR.
The syntax you are discussing is a short-form for something akin to:
VAR=value
export VAR
command
unset -v VAR
only without using the current process environment at all.
To complement Etan Reisner's helpful answer:
It's important to distinguish between shell variables and environment variables:
Note: The following applies to all POSIX-compatible shells; bash-specific extensions are marked as such.
A shell variable is a shell-specific construct that is limited to the shell that defines it (with the exception of subshells, which get their own copies of the current shell's variables),
whereas an environment variable is inherited by any child process created by the current process (shell), whether that child process is itself a shell or not.
Note that all-uppercase variable names should only be used for environment variables.
Either way, a child process only ever inherits copies of variables, whose modification (by the child) does not affect the parent.
All environment variables are also shell variables (the shell ensures that),
but the inverse is NOT true: shell variables are NOT environment variables, unless explicitly designated or inherited as such - this designation is called exporting.
note that the off-by-default -a shell option (set with set -a, or passed to the shell itself as a command-line option) can be used to auto-export all shell variables.
Thus,
any variables you create implicitly by assignment - e.g., TSAN_OPTIONS="suppressions=/somewhere/file" - are ONLY shell variables, but NOT ALSO environment variables,
EXCEPT - perhaps confusingly - when prepended directly to a command - e.g. TSAN_OPTIONS="suppressions=/somewhere/file" ./myprogram - in which case they are ONLY environment variables, only in effect for THAT COMMAND.
This is what Etan's answer describes.
Shell variables become environment variables as well under the following circumstances:
based on environment variables that the shell itself inherited, such as $HOME
shell variables created explicitly with export varName[=value] or, in bash, also with declare -x varName[=value]
by contrast, in bash, using declare without -x, or using local in a function, creates mere shell variables
shell variables created implicitly while the off-by-default -a shell option is in effect (with limited exceptions)
Once a shell variable is marked as exported - i.e., marked as an environment variable - any subsequent changes to the shell variable update the environment variable as well; e.g.:
export TSAN_OPTIONS # creates shell variable *and* corresponding environment variable
# ...
TSAN_OPTIONS="suppressions=/somewhere/file" # updates *both* the shell and env. var.
export -p prints all environment variables
unset [-v] MYVAR undefines shell variable $MYVAR and also removes it as an environment variable, if applicable.
in bash:
You can "unexport" a given variable without also undefining it as a shell variable with export -n MYVAR - this removes MYVAR from the environment, but retains its current value as a shell variable.
declare -p MYVAR prints variable $MYVAR's current value along with its attributes; if the output starts with declare -x, $MYVAR is exported (is an environment variable)
This question already has answers here:
What does the 'export' command do?
(3 answers)
Closed 7 years ago.
When I customize my environment, I add PATH=$PATH:$My-own-Path in file .bash_profile.
The tutorials tell me I should use this one: export PATH=$PATH:$My-own-Path
So, what is the difference?
To answer your exact specific question, in this particular case, there isn't any difference. Why?
Somewhere in the initialization process, the variable PATH has already been exported. A change in the variable's value which is already exported does not need another export; this is automatic. The processes fired hereafter will get the new value.
export makes the environment variable available to child processes
From man bash:
... The export and declare -x commands allow parameters and functions to be added to and deleted from the environment. If the value of a parameter in the environment is modified, the new value becomes part of the environment, replacing the old.
Also from man bash:
export [-fn] [name[=word]] ...
export -p
The supplied names are marked for automatic export to the environment of subsequently executed commands. If the -f option is given, the names refer to functions. If no names are given, or if the -p option is supplied, a list of names of all exported variables is printed. The -n option causes the export property to be removed from each name. If a variable name is followed by =word, the value of the variable is set to word. export returns an exit status of 0 unless an invalid option is encountered, one of the names is not a valid shell variable name, or -f is supplied with a name that is not a function.
Exported variables are available to other programs. Non-exported variables are not.
Example:
$ myVar=Foo # Create local
$ env | grep '^myVar='
$ export myVar # Export myVar to child process
$ env | grep '^myVar='
Foo
If you want to read more about this, check out export (GNU Bash manual).
Also, please note that non-exported variables will be available to subshells run with (...) and other similar notations:
$ thereVar=Bar
$ (echo $thereVar; echo $myVar; $myVar=testing; echo $myVar)
Bar
Foo
Testing
$echo $myVar
Foo
The subshell cannot affect variables in the parent shell.
For more information on subshells, please reference:
Command Grouping
Command Execution Environment
Every process has an area of memory called the environment block. In the environment block are environment variables. These look like ordinary variables, for example x=42.
In most shells (C shell is an exception) you move an ordinary variable into the environment block using export. That command can also create an environment variable without going through an intermediate stage. If the variable is already in the environment block then export will have no effect.
So why? When a new process is created, the default action is to copy various "core information" from parent to child. These include the current directory, the umask, the file descriptor table, the uid and gid, and the environment block.
Note that the child only gets a copy of the parent's environment block. The variable is not shared and cannot be passed back to the parent (except by using some other inter-process communication mechanism).
You can override this default behaviour using the env program, but this is rarely required.
So, if we set an environment variable in a shell script using export then all our child processes we create, when we call other programs, will get a copy of them. Some variable names are well-known and have a special meaning, and the PATH environment variable is probably the most important of those.
The PATH environment variable is used to find programs on UNIX/Linux. Directories in PATH are searched in left-right order each time we need to load a program. Bash also caches executable paths in a hash (KornShell calls them "tracked aliases").
OS: Solaris
Shell: Bash Shell
Scenario: Input the commands separately: "env", "export" and "set" (without any arguments) and there will be a list of variables and values returned.
My question: What's the difference among the returned values after inputting the three commands?
The env and export commands yield the same information, but not in the same format. And bash's export produces a very radically different output from the output of ksh or (Bourne) shell's version. Note that set and export are shell built-in commands, but env is an external command that has other uses than just listing the content of the environment (though that is one of its uses).
The set command lists the variables you've created. This includes environment variables, regular (non-environment) variables, and function definitions (which we'll ignore here).
Consider:
x1=abc
x2=def; export x2
export x3=ghi
There are two exported variables (x2 and x3), and one regular (non-exported) variable. The set command will list all three; export and env will only list the exported ones.
The output of the env command is mandated by the POSIX standard. This is simply the variable name and value followed by a newline:
name=value
Classically, the Bourne shell simply listed variables the same way for both set and export.
Korn shell encloses values in quotes if the value contains spaces or other characters that need protection, but otherwise uses the name=value notation.
The set command in bash generates assignments with the value protected in quotes. However, the output for export is a declare -x var=value with quote protection. The general idea is presumably that you can use export > file followed by source file to reset the environment variables to the values that were in the environment at the time you did the export.
Summary
Not all shell variables are environment variables.
The set command lists all shell variables and may list functions too.
The export command lists environment variables.
The set and export commands are built into the shell.
The env command with no arguments lists the environment it inherited from the process that executed it.
The set command shows you all of the shell variables defined in your session.
The export command lists a subset (usually) of the ones above. These are created with either export or declare -x : variables which are globally visible - ie., visible to child processes.
The env command is used to to enable porting scripts from account to another account or machine to machine.
env runs a program in a modified or different environment.