I happen to run some commands blindly, in order to get things done.
I started to work with Jenkins recently, and then I had to use this export command to run the Jenkins WAR archive. What does the export command do in general, and why do we need to run this command, while running Jenkins (after the Jenkins home is set)?
export in sh and related shells (such as Bash), marks an environment variable to be exported to child-processes, so that the child inherits them.
export is defined in POSIX:
The shell shall give the export attribute to the variables corresponding to the specified names, which shall cause them to be in the environment of subsequently executed commands. If the name of a variable is followed by = word, then the value of that variable shall be set to word.
I guess you're coming from a Windows background. So I'll contrast them (I'm kind of new to Linux too). I found a user's reply to my comment, to be useful in figuring things out.
In Windows, a variable can be permanent or not. The term environment variable includes a variable set in the cmd shell with the SET command, as well as when the variable is set within the Windows GUI, thus set in the registry, and becoming viewable in new cmd windows.
E.g., the documentation for the set command in Windows "Displays, sets, or removes environment variables. Used without parameters, set displays the current environment settings."
In Linux, set does not display environment variables. It displays shell variables which it doesn't call/refer to as environment variables. Also, Linux doesn't use set to set variables (apart from positional parameters and shell options, which I explain as a note at the end), only to display them and even then only to display shell variables. Windows uses set for setting and displaying, e.g., set a=5, but Linux doesn't.
In Linux, I guess you could make a script that sets variables on bootup, e.g., /etc/profile or /etc/.bashrc, but otherwise, they're not permanent. They're stored in RAM.
There is a distinction in Linux between shell variables, and environment variables. In Linux, shell variables are only in the current shell, and environment variables, are in that shell and all child shells.
You can view shell variables with the set command (though note that, unlike Windows, variables are not set in Linux with the set command).
set -o posix; set (doing that set -o posix once first, helps not display too much unnecessary stuff). So set displays shell variables.
You can view environment variables with the env command.
Shell variables are set with, e.g., just a = 5.
Environment variables are set with export. Export also sets the shell variable.
Here you see shell variable zzz set with zzz = 5, and see it shows when running set, but it doesn't show as an environment variable.
Here we see yyy set with export, so it's an environment variable. And see it shows under both shell variables and environment variables:
$ zzz=5
$ set | grep zzz
zzz=5
$ env | grep zzz
$ export yyy=5
$ set | grep yyy
yyy=5
$ env | grep yyy
yyy=5
$
Other useful QnAs:
https://unix.stackexchange.com/questions/176001/how-can-i-list-all-shell-variables
https://askubuntu.com/questions/26318/environment-variable-vs-shell-variable-whats-the-difference
Note: One point which elaborates a bit and is somewhat corrective to what I've written, is that, in Linux bash, 'set' can be used to set "positional parameters" and "shell options/attributes", and technically both of those are variables, though the man pages might not describe them as such.
But still, as mentioned, set won't set shell variables or environment variables). If you do set asdf then it sets $1 to asdf, and if you do echo $1 you see asdf.
If you do set a=5 it won't set the variable a, equal to 5. It will set the positional parameter $1 equal to the string of "a=5". So if you ever saw set a=5 in Linux it's probably a mistake unless somebody actually wanted that string a=5, in $1.
The other thing that Linux's set can set, is shell options/attributes. If you do set -o you see a list of them. And you can do for example set -o verbose, off, to turn verbose on (by the way, the default happens to be off, but that makes no difference to this). Or you can do set +o verbose to turn verbose off. Windows has no such usage for its set command.
In simple terms, environment variables are set when you open a new shell session. At any time if you change any of the variable values, the shell has no way of picking that change. That means the changes you made become effective in new shell sessions.
The export command, on the other hand, provides the ability to update the current shell session about the change you made to the exported variable. You don't have to wait until new shell session to use the value of the variable you changed.
Related
Here's a simple test script (named "testps")
#!/bin/bash
echo PS4=$PS4
and I have set and exported PS4 like this:
export PS4='$LINENO:'
When I run it, either with ./testps or bash ./testps, the result is:
PS4=+
Looks like the value of "PS4" has been reset.
The only way I have found so far to customize PS4 is to run the script with bash -l after having added export PS4='$LINENO:' in .bashrc.
What have I missed here?
Note also that when using ksh, PS4 is initialized with its value from the environment if any.
This behavior doesn't happen with all users -- a copy of PS4 from the environment is only ignored when running as root, since shell version 4.4.
Quoting from CHANGES in the bash source:
g. Shells running as root no longer inherit PS4 from the environment, closing a security hole involving PS4 expansion performing command substitution.
This was done because environment variables are passed to setuid executables, and some setuid executables (unwisely) use system(), popen(), etc. to invoke a shell. While ld.so ignores LD_LIBRARY_PATH, LD_PRELOAD and similar when running in setuid, bash historically did not do so with environment variables that could cause arbitrary execution.
As described in https://www.openwall.com/lists/oss-security/2016/09/26/9, this issue could be exploited as follows:
env -i SHELLOPTS=xtrace PS4='$(id)' ./test
...would run id, instead of printing $(id) as part of xtrace logs, even if invocation of ./test crosses a privilege boundary.
Personally, I set PS4 inside my scripts -- it's ignored unless they're run with set -x, after all, so why not establish a meaningful value regardless?
If you want to force it to happen with fewer side effects than you get from bash -l or bash -i, set BASH_ENV to have the name of a file that can be sourced to perform your desired initialization.
What is the difference between:
prompt$ TSAN_OPTIONS="suppressions=/somewhere/file" ./myprogram
and
prompt$ TSAN_OPTIONS="suppressions=/somewhere/file"
prompt$ ./myprogram
The thread-sanitizer library gives the first case as how to get their library (used within myprogram) to read the file given in options. I read it, and assumed it was supposed to be two separate lines, so ran it as the second case.
The library doesn't use the file in the second case, where the environment variable and the program execution are on separate lines.
What's the difference?
Bonus question: How does the first case even run without error? Shouldn't there have to be a ; or && between them? The answer to this question likely answers my first...
The format VAR=value command sets the variable VAR to have the value value in the environment of the command command. The spec section covering this is the Simple Commands. Specifically:
Otherwise, the variable assignments shall be exported for the execution environment of the command and shall not affect the current execution environment except as a side-effect of the expansions performed in step 4.
The format VAR=value; command sets the shell variable VAR in the current shell and then runs command as a child process. The child process doesn't know anything about the variables set in the shell process.
The mechanism by which a process exports (hint hint) a variable to be seen by child processes is by setting them in its environment before running the child process. The shell built-in which does this is export. This is why you often see export VAR=value and VAR=value; export VAR.
The syntax you are discussing is a short-form for something akin to:
VAR=value
export VAR
command
unset -v VAR
only without using the current process environment at all.
To complement Etan Reisner's helpful answer:
It's important to distinguish between shell variables and environment variables:
Note: The following applies to all POSIX-compatible shells; bash-specific extensions are marked as such.
A shell variable is a shell-specific construct that is limited to the shell that defines it (with the exception of subshells, which get their own copies of the current shell's variables),
whereas an environment variable is inherited by any child process created by the current process (shell), whether that child process is itself a shell or not.
Note that all-uppercase variable names should only be used for environment variables.
Either way, a child process only ever inherits copies of variables, whose modification (by the child) does not affect the parent.
All environment variables are also shell variables (the shell ensures that),
but the inverse is NOT true: shell variables are NOT environment variables, unless explicitly designated or inherited as such - this designation is called exporting.
note that the off-by-default -a shell option (set with set -a, or passed to the shell itself as a command-line option) can be used to auto-export all shell variables.
Thus,
any variables you create implicitly by assignment - e.g., TSAN_OPTIONS="suppressions=/somewhere/file" - are ONLY shell variables, but NOT ALSO environment variables,
EXCEPT - perhaps confusingly - when prepended directly to a command - e.g. TSAN_OPTIONS="suppressions=/somewhere/file" ./myprogram - in which case they are ONLY environment variables, only in effect for THAT COMMAND.
This is what Etan's answer describes.
Shell variables become environment variables as well under the following circumstances:
based on environment variables that the shell itself inherited, such as $HOME
shell variables created explicitly with export varName[=value] or, in bash, also with declare -x varName[=value]
by contrast, in bash, using declare without -x, or using local in a function, creates mere shell variables
shell variables created implicitly while the off-by-default -a shell option is in effect (with limited exceptions)
Once a shell variable is marked as exported - i.e., marked as an environment variable - any subsequent changes to the shell variable update the environment variable as well; e.g.:
export TSAN_OPTIONS # creates shell variable *and* corresponding environment variable
# ...
TSAN_OPTIONS="suppressions=/somewhere/file" # updates *both* the shell and env. var.
export -p prints all environment variables
unset [-v] MYVAR undefines shell variable $MYVAR and also removes it as an environment variable, if applicable.
in bash:
You can "unexport" a given variable without also undefining it as a shell variable with export -n MYVAR - this removes MYVAR from the environment, but retains its current value as a shell variable.
declare -p MYVAR prints variable $MYVAR's current value along with its attributes; if the output starts with declare -x, $MYVAR is exported (is an environment variable)
This question already has answers here:
What does the 'export' command do?
(3 answers)
Closed 7 years ago.
When I customize my environment, I add PATH=$PATH:$My-own-Path in file .bash_profile.
The tutorials tell me I should use this one: export PATH=$PATH:$My-own-Path
So, what is the difference?
To answer your exact specific question, in this particular case, there isn't any difference. Why?
Somewhere in the initialization process, the variable PATH has already been exported. A change in the variable's value which is already exported does not need another export; this is automatic. The processes fired hereafter will get the new value.
export makes the environment variable available to child processes
From man bash:
... The export and declare -x commands allow parameters and functions to be added to and deleted from the environment. If the value of a parameter in the environment is modified, the new value becomes part of the environment, replacing the old.
Also from man bash:
export [-fn] [name[=word]] ...
export -p
The supplied names are marked for automatic export to the environment of subsequently executed commands. If the -f option is given, the names refer to functions. If no names are given, or if the -p option is supplied, a list of names of all exported variables is printed. The -n option causes the export property to be removed from each name. If a variable name is followed by =word, the value of the variable is set to word. export returns an exit status of 0 unless an invalid option is encountered, one of the names is not a valid shell variable name, or -f is supplied with a name that is not a function.
Exported variables are available to other programs. Non-exported variables are not.
Example:
$ myVar=Foo # Create local
$ env | grep '^myVar='
$ export myVar # Export myVar to child process
$ env | grep '^myVar='
Foo
If you want to read more about this, check out export (GNU Bash manual).
Also, please note that non-exported variables will be available to subshells run with (...) and other similar notations:
$ thereVar=Bar
$ (echo $thereVar; echo $myVar; $myVar=testing; echo $myVar)
Bar
Foo
Testing
$echo $myVar
Foo
The subshell cannot affect variables in the parent shell.
For more information on subshells, please reference:
Command Grouping
Command Execution Environment
Every process has an area of memory called the environment block. In the environment block are environment variables. These look like ordinary variables, for example x=42.
In most shells (C shell is an exception) you move an ordinary variable into the environment block using export. That command can also create an environment variable without going through an intermediate stage. If the variable is already in the environment block then export will have no effect.
So why? When a new process is created, the default action is to copy various "core information" from parent to child. These include the current directory, the umask, the file descriptor table, the uid and gid, and the environment block.
Note that the child only gets a copy of the parent's environment block. The variable is not shared and cannot be passed back to the parent (except by using some other inter-process communication mechanism).
You can override this default behaviour using the env program, but this is rarely required.
So, if we set an environment variable in a shell script using export then all our child processes we create, when we call other programs, will get a copy of them. Some variable names are well-known and have a special meaning, and the PATH environment variable is probably the most important of those.
The PATH environment variable is used to find programs on UNIX/Linux. Directories in PATH are searched in left-right order each time we need to load a program. Bash also caches executable paths in a hash (KornShell calls them "tracked aliases").
Can someone please tell me what's the correct way to set a bunch of environment variables in the fish shell?
In my ~/.config/fish/config.fish file, I have a function to setup my environment variables like so:
function setTESTENV
set -x BROKER_IP '10.14.16.216'
set -x USERNAME 'foo'
set -x USERPASS 'bar'
end
When I type from the command prompt setTESTENV and do a env in the command line, I don't see this information.
Use Universal Variables.
If the variable has to be shared between all the current user Fish instances on the current computer and preserved across restarts of the shell you can set them using -U or --universal. For example:
set -Ux FOO bar
Using set with -g or --global doesn't set the variable persistently between shell instances.
Note:
Do not append to universal variables in config.fish file, because these variables will then get longer with each new shell instance. Instead, simply run set -Ux once at the command line.
Universal variables will be stored in the file ~/.config/fish/fish_variables as of Fish 3.0. In prior releases, it was ~/.config/fish/fishd.MACHINE_ID, where MACHINE_ID was typically the MAC address.
The variables you are declaring are keep in a local scope inside your function.
Use:
set -g -x
Here "g" is for global.
another option is to run:
export (cat env_file.txt |xargs -L 1)
where env_file.txt contains rows of the format VAR=VALUE
this has the benefit of keeping the variables in a format supported by other shells and tools
Environment Variables in Fish
I would like to add that, while #JosEduSol's answer is not incorrect and does help solve the OP problem, -g is only setting the scope to be global, while -x is causing the specified environment variable to be exported to child processes.
The reason the above fails, is because #cfpete is setting the env vars inside a function and the default scope will be local to that function.
OS: Solaris
Shell: Bash Shell
Scenario: Input the commands separately: "env", "export" and "set" (without any arguments) and there will be a list of variables and values returned.
My question: What's the difference among the returned values after inputting the three commands?
The env and export commands yield the same information, but not in the same format. And bash's export produces a very radically different output from the output of ksh or (Bourne) shell's version. Note that set and export are shell built-in commands, but env is an external command that has other uses than just listing the content of the environment (though that is one of its uses).
The set command lists the variables you've created. This includes environment variables, regular (non-environment) variables, and function definitions (which we'll ignore here).
Consider:
x1=abc
x2=def; export x2
export x3=ghi
There are two exported variables (x2 and x3), and one regular (non-exported) variable. The set command will list all three; export and env will only list the exported ones.
The output of the env command is mandated by the POSIX standard. This is simply the variable name and value followed by a newline:
name=value
Classically, the Bourne shell simply listed variables the same way for both set and export.
Korn shell encloses values in quotes if the value contains spaces or other characters that need protection, but otherwise uses the name=value notation.
The set command in bash generates assignments with the value protected in quotes. However, the output for export is a declare -x var=value with quote protection. The general idea is presumably that you can use export > file followed by source file to reset the environment variables to the values that were in the environment at the time you did the export.
Summary
Not all shell variables are environment variables.
The set command lists all shell variables and may list functions too.
The export command lists environment variables.
The set and export commands are built into the shell.
The env command with no arguments lists the environment it inherited from the process that executed it.
The set command shows you all of the shell variables defined in your session.
The export command lists a subset (usually) of the ones above. These are created with either export or declare -x : variables which are globally visible - ie., visible to child processes.
The env command is used to to enable porting scripts from account to another account or machine to machine.
env runs a program in a modified or different environment.