This question already has answers here:
Can I export a variable to the environment from a Bash script without sourcing it?
(13 answers)
Closed 3 years ago.
The community reviewed whether to reopen this question last year and left it closed:
Original close reason(s) were not resolved
I'm trying to write a shell script that, when run, will set some environment variables that will stay set in the caller's shell.
setenv FOO foo
in csh/tcsh, or
export FOO=foo
in sh/bash only set it during the script's execution.
I already know that
source myscript
will run the commands of the script rather than launching a new shell, and that can result in setting the "caller's" environment.
But here's the rub:
I want this script to be callable from either bash or csh. In other words, I want users of either shell to be able to run my script and have their shell's environment changed. So 'source' won't work for me, since a user running csh can't source a bash script, and a user running bash can't source a csh script.
Is there any reasonable solution that doesn't involve having to write and maintain TWO versions on the script?
Use the "dot space script" calling syntax. For example, here's how to do it using the full path to a script:
. /path/to/set_env_vars.sh
And here's how to do it if you're in the same directory as the script:
. set_env_vars.sh
These execute the script under the current shell instead of loading another one (which is what would happen if you did ./set_env_vars.sh). Because it runs in the same shell, the environmental variables you set will be available when it exits.
This is the same thing as calling source set_env_vars.sh, but it's shorter to type and might work in some places where source doesn't.
Your shell process has a copy of the parent's environment and no access to the parent process's environment whatsoever. When your shell process terminates any changes you've made to its environment are lost. Sourcing a script file is the most commonly used method for configuring a shell environment, you may just want to bite the bullet and maintain one for each of the two flavors of shell.
You're not going to be able to modify the caller's shell because it's in a different process context. When child processes inherit your shell's variables, they're
inheriting copies themselves.
One thing you can do is to write a script that emits the correct commands for tcsh
or sh based how it's invoked. If you're script is "setit" then do:
ln -s setit setit-sh
and
ln -s setit setit-csh
Now either directly or in an alias, you do this from sh
eval `setit-sh`
or this from csh
eval `setit-csh`
setit uses $0 to determine its output style.
This is reminescent of how people use to get the TERM environment variable set.
The advantage here is that setit is just written in whichever shell you like as in:
#!/bin/bash
arg0=$0
arg0=${arg0##*/}
for nv in \
NAME1=VALUE1 \
NAME2=VALUE2
do
if [ x$arg0 = xsetit-sh ]; then
echo 'export '$nv' ;'
elif [ x$arg0 = xsetit-csh ]; then
echo 'setenv '${nv%%=*}' '${nv##*=}' ;'
fi
done
with the symbolic links given above, and the eval of the backquoted expression, this has the desired result.
To simplify invocation for csh, tcsh, or similar shells:
alias dosetit 'eval `setit-csh`'
or for sh, bash, and the like:
alias dosetit='eval `setit-sh`'
One nice thing about this is that you only have to maintain the list in one place.
In theory you could even stick the list in a file and put cat nvpairfilename between "in" and "do".
This is pretty much how login shell terminal settings used to be done: a script would output statments to be executed in the login shell. An alias would generally be used to make invocation simple, as in "tset vt100". As mentioned in another answer, there is also similar functionality in the INN UseNet news server.
In my .bash_profile I have :
# No Proxy
function noproxy
{
/usr/local/sbin/noproxy #turn off proxy server
unset http_proxy HTTP_PROXY https_proxy HTTPs_PROXY
}
# Proxy
function setproxy
{
sh /usr/local/sbin/proxyon #turn on proxy server
http_proxy=http://127.0.0.1:8118/
HTTP_PROXY=$http_proxy
https_proxy=$http_proxy
HTTPS_PROXY=$https_proxy
export http_proxy https_proxy HTTP_PROXY HTTPS_PROXY
}
So when I want to disable the proxy,
the function(s) run in the login shell and sets the variables
as expected and wanted.
It's "kind of" possible through using gdb and setenv(3), although I have a hard time recommending actually doing this. (Additionally, i.e. the most recent ubuntu won't actually let you do this without telling the kernel to be more permissive about ptrace, and the same may go for other distros as well).
$ cat setfoo
#! /bin/bash
gdb /proc/${PPID}/exe ${PPID} <<END >/dev/null
call setenv("foo", "bar", 0)
END
$ echo $foo
$ ./setfoo
$ echo $foo
bar
This works — it isn't what I'd use, but it 'works'. Let's create a script teredo to set the environment variable TEREDO_WORMS:
#!/bin/ksh
export TEREDO_WORMS=ukelele
exec $SHELL -i
It will be interpreted by the Korn shell, exports the environment variable, and then replaces itself with a new interactive shell.
Before running this script, we have SHELL set in the environment to the C shell, and the environment variable TEREDO_WORMS is not set:
% env | grep SHELL
SHELL=/bin/csh
% env | grep TEREDO
%
When the script is run, you are in a new shell, another interactive C shell, but the environment variable is set:
% teredo
% env | grep TEREDO
TEREDO_WORMS=ukelele
%
When you exit from this shell, the original shell takes over:
% exit
% env | grep TEREDO
%
The environment variable is not set in the original shell's environment. If you use exec teredo to run the command, then the original interactive shell is replaced by the Korn shell that sets the environment, and then that in turn is replaced by a new interactive C shell:
% exec teredo
% env | grep TEREDO
TEREDO_WORMS=ukelele
%
If you type exit (or Control-D), then your shell exits, probably logging you out of that window, or taking you back to the previous level of shell from where the experiments started.
The same mechanism works for Bash or Korn shell. You may find that the prompt after the exit commands appears in funny places.
Note the discussion in the comments. This is not a solution I would recommend, but it does achieve the stated purpose of a single script to set the environment that works with all shells (that accept the -i option to make an interactive shell). You could also add "$#" after the option to relay any other arguments, which might then make the shell usable as a general 'set environment and execute command' tool. You might want to omit the -i if there are other arguments, leading to:
#!/bin/ksh
export TEREDO_WORMS=ukelele
exec $SHELL "${#-'-i'}"
The "${#-'-i'}" bit means 'if the argument list contains at least one argument, use the original argument list; otherwise, substitute -i for the non-existent arguments'.
You should use modules, see http://modules.sourceforge.net/
EDIT: The modules package has not been updated since 2012 but still works ok for the basics. All the new features, bells and whistles happen in lmod this day (which I like it more): https://www.tacc.utexas.edu/research-development/tacc-projects/lmod
Another workaround that I don't see mentioned is to write the variable value to a file.
I ran into a very similar issue where I wanted to be able to run the last set test (instead of all my tests). My first plan was to write one command for setting the env variable TESTCASE, and then have another command that would use this to run the test. Needless to say that I had the same exact issue as you did.
But then I came up with this simple hack:
First command ( testset ):
#!/bin/bash
if [ $# -eq 1 ]
then
echo $1 > ~/.TESTCASE
echo "TESTCASE has been set to: $1"
else
echo "Come again?"
fi
Second command (testrun ):
#!/bin/bash
TESTCASE=$(cat ~/.TESTCASE)
drush test-run $TESTCASE
You can instruct the child process to print its environment variables (by calling "env"), then loop over the printed environment variables in the parent process and call "export" on those variables.
The following code is based on Capturing output of find . -print0 into a bash array
If the parent shell is the bash, you can use
while IFS= read -r -d $'\0' line; do
export "$line"
done < <(bash -s <<< 'export VARNAME=something; env -0')
echo $VARNAME
If the parent shell is the dash, then read does not provide the -d flag and the code gets more complicated
TMPDIR=$(mktemp -d)
mkfifo $TMPDIR/fifo
(bash -s << "EOF"
export VARNAME=something
while IFS= read -r -d $'\0' line; do
echo $(printf '%q' "$line")
done < <(env -0)
EOF
) > $TMPDIR/fifo &
while read -r line; do export "$(eval echo $line)"; done < $TMPDIR/fifo
rm -r $TMPDIR
echo $VARNAME
Under OS X bash you can do the following:
Create the bash script file to unset the variable
#!/bin/bash
unset http_proxy
Make the file executable
sudo chmod 744 unsetvar
Create alias
alias unsetvar='source /your/path/to/the/script/unsetvar'
It should be ready to use so long you have the folder containing your script file appended to the path.
It's not what I would call outstanding, but this also works if you need to call the script from the shell anyway. It's not a good solution, but for a single static environment variable, it works well enough.
1.) Create a script with a condition that exits either 0 (Successful) or 1 (Not successful)
if [[ $foo == "True" ]]; then
exit 0
else
exit 1
2.) Create an alias that is dependent on the exit code.
alias='myscript.sh && export MyVariable'
You call the alias, which calls the script, which evaluates the condition, which is required to exit zero via the '&&' in order to set the environment variable in the parent shell.
This is flotsam, but it can be useful in a pinch.
You can invoke another one Bash with the different bash_profile.
Also, you can create special bash_profile for using in multi-bashprofile environment.
Remember that you can use functions inside of bashprofile, and that functions will be avialable globally.
for example, "function user { export USER_NAME $1 }" can set variable in runtime, for example: user olegchir && env | grep olegchir
Another option is to use "Environment Modules" (http://modules.sourceforge.net/). This unfortunately introduces a third language into the mix. You define the environment with the language of Tcl, but there are a few handy commands for typical modifications (prepend vs. append vs set). You will also need to have environment modules installed. You can then use module load *XXX* to name the environment you want. The module command is basically a fancy alias for the eval mechanism described above by Thomas Kammeyer. The main advantage here is that you can maintain the environment in one language and rely on "Environment Modules" to translate it to sh, ksh, bash, csh, tcsh, zsh, python (?!?!!), etc.
I created a solution using pipes, eval and signal.
parent() {
if [ -z "$G_EVAL_FD" ]; then
die 1 "Rode primeiro parent_setup no processo pai"
fi
if [ $(ppid) = "$$" ]; then
"$#"
else
kill -SIGUSR1 $$
echo "$#">&$G_EVAL_FD
fi
}
parent_setup() {
G_EVAL_FD=99
tempfile=$(mktemp -u)
mkfifo "$tempfile"
eval "exec $G_EVAL_FD<>'$tempfile'"
rm -f "$tempfile"
trap "read CMD <&$G_EVAL_FD; eval \"\$CMD\"" USR1
}
parent_setup #on parent shell context
( A=1 ); echo $A # prints nothing
( parent A=1 ); echo $A # prints 1
It might work with any command.
I don't see any answer documenting how to work around this problem with cooperating processes. A common pattern with things like ssh-agent is to have the child process print an expression which the parent can eval.
bash$ eval $(shh-agent)
For example, ssh-agent has options to select Csh or Bourne-compatible output syntax.
bash$ ssh-agent
SSH2_AUTH_SOCK=/tmp/ssh-era/ssh2-10690-agent; export SSH2_AUTH_SOCK;
SSH2_AGENT_PID=10691; export SSH2_AGENT_PID;
echo Agent pid 10691;
(This causes the agent to start running, but doesn't allow you to actually use it, unless you now copy-paste this output to your shell prompt.) Compare:
bash$ ssh-agent -c
setenv SSH2_AUTH_SOCK /tmp/ssh-era/ssh2-10751-agent;
setenv SSH2_AGENT_PID 10752;
echo Agent pid 10752;
(As you can see, csh and tcsh uses setenv to set varibles.)
Your own program can do this, too.
bash$ foo=$(makefoo)
Your makefoo script would simply calculate and print the value, and let the caller do whatever they want with it -- assigning it to a variable is a common use case, but probably not something you want to hard-code into the tool which produces the value.
Technically, that is correct -- only 'eval' doesn't fork another shell. However, from the point of view of the application you're trying to run in the modified environment, the difference is nil: the child inherits the environment of its parent, so the (modified) environment is conveyed to all descending processes.
Ipso facto, the changed environment variable 'sticks' -- as long as you are running under the parent program/shell.
If it is absolutely necessary for the environment variable to remain after the parent (Perl or shell) has exited, it is necessary for the parent shell to do the heavy lifting. One method I've seen in the documentation is for the current script to spawn an executable file with the necessary 'export' language, and then trick the parent shell into executing it -- always being cognizant of the fact that you need to preface the command with 'source' if you're trying to leave a non-volatile version of the modified environment behind. A Kluge at best.
The second method is to modify the script that initiates the shell environment (.bashrc or whatever) to contain the modified parameter. This can be dangerous -- if you hose up the initialization script it may make your shell unavailable the next time it tries to launch. There are plenty of tools for modifying the current shell; by affixing the necessary tweaks to the 'launcher' you effectively push those changes forward as well.
Generally not a good idea; if you only need the environment changes for a particular application suite, you'll have to go back and return the shell launch script to its pristine state (using vi or whatever) afterwards.
In short, there are no good (and easy) methods. Presumably this was made difficult to ensure the security of the system was not irrevocably compromised.
The short answer is no, you cannot alter the environment of the parent process, but it seems like what you want is an environment with custom environment variables and the shell that the user has chosen.
So why not simply something like
#!/usr/bin/env bash
FOO=foo $SHELL
Then when you are done with the environment, just exit.
You could always use aliases
alias your_env='source ~/scripts/your_env.sh'
I did this many years ago. If I rememeber correctly, I included an alias in each of .bashrc and .cshrc, with parameters, aliasing the respective forms of setting the environment to a common form.
Then the script that you will source in any of the two shells has a command with that last form, that is suitable aliased in each shell.
If I find the concrete aliases, I will post them.
Other than writings conditionals depending on what $SHELL/$TERM is set to, no. What's wrong with using Perl? It's pretty ubiquitous (I can't think of a single UNIX variant that doesn't have it), and it'll spare you the trouble.
my project, which uses mapreduce without hadoop, is composed of two files:
bash.sh and mapreduce.py.
I would like to use environment variables to communicate the information between bash.sh and mapreduce.py.
Within bash.sh I use export myvariable and on mapreduce.py, I use os.environ ['myvariable'].
I would like to edit myvariable within mapreduce.py and print the result via bash.sh. I tried to execute this istruction: os.environ ['myvariable'] = 'hello', but on bash.sh 'myvariable' is empty. Do you have any suggestions?
You can't do it from python, but some clever bash tricks can do
something similar. The basic reasoning is this: environment variables
exist in a per-process memory space. When a new process is created
with fork() it inherits its parent's environment variables. When you
set an environment variable in your shell (e.g. bash) like this:
/why-cant-environmental-variables-set-in-python-persist
So you can only make available to Bash script if the bash script is called inside python process space. A simple example can be
bash script
#!/bin/bash
echo "var from python is $myvariable"
python script
import os
os.environ ['myvariable'] = 'hello'
print(os.environ['myvariable'])
# all environment varaibles will be availbe to bash script in this case
os.system('sh ./ab.sh')
This is the way that you can try. Otherwise, no way to set it and make it available to bash script.
Setting an environment variable sets it only for the current process
and any child processes it launches. So using os.system will set it
only for the shell that is running to execute the command you
provided. When that command finishes, the shell goes away, and so does
the environment variable. Setting it using os.putenv or os.environ has
a similar effect; the environment variables are set for the Python
process and any children of it.
ENV via python
You can also try vice versa as you mention in question
Here is sequence
First set in bash script
call python script from bash ( based on argument to avoid loop)
update ENV in python
call bash again from python, if you call outside it will vanish.
bash script
#!/bin/bash
export myvariable="hellobash"
echo "myvariable form bash $myvariable"
if [ ! -z $1 ]; then
./py.py
else
echo "myvariable after updated from python $myvariable"
fi
Call bash script outside from python with the argument like myscript.sh bash, without argument in python myscript.sh
#!/usr/bin/python
import os
print("myvar form python",os.environ['myvariable'])
os.environ ['myvariable'] = 'hello'
print("myvar form python after update",os.environ['myvariable'])
os.system('sh ./ab.sh')
I wrote simple command that lets me run the last N commands from terminal history. It looks like this: $> r 3 which will replay the last 3 commands.
I have the following alias in my bash profile:
alias r="history -w; runlast $1"
And then the following simple perl script for the runlast command:
#!/usr/bin/env perl
use strict;
use warnings;
my $lines = $ARGV[0] || exit;
my #last_commands = split /\n/,
`bash -ic 'set -o history; history' | tail -$lines`;
#last_commands =
grep { $_ !~ /(^r |^history |^rm )/ }
map { local $_ = $_; s/^\s+\d+\s+//; $_ }
#last_commands;
foreach my $cmd (#last_commands) {
system("$cmd");
}
This works but my bash profile has aliases and other features (e.g. color output) I want the perl script to have access to. How do I load the bash profile for perl so it runs the bash commands with my bash profile? I read somewhere that if you "source the bash profile" for perl you can get it to work. So I tried adding source ~/.bash_profile; to my r command alias but that didn't have an effect. I'm not sure if I was doing that correctly, though.
The system forks a process in which it runs a shell, which is non-login and non-interactive; so no initialization is done and you get no aliases. Also note that the shell used is /bin/sh, which is generally a link to another shell. This is often bash but not always, so run bash explicitly.
To circumvent this you need to source the file with aliases, but as bash man page says
Aliases are not expanded when the shell is not interactive, unless the expand_aliases shell option is set using shopt (see the description of shopt under SHELL BUILTIN COMMANDS below).
Thus you need shopt -s expand_aliases, as mentioned. But there is another screw: on that same physical line aliases are not yet available; so it won't work like this in a one-liner.
I'd also recommend to put aliases in .bashrc, or in a separate file that is sourced.
Solutions
Add shopt -s expand_aliases to your ~/.bashrc, and before the aliases are defined (or the file with them sourced), and run bash as a login shell
system('/bin/bash', '-cl', 'source ~/.bashrc; command');
where -l is short for --login.
In my tests the source ~/.bashrc wasn't needed; however, the man page says
When bash is invoked as an interactive login shell, or as a non-interactive shell with the --login option, it first reads and executes commands from the file /etc/profile, if that file exists. After reading that file, it looks for ~/.bash_profile, ~/.bash_login, and ~/.profile, in that order, and reads and executes commands from the first one that exists and is readable.
and goes on to specify that ~/.bashrc is read when an interactive shel that is not login runs. So I added explicit sourcing.
In my tests sourcing .bashrc (with shopt added) while not running as a login shell didn't work, and I am not sure why.
This is a little heavy-handed. Also, initialization may be undesirable to run from a script.
Source ~/.bashrc and issue shopt command, and then a newline before the command
system('/bin/bash', '-c',
'source ~/.bashrc; shopt -s expand_aliases\ncommand');
Really. It works.
Finally, is this necessary? It asks for trouble, and there is probably a better design.
Other comments
The backticks (qx) is context-aware. If it's used in list context – its return assigned to an array, for example – then the command's output is returned as a list of lines. When you use it as the argument for split then it is in the scalar context though, when all output is returned in one string. Just drop split
my #last_commands = `bash -ic 'set -o history; history $lines`;
where I also use history N to get last N lines. In this case the newlines stay.
history N returns last N lines of history so there is no need to pipe to last
Regex substitution in a map can be done without changing the original
map { s/^\s+\d+\s+//r } #last_commands;
With /r modifier the s/// operator returns the new string, not changing the original. This "non-destructive substitution" has been available since v5.14
No need to explicitly use $_ in the last grep, and no need for parenthesis in regex
grep { not /^r |^history |^rm ?/ } ...
or
grep { not /^(?:r|history|rm)[ ]?/ } ...
where parens are now needed, but as it is only for grouping the ?: makes it not capture the match. I use [ ] to emphasize that that space is intended; this is not necessary.
I also added ? to make space optional since history (and r?) may have no space.
The proper solution is to have your Perl script just print the commands, and make your current interactive shell eval the string printed from your history. (I would probably get rid of Perl entirely but that's beside the point here.)
If the commands get evaluated in the current shell, you avoid many contextual problems which would be very hard or even intractable with system() or generally anything involving a new process. For example, a subprocess cannot have access to non-exported variables in the current shell. var="foo", echo "$var"; r 1 is going to be very hard to solve correctly with your current approach. Using the current interactive shell will also naturally and easily solve the problems you were having with trying to get a noninteractive subshell act like an interactive one.
Aliases suck anyway, so let's redefine r as a function:
r(){
history -w
eval $(printlast "$1")
}
... where refactoring runlast into a different script printlast is a trivial additional requirement. Or maybe just turn it into a (much simpler!) shell function:
printlast () {
history "$1" |
perl -ne 's/^\s*\d+\s+\*?//; print unless m/^(history|rm?)($|\s)'
}
With this, you can also get rid of history -w from the r definition.
Notice how we are using Perl where it is useful; but the main functionality makes sense to keep in the shell when you're dealing with the shell.
You can't source in a Bash script into a Perl script. The bash_profile has to be sourced in by the shell that executes the command. When Perl runs system, it forks a new shell each time.
You have to source in the bash_profile for each command that you run through system:
system('source ~/.bash_profile; ' + $cmd);
One more thing, system invokes a non-interactive shell. So, your Bash aliases defined in .bash_profile won't work unless you invoke:
shopt -s expand_aliases
inside that script
I have a number of Bash and Perl scripts which are unrelated in functionality, but are related in that they work within the same project.
The fact that they work in the same project means that I commonly specify the same directories, the same project specific commands, the same keywords at the top of every script.
Currently, this has not bitten me, but I understand that it would be easier to have all of these values in one place, then if something changes I can change a value once and have the various scripts pick up on those changes.
The question is - how is best to declare these values? A single Perl script that is 'required' in each script would require less changes to the Perl scripts, though doesn't provide a solution to the Bash script. A configuration file using a "key=value" format would perhaps be more universal, but requires each script to parse the configuration and has the potential to introduce issues. Is there a better alternative? Using environmental variables? Or a Bash specific way that Perl can easily execute and interpret?
When you run a shell script, it's done in a sub-shell so it cannot affect the parent shell's environment. So when you declare a variable as key=value its scope is limited to the sub-shell context. You want to source the script by doing:
. ./myscript.sh
This executes it in the context of the current shell, not as a sub shell.
From the bash man page:
. filename [arguments]
source filename [arguments]
Read and execute commands from filename in the current shell environment and return the exit status of the last command executed from filename.
If filename does not contain a slash, file names in PATH are used to find the directory containing filename.
Also you can use the export command to create a global environment variable. export governs which variables will be available to new processes, so if you say
FOO=1
export BAR=2
./myscript2.sh
then $BAR will be available in the environment of myscript2.sh, but $FOO will not.
Define environments variables :
user level : in your ~/.profile or ~/.bash_profile or ~/.bash_login or ~/.bashrc
system level : in /etc/profile or /etc/bash.bashrc or /etc/environment
For example add tow lines foreach variable :
FOO=myvalue
export FOO
To read this variable in bash script :
#! /bin/bash
echo $FOO
in perl script :
#! /bin/perl
print $ENV{'FOO'};
You could also source another file, so you do not create extra env variables, that may lead to unexpected behaviours.
source_of_truth.sh:
FOO="bar"
scritp1.sh
#!/usr/bin/env bash
source source_of_truth.sh
echo ${FOO}
# ... doing something
scritp2.sh
#!/usr/bin/env bash
source source_of_truth.sh
echo ${FOO}
# ... doing something else