If I want to inherit environment variables to child processes, i do something like:
export MYVAR=tork
Assume I have a a file site.conf containing assignments of values (that can contain spaces) to variables:
EMAIL="dev#example.com"
FULLNAME="Master Yedi"
FOO=bar
Now I would like to process this file whenever I open a new shell (e.g. with some code in ~/.bashrc or ~/.profile), so that any processes started from within that newly opened shell will inherit the assignments via environmental variables.
The obvious solution would be to prefix each line in site.conf with an export and just source the file. However I cannot do this since the file is also read (directly) by some other applications, so the format is fixed.
I tried something like
cat site.conf | while read assignment
do
export "${assignment}"
done
But this doesn't work, for various reasons (the most important being that export is executed in a subshell, so the variable will never be exported to the children of the calling shell).
Is there a way to programmatically export unknown variables in bash?
Run set -a before sourcing the file. This marks all new and modified variables that follow for export automatically.
set -a
source site.conf
set +a # Require export again, if desired.
The problem you observed is that the pipe executes the export in a subshell. You can avoid that simply by using input redirection instead of a pipe.
while read assignment; do
export "$assignment"
done < site.conf
This won't work, however, if (unlikely though it is) you have multiple assignments on one line, such as
EMAIL="dev#example.com" FULLNAME="Master Yedi"
Related
I have a set of environment variables that need to be set on the basis of the arguments specified in the shell script.
But the problem is that those variables are already defined in the bash profile
FOR EXAMPLE:
bash_profile has a variable called "KARAN":
export KARAN=/config/1
Now on running the shell script, this is what it should do:
export KARAN=/config/2 (Changed the bash profile's KARAN value to 2)
Your question is not clear. If your script needs to set the env var to a specific value just do so using export VAR=val. What I think you're asking is how to have a script modify the environment of the current shell. And that is impossible without the cooperation of both shells. That is because environment vars are inherited by child processes. But a child process cannot directly modify the environment of its parent process (or some other random process for that matter). To do so the two processes must coordinate the exchange of data. This is typically done by using the source command if the child process is a shell script. Or by having the child process write a series of export statements to stdout and having the parent shell capture and execute those statements. For example, let's say I have a script named set_env that looks like this
#!/bin/sh
echo export KARAN=/config_2
echo export VAR2=val2
The current shell would then do
eval $(set_env)
Note, however, eval is dangerous. I prefer to do this which is slightly safer:
set_env | source /dev/stdin
That, however, only works in shells like ksh and zsh. Due to how bash handles pipelines the source is actually executed in a child shell and therefore the vars won't be set in the current shell.
You can create a new Profile with all the new definitions. and then call the line below on top of your shell script. Similarly, you can create as many profiles as you want and use it.
source bash_profile_new
In vim I can access my bash environment variables such as $PWD and $PATH. I would like to know how to access my temporary shell variables in vim too.
For example, suppose I was in my terminal and define a variable foo="bar". Then I enter vim and try to access this variable with the following command :!echo $foo, but it does not recognize this variable. From my understanding, vim starts a new shell each time a bash command is invoked and then closes it immediately after. Is there a way to use the same shell in vim that my local variable foo was defined in?
No, you can't interact with the parent shell from a subprocess it spawned (without that shell's active participation, which isn't reasonably/practically available in the scenario at hand) -- but you can export your variables to make them accessible to new shells started in child processes.
Running
set -a
...will make any variable defined going forward be automatically exported to the environment, even without an explicit export command.
Since (unlike the C system() function) vim's system() honors the SHELL environment variable, if SHELL=/bin/bash (or :set shell=/bin/bash has been run in vim), you can also invoke exported functions from vim. That is, if you define the function and export it as follows:
foo() { echo "bar"; }
export -f foo
...then you can invoke it with !foo from inside vim.
Even then, however, this is running in a new, transient shell instance, not the original parent process.
Explanation
Environment variables and shell variables are two entirely different concepts, but as we manipulate them in a similar way in bash, it's easy to get confused.
Whenever a process is created (by fork), it may include an environment, given by its parent at fork-time. The child process may then access and modify its content. How this is done as a user depends on the program :
In vim, you can access an environment variable like this : :echo $foo
In bash, you can access it like this : $ echo "$foo"
In most programming languages, you can access it with a syntax coherent with the rest of the language, such as ENV['foo'] in ruby
On the other hand, a program may allocate memory for any internal use, but notably, it will quite often define and use variables. Once again, this depends on the program :
In vim, you would use the :let command to assign an internal variable
In bash, you would assign a variable with $ foo='bar', and then read it with $ echo "$foo"
In most programming languages, you have a variation of the foo='bar' syntax, sometimes with type declarations, etc
As you can see, bash uses the same syntax to read an environment variable and one of its own private variables, which can lead to some confusion.
When you execute vim from your bash shell, the environment is copied over from the parent process (bash) to the child (vim), but the private memory of bash (including the variables you may have defined) are not.
Thus, accessing them from the child process would require some inter-process communication mechanism, between parent and child. While technically doable, this option is not implemented in bash nor vim.
Solution
In order for your variable to be accessible from vim (or any forked process, for that matter), you need it to be present in the environment of your vim process.
Several options to do that :
$ export foo='bar' : This will mark your variable for export to the environment of subsequently executed commands. That's what you want in most cases.
$ foo='bar' vim : This adds your variable to the environment of this vim command. Very useful for troubleshooting, or for one-liners.
$ set -a : As you can see in bash manpage, this marks every subsequent definitions for export to the environment of subsequent commands. It's essentially equivalent to prepending every subsequent definition by export.
To go further
The question uses the :!echo $foo syntax to display the value of foo, which is yet another usecase. The ! here is actually an escape sequence that allows you to execute a shell command from vim.
However, vim cannot execute anything in the parent shell (the one you executed the vim command in), so it creates a new bash shell in a child process, executes echo in it, and displays the result.
In the current case, the result is mostly the same, but it could easily be misleading in other situations, so it's important to understand what is happening here.
There is another vim syntax, using expand, that allows one to lookup variables : :echo expand("$foo")
It however works entirely differently.
If no internal variable named foo exists, vim will invoke a shell to look it up (similarly to what ! would do).
This options is way slower than an environment lookup, and not recommended for most usecases.
If you want to use a value from your shell on the :substitute command, there's actually a way to do it.
I don't know if it solves your need but here we go.
Let's say we want to substitute Mydir by your PWD:
:s/Mydir/\=expand($PWD)/g
I'm looking how to export from a Makefile environment variables to be exposed in the userland environment so exporting these variables from the Makefile should be accessible from the user shell.
I have tried make's export but as I understand and have tried does not export to outside of Makefile.
The idea of this is to populate Docker Compose environment variables in a elegant way and have these variables ready to use in the user shell also.
This is a fragment of what I've tried with make's export:
include docker.env
export $(shell sed -n '/=/p' docker.env)
SHELL := /bin/bash
run:
#docker-compose -f my-service.yml up -d
According with ArchWiki, each process of Bash...
Each process stores their environment in the /proc/$PID/environ file.
so once Make execute a source, export or any other command to set a new environment variable it will be applied only for that process.
As workaround I've written in the bash startup file so the variables will be in the global environment as soon as a new bash shell is loaded:
SHELL := /bin/bash
RC := ~/.bashrc
ENV := $(shell sed -n '/=/p' docker.env)
test:
#$(foreach e,$(ENV),echo $(e) >> $(RC);) \
EDIT completely reworked the answer after the OP explained in a comment that he wants the environment variables to be defined for any user shell.
If your goal is to have a set of environment variables defined for any user shell (I assume this means interactive shell), you can simply add these definitions to the shell's startup file (.bashrc for bash). From GNU make manual:
Variables in make can come from the environment in which make is run.
Every environment variable that make sees when it starts up is
transformed into a make variable with the same name and value.
However, an explicit assignment in the makefile, or with a command
argument, overrides the environment. (If the ‘-e’ flag is specified,
then values from the environment override assignments in the makefile.
See Summary of Options. But this is not recommended practice.)
Example:
$ cat .bashrc
...
export FOOBAR=foobar
export BARFOO="bar foo"
...
$ cat Makefile
all:
#printf '$$(FOOBAR)=%s\n' '$(FOOBAR)'
#printf 'FOOBAR='; printenv FOOBAR
#printf '$$(BARFOO)=%s\n' '$(BARFOO)'
#printf 'BARFOO='; printenv BARFOO
$ make
$(FOOBAR)=foobar
FOOBAR=foobar
$(BARFOO)=bar foo
BARFOO=bar foo
If you want to keep these definitions separate, you can just source the file from .bashrc:
$ cat docker.env
export FOOBAR=foobar
export BARFOO="bar foo"
$ cat .bashrc
...
source <some-path>/docker.env
...
And finally, if you don't want to add the export bash command to your file, you can parse the file in your .bashrc:
$ cat docker.env
FOOBAR=foobar
BARFOO="bar foo"
$ cat .bashrc
...
while read -r line; do
eval "export $$line"
done < <(sed -n '/=/p' <some-path>/docker.env)
...
Of course, there are some constraints for the syntax of your docker.env file (no unquoted special characters, no spaces in variable names, properly quoted values...) If your syntax is not bash-compatible it is time to ask another question about parsing this specific syntax and converting it into bash-compatible syntax.
Make cannot change the calling shell's environment without its cooperation. Of course, if you are in control, you can make the calling shell cooperate.
In broad terms, you could replace the make command with a shell alias or function which runs the real make and also sets the environment variables from the result. I will proceed to describe in more detail one way to implement this.
Whether you call this alias or function of yours make or e.g. compose is up to you really. To wrap the real make is marginally harder -- inside the function, you need to say command make, because just make would cause an infinite loop with the alias or function calling itself recursively -- so I will demonstrate this. Let's define a function (aliases suck);
make () {
# run the real make, break out on failure
command make "$#" || return
# if there is no env for us to load, we are done
test -f ./docker.env || return 0
# still here? load it
. ./docker.env
}
If you want even stricter control, maybe define a variable in the function and check inside the Makefile that the variable is set.
$(ifneq '${_composing}','function_make')
$(error Need to use the wrapper function to call make)
$(endif)
The error message is rather bewildering if you haven't read this discussion, so maybe it needs to be improved, and/or documented in a README or something. You would change the make line in the function above into
_composing='function_make' \
command make "$#" || return
The syntax var=value cmd args sets the variable var to the string value just for the duration of running the command line cmd args; it then returns to its previous state (unset, or set to its previous value).
For this particular construction, the name of the variable just needs to be reasonably unique and transparent to a curious human reader; and the value is also just a reasonably unique and reasonably transparent string which the function and the Makefile need to agree on.
Depending on what you end up storing in the environment, this could introduce complications if you need this mechanism for multiple Makefiles. Running it in directory a and then switching to a similar directory b will appear to work, but uses the a things where the poor puny human would expect the b things. (If the variables you set contain paths, relative paths fix this scenario, but complicate others.)
Extending this to a model similar to Ruby's rvm or Python's virtualenv might be worth exploring; they typically add an indicator to the shell prompt to remind you which environment is currently active, and have some (very modest) safeguards in place to warn you when your current directory and the environment disagree.
Another wart: Hardcoding make to always load docker.env is likely to produce unwelcome surprises one day. Perhaps hardcode a different file name which is specific to this hook - say, .compose_post_make_hook? It can then in turn contain something like
. ./docker.env
in this particular directory.
This question already has answers here:
What does the 'export' command do?
(3 answers)
Closed 7 years ago.
When I customize my environment, I add PATH=$PATH:$My-own-Path in file .bash_profile.
The tutorials tell me I should use this one: export PATH=$PATH:$My-own-Path
So, what is the difference?
To answer your exact specific question, in this particular case, there isn't any difference. Why?
Somewhere in the initialization process, the variable PATH has already been exported. A change in the variable's value which is already exported does not need another export; this is automatic. The processes fired hereafter will get the new value.
export makes the environment variable available to child processes
From man bash:
... The export and declare -x commands allow parameters and functions to be added to and deleted from the environment. If the value of a parameter in the environment is modified, the new value becomes part of the environment, replacing the old.
Also from man bash:
export [-fn] [name[=word]] ...
export -p
The supplied names are marked for automatic export to the environment of subsequently executed commands. If the -f option is given, the names refer to functions. If no names are given, or if the -p option is supplied, a list of names of all exported variables is printed. The -n option causes the export property to be removed from each name. If a variable name is followed by =word, the value of the variable is set to word. export returns an exit status of 0 unless an invalid option is encountered, one of the names is not a valid shell variable name, or -f is supplied with a name that is not a function.
Exported variables are available to other programs. Non-exported variables are not.
Example:
$ myVar=Foo # Create local
$ env | grep '^myVar='
$ export myVar # Export myVar to child process
$ env | grep '^myVar='
Foo
If you want to read more about this, check out export (GNU Bash manual).
Also, please note that non-exported variables will be available to subshells run with (...) and other similar notations:
$ thereVar=Bar
$ (echo $thereVar; echo $myVar; $myVar=testing; echo $myVar)
Bar
Foo
Testing
$echo $myVar
Foo
The subshell cannot affect variables in the parent shell.
For more information on subshells, please reference:
Command Grouping
Command Execution Environment
Every process has an area of memory called the environment block. In the environment block are environment variables. These look like ordinary variables, for example x=42.
In most shells (C shell is an exception) you move an ordinary variable into the environment block using export. That command can also create an environment variable without going through an intermediate stage. If the variable is already in the environment block then export will have no effect.
So why? When a new process is created, the default action is to copy various "core information" from parent to child. These include the current directory, the umask, the file descriptor table, the uid and gid, and the environment block.
Note that the child only gets a copy of the parent's environment block. The variable is not shared and cannot be passed back to the parent (except by using some other inter-process communication mechanism).
You can override this default behaviour using the env program, but this is rarely required.
So, if we set an environment variable in a shell script using export then all our child processes we create, when we call other programs, will get a copy of them. Some variable names are well-known and have a special meaning, and the PATH environment variable is probably the most important of those.
The PATH environment variable is used to find programs on UNIX/Linux. Directories in PATH are searched in left-right order each time we need to load a program. Bash also caches executable paths in a hash (KornShell calls them "tracked aliases").
I have a number of Bash and Perl scripts which are unrelated in functionality, but are related in that they work within the same project.
The fact that they work in the same project means that I commonly specify the same directories, the same project specific commands, the same keywords at the top of every script.
Currently, this has not bitten me, but I understand that it would be easier to have all of these values in one place, then if something changes I can change a value once and have the various scripts pick up on those changes.
The question is - how is best to declare these values? A single Perl script that is 'required' in each script would require less changes to the Perl scripts, though doesn't provide a solution to the Bash script. A configuration file using a "key=value" format would perhaps be more universal, but requires each script to parse the configuration and has the potential to introduce issues. Is there a better alternative? Using environmental variables? Or a Bash specific way that Perl can easily execute and interpret?
When you run a shell script, it's done in a sub-shell so it cannot affect the parent shell's environment. So when you declare a variable as key=value its scope is limited to the sub-shell context. You want to source the script by doing:
. ./myscript.sh
This executes it in the context of the current shell, not as a sub shell.
From the bash man page:
. filename [arguments]
source filename [arguments]
Read and execute commands from filename in the current shell environment and return the exit status of the last command executed from filename.
If filename does not contain a slash, file names in PATH are used to find the directory containing filename.
Also you can use the export command to create a global environment variable. export governs which variables will be available to new processes, so if you say
FOO=1
export BAR=2
./myscript2.sh
then $BAR will be available in the environment of myscript2.sh, but $FOO will not.
Define environments variables :
user level : in your ~/.profile or ~/.bash_profile or ~/.bash_login or ~/.bashrc
system level : in /etc/profile or /etc/bash.bashrc or /etc/environment
For example add tow lines foreach variable :
FOO=myvalue
export FOO
To read this variable in bash script :
#! /bin/bash
echo $FOO
in perl script :
#! /bin/perl
print $ENV{'FOO'};
You could also source another file, so you do not create extra env variables, that may lead to unexpected behaviours.
source_of_truth.sh:
FOO="bar"
scritp1.sh
#!/usr/bin/env bash
source source_of_truth.sh
echo ${FOO}
# ... doing something
scritp2.sh
#!/usr/bin/env bash
source source_of_truth.sh
echo ${FOO}
# ... doing something else