I'm looking how to export from a Makefile environment variables to be exposed in the userland environment so exporting these variables from the Makefile should be accessible from the user shell.
I have tried make's export but as I understand and have tried does not export to outside of Makefile.
The idea of this is to populate Docker Compose environment variables in a elegant way and have these variables ready to use in the user shell also.
This is a fragment of what I've tried with make's export:
include docker.env
export $(shell sed -n '/=/p' docker.env)
SHELL := /bin/bash
run:
#docker-compose -f my-service.yml up -d
According with ArchWiki, each process of Bash...
Each process stores their environment in the /proc/$PID/environ file.
so once Make execute a source, export or any other command to set a new environment variable it will be applied only for that process.
As workaround I've written in the bash startup file so the variables will be in the global environment as soon as a new bash shell is loaded:
SHELL := /bin/bash
RC := ~/.bashrc
ENV := $(shell sed -n '/=/p' docker.env)
test:
#$(foreach e,$(ENV),echo $(e) >> $(RC);) \
EDIT completely reworked the answer after the OP explained in a comment that he wants the environment variables to be defined for any user shell.
If your goal is to have a set of environment variables defined for any user shell (I assume this means interactive shell), you can simply add these definitions to the shell's startup file (.bashrc for bash). From GNU make manual:
Variables in make can come from the environment in which make is run.
Every environment variable that make sees when it starts up is
transformed into a make variable with the same name and value.
However, an explicit assignment in the makefile, or with a command
argument, overrides the environment. (If the ‘-e’ flag is specified,
then values from the environment override assignments in the makefile.
See Summary of Options. But this is not recommended practice.)
Example:
$ cat .bashrc
...
export FOOBAR=foobar
export BARFOO="bar foo"
...
$ cat Makefile
all:
#printf '$$(FOOBAR)=%s\n' '$(FOOBAR)'
#printf 'FOOBAR='; printenv FOOBAR
#printf '$$(BARFOO)=%s\n' '$(BARFOO)'
#printf 'BARFOO='; printenv BARFOO
$ make
$(FOOBAR)=foobar
FOOBAR=foobar
$(BARFOO)=bar foo
BARFOO=bar foo
If you want to keep these definitions separate, you can just source the file from .bashrc:
$ cat docker.env
export FOOBAR=foobar
export BARFOO="bar foo"
$ cat .bashrc
...
source <some-path>/docker.env
...
And finally, if you don't want to add the export bash command to your file, you can parse the file in your .bashrc:
$ cat docker.env
FOOBAR=foobar
BARFOO="bar foo"
$ cat .bashrc
...
while read -r line; do
eval "export $$line"
done < <(sed -n '/=/p' <some-path>/docker.env)
...
Of course, there are some constraints for the syntax of your docker.env file (no unquoted special characters, no spaces in variable names, properly quoted values...) If your syntax is not bash-compatible it is time to ask another question about parsing this specific syntax and converting it into bash-compatible syntax.
Make cannot change the calling shell's environment without its cooperation. Of course, if you are in control, you can make the calling shell cooperate.
In broad terms, you could replace the make command with a shell alias or function which runs the real make and also sets the environment variables from the result. I will proceed to describe in more detail one way to implement this.
Whether you call this alias or function of yours make or e.g. compose is up to you really. To wrap the real make is marginally harder -- inside the function, you need to say command make, because just make would cause an infinite loop with the alias or function calling itself recursively -- so I will demonstrate this. Let's define a function (aliases suck);
make () {
# run the real make, break out on failure
command make "$#" || return
# if there is no env for us to load, we are done
test -f ./docker.env || return 0
# still here? load it
. ./docker.env
}
If you want even stricter control, maybe define a variable in the function and check inside the Makefile that the variable is set.
$(ifneq '${_composing}','function_make')
$(error Need to use the wrapper function to call make)
$(endif)
The error message is rather bewildering if you haven't read this discussion, so maybe it needs to be improved, and/or documented in a README or something. You would change the make line in the function above into
_composing='function_make' \
command make "$#" || return
The syntax var=value cmd args sets the variable var to the string value just for the duration of running the command line cmd args; it then returns to its previous state (unset, or set to its previous value).
For this particular construction, the name of the variable just needs to be reasonably unique and transparent to a curious human reader; and the value is also just a reasonably unique and reasonably transparent string which the function and the Makefile need to agree on.
Depending on what you end up storing in the environment, this could introduce complications if you need this mechanism for multiple Makefiles. Running it in directory a and then switching to a similar directory b will appear to work, but uses the a things where the poor puny human would expect the b things. (If the variables you set contain paths, relative paths fix this scenario, but complicate others.)
Extending this to a model similar to Ruby's rvm or Python's virtualenv might be worth exploring; they typically add an indicator to the shell prompt to remind you which environment is currently active, and have some (very modest) safeguards in place to warn you when your current directory and the environment disagree.
Another wart: Hardcoding make to always load docker.env is likely to produce unwelcome surprises one day. Perhaps hardcode a different file name which is specific to this hook - say, .compose_post_make_hook? It can then in turn contain something like
. ./docker.env
in this particular directory.
Related
I have several dirs with files stamp1.txt and stamp0.txt, and i want override cat command. I need it for example to suppress 'stamp1' files from archiving into library.
So i wrote little filter program called 'realname' and bash script to override original cat command.
function cat() {
local e=""
for s in $#
do
if realname $s; then
e=$e" "$s;
fi
done
command cat $e;
}
So command:
cat dir1/stamp1.txt dir2/stamp0.txt
will be converted to
cat dir2/stamp0.txt
And this example works just fine
ar cruv some_lib.a `cat dir1/stamp1.txt dir2/stamp0.txt`
But when i run some makefile to build some software - inside this process used original cat nor overrided.
How to override cat or any other command in way to get it work for make process without changing makefile(makefile is 3rdparty software and i don't want patch it every time when upgrade is needed)?
You can't do that with a shell function, because a shell function exists only in the local shell. It's not passed to programs like make. Also, GNU make always invokes /bin/sh by default, not /bin/bash, and your shell function above is written in bash syntax, so putting it in your ~/.bashrc will have no impact.
You could run:
$ make SHELL=/bin/bash
and add that shell function to your ~/.bashrc and that might work.
The only other thing you can do (assuming that your third party makefile invokes cat directly and doesn't use a variable like $(CAT) instead) is to create a cat shell script (not a function) and put it on your PATH before /bin and /usr/bin when you invoke make. Something like:
$ mkdir tmp
$ vi tmp/cat
...add commands...
$ chmod 755 tmp/cat
$ PATH=$(pwd)/tmp:$PATH make ...
Of course when you do this you can't use command cat ... in your script, you'll have to use a fully-qualified path like /bin/cat ...
Short version: you cannot (and you probably should not want to).
Longer version: You've defined a shell function. You could actually even export that shell function into the environment export -f cat. But unless the Makefile was saying bash -c cat ... instead of just cat (or other reference to the same effect), it would not give you the behavior you wanted.
If you really insisted... and the Makefile did not use hard coded path (e.g. /bin/cat). You could write your own cat, place it somewhere and make this location precede other possible hits for cat (just put it up front).
There is also some chance (look in the make file) it uses a variable (e.g. CAT) to know what to call, so you could just provide your own definition if that was the case.
In any case though. I would discourage you from using workaround like these because the actual behavior of the machinery gets obfuscated by doing so. There is something declared here... and something else in the environment giving it a different meaning. Which is a very common source of mistakes and eventually non-obvious (harder to resolve) bugs.
Example/clarification to the function bit. I have a Makefile:
all:
#echo foo
And define and export a function "overriding" echo. echo() { /bin/echo "$#" bar; } ; export -f echo. I run make and get:
$ make
foo
Because make just looks for echo in PATH (tries to exec and once it finds it, it runs it). If I changed it to have bash step in between, the exported function would kick in, but... that's an usual way to use commands in make and you'd have to edit the Makefile which you did not want:
all:
#bash -c 'echo foo'
This would yield you the result you wanted:
$ make
foo bar
The other option I've mentioned. I've put behavior of that function into a script /tmp/bin/echo reading:
#!/bin/bash
/bin/echo "$#" bar
And I've modified the PATH env var export PATH=/tmp/bin:$PATH. Now even with the first form of the Makefile:
all:
#echo foo
I get:
$ make
foo bar
But if the Makefile that is given says /bin/echo instead, I'd have no such luck. You could still change the binary... or change its behavior by forcing a shared library preload... but sounds quote extreme and fully exposes why this really might not be the best direction to take it.
I am trying to create something like a global variable that I will use in order to make my project easy to deploy for other developers.
I would like to have an .sh file where there is a variable defining the location of the project.
Later on I want to export this variable and make it accessable in every makefile that I am creating so that I can use this design to keep everything constant and in one place.
This is an example of what I am trying to build:
Creating and exporting the variables in script.sh:
#!/bin/bash
DIRECTORY='some path value here'
Importing the values in multiple Makefiles:
# start script and fetch the value
VAR := $(shell ./script.sh | sed -n '/^result: /s/^.*: //p')
all:
#echo VAR=$(VAR)
I would like to see how other people are dealing with the same problem.
Being a better developer is my goal here. :)
Feedback always welcomed.
Environment variables exported in the shell are visible from make, so in a shell script like this:
#!/bin/sh
VAR=value
export VAR
make $*
The Makefile will start with VAR defined to value. That's one way to get variables from a shell script into make.
If you don't want the shell script to run make, you can have a user source it:
$ source script.sh
$ make
The variables set in the script will be visible to make this way too.
Or course there doesn't seem to be any reason you need a shell script here. Stick your configuration into a fragment of a Makefile (which would look almost exactly like your shell script, but not use quotes for multiple word values) and then include Makefile.inc in your main makefile.
Also note that syntax like this:
#!/bin/sh or another commment
VAR=value
export VAR
It equally valid included in a Makefile or sourced into a shell script. So sometimes it's possible to use the same include file in both places!
I'm trying to access a variable declared by previous command (inside a Makefile).
Here's the Makefile:
all:
./script1.sh
./script2.sh
Here's the script declaring the variable I want to access,script1.sh:
#!/usr/bin/env bash
myVar=1234
Here's the script trying to access the variable previously defined, script2.sh:
#!/usr/bin/env bash
echo $myVar
Unfortunately when I run make, myVar isn't accessible. Is there an other way around? Thanks.
Make will run each shell command in its own shell. And when the shell exits, its environment is lost.
If you want variables from one script to be available in the next, there are constructs which will do this. For example:
all:
( . ./script1.sh; ./script2.sh )
This causes Make to launch a single shell to handle both scripts.
Note also that you will need to export the variable in order for it to be visible in the second script; unexported variables are available only to the local script, and not to subshells that it launches.
UPDATE (per Kusalananda's comment):
If you want your shell commands to populate MAKE variables instead of merely environment variables, you may have options that depend on the version of Make that you are running. For example, in BSD make and GNU make, you can use "variable assignment modifiers" including (from the BSD make man page):
!= Expand the value and pass it to the shell for execution and
assign the result to the variable. Any newlines in the result
are replaced with spaces.
Thus, with BSD make and GNU make, you could do this:
$ cat Makefile
foo!= . ./script1.sh; ./script2.sh
all:
#echo "foo=${foo}"
$
$ cat script1.sh
export test=bar
$
$ cat script2.sh
#!/usr/bin/env bash
echo "$test"
$
$ make
foo=bar
$
Note that script1.sh does not include any shebang because it's being sourced, and is therefore running in the calling shell, whatever that is. That makes the shebang line merely a comment. If you're on a system where the default shell is POSIX but not bash (like Ubuntu, Solaris, FreeBSD, etc), this should still work because POSIX shells should all understand the concept of exporting variables.
The two separate invocations of the scripts create two separate environments. The first script sets a variable in its environment and exits (the environment is lost). The second script does not have that variable in its environment, so it outputs an empty string.
You can not have environment variables pass between environments other than between the environments of a parent shell to its child shell (not the other way around). The variables passed over into the child shell are only those that the parent shell has export-ed. So, if the first script invoked the second script, the value would be outputted (if it was export-ed in the first script).
In a shell, you would source the first file to set the variables therein in the current environment (and then export them!). However, in Makefiles it's a bit trickier since there's no convenient source command.
Instead you may want to read this StackOverflow question.
EDIT in light of #ghoti's answer: #ghoti has a good solution, but I'll leave my answer in here as it explains a bit more verbosely about environment variables and what we can do and not do with them with regards to passing them between environments.
I have a number of Bash and Perl scripts which are unrelated in functionality, but are related in that they work within the same project.
The fact that they work in the same project means that I commonly specify the same directories, the same project specific commands, the same keywords at the top of every script.
Currently, this has not bitten me, but I understand that it would be easier to have all of these values in one place, then if something changes I can change a value once and have the various scripts pick up on those changes.
The question is - how is best to declare these values? A single Perl script that is 'required' in each script would require less changes to the Perl scripts, though doesn't provide a solution to the Bash script. A configuration file using a "key=value" format would perhaps be more universal, but requires each script to parse the configuration and has the potential to introduce issues. Is there a better alternative? Using environmental variables? Or a Bash specific way that Perl can easily execute and interpret?
When you run a shell script, it's done in a sub-shell so it cannot affect the parent shell's environment. So when you declare a variable as key=value its scope is limited to the sub-shell context. You want to source the script by doing:
. ./myscript.sh
This executes it in the context of the current shell, not as a sub shell.
From the bash man page:
. filename [arguments]
source filename [arguments]
Read and execute commands from filename in the current shell environment and return the exit status of the last command executed from filename.
If filename does not contain a slash, file names in PATH are used to find the directory containing filename.
Also you can use the export command to create a global environment variable. export governs which variables will be available to new processes, so if you say
FOO=1
export BAR=2
./myscript2.sh
then $BAR will be available in the environment of myscript2.sh, but $FOO will not.
Define environments variables :
user level : in your ~/.profile or ~/.bash_profile or ~/.bash_login or ~/.bashrc
system level : in /etc/profile or /etc/bash.bashrc or /etc/environment
For example add tow lines foreach variable :
FOO=myvalue
export FOO
To read this variable in bash script :
#! /bin/bash
echo $FOO
in perl script :
#! /bin/perl
print $ENV{'FOO'};
You could also source another file, so you do not create extra env variables, that may lead to unexpected behaviours.
source_of_truth.sh:
FOO="bar"
scritp1.sh
#!/usr/bin/env bash
source source_of_truth.sh
echo ${FOO}
# ... doing something
scritp2.sh
#!/usr/bin/env bash
source source_of_truth.sh
echo ${FOO}
# ... doing something else
I have a set of tools which I need to pass parameters depending on the project I'm working on. I'd like to be able to automatically set a couple of environment variables based on the current directory. So when I switched between directories, my commonly used env vars would also change. Example:
Let's current directory is foo, thus if I do:
~/foo$ ./myscript --var1=$VAR1
VAR1 would have some foo based value.
Then, let's say I switched to bar directory. If I do:
~/bar$ ./myscript --var1=$VAR1
VAR1 should now have some bar based value.
Is that possible? How?
the ondir program lets you specify actions to run when you enter and leave directories in a terminal
There is direnv which helps you do this stuff much easily and in an elegant way. Just define a .envrc file in your project directory with all the env variables needed and it will source it once you cd into that folder.
I've written another implementation of this, which is somewhat similar to ondir. I didn't actually know about ondir when I started working on it. There are some key differences that may be useful, however.
smartcd is written entirely in shell, and is fully compatible with bash and zsh, even the more esoteric options
smartcd will run scripts all the way down and up the directory hierarchy down to their common ancestor, not just for the two directories you're entering and leaving. This means you can have a ~/foo script that will execute whether you "cd ~/foo" or "cd ~/foo/bar"
it has "variable stashing" which is a more automatic way of dealing with your environment variables, whereas ondir requires you to explicitly and manually remove and/or reset your variables
smartcd can work with "autocd" turned on by hooking your prompt command (PROMPT_COMMAND in bash, precmd in zsh)
You can find smartcd at https://github.com/cxreg/smartcd
This is not something that is directly supported with the built-in features of bash or any other common shell. However, you can create your own "cd" command that will do whatever you want. For example, you could alias cd to do the cd and then run a special script (eg: ~/bin/oncd). That script could look up the new directory in a database and run some commands, or see if there's a special file (eg: .env) in the directory and load it, etc.
I do this sort of thing a lot. I create several identically named batch files in directories where I need them that only set the variables and call the common script. I even have a batch file that creates the other small files.
This is not pretty, but you can use a combination of exported environment variables and the value of $PWD.
For example:
export VAR1=prefix
export prefix${HOME////_}_foo=42
export prefix${HOME////_}_bar=blah
Then myscript needs only to eval echo \${$VAR1${PWD////_}} to get at the directory based value.
How about wrap your script with a function (the function can be placed either in your bash profile/bashrc file in the system ones to make available for all the users ).
myscript () { case $PWD in
/path/to/foo) path/to/myscript --var1=$VAR1 ;;
/path/to/bar) path/to/myscript --var2=$VAR1 ;;
*) ;;
case
}
Hence the function myscript will call the real "myscript" knowing what to do based on the current working directory.
Take this as an example:
hmontoliu#ulises:/tmp$ myscript () { case $PWD in /tmp) echo I\'m in tmp;; /var) echo I\'m in var;; *) echo I\'m neither in tmp nor in bar; esac; }
hmontoliu#ulises:/tmp$ myscript
I'm in tmp
hmontoliu#ulises:/tmp$ cd /var
hmontoliu#ulises:/var$ myscript
I'm in var
hmontoliu#ulises:/var$ cd /etc
hmontoliu#ulises:/etc$ myscript
I'm neither in tmp nor in bar