I found many answers here and elsewhere on the topic, but none that worked. Please help me out here.
I need to set some environment variables, which is partly done in some scripts, called from a master script, partly directly. Here is a minimal Makefile that shows the unwanted behaviour:
FC := ifort
SHELL := /bin/bash
some_target: load_ifort
$(FC) file.f
load_ifort:
source /usr/local2/bin/ifort-compilervars.sh ia32
export LM_LICENSE_FILE=/usr/local2/misc/intel2013/flexlm/server.lic
if I call make, I get an "ifort: command not found" error. If I execute the two comamnds by hand on the command line before calling make, ifort is found and everything is good.
What am I missing???
Each line in a recipe gets executed in a separate subshell. So you create one shell which sources the .sh file, then exits and forgets everything, then another shell which starts with a clean slate.
The straightforward solution in your case would be to collect all these commands in a single variable. I have factored out the LM_LICENSE_FILE assignment because that can be done in Make directly, but you could include that in the FC variable as well.
LM_LICENSE_FILE := /usr/local2/misc/intel2013/flexlm/server.lic
export LM_LICENSE_FILE
FC := source /usr/local2/bin/ifort-compilervars.sh ia32; \
ifort
some_target:
$(FC) file.f
If the shell commands can be straightforwardly run by Make as well, you could include them, or perhaps translate the sh file into Make commands by a simple script.
Another option would be to create a simple wrapper in your PATH; maybe call it fc:
#!/bin/sh
. /usr/local2/bin/ifort-compilervars.sh ia32
ifort "$#"
then just use fc where you currently have $(FC). (If the ifort-compilervars.sh file contains Bash constructs, in spite of the name, you should change the shebang to #!/bin/bash.)
As a rule, only one-liner shell commands "work". From the comment about "bash", it seems likely you are using GNU make. In your example, the word "source" is not found in the GNU make manual's index. (If you found this in a working example, it would be helpful to start from that). There are two types of variables of interest:
makefile variables, which live in the make program
environment variables, which are "exported"
The latter would include $PATH, which is used to find programs. For updating that, you do need shell commands. But (lacking some special provision in the make program), exported variables from a shell script are not passed up into the make program and made available for the next line of the makefile.
You could reorganize the makefile to provide a rule which combines the source command and other initialization into a shell command which then recurs (carrying those variables along) into a subprocess which would then do the compiles. Something like
build:
sh -c "source /usr/local2/bin/ifort-compilervars.sh ia32; \
export LM_LICENSE_FILE=/usr/local2/misc/intel2013/flexlm/server.lic; \
$(MAKE) some_target"
some_target: load_ifort
$(FC) file.f
Related
My default shell is bash in Ubuntu 14.04. I have a csh script file named clean.sh with the following make command:
#! /bin/csh -f
make -f commande.make del
And commande.make has
CKHOME=../CHEMKIN/DATA_BASES
LIN_DATA=${CKHOME}/LIN_FILES/
LINK_CKTP=${CKHOME}/LINK_CKTP_FILES/
#-----------------------------------------------------
include schema_cinetique.make
LINKFILE=${NAME}_LINK
LINKTPFILE=${NAME}_LINKTP
LINKFILE_OLD=${NAME_OLD}_LINK
LINKFILE_NEW=${NAME_NEW}_LINK
#-----------------------------------------------------
cplink :
${COPY} ${LINK_CKTP}${LINKFILE} LINK
cplink2 :
${COPY} ${LINK_CKTP}${LINKFILE} LINKZ1
tplink :
${COPY} ${LINK_CKTP}${LINKTPFILE} LINKTPZ1
calcul :
${COPY} jobtimp1 LJOBNZ1
${COPY} unsteadyf.dat1 DATZ1
del :
${DELETE} LINKZ1 LINKTPZ1 LJOBNZ1 DATZ1 SOLASUZ1
I opened the terminal and moved to the location and tried
./clean.sh
or
csh clean.sh &
or
csh -f clean.sh
Nothing worked.
I got the following line in the terminal,
LINKZ1 LINKTPZ1 LJOBNZ1 DATZ1 SOLASUZ1
make: LINKZ1: Command not found
make: *** [del] Error 127
So, how to run clean.sh file ?
You are confused. The Csh script contains a single command which actually runs identically in Bash.
#!/bin/bash
make -f commande.make del
Or, for that matter, the same with #!/bin/sh. Or, in this individual case, even sh clean.sh, since the shebang is then just a comment, and the commands in the file are available in sh just as well as in csh.
Once make runs, that is what parses and executes the commands in commande.make. make is not a "Fortran command", it is a utility for building projects (but the makefile named commande.make probably contains some instructions for how to compile Fortran code).
In the general case, Csh and Bash are incompatible, but the differences are in the shell's syntax itself (so, the syntax of loops and conditionals, etc, as well as variable assignments and various other shell builtins).
As an aside, Csh command files should probably not have a .sh extension, as that vaguely implies Bourne shell (sh) syntax. File extensions on Unix are just a hint to human readers, so not technically important; but please don't confuse them/us.
(As a further aside, nobody should be using Csh in 2022. There was a time when the C shell was attractive compared to its competition, but that was on the order of 40 years ago.)
The subsequent errors you are reporting seem to indicate that the makefile depends on some utilities which you have not installed. Figuring that out is a significant enough and separate enough question that you should probably ask a new question about that, probably with more debugging details. But in brief, it seems that make needs to be run with parameters to indicate what NAME and COPY (and probably some other variables) should be. Try with make -f commande.make COPY=cp DELETE=rm NAME=foobar for a start, but it's probably not yet anywhere near sufficient.
(I would actually assume that there will be a README file or similar to actually instruct you how to use commande.make since it seems to have some local conventions of its own.)
It seems the script is written having portability in mind, i.e. the name of the cp and rm binaries is kept in variables rather than hard-coding it. My best guess is that this has been done to make it possible to run the script on non UNIX systems, like Windows.
To make it work, export the respective variables before running the script. For the del action you are calling, only the DELETE variable is needed. It should be set to rm which is the command used to remove files on Linux:
export DELETE=rm
./clean.sh
Note: exporting the variable can also be done in one line when invoking the script, by prepending it to the command line:
DELETE=rm ./clean.sh
This behaviour is described in the bash manual:
The environment for any simple command or function may be augmented temporarily by prefixing it with parameter assignments, as described in Shell Parameters. These assignment statements affect only the environment seen by that command.
Say in ~/prj/abc/abcsim/abctsim/abcxyz/Makefile there is a line below.
TOOLCHAIN_DIR := $(PWD)/../../../prj1/tools/gcc_tools
If I'm in directory ~/test, and if I run make -C ~/prj/abc/abcsim/abctsim/abcxyz, this doesn't work because the $(PWD) variable is set to ~/test, not ~/prj/abc/abcsim/abctsim/abcxyz. How can I get the directory path where the Makefile exists?
In bash there's something for this : How can I get the source directory of a Bash script from within the script itself?
If you really use make -C (not make -f) and your Makefile is not included in another, you can simply use the CURDIR variable. GNU make sets it to the absolute path of the current directory when it starts, "after it has processed any -C options". So, in your case it should do exactly what you want.
Else, if you sometimes use make -f or if you have included Makefiles, you can put this as the first line of any of your Makefiles (or, at least, before any include statement):
HERE := $(dir $(lastword $(MAKEFILE_LIST)))
and then use $(HERE) to refer to this Makefile's directory. See the GNU make manual for the details.
Note: I was almost sure this question would be a duplicate. Surprisingly I searched SO for a clear answer and found only old answers that first suggest shell calls before using make built-ins or wrong answers (using firstword instead of lastword, for instance).
I have a script "set_env.py" that outputs the following uppon execution:
export MY_VAR_1=some_value
export MY_VAR_2=some_other_value
I cannot change this script, it is supplied in my current environment.
Further I have a Makefile that looks like this:
SHELL := /bin/bash
all: set_env
env | grep MY_VAR
set_env:
eval ./set_env.py
With this makefile I expected the grep to list my two variables, however it seems the environment is not set.
I suspect this is because make creates a sub-environment for each line and so the variables set on the first line will not be available in the second.
So the question is, how would I go about exporting the environment from the untouchable script in my makefile?
Actually, the output of the python is valid make.
One option then is to read the output of the python directly into the makefile.
The only fly in the ointment is that $(shell) doesn't cut the mustard.
include Environment.mk
PHONY: test
test:
env | grep MY_VAR
Environment.mk:
./set_env.py >$#-tmp
mv $#-tmp $#
How does this work?
The first thing that make tries to do is to ensure the makefile itself is up-to-date.
Since we have told it to include Environment.mk,
make must ensure that is up-to-date too.
Make finds a rule for Environment.mk
The python is run, creating Environment.mk
Environment.mk is read in, creating two make variables with the export attribute
The makefile is now up-to-date, so make proceeds on to the target (test in this case)
Make runs test's recipe, exporting any variables with the export attribute.
No recursion, but you should ensure the python always spits out make compatible syntax.
EDIT
As #raspy points out, this is not the whole story.
As it stands,
once Environment.mk has been created,
it will never be regenerated.
If set_env.py ever generates different output,
you should tell make what conditions these are by adding dependencies.
If set_env.py takes a trivial time to run,
I advise a simple .PHONY.
That way it will run every time you run make,
and Environment.mk will never be stale.
.PHONY: Environment.mk
Recursive make is your friend I'm afraid.
.PHONY: all
all:
eval $$(./set_env.py) && ${MAKE} test
.PHONY: test
test:
env | grep MY_VAR
There are a few moving parts here.
make all executes the shell command eval $(./set_env.py) && make test
The shell first does the command substitution
$(./set_env.py) is replaced by the export commands
The export commands are passed to the (shell) eval command
The environment variables are defined, but only for this shell invocation
The eval succeeds, so the execution passes to the command after the &&
Make runs recursively, but this second make has an augmented environment
I have a Makefile from which I want to call another external bash script to do another part of the building. How would I best go about doing this?
Just like calling any other command from a makefile:
target: prerequisites
shell_script arg1 arg2 arg3
Regarding your further explanation:
.PHONY: do_script
do_script:
shell_script arg1 arg2 arg3
prerequisites: do_script
target: prerequisites
Perhaps not the "right" way to do it like the answers already provided, but I came across this question because I wanted my makefile to run a script I wrote to generate a header file that would provide the version for a whole package of software. I have quite a bit of targets in this package, and didn't want to add a brand new prerequisite to them all. Putting this towards the beginning of my makefile worked for me
$(shell ./genVer.sh)
which tells make to simply run a shell command. ./genVer.sh is the path (same directory as the makefile) and name of my script to run. This runs no matter which target I specify (including clean, which is the downside, but ultimately not a huge deal to me).
Each of the actions in the makefile rule is a command that will be executed in a subshell. You need to ensure that each command is independent, since each one will be run inside a separate subshell.
For this reason, you will often see line breaks escaped when the author wants several commands to run in the same subshell:
targetfoo:
command_the_first foo bar baz
command_the_second wibble wobble warble
command_the_third which is rather too long \
to fit on a single line so \
intervening line breaks are escaped
command_the_fourth spam eggs beans
Currently using Makefile, I can easily call the bash script like this:
dump:
./script_dump.sh
And call:
make dump
This also works like mentioned in the other answer:
dump:
$(shell ./script_dump.sh)
But the downside is that you don't get the shell commands from the console unless you store in a variable and echo it.
So, to compile my executable, I need to have the library locations set up correctly. The problem is, the setup comes from a bunch of scripts that do the env variable exporting, and what needs to be set up may change (beyond my control) so I need to use those scripts instead of copying their functionality. To compile in regular command line, I need to do something like:
setup library1
setup library2
source some_other_setup_script.bash
g++ blah.c
# setup is a executable on my system that run some scripts
How would I write a makefile that accomplishes that? As far as I tried, the env variable exporting does not carry over (i.e. "export VAR=remember; echo $VAR" won't work)
You can also add environment variables properly with the machinery of GNU make, like so:
export TEST:="Something Good!"
test:
echo $$TEST
This (I think) has different semantics from:
TEST2:="Something not quite so useful?"
test2:
echo ${TEST2}
Which (again, I think) does the substitution within make before passing along to the shell. Note that the export command doesn't work within a target block, just unindented as an immediately executed command.
If variable exporting is not working the way it does on your command line, that suggests that Make is choosing a shell different from the one you're using, with different syntax for handling variables (export VAR=remember; echo $VAR works fine for me). Make uses /bin/sh by default, but you can override this with the SHELL variable, which Make does not import from the environment. I suggest setting SHELL (in the Makefile) to whatever you're using in your environment and trying the export VAR=remember experiment again.
Ultimately you will need to define the variable and execute the compiler in a shell list or even a script, rather than in separate make commands. There are a couple of refinements you could add, however. You could tell make about the script:
maintarget: script.sh blah.c
source script.sh; g++ blah.c
script.sh:
setup include script here
Another thing would be to just execute all that stuff in the same shell
maintarget: blah.c
run this; run that; run the other thing; g++ blah.c
I believe all make versions will run a ; list in the same shell, but you can always force a subshell with (list) or by calling specifically a shell script as a compiler command wrapper.
Don't forget to have the appropriate targets depend on your scripts themselves. BTW, some make versions (pmake aka bsd make) can execute a command when defining a make variable, and all versions of make then exports those. But I don't think gmake can do that.
You could write another shell script that executes all those commands, then prints out variable assignments that make can use. Run the script, pipe its output to a file, then include that file from your Makefile. For example:
Makefile:
all:
echo $(FOO)
test.mk: test.sh
./$< > $#
include test.mk
test.sh
echo "FOO=1"
Running "make" in the directory containing this Makefile produces:
make: Entering directory `/home/luser/build/mktest'
Makefile:7: test.mk: No such file or directory
./test.sh > test.mk
make: Leaving directory `/home/luser/build/mktest'
make: Entering directory `/home/luser/build/mktest'
echo 1
1
make: Leaving directory `/home/luser/build/mktest'
make creates test.mk by running the shell script, then includes it. test.mk contains the output of test.sh, and is parsed as a Makefile. See http://www.gnu.org/software/make/manual/make.html#Include for more details.
We use a variant of this in Mozilla's client.mk to let you define options in a "mozconfig" file:
http://mxr.mozilla.org/mozilla-central/source/client.mk#138
Restatement: How do I get a shell variable into a make file?
Something like:
MYVAR := $(shell echo $(MYVAR)) <any_makefile_additions_here>
So, this defines MYVAR inside a MAKEFILE when an environment variable named MYVAR is also set.
It might be of interest, that, in order to override an option that is already defined in a makefile, make supports (I am referring to GNU Make 3.82, but other version probably too) the option -e.
Example:
Makefile:
CC=gcc
...
Run make:
CC=gcc-4.7
make -e
will use gcc-4.7 instead of gcc.