I have a Makefile from which I want to call another external bash script to do another part of the building. How would I best go about doing this?
Just like calling any other command from a makefile:
target: prerequisites
shell_script arg1 arg2 arg3
Regarding your further explanation:
.PHONY: do_script
do_script:
shell_script arg1 arg2 arg3
prerequisites: do_script
target: prerequisites
Perhaps not the "right" way to do it like the answers already provided, but I came across this question because I wanted my makefile to run a script I wrote to generate a header file that would provide the version for a whole package of software. I have quite a bit of targets in this package, and didn't want to add a brand new prerequisite to them all. Putting this towards the beginning of my makefile worked for me
$(shell ./genVer.sh)
which tells make to simply run a shell command. ./genVer.sh is the path (same directory as the makefile) and name of my script to run. This runs no matter which target I specify (including clean, which is the downside, but ultimately not a huge deal to me).
Each of the actions in the makefile rule is a command that will be executed in a subshell. You need to ensure that each command is independent, since each one will be run inside a separate subshell.
For this reason, you will often see line breaks escaped when the author wants several commands to run in the same subshell:
targetfoo:
command_the_first foo bar baz
command_the_second wibble wobble warble
command_the_third which is rather too long \
to fit on a single line so \
intervening line breaks are escaped
command_the_fourth spam eggs beans
Currently using Makefile, I can easily call the bash script like this:
dump:
./script_dump.sh
And call:
make dump
This also works like mentioned in the other answer:
dump:
$(shell ./script_dump.sh)
But the downside is that you don't get the shell commands from the console unless you store in a variable and echo it.
Related
I found many answers here and elsewhere on the topic, but none that worked. Please help me out here.
I need to set some environment variables, which is partly done in some scripts, called from a master script, partly directly. Here is a minimal Makefile that shows the unwanted behaviour:
FC := ifort
SHELL := /bin/bash
some_target: load_ifort
$(FC) file.f
load_ifort:
source /usr/local2/bin/ifort-compilervars.sh ia32
export LM_LICENSE_FILE=/usr/local2/misc/intel2013/flexlm/server.lic
if I call make, I get an "ifort: command not found" error. If I execute the two comamnds by hand on the command line before calling make, ifort is found and everything is good.
What am I missing???
Each line in a recipe gets executed in a separate subshell. So you create one shell which sources the .sh file, then exits and forgets everything, then another shell which starts with a clean slate.
The straightforward solution in your case would be to collect all these commands in a single variable. I have factored out the LM_LICENSE_FILE assignment because that can be done in Make directly, but you could include that in the FC variable as well.
LM_LICENSE_FILE := /usr/local2/misc/intel2013/flexlm/server.lic
export LM_LICENSE_FILE
FC := source /usr/local2/bin/ifort-compilervars.sh ia32; \
ifort
some_target:
$(FC) file.f
If the shell commands can be straightforwardly run by Make as well, you could include them, or perhaps translate the sh file into Make commands by a simple script.
Another option would be to create a simple wrapper in your PATH; maybe call it fc:
#!/bin/sh
. /usr/local2/bin/ifort-compilervars.sh ia32
ifort "$#"
then just use fc where you currently have $(FC). (If the ifort-compilervars.sh file contains Bash constructs, in spite of the name, you should change the shebang to #!/bin/bash.)
As a rule, only one-liner shell commands "work". From the comment about "bash", it seems likely you are using GNU make. In your example, the word "source" is not found in the GNU make manual's index. (If you found this in a working example, it would be helpful to start from that). There are two types of variables of interest:
makefile variables, which live in the make program
environment variables, which are "exported"
The latter would include $PATH, which is used to find programs. For updating that, you do need shell commands. But (lacking some special provision in the make program), exported variables from a shell script are not passed up into the make program and made available for the next line of the makefile.
You could reorganize the makefile to provide a rule which combines the source command and other initialization into a shell command which then recurs (carrying those variables along) into a subprocess which would then do the compiles. Something like
build:
sh -c "source /usr/local2/bin/ifort-compilervars.sh ia32; \
export LM_LICENSE_FILE=/usr/local2/misc/intel2013/flexlm/server.lic; \
$(MAKE) some_target"
some_target: load_ifort
$(FC) file.f
How to use MAKEFILES variable environment? I writing in a bash MAKEFILES=/home/toker/mymake, but if I'm running a /home/toker/bundocode/gettingstart/testMF/Makefile then /home/toker/mymake doesn't executed. When I'm type $MAKEFILES in the bash then bash: /home/toker/mymake: Permission denied is displayed.
The MAKEFILES variable contains one or more makefiles, not the make program. From the above messages it looks like mymake is a make program you're trying to run. The shell decides what programs to run (by looking in directories listed in the $PATH environment variable) and the shell doesn't pay any special attention to the MAKEFILES variable. Only once make is running does it look at the MAKEFILES variable to see what other makefiles it might want to parse.
What is in the /usr/toker/mymake file? What do you mean by running a .../Makefile? One typically runs make, not a makefile. Then make reads in the makefile. There are ways to change a makefile to be executable but I don't think that's what you're looking for here.
ETA:
If the file /usr/toker/mymake is actually a makefile (BTW, convention typically uses .mk extensions to denote makefiles although there's no really official extension), then you can do this:
$ cat /usr/toker/mymake
$(info loaded /usr/toker/mymake)
foo: ; #echo building $#
$ export MAKEFILES=/usr/toker/mymake
$ cat Makefile
bar: foo ; #echo building $#
$ make
loaded /usr/toker/mymake
building foo
building bar
If you can show an example of exactly what you did (show the makefile and the commands you typed and the results you got) and explain what you don't understand about it, then we can see much more accurately what's going on than by you trying to describe it in words).
Is there a way to echo the (system, user, real) time spent in each target of a Makefile recursively when I do make all?
I'd like to benchmark the compilation of a project in a more granular way than just time make all. Ideally, it would echo a tree of the executed target, each one with the time spent in all its dependencies. It'd be great also if it could work with -j (parallel make). And by the way my Makefile is non-recursive (doesn't spawn another make instance for each main targets).
Thanks!
Gnu Make uses the $(SHELL) variable to execute commands in the targets.
By default it is set to /bin/sh.
You can set this variable to a script that will execute the command given with the "time" command. Something like this:
In your makefile specify the SHELL variable, somewhere at the top:
SHELL = ./report_time.sh
and in the file ./report_time.sh:
#!/bin/sh
shift # get rid of the '-c' supplied by make.
time sh -c "$*"
The replace the 'sh' command with the original SHELL specified in the Makefile if any.
This will report the timings.
However This will not tell you what target the report_time.sh script is running. One solution for this is to prepend the target name ($#) in each target entry in the makefile so that it will be passed to the report_time.sh script as well.
remake --profile is a drop-in replacement for make. It generates a target call tree in a callgrind format.
I want to tell make that it shall always use -j4 option even if I didn't specify it vie command line. Normally i would do this in some configuration file (i.e. ~/.makerc).
Does such file exist for gnu make?
Have a read about the $(MAKEFLAGS) variable:
export MAKEFLAGS=j4
However this will likely interfere with recursive-make-based builds (not that sensible people are using recursive make anyway!), by interfering with GNU make's ability to communicate with its sub-makes.
So the more sensible approach is probably a wrapper script or an alias or shell function.
Well, yes and no --- normally you would use an include file. Put your common configuration items together in a file, say common.mk and add
include common.mk
at the top of your makefile. If the flag doesn't have a matching way to configure it from inside the make file, you can use a function
function mk {
make -j4 $*
}
It doesn't exist, but you can do this by having a recursive call into make.
For example:
Makefile:
-include $(HOME)/.makerc
.DEFAULT_GOAL: all
# This will handle a default goal if make is just called without any target
all:
$(MAKE) $(MAKE_OPTIONS) -f Makefile.real $(MAKECMDGOALS)
# This handles all targets and passes it through
%:
$(MAKE) $(MAKE_OPTIONS) -f Makefile.real $(MAKECMDGOALS)
$(HOME)/.makerc:
MAKE_OPTIONS := -j4
I would like to expand a bit on the solution hinted in John Marshall's answer.
You can simply put a one-line wrapper script somewhere earlier in the $PATH with the following contents:
#!/bin/bash
$(type -ap make | sed -n 2p) -j4 "$#"
(The script doesn't have to be named make, and that would make it simpler, but I find it convenient if it is.)
I would argue that this is better than the other approaches for the following reasons:
Unlike MAKEFLAGS approach, it does not break recursive builds (which are actually quite common in my experience).
Unlike include .makerc approach, it can be applied locally without changing any existing makefiles or your workflow in any way.
Unlike shell alias or function approach, it is shell-agnostic (doesn't tie you to any particular shell) and works in any additional build scripts that you might have to use, too, as long as you launch them in the same environment.
I like the MAKEFLAGS approach suggested by John Marshall in lieu of make supporting something like an automatic .makerc project config file. However, I didn't want to have to remember to source a .env or similar environment variables beforehand (and unsetting them afterward).
A solution to this is to put the MAKEFLAGS assignment at the top of the Makefile itself:
#!/usr/bin/env make
MAKEFLAGS=s
.PHONY: foo
foo:
echo "hello, make"
Run it:
$ make foo
hello, make
Compared to running without the MAKEFLAGS=... line:
$ make foo
echo "hello, make"
hello, make
So, to compile my executable, I need to have the library locations set up correctly. The problem is, the setup comes from a bunch of scripts that do the env variable exporting, and what needs to be set up may change (beyond my control) so I need to use those scripts instead of copying their functionality. To compile in regular command line, I need to do something like:
setup library1
setup library2
source some_other_setup_script.bash
g++ blah.c
# setup is a executable on my system that run some scripts
How would I write a makefile that accomplishes that? As far as I tried, the env variable exporting does not carry over (i.e. "export VAR=remember; echo $VAR" won't work)
You can also add environment variables properly with the machinery of GNU make, like so:
export TEST:="Something Good!"
test:
echo $$TEST
This (I think) has different semantics from:
TEST2:="Something not quite so useful?"
test2:
echo ${TEST2}
Which (again, I think) does the substitution within make before passing along to the shell. Note that the export command doesn't work within a target block, just unindented as an immediately executed command.
If variable exporting is not working the way it does on your command line, that suggests that Make is choosing a shell different from the one you're using, with different syntax for handling variables (export VAR=remember; echo $VAR works fine for me). Make uses /bin/sh by default, but you can override this with the SHELL variable, which Make does not import from the environment. I suggest setting SHELL (in the Makefile) to whatever you're using in your environment and trying the export VAR=remember experiment again.
Ultimately you will need to define the variable and execute the compiler in a shell list or even a script, rather than in separate make commands. There are a couple of refinements you could add, however. You could tell make about the script:
maintarget: script.sh blah.c
source script.sh; g++ blah.c
script.sh:
setup include script here
Another thing would be to just execute all that stuff in the same shell
maintarget: blah.c
run this; run that; run the other thing; g++ blah.c
I believe all make versions will run a ; list in the same shell, but you can always force a subshell with (list) or by calling specifically a shell script as a compiler command wrapper.
Don't forget to have the appropriate targets depend on your scripts themselves. BTW, some make versions (pmake aka bsd make) can execute a command when defining a make variable, and all versions of make then exports those. But I don't think gmake can do that.
You could write another shell script that executes all those commands, then prints out variable assignments that make can use. Run the script, pipe its output to a file, then include that file from your Makefile. For example:
Makefile:
all:
echo $(FOO)
test.mk: test.sh
./$< > $#
include test.mk
test.sh
echo "FOO=1"
Running "make" in the directory containing this Makefile produces:
make: Entering directory `/home/luser/build/mktest'
Makefile:7: test.mk: No such file or directory
./test.sh > test.mk
make: Leaving directory `/home/luser/build/mktest'
make: Entering directory `/home/luser/build/mktest'
echo 1
1
make: Leaving directory `/home/luser/build/mktest'
make creates test.mk by running the shell script, then includes it. test.mk contains the output of test.sh, and is parsed as a Makefile. See http://www.gnu.org/software/make/manual/make.html#Include for more details.
We use a variant of this in Mozilla's client.mk to let you define options in a "mozconfig" file:
http://mxr.mozilla.org/mozilla-central/source/client.mk#138
Restatement: How do I get a shell variable into a make file?
Something like:
MYVAR := $(shell echo $(MYVAR)) <any_makefile_additions_here>
So, this defines MYVAR inside a MAKEFILE when an environment variable named MYVAR is also set.
It might be of interest, that, in order to override an option that is already defined in a makefile, make supports (I am referring to GNU Make 3.82, but other version probably too) the option -e.
Example:
Makefile:
CC=gcc
...
Run make:
CC=gcc-4.7
make -e
will use gcc-4.7 instead of gcc.