run before and after each target in a makefile - shell

I would like to run a command before and after each target in a makefile.
so something like this
pre:
#echo pre
#echo running | mailx -s "Start {target}" user#foo.com
post:
#echo post
#echo post | mailx -s "Finish {target}" user#foo.com
j:
long_running_command && echo $# > $#
k: j
long_running_command2 && echo $# > $#
I would like to run pre and post for j and k. Ideally, I would like to get an email for each task that starts and stops.

One way to do it is to modify all the recipes in your makefile to invoke some command. You can put it into a variable so it doesn't look too gross:
START = mailto --subject 'Started target $#' me#my.host
END = mailto --subject 'Finished target $#' me#my.host
j:
$(START)
long_running_command && echo $# > $#
$(END)
k: j
$(START)
long_running_command2 && echo $# > $#
$(END)
The nice thing about this is you can pick and choose which targets you want it for; maybe some of them don't need it. The disadvantage is if the command fails you won't get any "end" email at all.
If you really want to do it for every single target, then you can write a shell script that mimics the shell's behavior but also sends mail, while running the command.
$ cat mailshell
#!/bin/sh
# get rid of the -c flag
shift
mailto --subject "started $*" me#my.host
/bin/sh -c "$#"
r=$?
mailto --subject "ended $* with exit code $r" me#my.host
exit $r
(note this is totally untested but you get the idea I hope). Then in your makefile, set SHELL to that shell:
SHELL := mailshell
j:
long_running_command && echo $# > $#
k: j
long_running_command2 && echo $# > $#
I guess you could still pick and choose by setting SHELL as a target-specific variable only for those targets you wanted to use the shell.
One downside of this is that if you have recipes that have multiple lines you'll get an email for each line individually. You can work around this by enabling .ONESHELL: to force the entire recipe to be passed to a single shell. However, I believe that this may require your mailshell tool to be more sophisticated.

If you only have one command per recipe, you can do this by changing the configuration for the shell the commands are run in.
Have the config file run the pre commands directly and trap EXIT to run the after commands in.
For example:
$ cat Makefile
SHELL := BASH_ENV=/dev/fd/3 3<<<'echo before; trap "echo after" EXIT' /bin/bash
default:
echo default
other:
echo first
echo second
$ make default
echo default
before
default
after
However this may not be what you want if a recipe runs several commands, as the before and after code will run each time.
$ make other
echo first
before
first
after
echo second
before
second
after
And I don't know of a way (outside of recursive Makefiles) to have different recipes use different shells, so this won't work if you only want to set before/after for several recipes.

Related

Interactive input of a Makefile variable

I want to run makefile with input variable. What I want is that if I write down the project name, a folder with that name will be created.
So I write read command:
CC = gcc
CFLAGS = -W -Wall
FILE := hi
src = $(wildcard *.c)
OBJ = $(src:.c=.o)
all : $(FILE)
$(FILE) : $(OBJ)
$(CC) $(CFLAGS) -o $# $^
.PHONY: clean
clean :
rm *.o $(FILE)
move :
mkdir -p ../../bin/$(FILE);
mkdir -p ../../exe/$(FILE);
mv *.o ../../bin/$(FILE);
mv $(FILE) ../../exe/$(FILE)
afterclean :
rm ../../bin/$(FILE)/*.o;
rm ../../exe/$(FILE)/$(FILE)
execute :
./../../exe/$(FILE)/$(FILE)
read :
#read -p "Enter : " enter; \
$(FILE) := enter; \
echo $FILE
What I wanna do is if I get FILE name through read I want to change FILE variable, but I can't change it. How can I do that?
Well in short, you cannot easily do that (and you should likely not want to, scroll down for rationale). If you have a closer look at your Makefile you'd notice that you're mixing make and shell syntax... and their contexts.
In your case, it literally passes the following string to shell (value of SHELL, likely defaults to /bin/sh) with -c:
read -p "Enter : " enter; \
hi := enter; \
echo ILE
Which shows the effects of the intermixed syntax. $(FILE) (value hi) and $F (unset -> empty) are make variables substituted by make before invoking shell. (while read into enter variable is not used at all and instead literal string enter is used in attempted make variable assignment inside that running shell.)
If you wanted to run a shell command and assign a value from what it has done / learned to a make variable, you would have to do so using shell function (or generate a (temporary) file you would include, but that's even messier):
FILE := $(shell read -p "Enter: " enter ; echo $${enter})
That however always asks... unless you use conditional assignment (?=) in which case you could choose already from the command line (make FILE=something, at which point we're about to close the circle). I am generally unsure what your intentions were how to tell make when to ask and when to use default value of hi.
That leads me to why this notion sounds suspect to me to start with and why suggestion made by #HolyBlackCat is a superior way of customizing invocation of make.
Also any runtime user interactions generally break automation (which is what we have make for) and also make builds non-reproducible. So, they better are to be avoided.
In other words, if you really had to, I'd say write an interactive_make_call.sh around it for this type of invocation:
#!/bin/bash
read -p "Enter : " enter
make FILE="${enter}" "$#"
Or even:
#!/bin/bash
read -p "Enter : " enter
if [[ -n "${enter}" ]] ; then
make FILE="${enter}" "$#"
else
make "$#"
fi
To fallback on the default value of FILE from the Makefile if you just press enter.

Makefile result message during parallel build

Given a Makefile that's often times run with the -j flag for parallel builds. I want it to terminate with a result message. I would like this message to say if the build failed, and if it failed, what the error was. It doesn't have to say anything if the build succeeded (although it could) but it must warn the user when a target failed to build and why.
This behavior is already there during sequential builds, but not during parallel builds. Parallel builds interweaves the output and an error message is often overlooked because output from other targets might push the failed target's error off screen. A careless developer might see no errors on his/her screen and assume the build succeeded.
It's quite an intuitive feature and I've searched for an answer, but it doesn't seem like there's any straight forward solutions. Any ideas?
You basically run
make -j 8 2> >(tee /tmp/error.log)
test $? -ne 0 && echo "build errors:"
cat /tmp/error.log
and you get all of stderr after the build finishes.
-- EDIT --
Updating to use tee, to output on stdout and into file:
Make returns non-zero if one of its recipe's fails so you could do something like this from the command line (assuming bash shell):
make 2>&1 | tee build.log
[ ${PIPESTATUS}[0] -eq 0 ] || ( echo "MAKE FAILED!"; grep --color build.log "Error:" )
The ${PIPESTATUS}[0] gives you the exit code of the first command (make 2>&1) as opposed to the exit status of the entire command (which would the exit status of tee if the make failed). It is bash specific, so it won't work in zsh for example.
Alternatively you could add the same logic as the top level target of a recursive make.
ifndef IN_RECURSION
export IN_RECURSION:=1
$(info At top level -- defining default target)
_default:
#echo "doing recursive call of make"
#$(MAKE) $(MAKECMDGOALS) IN_RECURSION=1 2>&1 | tee build.log; \
[ ${PIPESTATUS}[0] -eq 0 ] || ( echo "MAKE FAILED!"; grep --color "Error:" build.log )
.PHONY: _default
endif
all:
....
Note that in this case the \ used to catinate the two recipe lines is crucial, as the second command must run in the same shell instance as the first.

How to compare two shell command output in Makefile?

My Makefile is:
.PHONY: check
check:
ifneq $(shell echo 123), $(shell echo 123)
$(error Not equal)
endif
When I run, I've got the error:
$ make
Makefile:3: *** Not equal. Stop.
But this should happen only when they're different, but they're not. Why?
ifneq cannot be indented. the way you've written it, it's being run via a shell command which means the $(error) is being evaluated first by the make command.
i'm guessing you want the make check to actually run two commands only when make check is invoked, and compare their output. you can do:
.PHONY: check
check:
if [ "`echo 123`" != "`echo 123`" ]; then \
echo "Not equal"; \
exit 1; \
fi
According to GNU Make docs, Conditional Parts cannot be used to control shell commands at the time of execution, since conditionals control what make actually "sees" in the makefile.
So to perform condition during compilation process, shell syntax is preferred, e.g.
SHELL := /bin/bash -e
.PHONY: check
check:
#test "$(shell echo 123)" = "$(shell echo 123)" \
|| { echo Not equal; exit 2; } \
&& { echo Equal; }

Delayed Windows cmd echo with GNU Make environment

I have few simple targets which create some files for me.
Example:
$(MAKE_INA):
#echo Building ASM compilation flags file $(notdir $(MAKE_INA))
#$(foreach i, $(sort $(ASMFLAGS) $(PFLAGS) $(ALL_INC_DIR) $(cppGetPreProcessorDefines)), $(shell echo $i >> $# ))
The target works fine, the file is being created and echo text displayed, but in that order (first the file is build then the echo is shown on cmd.exe console).
I guess that is related somehow with output buffering, but I was not able to find the way to flush the echos immediately.
Any hint? Is it even possible?
I am using Gnu Make 4.0
You are mixing up contexts here.
The first #echo line is a recipe line and is run by the shell when the target runs.
The second $(foreach) line is within the rule but is a make context line and is evaluated by make before running the recipe lines. Within that line $(shell) is also a make command and is run during the make expansion of the recipe instead of being run by the shell at recipe execution time.
To do what you want you can just use:
$(MAKE_INA):
#echo Building ASM compilation flags file $(notdir $(MAKE_INA))
#printf "%s\\n" $(sort $(ASMFLAGS) $(PFLAGS) $(ALL_INC_DIR) $(cppGetPreProcessorDefines)) >> $#
Which does the echoing at recipe execution time (so has the right order) and uses a single call to the printf built-in to output to the file instead of running N calls to echo.
Edit: For Windows cmd.exe compat you need to use echo $i >> $# & as the $(foreach) body so that cmd.exe runs multiple commands correctly.
If you did want to keep the N echo calls then you could use:
$(MAKE_INA):
#echo Building ASM compilation flags file $(notdir $(MAKE_INA))
#$(foreach i, $(sort $(ASMFLAGS) $(PFLAGS) $(ALL_INC_DIR) $(cppGetPreProcessorDefines)), echo $i >> $#; ))
Which has the $(foreach) output echo XXX >> $#; ....; echo ZZZ >> $#; as the recipe line to then execute during recipe execution.

How to source a script in a Makefile?

Is there a better way to source a script, which sets env vars, from within a makefile?
FLAG ?= 0
ifeq ($(FLAG),0)
export FLAG=1
/bin/myshell -c '<source scripts here> ; $(MAKE) $#'
else
...targets...
endif
Makefile default shell is /bin/sh which does not implement source.
Changing shell to /bin/bash makes it possible:
# Makefile
SHELL := /bin/bash
rule:
source env.sh && YourCommand
To answer the question as asked: you can't.
The basic issue is that a child process can not alter the parent's environment. The shell gets around this by not forking a new process when source'ing, but just running those commands in the current incarnation of the shell. That works fine, but make is not /bin/sh (or whatever shell your script is for) and does not understand that language (aside from the bits they have in common).
Chris Dodd and Foo Bah have addressed one possible workaround, so I'll suggest another (assuming you are running GNU make): post-process the shell script into make compatible text and include the result:
shell-variable-setter.make: shell-varaible-setter.sh
postprocess.py #^
# ...
else
include shell-variable-setter.make
endif
messy details left as an exercise.
If your goal is to merely set environment variables for Make, why not keep it in Makefile syntax and use the include command?
include other_makefile
If you have to invoke the shell script, capture the result in a shell command:
JUST_DO_IT=$(shell source_script)
the shell command should run before the targets. However this won't set the environment variables.
If you want to set environment variables in the build, write a separate shell script that sources your environment variables and calls make. Then, in the makefile, have the targets call the new shell script.
For example, if your original makefile has target a, then you want to do something like this:
# mysetenv.sh
#!/bin/bash
. <script to source>
export FLAG=1
make "$#"
# Makefile
ifeq($(FLAG),0)
export FLAG=1
a:
./mysetenv.sh a
else
a:
.. do it
endif
Using GNU Make 3.81 I can source a shell script from make using:
rule:
<tab>source source_script.sh && build_files.sh
build_files.sh "gets" the environment variables exported by source_script.sh.
Note that using:
rule:
<tab>source source_script.sh
<tab>build_files.sh
will not work. Each line is ran in its own subshell.
This works for me. Substitute env.sh with the name of the file you want to source. It works by sourcing the file in bash and outputting the modified environment, after formatting it, to a file called makeenv which is then sourced by the makefile.
IGNORE := $(shell bash -c "source env.sh; env | sed 's/=/:=/' | sed 's/^/export /' > makeenv")
include makeenv
Some constructs are the same in the shell and in GNU Make.
var=1234
text="Some text"
You can alter your shell script to source the defines. They must all be simple name=value types.
Ie,
[script.sh]
. ./vars.sh
[Makefile]
include vars.sh
Then the shell script and the Makefile can share the same 'source' of information. I found this question because I was looking for a manifest of common syntax that can be used in Gnu Make and shell scripts (I don't care which shell).
Edit: Shells and make understand ${var}. This means you can concatenate, etc,
var="One string"
var=${var} "Second string"
I really like Foo Bah's answer where make calls the script, and the script calls back to make. To expand on that answer I did this:
# Makefile
.DEFAULT_GOAL := all
ifndef SOME_DIR
%:
<tab>. ./setenv.sh $(MAKE) $#
else
all:
<tab>...
clean:
<tab>...
endif
--
# setenv.sh
export SOME_DIR=$PWD/path/to/some/dir
if [ -n "$1" ]; then
# The first argument is set, call back into make.
$1 $2
fi
This has the added advantage of using $(MAKE) in case anyone is using a unique make program, and will also handle any rule specified on the command line, without having to duplicate the name of each rule in the case when SOME_DIR is not defined.
If you want to get the variables into the environment, so that they are passed to child processes, then you can use bash's set -a and set +a. The former means, "When I set a variable, set the corresponding environment variable too." So this works for me:
check:
bash -c "set -a && source .env.test && set +a && cargo test"
That will pass everything in .env.test on to cargo test as environment variables.
Note that this will let you pass an environment on to sub-commands, but it won't let you set Makefile variables (which are different things anyway). If you need the latter, you should try one of the other suggestions here.
My solution to this: (assuming you're have bash, the syntax for $# is different for tcsh for instance)
Have a script sourceThenExec.sh, as such:
#!/bin/bash
source whatever.sh
$#
Then, in your makefile, preface your targets with bash sourceThenExec.sh, for instance:
ExampleTarget:
bash sourceThenExec.sh gcc ExampleTarget.C
You can of course put something like STE=bash sourceThenExec.sh at the top of your makefile and shorten this:
ExampleTarget:
$(STE) gcc ExampleTarget.C
All of this works because sourceThenExec.sh opens a subshell, but then the commands are run in the same subshell.
The downside of this method is that the file gets sourced for each target, which may be undesirable.
Depending on your version of Make and enclosing shell, you can implement a nice solution via eval, cat, and chaining calls with &&:
ENVFILE=envfile
source-via-eval:
#echo "FOO: $${FOO}"
#echo "FOO=AMAZING!" > $(ENVFILE)
#eval `cat $(ENVFILE)` && echo "FOO: $${FOO}"
And a quick test:
> make source-via-eval
FOO:
FOO: AMAZING!
An elegant solution found here:
ifneq (,$(wildcard ./.env))
include .env
export
endif
If you need only a few known variables exporting in makefile can be an option, here is an example of what I am using.
$ grep ID /etc/os-release
ID=ubuntu
ID_LIKE=debian
$ cat Makefile
default: help rule/setup/lsb
source?=.
help:
-${MAKE} --version | head -n1
rule/setup/%:
echo ID=${#F}
rule/setup/lsb: /etc/os-release
${source} $< && export ID && ${MAKE} rule/setup/$${ID}
$ make
make --version | head -n1
GNU Make 3.81
. /etc/os-release && export ID && make rule/setup/${ID}
make[1]: Entering directory `/tmp'
echo ID=ubuntu
ID=ubuntu
--
http://rzr.online.fr/q/gnumake
Assuming GNU make, can be done using a submake. Assuming that the shell script that exports the variables is include.sh in the current directory, move your Makefile to realmake.mk. Create a new Makefile:
all:
#. ./include.sh; \
$(MAKE) -f realmake.mk $(MAKECMDGOALS)
$(MAKECMDGOALS):
+#. ./include.sh; \
$(MAKE) -f realmake.mk $(MAKECMDGOALS)
Pay attention to the ./ preceding include.sh.
Another possible way would be to create a sh script, for example run.sh, source the required scripts and call make inside the script.
#!/bin/sh
source script1
source script2 and so on
make
target: output_source
bash ShellScript_name.sh
try this it will work, the script is inside the current directory.

Resources