Running a binary using makefile - makefile

From all targets I have, I need one to run the program.
From command line, I would state:
$ make main_run.err
or
$ make main_time.err
If it was specific to a certain program, I would write the rule:
main_run.err main_time.err: main.x
#echo ------------------- $(DEFSYM) Running ---------------------
/usr/bin/time -p -o main_time.err ./main.x > main_run.err || true
#echo -----------------------------------------------------------
Now, how to generalize it?
My first attempt was:
Command line:
$ make run main.x
And the rule:
run: $(filter-out $#,$(MAKECMDGOALS))
#echo --------------- $(DEFSYM) Running -----------------
/usr/bin/time -p -o time.err ./$< > run.err || true
#echo ---------------------------------------------------
This gave me some strange errors:
make: Circular run <- run dependency dropped.
------------- BUILD_150729_162018 Running -----------------
/usr/bin/time -p -o time.err ./main.x > run.err || true
-----------------------------------------------------------
make: 'main.x' is up to date.
My second and best attempt, the one I'm asking for help (unless there is another really easier way), is:
%_run.err %_time.err: %.x
#echo ------------------- $(DEFSYM) Running ---------------------
/usr/bin/time -p -o $#_time.err ./$< > $#_run.err || true
#echo -----------------------------------------------------------
This is almost ok! But filenames are wrong:
--------------- BUILD_150729_162239 Running -----------------
/usr/bin/time -p -o main_run.err_time.err ./main.x > main_run.err_run.err || true
-------------------------------------------------------------
And the command line is not good:
$ make main_run.err
I know I could use:
$ make run x=main
And define a variable $(x), but I prefer:
$ make run main.x
if possible.

This attempt:
%_run.err %_time.err: %.x
#echo ------------------- $(DEFSYM) Running ---------------------
/usr/bin/time -p -o $#_time.err ./$< > $#_run.err || true
#echo -----------------------------------------------------------
Is exactly the right idea. You just need to adjust the automatic variables you are using to match the change to a pattern rule.
Specifically you want the $* (stem) variable instead of the $# (target) variable.
From the manual:
$#
The file name of the target of the rule. If the target is an archive member, then ‘$#’ is the name of the archive file. In a pattern rule that has multiple targets (see Introduction to Pattern Rules), ‘$#’ is the name of whichever target caused the rule’s recipe to be run.
$*
The stem with which an implicit rule matches (see How Patterns Match). If the target is dir/a.foo.b and the target pattern is a.%.b then the stem is dir/foo. The stem is useful for constructing names of related files.
In a static pattern rule, the stem is part of the file name that matched the ‘%’ in the target pattern.
So you end up with:
%_run.err %_time.err: %.x
#echo ------------------- $(DEFSYM) Running ---------------------
/usr/bin/time -p -o $*_time.err ./$< > $*_run.err || true
#echo -----------------------------------------------------------
The original original:
main_run.err main_time.err: main.x
#echo ------------------- $(DEFSYM) Running ---------------------
/usr/bin/time -p -o main_time.err ./main.x > main_run.err || true
#echo -----------------------------------------------------------
Was incorrect because it did not tell make that both files were created by a single run of the target so make might have run it twice if you specified both main_run.err and main_time.err at the same time. The pattern rule version does tell make about that fact (see the last paragraph of the manual section on Pattern Rule Examples for an explanation of that).
To make the initial command line a bit prettier you could add:
.PHONY: run_%
run_%: %_run.err %_time.err
Which will let you run make run_main.
...
I say that but in a quick mock-up test that doesn't work and it needs to be:
.PHONY: run_%
run_%: %_run.err %_time.err ;
instead but I'm not at all sure why offhand.

Related

Why does make copy a file onto another file? (Target depends on an entire folder.)

I have a directory with test inputs and outputs. I wanted make to automatically test my program against this directory after build, for convenience. Thus I needed to somehow force the test target of Makefile to depend on the entire testing directory (it's called good, because it contains valid inputs and outputs for the program)
I read this question and the accepted answer and the comments about deleted files under this answer: Makefile rule that depends on all files under a directory (including within subdirectories) And, incorporating advice from this answer & comments, I came out with this:
my#comp:~/wtfdir$ cat Makefile
test : test.sh $(shell find good)
./test.sh
my#comp:~/wtfdir$
For the sake of MCVE, test.sh is very rudimentary:
my#comp:~/wtfdir$ cat test.sh
echo "blah"
my#comp:~/wtfdir$
However, I noticed, this behaves in a rather unexpected way:
my#comp:~/wtfdir$ ls good
test1 test1.out
my#comp:~/wtfdir$ make
./test.sh
blah
my#comp:~/wtfdir$ touch good/test1
my#comp:~/wtfdir$ make
cp good/test1 good/test1.out
./test.sh
blah
my#comp:~/wtfdir$
Why (expletive redacted) does modifying test1 cause make to overwrite test1.out with test1??? I'm not a big fan of data losses, you know.
What's going on here?
Your Make appears to be GNU Make. Here's why this happens. Your recipe:
test : test.sh $(shell find good)
./test.sh
adds to the prerequisites of test every file and directory that is listed
by find good in the current directory, which happen to be:
good
good/test1
good/test1.out
So to make target test, Make begins by determining if any of the specified
or built-in recipes require it to rebuild any of the prerequsities:
test.sh good good/test1 good/test1.out
Among its built-in recipes it finds:
%.out: %
# recipe to execute (built-in):
#rm -f $#
cp $< $#
as you can verify by running:
$ make --print-data-base | grep -A4 '%.out'
The rule for this recipe is matched by:
good/test1.out: good/test1
and by doing:
$ touch good/test1
you have made good/test1.out out of date with respect to good/test1.
So make executes the recipe:
#rm -f good/test1.out
cp good/test1 good/test1.out
the visible output of which is what you observed:
cp good/test1 good/test1.out
Then it proceeds with the recipe for test:
./test.sh
blah
There is always a risk of such booby-traps if you write a makefile that blindly
generates at runtime some set of preqrequisites or targets you don't know beforehand.
You could avoid this one in particular by explicitly deleting the offending
implicit pattern rule in your makefile by writing:
%.out: %
with no recipe. And you can avoid all possible booby-traps of this sort by disabling all
built-in recipes, with:
$ make --no-builtin-rules ...
but that will require you to write for yourself any builtin-recipes that your
makefile relies on.
The best solution for you is probably to amend your makefile as follows:
PREREQS := $(shell find good)
test : test.sh $(PREREQS)
./test.sh
$(PREREQS): ;
Then the last line explicitly specifies an empty recipe
for each of the $(PREREQS), and Make will not consult any pattern rules for targets
that have explicit recipes.
You should additionally make test a phony target:
.PHONY: test
for the avoidance of the booby-trap where something creates a file called test in the build directory.

How to loop against directories

I am trying to build a generic task that will execute other task. What I need it to do is to loop against directories and use each dir name executing other task for it.
This is what I have:
# GENERIC TASKS
all-%:
for BIN in `ls cmd`; do
#$(MAKE) --no-print-directory BIN=$(BIN) $*
done
But I get this error, could anyone explain to me how can I make it work
bash
➜ make all-build
for BIN in `ls cmd`; do
/bin/sh: -c: line 1: syntax error: unexpected end of file
make: *** [all-build] Error 2
UPDATE
this is how the complete flow of my makefile looks like:
all-%:
for BIN in `ls cmd`; do \
#$(MAKE) --no-print-directory BIN=$BIN $*; \
done
build-%:
#$(MAKE) --no-print-directory BIN=$* build
build:
docker build --no-cache --build-arg BIN=$(BIN) -t $(BIN) .
Each line of a make-recipe is executed in a distinct invocation of the shell.
Your recipe fails with a shell-syntax error because this line:
for BIN in `ls cmd`; do
is not a valid shell command. Nor is the third line:
done
To have all three lines executed in a single shell you must join them
into a single shell command with make's line-continuation character \:
# GENERIC TASKS
all-%:
for BIN in `ls cmd`; do \
#$(MAKE) --no-print-directory BIN=$$BIN $*; \
done
Note also BIN=$$BIN, not $(BIN). BIN is a shell variable here, not a make variable: $$ escapes $-expansion by make, to preserve the shell-expansion $BIN.
Using ls to drive the loop in Make is an antipattern even in shell script (you want for f in cmd/* if I'm guessing correctly) but doubly so in a Makefile. A proper design would be to let make know what the dependencies are, and take it from there.
all-%: %: $(patsubst cmd/%,%,$(wildcard cmd/*))
$(MAKE) --no-print-directory -$(MAKEFLAGS) BIN=$< $*

No rule to make target but the rule exists

I wanted to write some Makefile for testing C code via Unity test framework.
However when I want to test it I get the message make: *** No rule to make target 'unity_build/results/test_unity_dumb_example.unity_res', needed by 'unity_test'. Stop. . I can not figure out what is wrong because there is the rule for files that match the pattern: $(UNITY_PATHR)%.unity_res:.
Here is beginning of my Makefile:
UNITY_PATHU = ${UNITY_INSTALL_DIR}/
UNITY_PATHS = src/
UNITY_PATHS += src/unity_dumb_example/
UNITY_PATHT = unity_test/
UNITY_PATHB = unity_build/
UNITY_PATHD = unity_build/depends/
UNITY_PATHO = unity_build/objs/
UNITY_PATHR = unity_build/results/
UNITY_BUILD_PATHS = $(UNITY_PATHB) $(UNITY_PATHD) $(UNITY_PATHO) $(UNITY_PATHR)
# Tell compiler where to look for all test files
UNITY_SRCT = $(wildcard $(UNITY_PATHT)*.c)
UNITY_COMPILER=gcc -c
UNITY_LINKER=gcc
UNITY_DEPEND=gcc -MM -MG -MF
UNITY_CFLAGS=-I. -I$(UNITY_PATHU) -I$(UNITY_PATHS) -DTEST
UNITY_RESULTS = $(patsubst $(UNITY_PATHT)test_%.c,$(UNITY_PATHR)test_%.unity_res,$(UNITY_SRCT))
unity_test: $(UNITY_BUILD_PATHS) $(UNITY_RESULTS)
#echo "-----------------------\nIGNORES:\n-----------------------"
#echo `grep -s IGNORE $(UNITY_PATHR)*.unity_res`
#echo "-----------------------\nFAILURES:\n----------------------"
#echo `grep -s FAIL $(UNITY_PATHR)*.unity_res`
#echo "\nDONE"
$(UNITY_PATHR)%.unity_res: $(UNITY_PATHB)%.out
./$< > $# 2>&1
In GNU make manual I have read
The target is a pattern for matching file names; the ‘%’
matches any nonempty substring, while other characters match only themselves.
I do not understand why make complains because there is no misspell.
EDIT:
After full clean the output is as follows:
mkdir -p unity_build/
mkdir -p unity_build/depends/
mkdir -p unity_build/objs/
mkdir -p unity_build/results/
make: *** No rule to make target 'unity_build/results/test_unity_dumb_example.unity_res', needed by 'unity_test'. Stop.
All the needed paths exist.
When I run make in debug mode -d I can see that make is trying pattern rule with stem test_test_unity_dumb_example instead of test_unity_dumb_example, for example:
Trying pattern rule with stem 'test_test_unity_dumb_example'
test_test_ was my mistake but I have fixed it. I still can't make it work.
When I run with -p I can find something like this in the output:
# Implicit Rules
unity_build/results/%.unity_res: unity_build/%.out
# recipe to execute (from 'Makefile.unity', line 30):
./$< > $# 2>&1
SOLVED
The problem was with the prerequisite of the prerequisite of the $(UNITY_PATHB)%.out. Precisely, path to one crucial source file was ${UNITY_INSTALL_DIR}/src instead of ${UNITY_INSTALL_DIR}/ .
However I still find it weird that make was complaining about the rule to target that is 2 level above the target that could not be built.

How to execute a list of checks with Gnu Make?

I have a Makefile that should execute some simple commands on a set of imput files. Each does not have an output, just the success of the command should be checked:
# Makefile
check: test1 test2 test3
test1: one.xml
xmllint --schema my.xsd $#
test2: two.xml
xmllint --schema my.xsd $#
test3: three.xml
xmllint --schema my.xsd $#
I would like to have this more generic, that I don't have to list identical rules for each file. Which is especially inconvenient when I for example use $(wildcard ...):
# Makefile (sketch only)
XMLS:=$(wildcard *.xml)
check: xmllint
xmllint: %.xml # this will not work will it?
xmllint --schema my.xsd $#
The last rule isnor correct, of course, since the % is on the right side:
$ make check
*** No rule to make target '%.xml', needed by 'xmllint'. Stop.
How do I connect the $(XMLS) input files with the xmllint: ... rule, so that on make check the program xmllint is executed on all xml input files?
.PHONY: all check
# The files to be tested
XMLS:=$(wildcard *.xml)
# Turning the filenames into *test* names
XMLTESTS:=$(patsubst %.xml,%.test,$(XMLS))
# "make check" should run all tests
check: $(XMLTESTS)
# Each test depends on the XML file
# Now we have a pattern on both sides of the generic rule
%.test: %.xml
xmllint --schema my.xsd $<
Note that, for some (to me) unknown reason, putting .PHONY after the declarations and adding $(XMLTESTS) to it results in "nothing to be done for target 'all'". I have no idea why.
EDIT: .PHONY: does not work for implicit rules and pattern rules.

Suppress "Clock skew" warning for future-times in Makefile

I have a Makefile that does performs a task if it hasn't happened in the last hour. It does so like this:
HOUR_FROM_NOW = $(shell perl -e '($$s,$$m,$$h,$$d,$$M)=localtime(time()+3600); printf("%02d%02d%02d%02d\n",$$M+1,$$d,$$h,$$m);')
NOW_FILE = $(shell mkdir -p .make; touch .make/now; echo .make/now )
.PHONY: externals
externals: $(PROJECTS:%=.make/proj_%)
.make/proj_%: $(NOW_FILE)
$(MAKE) -s $(*F)
touch -t $(HOUR_FROM_NOW) $#
.PHONY: $(PROJECTS)
$(PROJECTS):
# do stuff, specifically, clone git-repo if not exists, else pull latest
That part works great, except that I now get warnings:
make: Warning: File `.make/proj' has modification time 3.5e+03 s in the future
make: Nothing to be done for `externals'.
make: warning: Clock skew detected. Your build may be incomplete.
Anyone know how to suppress those warnings? (Or to do a periodic task in a makefile)
Most versions of touch I have come across can do some date time maths which allows for setting the timestamp of a file directly via the --date option.
That and the fact that variables assigned with := are only "evaluated once" makes this a bit easier to read.
HOUR_AGO := .make/hour_ago
__UGLY := $(shell mkdir -p .make && touch --date='1hour ago' $(HOUR_AGO))
# The preceding line will be executed once
.make/proj_%: .make/hour_ago | .make
$(MAKE) -s $(*F)
#touch $#
.make:
mkdir -p $#
I'm using something very similar to this to periodically refresh login tokens.
Never would have thought of it if it wasn't for Dave's answer though.
The directory is created by specifying it as a order-only-prerequisite
I suspect that the + 3600 is at fault. What happens if you remove it?
I thought and thought, and then the stupid-obvious solution hit me ...
Instead of setting timestamps in the future with HOUR_FROM_NOW, I use the real time and compare with HOUR_AGO_FILE ...
HOUR_AGO = $(shell perl -e '($$s,$$m,$$h,$$d,$$M)=localtime(time()-3600); printf("%02d%02d%02d%02d\n",$$M+1,$$d,$$h,$$m);')
HOUR_AGO_FILE = $(shell mkdir -p .make; touch -t $(HOUR_AGO) .make/hour_ago; echo .make/hour_ago )
.PHONY: externals
externals: $(PROJECTS:%=.make/proj_%)
.make/proj_%: $(HOUR_AGO_FILE)
$(MAKE) -s $(*F)
#touch $#

Resources