gnu makefile when targets and dependencies are dynamic - makefile

I'm trying to create a makefile that creates a subsystem makefile if it does not exists, and then runs the make to create the subsystem using that subsystem makefile (using "echo" for simplicity)
If everything was static then I could use the following pattern
build_dir/config/makefile :
mkdir -p build_dir/config
echo create makefile > build_dir/config/makefile
use_make : build_dir/config/makefile
echo local makefile using `ls -l build_dir/config/makefile`
My problem is that the "config" is defined as part of the target (e.g Debug or Release)
I tried the following
.SECONDEXPANSION:
tgt_d : CONFIG = Debug
tgt_r : CONFIG = Release
tgt_d tgt_r : MAKEFILE_DIR = build_dir/${CONFIG}
${MAKEFILE_DIR}/makefile :
mkdir -p ${MAKEFILE_DIR}
echo create makefile > ${MAKEFILE_DIR}/makefile
use_make : ${MAKEFILE_DIR}/makefile
echo local makefile using `ls -l ${MAKEFILE_DIR}/makefile`
tgt_d tgt_r : use_make
My problem is that make is interpreting the ${MAKEFILE_DIR} in the target and the prerequisite directly (when it's undefined) not in a target-specific way (even when I added .SECONDEXPANSION or use ${${MAKEFILE_DIR}} - see output below
How can I achieve what I need to do ?
The next step is to define this in a general way so that I can use it for multiple targets (multiple Eclipse projects that are build together) using $< and % - so kudos to someone that takes me that extra step!
Thanks !
$ make --debug=v tgt_d
GNU Make 4.3
Built for x86_64-pc-cygwin
Copyright (C) 1988-2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Reading makefiles...
Reading makefile 'makefile'...
Updating makefiles....
Updating goal targets....
Considering target file 'tgt_d'.
File 'tgt_d' does not exist.
Considering target file 'use_make'.
File 'use_make' does not exist.
Considering target file 'build_dir//makefile'.
File 'build_dir//makefile' does not exist.
Finished prerequisites of target file 'build_dir//makefile'.
Must remake target 'build_dir//makefile'.
mkdir -p build_dir/Debug
echo create makefile > build_dir/Debug/makefile
Successfully remade target file 'build_dir//makefile'.
Finished prerequisites of target file 'use_make'.
Must remake target 'use_make'.
echo local makefile using `ls -l build_dir/Debug/makefile`
local makefile using -rw-r--r-- 1 yonatan.lehman 1049089 16 Oct 30 12:18 build_dir/Debug/makefile
Successfully remade target file 'use_make'.
Finished prerequisites of target file 'tgt_d'.
Must remake target 'tgt_d'.
Successfully remade target file 'tgt_d'.

The manual is pretty clear about when target-specific variables are in use and when they are not:
As with automatic variables, these values are only available within the context of a target’s recipe (and in other target-specific assignments).
So, clearly your use of them in targets and prerequisites cannot work. Secondary expansion has nothing to do with this (in any event, you're not actually using any secondary expansion here even though you've enabled the feature).
I think you'll have to use a recursive invocation of make:
tgt_d tgt_r :
$(MAKE) use_make MAKEFILE_DIR=build_dir/${CONFIG}

Related

Multiple rules for one target

I have large project and it's compiled on-demand on several different machines with different OS'es.
One of targets has foo.res in it's dependencies. This resource file can be compiled using either one of 2 different rules:
First rule requires text file foo.txt to be present. This rule also requires some of the non-cross platform software to be installed in the system. This rule can not be used in-field by the our engineers, as it won't run under old MacOS.
Second rule uses pre-compiled resource file foo.res in resources directory and simply copies it to build/. Obviously, this is unwanted workaround, but i have no other choice.
I've tried to describe 2 rules in one Makefile, but i've got such error:
Makefile:78: warning: overriding recipe for target 'build/foo.res'
Makefile:75: warning: ignoring old recipe for target 'build/foo.res'
Currently I use Makefile with one of the rules commented-out:
.PHONY: compile
build/app.bin: some_files... build/foo.res
(Here is compiler call)
build/:
-mkdir build/
.PHONY: clear
clear:
-rm -rf ./build
... Lots of stuff here ...
# build/foo.res: res/foo.res | build/.
# cp res/foo.res build/foo.res
build/foo.res: res/foo.txt | build/.
some_app -o build/foo.res res/foo.txt
I deploy pre-compiled res/foo.res to some machines and then I swap commented rules, so regular cp is used. This works fine, until some changes are introduced to Makefile in some of commits. Then it blocks fast-forward update in git.
How can I configure make to fire only one of the rules, based on matched dependencies?
You can't do this with explicit rules. An explicit rule tells make THIS is how you build this target. You can create two different pattern rules that can build the same target. A pattern rule says this is one way you can build that target.
So, change your makefile to:
build/%.res: res/%.res | build/.
cp $< $#
build/%.res: res/%.txt | build/.
some_app -o $# $<

Makefile rule with multiple targets for single invocation

According to the gnu make documentation, if a rule generates multiple targets by a single invocation (for instance with a recipe executing a tool with multiple output files), you can use the '&:' rule syntax to tell make.
I'm getting warnings when using this syntax, about target '&', however, when having multiple (but unique) targets in multiple rules. As if make mistakes the ampersand for a target name instead of being part of the target-prerequisite separator.
In my original project I've got two rules having multiple targets and a recipe generating those targets from a single statement/tool. The targets are unique for each of the two rules. I've created the following simple example to demonstrate the warning generated by make:
all: file_abbc
.PHONY: all clean
clean:
del /Q file_*
file_abbc: file_ab file_bc
copy file_ab+file_bc file_abbc
file_ab file_bc &: file_a file_b file_c
copy file_a+file_b file_ab
copy file_b+file_c file_bc
file_a file_b file_c &: content
copy content file_a
copy content file_b
copy content file_c
Warnings from running make on Windows on the above:
Makefile:17: warning: overriding recipe for target '&'
Makefile:13: warning: ignoring old recipe for target '&'
Why is make complaining about target '&' ?
You're using an old version of GNU make that doesn't understand &: rules, so just treats & as a file name. Use make --version to see which version you are running and upgrade if it is older that 4.3

Why is Make ignoring an explicit pattern rule?

I'm using GNU Make 4.0 to compile code on an IBM i system. Make is inexplicably choosing the wrong rule to build one type of object.
On this platform, program objects are built from modules, which are compiled from source code. There is also a convenience shortcut command that will create a program directly from a single piece of source code by creating a temporary module from the source code and then building a program from that. The problem I'm running into is that Make is using the shortcut command (crtbndrpg) instead of the two-step version (crtrpgmod + crtpgm), even though the target rule specifies that the program should be built from a module and not the shortcut.
There are two makefiles: a generic one that describes how to create IBM i objects, and a project-specific one that describes object dependencies for all items in this project and includes the generic one. My generic makefile looks like this (edited for simplicity):
# `IBMiMake`, a generic makefile that describes how to create IBM i objects.
OBJPATH := $(CURDIR)
override OBJPATH := $(shell echo "$(OBJPATH)" | tr '[:lower:]' '[:upper:]')
%.MODULE: %.RPGLE
$(eval crtcmd := crtrpgmod module($(OBJLIB)/$*) srcstmf('$<') $(CRTRPGMODFLAGS))
#system "$(crtcmd)" > $(LOGPATH)/$(notdir $<).log
%.PGM: %.RPGLE
$(eval crtcmd := crtbndrpg pgm($(OBJLIB)/$*) srcstmf('$<') $(CRTBNDRPGFLAGS))
system "$(crtcmd)" >$(LOGPATH)/$(notdir $<).log 2>&1
%.PGM:
$(eval crtcmd := crtpgm pgm($(OBJLIB)/$*) module($(basename $(filter %.MODULE,$(notdir $^)))) $(CRTPGMFLAGS))
system "$(crtcmd)" >$(LOGPATH)/$#.log 2>&1
The project-specific makefile looks like this (also edited for simplicity):
# `xpmake`, to create objects in this project.
ROOTDIR := $(shell dirname $(realpath $(lastword $(MAKEFILE_LIST))))
IBMIMAKE := $(ROOTDIR)/../SDE/IBMiMake
include $(IBMIMAKE)
# AB2001.B.MODULE -- CRTRPGMOD
AB2001.B.MODULE: AB2001.B.RPGLE
# AB2001.B.PGM -- CRTPGM
AB2001.B.PGM: AB2001.B.MODULE
To build the object in question:
bash-4.2$ make AB2001.B.PGM OBJPATH:='/qsys.lib/xp33make.lib' -f xp33make/xpmake -d --no-builtin-rules
What should happen: It should first create the module using the crtrpgmod command, which it does. It should then create the program using the crtpgm command. Instead of creating the program via crtpgm, however, it for some reason tries to use the crtbndrpg command to directly build the program from source code. The only thing I can think of is that perhaps Make is seeing AB2001.B.MODULE as an intermediate file and opting to bypass the crtrpgmod step. Could this be true? How do I get make to follow my rules and not try to overthink things?
Here is the output:
GNU Make 4.0
Built for powerpc-ibm-aix5.3.0.0
Copyright (C) 1988-2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Reading makefiles...
Reading makefile 'xp33make/xpmake'...
Reading makefile '/home/SMEEP/Source/xp33make/../SDE/IBMiMake' (search path) (no ~ expansion)...
Updating makefiles....
Considering target file '/home/SMEEP/Source/xp33make/../SDE/IBMiMake'.
Looking for an implicit rule for '/home/SMEEP/Source/xp33make/../SDE/IBMiMake'.
No implicit rule found for '/home/SMEEP/Source/xp33make/../SDE/IBMiMake'.
Finished prerequisites of target file '/home/SMEEP/Source/xp33make/../SDE/IBMiMake'.
No need to remake target '/home/SMEEP/Source/xp33make/../SDE/IBMiMake'.
Considering target file 'xp33make/xpmake'.
Looking for an implicit rule for 'xp33make/xpmake'.
No implicit rule found for 'xp33make/xpmake'.
Finished prerequisites of target file 'xp33make/xpmake'.
No need to remake target 'xp33make/xpmake'.
Updating goal targets....
Considering target file 'AB2001.B.PGM'.
File 'AB2001.B.PGM' does not exist.
Looking for an implicit rule for 'AB2001.B.PGM'.
Trying pattern rule with stem 'AB2001.B'.
Trying implicit prerequisite 'AB2001.B.CLLE'.
Trying pattern rule with stem 'AB2001.B'.
Trying implicit prerequisite 'AB2001.B.RPGLE'.
Found an implicit rule for 'AB2001.B.PGM'.
Considering target file 'AB2001.B.RPGLE'.
Looking for an implicit rule for 'AB2001.B.RPGLE'.
No implicit rule found for 'AB2001.B.RPGLE'.
Finished prerequisites of target file 'AB2001.B.RPGLE'.
No need to remake target 'AB2001.B.RPGLE'; using VPATH name '/home/SMEEP/Source/xp33make/AB2001.B.RPGLE'.
Considering target file 'AB2001.B.MODULE'.
Looking for an implicit rule for 'AB2001.B.MODULE'.
Trying pattern rule with stem 'AB2001.B'.
Trying implicit prerequisite 'AB2001.B.C'.
Trying pattern rule with stem 'AB2001.B'.
Trying implicit prerequisite 'AB2001.B.CLLE'.
Trying pattern rule with stem 'AB2001.B'.
Trying implicit prerequisite 'AB2001.B.RPGLE'.
Found prerequisite 'AB2001.B.RPGLE' as VPATH '/home/SMEEP/Source/xp33make/AB2001.B.RPGLE'
Found an implicit rule for 'AB2001.B.MODULE'.
Pruning file '/home/SMEEP/Source/xp33make/AB2001.B.RPGLE'.
Pruning file '/home/SMEEP/Source/xp33make/AB2001.B.RPGLE'.
Finished prerequisites of target file 'AB2001.B.MODULE'.
Prerequisite '/home/SMEEP/Source/xp33make/AB2001.B.RPGLE' is older than target 'AB2001.B.MODULE'.
Prerequisite '/home/SMEEP/Source/xp33make/AB2001.B.RPGLE' is older than target 'AB2001.B.MODULE'.
No need to remake target 'AB2001.B.MODULE'; using VPATH name '/QSYS.LIB/XP33MAKE.LIB/AB2001.B.MODULE'.
Finished prerequisites of target file 'AB2001.B.PGM'.
Must remake target 'AB2001.B.PGM'.
system "crtbndrpg pgm(XP33MAKE/AB2001.B) srcstmf('/home/SMEEP/Source/xp33make/AB2001.B.RPGLE')" >/home/SMEEP/Source/xp33make/Logs/2016-11-14_11.42.55-Mon/AB2001.B.RPGLE.log 2>&1
Putting child 30016df0 (AB2001.B.PGM) PID 1155363 on the chain.
Live child 30016df0 (AB2001.B.PGM) PID 1155363
Reaping losing child 30016df0 PID 1155363
/home/SMEEP/Source/xp33make/../SDE/IBMiMake:476: recipe for target 'AB2001.B.PGM' failed
Removing child 30016df0 PID 1155363 from chain.
Your example is still difficult to read: creating a minimal example is usually best (that is, construct an example that uses touch etc. to create files, and doesn't depend on your environment).
Also, the use of $(eval ...) inside the recipe to create a make variable assignment is a bit confusing. Don't think that just because you do this in the recipe there's some kind of scoping involved: those variables are still globally assigned.
In any event, the problem is that you have two ways to build a target that matches %.PGM:
%.PGM: %.RPGLE
%.PGM:
You apparently want to use the second one, but if the stem of a matching pattern rule is has equal length (here both have the same stem, .PGM) then make will always choose the first pattern that you define so it will always choose the first one if it can.
So, it will always use crtbndrpg to build that target, as long as make can figure out how to build %.RPGLE, so that pattern rule will match.
I'm not sure if they're omitted in the simplifying process, but your explicit rules lack their recipes.
Pattern rules with no recipe mean canceling any existing implicit rules.
https://www.gnu.org/software/make/manual/html_node/Canceling-Rules.html

How to undo intermediate file deletion

I have a software stack that creates some intermediate files as a part of build process. There is some problem come up and the build breaks. I want to have a look at those intermediate generated files. To my surprise those files are being deleted as a part of build process.
Removing intermediate files...
rm fact_test_without_proxies.c fact_test_main.c fact_test_without_proxies.o
I went through the Makefiles I don't see any explicit rules deleting them. Can there be any implicit rules to delete intermediate files. If yes how can I disable those implicit rules ?
I see the print Removing intermediate files... only if make is executed with --debug option.
skmt#tux:~/coding/factorial/ut$ make --debug
GNU Make 3.81
Copyright (C) 2006 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.
This program built for x86_64-pc-linux-gnu
Reading makefiles...
Updating goal targets....
File `check' does not exist.
File `test_dept_run' does not exist.
File `fact_test' does not exist.
File `fact_using_proxies.o' does not exist.
File `fact_test_without_proxies' does not exist.
File `fact_test_without_proxies.o' does not exist.
File `fact_test_without_proxies.c' does not exist.
File `fact_test_main.c' does not exist.
Must remake target `fact_test_main.c'.
nm -p fact_test.o | build_main_from_symbols >fact_test_main.c
Successfully remade target file `fact_test_main.c'.
Must remake target `fact_test_without_proxies.c'.
cp fact_test_main.c fact_test_without_proxies.c
Successfully remade target file `fact_test_without_proxies.c'.
Must remake target `fact_test_without_proxies.o'.
gcc -I../src -c -o fact_test_without_proxies.o fact_test_without_proxies.c
Successfully remade target file `fact_test_without_proxies.o'.
Must remake target `fact_test_without_proxies'.
gcc fact_test_without_proxies.o fact.o fact_test.o -o fact_test_without_proxies
fact.o: In function `unknown':
fact.c:(.text+0x67): undefined reference to `do_update'
collect2: ld returned 1 exit status
make: *** [fact_test_without_proxies] Error 1
Removing intermediate files...
rm fact_test_without_proxies.c fact_test_main.c fact_test_without_proxies.o
If you're using GNUMake, you can use the special target .PRECIOUS:
.PRECIOUS: fact_test_without_proxies.c fact_test_main.c fact_test_without_proxies.o
or just
.PRECIOUS: %.c %.o
Its only effect is that these files will not be deleted if Make is killed or interrupted.
You can also use .SECONDARY, which will preserve the specified files even if the build does not break.
e.g.
.SECONDARY:
There is a restriction on the use of targets, which affects the behaviour of .PRECIOUS:
I have targets A/%.foo: and B/%.foo: , so I have set:
.PRECIOUS: %.foo
and this did not work; I don't understand why, but expansion does not work this way; I had to explicitely list targets exactly as they are written:
.PRECIOUS: A/%.foo B/%.foo
But even after reading https://www.gnu.org/software/make/manual/html_node/Special-Targets.html I do not understand the difference between .PRECIOUS: and .SECONDARY: .
It's accepted to use those special targets without depends, but I think this would be very dirty coding and would expect side effects. Some people just put .PRECIOUS: or .SECONDARY: without dep, and later, they complain they have to run make clean after a broken build ...

How does "make" app know default target to build if no target is specified?

Most Linux apps are compiled with:
make
make install clean
As I understand it, the make command takes names of build targets as arguments. So for example install is usually a target that copies some files to standard locations, and clean is a target that removes temporary files.
But what target will make build if no arguments are specified (e.g. the first command in my example)?
By default, it begins by processing the first target that does not begin with a . aka the default goal; to do that, it may have to process other targets - specifically, ones the first target depends on.
The GNU Make Manual covers all this stuff, and is a surprisingly easy and informative read.
To save others a few seconds, and to save them from having to read the manual, here's the short answer. Add this to the top of your make file:
.DEFAULT_GOAL := mytarget
mytarget will now be the target that is run if "make" is executed and no target is specified.
If you have an older version of make (<= 3.80), this won't work. If this is the case, then you can do what anon mentions, simply add this to the top of your make file:
.PHONY: default
default: mytarget ;
References:
https://www.gnu.org/software/make/manual/html_node/How-Make-Works.html
GNU Make also allows you to specify the default make target using a special variable called .DEFAULT_GOAL. You can even unset this variable in the middle of the Makefile, causing the next target in the file to become the default target.
Ref: The Gnu Make manual - Special Variables
bmake's equivalent of GNU Make's .DEFAULT_GOAL is .MAIN:
$ cat Makefile
.MAIN: foo
all:
#echo all
foo:
#echo foo
$ bmake
foo
See the bmake(1) manual page.

Resources