This is related to my previous question: Why does .PHONY not work in this situation?.
I have a makefile system that I wrote to make it easy for developers who are not familiar with make, to do their tasks. In short, there is a generic portion which would be the same for all projects, and a set of makefiles that are specific for a given project. The project specific ones include the generic ones. It worked great on make 3.80 for some reason, but when I tried it out on make 3.81 I ran into a few problems. That forced me to make changes that are mentioned in the above post. Now I have some new problems, so I decided to make another post. Like in that post, I made a much smaller and simpler set of makefiles that show the problem. Unfortunatly, the "simple" case consists of 6 files. Sorry about that. First I'll start with the "project specific" ones (these are meant to be simple):
makefile:
TARGETS:=\
Lib1.mk \
Lib2.mk \
my_prog.mk \
include generic/top.mk
Lib1.mk:
BINARY:=Lib1
TYPE:=LIB
LOCATION:=a/location
include generic/rules.mk
Lib2.mk:
BINARY:=Lib2
TYPE:=LIB
LOCATION:=another/location
LIBS:=Lib1
include generic/rules.mk
my_prog.mk:
BINARY:=my_prog
TYPE:=EXE
LOCATION:=some/location
LIBS:=Lib1 Lib2
include generic/rules.mk
A quick description: Makefile simply lists the names of all the targets. A target is either a executable or a library. BINARY is the name of the library or executable (extensions are added by the generic part). TYPE is either EXE or LIB. LOCATION is where the binary should go. LIBS is whatever libraries this binary depends on. The real ones handles creating all the -L, rpath, etc. stuff for the user (as well as their equivalents for visual studio). Now for the generic ones (these do the REAL work):
generic/top.mk:
ALL_BINS:=
.PHONY: all
all:
include $(TARGETS)
all: $(ALL_BINS)
%.so %.exe:
mkdir -p $(dir $#)
touch $#
clean:
rm -rf out
and finally..
generic/rules.mk:
ifeq (EXE,$(TYPE))
$(BINARY).FULL_FILE_NAME:=out/$(LOCATION)/$(BINARY).exe
else
$(BINARY).FULL_FILE_NAME:=out/$(LOCATION)/lib$(BINARY).so
endif
$(BINARY).DEP_LIBS:=$(foreach a,$(LIBS),$($(a).FULL_FILE_NAME))
ALL_BINS+=$(BINARY)
$(BINARY): $($(BINARY).FULL_FILE_NAME)
$($(BINARY).FULL_FILE_NAME): $($(BINARY).DEP_LIBS)
BINARY:=
LOCATION:=
LIBS:=
Ok, in this state, things work fine. Make handles all the dependencies correctly, and if I touch any of the files, it will correctly "build" only the ones that it has to, and nothing more. The problem happens when you take the m_prog.mk line from makefile and move it to the top of the list, like so:
TARGETS:=\
my_prog.mk \
Lib1.mk \
Lib2.mk \
The problem seems to be that while its is going through rules.mk for my_prog.mk it does not yet know what the full library path for Lib1 and Lib2 (they are empty strings). So in the end, it considers my_prog to be dependent on nothing and it tries to build it out of order. In this example, you just see it "touch" my_prog first and then the other 2. Of course, when I have real compiler and linker commands in there, it throws an error.
Back when I simply had the .PHONY targets depend on each other (so my_prog depended on Lib1 and Lib2) life was easy and harmonious. Now that I can't do that, life became more difficult.
You may say, "heck just put it in the right order!". Well up to now, this has been handled automatically through make for the end users. In fact, most customers have been putting things in alphabetical order. They don't know or care what order they depend on each other. It would stink to have to tell them to re-order all of that now. Sorry for the length of this post. I'd appreciate any answers!
If you set variables using the := assignment operator, the assignment is evaluated immediately.
If you set variables using just = as the assignment operator, they're evaluated lazily, as late as possible (at the time of actual use).
See http://www.gnu.org/software/automake/manual/make/Flavors.html.
There are several ways to do what you want. The cleanest is probably by using vpath. Just modify rules.mk:
$(BINARY).DEP_LIBS:=$(foreach a,$(LIBS),$(a).so)
ALL_BINS+=$(BINARY)
vpath %.so out/$(LOCATION)
Related
I have a Makefile which, for a particular target, calls another Makefile. Suppose that the main Makefile contains
some_dir/some_target:
cd some_dir && make some_target
and that some_dir/Makefile contains
some_target: file1 file2
do_stuff
Here's my conundrum: What should the dependencies be for the target in the main Makefile? If I put no dependencies, then, according to the GNU make manual, some_dir/Makefile will only be invoked if some_dir/some_target doesn't exist. I could instead copy the dependencies from some_dir/Makefile. However, that creates the danger of later changing the dependencies in the subdirectory's Makefile and forgetting to update the main Makefile.
Is there a way to tell the main Makefile, "I don't know if the target is out of date or not. Go ask the other Makefile"?
Is there a way to tell the main Makefile, "I don't know if the target is out of date or not. Go ask the other Makefile"?
There is no provision specifically for delegating to another makefile, but you can achieve a similar result by ensuring that the recipe for the target in question is always run. There are mechanisms and conventions for that.
The old-school approach is to declare a dependency on a target that does not exist and is never actually built. You may see such a target being named FORCE, though that's only a convention. The name is not directly significant. Example:
some_dir/some_target: FORCE
cd some_dir && make some_target
# Dummy target
FORCE:
As long is there is not, in fact, a file named FORCE in the working directory, the FORCE target will be considered initially out of date, so every target that directly or indirectly depends on it will be built (see also below).
There is, of course, a weakness in that: what if a file named FORCE actually is created? However unlikely that may be, it screws up the whole scheme if it happens. Some make implementations, notably GNU's, have an implementation-specific way to address that. GNU's approach is to recognize a special, built-in target named .PHONY (do not overlook the leading .). All prerequisites of .PHONY are considered out of date on every build, notwithstanding anything on the filesystem. Inasmuch as make implementations that do not recognize that convention are unlikely to be troubled by its use, there is little drawback to putting it in play:
.PHONY: FORCE
You could also skip FORCE and just directly declare some_dir/some_target itself to be phony, as another answer suggests, but there are at least two problems with that:
it's not really phony in the usual sense. You expect that target to be built. Declaring it phony is therefore confusing.
if you happen to try to use that approach with a make that does not recognize .PHONY, then the whole scheme falls apart. If you instead use an intermediate phony target (such as FORCE, above) then your makefile still works even with such makes, except in the unlikely event that a file named the same as the dummy target is created.
But note well that however implemented, any such scheme has a significant drawback: if you force some_dir/some_target be considered out of date on every build, so that the sub-make will be run unconditionally, then every other target that depends directly or indirectly on some_dir/some_target will also be rebuilt every time. On the other hand, if you do not force it to be rebuilt, then it might not be rebuilt when it ought to be, as you already recognize. This is the topic of the well-known paper Recursive Make Considered Harmful. As an alternative, then, you should consider not using recursive make.
One option consists in forcing a sub-make:
dummy := $(shell $(MAKE) -C some_dir some_target)
top_target: some_dir/some_target
...
"I don't know if the target is out of date or not" - you can use .PHONY for this:
.PHONY: some_dir/some_target
some_dir/some_target:
cd some_dir && make some_target
https://www.gnu.org/software/make/manual/html_node/Special-Targets.html#Special-Targets:
The prerequisites of the special target .PHONY are considered to be phony targets. When it is time to consider such a target, make will run its recipe unconditionally, regardless of whether a file with that name exists or what its last-modification time is.
I haven't found an answer so far, so I think this is not a repeat question.
I have some Makefile along the lines of:
include prerequisite_2
all: prerequisite_1 prerequisite_2
clean:
rm *.mod
prerequisite_1:
mkdir somedir
prerequisite_2:
re-write existing file
The issue is that I want the prerequisite_2 to rebuild whenever the default goal is called (all) or when prerequisite_2 is called from the command line, and I know I can use touch prerequisite_2, FORCE or .PHONY to achieve this. However, I DO NOT want it to run every time (the written file contains dependency information for the Fortran files involved) as it doesn't make sense to also rebuild this when calling: make clean
Is it possible to emulate the effects of FORCE or .PHONY only when the depending targets are called?
You can see what the goal targets are by looking at the MAKECMDGOALS variable.
So you can do something like:
ifeq (,$(if $(MAKECMDGOALS),$(filter-out all prerequisite-2,$(MAKECMDGOALS))))
include prerequisite-2
endif
The if condition will be true if MAKECMDGOALS is the empty string, or if it contains only all and/or prerequisite-2 but not if it contains any other target.
Usually, this is not what you want though. Usually you want to disable the include only if certain targets (clean is the classic example) are used.
This exact situation is even discussed in the GNU make manual.
I have a rule in my Makefile which is intended to create a symlink to a file in a different directory:
VPATH = ../source
foo: foo
ln -s $< $#
Even though I intend for the target to resolve to ./foo and the dependency to resolve to ../source/foo, I understand why make sees it as circular. Is there a way to express this rule in a way that is not circular?
Note that the link does not depend on the changes of the link target; it only needs to exist. Therefore, there is no ordinary prerequisite needed at all, and the simplest fragment to do what you want would be
foo:
ln -sf ../source/$#
However, this won't work correctly if you still need your VPATH for other purposes. If that is so, then it seems to me the simplest way would be to ignore VPATH for this rule by using absolute path:
VPATH := ../source
$(CURDIR)/foo:
ln -sf ../source/$(#F)
Finally, if the file ../source/foo is also a target which is generated by Make, then maybe the best way would be:
VPATH := ../source
.SECONDEXPANSION:
$(CURDIR)/foo: | ../source/$$(#F)
ln -sf $|
Note that we are not depending on the changes of the prerequisite here, only on the existence of it.
By the way, the reason why I am using -f option is because, Make should support -B option. That option will not work unless you use -f here.
I think what you're trying here falls under the category of "VPATH abuse".
My experience is repeatedly pointing me toward the mantra "explicit is better than implicit" and this is one of the reasons why. Contrary to the assertions in the GNU make Manual, my experience has led me to believe that in larger, more complex projects, you need to be more explicity, not less, because the size makes it more difficult to locate files, unless their path is explicit.
I also believe that a lot of the need to use VPATH stems from the use of recursive make, where you're not building complete dependency trees; write your build system properly and you just don't need VPATH at all.
On a related subject, I'm a firm believer in only specifying one or two -I directories: your top-level src/ and include/ directories and make all inclussions relative to these paths. Again, on larger, more complex projects, seing #include "my/really/cool/thing.h" is soooo much informative than simply #include "thing.h".
That said, I am open to the idea of using VPATH for libraries, especially system libraries, because you can use the -lfoo syntax, but I wouldn't want to use it as a general rule, because it could threaten build reproducibility.
I am trying to compile with make. I have source in two directories, src and altsrc. I want make to look in altsrc for source files first, and then in src. I want the objects to go into directory obs. The relevant parts of my Makefile looks like:
VPATH=altsrc:src:obj
$(A_OUT): $(OBS)
$(FORTRAN) -o $(A_OUT) $(OBS) $(LFLAGS)
obs/%.o: %.f
$(FORTRAN) $(FFLAGS) $< -o $#
This Makefile actually compiles the code OK, but it has one really obnoxious side effect: It finds each source file (and there are alot) and copies them out of altsrc or src into the directory where I am running make. This is really annoying and leads to a bunch of confusion later on. I haven't found any documentation anywhere that says this should be an effect of the VPATH macro. Can anyone tell me how to solve this? Thanks.
The trouble is that Make is RCS-savvy, and will check to see if a source file (e.g. td.f) can and should be updated from RCS. Yes, Make knows how to use co. If the source file isn't in such an archive (such as the source files in altsrc), then Make shrugs and gets on with the work.
The quickest, dirtiest solution is to use make -r. This will disable the built-in rules, which should solve the problem nicely... unless you rely on the built-in rules elsewhere.
If that won't do, you can override that particular rule with a do-nothing rule of your own, or with a rule that actually updates the sources in place, or you can touch the sources so that Make won't consider them out of date, or put the RCS files someplace where Make can't see them, or maybe two or three other options.
I have a Makefile with several rules of this form
protolist.c: $(PROTOCOLS) Makefile src/genmodtable.sh
$(SHELL) $(srcdir)/src/genmodtable.sh \
$# $(filter-out %Makefile %genmodtable.sh, $^)
As the name implies, protolist.c winds up containing a list of all the "protocols" defined by the .c files in $(PROTOCOLS). The contents of this file do formally depend on everything in $(PROTOCOLS), the Makefile, and the generator script, but it's very rare for the file to actually change when one of those .c files is edited. Therefore, genmodtable.sh is coded to not change the timestamp of protolist.c if it's not going to make any change to its contents. This causes Make to skip rebuilding protolist.o and its dependencies when it's not really necessary.
That all works fine; the problem is that, because protolist.c now appears to be out of date with respect to its dependencies, Make thinks it has to try to regenerate protolist.c on every run. This isn't a performance issue -- the script is very fast -- but it is confusing behavior. I dimly recall an idiom, involving timestamp files, that could be used to stop Make from doing this, but I have not been able to reconstruct it or find it described anywhere. Does anyone know what it is?
(Also if anyone can suggest how to get rid of that silly $(filter-out ...) construct, that would be helpful, as that is the only GNUmakeism in this Makefile.)
This appears similar to an issue with Fortran programming and make, involving the files generated when compiling a module. (Not relevant, other than that is where I picked up how to do this.)
What you want is have make compare the timestamp of protolist.o to the timestamp of protolist.c, which remains 'old', and make the decision to run the recipe for protolist.c, depending on the timestamp of, well, a timestamp file, which gets updated each time the recipe is run.
In order to make this work, you have to link the two together with an empty rule.
protolist.o: protolist.c
[...]
protolist.c: protolist.c.time ;
protolist.c.time: $(PROTOCOLS) Makefile src/genmodtable.sh
$(SHELL) $(srcdir)/src/genmodtable.sh \
protolist.c $(filter-out %Makefile %genmodtable.sh, $^)
touch protolist.c.time
In my own makefiles, I have to declare the timestamp files as prerequisites of the special target .PRECIOUS, to prevent make from deleting them, but I'm using pattern rules; I'm not 100% sure, but I think this isn't necessary when using explicit rules, like here.
To avoid the $(filter-out ...) construct, can you not simply replace it with $(PROTOCOLS)?
(Although, personally, I would stick to Paul's First Rule of Makefiles: Don't hassle with writing portable makefiles, use a portable make instead.)