I have a Makefile with several rules of this form
protolist.c: $(PROTOCOLS) Makefile src/genmodtable.sh
$(SHELL) $(srcdir)/src/genmodtable.sh \
$# $(filter-out %Makefile %genmodtable.sh, $^)
As the name implies, protolist.c winds up containing a list of all the "protocols" defined by the .c files in $(PROTOCOLS). The contents of this file do formally depend on everything in $(PROTOCOLS), the Makefile, and the generator script, but it's very rare for the file to actually change when one of those .c files is edited. Therefore, genmodtable.sh is coded to not change the timestamp of protolist.c if it's not going to make any change to its contents. This causes Make to skip rebuilding protolist.o and its dependencies when it's not really necessary.
That all works fine; the problem is that, because protolist.c now appears to be out of date with respect to its dependencies, Make thinks it has to try to regenerate protolist.c on every run. This isn't a performance issue -- the script is very fast -- but it is confusing behavior. I dimly recall an idiom, involving timestamp files, that could be used to stop Make from doing this, but I have not been able to reconstruct it or find it described anywhere. Does anyone know what it is?
(Also if anyone can suggest how to get rid of that silly $(filter-out ...) construct, that would be helpful, as that is the only GNUmakeism in this Makefile.)
This appears similar to an issue with Fortran programming and make, involving the files generated when compiling a module. (Not relevant, other than that is where I picked up how to do this.)
What you want is have make compare the timestamp of protolist.o to the timestamp of protolist.c, which remains 'old', and make the decision to run the recipe for protolist.c, depending on the timestamp of, well, a timestamp file, which gets updated each time the recipe is run.
In order to make this work, you have to link the two together with an empty rule.
protolist.o: protolist.c
[...]
protolist.c: protolist.c.time ;
protolist.c.time: $(PROTOCOLS) Makefile src/genmodtable.sh
$(SHELL) $(srcdir)/src/genmodtable.sh \
protolist.c $(filter-out %Makefile %genmodtable.sh, $^)
touch protolist.c.time
In my own makefiles, I have to declare the timestamp files as prerequisites of the special target .PRECIOUS, to prevent make from deleting them, but I'm using pattern rules; I'm not 100% sure, but I think this isn't necessary when using explicit rules, like here.
To avoid the $(filter-out ...) construct, can you not simply replace it with $(PROTOCOLS)?
(Although, personally, I would stick to Paul's First Rule of Makefiles: Don't hassle with writing portable makefiles, use a portable make instead.)
Related
I have a Makefile which, for a particular target, calls another Makefile. Suppose that the main Makefile contains
some_dir/some_target:
cd some_dir && make some_target
and that some_dir/Makefile contains
some_target: file1 file2
do_stuff
Here's my conundrum: What should the dependencies be for the target in the main Makefile? If I put no dependencies, then, according to the GNU make manual, some_dir/Makefile will only be invoked if some_dir/some_target doesn't exist. I could instead copy the dependencies from some_dir/Makefile. However, that creates the danger of later changing the dependencies in the subdirectory's Makefile and forgetting to update the main Makefile.
Is there a way to tell the main Makefile, "I don't know if the target is out of date or not. Go ask the other Makefile"?
Is there a way to tell the main Makefile, "I don't know if the target is out of date or not. Go ask the other Makefile"?
There is no provision specifically for delegating to another makefile, but you can achieve a similar result by ensuring that the recipe for the target in question is always run. There are mechanisms and conventions for that.
The old-school approach is to declare a dependency on a target that does not exist and is never actually built. You may see such a target being named FORCE, though that's only a convention. The name is not directly significant. Example:
some_dir/some_target: FORCE
cd some_dir && make some_target
# Dummy target
FORCE:
As long is there is not, in fact, a file named FORCE in the working directory, the FORCE target will be considered initially out of date, so every target that directly or indirectly depends on it will be built (see also below).
There is, of course, a weakness in that: what if a file named FORCE actually is created? However unlikely that may be, it screws up the whole scheme if it happens. Some make implementations, notably GNU's, have an implementation-specific way to address that. GNU's approach is to recognize a special, built-in target named .PHONY (do not overlook the leading .). All prerequisites of .PHONY are considered out of date on every build, notwithstanding anything on the filesystem. Inasmuch as make implementations that do not recognize that convention are unlikely to be troubled by its use, there is little drawback to putting it in play:
.PHONY: FORCE
You could also skip FORCE and just directly declare some_dir/some_target itself to be phony, as another answer suggests, but there are at least two problems with that:
it's not really phony in the usual sense. You expect that target to be built. Declaring it phony is therefore confusing.
if you happen to try to use that approach with a make that does not recognize .PHONY, then the whole scheme falls apart. If you instead use an intermediate phony target (such as FORCE, above) then your makefile still works even with such makes, except in the unlikely event that a file named the same as the dummy target is created.
But note well that however implemented, any such scheme has a significant drawback: if you force some_dir/some_target be considered out of date on every build, so that the sub-make will be run unconditionally, then every other target that depends directly or indirectly on some_dir/some_target will also be rebuilt every time. On the other hand, if you do not force it to be rebuilt, then it might not be rebuilt when it ought to be, as you already recognize. This is the topic of the well-known paper Recursive Make Considered Harmful. As an alternative, then, you should consider not using recursive make.
One option consists in forcing a sub-make:
dummy := $(shell $(MAKE) -C some_dir some_target)
top_target: some_dir/some_target
...
"I don't know if the target is out of date or not" - you can use .PHONY for this:
.PHONY: some_dir/some_target
some_dir/some_target:
cd some_dir && make some_target
https://www.gnu.org/software/make/manual/html_node/Special-Targets.html#Special-Targets:
The prerequisites of the special target .PHONY are considered to be phony targets. When it is time to consider such a target, make will run its recipe unconditionally, regardless of whether a file with that name exists or what its last-modification time is.
I haven't found an answer so far, so I think this is not a repeat question.
I have some Makefile along the lines of:
include prerequisite_2
all: prerequisite_1 prerequisite_2
clean:
rm *.mod
prerequisite_1:
mkdir somedir
prerequisite_2:
re-write existing file
The issue is that I want the prerequisite_2 to rebuild whenever the default goal is called (all) or when prerequisite_2 is called from the command line, and I know I can use touch prerequisite_2, FORCE or .PHONY to achieve this. However, I DO NOT want it to run every time (the written file contains dependency information for the Fortran files involved) as it doesn't make sense to also rebuild this when calling: make clean
Is it possible to emulate the effects of FORCE or .PHONY only when the depending targets are called?
You can see what the goal targets are by looking at the MAKECMDGOALS variable.
So you can do something like:
ifeq (,$(if $(MAKECMDGOALS),$(filter-out all prerequisite-2,$(MAKECMDGOALS))))
include prerequisite-2
endif
The if condition will be true if MAKECMDGOALS is the empty string, or if it contains only all and/or prerequisite-2 but not if it contains any other target.
Usually, this is not what you want though. Usually you want to disable the include only if certain targets (clean is the classic example) are used.
This exact situation is even discussed in the GNU make manual.
I'm working on a rather large project in pure C, and for various reasons we are not using CMake.
We have a good system which uses various shell commands to require very little maintanence. It will automatically find new headers and C files and add the headers to the dependencies and will compile the C files and include them in the output. However, we've gotten to the point where any changes we make to the header files becomes a problem because it requires a recompilation of the entire project, rather than just the C files that include it directly or indirectly.
I've worked on a system to ensure that the only header files which are added as dependencies are ones which are required by the C file itself (recursively up the tree).
I had thought I would be able to solve this myself, but it seems that make has rather inconsistent rules for expansion of variables.
The solution I came up with was to try to use ag and get all the includes out of the file. This works as intended, and I pipe it into tr so that I can ensure that no newlines are added, thereby messing up make. The issue that I'm having though is in determining which file to search in. This is the recipe that it's using for the section in question:
$(GAMEOBJDIR)/%.o : $(GAMEDIR)/%.c $(shell ag -o '(?<=(^#include "))(.*?)(?=("$$))' $(GAMEDIR)/%.c | tr '\n' ' ')
$(CC) $(CFLAGS) $(if $(RELEASE),$(RELFLAGS),$(DBFLAGS)) -c -o $# $<
The problem is that $(GAMEDIR)/%.c is expanding to src/game/%.c and not src/game/<filename>.c. I'm unsure how to fix this issue based on the expansion rules of make.
Once I can figure this out I'll be able to make sure this walks up the chain of header files too, but until I can get this figured out I have no reason to work on that.
Make's rules for expansion are completely consistent... but sometimes they aren't what people want. That's not the same thing :)
Rules for expansion are explained in the manual. In your case you're working with a prerequisite list which means the variables are expanded as the makefile is parsed. That includes the shell command and all other variables. Because you are working with a pattern rule, it means that at the time the makefile is parsed it doesn't match any particular file, it's just defining the pattern rule itself. So, % can't be expanded here. It will be expanded later, when make starts walking the dependency graph and trying to locate ways to build targets.
In general your method cannot work efficiently. You could defer the expansion of these parts of the prerequisites list using Secondary Expansion. However, that means every time make wants to TRY to use this pattern it will run these commands--that will just be slow.
Have you considered using a more standard way to manage detecting prerequisites? Try reading this description for example to see if it would work for you.
For a project of mine I am automatically generating makefiles and including them, like this:
all:
#echo 'SUCCESS is $(SUCCESS)'
clean:
rm depend.mk
depend.mk:
#echo 'Creating $#'
#echo 'SUCCESS := 1' > $#
.PHONY: all clean
include depend.mk
This works, but the include line generates a warning message:
$ make
Makefile:13: depend.mk: No such file or directory
Creating depend.mk
SUCCESS is 1
I would like to silence that first warning line saying that depend.mk doesn't exist. I know it doesn't exist since I have a rule written to generate it, so the warning is unnecessary (unless of course there isn't a rule for it). I do NOT want make to ignore the error where the included file doesn't exist and there is no rule for it, so prefixing include with a - to ignore the error will not work for me. I'd like something similar to bash's convention of piping stderr to /dev/null like some_cmd 2>/dev/null but for including in make.
The sample above is a very simplified example of this case. In my actual project there are a lot of automatically generated makefiles (via clang's automatic dependency generation) being included, meaning a fresh run of make will flood my screen with these warning messages.
Is anything like this possible, or am I just going to have to deal with the annoying warning messages?
I've encountered and (re-re-re-re-)solved this problem a number of times myself. Really, the problem is in the thinking surrounding when the dependency files are generated and used.
This link has the detailed description of the "resolution": http://make.mad-scientist.net/papers/advanced-auto-dependency-generation/
Basically it comes down to the fact that dependency files are really only necessary for rebuilding, not the initial building of your library/executable. Resultantly you don't need to have a rule for generating dependency files up front (which is in fact less efficient), you instead should generate them during the object file step as intermediate files marked precious (so they're created and tracked as side-effect files that should never be cleaned up automatically). Subsequent builds will then have the files available, which is exactly what you were trying to achieve overall. You can then make it a "-include" on the dependency files, with the foreknowledge that your object file build step will fail if the dependency file generation fails, giving an immediate error, as you've mentioned is preferred, rather than an obscure and indirect one much later.
I've actually done a couple rather large build systems implementing this method, and it does work quite well, including ones that used non-GNU toolchains. To an outside user it appears identical, but internally it performs more efficiently and isn't hiding potentially important errors.
I tried many (many!) things to see if I could prevent or redirect the error message. No luck.
But when I tried -include (include with a leading dash), it didn't give an error, and make with clean, all, depend.mk and 'default' all worked properly and as expected.
Is there a particular reason you didn't want to use the -include variant? Seems to do exactly what you're looking for, and doesn't alter how the Makefile works in any way, just doesn't show the error during the first pass through the Makefile.
This is related to my previous question: Why does .PHONY not work in this situation?.
I have a makefile system that I wrote to make it easy for developers who are not familiar with make, to do their tasks. In short, there is a generic portion which would be the same for all projects, and a set of makefiles that are specific for a given project. The project specific ones include the generic ones. It worked great on make 3.80 for some reason, but when I tried it out on make 3.81 I ran into a few problems. That forced me to make changes that are mentioned in the above post. Now I have some new problems, so I decided to make another post. Like in that post, I made a much smaller and simpler set of makefiles that show the problem. Unfortunatly, the "simple" case consists of 6 files. Sorry about that. First I'll start with the "project specific" ones (these are meant to be simple):
makefile:
TARGETS:=\
Lib1.mk \
Lib2.mk \
my_prog.mk \
include generic/top.mk
Lib1.mk:
BINARY:=Lib1
TYPE:=LIB
LOCATION:=a/location
include generic/rules.mk
Lib2.mk:
BINARY:=Lib2
TYPE:=LIB
LOCATION:=another/location
LIBS:=Lib1
include generic/rules.mk
my_prog.mk:
BINARY:=my_prog
TYPE:=EXE
LOCATION:=some/location
LIBS:=Lib1 Lib2
include generic/rules.mk
A quick description: Makefile simply lists the names of all the targets. A target is either a executable or a library. BINARY is the name of the library or executable (extensions are added by the generic part). TYPE is either EXE or LIB. LOCATION is where the binary should go. LIBS is whatever libraries this binary depends on. The real ones handles creating all the -L, rpath, etc. stuff for the user (as well as their equivalents for visual studio). Now for the generic ones (these do the REAL work):
generic/top.mk:
ALL_BINS:=
.PHONY: all
all:
include $(TARGETS)
all: $(ALL_BINS)
%.so %.exe:
mkdir -p $(dir $#)
touch $#
clean:
rm -rf out
and finally..
generic/rules.mk:
ifeq (EXE,$(TYPE))
$(BINARY).FULL_FILE_NAME:=out/$(LOCATION)/$(BINARY).exe
else
$(BINARY).FULL_FILE_NAME:=out/$(LOCATION)/lib$(BINARY).so
endif
$(BINARY).DEP_LIBS:=$(foreach a,$(LIBS),$($(a).FULL_FILE_NAME))
ALL_BINS+=$(BINARY)
$(BINARY): $($(BINARY).FULL_FILE_NAME)
$($(BINARY).FULL_FILE_NAME): $($(BINARY).DEP_LIBS)
BINARY:=
LOCATION:=
LIBS:=
Ok, in this state, things work fine. Make handles all the dependencies correctly, and if I touch any of the files, it will correctly "build" only the ones that it has to, and nothing more. The problem happens when you take the m_prog.mk line from makefile and move it to the top of the list, like so:
TARGETS:=\
my_prog.mk \
Lib1.mk \
Lib2.mk \
The problem seems to be that while its is going through rules.mk for my_prog.mk it does not yet know what the full library path for Lib1 and Lib2 (they are empty strings). So in the end, it considers my_prog to be dependent on nothing and it tries to build it out of order. In this example, you just see it "touch" my_prog first and then the other 2. Of course, when I have real compiler and linker commands in there, it throws an error.
Back when I simply had the .PHONY targets depend on each other (so my_prog depended on Lib1 and Lib2) life was easy and harmonious. Now that I can't do that, life became more difficult.
You may say, "heck just put it in the right order!". Well up to now, this has been handled automatically through make for the end users. In fact, most customers have been putting things in alphabetical order. They don't know or care what order they depend on each other. It would stink to have to tell them to re-order all of that now. Sorry for the length of this post. I'd appreciate any answers!
If you set variables using the := assignment operator, the assignment is evaluated immediately.
If you set variables using just = as the assignment operator, they're evaluated lazily, as late as possible (at the time of actual use).
See http://www.gnu.org/software/automake/manual/make/Flavors.html.
There are several ways to do what you want. The cleanest is probably by using vpath. Just modify rules.mk:
$(BINARY).DEP_LIBS:=$(foreach a,$(LIBS),$(a).so)
ALL_BINS+=$(BINARY)
vpath %.so out/$(LOCATION)