I need to know the best way of dealing with this. Also you could answer this -- after reading the sample below --: Look at the makelib target in package.make and tell me if there is a way to force this be treated as not updated if the recipe (make -C ../lib/ -f lib.make) reports as nothing to be made (not using ordered prequisites)?
I need to explain this point using an example. I have inherited this and I need the best way to make this right.
A target which other targets will be depending on:
File lib.make
--------------
.DEFAULT_GOAL = thelib.dll
%.dll: file1.obj file2.obj
makelib file1.obj file2.obj -o thelib.dll
This by itself is pretty solid. You run it once (make -f lib.make) and it creates the lib. If you run it subsequently, having no modified files, then it will tell you it has nothing to do.
Now we're going to use this in a special way somewhere else:
File: package.make
------------------
.DEFAULT_GOAL: all
all: package
makelib:
#make -C ../lib/ -f lib.make
package: makelib file3 file4
#package_files file3 file4 ../lib/out/*.dll -o package
This is how lib.make is referenced inside package.make.
The issue is even though the package gets created when you call make -f package.make all make assumes that package target needs to be rebuilt every time since one of its dependencies -- makelib -- had to be remade
Make considers makelib out of date despite what happens after entering lib.make.
To correct this I thought of a few choices:
moving makelib to the ordered prerequisites ( after the |) but that's not quite right because in case of a newly built library my package wont' be updated
adding the dll (thelib.dll) as the dependency to the makelib target a second time but this would almost duplicate the logic and break the encapsulation.
removing makelib target and moving the line #make -C ../lib/ -f lib.make to inside the package recipe. There is a problem with this and that is that I have removed the dependency between the package and lib. If lib requires update, the package won't know about it and won't get updated.
using include lib.make and then rewriting package rule to something like: package: thelib.dll file3 file4. There are problems with this also and the least of them is for a make file to be included, it must be written as such. Otherwise a lot of overwriting/conflicting targets and definitions will be introduced.
Are there any suggestions other than and directly listing the dll as the dependency?
There are two main ways this works:
First, if you use recursive make (please remember to always invoke a sub-make using $(MAKE), never make directly) then you should make the target in the parent makefile be the actual file generated by the sub-make:
package: lib/thelib.dll ...
...
lib/thelib.dll: FORCE
$(MAKE) -f lib
FORCE:
Second, you can use non-recursive make which means you include the sub-makefile into the parent make and write it so that it expects that. You can play tricks with variables etc. to make this more generic, so it can be invoked from either the parent or subdirectory, if you want.
Related
At present, I have a makefile that has:
a target which links an executable image file from a bunch of object files
a pattern rule target that compiles the various object files the linker target depends on
I want to make the following changes.
Instead of compiling the object files outright, I want the pattern rule target mentioned above to create (for each object file that needs updating) an empty object_file_name.update file. Essentially, this target's job would be to take stock of all object files that actually need to be recompiled.
Write a new target that launches a Perl process which finds all these object_file_name.update files and, for each object file that must be recompiled, compiles it in this Perl process.
I know how to do 2) ... that part is not giving me any trouble. The part I'm worried about is 1). The reason is that that target would basically have to claim to update any needed object files while, in truth, merely creating an .update file for each such object file but not the object file itself.
I think I could trick GNU Make into not starting to try to link anything before all the object files have been built by declaring my dependencies accordingly (pseudo-code, not a valid GNU Make snippet):
# Phony target that reads the *.update files created by the pattern rule target below and then
# compiles each object file for which an *.update file exists.
COMPILE_OBJECTS :
...
# Pattern rule target to take stock of all object files that need updating. Creates an *.update file for
# each object file that needs recompiling.
%.o : %.c :
...
$(EXE_FILE_TO_LINK) : $(LIST_OF_OBJECT_FILE_PATHS) COMPILE_OBJECTS
...
but I still worry that this might result in undefined behavior because my pattern rule target would basically be lying to GNU Make about updating the needed object files. Is my worry justified?
Basically, I want to interject an intermediate layer between GNU Make and the compiler so that GNU Make doesn't compile each object file separately. Instead, the compiling would be done in a single Perl process that has access to the complete list of object files that need to be compiled, allowing me to do various fancy things that I couldn't do if GNU Make controlled compilation directly.
Yes, it's legal and I often use this pattern.
Consider the case where you only want to kick off a long build step if a file has changed.
target: config-file
target-creator $< -o $#
Now let's say we can't give make the dependencies for config-file (because the config file creation step lacks a dependency listing ability (BAH!)).
.PHONY: FORCE
FORCE: ;
config-file: FORCE
config-creator -o $#.tmp
cmp $#.tmp $# || mv $#.tmp $#
We ask make to build target
Make first has to build config-file
Make will always run the recipe for config-file,
as its dependency FORCE is out of date (being phony)
CRUCIALLY we only update config-file if config-creator decides something has actually changed
If cmp decides config-file.tmp and config-file are the same, and the last line of the recipe completes with no error
OTOH if cmp detects a mis-compare, it fails, and the shell goes on to execute the mv.
After running the recipe for config-file, make does actually check config-file's modification time. IF config-file has become younger than target, only then will target-creator be run.
The subtlety here is that even though config-file's recipe runs every time, config-file itself is not phony.
I want to optimise the compilation time of my makefile. One problem that waists my time is, after modifying one single file, make returns for instance,
File "frontend/parser_e.ml", line 1:
Error: The files expression/rc.cmi and frontend/gen/lexer_ref.cmi
make inconsistent assumptions over interface Utility
make: *** [frontend/parser_e.cmx] Error 2
rm frontend/parser_name.ml
Note that the files in trouble may change, but it happens quite often. What I have to do is make clean and then make, as a consequence it is not an incremental build and takes time.
So does anyone know what I should check in my makefile to reduce the chance of having this kind of error?
Edit 1:
Actually, all my ml-related files are in depth 1, except frontend/gen/*, which are in depth 2. Following the answer of #camlspotter, I modified a little bit the ocamldep part of my makefile. Now it looks like follows:
DIRS= -I frontend -I frontend/gen -I lib ...
depend: $(AUTOGEN)
# ocamldep -native $(DIRS) */*.ml */*.mli > depend # this is what was written before, I don't hink it is correct
ocamldep -native $(DIRS) *.ml *.mli > depend
As a consequence, make following another make gives immediately an inconsistence error.
One remark is I don't have AUTOGEN, is it normal?
Another remark is that make depend generates a depend that has 0 character, is it normal?
Edit 2:
I modified depend: by following Makefile of OCaml source code:
beforedepend:: */*.ml
depend: beforedepend
(for d in \
frontend frontend/gen lib ... ; \
do ocamldep $(DIRS) $$d/*.mli $$d/*.ml; \
done) > depend
I have actually around 20 folders, each has 1-5 ml files. This time, make rangs over for d in ..., and does not want to stop. But if I remove 3-4 folders, it succeeds to create a depend after several seconds.
Your Makefile does not cover all the necessary dependencies between modules.
The meaning of
File "frontend/parser_e.ml", line 1:
Error: The files expression/rc.cmi and frontend/gen/lexer_ref.cmi
make inconsistent assumptions over interface Utility
is:
frontend/parser_e.ml depends on expression/rc.ml and frontend/gen/lexer_ref.ml
Both expression/rc.ml and frontend/gen/lexer_ref.ml use module named Utility
expression/rc.ml and frontend/gen/lexer_ref.ml must agree with the type (interface) of Utility, but did not.
I think of two possibilities to cause this state:
There may be two different utility.ml, for example dir_a/utility.ml and dir_b/utility.ml. OCaml does not allow linking modules with the same name. You can workaround this using packed modules (see -pack compiler option). Your case is not this.
The both modules use the same utility.ml but the dependencies may not be perfectly known to your Makefile. This is your case.
A possible scenario of the second case is:
You have modified utility.ml or utility.mli and its interface (.cmi file) has been changed.
One of expression/rc.ml and frontend/gen/lexer_ref.ml is recompiled against this new interface of Utility, but the other IS NOT, since the dependency is not known.
The compiler has found the inconsistency between the two modules when they are used together in frontend/parser_e.ml.
For fix, you have to run ocamldep to capture all the necessary module dependencies and inform it to make. Note that:
Give proper options and arguments. Since you work with nested directories, you need -I option several times.
Make sure that the auto-generated .ml and .mli files are really generated before ocamldep runs. Since you seem to have .mly and .mll files and you have the issue around them, I suspect you miss something around here.
A good example of the dependency analysis of OCaml modules is found at OCaml compiler source code itself. It is good to check around its lines with beforedepend, depend and include .depend.
General hints:
Add include .depend to your Makefile and capture all the module dependencies into this .depend file, using ocamldep
Note that all the .ml and .mli files of your project must be scanned by ocamldep. Do not forget to add -I options properly or it misses some dependencies.
Before running ocamldep, make sure auto-generated .ml and .mli files such as the output of .mly and .mll are generated. Or it misses some dependencies.
Typical Makefile looks like:
beforedepend:: x.ml
x.ml: x.mly
ocamlyacc x.mly
beforedepend:: y.ml
y.ml: y.mll
ocamllex y.mll
depend: beforedepend
ocamldep -I <dir1> -I <dir2> <all the ml and mli paths> > .depend
include .depend
I am trying to create a subdirectory in my project (let's call it $PROJECT/child) that needs to pull in a Makefile (let's call it ../Makefile.inc) from its parent, $PROJECT/Makefile.inc. Later I want to copy $PROJECT/child somewhere else so it can run independently of $PROJECT.
There is a common Makefile that needs to be included in both projects and shipped when the subdirectory is copied, and I want it to be included in both cases. So I thought I would link it in during the child build, if it isn't found. (I don't want to just include ../Makefile.inc, because this will disappear when I copy the project, and I don't want the calling build system to be responsible for putting the Makefile.inc in place.)
With those constraints, here's a horrible hack that I've come up with to do this, within $PROJECT/child/Makefile:
HACK = $(shell test -f Makefile.inc || ln -f ../Makefile.inc .)
include $(HACK)Makefile.inc
Notice the extra special duct tape on that second command. I have to actually include $(HACK) even though it's going to end up empty, so that the $(shell ...) will be evaluated. ;-)
Is there a cleaner way to make this happen?
Give a rule to build Makefile.inc. (make will complain that Makefile.inc doesn't exist when it parses the include line, but it will go on parsing the main makefile, apply any rule to build included files, and go back and re-parse the main makefile with the included files.)
include Makefile.inc
Makefile.inc:
ln ../Makefile.inc $#
Here is the context :
I am working on a makefile to create a .h at every build, including another makefile that will use this header. I can't edit the second one.
Using a target all depending on my file, it compiles the first time, creating the missing MyHeader.h. The problem is, when I recompile, the header is not regenerated...
My makefile looks like this :
all: myHeader.h
myHeader.h:
scriptToBuildMyHeader.sh
include obscureAndPrivateMakefile.make
I also tried with a .phony target at the beginning. Same result : once created, it won't be regenerated at every build.
PS : I can't call a script before make.
Do some makfile-Masters have any ideas how to deal with that ?
Thanks!
Because myHeader.h has no dependencies, it will never be rebuilt once it exists. You can work around this by creating a dependency from myHeader.h to a phony target, eg:
forcebuild:
# dummy; do nothing and don't create this file
.PHONY: forcebuild
myHeader.h: forcebuild
scriptToBuildMyHeader.sh
This will however slow down your build considerably, as the header (and any source files including it) will need to be rebuilt every time.
The trouble is that because myHeader.h does not depend on anything, it exists and is therefore up to date on the second build. To make sure it is built each time, it has to depend on a non-existent file:
myHeader.h: .FORCE
scriptToBuildMyHeader.sh
.FORCE:
The name '.FORCE' (or, sometimes, FORCE) is used classically.
If you use GNU Make, you could make the 'non-existent' file into a phony target:
.PHONY: .FORCE
The advantage of this is that (GNU) make does not create the file .FORCE even if you run make -t - which would break the automatic rebuild of the header because that rule depends on there not being a file .FORCE that actually exists.
Here's another possible approach:
all: clean foo.txt
clean:
rm foo.txt
foo.txt:
echo > foo.txt
where I'm using echo > foo.txt to simulate creation of your header.
I have a project that I am working to release that actually contains 3 subprojects, all of which need to be compiled in one go. My makefile looks roughly like this:
all: a b c
a:
#cd a && make
b:
#cd b && make
c:
#cd c && make
Projects A and B both compile fine but for the 3rd project, it tells me there is nothing to be done although switching to the C directory and running make does in fact compile code.
To be a little more specific: Project C in the example above is actually Mozilla's SpiderMonkey. Whereas A and B are code/makefiles that I have written, C is just a raw copy of SpiderMonkey from the Mozilla website. The actually compile command for it is:
make JS_DIST=/usr JS_THREADSAFE=1 JS_HAS_FILE_OBJECT=1
In my master Makefile, I have:
spidermonkey:
#cd spidermonkey/src && $(MAKE) JS_DIST=/usr JS_THREADSAFE=1 JS_HAS_FILE_OBJECT=1
Running "make spidermonkey" outputs "make: Nothing to be done for `spidermonkey'." How do I get make to run the command?
EDIT:
I've tried adding the following lines to my makefile:
.PHONY: spidermonkey
As well as renaming the spidermonkey rule to sm, but still no change.
EDIT:
My bad! I had spaces when I should have had a tab. doh!
You probably have a file or directory at the toplevel called "spidermonkey". Make thinks this is what its supposed to create, and since it is already there, make stops.
One of the most important rules to follow when writing makefiles is each target should create one file with the same name as the target. In other words, if you have
a:
<some command>
That command should produce a single file called "a".
Rules which do not produce files but are only there as placeholders are called phony targets, and they should be declared like this:
.PHONY: a
Make will then always assume that a has to be remade.
Also, as a general rule do not use "make" to invoke make recursively, use $(MAKE) instead.
EDIT: changed "pseudo" to "phony"
Make only checks for the existance of a file (or directory) named the same as the rule target, and if there is (and it is newer than the dependencies) then from make's point of view there is nothing more to do.
So your problem is that you have a spidermonkey rule (with no dependencies) as well as a directory called spidermonkey, and then make thinks "the target is already made, nothing for me to do". To get make to do what you want, rename the spidermonkey rule (or the directory).
Speaking of recursive make by the way, this is not neccessarily a good idea,
see Recursive Make Considered Harmful.