I have such Makefile with a content for creating a script:
.PHONY cluster-run
cluster-run:
make $(TARGET) --just-print >> tmp_script.sh;
And another one nn.mk:
.PHONY nn-model
include Makefile
nn-model:
python run-nn.py
I have two separate Makefiles for readability, because their content is big and I have another '*.mk' files, like nn-lstm.mk, nn-conv.mk, etc.
I launch as follows:
make -f nn.mk cluster-run TARGET=nn-model
But make gives an error:
make nn-model --just-print >> tmp_script.sh;
make[1]: *** No rule to make target `nn-model'. Stop.
make: *** [cluster-run] Error 2
For me such behaviour is strange because target nn-model actually exists. How can I fix this problem?
First you should never use raw make in recipes. Always use the $(MAKE) variable.
Second, the problem is because when you run the sub-make you don't provide the -f option:
make nn-model --just-print >> tmp_script.sh;
Because of that, it reads Makefile but not nn.mk, and so there's no rule to build the target nn-model.
Remember if you run a sub-make like this it's starting an entirely new make process with a clean slate: none of the targets defined in the parent make process are known to the sub-make when it starts.
I don't know what you mean by target nn_model actually exists but there's definitely no file named nn_model or you wouldn't get that error.
So what's happening is that when you build cluster-run it invokes a recursive make, which reads Makefile, and asks it to build $(TARGET) (which will include nn-model).
Notice that the recursive make is a new make and does not inherit variables or rules from the parent make, so this make instance has no clue how to build nn-model If you want the child make to see this, then the child make must include the parent one...
Related
Consider the following (MCVE of a) Makefile:
my_target: prepare test.bin
prepare:
echo >test.dat
%.bin: %.dat
cp $? $#
If you run make in a clean directory, it fails:
echo >test.dat
make: *** No rule to make target 'test.bin', needed by 'my_target'. Stop.
Run it again and it succeeds:
echo >test.dat
cp test.dat test.bin
What seems to happen is that the rule to make *.bin from *.dat only recognises that it knows how to make test.bin if test.dat exists before anything is executed, even though according to the output it has already created test.dat before it tries to create test.bin.
This is inconvenient for me as I have to prepare a few files first (import them from a different, earlier part of the build).
Is there a solution? Perhaps some way to allow the rules to be (re)evaluated in the light of the files which are now present?
There are a number of issues with your makefile. However based on your comments I'm inclined to assume that the MCVE here is just a little too "M" and it's been reduced so much that it has a number of basic problems. So I won't discuss them, unless you want me to.
The issue here is that you're creating important files without indicating to make that that's what you're doing. Make keeps internally a cache of the contents of directories that it's worked with, for performance reasons, and that cache is only updated when make invokes a rule that it understands will modify it.
Here your target is prepare but the recipe actually creates a completely different file, test.dat. So, make doesn't modify its internal cache of the directory contents and when it checks the cache to see if the file test.dat exists, it doesn't.
You need to be sure that your makefile is written such that it doesn't trick make: if a recipe creates a file foo then the target name should be foo, not bar.
This happens for wildcard targets, like %.bin. They get evaluated at the first pass. You could add an explicit target of test.bin. Or, follow the advice of tkausl and have test.dat depend on prepare (a phony target). In this case, you don't need the double dependency anymore:
my_target: test.bin
you have to write
test.dat: prepare
or (when when you want to stay with wildcards)
%.dat: prepare
#:
Usually, you might want to create and use .stamp files instead of a prepare target.
Summary: I'm dealing with a make script that generates (and optionally 'makes') a makefile. Historically it used a make make "phony target" to do so. I want to change this to make makefile because it seems more coherent and representative of what's going on. But when I change it and switch to the .FORCE idiom so it will be generated dependent on an artificial phony target, it seems the makefile is created 4 extra times for a reason I do not understand.
Details: The way the script works is that you can write either:
make -f makefile.boot
or:
make -f makefile.boot make
In the first case, it assumes you want to use the rules in makefile.boot to generate a platform-specific makefile, and then run make on that file. In the second case it assumes you only want to create the makefile but not execute it.
Here is a stripped down version of makefile.boot in make make terms that works:
top: make
$(MAKE)
make:
#echo "Pretending to generate makefile..."
cp makefile.fake makefile
The makefile we "generate" wants to be a superset of makefile.boot. It wants to be able to do the make make generation process as well, but its top target is an actual build. So makefile.fake contains
top: product
make:
#echo "Pretending to generate makefile..."
cp makefile.fake makefile
product:
#echo "Pretending to make build product..."
echo "Build Product" >> product
It works, but I had a thought:
"make make" is confusing to read, and it would be clearer if it was "make makefile" instead
An immediate problem with that is when you have a real target instead of a phony one, then if the file exists and has no dependencies it won't get rebuilt. I wanted this makefile to be created every time you did make makefile or make -f makefile.boot makefile. So I used the .FORCE idiom to depend on a phony target. Updated makefile.boot:
.FORCE
top: makefile
$(MAKE)
makefile: .FORCE
#echo "Pretending to generate makefile..."
cp makefile.fake makefile
And an updated makefile.fake:
.FORCE:
top: product
makefile: .FORCE
#echo "Pretending to generate makefile..."
cp makefile.fake makefile
product:
#echo "Pretending to make build product..."
echo "Build Product" >> product
Which seems all well and good, but it now runs the makefile generation five times:
/test$ make -f makefile.boot
Pretending to generate makefile...
cp makefile.fake makefile
make
make[1]: Entering directory '/test'
Pretending to generate makefile...
cp makefile.fake makefile
Pretending to generate makefile...
cp makefile.fake makefile
Pretending to generate makefile...
cp makefile.fake makefile
Pretending to generate makefile...
cp makefile.fake makefile
Pretending to make build product...
echo "Build Product" >> product
make[1]: Leaving directory '/test'
The first one I want, and seems like the only one I asked for. Where are all the other calls coming from? What's triggering the four additional requests for makefile? Or in the absence of understanding, is there an alternative way of achieving my intent?
Do recall that makefile is a magic target in many make implementations, including GNU Make.
If the target makefile exists, then make will remake the makefile before doing anything else, and when it's finished it'll restart processing with the new makefile. That means that your makefile target may be run even if you don't ask for it.
Since the default action when you do make -f makefile.boot is to run make, then that's at least two potential runs of the makefile target's actions right there, before it's even looked at the product target. I can't quite make this add up to five runs, but I'd lay money that it's this special behaviour that's causing the unexpected repeats.
I've been trying to get a makefile, a, to include another makefile, b, if the target specified is not found in file a. I'm using this snippet to try and achieve this, but from echos I've put into the file I can see that makefile b is being accessed even when the target is found in a and run.
The snippet I'm using from the link above is:
foo:
frobnicate > foo
%: force
#echo "No target found locally, running default makefile"
#$(MAKE) -f Makefile $#
force: ;
Specifically I'm getting "Nothing to be done" outputs when makefile b is being used, and makefile a is behaving as expected. This is shown below:
$ make all # all target appears in both make files
No target found locally, running default makefile
make[1]: Entering directory `/home/user/currdir' # (b)
make[1]: Nothing to be done for `Makefile'.
make[1]: Leaving directory `/home/user/currdir'
Local all # (a)
Is there a better way to be doing this?
addition: After adding another echo to the % rule, I've found that $# is "Makefile", when it should be the target trying to be built.
I don't really understand your question based on the example you gave; there is no "a" or "b" in that example, just one Makefile.
However, the behavior you're seeing is due to GNU make's re-making makefiles capability. When you create match-anything pattern rules as you've done, you have to consider that every single target or prerequisite that make wants to build will match that rule. That's a large burden.
You can avoid having remade makefiles match by creating explicit rules for them, such as:
Makefile: ;
Why is the following makefile using Makefile target?
Makefile1:
Initially I have the following makefile which worked as expected when invoked as make abc xyz -s.
%::
echo $#
I would get
abc
xyz
Makefile2:
Now after adding an empty rule named test.
%:: test
echo $#
test:
the following invocation
make abc xyz -s
results in
Makefile
abc
xyz
Why am I getting Makefile as my output even though I am giving only abc and xyz as targets? Thanks in advance.
Because make always tries to rebuild the build files before building the actual targets. If it finds a rule for Makefile and if it is out-of-date, it will be rebuilt and reloaded and the requested targets will be built according to the new makefile. This is a feature so that if the build-files are themselves generated (rather common with autotools, cmake and similar), it won't use stale build instructions.
For more details see GNU Make Manual section 3.5
In the specific examples above the rule has target % and that matches absolutely anything, including Makefile. So make will find it as rule for remaking makefile and will evaluate it.
Now in the first case Makefile exists and is newer than all of it's dependencies trivially because there are none and none of it's dependencies need to be remade also because there are none. So make will conclude that Makefile does not need to be remade.
In the second case however Makefile exists, but it's dependency test needs to be remade. So make runs the (empty) rule and than comes back and runs the rule for Makefile. Because make does not check the timestamps after making dependencies. It simply assumes that when it remade them, the dependent targets need to be remade as well.
I tried to write a make rule of this form:
lib%.so: computations/%.h
make -C computations/ -f makefile $<
Output:
make -C computations/ -f makefile computations/test.h
make[1]: Entering directory `/home/shai/csm/csm2/src/computations'
make[1]: *** No rule to make target `computations/test.h'. Stop.
make[1]: Leaving directory `/home/shai/csm/csm2/src/computations'
make: *** [libtest.so] Error 2
Well, it seems that it looks for %.h after switching library. No biggie, I'll just try removing
lib%.so: %.h
make -C computations/ -f makefile $<
but now it doesn't recognize the rule at all!
Output:
shai#ubuntu:~/csm/csm2/src$ make libtest.so
`make: *** No rule to make target `libtest.so'. Stop.
Is make playing with my mind? How could a change in the dependencies make it stop recognizing the target?
How could a change in the dependencies make it stop recognizing the target?
Most likely because it can't find a file that fits the pattern of the dependency, because test.h is in another directory. You could use VPATH to have it search for files in other directories.
Edit: but this still won't fix your problem completely, because $< will be substituted by computations/test.h, which won't be found in the directory computations (as in the first error you got). You might try $(notdir $<), but I think it's an ugly hack (if it works; I haven't tried it). This results from the design of your makefile and project structure.
You're using make wrong.
Make's design is actually fairly simple at heart: you tell it
how to create targets (The body of the rule).
which targets it needs to create first (The dependency list).
Make then topological sorts the dependency information so it executes the bodies in the right order.
Your rule with $< (which expands to the list of dependencies) means you're telling the invoked make to create the dependency %.h not the target lib%.so.
The rule has to create the target. If the dependency needs to be created, you need to tell make how to do so with another, separate rule, not try to ensure it is up to date in this rule.
As you're trying to coordinate across directories, I'll recommend the paper Recursive make considered harmful, which can often be a nice way to organize things, though not the only possible way to do things.
It might just be the way you copy/pasted it, but you're missing a tab in the second form.