Here's a minimal example of what I'm trying to do.
Directory structure:
.
├── Makefile
└── subdir
├── a
└── Makefile
./subdir/Makefile:
a.copy: a
cp a a.copy
./Makefile:
.PHONY: build_subdir
a.copy: subdir/a.copy
cp subdir/a.copy a.copy
build_subdir:
$(MAKE) -C subdir
subdir/a.copy: build_subdir
The first time I run make, everything's fine:
$ make
make -C subdir
make[1]: Entering directory `/home/kkourt/src/tests/make/subdir'
cp a a.copy
make[1]: Leaving directory `/home/kkourt/src/tests/make/subdir'
cp subdir/a.copy a.copy
Re-runing make is also fine (nothing happens):
$ make
make -C subdir
make[1]: Entering directory `/home/kkourt/src/tests/make/subdir'
make[1]: `a.copy' is up to date.
make[1]: Leaving directory `/home/kkourt/src/tests/make/subdir'
If I update subdir/a, however, ./a is not updated:
$ touch subdir/a
$ make
make -C subdir
make[1]: Entering directory `/home/kkourt/src/tests/make/subdir'
cp a a.copy
make[1]: Leaving directory `/home/kkourt/src/tests/make/subdir'
If I run make a second time, ./a gets updated
$ make
make -C subdir
make[1]: Entering directory `/home/kkourt/src/tests/make/subdir'
make[1]: `a.copy' is up to date.
make[1]: Leaving director
Why does this happen? shouldn't the check of whether subdir/a.copy is older than a.copy happen after the build_subdir target is finished since it's its dependency? Is there a way to fix this?
Edit:
As MadScientist suggested, I added a dummy recipe and it works:
subdir/a.copy: build_subdir
#:
Because your rule subdir/a.copy: build_subdir has no recipe, make "knows" that it can't actually change the timestamp on subdir/a.copy.
You need add a dummy recipe to it, maybe like this:
subdir/a.copy: build_subdir ;
(note I can't test this right now but I think it'll work).
Related
I know there are many discussions here, why don't use recursive Makefiles. But not reverse. Use this concept (Reverse-Recursive) for usability purpose. With this system I only need one open shell for adding new or modified files to git and can also type make in the same path for rebuild the changed source and link them.
Short description to my Reverse-Recursive Makefile implementation. I have the following simplified tree here:
$ tree
.
├── dir
│ ├── Makefile
│ └── subdir
│ ├── Makefile
│ └── subsubdir
│ └── Makefile
└── Makefile
3 directories, 4 files
Content from the Root-Makefile:
all:
#echo 'Makefile-root: all'
arg1:
#echo 'Makefile-root: arg1'
arg2:
#echo 'Makefile-root: arg2'
.PHONY: all arg1 arg2
RR-Makefile: All other Makefiles in the sub-directories have the same content:
%::
make -C .. $#
all:
make -C ..
.PHONY: all
Output when I type make in subsubdir:
$ make
make -C ..
make[1]: Entering directory '/home/krj/projects/XU-Loader/test/dir/subdir'
make -C ..
make[2]: Entering directory '/home/krj/projects/XU-Loader/test/dir'
make -C ..
make[3]: Entering directory '/home/krj/projects/XU-Loader/test'
Makefile-root: all
make[3]: Leaving directory '/home/krj/projects/XU-Loader/test'
make[2]: Leaving directory '/home/krj/projects/XU-Loader/test/dir'
make[1]: Leaving directory '/home/krj/projects/XU-Loader/test/dir/subdir'
Output when I type make with an argument in subsubdir:
$ make arg1
make -C .. arg1
make[1]: Entering directory '/home/krj/projects/XU-Loader/test/dir/subdir'
make -C .. arg1
make[2]: Entering directory '/home/krj/projects/XU-Loader/test/dir'
make -C .. arg1
make[3]: Entering directory '/home/krj/projects/XU-Loader/test'
Makefile-root: arg1
make[3]: Leaving directory '/home/krj/projects/XU-Loader/test'
make[2]: Leaving directory '/home/krj/projects/XU-Loader/test/dir'
make[1]: Leaving directory '/home/krj/projects/XU-Loader/test/dir/subdir'
It's maybe only a cosmetic issue, but how can I suppress Leaving/Entering messages without to invoke make with the argument --no-print-directory in subsubdir?
$ make --no-print-directory
make -C ..
make -C ..
make -C ..
Makefile-root: all
Have tried to add MAKEFLAGS += -no-print-directory in each RR-Makefile but breaks the function of the recursion down to the root Makefile.
It was an usage fault from me. I used -no-print-directory instead of --no-print-directory, because I searched first on SO for a similar question and found this answer. Unfortunately don't investigaed the manual pages from GNU make.
With --no-print-directory this works, without any issues:
MAKEFLAGS += --no-print-directory
%::
make -C .. $#
all:
make -C ..
.PHONY: all
Can someone explain why make with sub-makes has different parallelization behavior?
I have two examples below. In one, all targets are in one Makefile where as in the second example each target has its own folder/makefile and they call each other using $(MAKE).
Running them with -j produce outputs which are vastly different. In the first case, all access to the top-level dependency (a.out below) is synchronized. Make waits for the target to build before running any targets which depend on that.
In the second case, the top-level target (a.out below) is subjected to concurrent access. This causes serious problems for us unless we use our own synchronization techniques to overcome that.
Examples 1:
jesaremi#u16-3:~/maketest$ cat Makefile
.ONESHELL:
all: d.out b.out c.out a.out
a.out: CALLER ?= self
a.out:
#
echo entering $# "(called by $(CALLER))"
sleep 10
echo exiting $#
b.out: a.out
#
echo entering $#
sleep 1
echo exiting $#
c.out: a.out
#
echo entering $#
sleep 1
echo exiting $#
d.out: a.out
#
echo entering $#
sleep 1
echo exiting $#
jesaremi#u16-3:~/maketest$ make
entering a.out (called by self)
exiting a.out
entering d.out
exiting d.out
entering b.out
exiting b.out
entering c.out
exiting c.out
jesaremi#u16-3:~/maketest$ make -j
entering a.out (called by self)
exiting a.out
entering d.out
entering c.out
entering b.out
exiting c.out
exiting d.out
exiting b.out
Example 2 (using sub-make):
jesaremi#js-u16-1:~/maketest$ cat */Makefile
---------- a/Makefile -------------
.ONESHELL:
a.out: CALLER ?= self
a.out:
#
echo entering $# "(called by $(CALLER))"
sleep 10
echo exiting $#
---------- b/Makefile -------------
.ONESHELL:
export CALLER:=b
a.out:
$(MAKE) -C ../a
b.out: a.out
#
echo entering $#
sleep 1
echo exiting $#
---------- c/Makefile -------------
.ONESHELL:
export CALLER:=c
a.out:
$(MAKE) -C ../a
c.out: a.out
#
echo entering $#
sleep 1
echo exiting $#
---------- d/Makefile -------------
.ONESHELL:
export CALLER:=d
a.out:
$(MAKE) -C ../a
d.out: a.out
#
echo entering $#
sleep 1
echo exiting $#
jesaremi#js-u16-1:~/maketest$ cat Makefile
all: d.out c.out b.out a.out
%.out:
$(MAKE) -C $*
.PHONY:all
jesaremi#js-u16-1:~/maketest$ make
make -C d
make[1]: Entering directory '/home/jesaremi/maketest/d'
make -C ../a
make[2]: Entering directory '/home/jesaremi/maketest/a'
entering a.out (called by d)
exiting a.out
make[2]: Leaving directory '/home/jesaremi/maketest/a'
make[1]: Leaving directory '/home/jesaremi/maketest/d'
make -C c
make[1]: Entering directory '/home/jesaremi/maketest/c'
make -C ../a
make[2]: Entering directory '/home/jesaremi/maketest/a'
entering a.out (called by c)
exiting a.out
make[2]: Leaving directory '/home/jesaremi/maketest/a'
make[1]: Leaving directory '/home/jesaremi/maketest/c'
make -C b
make[1]: Entering directory '/home/jesaremi/maketest/b'
make -C ../a
make[2]: Entering directory '/home/jesaremi/maketest/a'
entering a.out (called by b)
exiting a.out
make[2]: Leaving directory '/home/jesaremi/maketest/a'
make[1]: Leaving directory '/home/jesaremi/maketest/b'
make -C a
make[1]: Entering directory '/home/jesaremi/maketest/a'
entering a.out (called by self)
exiting a.out
make[1]: Leaving directory '/home/jesaremi/maketest/a'
jesaremi#js-u16-1:~/maketest$ make -j
make -C d
make -C c
make -C b
make -C a
make[1]: Entering directory '/home/jesaremi/maketest/d'
make -C ../a
make[1]: Entering directory '/home/jesaremi/maketest/c'
make -C ../a
make[1]: Entering directory '/home/jesaremi/maketest/b'
make -C ../a
make[2]: Entering directory '/home/jesaremi/maketest/a'
make[1]: Entering directory '/home/jesaremi/maketest/a'
entering a.out (called by self)
make[2]: Entering directory '/home/jesaremi/maketest/a'
entering a.out (called by c)
make[2]: Entering directory '/home/jesaremi/maketest/a'
entering a.out (called by d)
entering a.out (called by b)
exiting a.out
make[1]: Leaving directory '/home/jesaremi/maketest/a'
exiting a.out
make[2]: Leaving directory '/home/jesaremi/maketest/a'
exiting a.out
make[1]: Leaving directory '/home/jesaremi/maketest/c'
make[2]: Leaving directory '/home/jesaremi/maketest/a'
make[1]: Leaving directory '/home/jesaremi/maketest/d'
exiting a.out
make[2]: Leaving directory '/home/jesaremi/maketest/a'
make[1]: Leaving directory '/home/jesaremi/maketest/b'
Perhaps you're thinking that recusive invocations of make communicate to each other which targets they are building and somehow interlock so that no two sub-makes are trying to build the same target at the same time.
That's nowhere close to something make can do. All make can do is notify other sub-makes how many targets they are building so that you are ensured that no more than N total targets (for -jN) are invoked.
If you want to ensure that two different makefiles don't try to build the same target, that's up to you to do by organizing your makefiles; make can't do it for you.
In the second example, you list all the prerequisites as a single target:
all: d.out c.out b.out a.out
This recipe says, "before all can be built the targets d.out, c.out, b.out, and a.out must be finished". This says nothing about the relative relationship between the prerequisites (d.out, c.out, b.out, or a.out) themselves so when you run with -j the sub-makes for all of them will be invoked at the same time. If each of those tries to build the same shared target, then they will interfere with each other.
If you want to ensure that doesn't happen you must declare the prerequisite relationship in your makefile so make knows about it. For example you could do this:
all: d.out c.out b.out a.out
d.out c.out b.out: a.out
%.out:
$(MAKE) -C $*
.PHONY:all
Now, the recipes to build d.out, c.out, and b.out won't be started until after the recipe to build a.out is complete.
The question is already answered on file-level. But I have a bigger project which has quite a lot of inter-project-dependencies (caused by DBus headers, which were generated dynamically).
I ve created the following example (example files as ZIP - the real project is much more complex).
The top-level Makefile is the following:
sub-%:
$(MAKE) -C $(patsubst sub-%,%,$#)
default:
$(MAKE) -j12 sub-p1 sub-p2 sub-p3
The Makefile of the sub-project look like this (p1, p2 and p3):
all: p1
../lib/lib.a:
$(MAKE) -C ../lib lib.a
p1: ../lib/lib.a
cp -f ../lib/lib.a p1
And the Makefile of the lib looks like this:
lib.a:
sleep 2
date > $#
echo Done with building $#
THE PROBLEM: The library is built for each p*-project separately in parallel - in this example it's not a problem, but in our case it causes unsolvable problems.
When I call make on the top-level, I get the following output:
$ make
make -j12 sub-p1 sub-p2 sub-p3
make[1]: Entering directory '/home/kkr/tmp/parallelmake'
make -C p1
make -C p2
make -C p3
make[2]: Entering directory '/home/kkr/tmp/parallelmake/p1'
make -C ../lib lib.a
make[2]: Entering directory '/home/kkr/tmp/parallelmake/p2'
make -C ../lib lib.a
make[2]: Entering directory '/home/kkr/tmp/parallelmake/p3'
make -C ../lib lib.a
make[3]: Entering directory '/home/kkr/tmp/parallelmake/lib'
make[3]: Entering directory '/home/kkr/tmp/parallelmake/lib'
make[3]: Entering directory '/home/kkr/tmp/parallelmake/lib'
sleep 2
sleep 2
sleep 2
date > lib.a
date > lib.a
date > lib.a
make[3]: Leaving directory '/home/kkr/tmp/parallelmake/lib'
make[3]: Leaving directory '/home/kkr/tmp/parallelmake/lib'
make[3]: Leaving directory '/home/kkr/tmp/parallelmake/lib'
cp -f ../lib/lib.a p3
cp -f ../lib/lib.a p1
cp -f ../lib/lib.a p2
make[2]: Leaving directory '/home/kkr/tmp/parallelmake/p3'
make[2]: Leaving directory '/home/kkr/tmp/parallelmake/p1'
make[2]: Leaving directory '/home/kkr/tmp/parallelmake/p2'
make[1]: Leaving directory '/home/kkr/tmp/parallelmake'
QUESTION: Is it possible to synchronize the sub-projects somehow automatically?
Since the real project has 13 sub-projects - with most of them having inter-project-dependencies - a manual synchronization would be quite difficult.
There is clearly no way for a recursive make system to "automatically" synchronize this; they are completely separate processes and have no way to tell each other that they are running a given target.
You have two choices. The first, which is best but may be a lot of work, is to rework your build system so it's non-recursive. If you have a single make invocation which builds the entire system then there is only one instance of make, and it will globally coordinate the creation of all targets. You won't need the $(MAKE) -C ../lib lib.a rules at all.
If that's not feasible, then you need to rewrite your top-level makefile so that you enforce an ordering such that you never start making two directories that depend on the same external target, at the same time. For example instead of the top-level makefile you show above, write it like this:
default: p1 p2 p3
p%: ; $(MAKE) -C $#
.PHONY: default p1 p2 p3
Now suppose you want to ensure that p1 runs before p2 and p3 because you want p1 to build all the shared targets; then you'd add a rule like this to the top-level makefile:
p2 p3: p1
And you can add other orderings as needed.
I've gone over a number of other posts about recursive makefiles. Either I'm doing something wrong or there's a missing feature that I want. With a typical makefile I can check to see if it's out of date with -q, for example:
make -q || echo "out of date"
However, setting up a mechanism to force make to process dependent makefiles causes the root makefile to always be out of date. Here's a working example:
#./Makefile
all: myfile
myfile: dir
cp dir/myfile .
.PHONY: dir
dir:
$(MAKE) -C dir/
#./dir/Makefile
all: myfile
myfile:
touch myfile
#testing...
>>> make
make -C dir/
make[1]: Entering directory `dir'
touch myfile
make[1]: Leaving directory `dir'
cp dir/myfile .
>>> make
make -C dir/
make[1]: Entering directory `dir'
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory `dir'
cp dir/myfile . #<-- this shouldn't be necessary
>>> make -q || echo "out of date"
make -C dir/
out of date
Is there a mechanism that allows the recursive make call to return and say "actually everything's up to date here, no need to continue"?
There's an hacky way of doing it, maybe.
You could invoke $(MAKE) -q ... && touch something from inside the makefile, and have the something file as requisite for some rule in the outer makefile.
No, there's no way to do that. A recursive invocation of make is just another command that make invokes in a shell: the only result of that command available to make is the exit code. If we could always be sure that the command invoked was a make command, and that we were using the -q flag, then we could assume that if it exited with a 0 then nothing was done. But make does not assume those things.
You will need to rewrite your makefiles to be non-recursive to get the behavior you want.
#Dacav gave me the idea of calling make explicitly. Though without using a temporary files, target dependencies can be set dynamically using a shell-populated variable. I'm not sure the following is a good solution because even when calling make without -q it recursively calls make -q to populate OLD_CHILDREN. Moreover this will happen again unnecessarily for the child makefiles too (assuming multiple levels of recursion).
#./Makefile
CHILDREN=dir/
OLD_CHILDREN=$(foreach c, $(CHILDREN), $(shell make -q -C $(c) || echo $(c)))
all: myfile
myfile: $(OLD_CHILDREN)
cp dir/myfile .
.PHONY: $(CHILDREN)
$(CHILDREN):
$(MAKE) -C $#
Testing...
>>> make -q || echo out of date
make -C dir/
make: *** [dir/] Error 1 #huh??? whatever. still works, returning out of date
out of date
>>> make
make -C dir/
make[1]: Entering directory `dir'
touch myfile
make[1]: Leaving directory `dir'
cp dir/myfile .
>>> make
make: Nothing to be done for `all'.
>>> make -q || echo out of date
>>> #nothing. success :D
I have a working make, I have platform code and like several makes for each os in the folder. Right now I have one makefile which works. I renamed it to Makefile.ws and wrote this in Makefile
all:
make -f Makefile.w32
clean:
make -f Makefile.w32 clean
I ran it and got this error
> "make"
make -f Makefile.w32
make[1]: Entering directory `/c/nightly/test'
make -f Makefile.w32
make[3]: Makefile.w32: No such file or directory
make[3]: *** No rule to make target `Makefile.w32'. Stop.
make[2]: *** [all] Error 2
make[1]: *** [build] Error 2
make[1]: Leaving directory `/c/nightly/test'
"make": *** [all] Error 2
Oddly enough the clean works perfectly. Then I decided to write "make -f Makefile.w32 mingw32" and that did not work correctly. In fact it made a folder called mingw32 which I thought was very strange.
As for the mingw32 rule I just copy build which I suspect is the main/normal rule that is used to build
$(BUILD):
#[ -d $# ] || mkdir -p $#
#make --no-print-directory -C $(BUILD) -f $(CURDIR)/Makefile
mingw32:
#[ -d $# ] || mkdir -p $#
#make --no-print-directory -C $(BUILD) -f $(CURDIR)/Makefile
full .w32 source is here http://pastie.org/320035
First, what make are you running? Cygwin or MinGW, or something else?
make -f Makefile.w32
make[1]: Entering directory `/c/nightly/test'
make -f Makefile.w32
make[3]: Makefile.w32: No such file or directory
"Entering directory" is a hint. Why is it entering /c/nightly/test? Is there a Makefile.w32 there?
As to creating the directory "mingw32", the rule
mingw32:
#[ -d $# ] || mkdir -p $#
...
does exactly that. If "mingw32" does not exist, it creates it.
It would be easier to help you if you had a shorter example and clearly explain what you want to accomplish and what you expect to happen.
I think I see the problem in your second example. The mingw32 line should be changed so that it does not include the $(BUILD) variable:
mingw32:
#[ -d $# ] || mkdir -p $#
#make --no-print-directory -C mingw32 -f $(CURDIR)/Makefile
It is clear that he created the directory,
your first command in your given 2 rules, include a mkdir with the object name, i.e. either build or mingw32
Afterwards he changes into the current directory (i.e. no cd at all) and execute Makefile. But as you wrote you renamed Makefile into Makefile.ws, so offcourse you are getting File-Not-Found-error.
From the questions I can only recommend you to have a look at an short introduction or even better the manual for make.
Have you tried calling the secondary makefile using
$(MAKE) -f ...
instead of
make -f ...?