sub make with foreach fails randomly (with higher J factor) - makefile

Question

I am using GNU make, where I have a top level makefile, which invokes
another makefile, for different types of builds, like:
LIST_OF_TYPES: 32 64 ...
tgt-name: deps
$(foreach i,$(LIST_OF_TYPES), \
$(MAKE) -f $(MY_MAKEFILE) ARCH=$i mylib;)
when running with higher j factor like -j100 etc, one of the build fails, but
the return value is still 0 so, I cannot make out if the build really did work!
Is there anything wrong with the way I'm using the foreach construct?
or its just the higher j with foreach which is causing problems?

Answer 1

The top-level makefile you have listed is safe under -j (though badly sub-optimal). After the expansion of the $(foreach...), make effectively sees:
tgt-name: deps
$(MAKE) -f $(MY_MAKEFILE) ARCH=32 mylib; $(MAKE) -f $(MY_MAKEFILE) ARCH=64 mylib; ...
When one of these sub-makes fails (due to mis-handling of -j), the failure is not reported to the top level make. You need to use something like:
tgt-name: deps
$(MAKE) -f $(MY_MAKEFILE) ARCH=32 mylib && $(MAKE) -f $(MY_MAKEFILE) ARCH=64 mylib && ... && :
The && tells bash to exit immediately with an error if the previous command fails. (The : at the end is the bash builtin that does nothing but issue a successful exit—it will simplify your writing of the $(foreach ...).)
EDIT:
The proper way to this of course is to use make dependencies, not serial processing in the shell. You want make to see something like:
tgt-name: # default target
.PHONY: target-32
target-32: deps
$(MAKE) -f ${MY_MAKEFILE} arch=32 mylib
.PHONY: target-64
target-64: deps
$(MAKE) -f ${MY_MAKEFILE} arch=64 mylib
# etc. etc.
tgt-name: target-32 target-64
#echo $# Success
This is -j safe. Under -j make will make all of the target-% at the same time. Nice. (Though in this case it seems that your $MY_MAKEFILE is not -j safe (naughty!).) A few macros to replace the boiler plate:
LIST_OF_TYPES := 32 64 ...
LIST_OF_TARGETS := $(add-prefix,target-,${LIST_OF_TYPES})
tgt-name: # default target
.PHONY: ${LIST_OF_TARGETS}
${LIST_OF_TARGETS}: target-%: deps # Static Pattern Rule will (conveniently) set $*
$(MAKE) -f ${MY_MAKEFILE} arch=$* mylib
tgt-name: ${LIST_OF_TARGETS}
#echo $# Success
P.S. I suspect that you should be marking tgt-name as .PHONY

Answer 2

I've never seen this kind of use of foreach, but it seems to work for you. Usually I use a bash for loop
tgt-name: deps
for i in $(LIST_OF_TYPES); do $(MAKE) -f $(MY_MAKEFILE) ARCH=$$i mylib; done
But this is not the problem, since in either case the makes are run sequentially, AFAICS.
Since you build a library, there's the possible clash of two objects being inserted into an archive simultaneously. When this happens the archive might become corrupted.
As with every parallel execution, be it make jobs or threads, you must protect the shared resources (the archive in your case). You must add the objects at the end of the library build or protect the insertion with some lock (e.g. man lockfile or similar).
There might be other problems, of course. Look out for the simultaneous access to shared resources (object files, archives, ...) or incomplete defined dependencies.
Update:
foreach seems not to be a problem. Set LIST_OF_TYPES to a single type (e.g. 32 only) and then do a make -j100 mylib. If the problem is with the building of a single archive, it will fail with only one type as well.
You can also test with make ARCH=32 -j100 mylib. This should show the problem too.