How do I implement the make installcheck target when building a library? That is, I have a check target that creates a test program that links to the created library, called by some script in order to check if the library code functions properly, and I wish to perform the same checks but linking to the library after it has been installed; how can I implement this?
Since I can't find any guides on this, I will present the method that I have come up with, which has been patched together from reading the GNU automake manual and some general trial and error. It might be ugly, but it works...
If the check routines for building a library involve linking the library to a test program and seeing if that program works correctly, then to installcheck the library we need only do the same thing but link the test program to the installed library rather than the locally built library.
Let's call the library alpha (so we'll be creating libalpha.so and/or libalpha.a). Supposing alpha's source code is in the file alpha.cpp in the src directory, we'll create src/Makefile.am as usual:
# src/Makefile.am
lib_LTLIBRARIES = libalpha.la
libalpha_la_SOURCES = alpha.cpp
include_HEADERS = alpha.h
The check routine involves creating a binary beta that links to alpha. The source code for beta is in the file beta.cpp in the directory tests. The automake file tests/Makefile.am looks like this:
# tests/Makefile.am
check_PROGRAMS = beta
beta_SOURCES = beta.cpp
beta_CPPFLAGS = -I$(top_srcdir)/src
beta_LDADD = $(top_builddir)/src/libalpha.la
We will create our check and installcheck routines by creating "local" targets in tests/Makefile.am like so:
# tests/Makefile.am
# ...
check-local:
# recipe to run when 'make check' is called.
installcheck-local:
# recipe to run when 'make installcheck' is called.
The check and installcheck targets conflict because use of either target prevents the other target from executing properly (one target "taints" the build tree for the other target); in order for the other target to execute properly we need to remove beta and its object files and have the target recompile and re-link as it sees fit according to its nature (installcheck to installed files, check to local files).
We can solve this issue of tainted build trees by simply running make clean in the recipe of both targets. This will clearly remove the tainted builds, but is overzealous because we don't need to rebuild whenever we run the same target again. The build tree is only tainted whenever we run the other target.
We can only solve this complication by remembering which of the two targets had been called previously, which we can do via the creation/destruction of an intermediary file (let's call it taint). The check target is tainted whenever the file taint exists, which it resolves by cleaning, rebuilding and removing taint; and the installcheck target is tainted when the file taint does not exist, which it resolves by cleaning, rebuilding and creating taint.
Our targets will take the form:
# tests/Makefile.am
# ...
check-local:
# First, check to see if the build tree is tainted and rebuild if so
test ! -f taint || $(MAKE) $(AM_MAKEFLAGS) check_rebuild
# Then, run our check tests. Substitute with your shell scripts or testsuite files as appropriate
./beta
installcheck-local:
# First, check to see if the build tree is tainted and rebuild if so
test -f taint || $(MAKE) $(AM_MAKEFLAGS) installcheck_rebuild
# Then, run our installcheck tests. Substitute with your shell scripts or testsuite files as appropriate
./beta
The target check_rebuild needs to rebuild according to how check will run, and will look like this:
# tests/Makefile.am
# ...
.PHONY: check_rebuild
check_rebuild:
$(MAKE) $(AM_MAKEFLAGS) clean # remove tainted build tree
$(MAKE) $(AM_MAKEFLAGS) beta$(EXEEXT) # rebuild beta
rm -f taint # mark build tree as untainted
The target installcheck_rebuild likewise looks like this:
# tests/Makefile.am
# ...
.PHONY: installcheck_rebuild
installcheck_rebuild:
$(MAKE) $(AM_MAKEFLAGS) clean # remove tainted build tree
$(MAKE) $(AM_MAKEFLAGS) beta$(EXEEXT) \
beta_CPPFLAGS="-I$(DESTDIR)$(includedir)" \
beta_LDADD="$(DESTDIR)$(libdir)/libalpha.la" \
beta_DEPENDENCIES="$(DESTDIR)$(libdir)/libalpha.la"
echo 1 > taint # mark build tree as untainted
Note that rebuilding beta in installcheck_rebuild now requires the overriding of the automake variables so that they point to the installed library.
Finally, we need to add the taint file to DISTCLEANFILES so that running distcheck does not fail with "files left in build directory after distclean" errors.
And that should be it. The final tests/Makefile.am should look like this:
# tests/Makefile.am
check_PROGRAMS = beta
beta_SOURCES = beta.cpp
beta_CPPFLAGS = -I$(top_srcdir)/src
beta_LDADD = $(top_builddir)/src/libalpha.la
taint_file = .taint
check-local:
test ! -f $(taint_file) || $(MAKE) $(AM_MAKEFLAGS) check_rebuild
./beta # substitute with your actual test routines
installcheck-local:
test -f $(taint_file) || $(MAKE) $(AM_MAKEFLAGS) installcheck_rebuild
./beta # substitute with your actual test routines
.PHONY: check_rebuild
check_rebuild:
$(MAKE) $(AM_MAKEFLAGS) clean
$(MAKE) $(AM_MAKEFLAGS) beta$(EXEEXT)
rm -f $(taint_file)
.PHONY: installcheck_rebuild
installcheck_rebuild:
$(MAKE) $(AM_MAKEFLAGS) clean
$(MAKE) $(AM_MAKEFLAGS) beta$(EXEEXT) \
beta_CPPFLAGS="-I$(DESTDIR)$(includedir)" \
beta_LDADD="$(DESTDIR)$(libdir)/libalpha.la" \
beta_DEPENDENCIES="$(DESTDIR)$(libdir)/libalpha.la"
echo 1 > $(taint_file)
DISTCLEANFILES = $(taint_file)
Disclaimer
This has been checked on Linux for a "standard" build, but it may not work on other build environments or if you are trying to do something "exotic". Hopefully it should, but it is not something that I have bothered to check. If there are errors, the problem will likely be a missing or misused variable in one of my targets above.
For example, the check_rebuild target has the line:
check_rebuild:
# ...
$(MAKE) $(AM_MAKEFLAGS) beta$(EXEEXT)
# ...
The variables $(AM_MAKEFLAGS) and $(EXEEXT) are a part of how automake itself creates routines to populate in the Makefiles it creates, and had I neglected them in my targets above, then it might have caused the build to fail (or at least not function as expected).
I have tried to make sure the targets that I have suggested above are likewise canonically constructed, but I may have missed something out. Your best bet in case of build errors is to open the Makefiles generated by automake itself during the build, seeing how it is creating objects and mimicking those constructs in the corresponding Makefile.am files, as I have tried to do.
The other likely issue may be in the "hackish" way I've built the beta binary in the installcheck_rebuild target. Again, your best bet in diagnosing problems will be to see how automake is doing things in the Makefiles it generates, and trying to mimic that. Otherwise, a read of the automake manual will be in order, and failing that, you will likely need the help of people more knowledgeable than me. Good luck.
Related
Basically I have the usual Makefile construct:
target: dependency1 dependency2 dependency3
runtargetscript.sh
However in this case, the target only needs one of the dependencies and some dependencies may not be buildable. (so I cannot just build all dependencies)
Is it possible to tell make to trigger "target" when one of the dependencies changed/was created (i.e. normal behaviour) but NOT to try to rebuild any missing dependencies?
With GNU make you can use shell escapes to build the dependencies dynamically, adding them only if they already exist:
if_exist = $(shell if [ -e $(1) ]; then echo $(1); fi)
target: $(call if_exist,dependency1) $(call if_exist,dependency2) $(call if_exist,dependency3)
runtargetscript.sh
This will run the script if target does not exist, or if it is older than any of the dependencies that do exist at the time the makefile was read, but will not attempt to build them if they do not exist at that time.
Note the important caveat there -- if the file(s) do not exist, but some other unrelated rule runs an action that creates them, it won't rebuild target, unless you rerun make target again.
Assuming your rule body does not do anything special depending on which dependencies are newer than the target you can use the -W flag to make to instruct it to consider certain targets as always new (and thus not in need of building).
So for the given example assuming you can (and want) to build dependency2 but not dependency1 or dependency3 you would run:
make -W dependency1 -W dependency3 target
Edit: As pointed out in the comments this does not work correctly when dependency2 is not newer than target as target will still be built.
In that case I believe the only solution (given the comments below) is to use something like:
DEPENDENCY_BIN := $(or $(wildcard /path/to/mysql),$(wildcard /path/to/sqlite3),/path/that/does/not/exist)
target: $(DEPENDENCY_BIN) dependency2
I'd like a set of makefile rules that create a symlink to one of several code modules before building the project. The name of the make target would determine the file to which the symlink points. For example:
The user invokes 'make R3000'
Make sees that 'data.asm' doesn't exist yet, so a symlink is created from 'data_R3000.asm' to 'data.asm'
The build process continues, using data.asm
How can I set up make rules to do this?
Maybe something like:
MODULES := $(patsubst data_%.asm,%,$(wildcard data_*.asm))
all:
...
data.asm:
[ -n "$(filter $(MAKECMDGOALS),$(MODULES))" ] || { echo unknown module: $(MAKECMDGOALS) ; exit 1; }
ln -s $(filter $(MAKECMDGOALS),$(MODULES)) $#
Then make sure data.asm is listed as a prerequisite in the appropriate rules.
I would do something like this:
.PHONY mklink
mklink:
test -e data_$(MAKECMDGOALS).asm || exit 1
ln -s data_$(MAKECMDGOALS).asm data.asm
and then make all (and other targets) dependent on mklink. The reason you shouldn't make data.asm your target in the rule is that if you run make R3000, then data.asm will be created, and then if you run make L2000, the data.asm file will be pointing to the wrong directory, and will not be overwritten (I'll assuming this is not what you want). The test line checks if the link target exists, and if not, it exits with 1, causing the target to fail. You should also add a check that MAKECMDGOALS is exactly one element as well.
I am using GNU make, where I have a top level makefile, which invokes
another makefile, for different types of builds, like:
LIST_OF_TYPES: 32 64 ...
tgt-name: deps
$(foreach i,$(LIST_OF_TYPES), \
$(MAKE) -f $(MY_MAKEFILE) ARCH=$i mylib;)
when running with higher j factor like -j100 etc, one of the build fails, but
the return value is still 0 so, I cannot make out if the build really did work!
Is there anything wrong with the way I'm using the foreach construct?
or its just the higher j with foreach which is causing problems?
The top-level makefile you have listed is safe under -j (though badly sub-optimal). After the expansion of the $(foreach...), make effectively sees:
tgt-name: deps
$(MAKE) -f $(MY_MAKEFILE) ARCH=32 mylib; $(MAKE) -f $(MY_MAKEFILE) ARCH=64 mylib; ...
When one of these sub-makes fails (due to mis-handling of -j), the failure is not reported to the top level make. You need to use something like:
tgt-name: deps
$(MAKE) -f $(MY_MAKEFILE) ARCH=32 mylib && $(MAKE) -f $(MY_MAKEFILE) ARCH=64 mylib && ... && :
The && tells bash to exit immediately with an error if the previous command fails. (The : at the end is the bash builtin that does nothing but issue a successful exit—it will simplify your writing of the $(foreach ...).)
EDIT:
The proper way to this of course is to use make dependencies, not serial processing in the shell. You want make to see something like:
tgt-name: # default target
.PHONY: target-32
target-32: deps
$(MAKE) -f ${MY_MAKEFILE} arch=32 mylib
.PHONY: target-64
target-64: deps
$(MAKE) -f ${MY_MAKEFILE} arch=64 mylib
# etc. etc.
tgt-name: target-32 target-64
#echo $# Success
This is -j safe. Under -j make will make all of the target-% at the same time. Nice. (Though in this case it seems that your $MY_MAKEFILE is not -j safe (naughty!).) A few macros to replace the boiler plate:
LIST_OF_TYPES := 32 64 ...
LIST_OF_TARGETS := $(add-prefix,target-,${LIST_OF_TYPES})
tgt-name: # default target
.PHONY: ${LIST_OF_TARGETS}
${LIST_OF_TARGETS}: target-%: deps # Static Pattern Rule will (conveniently) set $*
$(MAKE) -f ${MY_MAKEFILE} arch=$* mylib
tgt-name: ${LIST_OF_TARGETS}
#echo $# Success
P.S. I suspect that you should be marking tgt-name as .PHONY
I've never seen this kind of use of foreach, but it seems to work for you. Usually I use a bash for loop
tgt-name: deps
for i in $(LIST_OF_TYPES); do $(MAKE) -f $(MY_MAKEFILE) ARCH=$$i mylib; done
But this is not the problem, since in either case the makes are run sequentially, AFAICS.
Since you build a library, there's the possible clash of two objects being inserted into an archive simultaneously. When this happens the archive might become corrupted.
As with every parallel execution, be it make jobs or threads, you must protect the shared resources (the archive in your case). You must add the objects at the end of the library build or protect the insertion with some lock (e.g. man lockfile or similar).
There might be other problems, of course. Look out for the simultaneous access to shared resources (object files, archives, ...) or incomplete defined dependencies.
Update:
foreach seems not to be a problem. Set LIST_OF_TYPES to a single type (e.g. 32 only) and then do a make -j100 mylib. If the problem is with the building of a single archive, it will fail with only one type as well.
You can also test with make ARCH=32 -j100 mylib. This should show the problem too.
I want to create a Makefile (in a parent dir) to call several other Makefiles (in sub dirs) such that I can build several binaries (one per project sub dir) by invoking just the one parent Makefile.
My research has been hampered by finding loads of stuff on recursive Makefiles, but I think this is where you are trying to build several directories Makefiles into a single binary?
Maybe what I want to do is better handled by a shell script perhaps invoking make in each sub directory in turn, but I thought a Makefile might be a more elegant solution?
any pointers gratefully received
PS using linux and the GNU tool chain
The for loop solution given in the first answer above actually shouldn't be used, as-is. In that method, if one of your sub-makes fails the build will not fail (as it should) but continue on with the other directories. Not only that, but the final result of the build will be whatever the exit code of the last subdirectory make was, so if that succeeded the build succeeds even if some other subdirectory failed. Not good!!
You could fix it by doing something like this:
all:
#for dir in $(SUBDIRS); \
do \
$(MAKE) -C $${dir} $# || exit $$?; \
done
However now you have the opposite problem: if you run "make -k" (continue even if there are errors) then this won't be obeyed in this situation. It'll still exit on failure.
An additional issue with both of the above methods is that they serialize the building of all subdirectories, so if you enable parallel builds (with make's -j option) that will only happen within a single subdirectory, instead of across all subdirectories.
Eregrith and sinsedrix have solutions that are closer to what you want, although FYI you should never, ever use "make" when you are invoking a recursive make invocation. As in johfel's example you should ALWAYS use $(MAKE).
Something like this is what you want:
SUBDIRS = subdir1 subdir1 subdir3 ...
all: $(addprefix all.,$(SUBDIRS))
all.%:
# $(MAKE) -C '$*' '$(basename $#)'
.PHONY: $(addprefix all.,$(SUBDIRS))
And of course you can add more stanzas like this for other targets such as "install" or whatever. There are even more fancy ways to handle building subdirectories with any generic target, but this requires a bit more detail.
If you want to support parallel builds you may need to declare dependencies at this level to avoid parallel builds of directories which depend on each other. For example in the above if you cannot build subdir3 until after both subdir1 and subdir2 are finished (but it's OK for subdir1 and subdir2 to build in parallel) then you can add something like this to your makefile:
all.subdir3 : all.subdir1 all.subdir2
You can call targets in subdirectory makefiles via
all:
$(MAKE) -C subdirectory1 $#
$(MAKE) -C subdirectory2 $#
...
or better
SUBDIRS=subd1 subd2 subd3
all:
#for dir in $(SUBDIRS); \
do \
$(MAKE) -C $${dir} $#; \
done
you should indeed use cmake to generate the Makefile automatically from a given CMakeLists.txt configuration file.
Here's a random link to get you started. Here you can find a simple sample project, including multiple subdirectories, executables, and a shared library.
Each makefile can have several target, it's still true with recursive makefiles, usually it's written:
all: target1 target2 target3
target1 :
make -C subdir
Then make all
At work we use a common makefile that other makefiles include (via the include statement) and it has a generic "clean" target that kills some common files. I want to add on to that target in my new makefile so I can delete some specific files, but if I add a clean target in my makefile, it just overrides the old one.
I know I can just make a new target with a new name and have it call clean, and then do other stuff, but for sake of consistency I'd like to be able to just call make clean and have it do everything.
Is that possible?
I've seen this done at several shops. The most common approach is to use double-colon rules, assuming you're using something like GNU make. In your common makefile you would have something like this:
clean::
# standard cleanup, like remove all .o's:
rm -f *.o
Note that there are two colons following clean, not just one!
In your other makefile you just declare clean again, as a double-colon rule:
clean::
# custom cleanup, like remove my special generated files:
rm -f *.h.gen
When you invoke make clean, GNU make will automagically run both of these "branches" of the clean rule:
% make clean
rm -f *.o
rm -f *.h.gen
It's simple to set up and it composes quite neatly I think. Note that specifically because it is a double-colon rule, you don't get the "overriding commands" errors you normally get when you define two rules for the same target. That's sort of the point of double-colon rules.
You can write your own clean and make it a preq of the common clean.
clean: myclean
myclean:
rm whatever
Yours will run first. If for some reason you want the common clean to run first then the solution will be more complicated.
EDIT:
Here is the best solution I can see which runs the common rule before the local one:
include Makefile.common
clean:
$(MAKE) -f Makefile.common $#
rm whatever additional things
The include directive is necessary because the local makefile relies on the common one for things other than clean. The local clean rule overrides the common clean rule, but invokes the common clean rule before doing the additional work. (This overriding will cause some warnings, which is a nuisance; I don't know a good way to silence them.)
Use implicit rules:
existing-target: my-extention
my-extention:
echo running command 1
echo running command 2
Very simple make tutorial to ramp up.
When using :: you can run into issues since make complains when you mix single colon : and double colon :: rules:
a:
echo a
a::
echo aa
will result in:
. . .
*** target file `a' has both : and :: entries. Stop.
It seems like the common makefile's rule should be called something like common-clean. Then each main makefile would declare their clean rule as
clean: common-clean
and you're set.
If that isn't an option, you could take a look at double colon rules, but those introduce a whole other set of issues to consider.
Adding another possible solution I've seen for posterity... I know the OP was wary about changing the common makefile, but something like this works and involves minimal changes.
local makefile 1:
CLEAN=MyExe1 MyExe2
....
include /my/common/makefile
local makefile 2:
CLEAN=MyExe3 MyExe4
....
include /my/common/makefile
common makefile:
clean:
rm -f *.dep *.o *.a $(CLEAN)
Basically the idea is to define some variable (in this case CLEAN) in each local makefile with all the specific items you want to delete. Then the common makefile runs rm -f on all the common file types to delete, plus whatever was specifically flagged for deletion in each local makefile via the CLEAN variable. If there's nothing specific to delete, simply omit the variable declaration or leave it empty (CLEAN=)
So now if we run make clean for local makefile 1, it executes
rm -f *.dep *.o *.a MyExe1 MyExe2
And if we run make clean for local makefile 2, it executes
rm -f *.dep *.o *.a MyExe3 MyExe4
I've found a better solution:
.PHONY: my-extra-clean
clean: my-extra-clean
my-extra-clean:
rm <whatever-you-want>
include Makefile.common
The key line is clean: my-extra-clean. Ie, you can add dependencies in separate stanzas in different makefiles to add behaviour. my-extra-clean is run as a dependency of the root clean target.
For ours, we define a variable, EXTRAFILESTOCLEAN, then when the clean rule runs, it has a step to remove anything specified in the EXTRAFILESTOCLEAN variable
clean:
rm -f *.o
ifdef $(EXTRAFILESTOCLEAN)
rm -f $(EXTRAFILESTOCLEAN)
endif
That can cause unexpected problems if you set that variable to weird values, but you could guard against those by adding prefixes or other tests.
It's in the docs: https://www.gnu.org/software/make/manual/html_node/Overriding-Makefiles.html
So instead of include Makefile you use a wildcard target and forward it to the base Makefile:
# -include base.Makefile <--- not this
%:
#$(MAKE) -f base.Makefile $#