I have a bit of file processing to do in Automake and can't figure out how to do it correctly (using either automake rules of gnu make extensions). For example, I have a file called refName.in and wish to produce the output refName.out. I have two relevant command-lines:
produce-out refName.in refName.out
produce-deps refName.in
The first simply takes the input file and produces the output. The second provides a list of dependencies for the file.
What am I supposed to do in Automake to get this setup working? I wish to have full dependency tracking, such that if any file in the list produced by produce-deps will trigger produce-out to be called.
(NOTE: I have no problem changing how these commands work. One command can produce the output and dependencies if needed. Different command-line switches, etc. can also be modified.)
(PS: If need be a GNU make extension is totally okay. I already use patterned rules in the Makefile.am and other extensions.)
The only automake specific thing you need for this is to add the built file to BUILT_SOURCES:
BUILT_SOURCES := refName.out
# You might well want refName.in in EXTRA_DIST
EXTRA_DIST := refName.in
# then any standard Make rules will do:
refName.out: refName.in $(shell produce-deps refName.in)
produce-out refName.in refName.out
If you want a more sophisticated way of doing the dependencies you can use these rules with GNU make's include, which will only cause produce-deps to be run when refName.deps is missing or refName.in changes:
refName.deps: refName.in
echo 'ref-deps := "' $(shell produce-deps refName.in) '"' > refName.deps
include refName.deps
refName.out: refName.in $(ref-deps)
produce-out refName.in refName.out
You can make these rules generic with the usual GNU make % rules.
Related
I am having a problem creating Makefile build that has dependency of some other file content.
I know that I can write something like that in makefile to generate prerequisites on-the-fly:
result/nuclease.stat : $(shell cat config/nuclease.sample.list)
#bash
make result/nuclease.stat
But the problem is that the file config/nuclease.sample.list might not exist and I have a target for it.
If I could force make to require config/nuclease.sample.list before $(shell cat config/nuclease.sample.list) gets evaluated - problem is solved.
What kind of dependency I should make?
One more thing, I would like keep all gnu make's good qualities: download files only once, process them only once and etc..
Please check
The most straightforward answer is to use constructed include files. In this method you'd include a makefile that defined the dependency relationships you wanted, and provide a make rule that generated that included file with the information you want. That's all you have to do: make knows how to do the rest. Based on your question it might be something like:
include nuclease-sample-list.mk
nuclease-sample-list.mk: config/nuclease.sample.list
echo "result/nuclease.stat : $$(cat $<)" > $#
That's all you have to do. If you're using an older version of GNU make you might need to use -include instead of include to avoid seeing warning messages.
I encountered such pattern in makefile
CXXOBJ = f1.o f2.o f3.o
$(CXXOBJ): %.o: %.cpp
g++ -c $< -o $#
f1.o: f1.cpp f1.hpp f2.hpp
f2.o: f2.cpp f2.hpp f3.hpp macros.h
f3.o: f3.cpp f3.hpp
It works (at least with GNU make 4.0).
It uses generic recipe from 4th line,
but in addition uses dependencies defined at the bottom.
Questions
Is it standard make behavior? (or is it specific to GNU-make?)
Is it standard way to write make file? (i.e. are people usualy doing it this way or is it something 'exotic'?)
How exactly does it work?
How does make combine 2 distinct rules for same file? (just append dependency list or something more?)
(I was browsing through GNU-make manual, but could not find relevant part)
This is called static pattern rules (https://www.gnu.org/software/make/manual/html_node/Static-Usage.html). It is specific to GNU make. It might be useful when different targets require different recipes to build, but match the same pattern.
As for third question, there are no distinct rules for the same file. Everything is quite well defined, each target have corresponding .cpp file.
GNU Make manual:
One file can be the target of several rules. All the dependencies
mentioned in all the rules are merged into one list of dependencies
for the target....
There can only be one set of commands to be executed for a file. If
more than one rule gives commands for the same file, make uses the
last set given and prints an error message...
Is there a generic was to specify a rule for something that goes from something with an extension to something with no extension? (in this case, '.in' to '')?
I have a project that's using automake. I have some config file templates that need substitutions before installing them. For example, I have a 'foo.ini.in' file that I want to use to create 'foo.ini'. I have a few of these scattered throughout the codebase. However, for each one, I need to create a custom Makefile rule to make them work. Is there a better way? I currently do this in Makefile.am:
bazdir=$(libdir)/#PACKAGE_NAME#/baz
EXTRA_DIST = foo.ini.in bar.ini.in
CLEANFILES = foo.ini bar.ini
foo.ini: foo.ini.in
sed -e 's|#LIBDIR#|$(bazdir)|g' $< > $#
bar.ini: bar.ini.in
sed -e 's|#LIBDIR#|$(bazdir)|g' $< > $#
This works fine, but I need to duplicate the rule for every file. I'd like to write one rule that will do the substitution for any 'ini.in' files to 'ini' files. I've tried:
%.ini.in: %.ini
sed ....
but autoconf complains that % pattern substitution is a non-portable gnumake-ism. It also doesn't like
.ini.in.ini:
sed .....
.. which I can't really blame it for not liking, because I can't even parse that.
Is there a way to do this?
NOTE: I can not use AC_CONFIG_FILES to do this substitution, because 'bazdir' is not fully expanded in that case.
Fortunately, there's a much better way: let the 'configured' files be generated from the templates using the configure script itself. As I mentioned in the comment, I don't see any reason to try to expand $bazdir in the Makefile.am itself. Maybe you can clear this up for me if there's some special reason for this.
In the configure.ac script, substitution variables are defined with the AC_SUBST macro. I'm guessing you want to replace LIBDIR with the value of $bazdir. Note that LIBDIR isn't a good choice of name, as libdir is already used in configure scripts; so let's use a variable name prepended with a project name: BAZ_LIBDIR
1/. set bazdir in the configure script: bazdir="${libdir}/${PACKAGE_NAME}/baz", and substitute with: AC_SUBST(BAZ_LIBDIR, $bazdir). Alternatively, just assign the value to BAZ_LIBDIR and use the single argument form of AC_SUBST. BAZ_LIBDIR is now ready for substitution...
2/. At the end of configure.ac, specify the files to be generated from their <file>.in templates, with AC_CONFIG_FILES. Typically this will list Makefile's, as well as data files, where the .in suffix is implicit.
Assuming a projectdir tree... could be any sort of tree layout of course:
BAZ_LIBDIR="${libdir}/${PACKAGE_NAME}/baz"
AC_SUBST(BAZ_LIBDIR)
...
AC_CONFIG_FILES([Makefile projectdir/Makefile])
AC_CONFIG_FILES([projectdir/foo.ini projectdir/bar.ini])
...
AC_OUTPUT
Instances of #BAZ_LIBDIR# in the <file>.ini.in files will be replaced with the substitution value. No arcane sed invocations are required to generate <file>.ini files.
Another nice feature is that you needn't add foo.ini.in or bar.ini.in to the EXTRA_DIST variable in the Makefile.am - and make distclean will clean the bar.ini and foo.ini files.
My Makefile.am includes a file (with various defined variables), for example:
include make.config
...
The problem is that this file is in turn generated by a tool (i.e. config.generator.sh) based on some input file (i.e. input.dat). The straightforward and wrong idea would be to add a rule to generate make.config:
make.config : input.dat
config.generator.sh input.dat > make.config
include make.config
...
Although this content is perfectly working makefile on its own without automake, the idea is doomed with automake. The make.config file is included by automake before I even have a chance to execute make (and it fails as the file is not yet generated):
automake: cannot open < make.config: No such file or directory
Is there a way to postpone effect of include directive until make is run (possibly by using another directive)?
There is probably a way to simply run arbitrary commands before any makefile generation is done (i.e. AC_CONFIG_COMMANDS*). But the question is more complicated because the config.generator.sh is supposed to use executables which are in turn also generated during the same build process (so there is a dependency chain which logically has to be managed by makefiles from the same project). The documentation simply confirms the logic without providing alternatives.
The solution is described in this email of Automake's mailing list.
The idea is to use include directives inside small regular "wrapper" makefile and include Automake-generated Makefile into it (note the upper case M). Because makefile is not an Automake template, the include works as expected triggering builds for non-existing files.
Note that:
By default make utility will search for makefile first (not for Makefile) making this approach working seamlessly.
It is still recommended to specify all rules inside Makefile.am and keep the "wrapper" makefile simple. The rules for non-existing files will naturally come from the generated Makefile anyway.
I've come across the same annoying problem today when moving my OCaml project to Autotools. My solution is to use autoconf's substitution to go around automake. For the above example, I'd add a substitution to configure.ac:
AC_SUBST([include_make_config], ["include make.config"])
and adjust Makefile.am, replacing the include directive with the autoconf variable reference:
make.config : input.dat
config.generator.sh input.dat > make.config
#include_make_config#
...
automake doesn't touch the #include_make_config# line so it gets carried over into the generated Makefile.in. When autoconf takes over, it substitutes the variable with include make.config in the final Makefile.
Note: I use this with OCaml's ocamldep dependency generator.
We are using automake & autoconf to build our multi-package software. I was wondering how to fill a variable with the output of e.g. shell-scripts once and reuse this, e.g. for needed include dirs
INCLUDES := -I`some-ext-config --incdir`
Using := instead of = here makes this variable filled once so some-ext-config will only be called once (AFAIK this comes from plain make). Of course INCLUDES is the depreciated cousin of AM_CPPFLAGS, but would I have used that one instead, the shell script would have been called for each compile.
Using INCLUDES instead of AM_CPPFLAGS is an acceptable solution for me (though I imagine there might be portability issues), but I have no solution for e.g. LDFLAGS for a libtool library
libmylib_la_LDFLAGS := `some-ext-config --ldflags` # will always be evaluated
What is the general solution inside automake if I want to make sure these external tools are not called multiple times? I would like to stay away from using an obvious AC_SUBST in configure.ac since we have to make sure our packages can be build both from subdirectories (some configure.ac in there) and with an recursive make from the top-level and a configure.ac there which shouldn't need to know too much about the different subprojects.
:= is GNU-make specific, so you are advised to use just = in automake. If you do not want to run the shell script everytime INCLUDES (or AM_CPPFLAGS, does not matter, it would occur with either), then run the script in configure.ac and use variable substitution via AC_SUBST. That is essentially what pkg-config would do — and come to speak of it, you could just use that instead of some-ext-config if there is a .pc file.
# configure.ac
libfoo_CPPFLAGS=$(some-ext-config --incdir);
libfoo_LIBS=$(some-ext-config --libs);
AC_SUBST([libfoo_CPPFLAGS])
AC_SUBST([libfoo_LIBS])
# Makefile.am
AM_CPPFLAGS = -Iwhatever ${libfoo_CPPFLAGS}
bin_PROGRAMS = foo
foo_LDADD = ${libfoo_LIBS}
This is a more lengthy explanation of what I suggested in a comment to jørgensen's answer.
I understand your top-level configure.ac must generate the makefiles of multiple sub-projects, and performs the required tests so that you don't have to run the configure in any subproject (a sub-configure serves only when you want to work on this particular sub-project).
In that case, you want to avoid duplicating as much stuff as possible from various configure.ac. I suggest you factor all the code of the sub-configure that must also be performed by the top-level configure in an m4 macro. This includes tests, AC_SUBSTS, and Makefile declarations.
For instance using only one-subproject. Here is a top-level ./configure.ac:
AC_INIT([toplevel], [1.0])
AM_INIT_AUTOMAKE([foreign -Werror])
SUB1_COMMON([sub1/]) dnl Stuff from the subproject
AC_CONFIG_FILES([Makefile])
AC_OUTPUT
With ./Makefile.am:
ACLOCAL_AMFLAGS = -I sub1/m4
SUBDIRS = sub1
Now here is sub1/configure.ac for the sub-project:
AC_INIT([sub1], [1.0])
AM_INIT_AUTOMAKE([foreign -Werror])
AC_CONFIG_MACRO_DIR([m4])
SUB1_COMMON
AC_OUTPUT
With SUB1_COMMON defined in m4/common.m4:
AC_DEFUN([SUB1_COMMON],
[AC_SUBST([PYTHON3LIB], [`pkg-config --libs python3`])
AC_CONFIG_FILES([$1Makefile])])
And finally sub1/Makefile.am is just:
ACLOCAL_AMFLAGS = -I m4
# Build something.
...
The SUB1_COMMON contains all the code you want to share between the two configure.ac files, and use argument $1 to relocate the config files appropriately. In this example, the variable PYTHON3LIB will be defined regardless of which configure were run.