Is there a simple way to automatically load a file containing local configurations into a Makefile?
I have a file, .env, containing key=value pairs on each line:
var1=val1
var2=val2
Currently I am importing each variable into my Makefile manually like this:
var1=$$(grep '^var1=' .env | cut -d= -f2-)
This feels clumsy and prone to bugs when adding more variables.
I have also tried adding an extra target to the Makefile to read each line of .env and pass it to export, but values exported in one target are not available to others.
I expect there is a built in feature for doing this in make, but I have not been able to find documentation for it.
Assuming your include file is comprised entirely of valid make assignments then the include directive is likely what you want here.
The include directive tells make to suspend reading the current makefile and read one or more other makefiles before continuing. The directive is a line in the makefile that looks like this:
include filenames…
filenames can contain shell file name patterns.
Related
Is there a generic was to specify a rule for something that goes from something with an extension to something with no extension? (in this case, '.in' to '')?
I have a project that's using automake. I have some config file templates that need substitutions before installing them. For example, I have a 'foo.ini.in' file that I want to use to create 'foo.ini'. I have a few of these scattered throughout the codebase. However, for each one, I need to create a custom Makefile rule to make them work. Is there a better way? I currently do this in Makefile.am:
bazdir=$(libdir)/#PACKAGE_NAME#/baz
EXTRA_DIST = foo.ini.in bar.ini.in
CLEANFILES = foo.ini bar.ini
foo.ini: foo.ini.in
sed -e 's|#LIBDIR#|$(bazdir)|g' $< > $#
bar.ini: bar.ini.in
sed -e 's|#LIBDIR#|$(bazdir)|g' $< > $#
This works fine, but I need to duplicate the rule for every file. I'd like to write one rule that will do the substitution for any 'ini.in' files to 'ini' files. I've tried:
%.ini.in: %.ini
sed ....
but autoconf complains that % pattern substitution is a non-portable gnumake-ism. It also doesn't like
.ini.in.ini:
sed .....
.. which I can't really blame it for not liking, because I can't even parse that.
Is there a way to do this?
NOTE: I can not use AC_CONFIG_FILES to do this substitution, because 'bazdir' is not fully expanded in that case.
Fortunately, there's a much better way: let the 'configured' files be generated from the templates using the configure script itself. As I mentioned in the comment, I don't see any reason to try to expand $bazdir in the Makefile.am itself. Maybe you can clear this up for me if there's some special reason for this.
In the configure.ac script, substitution variables are defined with the AC_SUBST macro. I'm guessing you want to replace LIBDIR with the value of $bazdir. Note that LIBDIR isn't a good choice of name, as libdir is already used in configure scripts; so let's use a variable name prepended with a project name: BAZ_LIBDIR
1/. set bazdir in the configure script: bazdir="${libdir}/${PACKAGE_NAME}/baz", and substitute with: AC_SUBST(BAZ_LIBDIR, $bazdir). Alternatively, just assign the value to BAZ_LIBDIR and use the single argument form of AC_SUBST. BAZ_LIBDIR is now ready for substitution...
2/. At the end of configure.ac, specify the files to be generated from their <file>.in templates, with AC_CONFIG_FILES. Typically this will list Makefile's, as well as data files, where the .in suffix is implicit.
Assuming a projectdir tree... could be any sort of tree layout of course:
BAZ_LIBDIR="${libdir}/${PACKAGE_NAME}/baz"
AC_SUBST(BAZ_LIBDIR)
...
AC_CONFIG_FILES([Makefile projectdir/Makefile])
AC_CONFIG_FILES([projectdir/foo.ini projectdir/bar.ini])
...
AC_OUTPUT
Instances of #BAZ_LIBDIR# in the <file>.ini.in files will be replaced with the substitution value. No arcane sed invocations are required to generate <file>.ini files.
Another nice feature is that you needn't add foo.ini.in or bar.ini.in to the EXTRA_DIST variable in the Makefile.am - and make distclean will clean the bar.ini and foo.ini files.
Consider having a makefile, which can generate some files using generating lines listed in a file. For example file 001 using first line, 002 using second line, etc.
This file can be changed (it has it's own dependences, but this doesn't metter).
If some lines in this file changed, appropriate files should be remade. But other files shouldn't.
The solution I found is this one: for every file there is flag-file which content is generating line it was made last time. After remaking file with generating lines, i check all this file, and remove them if line changed. So files which have dependences on removed files will be remade, and other files won't. But this works too slow if use msys-make.
Can you suggest any other solution, which doesn't need many extra calls to file system and executable runs.
If I understand your description correctly, what you're describing is a Makefile which depends on another file that is functionally a Makefile but for unknown reasons uses a different format.
Turn that file into Makefile format and include it into the original Makefile. (If you're using GNU Make.)
We have what may be similar to your problem. We have xml files, say foobar.xml which contains the dependencies for foobar.out
<files>
<file>a</file>
<file>b</file>
<file>c</file>
</files>
We decided to adhere to this simple layout, so we don't need to parse xml. We use makepp (because we got fed up with gmake not noticing dependencies like changed commands). This gives us the builtin &sed command (the expression is actually Perl programming, but as you see, you don't need to get into it much). Here's what we do with three simple substitutions for the three kinds of lines:
%.d: %.xml
&sed 's!<files>!$(stem).out: \\! || s!<file>(.+)</file>!$$1 \\! || s!</files>!!' \
$(input) -o $(output)
include foobar.d
This produces foobar.d which we then include:
foobar.out: \
a \
b \
c \
Stackoverflow is swallowing the last empty line here, which avoids having to worry about the trailing backslash.
I have a rather long list of preprocessor definitions that I want to make available to several C programs that are compiled with gcc.
Basically I could create a huge list of -DDEF1=1 -DDEF2=2 ... options to pass to gcc, but that would create a huge mess, is hard to use in a versioning-system and may at some time in the future break the command line length limit.
I would like to define my defines in a file.
Basically the -imacros would do what I want except that it only passes it to the first source file: (below from the gcc documentation):
-include file Process file as if #include "file" appeared as the first line of the primary source file. However, the first directory searched
for file is the preprocessor's working directory instead of the
directory containing the main source file. If not found there, it is
searched for in the remainder of the #include "..." search chain as
normal. If multiple -include options are given, the files are included
in the order they appear on the command line.
-imacros file Exactly like -include, except that any output produced by scanning file is thrown away. Macros it defines remain defined.
This allows you to acquire all the macros from a header without also
processing its declarations. All files specified by -imacros are
processed before all files specified by -include.
I need to have the definitions available in all source files, not just the first one.
Look at the bottom of this reference.
What you might want is the #file option. This option tells GCC to use file for command-line options. This file can of course contain preprocessor defines.
Honestly - it sounds like you need to do a bit more in your build environment.
For example, one suggestion is that it sounds like you should create a header file that is included by all your source files and #define all your definitions.
You could also use -include, but specify an explicit path - which should be determined in your Makefile/build environment.
The -imacros would work, if your Makefile were building each source file independently, into its own object file (which is typical). Its sounds like you're just throwing all the sources into building a single object.
Say I have a header file which is included by many source files, maybe with a very deep hierarchy. It is very boring to list this common header file in the prerequisites of each source object, and not sure whether there is an elegant solution. Thanks!
You can generate such dependencies with gcc -M. From TFM:
-M Instead of outputting the result of preprocessing, output a rule suitable for make describing the dependencies of the main source file. The preprocessor outputs one make rule containing the object file name for that source file, a colon, and the names of all the included files, including those coming from -include or -imacros command line options.
Also see Generating Prerequisites Automatically.
My Makefile.am includes a file (with various defined variables), for example:
include make.config
...
The problem is that this file is in turn generated by a tool (i.e. config.generator.sh) based on some input file (i.e. input.dat). The straightforward and wrong idea would be to add a rule to generate make.config:
make.config : input.dat
config.generator.sh input.dat > make.config
include make.config
...
Although this content is perfectly working makefile on its own without automake, the idea is doomed with automake. The make.config file is included by automake before I even have a chance to execute make (and it fails as the file is not yet generated):
automake: cannot open < make.config: No such file or directory
Is there a way to postpone effect of include directive until make is run (possibly by using another directive)?
There is probably a way to simply run arbitrary commands before any makefile generation is done (i.e. AC_CONFIG_COMMANDS*). But the question is more complicated because the config.generator.sh is supposed to use executables which are in turn also generated during the same build process (so there is a dependency chain which logically has to be managed by makefiles from the same project). The documentation simply confirms the logic without providing alternatives.
The solution is described in this email of Automake's mailing list.
The idea is to use include directives inside small regular "wrapper" makefile and include Automake-generated Makefile into it (note the upper case M). Because makefile is not an Automake template, the include works as expected triggering builds for non-existing files.
Note that:
By default make utility will search for makefile first (not for Makefile) making this approach working seamlessly.
It is still recommended to specify all rules inside Makefile.am and keep the "wrapper" makefile simple. The rules for non-existing files will naturally come from the generated Makefile anyway.
I've come across the same annoying problem today when moving my OCaml project to Autotools. My solution is to use autoconf's substitution to go around automake. For the above example, I'd add a substitution to configure.ac:
AC_SUBST([include_make_config], ["include make.config"])
and adjust Makefile.am, replacing the include directive with the autoconf variable reference:
make.config : input.dat
config.generator.sh input.dat > make.config
#include_make_config#
...
automake doesn't touch the #include_make_config# line so it gets carried over into the generated Makefile.in. When autoconf takes over, it substitutes the variable with include make.config in the final Makefile.
Note: I use this with OCaml's ocamldep dependency generator.