First, a little bit of background as to why I'm asking this question: Our product's daily build script (as run under Debian Linux by Jenkins), does roughly this:
Creates and enters a build environment using debootstrap and chroot
Checks out our codebase (including some 3rd party libraries) from SVN
Runs configure and make as necessary to build all of the code
Packages up the results into an install file that can be uploaded to our headless server boxes using our install tool.
This mostly works fine, but every so often (maybe one daily build out of 10), the part of the script that builds one of our third-party libraries will error out with an error like this one:
CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/bash
/root/software/3rdparty/libogg/missing autoconf
/root/software/3rdparty/libogg/missing: line 81: autoconf: command not found
WARNING: 'autoconf' is missing on your system.
You should only need it if you modified 'configure.ac',
or m4 files included by it.
The 'autoconf' program is part of the GNU Autoconf package:
<http://www.gnu.org/software/autoconf/>
It also requires GNU m4 and Perl in order to run:
<http://www.gnu.org/software/m4/>
<http://www.perl.org/>
make: *** [configure] Error 127
As far as I can tell, this happens occasionally because the timestamps of the files in the third-party library are different (e.g. off by a second or two from each other just due to the timing of when they were checked out from the SVN server during that particular build). That causes the configure script to think that it needs to auto-regenerate a file, so then it tries to call 'automake' to do so, and errors out because automake is not installed.
Of course the obvious thing to do here would be to install automake in the build environment, but the build environment is not one that I can easily modify (due to institutional reasons), so I'd like to avoid having to do that if possible. What I'd like to do instead is figure out how to get the configure scripts (which I can modify) to ignore the timestamps and just always do the basic build that they do when the timestamps are equal.
I tried to finesse the problem by manually running 'touch' on some files to force their timestamps to be the same, and that seemed to make the problem occur less often, but it still happens:
./configure --prefix="$PREFIX" --disable-shared --enable-static && \
touch config* aclocal* Makefile* && \
make clean && make install ) || Failure "libogg"
Can anyone familiar with how automake works supply some advice on how I might make the "configure" calls in our daily build work more reliably, without modifying the build environment?
You could try forcing SVN to use commit times on checkout on your Jenkins server. These commit times can also be set in SVN if they don't work out for some reason. You could use touch -d or touch -r instead of just touch to avoid race conditions there.
Related
I'm trying to compile a code to run in parallel on a supercomputer. I know that others have compiled this code to run on the same computer, but for some reason I am having trouble even when using the same methodology as them. For now I'm just trying to compile the code to run in serial as that should be easier to troubleshoot.
configure seems to work correctly.
However make install returns the following:
> make install
CDPATH="${ZSH_VERSION+.}:" && cd .. && /bin/sh /home1/username123/code123/config/missing aclocal-1.13 -I ./config -I /home1/username123/code123/build-tools/aclocal -I /usr/local/share/aclocal
aclocal-1.13: error: couldn't open directory '/usr/local/share/aclocal': No such file or directory
Makefile:534: recipe for target '../aclocal.m4' failed
make: *** [../aclocal.m4] Error 1
aclocal is indeed not located at /usr/local/share/aclocal, it is located at /usr/bin/aclocal - but as /usr/bin is in my path, I don't understand why the location is an issue.
As has been made clear in comments on the question, the problem was that the project sources were copied onto the target system in a way that failed to preserve their original timestamps. The Autotools, through make, use file timestamps to determine which files are out of date, and in particular, Autools-generated Makefiles contain rules for rebuilding the build system itself that can be triggered this way.
It is not ordinarily necessary or desirable to rebuild an Autotools project's build system, except in conjunction with actually performing maintenance on it. It is often the case, in fact, that the necessary support for that is not available locally. To avoid the build system thinking that it needs to rebuild itself, it is important to preserve the file timestamps from the distribution archive. For some packages, it also works to pass the --disable-maintainer-mode argument to the configure script, but by no means do all Autotools configure scripts support that.
The archive extraction tools for the typical archive formats in which Autotools-based packages are distributed do, by default, preserve timestamps when unpacking, so the ordinary procedure of
unpack the archive on the target system (e.g. tar xzf foo-1.2.3.tar.gz)
change to the unpacked source directory (e.g. cd foo-1.2.3)
configure; make; make install
normally does the right thing. Substituting something else for (1), however, such as copying the unpacked source tree from somewhere else, may cause the package to think it needs to rebuild the build system. If that works correctly then it's no big deal, but it is not uncommon that it doesn't. That's what happened here, and following the standard procedure described above solved the problem.
A few months ago I installed an application from source by executing
./configure --whatever
make
sudo make install
I reinstalled my OS recently and now I would like to install this application again, but I don't remember what compile flags I used back then. However, since I kept all files from my home, I still have the source code and my original build. Is there a way to recover the ./configure parameters I used originally?
just re-run make? after all, i think you are mainly after re-running the installation phase rather than the configure-step.
however, to specifically answer your question: autotools record your configure-invocation in the config.status file.
you can call it to re-run your configure phase "in the same conditions":
./config.status --recheck
(this is mainly needed internally by autotools: if you hack your Makefile.am or your configure.ac, automake will rebuild the build-system so the changes have an effect; in order to do this correctly, it must invoke configure in the same way as it was originally called (by the user))
I am trying make/install inkscape on WinXp. Although I am rather inexperienced on this subject, I have a reasonably well developed MingW infrastructure, and have successfully built/installed a variety of packages, such as Cairo, Pango, GTK+, Glade, etc.
At the moment I am trying to make/install inkscape. Following a huge amount of work, it all configures and compiles, with MSys:
cd ${LOCALBUILDDIR} && \
cd inkscape-0.91 && \
./configure --prefix=${LOCALDESTDIR} CFLAGS='-std=gnu++11' CXXFLAGS='-std=gnu++11' && \
make CFLAGS='-std=gnu++11' CXXFLAGS='-std=gnu++11' && \
make install
and after successful config/compile I use
cd ${LOCALBUILDDIR} && \
cd inkscape-0.91 && \
make CFLAGS='-std=gnu++11' CXXFLAGS='-std=gnu++11' && \
make install
to prevent recreating/overwriting the config etc files for the edits discussed below.
This fails on the link with:
make[3]: Entering directory `/build32/inkscape-0.91/src'
CXXLD inkscape.exe
libtool: link: cannot find the library `/usr/lib/libintl.la' or unhandled argument `/usr/lib/libintl.la'
At no time ever have I had any Dir's called "/usr". I appreciate that is the default on Unix etc, but my installation has its libs in D:/Apps/MingW/local32/lib", and which certainly includes libintl.la (and all the required gettext and libintl bits, as it would need for the success with all the other packages). Indeed, the inkscape configure and compile steps report finding libintl etc. correctly. Also, it obtains the path correctly for the compiler, etc. as it would need to for the success with the compiler.
I am guessing that somewhere in the inkscape configure/in/ac/m4 etc bits, it has hard-wired, or something, the link step for libintl for Unix default. I have tried many permutations to manually alter "/usr/lib" lines in the config and libtool files, but obtain the same link crash.
I would be grateful for any hints as to where to look for/correct this in inkscape (or even generally).
OK, sussed it. It wasn't libintl per se, even though that is the "error/crash message".
The problem was in libpopt.la, which had hardwired "/usr/lib" dependency, rather than correctly creating the .la for the Windows/MingW environment.
popt was required for inkscape.
I kept running AgenRansack on the inkscape build dir's looking for "/usr" etc, when I should have run AgentRansack on my /local32/lib dir.
It now compiles and links without complaint ... but doesn't launch ... oh well, on to that later :-(
I'm currently learning how to use the autoconf/automake toolchain. I seem to have a general understanding of the workflow here - basically you have a configure.ac script which generates an executable configure file. The generated configure script is then executed by the end user to generate Makefiles, so the program can be built/installed.
So the installation for a typical end-user is basically:
./configure
make
make install
make clean
Okay, now here's where I'm confused:
As a developer, I've noticed that the auto-generated configure script sometimes won't run, and will error with:
config.status: error: cannot find input file: `somedir/Makefile.in'
This confuses me, because I thought the configure script is supposed to generate the Makefile.in. So Googling around for some answers, I've discovered that this can be fixed with an autogen.sh script, which basically "resets" the state of the autoconf environment. A typical autogen.sh script would be something like:
aclocal \
&& automake --add-missing \
&& autoconf
Okay fine. But as an end-user who's downloaded countless tarballs throughout my life, I've never had to use an autogen.sh script. All I did was uncompress the tarball, and do the usual configure/make/make install/make clean routine.
But as a developer who's now using autoconf, it seems that configure doesn't actually run unless you run autogen.sh first. So I find this very confusing, because I thought the end-user shouldn't have to run autogen.sh.
So why do I have to run autogen.sh first - in order for the configure script to find Makefile.in? Why doesn't the configure script simply generate it?
In order to really understand the autotools utilities you have to remember where they come from: they come from an open source world where there are (a) developers who are working from a source code repository (CVS, Git, etc.) and creating a tar file or similar containing source code and putting that tar file up on a download site, and (b) end-users who are getting the source code tar file, compiling that source code on their system and using the resulting binary. Obviously the folks in group (a) also compile the code and use the resulting binary, but the folks in group (b) don't have or need, often, all the tools for development that the folks in group (a) need.
So the use of the tools is geared towards this split, where the people in group (b) don't have access to autoconf, automake, etc.
When using autoconf, people generally check in the configure.ac file (input to autoconf) into source control but do not check in the output of autoconf, the configure script (some projects do check in the configure script of course: it's up to you).
When using automake, people generally check in the Makefile.am file (input to automake) but do not check in the output of automake: Makefile.in.
The configure script basically looks at your system for various optional elements that the package may or may not need, where they can be found, etc. Once it finds this information, it can use it to convert various XXX.in files (typically, but not solely, Makefile.in) into XXX files (for example, Makefile).
So the steps generally go like this: write configure.ac and Makefile.am and check them in. To build the project from source code control checkout, run autoconf to generate configure from configure.ac. Run automake to generate Makefile.in from Makefile.am. Run configure to generate Makefile from Makefile.in. Run make to build the product.
When you want to release the source code (if you're developing an open source product that makes source code releases) you run autoconf and automake, then bundle up the source code with the configure and Makefile.in files, so that people building your source code release just need make and a compiler and don't need any autotools.
Because the order of running autoconf and automake (and libtool if you use it) can be tricky there are scripts like autogen.sh and autoreconf, etc. which are checked into source control to be used by developers building from source control, but these are not needed/used by people building from the source code release tar file etc.
Autoconf and automake are often used together but you can use autoconf without automake, if you want to write your own Makefile.in.
For this error:
config.status: error: cannot find input file: `somedir/Makefile.in'
In the directory where the configure.ac is located in the Makefile.am add a line with the subdirectory somedir
SUBDIRS = somedir
Inside somedir put a Makefile.am with all the description. then run automaker --add-missing
A better description can be found in 7.1 Recursing subdirectories automake manual.
https://www.gnu.org/software/automake/manual/automake.html
I have started recently to work with automake and autoconf and I am a little confused about how to distribute the code.
Usually when I get a code that works with a configure file, the only thing that I get is a confiure file and the code itself with the Makefile.am and so on. Usually I do
./configure
make
sudo make install
and thats all but when I generate my configure from a configure.ac file it toss out lots of files that I thought where just temporary but if I give the code to a partner and he makes configure, it doesn't work, it needs either remake the autoreconf or have all this files (namely instal.sh,config.sub...).
Is there something that I am missing? How can I distribute the code easily and clean?
I have searched a lot but I think I am searching for the right thing because I cannot find anything useful.
Thanks a lot in advance!
Automake provides a make dist target. This automatically creates a .tar.gz from your project. This archive is set up in such a way that the recipient can simply extract it and run the usual ./configure && make && make install invocation.
It is generally not recommended to check the files generated by Autotools into your repository. This is because they are derived objects. You wouldn't check in your .o files!
Usually, it is a good idea to provide a autogen.sh script that carries out any actions required to re-create the Autotools build infrastructure in a new version control system checkout. Often, it can be as simple as:
#!/bin/sh
autoreconf -i
Then set it chmod +x, and the instructions for compiling from a clean checkout can be ./autogen.sh && ./configure && make.