I'm trying to compile a code to run in parallel on a supercomputer. I know that others have compiled this code to run on the same computer, but for some reason I am having trouble even when using the same methodology as them. For now I'm just trying to compile the code to run in serial as that should be easier to troubleshoot.
configure seems to work correctly.
However make install returns the following:
> make install
CDPATH="${ZSH_VERSION+.}:" && cd .. && /bin/sh /home1/username123/code123/config/missing aclocal-1.13 -I ./config -I /home1/username123/code123/build-tools/aclocal -I /usr/local/share/aclocal
aclocal-1.13: error: couldn't open directory '/usr/local/share/aclocal': No such file or directory
Makefile:534: recipe for target '../aclocal.m4' failed
make: *** [../aclocal.m4] Error 1
aclocal is indeed not located at /usr/local/share/aclocal, it is located at /usr/bin/aclocal - but as /usr/bin is in my path, I don't understand why the location is an issue.
As has been made clear in comments on the question, the problem was that the project sources were copied onto the target system in a way that failed to preserve their original timestamps. The Autotools, through make, use file timestamps to determine which files are out of date, and in particular, Autools-generated Makefiles contain rules for rebuilding the build system itself that can be triggered this way.
It is not ordinarily necessary or desirable to rebuild an Autotools project's build system, except in conjunction with actually performing maintenance on it. It is often the case, in fact, that the necessary support for that is not available locally. To avoid the build system thinking that it needs to rebuild itself, it is important to preserve the file timestamps from the distribution archive. For some packages, it also works to pass the --disable-maintainer-mode argument to the configure script, but by no means do all Autotools configure scripts support that.
The archive extraction tools for the typical archive formats in which Autotools-based packages are distributed do, by default, preserve timestamps when unpacking, so the ordinary procedure of
unpack the archive on the target system (e.g. tar xzf foo-1.2.3.tar.gz)
change to the unpacked source directory (e.g. cd foo-1.2.3)
configure; make; make install
normally does the right thing. Substituting something else for (1), however, such as copying the unpacked source tree from somewhere else, may cause the package to think it needs to rebuild the build system. If that works correctly then it's no big deal, but it is not uncommon that it doesn't. That's what happened here, and following the standard procedure described above solved the problem.
Related
I'm working on several supercomputers, which have different autotools versions.
When I do make after ./configure, some of them give me an error about wrong version of aclocal.
If I re-configure my configure.ac on one platform, then same thing happens on the other one.
For example, one platform would give me this:
CDPATH="${ZSH_VERSION+.}:" && cd . && aclocal-1.15
/bin/sh: aclocal-1.15: command not found
make: *** [aclocal.m4] Error 127
If I run autoreconf, it would work fine on it. However, if I use that newly generated configure.ac on some other platforms, they will give me the same error with different version number:
CDPATH="${ZSH_VERSION+.}:" && cd . && aclocal-1.13
/bin/sh: aclocal-1.15: command not found
make: *** [aclocal.m4] Error 127
I know I can just run autoreconf -fi every time I go to a different platform, but I heard it isn't a good practice to make end users to do it as it requires them to install autotools.
They have three different versions, and I don't know how to handle this. Is there any ways to automatically run autoreconf when I run ./configure to regenerate configure? Or is there a better approach to solve this problem?
I see two main ways of distributing your source code to the different supercomputers:
tarballs generated by make dist
version controlled source tree
If you are using make dist generated tarballs (case 1) to distribute your source code to the different supercomputers, you can use one single source tree extracted from the tarball for all different supercomputers, as long as you do out of source tree builds in different per-supercomputer directories.
If you are using a version controlled source tree (case 2) for both developing your source code and distributing it to the different supercomputers, you have two options:
keep the make dist generated files inside version control
keep all generated files outside version control
If you keep the make dist generated files inside of version control (case 21), every autoreconf run on a machine different to the one which generated those files will produce a changed set of generated files, and thus changes to the files without actual raw changes. So in this case, you need to define a single system on which you all development work which touches the build system. All other systems can only do development inside the source code. Then you can use one single version controlled source tree on all different supercomputers.
If you keep all generated files outside of version control (case 22), you will need different copies of the version controlled source tree for each different machine to avoid the tool version mismatches.
In the age of distributed version control systems like git, this (case 22) would clearly be my preferred option.
First, a little bit of background as to why I'm asking this question: Our product's daily build script (as run under Debian Linux by Jenkins), does roughly this:
Creates and enters a build environment using debootstrap and chroot
Checks out our codebase (including some 3rd party libraries) from SVN
Runs configure and make as necessary to build all of the code
Packages up the results into an install file that can be uploaded to our headless server boxes using our install tool.
This mostly works fine, but every so often (maybe one daily build out of 10), the part of the script that builds one of our third-party libraries will error out with an error like this one:
CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/bash
/root/software/3rdparty/libogg/missing autoconf
/root/software/3rdparty/libogg/missing: line 81: autoconf: command not found
WARNING: 'autoconf' is missing on your system.
You should only need it if you modified 'configure.ac',
or m4 files included by it.
The 'autoconf' program is part of the GNU Autoconf package:
<http://www.gnu.org/software/autoconf/>
It also requires GNU m4 and Perl in order to run:
<http://www.gnu.org/software/m4/>
<http://www.perl.org/>
make: *** [configure] Error 127
As far as I can tell, this happens occasionally because the timestamps of the files in the third-party library are different (e.g. off by a second or two from each other just due to the timing of when they were checked out from the SVN server during that particular build). That causes the configure script to think that it needs to auto-regenerate a file, so then it tries to call 'automake' to do so, and errors out because automake is not installed.
Of course the obvious thing to do here would be to install automake in the build environment, but the build environment is not one that I can easily modify (due to institutional reasons), so I'd like to avoid having to do that if possible. What I'd like to do instead is figure out how to get the configure scripts (which I can modify) to ignore the timestamps and just always do the basic build that they do when the timestamps are equal.
I tried to finesse the problem by manually running 'touch' on some files to force their timestamps to be the same, and that seemed to make the problem occur less often, but it still happens:
./configure --prefix="$PREFIX" --disable-shared --enable-static && \
touch config* aclocal* Makefile* && \
make clean && make install ) || Failure "libogg"
Can anyone familiar with how automake works supply some advice on how I might make the "configure" calls in our daily build work more reliably, without modifying the build environment?
You could try forcing SVN to use commit times on checkout on your Jenkins server. These commit times can also be set in SVN if they don't work out for some reason. You could use touch -d or touch -r instead of just touch to avoid race conditions there.
I'm currently learning how to use the autoconf/automake toolchain. I seem to have a general understanding of the workflow here - basically you have a configure.ac script which generates an executable configure file. The generated configure script is then executed by the end user to generate Makefiles, so the program can be built/installed.
So the installation for a typical end-user is basically:
./configure
make
make install
make clean
Okay, now here's where I'm confused:
As a developer, I've noticed that the auto-generated configure script sometimes won't run, and will error with:
config.status: error: cannot find input file: `somedir/Makefile.in'
This confuses me, because I thought the configure script is supposed to generate the Makefile.in. So Googling around for some answers, I've discovered that this can be fixed with an autogen.sh script, which basically "resets" the state of the autoconf environment. A typical autogen.sh script would be something like:
aclocal \
&& automake --add-missing \
&& autoconf
Okay fine. But as an end-user who's downloaded countless tarballs throughout my life, I've never had to use an autogen.sh script. All I did was uncompress the tarball, and do the usual configure/make/make install/make clean routine.
But as a developer who's now using autoconf, it seems that configure doesn't actually run unless you run autogen.sh first. So I find this very confusing, because I thought the end-user shouldn't have to run autogen.sh.
So why do I have to run autogen.sh first - in order for the configure script to find Makefile.in? Why doesn't the configure script simply generate it?
In order to really understand the autotools utilities you have to remember where they come from: they come from an open source world where there are (a) developers who are working from a source code repository (CVS, Git, etc.) and creating a tar file or similar containing source code and putting that tar file up on a download site, and (b) end-users who are getting the source code tar file, compiling that source code on their system and using the resulting binary. Obviously the folks in group (a) also compile the code and use the resulting binary, but the folks in group (b) don't have or need, often, all the tools for development that the folks in group (a) need.
So the use of the tools is geared towards this split, where the people in group (b) don't have access to autoconf, automake, etc.
When using autoconf, people generally check in the configure.ac file (input to autoconf) into source control but do not check in the output of autoconf, the configure script (some projects do check in the configure script of course: it's up to you).
When using automake, people generally check in the Makefile.am file (input to automake) but do not check in the output of automake: Makefile.in.
The configure script basically looks at your system for various optional elements that the package may or may not need, where they can be found, etc. Once it finds this information, it can use it to convert various XXX.in files (typically, but not solely, Makefile.in) into XXX files (for example, Makefile).
So the steps generally go like this: write configure.ac and Makefile.am and check them in. To build the project from source code control checkout, run autoconf to generate configure from configure.ac. Run automake to generate Makefile.in from Makefile.am. Run configure to generate Makefile from Makefile.in. Run make to build the product.
When you want to release the source code (if you're developing an open source product that makes source code releases) you run autoconf and automake, then bundle up the source code with the configure and Makefile.in files, so that people building your source code release just need make and a compiler and don't need any autotools.
Because the order of running autoconf and automake (and libtool if you use it) can be tricky there are scripts like autogen.sh and autoreconf, etc. which are checked into source control to be used by developers building from source control, but these are not needed/used by people building from the source code release tar file etc.
Autoconf and automake are often used together but you can use autoconf without automake, if you want to write your own Makefile.in.
For this error:
config.status: error: cannot find input file: `somedir/Makefile.in'
In the directory where the configure.ac is located in the Makefile.am add a line with the subdirectory somedir
SUBDIRS = somedir
Inside somedir put a Makefile.am with all the description. then run automaker --add-missing
A better description can be found in 7.1 Recursing subdirectories automake manual.
https://www.gnu.org/software/automake/manual/automake.html
This question has been asked many time but I am not able to resolve the problem from them so I am asking
I had installed Cygwin a few days ago.I tried using ./configure command but it says
-bash: ./configure: No such file or directory
I tried using
where configure
but I got the output
INFO: Could not find files for the given pattern(s).
then I tried grep configureand I got this output
/etc/bash_completion.d/configure
/usr/i686-pc-cygwin/sys-root/usr/share/libtool/libltdl/configure
/usr/share/ELFIO/configure
/usr/share/libtool/libltdl/configure
I tried to export the path and then run the ./configure but it also didn't worked.
I find no executable file named as configure in my cygwin bin directory.
Does it mean that I have to add configure file manually?How can I correct it?
NOTE :- I had also tried sh configure but it also didn't worked
If a software project is set up to be built using autoconf, that tool generates a script canonically called configure. It queries the system for various parameters that are subsequently used in the build, and is specific to the software package to be built. Different software projects have different configure scripts. They are all called configure, but their contents are not the same.
So, to actually build such a software project once that script was set up (usually done by the maintainers when packaging the source tarball for distribution), you call:
tar xzf <tarball>.gz # or xjf <tarball>.bz2 or whatever
cd <sourcedir> # the one you just untarred
./configure
make
make install
Note the prefix ./, which means "located in this directory" (i.e. the top directory of that project's source tree).
Actually, the better procedure is the so-called "out-of-tree build", when you set up a different directory for the binaries to be built in, so the source tree remains unmodified:
tar xzf <tarball>.gz # or xjf <tarball>.bz2 or whatever
mkdir builddir
cd builddir
../<sourcedir>/configure
make
make install
So, there is supposed to be no configure executable in your PATH, you are supposed to call the script of that name from the source tree you are trying to build from.
If I correctly understood...
Configure is not an application that should be installed on your system, but script that should be delivered with source code to prepare for make command. File named configure should be in the main directory of source code.
I understand that this is an old question. However many might find this solution helpful.
Normally we use the make command to compile a downloaded source in cygwin. In many cases it contains a autogen.sh file. Running that file with
bash autogen.sh
will in many case solve the problem. At least it solved my issue and i could then use the make command
I have started recently to work with automake and autoconf and I am a little confused about how to distribute the code.
Usually when I get a code that works with a configure file, the only thing that I get is a confiure file and the code itself with the Makefile.am and so on. Usually I do
./configure
make
sudo make install
and thats all but when I generate my configure from a configure.ac file it toss out lots of files that I thought where just temporary but if I give the code to a partner and he makes configure, it doesn't work, it needs either remake the autoreconf or have all this files (namely instal.sh,config.sub...).
Is there something that I am missing? How can I distribute the code easily and clean?
I have searched a lot but I think I am searching for the right thing because I cannot find anything useful.
Thanks a lot in advance!
Automake provides a make dist target. This automatically creates a .tar.gz from your project. This archive is set up in such a way that the recipient can simply extract it and run the usual ./configure && make && make install invocation.
It is generally not recommended to check the files generated by Autotools into your repository. This is because they are derived objects. You wouldn't check in your .o files!
Usually, it is a good idea to provide a autogen.sh script that carries out any actions required to re-create the Autotools build infrastructure in a new version control system checkout. Often, it can be as simple as:
#!/bin/sh
autoreconf -i
Then set it chmod +x, and the instructions for compiling from a clean checkout can be ./autogen.sh && ./configure && make.