My .la file has full pathnames in both the dependency_libs= section and the libdir= section which makes it difficult to copy my libraries to a different machine (same arch but different path structure). What is the solution to this, besides having some script to hack the .la file to adjust for the paths on the new machine?
==Details==
When I ./configure; make; make install my libfoo, depending on how I use the --prefix, --exec-prefix, and DESTDIR= flags, I'll get an entry in the libfoo.la file that reads libdir=/dir1/lib and I'll have the actual .so files in the same dir as libfoo.la. All is well (in terms of linking something with libfoo as it is) until I package these up and put them on another machine.
Let's say I have libbar on my second machine which depends on libfoo. When I use my -L/dir2/lib flag to look for -lfoo, the libbar compilation/linking fails because of the libfoo.la file is expecting foo to be installed in /dir1/lib (from the first machine) when it's actually in /dir2/lib. I then need to replace all of the dir1's with the correct path, both of which can be long and complicated.
The dependency_libs= line also comes into play in a similar manner.
How do I avoid this problem?
Related
A user of xnec2c was trying to build on OSX and had autoconf issues because PKG_CHECK_MODULES could not be found since MacPorts puts it in a funny spot.
The user made autoconf work like so:
ACLOCAL_PATH=/opt/local/share/aclocal ./autogen.sh
ACLOCAL_PATH=/opt/local/share/aclocal ./configure
I would like to make it build on OSX without special user path hacks for ACLOCAL_PATH. Can that be done?
I started writing a possible fix below and realized it could an xyproblem so posed the question just above. However, if this starts any gears turning, then I would be open to a bit of special-casing for OSX:
For example, would it be possible (if not advisable) to detect:
Is PKG_CHECK_MODULES missing?
If so:
is it OSX?
Is [ -d /opt/local/share/aclocal ] true?
Does the macro exist there?
While aclocal has a few ways of appending to its search path (see https://www.gnu.org/software/automake/manual/html_node/Macro-Search-Path.html), you cannot modify that macro search path using code in configure.ac:
When the shell code in configure is run, it is too late, as the available macros have already been expanded. When autoconf (is it autoconf or something else? anyway, m4 called from autoreconf) generates configure from configure.ac by having m4 expand the macros it is also too late: aclocal has already collected the m4 macros it could find.
So what you would need is a step before the autoreconf run - which is beyond what I would consider a buildsystem needs to do.
What you can do: Put static strings into the top level Makefile.am file like e.g.
ACLOCAL_AMFLAGS = -I auto-m4 -I project-m4 -I /opt/local/share/aclocal
(this example uses auto-m4/ with AC_CONFIG_MACRO_DIR([auto-m4]) for the *.m4 files automatically put there by autoreconf/autopoint/libtoolize and project-m4/ for the project specific *.m4 files).
Of course, you should already have
m4_pattern_forbid([PKG_CHECK_MODULES])dnl
before invoking PKG_CHECK_MODULES for the first time so that the problem of the missing *.m4 file will be detected at the earliest possible time, i.e. when autoconf is about to generate a configure file with PKG_CHECK_MODULES unexpanded.
You could use some m4 code to print a lengthy error message if PKG_CHECK_MODULES is not defined. Something along the lines of (untested)
m4_ifndef([PKG_CHECK_MODULES], [dnl
m4_fatal([Could not find the PKG_CHECK_MODULES macro. Check that the pkg.m4 file is available and aclocal finds it (e.g. set ACLOCAL_PATH=/opt/local/share/aclocal).
])dnl
PKG_CHECK_MODULES([FOO], [foo])
Personally, I would go with m4_pattern_forbid and make sure OSX builds with homebrew work OOTB, and then document idiosyncrasies for building on rare and buggy systems like OSX with macports or SunOS without GNU tools in the INSTALL file.
Isn't it a bug in macports/OSX that aclocal there cannot find its *.m4 files? Shouldn't there be a dirlist file pointing to /opt/local/share/aclocal? Or perhaps they macports users should have an aclocal in their PATH which actually finds the macports macro files?
In any case, I would not consider it my build systems's job to fix a buggy system. You need to draw the line somewhere.
I'm trying to compile a code to run in parallel on a supercomputer. I know that others have compiled this code to run on the same computer, but for some reason I am having trouble even when using the same methodology as them. For now I'm just trying to compile the code to run in serial as that should be easier to troubleshoot.
configure seems to work correctly.
However make install returns the following:
> make install
CDPATH="${ZSH_VERSION+.}:" && cd .. && /bin/sh /home1/username123/code123/config/missing aclocal-1.13 -I ./config -I /home1/username123/code123/build-tools/aclocal -I /usr/local/share/aclocal
aclocal-1.13: error: couldn't open directory '/usr/local/share/aclocal': No such file or directory
Makefile:534: recipe for target '../aclocal.m4' failed
make: *** [../aclocal.m4] Error 1
aclocal is indeed not located at /usr/local/share/aclocal, it is located at /usr/bin/aclocal - but as /usr/bin is in my path, I don't understand why the location is an issue.
As has been made clear in comments on the question, the problem was that the project sources were copied onto the target system in a way that failed to preserve their original timestamps. The Autotools, through make, use file timestamps to determine which files are out of date, and in particular, Autools-generated Makefiles contain rules for rebuilding the build system itself that can be triggered this way.
It is not ordinarily necessary or desirable to rebuild an Autotools project's build system, except in conjunction with actually performing maintenance on it. It is often the case, in fact, that the necessary support for that is not available locally. To avoid the build system thinking that it needs to rebuild itself, it is important to preserve the file timestamps from the distribution archive. For some packages, it also works to pass the --disable-maintainer-mode argument to the configure script, but by no means do all Autotools configure scripts support that.
The archive extraction tools for the typical archive formats in which Autotools-based packages are distributed do, by default, preserve timestamps when unpacking, so the ordinary procedure of
unpack the archive on the target system (e.g. tar xzf foo-1.2.3.tar.gz)
change to the unpacked source directory (e.g. cd foo-1.2.3)
configure; make; make install
normally does the right thing. Substituting something else for (1), however, such as copying the unpacked source tree from somewhere else, may cause the package to think it needs to rebuild the build system. If that works correctly then it's no big deal, but it is not uncommon that it doesn't. That's what happened here, and following the standard procedure described above solved the problem.
When building a binary or library, specifying the rpath, i.e.
-Wl,rpath,<path/to/lib>
tells the linker where to find the required library at runtime of the binary.
What is the UNIX philosphy regarding absolute and relative paths here? Is it better to use an absolute path so the lib can be found from everywhere? Or is it better to make it relative so copying an entire directory or renaming a higher level path won't render the binary unusable?
Update
Using $ORIGIN is usually the preferred way of building binaries. For libraries I like to put in the absolute path, because otherwise you won't be able to link to the library. A symbolic link will change the $ORIGIN to point to the path of the link and not of the link target.
In the case of rpath, it makes no sense to use a relative path, since a relative path will be relative to the current working directory, NOT relative to the directory where the binary/library was found. So it simply won't work for executables found in $PATH or libraries in pretty much any case.
Instead, you can use the $ORIGIN "special" path to have a path relative to the executable with-Wl,-rpath,'$ORIGIN' -- note that you need quotes around it to avoid having the shell interpret it as a variable, and if you try to do this in a Makefile, you need $$ to avoid having make interpret the $ as well.
What is the UNIX philosphy regarding absolute and relative paths here?
Using relative path makes an executable that only works when invoked from a particular directory, which is almost never what you want. E.g. if the executable is in /app/foo/bin/exe and has DT_RUNPATH of lib/, and a dependent library is in /app/foo/lib/libfoo.so, then the exe would only run when invoked from /app/foo, and not when invoked from any other directory.
Using absolute path is much better: you can do cd /tmp; /app/foo/bin/exe and have the executable still work. This is still however less than ideal: you can't easily have multiple versions of the binary (important during development), and you dictate to end-users where they must install the package.
On systems that support $ORIGIN, using DT_RUNPATH of $ORIGIN/../lib would give you an executable what works when installed anywhere and invoked from any directory, so long as relative paths to bin/ and lib/ are preserved.
From The Linux Programming Interface:
Using $ORIGIN in rpath
Suppose that we want to distribute an application that uses some of
its own shared libraries, but we don’t want to require the user to
install the libraries in one of the standard directories. Instead, we
would like to allow the user to unpack the application under an
arbitrary directory of their choice and then immediately be able to
run the application. The problem is that the application has no way of
determining where its shared libraries are located, unless it requests
the user to set LD_LIBRARY_PATH or we require the user to run some
sort of installation script that identifies the required directories.
Neither of these approaches is desirable. To get around this
problem, the dynamic linker is built to understand a special string,
$ORIGIN (or, equivalently, ${ORIGIN}), in an rpath
specification. The dynamic linker interprets this string to mean “the
directory containing the application.” This means that we can, for
example, build an application with the following command:
$ gcc -Wl,-rpath,'$ORIGIN'/lib ...
This presumes that at run time the application’s shared libraries will
reside in the subdirectory lib under the directory that contains the
application executable. We can then provide the user with a simple
installation package that contains the application and associated
libraries, and the user can install the package in any location and
then run the application (i.e., a so-called “turn-key application”).
I'm trying to build a project that supports easily building 'stripped-down' distributions with undesired features removed. The project uses automake and is structured with potentially removable features in their own feature.am files that are included in the top-level Makefile.am file as
include feature.am
The problem is, if you remove a feature (and its feature.am file), autoreconf fails with
automake: error: cannot open < feature.am: No such file or directory
Is there a way to simply ignore this error and continue? I tried using
-include feature.am
like GNU make does, but this ends up simply copying that line into the Makefile.in file (and thus the Makefile), rather than having automake read it.
This is not possible because of the way automake works (which has next to nothing to do with how make includes files by itself.)
You can use AM_CONDITIONAL and autoconf's AC_ARG_ENABLE to build or not build the components that you don't want built. If you would like to have separate tarballs, that's more complicated and I would suggest you just ship separate packages for those features.
I have recently installed Hombrew on Yosemite with Xcode installed and would like to better understand what brew doctor is warning me about. I understand that these warnings are nothing to worry about (until they are) and am not asking how or whether to respond to them; but I would like to do the best I can to understand the likely causes so that I can be prepared when issues come up (and also to head off issues that others in my office may encounter on similar systems).
Are any of the items below from obvious sources on a 10.10 machine with Xcode and non-brew Fortran installed? Do any of these items ring a bell?
Warning: Some directories in /usr/local/share/man aren't writable.
This can happen if you "sudo make install" software that isn't managed
by Homebrew. If a brew tries to add locale information to one of these
directories, then the install will fail during the link step.
You should probably `chown` them:
/usr/local/share/man/de
/usr/local/share/man/de/man1
/usr/local/share/man/mann
Warning: Broken symlinks were found. Remove them with `brew prune`:
/usr/local/lib/libasan.dylib
/usr/local/lib/libatomic.dylib
/usr/local/lib/libcilkrts.dylib
/usr/local/lib/libgcc_s_ppc64.1.dylib
/usr/local/lib/libgcc_s_x86_64.1.dylib
/usr/local/lib/libgfortran.dylib
/usr/local/lib/libgmp.dylib
/usr/local/lib/libgmpxx.dylib
/usr/local/lib/libgomp.dylib
/usr/local/lib/libitm.dylib
/usr/local/lib/libmpc.dylib
/usr/local/lib/libmpfr.dylib
/usr/local/lib/libquadmath.dylib
/usr/local/lib/libssp.dylib
/usr/local/lib/libstdc++.dylib
/usr/local/lib/libubsan.dylib
/usr/local/lib/ppc64/libgfortran.2.0.0.dylib
/usr/local/lib/ppc64/libgfortran.2.dylib
/usr/local/lib/ppc64/libgfortran.dylib
/usr/local/lib/x86_64/libgfortran.2.0.0.dylib
/usr/local/lib/x86_64/libgfortran.2.dylib
/usr/local/lib/x86_64/libgfortran.dylib
Warning: "config" scripts exist outside your system or Homebrew directories.
`./configure` scripts often look for *-config scripts to determine if
software packages are installed, and what additional flags to use when
compiling and linking.
Having additional scripts in your path can confuse software installed via
Homebrew if the config script overrides a system or Homebrew provided
script of the same name. We found the following "config" scripts:
/opt/ImageMagick/bin/Magick++-config
/opt/ImageMagick/bin/Magick-config
/opt/ImageMagick/bin/MagickCore-config
/opt/ImageMagick/bin/MagickWand-config
/opt/ImageMagick/bin/Wand-config
Warning: Unbrewed header files were found in /usr/local/include.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.
Unexpected header files:
/usr/local/include/fakemysql.h
/usr/local/include/fakepq.h
/usr/local/include/fakesql.h
/usr/local/include/gmp.h
/usr/local/include/gmpxx.h
/usr/local/include/graphviz/arith.h
/usr/local/include/graphviz/cdt.h
/usr/local/include/graphviz/cgraph.h
/usr/local/include/graphviz/color.h
/usr/local/include/graphviz/geom.h
/usr/local/include/graphviz/graphviz_version.h
/usr/local/include/graphviz/gvc.h
/usr/local/include/graphviz/gvcext.h
/usr/local/include/graphviz/gvcjob.h
/usr/local/include/graphviz/gvcommon.h
/usr/local/include/graphviz/gvconfig.h
/usr/local/include/graphviz/gvplugin.h
/usr/local/include/graphviz/gvplugin_device.h
/usr/local/include/graphviz/gvplugin_layout.h
/usr/local/include/graphviz/gvplugin_loadimage.h
/usr/local/include/graphviz/gvplugin_render.h
/usr/local/include/graphviz/gvplugin_textlayout.h
/usr/local/include/graphviz/gvpr.h
/usr/local/include/graphviz/pack.h
/usr/local/include/graphviz/pathgeom.h
/usr/local/include/graphviz/pathplan.h
/usr/local/include/graphviz/textspan.h
/usr/local/include/graphviz/types.h
/usr/local/include/graphviz/usershape.h
/usr/local/include/graphviz/xdot.h
/usr/local/include/itcl.h
/usr/local/include/itcl2TclOO.h
/usr/local/include/itclDecls.h
/usr/local/include/itclInt.h
/usr/local/include/itclIntDecls.h
/usr/local/include/itclMigrate2TclCore.h
/usr/local/include/itclTclIntStubsFcn.h
/usr/local/include/mpc.h
/usr/local/include/mpf2mpfr.h
/usr/local/include/mpfr.h
/usr/local/include/mysqlStubs.h
/usr/local/include/odbcStubs.h
/usr/local/include/pqStubs.h
/usr/local/include/tcl.h
/usr/local/include/tclDecls.h
/usr/local/include/tclOO.h
/usr/local/include/tclOODecls.h
/usr/local/include/tclPlatDecls.h
/usr/local/include/tclThread.h
/usr/local/include/tclTomMath.h
/usr/local/include/tclTomMathDecls.h
/usr/local/include/tdbc.h
/usr/local/include/tdbcDecls.h
/usr/local/include/tdbcInt.h
/usr/local/include/tk.h
/usr/local/include/tkDecls.h
/usr/local/include/tkPlatDecls.h
Warning: Unbrewed .la files were found in /usr/local/lib.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.
Unexpected .la files:
/usr/local/lib/libasan.la
/usr/local/lib/libatomic.la
/usr/local/lib/libcilkrts.la
/usr/local/lib/libgfortran.la
/usr/local/lib/libgmp.la
/usr/local/lib/libgmpxx.la
/usr/local/lib/libgomp.la
/usr/local/lib/libitm.la
/usr/local/lib/libmpc.la
/usr/local/lib/libmpfr.la
/usr/local/lib/libquadmath.la
/usr/local/lib/libssp.la
/usr/local/lib/libssp_nonshared.la
/usr/local/lib/libstdc++.la
/usr/local/lib/libsupc++.la
/usr/local/lib/libubsan.la
Warning: Unbrewed .pc files were found in /usr/local/lib/pkgconfig.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.
Unexpected .pc files:
/usr/local/lib/pkgconfig/libcdt.pc
/usr/local/lib/pkgconfig/libcgraph.pc
/usr/local/lib/pkgconfig/libgvc.pc
/usr/local/lib/pkgconfig/libgvpr.pc
/usr/local/lib/pkgconfig/libpathplan.pc
/usr/local/lib/pkgconfig/libxdot.pc
/usr/local/lib/pkgconfig/tcl.pc
/usr/local/lib/pkgconfig/tk.pc
Warning: Unbrewed static libraries were found in /usr/local/lib.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.
Unexpected static libraries:
/usr/local/lib/libatomic.a
/usr/local/lib/libcilkrts.a
/usr/local/lib/libgfortran.a
/usr/local/lib/libgmp.a
/usr/local/lib/libgmpxx.a
/usr/local/lib/libgomp.a
/usr/local/lib/libitm.a
/usr/local/lib/libmpc.a
/usr/local/lib/libmpfr.a
/usr/local/lib/libquadmath.a
/usr/local/lib/libssp.a
/usr/local/lib/libssp_nonshared.a
/usr/local/lib/libstdc++.a
/usr/local/lib/libsupc++.a
/usr/local/lib/libtclstub8.6.a
/usr/local/lib/libtkstub8.6.a
There is a variety of warnings here, probably all with a very similar cause: having installed software from source (by compiling with configure; make; make install, or sometimes unpacking a very specific (e.g. scientific) package, such as fortran). That would be mostly software with a unix background, and the software decided to install itself in /usr/local.
I don't know your background, so I'm going to expand somewhat (I'll probably miss a few things or may have a few things (somewhat) incorrect):
In unix & linux, a piece of software is often distributed across various subdirectories, and not inside a single package (which is more Mac style). It can consist of binary that goes into /usr/local/bin/, a library with routines used by the binary that is put in /usr/local/lib/, some header file for function declarations that lives in /usr/local/include/, a manual page that is put in /usr/local/man/ or additional (e.g. configuration) data that could go into /usr/local/share/. Practically any combination can exist.
That looks a bit scattered, but it works. It does mean that pieces of different software live in the same subdirectory, as you can see form the various lists of filenames.
Note that /usr/local/ is in so far unique, that it tends to be a preferred directory to install extra software: software that the system (OS) can do without. This is also why Homebrew likes to install software there. Other software managers avoid the unix default of /usr/local/ and install in /opt/local/ (Macports) or /sw/ (Fink). But the fact that Homebrew picks the default "extra" software installation part can mean it clashes with other installed software.
Your system software lives in plain /usr/, or /System/ and /Library/, and stays out of /usr/local/; if you remove /usr/local/ completely, your system will still work.
So, with that in mind, the warnings:
Warning: Some directories in /usr/local/share/man aren't writable.
This can happen if you "sudo make install" software that isn't managed
by Homebrew. If a brew tries to add locale information to one of these
directories, then the install will fail during the link step. You
should probably chown them:
/usr/local/share/man/de
/usr/local/share/man/de/man1
/usr/local/share/man/mann
Other software installed its manual pages here, likely with the sudo command (you may simply have had a dialog asking for your password; sudo then happened under the hood). In that case, those specific directories are "owned" by "root", and not by the usual Homebrew user (your login). That would mean Homebrew can't, in a future event, not write in those directories, if it ever comes across software that likes its manual pages there, since it's not owned by the Homebrew user.
You can alleviate this by changing the permissions to yourself:
$ sudo chown -R <user>:<group> /usr/local/share/man/de
$ sudo chown -R <user>:<group> /usr/local/share/man/mann
(do an $ ls -l $HOME to find your <user> and <group> to fill in: it's the hopefully obvious columns.)
Warning: Broken symlinks were found. Remove them with `brew prune`:
A sym(bolic )link is just a pointer to a file that exist somewhere else. If it's broken, the original file doesn't exist anymore, or the symlink is pointing to the wrong place. You can see what it is pointing at by doing for example:
ls -l /usr/local/lib/libasan.dylib
The .dylib files are all dynamic libraries: library files that contain functions, and are meant to be used by other programs (but, specifically, not fully included inside that program; see later on static libraries). A broken library file is a potentially bad thing: new software that wants to use those files may think "ah, I can use this library" and then things go bad during compilation, when it turns out the file doesn't exist anymore. So, it may be wise to prune (remove) the symbolic links. (A symbolic link can normally be safely removed, even if it points correctly: it's only a pointer, the original file is left untouched.)
Warning: "config" scripts exist outside your system or Homebrew directories.
`./configure` scripts often look for *-config scripts to determine if
software packages are installed, and what additional flags to use when
compiling and linking.
Having additional scripts in your path can confuse software installed via
Homebrew if the config script overrides a system or Homebrew provided
script of the same name. We found the following "config" scripts:
configure is part of the chain to build software from source on the command line. It looks around on your system for existing libraries, and also ask for the configuration of programs using these -config scripts. Homebrew had a look around, and found such scripts outside the usual directories. Thus, configure could find multiple versions of the same config script, get confused which one to use and use the wrong one.
In this case, it's all ImageMagick stuff. Depending how you use it, you could opt to remove the entire /opt/ImageMagick/ directory (and perhaps you're then left with an empty /opt/ directory, in which case you can also remove /opt/; it's not part of the usual system).
Warning: Unbrewed header files were found in /usr/local/include.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.
Warning: Unbrewed .la files were found in /usr/local/lib.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.
Warning: Unbrewed static libraries were found in /usr/local/lib.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.
These are all related, and related to the dynamic libraries mentioned before. The .a files are static libraries: they (also) contain often-used functions, but those files get included inside the binary when compiling software from source (thus, the binary becomes bigger than with dynamic libraries). The .h files are header files, which tell programs what the library files contain; the .la have some extra information on the .a library files.
The warning tells you that Homebrew may get confused when installing new software: perhaps it will try to use these libraries, but they happen to be the (oh so slightly) wrong version and things don't work. You don't know until you bump into it, unfortunately.
Warning: Unbrewed .pc files were found in /usr/local/lib/pkgconfig.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.
pkgconfig files are similar to those config scripts mentioned earlier: they contain configuration information if you need some earlier installed software to build new software. Again, as before: versions may mismatch slightly and then things could not work as intended (it doesn't install, or crashes, or makes your computer pass the Turing test).
As for how you may have gotten all those files in /usr/local.? Probably you installed a package that puts everything in /usr/local/ as well. As mentioned, it is a default place to put things. Since it features libgfortran and libgomp, I'm inclined to think about some scientific package.
All in all, most warnings are about potential incompatibilities: Homebrew, like most package/software managers, tries to hold some grip (not too tight though) on what it installs and what is around, because backwards incompatibilities and such can mean that newly installed software from source ("brewed" software) doesn't properly install or run. As an example, Macports is more strict (at least it was years ago when I used it), and will download matching, known compatible, versions of required extra software (thus, you could end up with four different versions of a C compiler).
Homebrew tries to be lenient with respect to installed libraries and such, but it does warn you that bad things can happen, either during compilation or later on.