Choose right library version - gcc

I have a project which uses libibverbs. I have to compile this project on a server to run it there. The server has libibverbs installed system-wide, but unfortunately it doesn't support a feature which I need.
I decided to compile and use my own version of libibverbs, which has the corresponding feature. So I compiled the library, installed it to my home directory, and updated following environment variables: PATH, LD_LIBRARY_PATH, C_INCLUDE_PATH, LIBRARY_PATH, CPLUS_INCLUDE_PATH
Now I have to compile my project. First I call configure and it fails with following error message:
conftest.c:(.text.startup+0x7): undefined reference to `ibv_open_xrc_domain'
This is the symbol, which is missing in the system-wide version, but is present in the version I installed. First entry in LIBRARY_PATH is the path to the new libibversion, so I expected it is going to be used first, but it seems that the old version is used anyway.
Compilation command which ./configure uses, contains flag -L/libibverbs/1.1.4/lib, which points to the directory with the new library version. This flag goes just after -L/usr/lib/../lib64 which point to the directory, with system-wide libibverbs
If I put -L with new version in the end of command, the conftest compiles successfully.
To be clear following fails: https://gist.github.com/planetA/a421669269b14e69026c53f56fa45b2b
And following works:
https://gist.github.com/planetA/3b0e22bf6aca3a1c67f30bfa3666d9a8
Could you help me to enforce picking the new version of the library in a way that configure catches it?

LD_LIBRARY_PATH specifies directories to be searched, before the
defaults, for a library that is to be loaded in a process at runtime.
It does not affect the directories that are searched for a library
in order to link it at buildtime.
The linker searches for libraries in the directories that are specified
with the -Ldir option in its commandline, before it searches its
default directories. Your configure script tests whether it can find a libibverbs library in the linker's search directories that defines the function
ibv_open_xrc_domain, and fails because it cannot. The value of LD_LIBRARY_PATH does
not matter to this test.
For GNU make, the -L-options it should pass to the linker
are conventionally specified in the environment variable LDFLAGS. GNU autoconf - which
generates your configure script - follows this convention. autoconf generates
the configure script from the project's configure.ac file.
So, if your want your modified package to generate a generate a configure
script such that running ./configure will in turn generate makefiles
in which -L/my/library/version/is/here is passed to the linker then
you need to modify the project's configure.ac like:
LDFLAGS="$LDFLAGS -L/my/library/version/is/here"
and you need to do this in the configure.ac before it runs the test for
the libibverbs library. After making this change you will need to reconfigure
the package by running autoreconf in the project directrory, to regenerate
the configure script.
If you don't want to change the configure.ac like this then you can achieve the same
effect by:
./configure LDFLAGS="$LDFLAGS -L/my/library/version/is/here"
or:
export LDFLAGS="$LDFLAGS -L/my/library/version/is/here"
... # in same shell or a subshell
./configure
Then until the next time you run the configure script the project's makefiles
will pass -L/my/library/version/is/here to the linker. But the next time
you run ./configure you must remember to set LDFLAGS in the same way, or
the regenerated makefiles will revert to the default behaviour.

Related

Autotools: GCC's Makefile.in has a bug (7.3.0 and earlier). How to re-program automake to find 'ar' and 'objdump'

GCC's cross compiling autotools is supposed to be flexible, but I've isolated a bug that's been breaking cross compiler builds that ought to work.
Note: Some systems will "poison" default compiler tool names to prevent using wrong tools by default. On my system, x86_64-pc-gnu-linux-ar will execute but "ar" is not found.
I need to build cross compiler toolchains with custom names. gcc's configure script supports this with --program-prefix or --program-transform-name. However, when using a custom name, all compile time tools have to be explicitly named on the configure line. gcc configure is not intelligent enough to find tools it has just built with a name change. (too stupid).
The GCC manual states how to explicitly name tools:
configure AR=x86_foo_b_ar AR_FOR_TARGET=ARMv6m_foo_b_ar ...
However, it doesn't work right. Autoools sometimes ignores the supplied names and the build fails. In particular, it ignores 'AR' and 'OBJDUMP' variables.
Apparently the toplevel gcc configure was created at a later date than lower level configures.
Makefile.in without Makefile.am in GCC?
Makefile.am does not exist in some subdirectories, but it does exist in newer subdirectories.
This causes inconsistencies in variable passing from the top-level makefile.
Internally, the top level "configure" script has variables AR_FOR_HOST (alias for AR), AR_FOR_BUILD, and AR_FOR_TARGET. These variables are used to re-define "AR" when entering sub-directories to force a generic make script to compile for a particular target.
I've even gone so far as to define the internal variables correctly as well as "AR" and "OBJDUMP" on the configure command line. ( Shouldn't be needed ).
gcc-7.3.0/configure --host=x86_64-pc-linux-gnu --program-prefix=armv6m-softfloat-eabi-newlib- AR_FOR_BUILD=/usr/bin/x86_64-pc-linux-gnu-ar AR=/usr/bin/x86_64-pc-linux-gnu-ar AR_FOR_HOST=/usr/bin/x86_64-pc-linux-gnu-ar AR_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/ar AS_FOR_BUILD=/usr/bin/x86_64-pc-linux-gnu-as AS=/usr/bin/x86_64-pc-linux-gnu-as AS_FOR_HOST=/usr/bin/x86_64-pc-linux-gnu-as AS_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/as DLLTOOL_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/dlltool LD_FOR_BUILD=/usr/bin/x86_64-pc-linux-gnu-ld LD=/usr/bin/x86_64-pc-linux-gnu-ld LD_FOR_HOST=/usr/bin/x86_64-pc-linux-gnu-ld LD_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/ld LIPO_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/lipo NM_FOR_BUILD=/usr/bin/x86_64-pc-linux-gnu-nm NM=/usr/bin/x86_64-pc-linux-gnu-nm NM_FOR_HOST=/usr/bin/x86_64-pc-linux-gnu-nm NM_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/nm OBJCOPY_FOR_BUILD=/usr/bin/x86_64-pc-linux-gnu-objcopy OBJCOPY=/usr/bin/x86_64-pc-linux-gnu-objcopy OBJCOPY_FOR_HOST=/usr/bin/x86_64-pc-linux-gnu-objcopy OBJCOPY_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/objcopy OBJDUMP_FOR_BUILD=/usr/bin/x86_64-pc-linux-gnu-objdump OBJDUMP=/usr/bin/x86_64-pc-linux-gnu-objdump OBJDUMP_FOR_HOST=/usr/bin/x86_64-pc-linux-gnu-objdump OBJDUMP_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/objdump RANLIB_FOR_BUILD=/usr/bin/x86_64-pc-linux-gnu-ranlib RANLIB=/usr/bin/x86_64-pc-linux-gnu-ranlib RANLIB_FOR_HOST=/usr/bin/x86_64-pc-linux-gnu-ranlib RANLIB_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/ranlib READELF_FOR_BUILD=/usr/bin/x86_64-pc-linux-gnu-readelf READELF=/usr/bin/x86_64-pc-linux-gnu-readelf READELF_FOR_HOST=/usr/bin/x86_64-pc-linux-gnu-readelf READELF_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/readelf STRIP_FOR_BUILD=/usr/bin/x86_64-pc-linux-gnu-strip STRIP=/usr/bin/x86_64-pc-linux-gnu-strip STRIP_FOR_HOST=/usr/bin/x86_64-pc-linux-gnu-strip STRIP_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/strip CC_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/cc CXX_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/cxx WINDRES_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/windres WINDMC_FOR_TARGET=/usr/libexec/gcc/armv6m-softfloat-eabi-newlib/windmc --target=armv6m-softfloat-eabi --build=x86_64-pc-linux-gnu --prefix=/usr --bindir=/usr/x86_64-pc-linux-gnu/armv6m-softfloat-eabi-newlib/gcc-bin/7.3.0 --includedir=/usr/lib/gcc/armv6m-softfloat-eabi-newlib/7.3.0/include --datadir=/usr/share/gcc-data/armv6m-softfloat-eabi-newlib/7.3.0 --mandir=/usr/share/gcc-data/armv6m-softfloat-eabi-newlib/7.3.0/man --infodir=/usr/share/gcc-data/armv6m-softfloat-eabi-newlib/7.3.0/info --with-gxx-include-dir=/usr/lib/gcc/armv6m-softfloat-eabi-newlib/7.3.0/include/g++-v7 --with-python-dir=/share/gcc-data/armv6m-softfloat-eabi-newlib/7.3.0/python --enable-languages=c --enable-obsolete --enable-secureplt --disable-werror --with-system-zlib --enable-nls --without-included-gettext --enable-checking=release --with-bugurl=https://bugs.gentoo.org/ --with-pkgversion=Gentoo 7.3.0-r3 p1.4 --disable-esp --enable-poison-system-directories --disable-bootstrap --with-newlib --enable-multilib --disable-altivec --disable-fixed-point --with-float=soft --disable-libgcj --disable-libgomp --disable-libmudflap --disable-libssp --disable-libcilkrts --disable-libmpx --disable-vtable-verify --disable-libvtv --disable-libquadmath --enable-lto --without-isl --disable-libsanitizer --enable-default-pie --enable-default-ssp
I'm wanting gcc to both make and use tools that start with the prefix: armv6m-softfloat-eabi-newlib-
(Arm cortex m0 chipset is what I am using)
But "make" still fails when attempting to execute "ar" in the .../libcpp directory. The reason is that .../libcpp/Makefile.in is not updated by automake. It's a hand crafted file. On line 28 of the old .../libcpp/Makefile.in it says "AR = ar"
So, the AR variable is hardcoded to "ar" But, "ar" doesn't exist on my system. I've tried editing .../libcpp/Makefile.in with "AR = dummyname" , and the build crashes with "can't fine dummyname" instead of can't find "ar". So, the bug is on line 28.
All other variables in the .../libcpp/Makefile.in are of the form:
CC = #CC#
INSTALL = #INSTALL#
etc..
On a positive note: The compiler used by .../libcpp IS the fully qulaified name I gave to gcc-7.3.0/configure. That success made me think I could fix the bug by editing the makefile to read:
AR = #AR#
But the build fails with "Can't find AR#"
I'm not familiar enough with autotools to hand edit the Makefile.in and fix the bug.
What's the #variable# name format do?
Does the configure.ac in the subdirectory have to define "AR" in some way for #AR# to be linked to the value in the toplevel directory?
I've tried a few other tests while building different gcc versions. Re-running autoconfig, automake, is hell because GCC uses AC_PREREQ() macro.
For example, I have autotools 2.69 installed ... but gcc 7.3.0 fails and complains that I must use autotools 2.64, ONLY. eg: AC_PREREQ(2.64)
So, fixing the bug via autotools doesn't seem practical.
I'm hoping to simply patch the .../libcpp/Makefile.in, since that file is exactly the same in so many versions of gcc.
Questions:
Why is "ar" hard-coded ? Is this a serious legacy issue? and what is a minimal patch that won't interfere with other configurations of GCC?
Is it better to modify the shell or the Makefile; eg: like the top level configure shell script could define a bash function that would be inherited by make as "if" it were a program.
if [ -z ${AR##*-*} ] ; then
ar() { $AR }
fi
Edit: A quick-fix patch for gcc-7.3.0
This is not a "correct" fix, but just a work-around.
I've found three places where the sub-directories ignore variables passed in from the toplevel configure.
.../libcpp/Makefile.in on line 29
.../gcc/configure just before line 29531
.../libcc1/configure just before 14574
The second and third errors are from a defective macro in configure.ac. I haven't traced it back because I can't run autoconfig anyway.
I added a line to the configure(s), to see if passing the default OBJDUMP override variable would allow gcc to compile. It does. I'm not sure I've chosen the right override variable for all cases of gcc compile switches, but at least it proves where the bug is.
Patch file follows:
--- gcc-old/libcpp/Makefile.in
+++ gcc-new/libcpp/Makefile.in
## -28,3 +28,3 ##
INSTALL = #INSTALL#
-AR = ar
+AR ?= ar
ARFLAGS = cru
--- gcc-old/gcc/configure
+++ gcc-new/gcc/configure
## -29531,4 +29531,6 ##
;;
esac
+ if [ -n $OBJDUMP ]; then export_sym_check="$OBJDUMP -T"; fi
+
if test x"$enable_plugin" = x"yes"; then
--- gcc-old/libcc1/configure
+++ gcc-new/libcc1/configure
## -14574,4 +14574,6 ##
;;
esac
+ if [ -n $OBJDUMP ]; then export_sym_check="$OBJDUMP -T"; fi
+
if test x"$enable_plugin" = x"yes"; then
TL;DR: there are a lot of things you could try, but the very first would be to specify AR on the command line when you run make:
make AR=x86_foo_b_ar
That shouldn't be necessary when you've already specified the same to configure, but if it doesn't work then that suggests a problem one or more levels up from the Makefile.in you're looking at. Variable definitions specified on the make command line override definitions in makefiles.
"make" still fails when attempting to execute "ar" from the .../libcpp directory. The reason is that .../libcpp/Makefile.in is not updated by automake. It's a hand crafted file.
To be clear, since understanding the system you are trying to use is immensely helpful in troubleshooting it, automake does not run at configuration or build time. It is used by the package maintainer to build one or more Makefile.in files to be included in source distributions, such as the one you obtained. Of course, this is not the only way to create Makefile.in files, and the configure script does not care how you create them (or other input files).
I'm not familiar enough with autotools to hand edit the Makefile.in and fix the bug. What's the #variable# name format do?
Does the configure.ac in the subdirectory have to define "AR" in some way for #AR# to be linked to the value in the toplevel directory?
The #variable# construction is used for values that are expected to be substituted by the configure script when it builds a corresponding output file. For that to take place, there needs to be at least a corresponding AC_SUBST([variable]) or its equivalent in the configure.ac (sometimes named configure.in, instead). Normally, that's preceded somewhere in configure.ac by code assigning an appropriate value to shell variable variable.
If you modify configure.ac then you need to rebuild the configure script, and in that case it's probably safest to rebuild the whole build system, as a package maintainer would do. There may be a script provided for that purpose in the package (autogen.sh is a common name for such scripts), but the default mechanism is to run the Autotools program autoreconf in the top-level directory of the project source tree.
I've tried a few other tests while building different gcc versions.
Re-running autoconfig, automake, is hell because GCC uses AC_PREREQ()
macro.
For example, I have autotools 2.69 installed ... but gcc 7.3.0 fails
and complains that I must use autotools 2.64, ONLY. eg:
AC_PREREQ(2.64)
That description is not consistent with the documentation of AC_PREREQ, nor with my experience with that macro. AC_PREREQ tests for the specified Autoconf version or newer. It does not demand an exact Autoconf version. There may be something else in the build system that does so, but it's not AC_PREREQ.
In any case, one alternative would be to obtain and install Autoconf 2.64. You may even be able to install it alongside your existing version. Some systems even provide pre-built packages for exactly that purpose.
So, fixing the bug via autotools doesn't seem practical. I'm hoping to
simply patch the .../libcpp/Makefile.in, since that file is exactly
the same in so many versions of gcc.
Patching a Makefile.in does not require afterward re-running the autotools, so it's at least conceivable that that would work. Even for Makefile.in files that were generated by Automake. You could consider having a look at how AR is defined in some of the Automake-generated Makefile.in files in the project (supposing there are any) for an idea of how it should look.
Why is "ar" hard-coded ? Is this a serious legacy issue?
I can only speculate. As a threshold matter, I'm inclined to suppose that in that Makefile, the archiver of the build system is the one wanted (not that of the intended host system, nor a cross-ar for host-target). It is reasonable in that case for AR = ar to be provided as a default, because that can be overridden via a declaration of that variable on the command-line.
That you are in fact not getting the AR you specify to configure looks like a bug to me -- probably a regression introduced at some point when some of the higher-level bits of the build system were updated. I have no trouble imagining such an issue slipping by, as a system configuration such as yours, in which the system's own archiver goes only by a non-standard name, is very uncommon.
and what is a
minimal patch that won't interfere with other configurations of GCC?
The first thing to try is to pass the AR definition on the top-level make command line:
make AR=x86_foo_b_ar
Such definitions will be passed on to recursively-invoked sub-makes, and definitions on the command line (but not, by default, from the environment) override definitions in Makefiles.
Is it better to modify the shell or the Makefile; eg: like the top
level configure shell script could define a bash function that would
be inherited by make as "if" it were a program.
The top-level configure script could be modified to define a shell function and export it to child processes, but not to its parent or siblings. This is nothing specific to configure; the shell just doesn't work that way. Whatever changes you make, if any, would be best made in Makefile.in files before running configure, or in the generated Makefiles afterward.

Generate list files with CMake

I hope this is a simple question and I'm just missing something fundamental.
I'm trying to emulate a binary build manager for an embedded Cortex-M0 target using a CMake project. I'm having some trouble figuring out how to generate list files for each dependency of my executable target.
The current build system, when building a file called main.c passes -Wa,-alh=.\CortexM0\ARM_GCC_493\Debug/main.lst as an argument to gcc. I can't figure out how to get CMake to use the current filename without the extension to save the file.
I've looked at the get_filename_component command, but it appears only to get the filename of the output:
add_executable(TestExe main.c)
get_filename_component(curr_name TestExe NAME_WM)
message(${curr_name})
As expected, this prints TestExe instead of the hoped for main
Is there a simple variable I'm overlooking that I could put in my toolchain file's CMAKE_C_FLAGS like -Wa,-alh=${CURR_SOURCE}.lst? Or some other method that I'm not seeing?
System info:
Windows 10
Msys shell
CMake 3.7.2
arm-none-eabi-gcc v4.9.3
You can use Expansion Rules and extend CMAKE_C_COMPILE_OBJECT:
set(CMAKE_C_COMPILE_OBJECT "${CMAKE_C_COMPILE_OBJECT} -Wa,-alh=<OBJECT>.lst")
But there is unfortunately
no Expansion Rule that does give the current source file without path and extension
so you will get in the above example main.c.o.lst as an output name
Footnote: In CMake generated makefile projects, if you just need the assembly file can just do make main.s or for the pre-processed file make main.i.

how to make use of old system libraries

We have a project and shared libraries libprivate.so (private so) which was using old libraries libcurl.so.3. The system was upgraded with new system libraries libcurl.so.4.
For some internal issues, right now we do not want to make use of latest libraries libcurl.so.4, we want to make use of libcurl.so.3.
Hence I copied libcurl.so.3 in local folder and set LD_LIBRARY_PATH according. When I link my entire project it says that there is version conflict between libcurl.so.4 and libcurl.so.3 required libprivate.so (libprivate.so is compiled long time ago with libcurl.3.so).
Should I not worry about this warning and proceed further?
When I correctly specify LD_LIBRARY_PATH which has libcurl.so.3, why it is taking from system directory /usr/lib64/libcurl.so.4? when I do ldd my_binary, it takes from libcurl.so.4. How do I stop it? Specifying -L with specific location also doesn't work. Modiying /etc/ld.conf will do for the entire system. I want to make this when I ran my project.
Specifying explicit path it works like /home/mydir/libcurl.so.3, but I do not want to do it.
I want to have these conditions only when I execute my project. In other cases it can make use of latest libraries.
Thanks for your help
If the command you show in your comment is correct:
gcc test.c -L~/lib/x86_64/ -lcurl -o test
... then you need a space between -L and ~/lib/x86_64/ or the shell won't expand the ~, so the linker is not looking in the right directory.
So you need either:
gcc test.c -L ~/lib/x86_64/ -lcurl -o test
or:
gcc test.c -L$HOME/lib/x86_64/ -lcurl -o test
(You don't need a space here because variables are expanded anywhere in a word, but ~ is only expanded at the start of a word.)

How to specify non-default shared-library path in GCC Linux? Getting "error while loading shared libraries" when running

There is a laptop on which I have no root privilege.
onto the machine I have a library installed using configure --prefix=$HOME/.usr .
after that, I got these files in ~/.usr/lib :
libXX.so.16.0.0
libXX.so.16
libXX.so
libXX.la
libXX.a
when I compile a program that invokes one of function provided by the library with this command :
gcc XXX.c -o xxx.out -L$HOME/.usr/lib -lXX
xxx.out was generated without warning, but when I run it error like this was thrown:
./xxx.out: error while loading shared libraries: libXX.so.16: cannot open shared object file: No such file or directory , though libXX.so.16 resides there.
my clue-less assumption is that ~/.usr/lib wasn't searched when xxx.out is invoked.
but what can I do to specify path of .so , in order that xxx.out can look there for .so file?
An addition is when I feed -static to gcc, another error happens like this:
undefined reference to `function_proviced_by_the_very_librar'
It seems .so does not matter even though -L and -l are given to gcc.
what should I do to build a usable exe with that library?
For other people who has the same question as I did
I found a useful article at tldp about this.
It introduces static/shared/dynamic loaded library, as well as some example code to use them.
There are two ways to achieve that:
Use -rpath linker option:
gcc XXX.c -o xxx.out -L$HOME/.usr/lib -lXX -Wl,-rpath=/home/user/.usr/lib
Use LD_LIBRARY_PATH environment variable - put this line in your ~/.bashrc file:
export LD_LIBRARY_PATH=/home/user/.usr/lib
This will work even for a pre-generated binaries, so you can for example download some packages from the debian.org, unpack the binaries and shared libraries into your home directory, and launch them without recompiling.
For a quick test, you can also do (in bash at least):
LD_LIBRARY_PATH=/home/user/.usr/lib ./xxx.out
which has the advantage of not changing your library path for everything else.
Should it be LIBRARY_PATH instead of LD_LIBRARY_PATH.
gcc checks for LIBRARY_PATH which can be seen with -v option

using agg (antigrain) library and getting gsl shell working

My main aim is to get the GSL Shell working on my OSX 10.7 system. So far I have the correct version of lua with the correct patches running. I also have a working version of GSL which compiles and runs example programs. I can build agg perfectly and also run their example programs by running make in the macosx_sdl folder.
My first question is how on earth do I create my own project with agg? I know that you are supposed to simply add the files to your project file and go, but this does not seem to want to compile for me. Is it simply a case of adding the include directory and the libagg.a?
Finally, how do I build gsl shell? Currently it complains about the agg-plot folder a lot, so where do I put the agg files to make this build, then when i've done it where do I place the agg files so that the lua scripts can get to them?!
Hope someone can help!
In general to use the AGG library you need to make sure that the compiler is able to find the headers files and, during the linking, the libraries, either in form of a static or dynamic libraries.
To make the headers files and the libraries available you need to take into account the system that is used to build the software. If a traditional makefile sistem is used you need to add some flags to make sure that the headers file can be found. This can be achieved by adding into the makefile something like:
CFLAGS += -I/path/to/agg/headers
and for the linker:
LIBS += -L/path/to/agg/library -lagg -lm
In the specific case of GSL Shell 1.1 the file "make-packages" is used in the Makefile to configure the required packages. You can add here the flags required to locate the AGG library:
AGG_INCLUDES = -I/usr/include/agg2
AGG_LIBS = -lagg -lX11 -lpthread -lsupc++
you should just modify the path provided with the "-I" option and, in AGG_LIBS, add an option "-L/path/to/agg/library" to specify the path where the AGG libraries are located.
Please note also that the agg libraries depends on other libraries. For example on linux it needs at least the X11 library. The libraries supc++ may be needed if the linking is made by invoking gcc instead of g++ because if gcc is used the C++ runtime libraries are not included.

Resources