I use gcc more than any other compiler, so I will shape my example with this compiler suite, but i have experienced this problem with almost all the suite that i have tried like gcc, mingw, clang and msvc.
gcc offers this flags:
-l you write the name foo, gcc will find a corrisponding library named libfoo
-L you append the path where the libs lives and gcc tries to match the required libraries to the ones that it finds in that path
-rpath basically a pool of different path for the same lib so the executable is "smart" enough to look for alternatives if he needs one.
the problem is big for me, no one of this solves my problem and each one of this flags suffer the same problem: ambiguity.
if I just want to link a library that i know there is no way to do this without including a dose of ambiguity in the best case scenario, what i want is:
linking 1 specific library only, and only the 1 that I specify with a precise name and path
avoid auto-completion mechanism like the one on the name given to -l because my libs are named foo.so not libfoo.so
relative path for the linked libs
only consider the explicitly given set of libraries, no other automation of any kind should be involved, no pool of libs, no search-paths, no nothing else, I prefer list of errors instead of an executable linked to a random library
I often deal with different libs in different releases, they often share the same name for historical and compatibility reasons, it's a nightmare compiling and linking with gcc because I never got the one that I want linked to my executable.
Thanks.
The easiest way to do this is to simply specify the library.
gcc -o test test.o /path/my_library.so /path/to_other_library.a
The obvious downside to this approach if of course that if you move that library then your application won't work anymore but since you state that you have the libraries at fixed locations it should work in your case.
Related
I have a project, where I want to link one of system libraries statically. The project uses GNU build system.
In configure.ac I have:
AC_CHECK_LIB(foobar, foobar_init)
On development machine this library is installed in /usr/lib/x86_64-linux-gnu. It is detected, but it is linked dynamically, which causes issues, as it is not present on some machines. Linking it statically (-Wl,-Bstatic etc.) works fine, but I don't know how to set this up in autotools. I tried forcing this into Makefile.am link flags for the project, but it still gives a preference to dynamic library.
I also tried using --enable-static with ./configure, but it seems to have no effect on system libraries.
If you want to link the whole program statically, then you should pass the --disable-shared option to configure. You might or might not need also to pass --enable-static, depending on the default value for that option (which you can influence via your configure.ac file). You really should consider doing this.
You should also consider making this the installer's problem, not the build system's. Let it be the installer's responsibility to ensure that all the shared libraries needed by the program are provided by the systems where it is installed. This is very common; in fact, it is one of the inspirations for package-management systems such as yum / dnf and apt, and their underlying packaging formats.
If you insist on linking only one library statically, while linking everything else dynamically, then you'll need to jump through a few more hoops. The objective will be to emit link options that cause just that library to be linked statically, without changing other libraries' linking. With the GNU toolchain, and supposing that the program is otherwise being linked dynamically, that would be this combination of options:
-Wl,-Bstatic -lfoobar -Wl,-Bdynamic
Now consider the documentation of the AC_CHECK_LIB() macro:
Macro: AC_CHECK_LIB (library, function, [action-if-found],
[action-if-not-found], [other-libraries])
[...] action-if-found is a list of shell commands to run if the link with the library succeeds; action-if-not-found is a list of shell
commands to run if the link fails. If action-if-found is not
specified, the default action prepends -llibrary to LIBS and defines
'HAVE_LIBlibrary' (in all capitals). [...]
Note in particular the default behavior in the event that the optional arguments are not provided (your present case) -- that's not quite what you want, at least not by itself. I suggest providing at least an alternative behavior for action-if-found case, and you could consider also making configure fail in the action-if-not-found case. The latter is left as an exercise; implementing just the former might look like this:
AC_CHECK_LIB([foobar], [foobar_init], [
LIBS="-Wl,-Bstatic -lfoobar -Wl,-Bdynamic $LIBS"
AC_DEFINE([HAVE_LIBFOOBAR], [1], [Define to 1 if you have libfoobar.])
])
You should also pay attention to the order of your AC_CHECK_LIB() invocations. As its docs go on to say:
This macro is intended to support building LIBS in a right-to-left
(least-dependent to most-dependent) fashion such that library
dependencies are satisfied as a natural side effect of consecutive
tests. Linkers are sensitive to library ordering so the order in which
LIBS is generated is important to reliable detection of libraries.
If you find that you still aren't getting what you want, then have a look at the link commands that make actually executes. You need to understand what's wrong about them before you can determine how to fix the problem.
With all that said, I observe that the above treatment is basically a hack, and it makes your build system much less resilient. It introduces dependencies on GNU toolchain options (which some other toolchains may nevertheless accept), and it assumes dynamic linking is being performed overall. It may be possible to resolve those issues with additional Autoconf code, but I urge you to instead go with one of the first two alternatives I described.
Background:
I am trying to provide a library wrapped within an interface, to be used in a third party application (MATLAB).
My problem is that both my library and the third party application rely on Boost libraries.
To make things even worse, not only they rely on two different versions of Boost, but my library is built against a patched version of Boost, and they are statically linked. Third party libraries are instead dynamically linked against Boost.
This is how I link (simplified output of make -n)
g++ -fPIC ${CPPFLAGS} -shared -lc -ldl ... -Wl,-h,mylib.so -o mylib.so \
${MYSTATICLIBS} \
-L${MATLABLIBSPATH} -lmx -lmex \
-L${MYBOOSTVERSIONPATH} -lboost_thread-gcc49-mt-1_52 ...
When I run the library fron within MATLAB then, the wrong boost functions are called from my internal functions (and everything crashes).
My naive intuition was that since i statically linked my Boost, all calls to them from within my library should point to my Boost, while calls from within libmx.so or libmex.so would point to their dynamically loaded dependencies. However this proves my naive intuition was very wrong.
I found different hints here on SO and around the web describing how linking order may break similar situations, however everything describes either all static or all dynamic linking of conflicting symbols.
Suggested solutions are
using -Bsymbolic -Bsymbolic-functions
upgrade my library to the same boost version
changing linking order ( first boost, then -lmx -lmex )
sell all my earthly possessions, buy a bar on the beach somewhere hot and try making a living out of it
other??
Can someone explain if and why these solution can work?
( my other problem is, I am not able to test with MATLAB myself, as I have no access to the machines on which it is installed. this is why I'm also asking "if" the proposed solutions would work)
First of all, welcome to the wonderful world of Linux symbol interposition ;)
I think what happens is your library exports all symbols from Boost library it has been linked with (can be verified with readelf --dyn-syms -W). At runtime, dynamic linker will note that and instead rebind symbols to a different version of Boost (because it happens to be loaded earlier and so overrides symbols used by your library).
What you want to do is to tell gcc that you do not want to export Boost symbols or you do not want them to be interposable at runtime. There are several solutions for this, the best being to compile and link with -fvisibility=hidden (note that Boost will also need to be linked with this flag). This will prevent any functions from being exported from your library unless you explicitly mark them with __attribute__((visibility("hidden"))) (you'll need to annotate library interface functions of course).
Another option is -Bsymbolic - this would not prevent spurious Boost exports from your library but would at least prevent dynamic linker from overriding them with random versions of Boost functions which happen to be loaded by other libraries.
Now to remaining questions.
upgrade my library to the same boost version
You could do that but this may break if 3-rd party upgrades their Boost at some point.
changing linking order ( first boost, then -lmx -lmex )
Does not make much sense... Have you tried?
I've used zlib for ages and never thought about the fact that it is named slightly unconventionally. While most libraries on Linux follow the naming convention of lib<name>.so for shared objects and lib<name>.a for archives, zlib is named zlib.so/zlib.a. My question is: how does gcc/ld know to look for zlib.so when I use -lz as a link flag?
I understand that for linking, gcc invokes ld, which searches for libraries in certain default paths and any path specified with -L, and it appends the lib and .so or .a. parts as necessary. Oddly, gcc's manual page for linking options only mentions that the linker can find archives; there is no mention of the .so extension. The man page for ld at least mentions both extensions, but still only mentions searching by prepending lib to the specified library name. How does ld know to add the lib after the z for zlib? I've never seen this happen to another library.
gcc has several different methods for linking libraries, shared or static. If you specify -lz, gcc is going to look for libz.so (possibly with some version bits between the libz and the .so, but the important part is the file name will start with libz and end with .so), or for libz.a (again, possibly with version info) if you are compiling statically, or as a fallback if the shared library does not exist. If you specify -lzlib it will look for libzlib.so (which is not the standard name - the package is often named zlib, but the library itself is libz). Another way of linking would be to not use the -l<lib> option, and just specify /path/to/zlib.so or -L /path/to zlib.so (or zlib.a if you want). In this case, the library doesn't have to have the lib prefix, but you would have to explicitly provide any version info, unless provisions are made for a symbolic link or something similar to provide the literal name zlib.so.
Applications can also load shared libraries at runtime via dlopen() and it's other associated functions, in which case the library can also be named whatever you want it to be (this doesn't work for static libraries, of course).
So, if the library you are looking at is actually called zlib.so, then it is not being found by gcc ... -lz, unless it just happens to be a symbolic link to libz.so (or vice versa, in which case gcc is really just using libz.so, which happens to have the same content as your zlib.so). However gcc might be using it if the build process explicitly names the library in the link stage (not using -l<lib>) or if your application loads it via dlopen() (but in that case, it's not really linked to your program - it's just loaded at run time).
I have my soft.c and I compile it successfully and link it against /usr/lib/myLib.so.
Everything it's ok except for the fact that it's not what I'm looking for, i would like to manage the final path considered for the linking, in other words i want GCC to consider the library /usr/lib/myLib.so but I also want the final linking to be a relative path, kinda like ../myLib.so.
The usual scenario is
./soft linked to -> /usr/lib/myLib.so
I want
./soft linked to -> ../myLib.so
Obiviously I also need GCC to manage any cross referenced library, for example if /usr/lib/myLib.so is linked to other libs i want all the libraries involved to be relocated in the relative path so ../myLib.so can fully work from a relative path .
How to achieve that ?
Thanks.
You could include run-time search path information in the binaries, using the -rpath option to the linker (passed to gcc as -Wl,-rpath,.
Note that this is generally frowned upon by distribution maintainers, as there should be a single system-wide library search path so conflicts are caught early.
I am compiling a program in which a header file is defined in multiple places. Contents of each of the header file is different, though the variable names are the same internal members within the structures are different .
Now at the linking time it is picking up from a library file which belongs to a different header not the one which is used during compilation. Due to this I get an error at link time.
Since there are so many libraries with the same name I don't know which library is being picked up. I have lot of oems and other customized libraries which are part of this build.
I checked out the options in gcc which talks about selecting different library files to be included. But no where I am able to see an option which talks about which libraries are being picked up the linker.
If the linker is able to find more than one library file name, then which does the linker pick up is something which I am not able to understand. I don't want to specify any path, rather I want to understand how the linker is resolving the multiple libraries that it is able to locate. I tried putting -v option, but that doesn't list out the path from which the gcc picks up the library.
I am using gcc on linux.
Any help in this regard is highly appreciated.
Regards,
Chitra
Passing -Wl,-t to gcc will tell ld to dump which files it's reading.