GCC linking to a shared object's linker name - gcc

Suppose I have:
/usr/lib/libsomething.so.1 on machine A;
/usr/lib/libsomething.so.2 on machine B.
Both machines have /usr/lib/libsomething.so symlinking to their respective libs.
If I link using gcc with -lsomething (or even /usr/lib/libsomething.so) it will follow the symlink, and ldd on machine A produces something like:
libsomething.so.1 => /usr/lib/libsomething.so.1
This means it won't be able to find the library on machine B.
Now I know these are major version number changes and I know they may not be compatible, but I'm willing to take that risk. What I'd like to tell the linker is to look for libsomething.so, and don't follow the symlink so ldd will show
libsomething.so => /usr/lib/libsomething.so.1
on A but
libsomething.so => /usr/lib/libsomething.so.2
on B. And then the loader will follow the symlink to whatever version is there.
Also, I don't want delayed loading with dlopen or anything. I want it to link to the shared object at compile time.
Is this even possible?

Making executable which uses any available version of shared library is, of course, possible.
The problem was that you linked your executable to version-specific soname (libsomething.so.1 and libsomething.so.2). You should have done it with unversioned soname libsomething.so instead.
In order to achieve this, on the build machine you should compile and install library with soname (ELF SONAME) equal to libsomething.so (without version) so that linker can choose this soname while executable is built.
According to the Shared Libraries HOWTO, you can pass required unversioned soname while building the library:
gcc -shared -Wl,-soname,libsomething.so -o libsomething.so.X objectsomething.o
Then, as soon as you install the library and run ldconfig, you have:
symlink /lib/libsomething.so pointing to /lib/libsomething.so.1 on machine A;
symlink /lib/libsomething.so pointing to /lib/libsomething.so.2 on machine B.
The loader (run ldd) will choose unversioned symlinks regardless where it points to:
libsomething.so => /lib/libsomething.so (0xNNNNNNNN) on machine A;
libsomething.so => /lib/libsomething.so (0xNNNNNNNN) on machine B.
Linux dynamic loader (ld.so) resolves libraries based on their soname value written in the executable (ELF NEEDED). The value is copied from library file (ELF SONAME) while building the executable. As long as there is a symlink on the target system matching the soname recorded in the executable, the library pointed by this symlink will be loaded.
Let's run through your setup and show commands to verifing assumptions.
I used Fedora 18 X86_64 for the test and adjusted output to i686 for clarity.
Compile both libsomething.so.1 and libsomething.so.2. Make sure SONAME is set to unversioned libsomething.so:
readelf -a libsomething.so.1 | grep SONAME
0xNNNNNNNN (SONAME) Library soname: [libsomething.so]
readelf -a libsomething.so.2 | grep SONAME
0xNNNNNNNN (SONAME) Library soname: [libsomething.so]
Install the libraries into their respective machines under /lib/ directory. Run ldconfig -v on both machines and verify the output.
ldconfig -v 2>&1 | grep something
libsomething.so -> libsomething.so.1 (changed)
ldconfig -v 2>&1 | grep something
libsomething.so -> libsomething.so.2 (changed)
Compile executable and make sure that it refers to the same soname without version in NEEDED.
readelf -a executable | grep NEEDED
0xNNNNNNNN (NEEDED) Shared library: [libsomething.so]
You executable depends on unversioned libsomething.so now. Copy executable to both machines and run ldd against both copies.
ldd executable
libsomething.so => /lib/libsomething.so (0xNNNNNNNN)
The last output is the same on both machines as the executable was built with soname without version. This makes loader take unversioned symlinks on targets machines. And depending on the machine, the symlink can point to different implementation of the library libsomething.so.1 or libsomething.so.2.

This means it won't be able to find the library on machine B.
And it's not supposed to anyway.
By the very definition of soversions, libsomething.so.2 denotes that the API/ABI is incompatible to libsomething.so.1. Therefore, just adding libsomething.so in the program's table of libraries to be loaded would be factually wrong. The libsomething.so symlink merely serves as a hint to ld as to which soversion to pick by default.
Of whatever file ld actually ended up opening, it will take the DTNAME/SONAME field to encode in the program. If you don't want that, don't equip libsomething with a soname. But it can easily become pain... starting with running into unavailable symbols when trying to run the program.

Related

Modify default library search dirs that gcc passes to ld

I want to force new GCC 12 on my old debian (that only has GCC 6 by default) to use fresh libstdc++ headers with new header-only features, but link with old stdlibc++,gcc_s (and other system/compiler libs used by GCC6) to keep binary compatibility with native runtime of old debian (so that users of old GCC6 can link with my binaries without having GCC12).
Of course I know that some functionality in the old runtime will be missing, and ABI is also different, but I guess I can fight with that. Afterall RedHat seems to be using similar scheme for their devtoolset packages (they try to link missing functionality of new runtime statically to your binary if these symbols are not found in native old runtime)
So far I am stuck with -L arguments that GCC is passing to ld.
Here is complete output of /usr/local/gcc12/bin/x86_64-linux-gnu-gcc-12 main.cpp -Wl,-v -v command for simple hello-world main.cpp:
https://pastebin.com/JhYSfg4x
The question: Where does GCC take all these -L paths from, and how do I remove/modify them? I don't want to accidentally link with new version of libraries that were built with GCC12:
-L/usr/lib/gcc/x86_64-linux-gnu -L/usr/local/gcc12/lib/gcc/x86_64-linux-gnu/12 -L/usr/local/gcc12/lib/gcc/x86_64-linux-gnu/12/../../../../lib64 -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/usr/lib/gcc -L/usr/local/gcc12/lib/gcc/x86_64-linux-gnu/12/../../.. /tmp/ccXfhCs4.o
System ld.conf does not mention any paths to /usr/local/gcc12 folder.
-nostdlib and -nodefaultlibs are removing some standard -l flags, but they are not doing anything to -L flags.
Update: I ended up just removing all *.a, .so, *.la files from include, lib and lib64 folders of gcc12, and I also added -L path to native libraries. This way I am sure gcc12 can't pickup one of its libraries for li nking. Not sure if this is good solution, but it works.

ldd -r equivalent on macOS

I am trying to make my software available on macOS and in my toolchain I use ldd -r MyModel.so to verify is everything went well but I can't really find anything conclusive on macOS that would have the same behaviour.
otool or nm seems the two directions to go but I am not sure how to be sure what options would behave the same. Or is there another tool ?
ldd uses ld to load executable files, and recursively loads
dynamically-linked libraries. So using ldd requires being on the target system
(e.g., Linux). Thus, ldd cannot be used for ELF files on macOS.
Assuming that the question is about analyzing Mach-O files on macOS, I do not know of any tool that works for Mach-O files as ldd does for ELF files.
Both otool and nm perform a static analysis.
A possibility is:
otool -L /usr/bin/true
Relevant:
MacOSX: which dynamic libraries linked by binary?
Inspect and get binary from ELF file on MAC

Linking .a files in Mac OSx

I have added libxxx.a in /usr/lib but when I perform otool -L myproject.so, libxxx.a was not included in the list of libraries. I have also included libxxx.a in my build file so I was thinking that I have successfully added it.
How can I like .a file?
I didn't have a problem with .dylib files though.
otool won't show the static library as they are included within the executable binary (a .dylib in this case). This is because static libraries are a collection of object (.o) files and it's pretty much the same as adding file1.o ... fileN.o to the linker command line and you can't see object files from otool either.
One way to check that your static library is part of the executable (other than it successfully linking) is to use the nm command which lists symbols. Providing the executable binary is not stripped, you would do something like:
$ nm /path/to/libLibrary.dylib | grep aClassOrFunctionInStaticLibrary
and the symbol being searched should have the letter t next to it, to indicate that it's part of the executable text section.
Also as mentioned by #PaulR, /usr/lib is part of the operating system and you should not add files there; use /usr/local/lib instead as /usr/local is designed for site-specific additions to the system and files there will survive an operating system update.

Why is my shared library not found?

I'm trying to compile an example program that links to the shared library produced by Sundown. I'm compiling the program like so.
$ gcc -o sd sundown.c -L. -lsundown
Yet, when I run it I get the following error.
./sd: error while loading shared libraries: libsundown.so: cannot open shared object
file: No such file or directory
The output of ls is.
$ ls
libsundown.so libsundown.so.1 sundown.c sd
Why is the shared library not found by ld?
Short solution:
add . (or whatever it is from your -L flag) to your LD_LIBRARY_PATH. When you run sd, it'll look for libraries in the standard places and the LD_LIBRARY_PATH. Note that since you've added ., this will only work if you run sd from the same directory libsundown.so is in.
I plan on distributing the compiled binary. How can I do so that the library can be distributed without forcing people to edit their LD_LIBRARY_PATH?
You should install libsundown.so in one of the standard places, like /usr/lib or /usr/local/lib. You can do that with an installer or a make file, or something as simple as a INSTALL or README that tells the user to stick the libraries there and ensure the permissions are set to something sensible.
On Centos systems with /usr/lib and /usr/lib64, if you install 64-bit libs manually into /usr/lib then at runtime, the library may not be visible even though at build time it is visible (I used autotools and it was able to find my zopfli library from /usr/lib without any problem). When I execute the my_binary that links to /usr/lib/libzopfli.so.1 I got
libzopfli.so.1 => not found
After moving libzopfly.so.1 from /usr/lib to /usr/lib64, then everything works fine.

g++ library search failure

I have a directory structure as below:
/
|
+--- /lib
| libsomething.a
| libsomething.so
|
+----/obj
anObjFile.o
When I run the following command from within the obj directory:
g++ -L../lib -lsomething anObjFile.o
I get undefined reference errors. Apparently gcc is failing to locate libsomething.a.
But now if I delete libsomething.so and then rerun the command, linking succeeds.
As per gcc documentation -lsomething should expand to libsomething.a. Then why is presence of libsomething.so causing the library search to fail? Also, how can I resolve this problem?
I'm on Linux Mint 12 with gcc version 4.6.1.
The .so file is a dynamic library, meaning it is used each time you run your program. In that case you need to tell your OS (Mint in this case) where that dynamic library is located so your prgramm can find it at run time.
The .a file is a static library, meaning functions needed from it will be copied into your prgramm while linking. So your prgramm can run without the library file.
If nothing else is said gcc will assume you want to use the dynamic lib. If it can find a .so file it will use it instead of a .a file. If you want to tell gcc to make your programm static (use the .a file) you can add -static to your gcc call.
However the reason why the linking fails when the dynamic library is used is not clear. How did you install/build the library?

Resources