According to the manual page for ld (and gcc used for linking by extension), if a -L option appears on the command line, it applies to all libraries specified by -l and takes precedence over the default search locations. However, that is not working in my link step. I have this on the command line:
-L /users/me/mylib -lpcre -lz
and /users/me/mylib contains (copies) of libpcre.so and libz.so
These libraries exist in other locations on the system (although not necessarily the same versions) and what I see (with ldd on Linux and otool on Mac) is a path that references the libraries in those locations. Some of those locations are on the LD_LIBRARY_PATH (which I cannot control in the build environment I am running in) and it appears that somehow those locations are being picked up in preference to my explicit setting with -L.
Just to be clear, this a link step problem and not a runtime problem. There is a lot of info on the web on how to affect/override library locations when executing and I am familiar with all that. In some sense what I am trying to do with the -L is create a completely specified setup. I know I can fix things up with install_name_tool on MacOS but I'd really like to understand why -L isn't doing what it claims to.
One thing I learned using gcc -Wl,-v is that gcc appears to forward all the LD_LIBRARY_PATH directories to ld. However, it places them after the ones explicitly listed by me and man ld says they are searched in order they appear on the line.
Just to be clear, this a link step problem and not a runtime problem.
From what you describe as the problem, I don't think you are right about this - it sounds like a runtime problem for which you are (justifiably) looking for a solution that you can employ while linking that will solve the problem you have at runtime.
The reason I say it does not appear to be a problem with linking is that it sounds like your linking is working as it is intended. LD (or GCC) are not complaining about the linking, and your linked executables are being produced just fine. The issue you are having is that when you subsequently go to run those executables, the loader is finding libraries other than the ones you intend. The purpose of the -L flag during linking is to let the linker know where it can find suitable libraries to use in preparing the linked binary. That is completely separate from where the loader will search for the required libraries at runtime.
As you say, you are already aware that there are ways you could employ at runtime (such as changing LD_LIBRARY_PATH) that would avoid the issue by changing the set of paths that the loader searches for libraries, but you'd rather not have to do that because for whatever reason you won't necessarily have control over the runtime environment, which is fair enough.
Luckily, there is a facility that I believe will get you what you want. Take a look at the ld option called -rpath (see the GNU ld man page for full documentation). Basically, if you add paths during linking using the -rpath option, those paths gets stored in the linked executable as preferred locations to find the libraries at runtime, in much the same way they would be searched if included in LD_LIBRARY_PATH. This should work on Linux or Mac OS X (at least since 10.5).
Passing the -rpath option to ld via gcc requires using the -Wl option to pass the flag through. To obtain an ld command line that contains ld -rpath /custom/path/to/libs requires a gcc invocation something like: gcc -Wl,-rpath,/custom/path/to/libs
In short, try replacing what you currently have: -L/users/me/mylib -lpcre -lz
With: -L/users/me/mylib -Wl,-rpath,/users/me/mylib -lpcre -lz
The resulting executable (or library) will then have /users/me/mylib stored as the place to go to find libraries, and it should find libpcre.so and libz.so there without needing to control LD_LIBRARY_PATH.
Related
I want to force new GCC 12 on my old debian (that only has GCC 6 by default) to use fresh libstdc++ headers with new header-only features, but link with old stdlibc++,gcc_s (and other system/compiler libs used by GCC6) to keep binary compatibility with native runtime of old debian (so that users of old GCC6 can link with my binaries without having GCC12).
Of course I know that some functionality in the old runtime will be missing, and ABI is also different, but I guess I can fight with that. Afterall RedHat seems to be using similar scheme for their devtoolset packages (they try to link missing functionality of new runtime statically to your binary if these symbols are not found in native old runtime)
So far I am stuck with -L arguments that GCC is passing to ld.
Here is complete output of /usr/local/gcc12/bin/x86_64-linux-gnu-gcc-12 main.cpp -Wl,-v -v command for simple hello-world main.cpp:
https://pastebin.com/JhYSfg4x
The question: Where does GCC take all these -L paths from, and how do I remove/modify them? I don't want to accidentally link with new version of libraries that were built with GCC12:
-L/usr/lib/gcc/x86_64-linux-gnu -L/usr/local/gcc12/lib/gcc/x86_64-linux-gnu/12 -L/usr/local/gcc12/lib/gcc/x86_64-linux-gnu/12/../../../../lib64 -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/usr/lib/gcc -L/usr/local/gcc12/lib/gcc/x86_64-linux-gnu/12/../../.. /tmp/ccXfhCs4.o
System ld.conf does not mention any paths to /usr/local/gcc12 folder.
-nostdlib and -nodefaultlibs are removing some standard -l flags, but they are not doing anything to -L flags.
Update: I ended up just removing all *.a, .so, *.la files from include, lib and lib64 folders of gcc12, and I also added -L path to native libraries. This way I am sure gcc12 can't pickup one of its libraries for li nking. Not sure if this is good solution, but it works.
I'm a student doing research involving extending the TM capabilities of gcc. My goal is to make changes to gcc source, build gcc from the modified source, and, use the new executable the same way I'd use my distro's vanilla gcc.
I built and installed gcc in a different location (not /usr/bin/gcc), specifically because the modified gcc will be unstable, and because our project goal is to compare transactional programs compiled with the two different versions.
Our changes to gcc source impact both /gcc and /libitm. This means we are making a change to libitm.so, one of the shared libraries that get built.
My expectation:
when compiling myprogram.cpp with /usr/bin/g++, the version of libitm.so that will get linked should be the one that came with my distro;
when compiling it with ~/project/install-dir/bin/g++, the version of libitm.so that will get linked should be the one that just got built when I built my modified gcc.
But in reality it seems both native gcc and mine are using the same libitm, /usr/lib/x86_64-linux-gnu/libitm.so.1.
I only have a rough grasp of gcc internals as they apply to our project, but this is my understanding:
Our changes tell one compiler pass to conditionally insert our own "function builtin" instead of one it would normally use, and this is / becomes a "symbol" which needs to link to libitm.
When I use the new gcc to compile my program, that pass detects those conditions and successfully inserts the symbol, but then at runtime my program gives a "relocation error" indicating the symbol is not defined in the file it is searching in: ./test: relocation error: ./test: symbol _ITM_S1RU4, version LIBITM_1.0 not defined in file libitm.so.1 with link time reference
readelf shows me that /usr/lib/x86_64-linux-gnu/libitm.so.1 does not contain our new symbols while ~/project/install-dir/lib64/libitm.so.1 does; if I re-run my program after simply copying the latter libitm over the former (backing it up first, of course), it does not produce the relocation error anymore. But naturally this is not a permanent solution.
So I want the gcc I built to use the shared libs that were built along with it when linking. And I don't want to have to tell it where they are every time - my feeling is that it should know where to look for them since I deliberately built it somewhere else to behave differently.
This sounds like the kind of problem any amateur gcc developer would have when trying to make a dev environment and still be able to use both versions of gcc, but I had difficulty finding similar questions. I am thinking this is a matter of lacking certain config options when I configure gcc before building it. What is the right configuration to do this?
My small understanding of the instructions for building and installing gcc led me to do the following:
cd ~/project/
mkdir objdir
cd objdir
../source-dir/configure --enable-languages=c,c++ --prefix=/home/myusername/project/install-dir
make -j2
make install
I only have those config options because they seemed like the ones closest related to "only building the parts I need" and "not overwriting native gcc", but I could be wrong. After the initial config step I just re-run make -j2 and make install every time I change the code. All these steps do complete without errors, and they produce the ~/project/install-dir/bin/ folder, containing the gcc and g++ which behave as described.
I use ~/project/install-dir/bin/g++ -fgnu-tm -o myprogram myprogram.cpp to compile a transactional program, possibly with other options for programs with threads.
(I am using Xubuntu 16.04.3 (64 bit), within VirtualBox on Windows. The installed /usr/bin/gcc is version 5.4.0. Our source at ~/project/source-dir/ is a modified version of 5.3.0.)
You’re running into build- versus run-time linking differences. When you build with -fgnu-tm, the compiler knows where the library it needs is found, and it tells the linker where to find it; you can see this by adding -v to your g++ command. However when you run the resulting program, the dynamic linker doesn’t know it should look somewhere special for the ITM library, so it uses the default library in /usr/lib/x86_64-linux-gnu.
Things get even more confusing with ITM on Ubuntu because the library is installed system-wide, but the link script is installed in a GCC-private directory. This doesn’t happen with the default GCC build, so your own GCC build doesn’t do this, and you’ll see libitm.so in ~/project/install-dir/lib64.
To fix this at run-time, you need to tell the dynamic linker where to find the right library. You can do this either by setting LD_LIBRARY_PATH (to /home/.../project/install-dir/lib64), or by storing the path in the binary using -Wl,-rpath=/home/.../project/install-dir/lib64 when you build it.
I'm trying to compile CMake using a non-default GCC installed in /usr/local/gcc530, on Solaris 2.11.
I have LD_LIBRARY_PATH=/usr/local/gcc530/lib/sparcv9
Bootstrap proceeds fine, bootstrapped cmake successfully compiles various object files, but when it tries to link the real cmake (and other executables), I get pages of "undefined reference" errors to various standard library functions, because, as running the link command manually with -Wl,-verbose shows, the linker links with /usr/lib/64/libstdc++.so of the system default, much older GCC.
This is because apparently CMake tries to find curses/ncurses libraries (even if I tell it BUILD_CursesDialog:BOOL=OFF), finds them in /usr/lib/64, and adds -L/usr/lib/64 to build/Source/CMakeFiles/cmake.dir/link.txt, which causes the linker to use libstdc++.so from there, and not my actual GCC's own.
I found a workaround: I can get the path to proper libraries from $CC -m64 -print-file-name=libstdc++.so then put it with -L into LDFLAGS when running ./configure, and all works well then.
Is there a less hacky way? It's really weird that I can't tell GCC to prioritize its own libraries.
Also, is there some way to have CMake explain where different parts of a resulting command line came from?
I am a regular C programmer, and there's something I have wondered for some time about GNU GCC/LD internals.
I have noticed that, when passing a sequence of object files to GCC (e.g gcc main.o otherfile.o), GCC automatically links libc.a library file without my explicitly specifying -lc in the options. Similarly, when I compile a program using ncurses, I just need to specify -lncurses and libtinfo.a gets linked automatically (no need to specify -ltinfo). In other words, even though ncurses functions use functions from libtinfo (for instance, unctrl()), I don't need to explicitly link libtinfo.
How can it be possible?
Does GCC/LD have a list of "default libraries" where it looks for missing symbols when linking? If such a table exists, where is it and how can it be configured?
A direct answer is that you can see what libraries are linked by command line options by adding -v to the linking command line. This will show you the commands as they are executed.
The C library and the GCC support library or libraries are linked automatically; other libraries have to be specified manually.
The case of -lncurses and libtinfo.a (libtinfo.so?) is rather different. There, the link command used to build libncurses.so tells the linker that this library also needs -ltinfo, so it automatically picks up the extra library.
I am currently working on a simple data synchonizer in a mixture of Fortran and C/C++ by using OpenMPI libraries. The synchonizer compiles and links correctly, as far as I can see:
f95 -o fortran_mpi_test *.o -L/usr/lib/gcc/x86_64-redhat-linux/4.1.1/
-L/usr/lib64/openmpi/1.4-gcc/lib/ -lmpi -lmpi_cxx -lstdc++
But when I execute the resulting executable on the same machined I get an error stating that one of the shared libraries is not found. That is confirmed by ldd.
Nevertheless the missing library libmpi_cxx.so.0 is located in one of the specified folders.
Could anyone give me a hint what I could have done wrong?
Check your environment variables. If your LIBRARY_PATH, LD_LIBRARY_PATH or similar vars have gotten out of sync or set to silly values you might not be searching the same directories for static libraries as you do for dynamics.
Also check out the ld.so manpage