I am currently working on a simple data synchonizer in a mixture of Fortran and C/C++ by using OpenMPI libraries. The synchonizer compiles and links correctly, as far as I can see:
f95 -o fortran_mpi_test *.o -L/usr/lib/gcc/x86_64-redhat-linux/4.1.1/
-L/usr/lib64/openmpi/1.4-gcc/lib/ -lmpi -lmpi_cxx -lstdc++
But when I execute the resulting executable on the same machined I get an error stating that one of the shared libraries is not found. That is confirmed by ldd.
Nevertheless the missing library libmpi_cxx.so.0 is located in one of the specified folders.
Could anyone give me a hint what I could have done wrong?
Check your environment variables. If your LIBRARY_PATH, LD_LIBRARY_PATH or similar vars have gotten out of sync or set to silly values you might not be searching the same directories for static libraries as you do for dynamics.
Also check out the ld.so manpage
Related
I'm building a simple C++ program and I want to temporarily substitute a system supplied shared library with a more recent version of it, for development and testing.
I tried setting the LD_LIBRARY_PATH variable but the linker (ld) failed with:
/usr/bin/ld: cannot find -lyaml-cpp
I expected that to work because according to the ld man page:
The linker uses the following search
paths to locate required shared
libraries: ... For a native linker,
the contents of the environment variable
"LD_LIBRARY_PATH"...
I then tried setting the LIBRARY_PATH, and that worked.
According to the GCC manual:
The value of
LIBRARY_PATH is a colon-separated list
of directories, much like PATH. When
configured as a native compiler, GCC
tries the directories thus specified
when searching for special linker
files, if it can't find them using
GCC_EXEC_PREFIX. Linking using GCC
also uses these directories when
searching for ordinary libraries for
the -l option (but directories
specified with -L come first).
As the (GCC) manual suggests, LIBRARY_PATH works because I link with GCC.
But..
Since I link with gcc why ld is
being called, as the error message
suggests?
What's the point of
having two variables serving the same
purpose? Are there any other
differences?
LIBRARY_PATH is used by gcc before compilation to search directories containing static and shared libraries that need to be linked to your program.
LD_LIBRARY_PATH is used by your program to search directories containing shared libraries after it has been successfully compiled and linked.
EDIT:
As pointed below, your libraries can be static or shared. If it is static then the code is copied over into your program and you don't need to search for the library after your program is compiled and linked. If your library is shared then it needs to be dynamically linked to your program and that's when LD_LIBRARY_PATH comes into play.
LD_LIBRARY_PATH is searched when the program starts, LIBRARY_PATH is searched at link time.
caveat from comments:
When linking libraries with ld (instead of gcc or g++), the LIBRARY_PATH or LD_LIBRARY_PATH environment variables are not read.
When linking libraries with gcc or g++, the LIBRARY_PATH environment variable is read (see documentation "gcc uses these directories when searching for ordinary libraries").
Since I link with gcc why ld is being called, as the error message suggests?
gcc calls ld internally when it is in linking mode.
LIBRARY_PATH is used by linker (ld)
LD_LIBRARY_PATH is used by loader (ld.so)
According to the manual page for ld (and gcc used for linking by extension), if a -L option appears on the command line, it applies to all libraries specified by -l and takes precedence over the default search locations. However, that is not working in my link step. I have this on the command line:
-L /users/me/mylib -lpcre -lz
and /users/me/mylib contains (copies) of libpcre.so and libz.so
These libraries exist in other locations on the system (although not necessarily the same versions) and what I see (with ldd on Linux and otool on Mac) is a path that references the libraries in those locations. Some of those locations are on the LD_LIBRARY_PATH (which I cannot control in the build environment I am running in) and it appears that somehow those locations are being picked up in preference to my explicit setting with -L.
Just to be clear, this a link step problem and not a runtime problem. There is a lot of info on the web on how to affect/override library locations when executing and I am familiar with all that. In some sense what I am trying to do with the -L is create a completely specified setup. I know I can fix things up with install_name_tool on MacOS but I'd really like to understand why -L isn't doing what it claims to.
One thing I learned using gcc -Wl,-v is that gcc appears to forward all the LD_LIBRARY_PATH directories to ld. However, it places them after the ones explicitly listed by me and man ld says they are searched in order they appear on the line.
Just to be clear, this a link step problem and not a runtime problem.
From what you describe as the problem, I don't think you are right about this - it sounds like a runtime problem for which you are (justifiably) looking for a solution that you can employ while linking that will solve the problem you have at runtime.
The reason I say it does not appear to be a problem with linking is that it sounds like your linking is working as it is intended. LD (or GCC) are not complaining about the linking, and your linked executables are being produced just fine. The issue you are having is that when you subsequently go to run those executables, the loader is finding libraries other than the ones you intend. The purpose of the -L flag during linking is to let the linker know where it can find suitable libraries to use in preparing the linked binary. That is completely separate from where the loader will search for the required libraries at runtime.
As you say, you are already aware that there are ways you could employ at runtime (such as changing LD_LIBRARY_PATH) that would avoid the issue by changing the set of paths that the loader searches for libraries, but you'd rather not have to do that because for whatever reason you won't necessarily have control over the runtime environment, which is fair enough.
Luckily, there is a facility that I believe will get you what you want. Take a look at the ld option called -rpath (see the GNU ld man page for full documentation). Basically, if you add paths during linking using the -rpath option, those paths gets stored in the linked executable as preferred locations to find the libraries at runtime, in much the same way they would be searched if included in LD_LIBRARY_PATH. This should work on Linux or Mac OS X (at least since 10.5).
Passing the -rpath option to ld via gcc requires using the -Wl option to pass the flag through. To obtain an ld command line that contains ld -rpath /custom/path/to/libs requires a gcc invocation something like: gcc -Wl,-rpath,/custom/path/to/libs
In short, try replacing what you currently have: -L/users/me/mylib -lpcre -lz
With: -L/users/me/mylib -Wl,-rpath,/users/me/mylib -lpcre -lz
The resulting executable (or library) will then have /users/me/mylib stored as the place to go to find libraries, and it should find libpcre.so and libz.so there without needing to control LD_LIBRARY_PATH.
I am a novice to cmake and boost so this question might be missing something obvious:
I am building a project with cmake on linux (ubuntu) and I am trying to use boost logging in that project. Here is what I do to generate the Makefile:
rm CMakeCache.txt
cmake ../ -DCMAKE_EXE_LINKER_FLAGS="-lboost_log -lboost_log_setup -lpthread -std=c++11" -DCMAKE_SHARED_LINKER_FLAGS="-lboost_log_setup -lboost_log -lpthread" -DCMAKE_MODULE_LINKER_FLAGS="-lboost_log_setup -lboost_log -lpthread" -DCMAKE_CXX_FLAGS="-DBOOST_LOG_DYN_LINK -std=c++11"
Compile goes through fine. (Some of these flags may be overkill -- I should only need the CMAKE_EXE_LINKER_FLAGS).
When I run the executable, I get the following unresolved reference:
-- ImportError: /home/mranga/gr-msod-sensor/gr-msod_sensor/build/lib/libgnuradio-msod_sensor.so: undefined symbol: _ZN5boost3log11v2_mt_posix3aux25unhandled_exception_countEv
What flags am I missing? My boost library is set up and LD_LIBRARY_PATH points to the right location.
When I manually built a test program using the same linker flags, it compiles and runs fine so boost is installed correctly. I hope I have not missed the obvious.
(Moved question from the GNU Radio mailing list -- sorry if you are reading this post for a second time).
I believe the order of libraries in the linker command line in -DCMAKE_EXE_LINKER_FLAGS is incorrect. boost_log_setup depends on boost_log, so boost_log_setup should go first.
You seem to be linking against the non-multithreaded version:
-lboost_log
but the run-time linker seems to explicitely look for the multithreaded variant (the Boost doc site on that):
_ZN5boost3log11v2_mt_posix3aux25unhandled_exception_countEv
^^
My guess hence is that you should try linking with
-lboost_log_mt
but the question whether that is right or not depends too much on your individual project to make it possible for me to clearly answer this.
There is a laptop on which I have no root privilege.
onto the machine I have a library installed using configure --prefix=$HOME/.usr .
after that, I got these files in ~/.usr/lib :
libXX.so.16.0.0
libXX.so.16
libXX.so
libXX.la
libXX.a
when I compile a program that invokes one of function provided by the library with this command :
gcc XXX.c -o xxx.out -L$HOME/.usr/lib -lXX
xxx.out was generated without warning, but when I run it error like this was thrown:
./xxx.out: error while loading shared libraries: libXX.so.16: cannot open shared object file: No such file or directory , though libXX.so.16 resides there.
my clue-less assumption is that ~/.usr/lib wasn't searched when xxx.out is invoked.
but what can I do to specify path of .so , in order that xxx.out can look there for .so file?
An addition is when I feed -static to gcc, another error happens like this:
undefined reference to `function_proviced_by_the_very_librar'
It seems .so does not matter even though -L and -l are given to gcc.
what should I do to build a usable exe with that library?
For other people who has the same question as I did
I found a useful article at tldp about this.
It introduces static/shared/dynamic loaded library, as well as some example code to use them.
There are two ways to achieve that:
Use -rpath linker option:
gcc XXX.c -o xxx.out -L$HOME/.usr/lib -lXX -Wl,-rpath=/home/user/.usr/lib
Use LD_LIBRARY_PATH environment variable - put this line in your ~/.bashrc file:
export LD_LIBRARY_PATH=/home/user/.usr/lib
This will work even for a pre-generated binaries, so you can for example download some packages from the debian.org, unpack the binaries and shared libraries into your home directory, and launch them without recompiling.
For a quick test, you can also do (in bash at least):
LD_LIBRARY_PATH=/home/user/.usr/lib ./xxx.out
which has the advantage of not changing your library path for everything else.
Should it be LIBRARY_PATH instead of LD_LIBRARY_PATH.
gcc checks for LIBRARY_PATH which can be seen with -v option
I'm building a simple C++ program and I want to temporarily substitute a system supplied shared library with a more recent version of it, for development and testing.
I tried setting the LD_LIBRARY_PATH variable but the linker (ld) failed with:
/usr/bin/ld: cannot find -lyaml-cpp
I expected that to work because according to the ld man page:
The linker uses the following search
paths to locate required shared
libraries: ... For a native linker,
the contents of the environment variable
"LD_LIBRARY_PATH"...
I then tried setting the LIBRARY_PATH, and that worked.
According to the GCC manual:
The value of
LIBRARY_PATH is a colon-separated list
of directories, much like PATH. When
configured as a native compiler, GCC
tries the directories thus specified
when searching for special linker
files, if it can't find them using
GCC_EXEC_PREFIX. Linking using GCC
also uses these directories when
searching for ordinary libraries for
the -l option (but directories
specified with -L come first).
As the (GCC) manual suggests, LIBRARY_PATH works because I link with GCC.
But..
Since I link with gcc why ld is
being called, as the error message
suggests?
What's the point of
having two variables serving the same
purpose? Are there any other
differences?
LIBRARY_PATH is used by gcc before compilation to search directories containing static and shared libraries that need to be linked to your program.
LD_LIBRARY_PATH is used by your program to search directories containing shared libraries after it has been successfully compiled and linked.
EDIT:
As pointed below, your libraries can be static or shared. If it is static then the code is copied over into your program and you don't need to search for the library after your program is compiled and linked. If your library is shared then it needs to be dynamically linked to your program and that's when LD_LIBRARY_PATH comes into play.
LD_LIBRARY_PATH is searched when the program starts, LIBRARY_PATH is searched at link time.
caveat from comments:
When linking libraries with ld (instead of gcc or g++), the LIBRARY_PATH or LD_LIBRARY_PATH environment variables are not read.
When linking libraries with gcc or g++, the LIBRARY_PATH environment variable is read (see documentation "gcc uses these directories when searching for ordinary libraries").
Since I link with gcc why ld is being called, as the error message suggests?
gcc calls ld internally when it is in linking mode.
LIBRARY_PATH is used by linker (ld)
LD_LIBRARY_PATH is used by loader (ld.so)