I will use a simplified example to explain my case.
I have two libme.so files which have different implementations and I already verified that they can work in a single design at the same time. One is located at ./root/v1/ and the other one is located at ./root/v2/
My "main" .so file is linked with those 2 libme.so files and is located at ./root/libtest.so
Now I have the requirement to make the things relocatable. That is, if I copy the entire "root" directory to another location or even another machine (assuming binary compatible), things should still work fine.
My question is, to make this work, what gcc command line should I use to build libtest.so?
I've tried the following two:
(1) (assuming I'm in "root" directory)
>gcc -shared -o libtest.so ./v1/libme.so ./v2/libme.so
This will make libtest.so has those 2 link dependencies both with absolute paths. This can be verified via ldd:
>ldd libtest.so
/home/design/root/v1/libme.so
/home/design/root/v2/libme.so
Obviously the paths are fixed. So once I relocate the "root" directory, it cannot find libme.so at runtime. Note setting LD_LIBRARY_PATH doesn't work in this case since the path from ldd is absolute path. The runtimer loader will not search LD_LIBRARY_PATH to find libme.so.
(2)
>gcc -shared -o libtest.so -lme -L./v1 -L./v2
This will work only if we have a single version of libme.so. In this case, the version in ./v2 will not get linked in. The same issue exists for -rpath.
Given that, what other options do I have?
Note there are some restrictions:
(1) cannot rename libme.so to other names such as libme_v1.so
(2) libtest.so has to link in both versions of libme.so
Teppic is right. This turned out to be an issue only happening on one of my machines. Switching to another RHEL5.8 machine and I'm all set. It looks like the original machine installed a linker customized by someone.
Related
According to the manual page for ld (and gcc used for linking by extension), if a -L option appears on the command line, it applies to all libraries specified by -l and takes precedence over the default search locations. However, that is not working in my link step. I have this on the command line:
-L /users/me/mylib -lpcre -lz
and /users/me/mylib contains (copies) of libpcre.so and libz.so
These libraries exist in other locations on the system (although not necessarily the same versions) and what I see (with ldd on Linux and otool on Mac) is a path that references the libraries in those locations. Some of those locations are on the LD_LIBRARY_PATH (which I cannot control in the build environment I am running in) and it appears that somehow those locations are being picked up in preference to my explicit setting with -L.
Just to be clear, this a link step problem and not a runtime problem. There is a lot of info on the web on how to affect/override library locations when executing and I am familiar with all that. In some sense what I am trying to do with the -L is create a completely specified setup. I know I can fix things up with install_name_tool on MacOS but I'd really like to understand why -L isn't doing what it claims to.
One thing I learned using gcc -Wl,-v is that gcc appears to forward all the LD_LIBRARY_PATH directories to ld. However, it places them after the ones explicitly listed by me and man ld says they are searched in order they appear on the line.
Just to be clear, this a link step problem and not a runtime problem.
From what you describe as the problem, I don't think you are right about this - it sounds like a runtime problem for which you are (justifiably) looking for a solution that you can employ while linking that will solve the problem you have at runtime.
The reason I say it does not appear to be a problem with linking is that it sounds like your linking is working as it is intended. LD (or GCC) are not complaining about the linking, and your linked executables are being produced just fine. The issue you are having is that when you subsequently go to run those executables, the loader is finding libraries other than the ones you intend. The purpose of the -L flag during linking is to let the linker know where it can find suitable libraries to use in preparing the linked binary. That is completely separate from where the loader will search for the required libraries at runtime.
As you say, you are already aware that there are ways you could employ at runtime (such as changing LD_LIBRARY_PATH) that would avoid the issue by changing the set of paths that the loader searches for libraries, but you'd rather not have to do that because for whatever reason you won't necessarily have control over the runtime environment, which is fair enough.
Luckily, there is a facility that I believe will get you what you want. Take a look at the ld option called -rpath (see the GNU ld man page for full documentation). Basically, if you add paths during linking using the -rpath option, those paths gets stored in the linked executable as preferred locations to find the libraries at runtime, in much the same way they would be searched if included in LD_LIBRARY_PATH. This should work on Linux or Mac OS X (at least since 10.5).
Passing the -rpath option to ld via gcc requires using the -Wl option to pass the flag through. To obtain an ld command line that contains ld -rpath /custom/path/to/libs requires a gcc invocation something like: gcc -Wl,-rpath,/custom/path/to/libs
In short, try replacing what you currently have: -L/users/me/mylib -lpcre -lz
With: -L/users/me/mylib -Wl,-rpath,/users/me/mylib -lpcre -lz
The resulting executable (or library) will then have /users/me/mylib stored as the place to go to find libraries, and it should find libpcre.so and libz.so there without needing to control LD_LIBRARY_PATH.
We know that the command for making the executable is :
gcc -L/home/username/foo -Wall -o test main.c -lfoo
But if run ./test ,it will give error :
/test
./test: error while loading shared libraries: libfoo.so: cannot open shared object file: No such file or directory
There are 2 solutionss for it:
1)
LD_LIBRARY_PATH=/home/username/foo
2)
gcc -L/home/username/foo -Wl,-rpath=/home/username/foo -Wall -o test main.c -lfoo
my question is why do need to provide the library path to the loader??While making the execuatble I have explicilty mentioned the path such that the linker could create the excutable and I beleive there must be a refernce of the location of the library embedded in th execuatble
This a conceptual doubt i have ,plz clear it out.Thank you
It's just a choice that the linker designers made -- but a good one.
The reason it is good is that it lets you build a library, link against it, and then install the executable and library without requiring a re-link.
The cost is that you have to use LD_LIBRARY_PATH or the like to run the executable in the build tree; but normally this is no problem, as you can add the needed setting to make check or what have you.
If the default were flipped, then you would have to either relink at install time (which is a bit unfriendly since it is reasonably common to make install as root); or provide a flag to turn off the behavior at build time (but then you're back to the above...).
-Wl,rptath: Its actually build the executable feeding the information that where the loader will check for the library for loading.
Lets me give an example with 2 cases:
1)
If I transfer the library and executable to another machine and place them in 2 different location then I need to mention the path where the library is located by setting LD_LIBRARY_PATH.Then the loader will understand where to look for the loader while executing.
2)If i make the make the executable with -Wl,rpath= then I need to place the execuatble in the same path in another machine as the value of -Wl,rpath=.
So in simple terms we can say that we have the independency of placing the library with LD_LIBRARY_PATH
We have a project and shared libraries libprivate.so (private so) which was using old libraries libcurl.so.3. The system was upgraded with new system libraries libcurl.so.4.
For some internal issues, right now we do not want to make use of latest libraries libcurl.so.4, we want to make use of libcurl.so.3.
Hence I copied libcurl.so.3 in local folder and set LD_LIBRARY_PATH according. When I link my entire project it says that there is version conflict between libcurl.so.4 and libcurl.so.3 required libprivate.so (libprivate.so is compiled long time ago with libcurl.3.so).
Should I not worry about this warning and proceed further?
When I correctly specify LD_LIBRARY_PATH which has libcurl.so.3, why it is taking from system directory /usr/lib64/libcurl.so.4? when I do ldd my_binary, it takes from libcurl.so.4. How do I stop it? Specifying -L with specific location also doesn't work. Modiying /etc/ld.conf will do for the entire system. I want to make this when I ran my project.
Specifying explicit path it works like /home/mydir/libcurl.so.3, but I do not want to do it.
I want to have these conditions only when I execute my project. In other cases it can make use of latest libraries.
Thanks for your help
If the command you show in your comment is correct:
gcc test.c -L~/lib/x86_64/ -lcurl -o test
... then you need a space between -L and ~/lib/x86_64/ or the shell won't expand the ~, so the linker is not looking in the right directory.
So you need either:
gcc test.c -L ~/lib/x86_64/ -lcurl -o test
or:
gcc test.c -L$HOME/lib/x86_64/ -lcurl -o test
(You don't need a space here because variables are expanded anywhere in a word, but ~ is only expanded at the start of a word.)
There is a laptop on which I have no root privilege.
onto the machine I have a library installed using configure --prefix=$HOME/.usr .
after that, I got these files in ~/.usr/lib :
libXX.so.16.0.0
libXX.so.16
libXX.so
libXX.la
libXX.a
when I compile a program that invokes one of function provided by the library with this command :
gcc XXX.c -o xxx.out -L$HOME/.usr/lib -lXX
xxx.out was generated without warning, but when I run it error like this was thrown:
./xxx.out: error while loading shared libraries: libXX.so.16: cannot open shared object file: No such file or directory , though libXX.so.16 resides there.
my clue-less assumption is that ~/.usr/lib wasn't searched when xxx.out is invoked.
but what can I do to specify path of .so , in order that xxx.out can look there for .so file?
An addition is when I feed -static to gcc, another error happens like this:
undefined reference to `function_proviced_by_the_very_librar'
It seems .so does not matter even though -L and -l are given to gcc.
what should I do to build a usable exe with that library?
For other people who has the same question as I did
I found a useful article at tldp about this.
It introduces static/shared/dynamic loaded library, as well as some example code to use them.
There are two ways to achieve that:
Use -rpath linker option:
gcc XXX.c -o xxx.out -L$HOME/.usr/lib -lXX -Wl,-rpath=/home/user/.usr/lib
Use LD_LIBRARY_PATH environment variable - put this line in your ~/.bashrc file:
export LD_LIBRARY_PATH=/home/user/.usr/lib
This will work even for a pre-generated binaries, so you can for example download some packages from the debian.org, unpack the binaries and shared libraries into your home directory, and launch them without recompiling.
For a quick test, you can also do (in bash at least):
LD_LIBRARY_PATH=/home/user/.usr/lib ./xxx.out
which has the advantage of not changing your library path for everything else.
Should it be LIBRARY_PATH instead of LD_LIBRARY_PATH.
gcc checks for LIBRARY_PATH which can be seen with -v option
I have followed the instructions at Cross linux from scratch for powerpc, but I can't get past the gcc-final stage due to an ld error crti.o: no such file or directory.
My tool chain is separated into two directories: /opt/builder/tools and /opt/builder/cross-tools, with Linux headers and eglibc in the first and cross-utilities in the second. Also I have created links /tools and /cross-tools for those directories.
I have tried /cross-tools/bin/powerpc-unknown-linux-gcc -print-search-path and there was a directory /opt/builder/tools/lib there in the "libraries" section. But gcc passes names of crt....o objects to ld without directory prefixes (I have used strace to find it out).
The crt....o files are good because, when I copied them to the directory with hello.c and compiled it, everything was ok.
Could you give some advice?
Thanks in advance,
Sergey Naumov.
The libc-provided crt files should be in $ROOT/$TARGET/lib/ (i.e., The gcc-compiled crt files will later be installed in $ROOT/lib/gcc/$TARGET/$GCC_VERSION/.