I have boost C++ libraries already installed on my Fedora10 machine but I want to use a newer version that I keep at some location in my home folder. I want g++ to use include and library files from my home folder location instead of default (/usr/include and /usr/lib64).
For that matter, I also have declared CPLUS\_INCLUDE\_PATH and LIBRARY\_PATH environment variables in my ~/.bashrc file as explained here.
Now when I run,
g++ -o hello.so -fPIC hello.cpp -shared -lboost_python
The preprocessor uses include files from my home folder location, overriding the default location (as it should, because CPLUS\_INCLUDE\_PATH has a higher precedence in the search path). But the linker does not seem to follow the same precedence rule. It always uses libboost_python.so from the default location /usr/lib64 instead of first searching LIBRARY\_PATH. It only links to the libboost\_python.so library in my home folder when I explicitly specify with -L switch. This is really inconvenient.
The -L switch is the standard way of telling the compiler where to find the libraries. Write a makefile that builds your compiler/linker switches - you'll find it's worth investing your time. You can do something like:
MY_LIBPATH += -L$(BOOST_LIB_PATH)
MY_INCPATH += -I$(BOOST_INC_PATH)
hello.so: hello.cpp
g++ -o $# -fPIC $(MY_INCPATH) $(MY_LIBPATH) hello.cpp -shared -lboost_python
And then you can control this via environment (of course there could be many variations on how to structure the makefile.)
Related
I want to force new GCC 12 on my old debian (that only has GCC 6 by default) to use fresh libstdc++ headers with new header-only features, but link with old stdlibc++,gcc_s (and other system/compiler libs used by GCC6) to keep binary compatibility with native runtime of old debian (so that users of old GCC6 can link with my binaries without having GCC12).
Of course I know that some functionality in the old runtime will be missing, and ABI is also different, but I guess I can fight with that. Afterall RedHat seems to be using similar scheme for their devtoolset packages (they try to link missing functionality of new runtime statically to your binary if these symbols are not found in native old runtime)
So far I am stuck with -L arguments that GCC is passing to ld.
Here is complete output of /usr/local/gcc12/bin/x86_64-linux-gnu-gcc-12 main.cpp -Wl,-v -v command for simple hello-world main.cpp:
https://pastebin.com/JhYSfg4x
The question: Where does GCC take all these -L paths from, and how do I remove/modify them? I don't want to accidentally link with new version of libraries that were built with GCC12:
-L/usr/lib/gcc/x86_64-linux-gnu -L/usr/local/gcc12/lib/gcc/x86_64-linux-gnu/12 -L/usr/local/gcc12/lib/gcc/x86_64-linux-gnu/12/../../../../lib64 -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/usr/lib/gcc -L/usr/local/gcc12/lib/gcc/x86_64-linux-gnu/12/../../.. /tmp/ccXfhCs4.o
System ld.conf does not mention any paths to /usr/local/gcc12 folder.
-nostdlib and -nodefaultlibs are removing some standard -l flags, but they are not doing anything to -L flags.
Update: I ended up just removing all *.a, .so, *.la files from include, lib and lib64 folders of gcc12, and I also added -L path to native libraries. This way I am sure gcc12 can't pickup one of its libraries for li nking. Not sure if this is good solution, but it works.
I want to create a static library libmylib.a from mylib.c/.h and link it to a project to use this library in bootloader code using the arm-none-eabi-gcc cross compiler in ubuntu 20.04 LTS.
I have an electronic engineering background, so I'm kind of new in this compiler and linker stuff.
What I know:
I've been searching about this, and found out that '.a' are just packed '.o' files, and that's it. You can do it using ar in linux. I don't know how to manage the dependencies for this '.a' file, for example, or how to link it to the project.
What I want to know:
I really want to understand how it works, to compile and generate the bin, elf or hex files using these static libraries for arm using the arm-none-eabi-gcc cross compiler (found some for linux), but I don't know how to search for this properly, how to learn it in a linear way. If you guys could help me on this I would be really grateful.
First you create your library objects. Let us say that you have a foo function written in foo.c, then you do:
arm-none-eabi-gcc -c foo.c
The -c options tells the compiler to stop after assembling and no go further.
Then you need to create the .a file
arm-none-eabi-ar -rc libfoo.a foo.o
this command creates a static library called libfoo.a
At the end you compile your main with:
arm-none-eabi-gcc -L. -lfoo main.c -o main
Note that in -l flag we don put "lib" and ".a", those are automagically added. The -L. flag tells gcc to look into the current folder for library files.
Is there a way to get compilers to prefer libraries from LIBRARY_PATH instead of system paths. I am particularly looking for Clang. I partially solved the problem for GCC while writing this question, but it's also not quite clear.
Background
LIBRARY_PATH is a convenient environment variable to allow transparently linking libraries in non-standard directories, e.g. user installations, and in my case environment modules that provide different versions of a library.
The idea is to do a module load libfoo/version and the compiler will transparently use the right libfoo.so.
For a shared library, one also needs to set LD_LIBRARY_PATH for ld.so to find the right library. If there are multiple libfoo.so in both LD_LIBRARY_PATH and /usr/lib, ld.so specifies that LD_LIBRARY_PATH is searched before default paths.
Problem
I have encountered issues when the library defines a soname - which is different between the two libfoo.so versions (which are symlinks to libfoo.so.1 and libfoo.so.2 respectively) in /usr/lib and LIBRARY_PATH. Then ld will link against the soname in /usr/lib and LD_LIBRARY_PATH can no longer prioritize the intended library.
I first encountered this with boost, but here's a small example:
echo "void foo() {}" > foo.c
# create an old libfoo version in /usr/lib
sudo gcc foo.c -fpic -shared -o /usr/lib/libfoo.so.1 -Wl,-soname,libfoo.so.1
sudo ln -s libfoo.so.1 /usr/lib/libfoo.so
# create the new libfoo that we want to transparently override
mkdir -p /tmp/XXX/lib
gcc foo.c -fpic -shared -o /tmp/XXX/lib/libfoo.so.2 -Wl,-soname,libfoo.so.2
ln -s libfoo.so.2 /tmp/XXX/lib/libfoo.so
export LIBRARY_PATH=/tmp/XXX/lib
export LD_LIBRARY_PATH=/tmp/XXX/lib
echo "void foo(); int main() { foo(); }" > main.c
gcc main.c -lfoo
ldd a.out| grep foo
# under some conditions this shows libfoo.so.1 instead of .2
libfoo.so.1 => /usr/lib/libfoo.so.1
GCC
I initially encountered this issue with a custom installation of GCC while it was working as expected for the system installation.
gcc --print-search-dirs reveals a pattern:
/tmp/XXX/lib/x86_64-pc-linux-gnu/7.2.0/
/tmp/XXX/lib/x86_64-linux-gnu/
/tmp/XXX/lib/../lib64/
/opt/gcc/7.2.0/lib/gcc/x86_64-pc-linux-gnu/7.2.0/
/opt/gcc/7.2.0/lib/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../x86_64-pc-linux-gnu/lib/x86_64-pc-linux-gnu/7.2.0/
/opt/gcc/7.2.0/lib/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../x86_64-pc-linux-gnu/lib/x86_64-linux-gnu/
/opt/gcc/7.2.0/lib/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../x86_64-pc-linux-gnu/lib/../lib64/
/opt/gcc/7.2.0/lib/gcc/x86_64-pc-linux-gnu/7.2.0/../../../x86_64-pc-linux-gnu/7.2.0/
/opt/gcc/7.2.0/lib/gcc/x86_64-pc-linux-gnu/7.2.0/../../../x86_64-linux-gnu/
/opt/gcc/7.2.0/lib/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../lib64/
/lib/x86_64-pc-linux-gnu/7.2.0/
/lib/x86_64-linux-gnu/
/lib/../lib64/
/usr/lib/x86_64-pc-linux-gnu/7.2.0/
/usr/lib/x86_64-linux-gnu/
/usr/lib/../lib64/
/tmp/XXX/lib/
/opt/gcc/7.2.0/lib/gcc/x86_64-pc-linux-gnu/7.2.0/../../../../x86_64-pc-linux-gnu/lib/
/opt/gcc/7.2.0/lib/gcc/x86_64-pc-linux-gnu/7.2.0/../../../
/lib/
/usr/lib/
In addition to the normal search priority - where LIBRARY_PATH comes before system paths, GCC prioritizes a few "prefixes", including ../lib64. This can be worked around by creating another symlink:
ln -s lib /tmp/XXX/lib64
I thought this was related to the --libdir parameter during configure, which I omitted and is /usr/lib in the system installation, but even if i specify --libdir=$PREFIX/lib --libexecdir=$PREFIX/lib, it prefers ../lib64.
How to compile or control gcc at runtime so that it at least uses the ../lib instead of ../lib64 postfix?
Clang
Clang is even more uncooperative. It does not include LIBRARY_PATH in the output for --print-search-dirs and does not even include a -L/tmp/XXX/lib to it's call to ld if the libfoo.so can be found already in /usr/lib.
How can I get Clang to prioritize my library path transparently?
Notes
Examples are from Archlinux, but I also tested under Ubuntu 16.04 which behaves similarly.
Related questions for GCC: Why does g++ look in LIBRARY_PATH/../lib64 and where is this documented? and g++ searches /lib/../lib/, then /lib/.
Overriding the search order with -L works, but is not transparent.
gcc --print-search-dirs lists more directories than gcc -v. The latter filters out non-exiting paths.
I found out the reason why a custom installation of GCC is different. Debian distribution patches the GCC makefiles, that's how it gets the right priority of LIBRARY_PATH. Before building GCC, find gcc/config/i386/t-linux64 and there change all MULTILIB_OSDIRNAMES to these lines:
MULTILIB_OSDIRNAMES = m64=../lib$(call if_multiarch,:x86_64-linux-gnu)
MULTILIB_OSDIRNAMES+= m32=../lib32$(call if_multiarch,:i386-linux-gnu)
MULTILIB_OSDIRNAMES+= mx32=../libx32$(call if_multiarch,:x86_64-linux-gnux32)
Also add --libexecdir=/your/custom/path/lib --libdir=/your/custom/path/lib to configure.
We have a project and shared libraries libprivate.so (private so) which was using old libraries libcurl.so.3. The system was upgraded with new system libraries libcurl.so.4.
For some internal issues, right now we do not want to make use of latest libraries libcurl.so.4, we want to make use of libcurl.so.3.
Hence I copied libcurl.so.3 in local folder and set LD_LIBRARY_PATH according. When I link my entire project it says that there is version conflict between libcurl.so.4 and libcurl.so.3 required libprivate.so (libprivate.so is compiled long time ago with libcurl.3.so).
Should I not worry about this warning and proceed further?
When I correctly specify LD_LIBRARY_PATH which has libcurl.so.3, why it is taking from system directory /usr/lib64/libcurl.so.4? when I do ldd my_binary, it takes from libcurl.so.4. How do I stop it? Specifying -L with specific location also doesn't work. Modiying /etc/ld.conf will do for the entire system. I want to make this when I ran my project.
Specifying explicit path it works like /home/mydir/libcurl.so.3, but I do not want to do it.
I want to have these conditions only when I execute my project. In other cases it can make use of latest libraries.
Thanks for your help
If the command you show in your comment is correct:
gcc test.c -L~/lib/x86_64/ -lcurl -o test
... then you need a space between -L and ~/lib/x86_64/ or the shell won't expand the ~, so the linker is not looking in the right directory.
So you need either:
gcc test.c -L ~/lib/x86_64/ -lcurl -o test
or:
gcc test.c -L$HOME/lib/x86_64/ -lcurl -o test
(You don't need a space here because variables are expanded anywhere in a word, but ~ is only expanded at the start of a word.)
The issue I'm having is that I need to compile my code using specific libraries that are in different path locations. I need to use -lncurses library from ./ramdisk/libs path, the problem is that this directory also cointains a version of lthr library that I don't want to be linked. The makefile is pulling both libraries from the same location which is not what I want. I can't change the contents of these library directories in the filesystem, so I need to find a way to tell the Makefile to link lncurses library from path A and link lthr library from path B instead using the lthr from path A.
Any suggestions?
CC=icc
NCE=-L./ramdisk/libs
CFLAGS+=-I$(ROOTDIR)/../../include
#LDFLAGS=-static -lthr
$(DESTDIR)/nce: nce
mkdir -p $(DESTDIR)
$(INSTALL) -m 777 nce $(DESTDIR)
nce: nce.c
$(CC) $(CFLAGS) nce.c $(LDFLAGS) -o nce -lthr $(NCE) -lncurses
You can (probably) bypass the search by giving the full path to the library archive. So instead of specifying -lncurses, you might try ./ramdisk/libs/libncurses.a (or whatever). You didn't specify whether it was a shared lib or not, and I'm not entirely sure that this works for shared libraries, but probably worth a try.
[edit]
Since this is a shared lib issues, maybe something like:
CC=icc
THR=/full/path/to/wherever/libthr/lives
NCE=/full/path/to/ramdisk/libs
CFLAGS+=-I$(ROOTDIR)/../../include
LDFLAGS=-static
nce: nce.c
$(CC) $(CFLAGS) nce.c $(LDFLAGS) -o nce -L$(THR) -W,-rpath=$(THR) -lthr -L$(NCE) -W,-rpath=$(NCE) -lncurses
I'm kind of shooting in the dark here as I'm not familiar with icc, but the idea is to make sure the linker puts thr's path on the runtime linker's search path before the one on the ramdisk so that thr gets found there first.
You could copy the remote library to a local working directory.
ncurses would source from one location while thr would source from another.