gcc - A static library with undefined symbols? - gcc

I'm trying to build a project using a static library, so that the binary can be used even if the library isn't installed. However, I get lots of errors about undefined symbols when I try to do so.
Looking at the library, I see it has tons of undefined symbols, even though it's a .a static lib:
nm - u /usr/local/lib/libthis.a
....
U EVP_DigestFinal_ex
U EVP_DigestInit_ex
U EVP_DigestUpdate
U EVP_MD_CTX_cleanup
U EVP_MD_CTX_init
Those seem to be from openssl; others seem to be from libbzip2; etc.
Questions:
1. Why does the static (.a) lib have dependencies on shared objects (e.g. libopenssl) that aren't statically compiled?
2. How do I solve this? Trying to manually add -lssl doesn't seem to work. How do I get the binary to compile and not have external dependcies?

Why does the static (.a) lib have dependencies on shared objects (e.g. libopenssl) that aren't statically compiled?
Just about every static library that you can build will have unresolved symbols, e.g.
int my_open_for_read(const char *filename)
{
return open(filename, O_RDONLY); // unresolved reference to open
}
As Marc Glisse pointed out, this a plain unresolved symbol, not a dependency on libc.so.
How do I solve this?
There is no problem to solve here. When you link your binary, you get to decide which libraries to link statically, and which to link dynamically.
Trying to manually add -lssl doesn't seem to work.
This should work:
gcc main.o -lthis -lssl
Possibly you did something like
gcc main.o -lssl -lthis
which is wrong: the order of libraries on the link line matters.
How do I get the binary to compile and not have external dependcies?
Most OSes support using fully-static binaries. Generally this should not be your goal: it makes for less portable binaries, and their use is strongly discouraged.
If you really do want to produce a fully-static binary, link it with -static flag.
Why do you say full static is less portable?
Because they are.
if the user doesn't have the exact same build of the lib, the binary won't be portable with shared libs, but will be portable with static.
This is incorrect: most shared libraries support backward compatibility, e.g. libc.so.6 version 2.22 will happily run executables linked against version 2.3.6 from 10 years ago.
If you do ldd firefox
You need to pay attention to what you are doing:
file -L `which /usr/bin/firefox`
/usr/bin/firefox: POSIX shell script, ASCII text executable
If you look inside the shell script, you'll discover that it invokes /usr/lib/firefox/firefox, and that binary is dynamically linked:
ldd /usr/lib/firefox/firefox
linux-vdso.so.1 => (0x00007ffca278d000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f511731b000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f5117117000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f5116e13000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f5116b0d000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f51168f7000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f5116532000)
/lib64/ld-linux-x86-64.so.2 (0x00007f5117757000)

Related

Why does the program which is compiled against the installed glibc not run normally?

Thanks in advance.
my development environment:
$ cat /proc/version
Linux version 5.4.0-66-generic (buildd#lgw01-amd64-016) (gcc version 7.5.0 (Ubuntu 7.5.0-3ubuntu1~18.04)) #74~18.04.2-Ubuntu SMP Fri Feb 5 11:17:31 UTC 2021
$ ld --version
GNU ld (GNU Binutils for Ubuntu) 2.30
Copyright (C) 2018 Free Software Foundation, Inc.
$ getconf GNU_LIBC_VERSION
glibc 2.27
$ #my glibc source version is 2.32.9000-development
$ cat ./version.h
/* This file just defines the current version number of libc. */
#define RELEASE "development"
#define VERSION "2.32.9000"
For some reasons, I need to modify and test glibc. I follow the steps of this website(https://sourceware.org/glibc/wiki/Testing/Builds#Compile_against_glibc_in_an_installed_location) to modify glibc and write test programs.
compile glibc.(confgure and make)
install glibc.(make install to a directory)
...other steps in the website above.
I successfully modified some pthread functions and passed the test (the test program I wrote can compiled against the install glibc and ran successfully). ldd the program.
$ ldd ./exec/1-1.out
linux-vdso.so.1 (0x00007ffcbf367000)
libpthread.so.0 => /home/cjl-target/gnu/install/lib64/libpthread.so.0 (0x00007fcadcea9000)
libc.so.6 => /home/cjl-target/gnu/install/lib64/libc.so.6 (0x00007fcadcaed000)
/home/cjl-target/gnu/install/lib64/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2 (0x00007fcadd2ca000)
As shown above, the shared libraries that the program depends on all point to the glibc installation path.
But when I compiled message-queue's test program(test mq_unlink) and ran it, failed as bellow:
./exec/1-1.out: symbol lookup error: /lib/x86_64-linux-gnu/libpthread.so.0: undefined symbol: __libc_vfork, version GLIBC_PRIVATE
check the library that is depended by the program:
$ ldd ./exec/1-1.out
linux-vdso.so.1 (0x00007ffce3f72000)
librt.so.1 => /home/cjl-target/gnu/install/lib64/librt.so.1 (0x00007f0a389a2000)
libc.so.6 => /home/cjl-target/gnu/install/lib64/libc.so.6 (0x00007f0a385e6000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f0a383c7000)
/home/cjl-target/gnu/install/lib64/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2 (0x00007f0a38dac000)
As shown above, the shared libraries libpthread.so.0 points to the system library. Why?
my compile script is(from the website above):
# dobuild.sh
SYSROOT=/home/xxx/xxx/xxx #the glibc's installation path
(set -x; \
gcc \
-L${SYSROOT}/usr/lib64 \
-I${SYSROOT}/usr/include \
--sysroot=${SYSROOT} \
-Wl,-rpath=${SYSROOT}/lib64 \
-Wl,--dynamic-linker=${SYSROOT}/lib64/ld-linux-x86-64.so.2 \
-Wall $*
)
when I compile the pthread's test program:./dobuild 1-1.c -pthread -Wall
when I compile the mq's test program:./dobuild 1-1.c -lrt -Wall
In addition, it is confusing that when invoke the pthread_create in the mq_unlink's test program, compiling it ./dobuild 1-1.c -lrt -pthread, the ldd result shows that all dependent libraries point to the installed glibc.
I've tried multiple variations of this, but none of them seem to work. Any ideas?
First, you should stop using ldd -- in the presence of multiple GLIBCs on a host, ldd is more likely to mislead than to illuminate.
If you want to see which libraries are really loaded, do this instead:
LD_TRACE_LOADED_OBJECTS=1 ./exec/1-1.out
Second, you should almost never use $* in shell scripts. Use "$#" instead (note: quotes are important). See this answer.
Third, the behavior you are observing is easily explained. To understand it, you need to know the difference between DT_RPATH and DT_RUNPATH, described here.
You can verify that your binaries are currently using RUNPATH, like so:
readelf -d 1-1.out | grep 'R.*PATH'
And you can verify that everything starts working as you expect by adding -Wl,--disable-new-dtags to the link command (which would cause the binary to use RPATH instead).
To summarize:
RUNPATH affects the search for the binary itself, but not for any libraries the binary depends on.
RPATH affects the search path for the binary and all libraries it depends on.
with RUNPATH, expected libpthread.so.0 is found only when the binary depends on it directly, but not when the dependency on libpthread is indirect (via librt).
with RPATH, expected libpthread.so.0 is found regardless of whether the dependency is direct or indirect.
Update:
If I want to use DT_RUNPATH, how to set the library runpath for librt?
You would need to link librt.so with -rpath=${SYSROOT}/lib64.
You could edit the rt/Makefile, or build with:
make LDFLAGS-rt.so='-Wl,--enable-new-dtags,-z,nodelete,-rpath=${SYSROOT}/lib64'
You would need to do the same for any other library that may bring transitive dependency on other parts of GLIBC. I don't know of a general way to do this, but setitng LDFLAGS-lib.so='-Wl,-rpath=${SYSROOT}/lib64' and rebuilding everything might do the trick.

Go build with another glibc

I have installed another version of GLIBC and want to compile Golang code against this new GLIBC.
I have tried the following command for dynamic compilation:
go build --ldflags '-linkmode external -L /path/to/another_glibc/
But when I run ldd "go_executable", it still shows linked to default glibc.
Output:
linux-vdso.so.1 => (0x00007fff29da7000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f128a93c000)
/lib64/ld-linux-x86-64.so.2 (0x00007f128ad06000)
Expected Output:
linux-vdso.so.1 => (0x00007fff45fa7000)
libc.so.6 => /another_glibc/lib/libc.so.6 (0x00007f5cd2067000)
/another_glibc/ld-2.29.so => /lib64/ld-linux-x86-64.so.2 (0x00007f5cd2420000)
What is missing here?
This is not an answer to the question, just a warning:
If you, like me, came here because you were compiling to deploy on another machine and got "version `GLIBC_2.32' not found" (or similar), but you were not intentionally using CGo, stop here.
Go on Linux dynamically links C libraries to have faster and smaller builds, but it is able to supplement them for example when cross-compiling.
You can do export CGO_ENABLED=0 to disable CGo and get rid of the dependencies.
Before doing go build
Set
CGO_LDFLAGS
Dynamic:
export CGO_LDFLAGS="-Xlinker -rpath=/path/to/another_glibc/lib"
Static:
export CGO_LDFLAGS="-Xlinker -rpath=/path/to/another_glibc/lib -static"
CGO_LDFLAGS lets you set GCC-like ld flags for Go.
bitbyter's answer is not correct for the dynamic case because it requires that the system dynamic linker is compatible with the non-system glibc, which is unlikely. You can set the dynamic linker like this:
export CGO_LDFLAGS="-Xlinker -rpath=/path/to/another_glibc/lib64"
CGO_LDFLAGS="$CGO_LDFLAGS -Xlinker --dynamic-linker="/path/to/another_glibc/lib64/ld-linux-x86-64.so.2"
The dynamic linker name is specific to the architecture, so you have to research its name.

Can it be bad to link against shared libraries that are not being used?

Will gcc/ld really include the references to those shared libraries on the final executable, even if there's no reference to any function from those? If so, is there any potential problems that I may face by doing something like that?
Look at this test:
If you have
//file.c
int main(){
return 1;
}
And compile
gcc file.c -lm
Then
$ ldd a.out
linux-vdso.so.1 => (0x00007fff3ece6000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f1898e59000)
/lib64/ld-linux-x86-64.so.2 (0x00007f1899269000)
No link to mathlib
The issues you may have are larger linking times and the risk of an accidental symbol collision.
In addition to linkage times and possible symbol conflicts cited by #xvan, if you happen to build packages (such as RPMs), those extra linkages result in unnecessary dependencies, which result in extra packages being installed on the target system.
Further reading:
Automatic Dependencies (Maximum RPM: Taking the Red Hat Package Manager to the Limit)
Re: using "--nodeps" with rpmbuild is it possible?

Why does setting LD_LIBRARY_PATH change the shared library dependencies?

I have a simple hello_world.cpp program. I compiled it using g++4.4.7 on a CentOS 6.6 system. When I look at ldd a.out:
linux-vdso.so.1 => (0x00007fffbd79e000)
libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00002ab6f6819000)
libm.so.6 => /lib64/libm.so.6 (0x00002ab6f6b1f000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00002ab6f6da4000)
libc.so.6 => /lib64/libc.so.6 (0x00002ab6f6fba000)
/lib64/ld-linux-x86-64.so.2 (0x00002ab6f65f7000)
When I load a module for gcc-4.9.2, LD_LIBRARY_PATH is set to /path/to/gcc-4.9.2/lib64 and running ldd a.out yields :
linux-vdso.so.1 => (0x00007ffff9393000)
libstdc++.so.6 => /path/to/gcc-4.9.2/lib64/libstdc++.so.6 (0x00002b2b7c104000)
libm.so.6 => /lib64/libm.so.6 (0x00002b2b7c435000)
libgcc_s.so.1 => /path/to/gcc-4.9.2/lib64/libgcc_s.so.1 (0x00002b2b7c6b9000)
libc.so.6 => /lib64/libc.so.6 (0x00002b2b7c8cf000)
/lib64/ld-linux-x86-64.so.2 (0x00002b2b7bee2000)
QUESTION : Why is the 4.9.2 version of the gcc libraries used when LD_LIBRARY_PATH is set, even though I compiled with with 4.4.7?
This seems to pose a problem of knowing which version of a library is being used. A user may compile a program with one compiler version, load a different compiler versions (via module) and then run the executable which uses a different library version than expected.
I don't know whether you're aware that the use of Environment Modules
is not a necessary or normal part of using GCC, or even using multiple versions of GCC.
Setting LD_LIBRARY_PATH to override the system dynamic linker's default directory search order
is a practice that has been long and loudly censured, for reasons that include the
one expressed in the last paragraph of your post. See e.g. Gurus say that LD_LIBRARY_PATH is bad
Doing so within environment modules in a system tailored for the use of multiple versions of tools or toolchains
can be regarded as a regulated application of a hazardous practice. In that context, however, the
setting of LD_LIBRARY_PATH in your gcc-4.9.2 environment module is doing exactly what
such a setting is supposed to do: overriding the
dynamic linker's default directory search order. See the documentation: 3.3.1.
In principle it is possible that a program compiled and linked with an older or
later version of GCC would behave unexpectedly if you choose to run it in the
environment of the gcc-4.9.2 module. Environment modules are a niche convenience,
and they come with this risk. Although if you built the program in the native
environment, or specifically in a module-enabled environment for gcc 4.4.7, then
the risk attendant on running it in your gcc-4.9.2 environment is presumably one
that you don't have to take.

Compile with older libc (version `GLIBC_2.14' not found)

I have to compile a program on a current ubuntu (12.04). This program should then run on a cluster using CentOS with an older Kernel (2.6.18). I cannot compile on the cluster directly, unfortunately. If I just compile and copy the program without any changes I get the error message "kernel too old".
The way I understood it, the reason for this is not so much the Kernel version, but the version of libc that was used for compilation. So I tried to compile my program dynamically linking the libc from the cluster and statically linking everything else.
Research
There are already a lot of questions about this on SO but none of the answers really worked for me. So here is my research on that topic:
This question explains the reason for the Kernel too old message
This question is similar but more specialized and has no answers
Linking statically as proposed here didn't work because the libc is too old on the cluster. One answer also mentions to build using the old libc, but doesn't explain how to do this.
One way is to compile in a VM running an old OS. This worked but is complicated. I also read that you should not link libc statically
Apparently it is possible to compile for a different libc version with the option -rpath but this did not work for me (see below)
Current state
I copied the following files from the cluster into the directory /path/to/copied/libs
libc-2.5.so
libgcc_s.so.1
libstdc++.so.6
and am compiling with the options -nodefaultlibs -Xlinker -rpath=/path/to/copied/libs -Wl,-Bstatic,-lrt,-lboost_system,-lboost_filesystem -Wl,-Bdynamic,-lc,-lstdc++,-lgcc_s
The output of ldd on the compiled binary is
mybin: /path/to/copied/libs/libc.so.6: version `GLIBC_2.14' not found (required by mybin)
mybin: /path/to/copied/libs/libstdc++.so.6: version `GLIBCXX_3.4.15' not found (required by mybin)
linux-vdso.so.1 => (0x00007ffff36bb000)
libc.so.6 => /path/to/copied/libs/libc.so.6 (0x00007fbe3789a000)
libstdc++.so.6 => /path/to/copied/libs/libstdc++.so.6 (0x00007fbe37599000)
libgcc_s.so.1 => /path/to/copied/libs/libgcc_s.so.1 (0x00007fbe3738b000)
/lib64/ld-linux-x86-64.so.2 (0x00007fbe37bf3000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fbe37071000)
I'm somewhat confused by the error, because it uses the correct path (i.e. the libc from the cluster) but still complains about a missing glibc version. When running ldd on the cluster it returns not a dynamic executable and running the binary results in the same two errors mentioned above. It also looks like there are other libraries included (linux-vdso.so.1, ld-linux-x86-64.so.2 and libm.so.6). Should I use the older versions for those as well?
So now I have two main questions:
Is this even the correct approach here?
If yes: how do I link the old libc correctly?
See this answer.
Is this even the correct approach here
No: you can't use mismatched versions of glibc as your link command does. You used crt0.o and ld-linux.so from new (system-installed) libc, but libc.so.6 from an old (copied from cluster) libc. That is just not going to work.
-rpath sets the DT_RPATH tag but doesn't tell the linker to look there for libs, you want -L for that.

Resources