I have to compile a program on a current ubuntu (12.04). This program should then run on a cluster using CentOS with an older Kernel (2.6.18). I cannot compile on the cluster directly, unfortunately. If I just compile and copy the program without any changes I get the error message "kernel too old".
The way I understood it, the reason for this is not so much the Kernel version, but the version of libc that was used for compilation. So I tried to compile my program dynamically linking the libc from the cluster and statically linking everything else.
Research
There are already a lot of questions about this on SO but none of the answers really worked for me. So here is my research on that topic:
This question explains the reason for the Kernel too old message
This question is similar but more specialized and has no answers
Linking statically as proposed here didn't work because the libc is too old on the cluster. One answer also mentions to build using the old libc, but doesn't explain how to do this.
One way is to compile in a VM running an old OS. This worked but is complicated. I also read that you should not link libc statically
Apparently it is possible to compile for a different libc version with the option -rpath but this did not work for me (see below)
Current state
I copied the following files from the cluster into the directory /path/to/copied/libs
libc-2.5.so
libgcc_s.so.1
libstdc++.so.6
and am compiling with the options -nodefaultlibs -Xlinker -rpath=/path/to/copied/libs -Wl,-Bstatic,-lrt,-lboost_system,-lboost_filesystem -Wl,-Bdynamic,-lc,-lstdc++,-lgcc_s
The output of ldd on the compiled binary is
mybin: /path/to/copied/libs/libc.so.6: version `GLIBC_2.14' not found (required by mybin)
mybin: /path/to/copied/libs/libstdc++.so.6: version `GLIBCXX_3.4.15' not found (required by mybin)
linux-vdso.so.1 => (0x00007ffff36bb000)
libc.so.6 => /path/to/copied/libs/libc.so.6 (0x00007fbe3789a000)
libstdc++.so.6 => /path/to/copied/libs/libstdc++.so.6 (0x00007fbe37599000)
libgcc_s.so.1 => /path/to/copied/libs/libgcc_s.so.1 (0x00007fbe3738b000)
/lib64/ld-linux-x86-64.so.2 (0x00007fbe37bf3000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fbe37071000)
I'm somewhat confused by the error, because it uses the correct path (i.e. the libc from the cluster) but still complains about a missing glibc version. When running ldd on the cluster it returns not a dynamic executable and running the binary results in the same two errors mentioned above. It also looks like there are other libraries included (linux-vdso.so.1, ld-linux-x86-64.so.2 and libm.so.6). Should I use the older versions for those as well?
So now I have two main questions:
Is this even the correct approach here?
If yes: how do I link the old libc correctly?
See this answer.
Is this even the correct approach here
No: you can't use mismatched versions of glibc as your link command does. You used crt0.o and ld-linux.so from new (system-installed) libc, but libc.so.6 from an old (copied from cluster) libc. That is just not going to work.
-rpath sets the DT_RPATH tag but doesn't tell the linker to look there for libs, you want -L for that.
Related
My os is Kali, running GLIBC_2.32. I need to build an CGO application for a debian 10 system, which is running GLIBC_2.28.
If I go build with dynamic linking, it can't be run on the debian system, it shows GLIBC mismatch:
version `GLIBC_2.29` not found
version `GLIBCXX_3.4.29` not found
version `GLIBC_2.32` not found
So I tried static linking: CGO_LDFLAGS='-static' go build. A gui library uses OpenGL and it shows error:
# github.com/go-gl/gl/v3.2-core/gl
/usr/bin/ld: cannot find -lGL
After searching a while I found the libGL is related to the gpu driver and can't be statically linked.
Then I tried linking libGL.so dynamically and statically linking other libraries by:
CGO_LDFLAGS='-L/usr/lib/x86_64-linux-gnu -Bdynamic -lGL -static' go build
But same error: "cannot find -lGL"
I don't want to use docker, it's too heavy. And I don't think upgrading from debian 10 to 11 solves the problem, there maybe some other clients running different os in the future. What's the best solution?
Then I tried linking libGL.so dynamically and statically linking other libraries by:
CGO_LDFLAGS='-L/usr/lib/x86_64-linux-gnu -Bdynamic -lGL -static' go build
The -static flag tells the linker: perform a completely static link. It doesn't matter whether you put it before or after -lGL, the meaning is the same.
To link some libraries statically and link others dynamically, see this answer.
That said, what you are trying to do is impossible: if you have any dynamic libraries in your link, then libc.so.6 must also be dynamically linked.
I don't want to use docker, it's too heavy.
Too bad. You'll have to use docker, or set up a chroot environment, or build yourself a "Linux to older Linux" crosscompiler. The docker is likely easiest to implement.
I am trying to cross compile a small rust application for the RPI. I am cross compiling because compiling directly on the PI takes way too long and it hits 75C.
I followed various instructions, but what I ended up doing is this:
Install "armv7-unknown-linux-gnueabihf" target with rustup
Download rpi tools from here: https://github.com/raspberrypi/tools
Add the "tools/arm-bcm2708/arm-linux-gnueabihf/bin/" folder to PATH
Add ".cargo/config" file with:
[target.armv7-unknown-linux-gnueabihf]
linker = "arm-linux-gnueabihf-gcc"
run "cargo build --target armv7-unknown-linux-gnueabihf --release"
scp the file to the RPI
chmod +x the_file
do "./the_file"
I get bash: ./the_file: No such file or directory
Yes, I am indeed in the right directory.
So this is the output from "file":
ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), dynamically
linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 2.6.32,
with debug_info, not stripped
I'm not experienced enough with this sort of stuff to determine if the binary that I produced is suitable to be run on an RPI3 B.
Did I produce the correct "type" of binary?
P.S. I am running DietPi distro on the PI. It is based on debian if that's of any relevance.
So I solved this by cheating. I found https://github.com/rust-embedded/cross which took about 30 seconds to get going and now I can cross compile to pretty much anything. I highly recommend it!
The error message "No such file or directory" is not about the your executable but about the dynamic libraries linked to it which are missing from the target system.
To find out which libraries your executable needs you have to run the following command.
ldd /usr/bin/lsmem
This will output something like this
linux-vdso.so.1 (0x00007fffc87f1000)
libsmartcols.so.1 => /lib/x86_64-linux-gnu/libsmartcols.so.1 (0x00007fe82fe71000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fe82fc7f000)
/lib64/ld-linux-x86-64.so.2 (0x00007fe82fedd000)
Now you have to check that all this libraries are available on your system. rust-cross probably uses the correct linker for your target so that is probably the reason this works with it. To modify the linker see https://stackoverflow.com/a/57817848/5809980
I am using the devtoolset-1.0 for CentOS 6.3 in order to upgrade temporarily the GCC version. Although I am now able to compile my C++ application, the final binary is missing some symbols:
$ ldd -d -r myapp
$ [..]
$ libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x0000003216e00000)
$ [..]
$ undefined symbol: _ZNSt8__detail15_List_node_base11_M_transferEPS0_S1_ (./myapp)
$ undefined symbol: _ZNSt8__detail15_List_node_base7_M_hookEPS0_ (./myapp)
$ undefined symbol: _ZNSt8__detail15_List_node_base9_M_unhookEv (./myapp)
I figured out, that these are some new functions, which aren't found in the 'old' libstdc++, but in a newer libstdc++. On my system, both libstdc++ (default version 4.4.7) and devtoolset-1.0-libstdc++-devel (4.7 via devtoolset) are installed. Interestingly, the libstdc++ from devtoolset links agains to the old one:
$ cat /opt/centos/devtoolset-1.0/root/usr/lib/gcc/x86_64-redhat-linux/4.7.0/libstdc++.so
$ /* GNU ld script
$ Use the shared library, but some functions are only in
$ the static library, so try that secondarily. */
$ OUTPUT_FORMAT(elf64-x86-64)
$ INPUT ( /usr/lib64/libstdc++.so.6 -lstdc++_nonshared )
What I actually want is to replace the libstdc++ binding, but I don't know how to achieve that. I already tried to set LD_LIBRARY_PATH and pointing to the devtoolset directory, but the libstdc++ was still set to the old location. Also a symbolic link did not result in a success, because it is a ld script and not actual shared library.
the final binary is missing some symbols
That looks like a bug in devtoolset-1-gcc, which I assume has been fixed in more recent versions of devtoolset.
Interestingly, the libstdc++ from devtoolset links agains to the old one:
Yes, that's how the devtoolset gcc is supposed to work (see this answer for more details).
What I actually want is to replace the libstdc++ binding, but I don't know how to achieve that.
You can't do that with the devtoolset of GCC, because it doesn't even have a new libstdc++.so library. As you found, that file is actually a linker script, which links your binary to libstdc++_nonshared.a /usr/lib64/libstdc++.so
Since there is no new libstdc++.so you can't link to it.
The GCC compiler and its libraries (very specially g++ and corresponding libstdc++ runtime) need to match. Compiling with a newer compiler will (very often, practically guaranteed if a new version of the language is supported) give binaries that don't work with an older library. Older binaries might work with a newer library, no guarantees here.
I installed Matlab in my Linux Mint 14 Nadia (a uname -a shows: Linux Ideapad-Z570 3.5.0-17-generic #28-Ubuntu SMP Tue Oct 9 19:31:23 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux) and when calling it from the command line I would get a: "/lib64/libc.so not found".
I followed the help on mathworks by making a link in /lib64 as:
ln -s /lib/x86_64-linux-gnu/libc.so.6 .
That solved the issue.
Now, if I do a locate of this library I get:
locate "libc.so"
/lib/i386-linux-gnu/libc.so.6
/lib/x86_64-linux-gnu/libc.so.6
/usr/lib/x86_64-linux-gnu/libc.so
I will be compiling with gcc in this computer and I would like to have full 64bit compilations. What does exactly mean to have all these different libc.so libraries? which one will the gnu compiler be using? do I need to do anything different with gcc to compile for 64 bits?
I would also love to optimize as much as I can for my new i7 core!!!
/lib/i386-linux-gnu/libc.so.6
This is is 32-bit version of the library.
/lib/x86_64-linux-gnu/libc.so.6
This is the 64-bit version of the library.
Both are usually symbolic links to the actual library file, which will usually be named according to the glibc release number, for example libc-2.15.so
/usr/lib/x86_64-linux-gnu/libc.so
This is not a library, but a linker script file, which refers to the above symlinks.
Why do we need all these:
First, regardless of libc version installed, the linker will always search for libc.so, because the compiler driver will always pass to the linker the -lc options. The name libc stays the same and denotes to most recent version of the library.
The symlinks libc.so.6 are named after the soname of the library, which, more or less corresponds to the ABI version of the library. The executables, linked against libc.so in fact contain runtime dependencies on libc.so.6.
If we imagine the someday a grossly ABI incompatible libc is released, it's soname could be named libc.so.7, for example and this version coukld coexists with the older libc.so.6 version, thus executables linked against one or the other can coexist in the same system,
And finally, the name libc-2.15.so refers to the libc release, when you install a new libc package, the name will change to libc-2.16.so. Provided that it is binary compatible with the previous release, the libc.so.6 link will stay named that way and the existing executables will continue to work.
To find which one to use, you have to first find the order that ld (the linker) uses to find libraries, like so:
ld --verbose | grep SEARCH
For me it gave me this output:
SEARCH_DIR("/usr/x86_64-unknown-linux-gnu/lib64"); SEARCH_DIR("/usr/x86_64-unknown-linux-gnu/lib"); SEARCH_DIR("/usr/lib"); SEARCH_DIR("/usr/local/lib");
This means that on my computer, ld looks in these directories, in order:
/usr/x86_64-unknown-linux-gnu/lib64
/usr/x86_64-unknown-linux-gnu/lib
/usr/lib
/usr/local/lib
So if libc was in /usr/x86_64-unknown-linux-gnu/lib64, and libc was also in /usr/lib, it would use the /usr/x86_64-unknown-linux-gnu/lib64 version, because it was listed first.
The symlink you created will have no effect whatsoever on GCC. The 32-bit version is only used when you compile using the -m32 GCC flag. GCC will not attempt to generate 32-bit binaries unless you specifically tell it to (by using that flag.)
I have inherited a piece of software which is having some issues. I believe the issues are related to the version of libc that is being statically linked.
I am building this on a Windows XP machine, targeting an x86 QNX Neutrino 6.3.2 machine.
Previously, the software built with GCC 2.95.3 (Well, technically, it's QNX's QCC that wraps and calls GCC)
Someone added a feature and had to port it to build with GCC 3.3.5 because the new feature needed it.
Now, the software is mine. I need to make some additions but have noticed weird behavior. After some digging, I found that there are static links to both libc for 2.95.3 and 3.3.5. According to QNX's web site, :
GCC 2.95.3 (from 6.2.1 or 6.3) and GCC 3.3.5 use different C++ ABIs
and have different name mangling. As a result, you can't link C++
binaries (objects, executables, libraries) built with GCC 2.95.3 with
binaries built with GCC 3.3.5.
This is a breaking ABI change, so I am obviously concerned. I wrote a small test for this
#include <stdio.h>
int main()
{
FILE *stream_ptr = popen("fakename","r"); /// use libc
return 0;
}
and built it with 3.3.5:
QCC -V3.3.5,gcc_ntox86 small.cpp -o small.out
then used strings to see what has been statically linked for this program
strings -a small.out | grep GCC
GCC: (GNU) 3.3.5 (qnx-nto)
GCC: (GNU) 3.3.5 (qnx-nto)
GCC: (GNU) 2.95.3
GCC: (GNU) 3.3.5 (qnx-nto)
As you can see, libc for GCC 2.95.3 has been statically linked.
My first question is: How can I make this link with a 3.3.5 version of libc?
My second question is: Why does it link with 2.95.3 in the first place?
What am I doing wrong/missing? Any suggestions are welcome.
(There's probably 60 other things in the project linking with 2.95.3 objects, and I need to fix them all, so implementing popen() and 59 of his closest friends myself isn't the best of ideas...)
Thanks,
Karl
UPDATE:
So I haven't figured out how to fix this yet, but a little bit of background for QNX 6.3.2 so folks who stumble upon this later don't have to figure this out the hard way:
You can use the verbose option for the linker ld --verbose and have it spit out everything it does. Note that I got the following output when I did that:
attempt to open C:/QNX632/host/win32/x86/usr/lib/gcc-lib/i386-pc-nto-qnx6.3.0/3.3.5//libc.a failed
attempt to open C:/QNX632/target/qnx6/x86/lib/gcc/3.3.5/libc.a failed
attempt to open C:/QNX632/target/qnx6/usr/i386-pc-nto-qnx6.3.0/lib//libc.a failed
attempt to open C:/QNX632/target/qnx6/usr/lib/libc.a failed
attempt to open C:/QNX632/target/qnx6/x86/lib//libc.a succeeded
As one can see, the linker is attempting to open the 3.3.5 version of libc.a, but it's simply not there. I took a look at 3 other coworkers computers, and the 3.3.5 version of libc.a is not there. How this is even working across a breaking ABI change, I'm not sure, but I am suspicious that some of the wonkiness in this project has to do with this discrepancy.
While this answers my original questions,
1) You can't make it link with nonexistant libc.a files,
2) It picks the 2.95.3 version because the 3.3.5 version isn't there,
it brings up new questions:
3) Why doesn't QNX ship a 3.3.5 version of libc.a with this version of Momentics? (or if they do, where do they hide it because I missed it.)
4) Are there any viable workarounds? I was able to build everything but the two most important servers in the project without using libc, but until I get the last two fixed up, I'm still searching for a solution.
Update to the Update:
Working with the QNX folks, they built me an unsupported, untested engineering version of libc.a, libm.a and libsocket.a with GCC 3.3.5, and everything has been good since.
When I compile for QNX 6.3.2, I always use 3.3.5 with the GNU C/C++ libraries. If you don't specify GNU, you will get Dinkum libraries by default. I have had problems in the past with Dinkum thread safety. Try these flags:
qcc -V3.3.5,gcc_ntox86 -Y_gpp
The -Y_gpp directs qcc to use the GNU libraries.
GCC 3.3 is prehistoric, isn't there a newer version for QNX?
There should be some option for the compiler or linker which will tell it to be verbose, which you could use to see all the library paths and libraries being linked to. That might show you how an older lib is being linked to.
In case someone else runs into a similar problem, to the best of my knowledge, here are the answers to the four questions I asked. They are not encouraging.
1) You can't make it link with nonexistent libc.a files. Of course.
2) It picks the 2.95.3 version of libc.a because the 3.3.5 version isn't there.
3) During discussions with the QNX folks, they stated that for QNX Neutrino 6.3.2, the official, tested compiler is only 2.95.3, even though GCC 3.3.5 is included in the shipped version of Momentics, it is not tested nor supported. It just happens to be there.
4) Options:
a) Go to a newer version of QNX which uses a newer version of GCC
b) Get source for libc (and libm as it turns out) and build it with GCC 3.3.5.
This one may pan out. Still waiting on QNX tech support.
c) Get already-built libraries from the QNX folks.
d) Don't use GCC 3.3.5 to build for Neutrino 6.3.2
Sincerely,
Karl