I've created very minimal chroot environment on sdb and mounted it on /mnt/sdb. I've also created a symbolic link /mnt/sdb/bin/cc that points to /usr/bin/gcc.
ldd /mnt/sdb/bin/cc returned
linux-gate.so.1 => (0xb7829000)
libc.so.6 => /lib/i686/cmov/libc.so.6 (0xb76dd000)
/lib/ld-linux.so.2 (0xb782a000)
So I copied the necessary libraries by running:
cp /lib/i686/cmov/libc.so.6 /mnt/sdb/lib/i686/cmov/libc.so.6
cp /lib/ld-linux.so.2 /mnt/sdb/lib/ld-linux.so.2
Glancing through this article, I figured that since linux-gate.so.1 is a part of the kernel, I don't need to copy it over.
However, after I run chroot /mnt/sdb /bin/sh then try cc I get
cc: error while loading shared libraries: libm.so.6: cannot open shared object file: No such file or directory
How come ldd couldn't tell cc needed libm.so.6? Is there an easy way to get cc to work in the chrooted environment without simply copying over all the libraries? I'd just like to use cc temporarily so that I can build tcc with it, then build everything else with tcc (I've also tried simply building tcc outside then using it in chroot, but I'm afraid that that might deserve its own post).
Note:
I'm using Debian in Virtualbox, and the only program that currently runs in the chroot environment is a single (static) busybox binary.
Related
I'm very new to Yesod and I'm having trouble building Yesod statically
so I can deploy to Heroku.
I have changed the default .cabal file to reflect static compilation
if flag(production)
cpp-options: -DPRODUCTION
ghc-options: -Wall -threaded -O2 -static -optl-static
else
ghc-options: -Wall -threaded -O0
And it no longer builds. I get a whole bunch of warnings and then a
slew of undefined references like this:
Linking dist/build/personal-website/personal-website ...
/usr/lib/ghc-7.0.3/libHSrts_thr.a(Linker.thr_o): In function
`internal_dlopen':
Linker.c:(.text+0x407): warning: Using 'dlopen' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
/usr/lib/ghc-7.0.3/unix-2.4.2.0/libHSunix-2.4.2.0.a(HsUnix.o): In
function `__hsunix_getpwent':
HsUnix.c:(.text+0xa1): warning: Using 'getpwent' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
/usr/lib/ghc-7.0.3/unix-2.4.2.0/libHSunix-2.4.2.0.a(HsUnix.o): In
function `__hsunix_getpwnam_r':
HsUnix.c:(.text+0xb1): warning: Using 'getpwnam_r' in statically
linked applications requires at runtime the shared libraries from the
glibc version used for linking
/usr/lib/libpq.a(thread.o): In function `pqGetpwuid':
(.text+0x15): warning: Using 'getpwuid_r' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
/usr/lib/libpq.a(ip.o): In function `pg_getaddrinfo_all':
(.text+0x31): warning: Using 'getaddrinfo' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
/usr/lib/ghc-7.0.3/site-local/network-2.3.0.2/
libHSnetwork-2.3.0.2.a(BSD__63.o): In function `sD3z_info':
(.text+0xe4): warning: Using 'gethostbyname' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
/usr/lib/ghc-7.0.3/site-local/network-2.3.0.2/
libHSnetwork-2.3.0.2.a(BSD__164.o): In function `sFKc_info':
(.text+0x12d): warning: Using 'getprotobyname' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
/usr/lib/ghc-7.0.3/site-local/network-2.3.0.2/
libHSnetwork-2.3.0.2.a(BSD__155.o): In function `sFDs_info':
(.text+0x4c): warning: Using 'getservbyname' in statically linked
applications requires at runtime the shared libraries from the glibc
version used for linking
/usr/lib/libpq.a(fe-misc.o): In function `pqSocketCheck':
(.text+0xa2d): undefined reference to `SSL_pending'
/usr/lib/libpq.a(fe-secure.o): In function `SSLerrmessage':
(.text+0x31): undefined reference to `ERR_get_error'
/usr/lib/libpq.a(fe-secure.o): In function `SSLerrmessage':
(.text+0x41): undefined reference to `ERR_reason_error_string'
/usr/lib/libpq.a(fe-secure.o): In function `initialize_SSL':
(.text+0x2f8): undefined reference to `SSL_check_private_key'
/usr/lib/libpq.a(fe-secure.o): In function `initialize_SSL':
(.text+0x3c0): undefined reference to `SSL_CTX_load_verify_locations'
(... snip ...)
If I just compile with just -static and without -optl-static
everything builds fine but the application crashes when it tries to
start on Heroku.
2011-12-28T01:20:51+00:00 heroku[web.1]: Starting process with command
`./dist/build/personal-website/personal-website -p 41083`
2011-12-28T01:20:51+00:00 app[web.1]: ./dist/build/personal-website/
personal-website: error while loading shared libraries: libgmp.so.10:
cannot open shared object file: No such file or directory
2011-12-28T01:20:52+00:00 heroku[web.1]: State changed from starting
to crashed
I tried adding libgmp.so.10 to the LD_LIBRARY_PATH as suggested in here
and then got the following error:
2011-12-28T01:31:23+00:00 app[web.1]: ./dist/build/personal-website/
personal-website: /lib/libc.so.6: version `GLIBC_2.14' not found
(required by ./dist/build/personal-website/personal-website)
2011-12-28T01:31:23+00:00 app[web.1]: ./dist/build/personal-website/
personal-website: /lib/libc.so.6: version `GLIBC_2.14' not found
(required by /app/dist/build/personal-website/libgmp.so.10)
2011-12-28T01:31:25+00:00 heroku[web.1]: State changed from starting
to crashed
2011-12-28T01:31:25+00:00 heroku[web.1]: Process exited
It seems that the version of libc that I'm compiling against is
different. I tried also adding libc to the batch of libraries the
same way I did for libgmp but this results in a segmentation fault
when the application starts on the Heroku side.
Everything works fine on my PC. I'm running 64bit archlinux with ghc
7.0.3. The blog post on the official Yesod blog looked pretty easy
but I'm stumped at this point. Anyone have any ideas? If there's a way to get this thing working without building statically I'm open to that too.
EDIT
Per Employed Russians answer I did the following to fix this.
First created a new directory lib under the project directory and copied the missing shared libraries into it. You can get this information by running ldd path/to/executable and heroku run ldd path/to/executable and comparing the output.
I then did heroku config:add LD_LIBRARY_PATH=./lib so when the application is started the dynamic linker will look for libraries in the new lib directory.
Finally I created an ubuntu 11.10 virtual machine and built and deployed to Heroku from there, this has an old enough glibc that it works on the Heroku host.
Edit:
I've since written a tutorial on the Yesod wiki
I have no idea what Yesod is, but I know exactly what each of your other errors means.
First, you should not try to link statically. The warning you get is exactly right: if you link statically, and use one of the routines for which you are getting the warning, then you must arrange to run on a system with exactly the same version of libc.so.6 as the one you used at build time.
Contrary to popular belief, static linking produces less, not more, portable executables on Linux.
Your other (static) link errors are caused by missing libopenssl.a at link time.
But let's assume that you are going to go the "sane" route, and use dynamic linking.
For dynamic linking, Linux (and most other UNIXes) support backward compatibility: an old binary continues to work on newer systems. But they don't support forward compatibility (a binary built on a newer system will generally not run on an older one).
But that's what you are trying to do: you built on a system with glibc-2.14 (or newer), and you are running on a system with glibc-2.13 (or older).
The other thing you need to know is that glibc is composed of some 200+ binaries that must all match exactly. Two key binaries are /lib/ld-linux.so and /lib/libc.so.6 (but there are many more: libpthread.so.0, libnsl.so.1, etc. etc). If some of these binaries came from different versions of glibc, you usually get a crash. And that is exactly what you got, when you tried to place your glibc-2.14 libc.so.6 on the LD_LIBRARY_PATH -- it no longer matches the system /lib/ld-linux.
So what are the solutions? There are several possibilities (in increasing difficulty):
You could copy ld-2.14.so (the target of /lib/ld-linux symlink) to the target system, and invoke it explicitly:
/path/to/ld-2.14.so --library-path <whatever> /path/to/your/executable
This generally works, but can confuse an application that looks at argv[0], and breaks for applications that re-exec themselves.
You could build on an older system.
You could use appgcc (this option has disappeared, see this for description of what it used to be).
You could set up a chroot environment matching the target system, and build inside that chroot.
You could build yourself a Linux-to-olderLinux crosscompiler
You have several issues.
You should not build production binaries on bleeding edge distributions. The libraries on the production system will not be forward compatible.
You should not link glibc statically - it will always at runtime try to load additional libraries. For example cpu-based assembly. That is what your first warnings are about.
The last linker errors look like they are related to a missing openssl library on the command line.
But all in all - downgrade your distribution.
I had similar problems launching to Heroku (which uses glibc-2.11) where I had an application that required glibc-2.14, but I did not have access to the source and could not re-build it. I tried many things and nothing worked.
My workaround was to launch the service on Amazon Elastic Beanstalk and just provide an API interface.
I found the information provided useful as well, I think the various descriptions miss a critical issue I also ran into while forcing an updated version of Vagrant to start working again.
It's the dependency references internal to something like complicated installs, like Yesod to Heroku. Those interanl refences need to be preserved.
This is the script I wrote to make problems go away (at least, hopefully, for a little while):
#!/bin/bash
cd $HOME/
GLIBC_VERSION="2.17"
GLIBC_PREFIX="/usr/glibc/"
VAGRANT_VERSION="2.2.19"
# Install the basic build system utilities.
yum groupinstall -y "Development tools"
yum install -y curl patchelf
# Grab the tarball with the GNU libc source code.
curl -Lfo glibc-${GLIBC_VERSION}.tar.gz "https://ftp.gnu.org/gnu/glibc/glibc-${GLIBC_VERSION}.tar.gz"
echo "a3b2086d5414e602b4b3d5a8792213feb3be664ffc1efe783a829818d3fca37a glibc-${GLIBC_VERSION}.tar.gz" | sha256sum -c || exit 1
# Extract the secrets and get ready to rumble.
tar xzvf glibc-${GLIBC_VERSION}.tar.gz
# The configure script requrires an independent build directory.
mkdir -p glibc-build && cd glibc-build
# Configure glibc with a GLIBC_PREFIX so it doesn't conflict with distro libc files..
../glibc-${GLIBC_VERSION}/configure --prefix="${GLIBC_PREFIX}" --libdir="${GLIBC_PREFIX}/lib" \
--libexecdir="${GLIBC_PREFIX}/lib" --enable-multi-arch
# Compile and then install GNU libc.
make -j8 && make install
# Download and install Vagrant.
curl -Lfo vagrant_${VAGRANT_VERSION}_x86_64.rpm "https://releases.hashicorp.com/vagrant/${VAGRANT_VERSION}/vagrant_${VAGRANT_VERSION}_x86_64.rpm"
echo "990e8d2159032915f21c0f1ccdcbca1a394f7937e06e43dc1dabe605d208dc20 vagrant_${VAGRANT_VERSION}_x86_64.rpm" | sha256sum -c || exit 1
yum install -y vagrant_${VAGRANT_VERSION}_x86_64.rpm
# Patch the binaries and shared libraries inside the Vagrant directory, so they use the new version of GNU libc.
(find /opt/vagrant/ -type f -exec file {} \; )| grep "dynamically linked" | awk -F':' '{print $1}' | while read FILE ; do
patchelf --set-rpath /opt/vagrant/embedded/lib:/opt/vagrant/embedded/lib64:/usr/glibc/lib:/usr/lib64:/lib64:/lib --set-interpreter /usr/glibc/lib/ld-linux-x86-64.so.2 "${FILE}"
done
The script should be pretty easy to understand, and adapt easily to whatever MacGuffin you want to make work, provied you understand it.
The only tricky part is the rpath you pass to patchelf. Upi need to make sure you preserve the search paths, and precedence your software requires. Or you end up fixing one problem only to create another equally frustrating roadblock.
P.S. Don't forget the update the hashes for any file you down. In particular, you need to compile/install a different version of GNU libc, you will need to update that hash to match the version you want to use.
I am trying to cross compile a small rust application for the RPI. I am cross compiling because compiling directly on the PI takes way too long and it hits 75C.
I followed various instructions, but what I ended up doing is this:
Install "armv7-unknown-linux-gnueabihf" target with rustup
Download rpi tools from here: https://github.com/raspberrypi/tools
Add the "tools/arm-bcm2708/arm-linux-gnueabihf/bin/" folder to PATH
Add ".cargo/config" file with:
[target.armv7-unknown-linux-gnueabihf]
linker = "arm-linux-gnueabihf-gcc"
run "cargo build --target armv7-unknown-linux-gnueabihf --release"
scp the file to the RPI
chmod +x the_file
do "./the_file"
I get bash: ./the_file: No such file or directory
Yes, I am indeed in the right directory.
So this is the output from "file":
ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), dynamically
linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 2.6.32,
with debug_info, not stripped
I'm not experienced enough with this sort of stuff to determine if the binary that I produced is suitable to be run on an RPI3 B.
Did I produce the correct "type" of binary?
P.S. I am running DietPi distro on the PI. It is based on debian if that's of any relevance.
So I solved this by cheating. I found https://github.com/rust-embedded/cross which took about 30 seconds to get going and now I can cross compile to pretty much anything. I highly recommend it!
The error message "No such file or directory" is not about the your executable but about the dynamic libraries linked to it which are missing from the target system.
To find out which libraries your executable needs you have to run the following command.
ldd /usr/bin/lsmem
This will output something like this
linux-vdso.so.1 (0x00007fffc87f1000)
libsmartcols.so.1 => /lib/x86_64-linux-gnu/libsmartcols.so.1 (0x00007fe82fe71000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fe82fc7f000)
/lib64/ld-linux-x86-64.so.2 (0x00007fe82fedd000)
Now you have to check that all this libraries are available on your system. rust-cross probably uses the correct linker for your target so that is probably the reason this works with it. To modify the linker see https://stackoverflow.com/a/57817848/5809980
I'm trying to compile an example program that links to the shared library produced by Sundown. I'm compiling the program like so.
$ gcc -o sd sundown.c -L. -lsundown
Yet, when I run it I get the following error.
./sd: error while loading shared libraries: libsundown.so: cannot open shared object
file: No such file or directory
The output of ls is.
$ ls
libsundown.so libsundown.so.1 sundown.c sd
Why is the shared library not found by ld?
Short solution:
add . (or whatever it is from your -L flag) to your LD_LIBRARY_PATH. When you run sd, it'll look for libraries in the standard places and the LD_LIBRARY_PATH. Note that since you've added ., this will only work if you run sd from the same directory libsundown.so is in.
I plan on distributing the compiled binary. How can I do so that the library can be distributed without forcing people to edit their LD_LIBRARY_PATH?
You should install libsundown.so in one of the standard places, like /usr/lib or /usr/local/lib. You can do that with an installer or a make file, or something as simple as a INSTALL or README that tells the user to stick the libraries there and ensure the permissions are set to something sensible.
On Centos systems with /usr/lib and /usr/lib64, if you install 64-bit libs manually into /usr/lib then at runtime, the library may not be visible even though at build time it is visible (I used autotools and it was able to find my zopfli library from /usr/lib without any problem). When I execute the my_binary that links to /usr/lib/libzopfli.so.1 I got
libzopfli.so.1 => not found
After moving libzopfly.so.1 from /usr/lib to /usr/lib64, then everything works fine.
I'm currently switch from ubuntu 11.04 (gcc 4.5) to ubuntu 12.04 (but I have the same problem with ubuntu 11.10 too, gcc 4.6) and am not able to compile some of my projects anymore.
I have a library, lets call it liba, which I build myself so I get
liba.so.0.0.1 (real library)
liba.so.0 -> liba.so.0.0.1 (symbolic link)
liba.so -> liba.so.0.0.1 (symbolic link)
but when I try to build a program using liba with:
gcc -o myprogram myprogram.c -la
I get an error that my library can not be found:
/usr/bin/ld.bfd.real: cannot find -la
On the other hand if I delete liba.so and rename liba.so.0.0.1 to liba.so everything works fine. Alternatively calling gcc with the full shared library its file name works find:
gcc -o myprogram myprogram.c -l:liba.so.0.0.1
Unfortunately due to versioning reasons in my build system these two solutions are not desirable to me.
Ideas?
EDIT: nevermind I found the problem is with 'new' vmware not supporting symlinks in shared folders. USing NFS now and everything is fine.
Make sure the location where liba lives is mentioned in /etc/ld.so.conf and run ldconfig. Alternativly you can give the search path with -Lpath.
Suppose I have:
/usr/lib/libsomething.so.1 on machine A;
/usr/lib/libsomething.so.2 on machine B.
Both machines have /usr/lib/libsomething.so symlinking to their respective libs.
If I link using gcc with -lsomething (or even /usr/lib/libsomething.so) it will follow the symlink, and ldd on machine A produces something like:
libsomething.so.1 => /usr/lib/libsomething.so.1
This means it won't be able to find the library on machine B.
Now I know these are major version number changes and I know they may not be compatible, but I'm willing to take that risk. What I'd like to tell the linker is to look for libsomething.so, and don't follow the symlink so ldd will show
libsomething.so => /usr/lib/libsomething.so.1
on A but
libsomething.so => /usr/lib/libsomething.so.2
on B. And then the loader will follow the symlink to whatever version is there.
Also, I don't want delayed loading with dlopen or anything. I want it to link to the shared object at compile time.
Is this even possible?
Making executable which uses any available version of shared library is, of course, possible.
The problem was that you linked your executable to version-specific soname (libsomething.so.1 and libsomething.so.2). You should have done it with unversioned soname libsomething.so instead.
In order to achieve this, on the build machine you should compile and install library with soname (ELF SONAME) equal to libsomething.so (without version) so that linker can choose this soname while executable is built.
According to the Shared Libraries HOWTO, you can pass required unversioned soname while building the library:
gcc -shared -Wl,-soname,libsomething.so -o libsomething.so.X objectsomething.o
Then, as soon as you install the library and run ldconfig, you have:
symlink /lib/libsomething.so pointing to /lib/libsomething.so.1 on machine A;
symlink /lib/libsomething.so pointing to /lib/libsomething.so.2 on machine B.
The loader (run ldd) will choose unversioned symlinks regardless where it points to:
libsomething.so => /lib/libsomething.so (0xNNNNNNNN) on machine A;
libsomething.so => /lib/libsomething.so (0xNNNNNNNN) on machine B.
Linux dynamic loader (ld.so) resolves libraries based on their soname value written in the executable (ELF NEEDED). The value is copied from library file (ELF SONAME) while building the executable. As long as there is a symlink on the target system matching the soname recorded in the executable, the library pointed by this symlink will be loaded.
Let's run through your setup and show commands to verifing assumptions.
I used Fedora 18 X86_64 for the test and adjusted output to i686 for clarity.
Compile both libsomething.so.1 and libsomething.so.2. Make sure SONAME is set to unversioned libsomething.so:
readelf -a libsomething.so.1 | grep SONAME
0xNNNNNNNN (SONAME) Library soname: [libsomething.so]
readelf -a libsomething.so.2 | grep SONAME
0xNNNNNNNN (SONAME) Library soname: [libsomething.so]
Install the libraries into their respective machines under /lib/ directory. Run ldconfig -v on both machines and verify the output.
ldconfig -v 2>&1 | grep something
libsomething.so -> libsomething.so.1 (changed)
ldconfig -v 2>&1 | grep something
libsomething.so -> libsomething.so.2 (changed)
Compile executable and make sure that it refers to the same soname without version in NEEDED.
readelf -a executable | grep NEEDED
0xNNNNNNNN (NEEDED) Shared library: [libsomething.so]
You executable depends on unversioned libsomething.so now. Copy executable to both machines and run ldd against both copies.
ldd executable
libsomething.so => /lib/libsomething.so (0xNNNNNNNN)
The last output is the same on both machines as the executable was built with soname without version. This makes loader take unversioned symlinks on targets machines. And depending on the machine, the symlink can point to different implementation of the library libsomething.so.1 or libsomething.so.2.
This means it won't be able to find the library on machine B.
And it's not supposed to anyway.
By the very definition of soversions, libsomething.so.2 denotes that the API/ABI is incompatible to libsomething.so.1. Therefore, just adding libsomething.so in the program's table of libraries to be loaded would be factually wrong. The libsomething.so symlink merely serves as a hint to ld as to which soversion to pick by default.
Of whatever file ld actually ended up opening, it will take the DTNAME/SONAME field to encode in the program. If you don't want that, don't equip libsomething with a soname. But it can easily become pain... starting with running into unavailable symbols when trying to run the program.