Use a static version of libc from an application that uses a dynamic shared library - static-libraries

I am trying to compile the following components:
appl - an application which links to a dynamic shared library (libwrapper.so)
libwrapper.so - A library used by appl. This library invokes functions from a libbackend archive.
libbackend.a - A static archive that uses libc functions.
I want to ensure that the libc functions invoked by libbackend always uses a fixed libc implementation. So I am also archiving the needed libc.a in libbackend, in my compilation environment.
I want to achieve that when libbackend invokes any libc function, the archived libc function from the compilation environment should be invoked, not the function from libc.so version in the target environment.
Can this be achieved? Also, can this be achieved without changing the way appl is being compiled; i.e., can be this achieved by changing makefile of only libwrapper and libbackend?
This is what I have tried so far, and does not work:
[dev-env]# ls
appl.c backend.c backend.h Makefile wrapper.c wrapper.h
[dev-env]# cat appl.c
#include <stdio.h>
#include "wrapper.h"
void main()
{
printf("in appl\n");
wrapper();
}
[dev-env]# cat wrapper.h
void wrapper();
[dev-env]# cat wrapper.c
#include <stdio.h>
#include "wrapper.h"
#include "backend.h"
void wrapper()
{
printf("In wrapper\n");
backend();
}
[dev-env]# cat backend.h
void backend();
[dev-env]# cat backend.c
#include <stdio.h>
#include <gnu/libc-version.h>
#include "backend.h"
void backend()
{
printf("in backend\n");
printf("GNU libc version: %s\n", gnu_get_libc_version());
}
[dev-env]# cat Makefile
LIBC=$(shell gcc --print-file-name=libc.a)
all: libbackend.a libwrapper.so appl
libbackend.a: backend.c backend.h
gcc -static -fPIC -c backend.c -o backend.o
ar rcs libbackend.a $(LIBC) backend.o
libwrapper.so: wrapper.c wrapper.h libbackend.a
gcc -c wrapper.c
gcc -shared -o libwrapper.so wrapper.o libbackend.a
appl: appl.c
gcc -o appl appl.c -L . -lwrapper
clean:
rm *.o *.a *.so appl
[dev-env]#
From compilation environment:
[dev-env]# make
gcc -static -fPIC -c backend.c -o backend.o
ar rcs libbackend.a /usr/lib/gcc/x86_64-linux-gnu/8/../../../x86_64-linux-gnu/libc.a backend.o
gcc -c wrapper.c
gcc -shared -o libwrapper.so wrapper.o libbackend.a
gcc -o appl appl.c -L . -lwrapper
[dev-env]# export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:.
[dev-env]# ./appl
in appl
In wrapper
in backend
GNU libc version: 2.28
In target environment:
[target]# ./appl
in appl
In wrapper
in backend
GNU libc version: 2.30
If what I am intending works, on target I should have seen the version output as "2.28".

Can this be achieved?
No.
This will lead to totally unpredictable behavior on any system with a different version of libc.so.6 installed. It's not something that you should even attempt.
I want to ensure that the libc functions invoked by libbackend always uses a fixed libc implementation.
Why? What problem are you imagining this will solve?
It would not solve anything, but is guaranteed to introduce new problems.

Related

Why can't I rename a shared library after it's been built?

Here's a simple example:
lib.c:
#include <stdio.h>
void hello_world(void) {
puts("Hello, world!");
}
program.c:
void hello_world(void);
int main() {
hello_world();
return 0;
}
If I compile the library normally and link it, everything works fine:
$ cc lib.c -fPIC -shared -o libmylib.dylib
$ cc program.c -L. -lmylib
$ ./a.out
Hello, world!
But if I rename the library after building it (but before building the program), it doesn't work:
$ cc lib.c -fPIC -shared -o totally_different_name.dylib
$ mv totally_different_name.dylib libmylib.dylib
$ cc program.c -L. -lmylib
$ ./a.out
dyld: Library not loaded: totally_different_name.dylib
Referenced from: /private/tmp/./a.out
Reason: image not found
[1] 13229 abort ./a.out
Why is this the case? Why is dyld looking for the original library name?
This works okay on Linux, but when I try on macOS it completely fails (I don't know if it's a macOS vs Linux thing, or a clang vs gcc thing, since I'm using clang on macOS and gcc on Linux).
If it's relevant, cc --version is Apple LLVM version 8.0.0 (clang-800.0.42.1) x86_64-apple-darwin16.4.0.
Thanks to Florian Zwoch's comment, I was able to find a way to fix this. Run this command after renaming the dylib:
install_name_tool -id libmylib.dylib libmylib.dylib
That command changes the internal ID of the dylib. The command syntax is install_name_tool -id <new-id> <dylib-path>.
Originally, the dylib has the ID of totally_different_name.dylib. Renaming the file doesn't change the ID it has stored internally, and that has to be updated separately with install_name_tool.

How to get gcc LTO work with library archives?

gcc experts,
I'm trying to use gcc lto with library archives, as the gcc comes with my system (RedHat Enterprise Linux 5.7) doesn't work with -flto (neither for my Ubuntu 14.10), so I build binutils && gcc from scratch.
Here is what I did:
1. Build binutils-2.22 with --enable-plugins
2. Build gcc-4.7.2 with --with-plugin-ld=/path/to/ld/built/in/step1 --enable-lto
3. Then for the following simple test:
// 1.c:
int foo(void)
{ return 0; }
// 2.c:
extern int foo(void)
int main(void)
{ return foo(); }
The following can get foo() inlined:
my_gcc -O3 -flto -c -o 1.o 1.c
my_gcc -O3 -flto -c -o 2.o 2.c
my_gcc -O3 -flto -o a.out 1.o 2.o
While the following can't:
my_gcc -O3 -flto -c -o 1.o 1.c
my_gcc -O3 -flto -c -o 2.o 2.c
my_ar cr --plugin <my_gcc>/libexec/gcc/x86_64-redhat-linux/4.7.2/liblto_plugin.so 1.a 1.o
my_ar cr --plugin <my_gcc>/libexec/gcc/x86_64-redhat-linux/4.7.2/liblto_plugin.so 2.a 2.o
gcc -O3 -flto -fuse-linker-plugin -o a.out 1.a 2.a
As the building system for the product I'm working on has to use archives, then what I can do to let lto work with library archive?
Your help will be much much appreciated.
Thanks a lot.
When linking, the order in which the libraries are listed on the command line, matters. So when compiling from the archives, you should swap 1.a and 2.a:
gcc -O3 -flto -fuse-linker-plugin -o a.out 2.a 1.a
I tested with gcc 4.9.2 and the disassembly, obtained with objdump -d a.out, shows that foo() is being inlined.

Convert .a to .dylib in Mac osx

Is it possible to convert .a files to .dylib files in Mac osx?
I currently have libraryname.a and it can't seem to include it in my program as only .dylib libraries are included.
Is there also a command that shows static libraries used in a program via mac osx terminal?
Yes, this is possible. To convert foo.a into libfoo.dylib, try this command:
clang -fpic -shared -Wl,-all_load foo.a -o libfoo.dylib
On Linux, here's the equivalent command using gcc:
gcc -fpic -shared -Wl,-whole-archive foo.a -Wl,-no-whole-archive -o foo.so
Here's a complete example.
Let's start by creating (and testing) libfoo.a:
$ cat > foo.h
int foo();
$ cat > foo.c
int foo() {
return 42;
}
$ cat > main.c
#include "foo.h"
int main() {
return foo();
}
$ clang -c foo.c -o foo.o
$ ar -r libfoo.a foo.o
ar: creating archive libfoo.a
$ clang libfoo.a main.c -o main.out
$ ./main.out; echo $?
42
Now let's convert it into libbar.dylib and test again:
$ clang -fpic -shared -Wl,-all_load libfoo.a -o libbar.dylib
$ clang -L. -lbar main.c -o main.out
$ ./main.out; echo $?
42

linking OpenMP statically with GCC

Given the following file print.cpp
#include <stdio.h>
int main() {
printf("asdf\n");
}
I can link this statically like this
g++ -static print.cpp
or like this
g++ -static-libgcc -Wl,-Bstatic -lc print.cpp -o print
But now let's add a little OpenMP and call the file print_omp.cpp
#include <omp.h>
#include <stdio.h>
int main() {
printf("%d\n", omp_get_num_threads());
}
I can link this statically like this (I checked it with ldd)
g++ -fopenmp -static print_omp.cpp
However, this does not work
g++ -fopenmp -static-libgcc -Wl,-Bstatic -lc print_omp.cpp -o print
I have tried various combinations of -Wl,--whole-archive -lpthread -Wl,--no-whole-archive and -lgomp -lpthread but no luck (I get various problems linking to pthreads). Can someone explain how I can do this without using the -static option?
GCC says
On glibc-based systems, OpenMP enabled applications cannot be statically linked due to limitations of the underlying pthreads-implementation
However, since g++ -fopenmp -static print_omp.cpp works just fine this does not make sense to me.
Edit:
I figured this out. The library GOMP comes with GCC whereas pthreads and libc come from GLIBC. So I can link GOMP statically like this
ln -s `g++ -print-file-name=libgomp.a`
g++ foo.cpp -static-libgcc -static-libstdc++ -L. -o foo -O3 -fopenmp
ldd shows
linux-vdso.so.1 => (0x00007fff71dbe000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fc231923000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fc23155c000)
/lib64/ld-linux-x86-64.so.2 (0x00007fc231b5c000)
However, if I the try this
ln -s `g++ -print-file-name=libpthread.a`
g++ foo.cpp -static-libgcc -static-libstdc++ -L. -o foo -O3 -fopenmp
It won't link. Pthreads and libc must be linked statically together. So once I add
ln -s `g++ -print-file-name=libc.a`
g++ foo.cpp -static-libgcc -static-libstdc++ -L. -o foo -O3 -fopenmp
ldd returns
not a dynamic executable
I really don't get why you may want to link only libgomp statically, but having separate compilation and linking commands may help. For instance assume main.cpp contains:
#include <omp.h>
#include <stdio.h>
int main() {
#pragma omp parallel
{
printf("%d\n", omp_get_thread_num());
}
}
Then:
~/tmp$ ls
main.cpp
~/tmp$ g++ -Wall -Werror -pedantic -fopenmp main.cpp -c
~/tmp$ ls
main.cpp main.o
~/tmp$ locate libgomp.a
${SOME_PATH_TO_LIBGOMP}/libgomp.a
~/tmp$ g++ -Wall -Werror -pedantic main.o -o main.x ${SOME_PATH_TO_LIBGOMP}/libgomp.a -pthread
~/tmp$ ls
main.cpp main.o main.x
~/tmp$ ldd main.x
linux-gate.so.1 => (0xb7747000)
libstdc++.so.6 => /production/install/gnu/compiler/gcc/lib/libstdc++.so.6 (0xb765c000)
libm.so.6 => /lib/i386-linux-gnu/libm.so.6 (0xb75fa000)
libgcc_s.so.1 => /production/install/gnu/compiler/gcc/lib/libgcc_s.so.1 (0xb75de000)
libpthread.so.0 => /lib/i386-linux-gnu/libpthread.so.0 (0xb75c2000)
libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0xb7413000)
/lib/ld-linux.so.2 (0xb7748000)
The most clean solution I've found for this is by modifying the libgomp.spec. Mine is at /usr/local/lib64/libgomp.spec. Change the content as follow:
*link_gomp: -l:libgomp.a %{static: -ldl }

Beginner's question, trying to understand how the linker searches for a static library

I have a working setup, where all files are in the same directory (Desktop). The Terminal output is like so:
$ gcc -c mymath.c
$ ar r mymath.a mymath.o
ar: creating archive mymath.a
$ ranlib mymath.a
$ gcc test.c mymath.a -o test
$ ./test
Hello World!
3.14
1.77
10.20
The files:
mymath.c:
float mysqrt(float n) {
return 10.2;
}
test.c:
#include <math.h>
#include <stdio.h>
#include "mymath.h"
main() {
printf("Hello World!\n");
float x = sqrt(M_PI);
printf("%3.2f\n", M_PI);
printf("%3.2f\n", sqrt(M_PI));
printf("%3.2f\n", mysqrt(M_PI));
return 0;
}
Now, I move the archive mymath.a into a subdirectory /temp. I haven't been able to get the linking to work:
$ gcc test.c mymath.a -o test -l/Users/telliott_admin/Desktop/temp/mymath.a
i686-apple-darwin10-gcc-4.2.1: mymath.a: No such file or directory
$ gcc test.c -o test -I/Users/telliott_admin/Desktop/temp -lmymath
ld: library not found for -lmymath
collect2: ld returned 1 exit status
What am I missing? What resources would you recommend?
Update: Thanks for your help. All answers were basically correct. I blogged about it here.
$ gcc test.c /Users/telliott_admin/Desktop/temp/mymath.a -o test
edit: gcc only needs the full path to the library for static libraries. You use -L to give a path where gcc should search in conjunction with -l.
To include the math libraries, use -lm, not -lmath. Also, you need to use -L with the subdirectory to include the library when linking (-I just includes the header for compiling).
You can compile and link with:
gcc test.c -o test -I/Users/telliott_admin/Desktop/temp /Users/telliott_admin/Desktop/temp/mymath.a
or with
gcc test.c -o test -I/Users/telliott_admin/Desktop/temp -L/Users/telliott_admin/Desktop/temp -lmymath
where mymath.a is renamed libmymath.a.
See link text for comments (search for "bad programming") on the practices of using -l:
In order for ld to find a library with -l, it must be named according to the pattern libyourname.a. Then you use -lmymath
So, there is no way to get it to take /temp/mymath.a with -l.
If you named it libmymath.a, then -L/temp -lmymath would find it.

Resources