I want to cross-compile zsh for arm (an android device) but statically. I want the result to be a bunch of binaries that does not require a bunch of libs that android does not have. I don't care about the size of the binary.
I have compiled (statically) ncurses for android and I try to compile zsh:
ttouch zsh$ CFLAGS="-Wl,-static -static-libgcc -L/media/files/Lab/compilenv/ncurses-5.9/root/lib -lncurses" ./configure --host=arm-linux --disable-dynamic --disable-restricted-r --disable-gdbm --with-term-lib=ncurses --prefix=$(pwd)/root/
<everything ok>
ttouch zsh$ make -j16
<everything ok>
ttouch zsh$ readelf -d Src/zsh | grep NEEDED
0x00000001 (NEEDED) Shared library: [libncurses.so.5]
0x00000001 (NEEDED) Shared library: [librt.so.1]
0x00000001 (NEEDED) Shared library: [libm.so.6]
0x00000001 (NEEDED) Shared library: [libc.so.6]
0x00000001 (NEEDED) Shared library: [libgcc_s.so.1]
so, how can I compile zsh statically?
Related
I'm trying to build ffmpeg shared libraries for decoding video on Linux systems. The build is done on Arch Linux but the resulting shared libraries need to be as portable as possible.
Fortunately, the functionality built into ffmpeg is sufficient, so I don't want to link with any libraries that might or might not be present on the target system. I'm taking libvdpau as an example of an unwanted dependency here, but there are many more.
Here's what I tried:
$ git clone https://git.ffmpeg.org/ffmpeg.git --branch n4.3.1 --depth 1
$ cd ffmpeg
$ mkdir build
$ cd build
$ ../configure \
--enable-shared \
--disable-programs --disable-doc \
--disable-avdevice --disable-postproc --disable-avfilter \
--disable-autodetect
$ make -j9
Notice --disable-autodetect which should prevent autodetection of which dependencies happen to be present on the build system.
And yet, when I check the resulting libraries, all of them have dependencies that I didn't ask for. For example:
$ ldd libavcodec/libavcodec.so
linux-vdso.so.1 (0x00007ffcd73cd000)
libswresample.so.3 => /usr/lib/libswresample.so.3 (0x00007fba1e45c000)
libavutil.so.56 => /usr/lib/libavutil.so.56 (0x00007fba1e1a7000)
libm.so.6 => /usr/lib/libm.so.6 (0x00007fba1e061000)
libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007fba1e03f000)
libc.so.6 => /usr/lib/libc.so.6 (0x00007fba1de76000)
libsoxr.so.0 => /usr/lib/libsoxr.so.0 (0x00007fba1ddf4000)
libva-drm.so.2 => /usr/lib/libva-drm.so.2 (0x00007fba1dded000)
libva.so.2 => /usr/lib/libva.so.2 (0x00007fba1ddc0000)
libva-x11.so.2 => /usr/lib/libva-x11.so.2 (0x00007fba1ddb8000)
libvdpau.so.1 => /usr/lib/libvdpau.so.1 (0x00007fba1ddb3000) <- Why is this here?
libX11.so.6 => /usr/lib/libX11.so.6 (0x00007fba1dc72000)
libdrm.so.2 => /usr/lib/libdrm.so.2 (0x00007fba1dc5d000)
libmfx.so.1 => /usr/lib/libmfx.so.1 (0x00007fba1dc4d000)
libdl.so.2 => /usr/lib/libdl.so.2 (0x00007fba1dc47000)
/usr/lib64/ld-linux-x86-64.so.2 (0x00007fba1f971000)
libgomp.so.1 => /usr/lib/libgomp.so.1 (0x00007fba1dc06000)
libXext.so.6 => /usr/lib/libXext.so.6 (0x00007fba1dbf1000)
libXfixes.so.3 => /usr/lib/libXfixes.so.3 (0x00007fba1dbe8000)
libxcb.so.1 => /usr/lib/libxcb.so.1 (0x00007fba1dbbc000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007fba1d9df000)
libgcc_s.so.1 => /usr/lib/libgcc_s.so.1 (0x00007fba1d9c5000)
libXau.so.6 => /usr/lib/libXau.so.6 (0x00007fba1d9c0000)
libXdmcp.so.6 => /usr/lib/libXdmcp.so.6 (0x00007fba1d9b8000)
I also tried listing all disable flags explicitly, such as --disable-vdpau, but this didn't change anything; libvdpau.so.1 is still listed in ldd output.
Maybe they are just being linked, but not actually used? readelf -d seems to point in that direction:
$ ls */*.so
libavcodec/libavcodec.so libavutil/libavutil.so libswscale/libswscale.so
libavformat/libavformat.so libswresample/libswresample.so
$ readelf -d */*.so | grep 'Shared library:' | sort -u
0x0000000000000001 (NEEDED) Shared library: [libavcodec.so.58]
0x0000000000000001 (NEEDED) Shared library: [libavutil.so.56]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
0x0000000000000001 (NEEDED) Shared library: [libm.so.6]
0x0000000000000001 (NEEDED) Shared library: [libpthread.so.0]
0x0000000000000001 (NEEDED) Shared library: [libswresample.so.3]
This is a nice and tight set of dependencies, the way I actually want them to be.
So maybe the build system is stupid and always adds all detected libraries to the linker command line anyway? To suppress this, I tried adding --extra-ldflags=-Wl,--as-needed, but that doesn't seem to have any effect either.
The resulting linker command line looks like this (abridged):
$ make -n libavcodec/libavcodec.so.58
gcc \
-shared \
-Wl,-soname,libavcodec.so.58 \
-Wl,-Bsymbolic \
-Wl,--version-script,libavcodec/libavcodec.ver \
-Llibavcodec -Llibavdevice -Llibavfilter -Llibavformat -Llibavresample -Llibavutil -Llibpostproc -Llibswscale -Llibswresample \
-Wl,--as-needed \
-Wl,-z,noexecstack \
-Wl,--warn-common \
-Wl,-rpath-link=:libpostproc:libswresample:libswscale:libavfilter:libavdevice:libavformat:libavcodec:libavutil:libavresample \
-Wl,--as-needed \
-o libavcodec/libavcodec.so.58 \
[...all .o files...] \
-lswresample -lavutil -pthread -lm -lm -pthread -lm
It looks like -Wl,--as-needed was already added anyway.
Interestingly, -Llibavdevice is also listed, even though I passed --disable-avdevice --disable-postproc --disable-avfilter to the configure script. That might be a clue: it would pick up my system's libavdevice, which of course comes with far more dependencies. Still I'd expect -Wl,--as-needed to trim these unbuilt dependencies out as well.
Where do these dependencies come from, and how do I get rid of them?
Of course: when running ldd, any dependencies on other ffmpeg libraries are resolved to those installed on my system. And those have many more dependencies. So the problem wasn't with the build system, but rather with the way I was inspecting the resulting output.
Setting LD_LIBRARY_PATH reveals the true list of transitive dependencies:
$ LD_LIBRARY_PATH=libavcodec:libavformat:libavutil:libswresample:libswscale ldd libavcodec/libavcodec.so
linux-vdso.so.1 (0x00007fffaab0f000)
libswresample.so.3 => libswresample/libswresample.so.3 (0x00007f9a8f213000)
libavutil.so.56 => libavutil/libavutil.so.56 (0x00007f9a8ef6c000)
libm.so.6 => /usr/lib/libm.so.6 (0x00007f9a8ede0000)
libpthread.so.0 => /usr/lib/libpthread.so.0 (0x00007f9a8edbe000)
libc.so.6 => /usr/lib/libc.so.6 (0x00007f9a8ebf5000)
/usr/lib64/ld-linux-x86-64.so.2 (0x00007f9a906e1000)
I've built a simple program like this:
g++ application.cpp -o application.exe
and then executed the command;
ldd application.exe
...
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6
...
I want to list all the symbols of the libc library:
nm /lib/x86_64-linux-gnu/libc.so.6
nm: /lib/x86_64-linux-gnu/libc.so.6: no symbols
nm --defined-only /lib/x86_64-linux-gnu/libc.so.6
nm: /lib/x86_64-linux-gnu/libc.so.6: no symbols
Why nm reports no symbols? If libc.so.6 is not a library, but a some kind of a link to the actual library, then how can I find the actual library?
By default, nm reads the .symtab section in ELF objects, which is optional in non-relocatable objects. With the -D/--dynamic option, you can instruct nm to read the dynamic symbol table (which are the symbols actually used at run time). You may also want to use --with-symbol-versions because glibc uses symbol versioning extensively.
Alternatively, you can use eu-readelf --symbols=.dynsym or objdump -Tw. (readelf -sDW does not include symbol versioning information.)
I'm trying to compile my program to armv6, but the ELF file is built for armv7, even when I use -march=armv6 option. All .o files has been compiled to armv6, only the ELF file is wrong.
Basically I'm compiling like this:
arm-linux-gnueabihf-g++ -static -march=armv6 -mfpu=vfp -mfloat-abi=hard -marm -Wa,-march=armv6 -o "Bridge"
Checking all .o files I get this:
$ readelf -a -W Bridge.o | grep Tag
Tag_CPU_name: "6"
Tag_CPU_arch: v6
Tag_ARM_ISA_use: Yes
Tag_THUMB_ISA_use: Thumb-1
Tag_FP_arch: VFPv2
Tag_ABI_PCS_wchar_t: 4
Tag_ABI_FP_denormal: Needed
Tag_ABI_FP_exceptions: Needed
Tag_ABI_FP_number_model: IEEE 754
Tag_ABI_align_needed: 8-byte
Tag_ABI_align_preserved: 8-byte, except leaf SP
Tag_ABI_enum_size: int
Tag_ABI_VFP_args: VFP registers
Tag_ABI_optimization_goals: Aggressive Speed
Tag_CPU_unaligned_access: v6
But checking the ELF file:
$ readelf -a -W Bridge | grep Tag
Tag_CPU_name: "7-A"
Tag_CPU_arch: v7
Tag_CPU_arch_profile: Application
Tag_ARM_ISA_use: Yes
Tag_THUMB_ISA_use: Thumb-2
Tag_FP_arch: VFPv3
Tag_Advanced_SIMD_arch: NEONv1
Tag_ABI_PCS_wchar_t: 4
Tag_ABI_FP_rounding: Needed
Tag_ABI_FP_denormal: Needed
Tag_ABI_FP_exceptions: Needed
Tag_ABI_FP_number_model: IEEE 754
Tag_ABI_align_needed: 8-byte
Tag_ABI_align_preserved: 8-byte, except leaf SP
Tag_ABI_enum_size: int
Tag_ABI_VFP_args: VFP registers
Tag_CPU_unaligned_access: v6
I'm using Linaro GCC 7.2.1, and I've also tried with old versions, and different compilation flags combinations. Can someone tell me how to compile an armv6 ELF file?
Problem identified.
The Linaro toolchain binary was able to compile the object files for the armv6 platform, however, the linker actually did not interpret this flag.
Reason: the toolchain was compiled with the armv7 configuration.
Solution: I downloaded the source code for the Linaro toolchain, configured it to support armv6 and compiled it using the cross-ng-tool.
first I'll create a small program:
#include <gmpxx.h>
void function() {
//mpf_class num;
return;
}
int main() {}
Notice, that I am including a 3rd party library, but not using any of its contents.
I go to compile the program:
$ g++ main.cpp -Wl,--as-needed -lgmp -lgmpxx
$ readelf -d a.out | grep NEEDED
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
Even though I linked two additional libraries to this program, I passed the flag --as-nedded which discovered that those two libraries were not actually needed at run-time. Therefore, readelf shows us that they were not actually linked.
(gcc actually links additional c and c++ libraries to your binary besides libc, but the --as-needed flag discovered that those too, were not needed.)
Here I reference the library code, but still don't actually need it:
#include <gmpxx.h>
void function() {
mpf_class num;
return;
}
int main() {}
$ g++ main.cpp -Wl,--as-needed -lgmp -lgmpxx
$ readelf -d a.out | grep NEEDED
0x0000000000000001 (NEEDED) Shared library: [libgmp.so.10]
0x0000000000000001 (NEEDED) Shared library: [libstdc++.so.6]
0x0000000000000001 (NEEDED) Shared library: [libgcc_s.so.1]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
Question:
I was under the assumption that unused functions get thrown away, and don't become part of the binary. Assuming the compiler realizes that the "function" code is not needed, and can toss it, why can't it also realize that the linking of gmp is also no longer needed?
Interestingly, running the command with full optimization,
$ g++ -Ofast main.cpp -Wl,--as-needed -lgmp -lgmpxx
$ readelf -d a.out | grep NEEDED
0x0000000000000001 (NEEDED) Shared library: [libgmp.so.10]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
results in some of gcc's implicitly linked libraries to be excluded when previously unoptimized were kept due to gmp's inclusion.
Question:
Is the compiler capable of excluding that 3rd party library given that the library code will never actually be executed?
If the compiler is absolutely sure some code won't ever be used, it is allowed to just throw it away. If so, the call won't show up in the object, and the library ommited.
In the example function that isn't called, it is by default extern, so the compiler can't know if it is called from another file, and isn't allowed to get rid of it. But most compilers just compile a function at a time, and do little intra-function analysis. So I doubt any current compiler will compile that file (with a static void function()), notice the function isn't called, and delete it. Perhaps a smart linker (which does whole-program analysis) will do so, but I haven't looked too closely at that.
I'm using CMake in a project, and I'm trying to statically link some libraries.
I've set:
set(BUILD_SHARED_LIBS OFF)
set(CMAKE_EXE_LINKER_FLAGS "-static-libgcc -static-libstdc++ -static")
set_target_properties(icarus PROPERTIES LINK_SEARCH_END_STATIC 1)
And I've made sure when looking for the actual libraries that I have the *.a version of them.
Currently the project imports:
libPocoNet.a
libPocoUtil.a
libPocoXML.a
libPocoFoundation.a
libmysqlclient.a
libmysqlpp.a
libcrypto++.a
CUDA
All libraries are found, and when doing a dynamic/shared linkage, they work fine.
I have also tried to set compilation flags:
set(GCC_CXX_FLAGS ${GCC_CXX_FLAGS} "-static-libgcc -static-libstdc++ -static")
But to no avail.
While I get no issues while compiling, linkage is throwing alot of undefined reference errors for calls found in the above libraries, i.e:
undefined reference to `mysql_thread_init'
undefined reference to `mysql_real_query'
undefined reference to `pthread_mutex_unlock'
undefined reference to `Poco::ErrorHandler::handle()'
Not in that particular order, and numerous errors for each library.
Looking at the last line of GCC I see:
/usr/bin/c++ -g -g -static-libgcc -static-libstdc++ -static [list of *.cpp files]
-o icarus -rdynamic /usr/local/lib/libPocoFoundation.a /usr/local/lib/libPocoNet.a
/usr/local/lib/libPocoUtil.a /usr/local/lib/libPocoXML.a
-Wl,-Bstatic -lmysqlclient -lmysqlpp -lcrypto++
Which makes me wonder:
Why are Poco libraries linked as -rdynamic, and there is no -Wl -Bstatic flag? As if they are skipped/excluded from static linkage.
mysqlclient, mysqlpp and crypto++ seem to be set for static linkage, yet I still get errors
So, could someone please explain to me:
How do I setup for partial static linkage using CMake
Is CMAKE_EXE_LINKER_FLAGS the only one I need to set?
Should I be forcing static linking for mentioned libraries but not for entire project?
Please excuse me if those are too many or too localized questions, I haven't tried this before, and I can't seem to find much info on the net.
I've managed to solve my problem by using the following:
#Dynamic/Shared Libs
...
#Static start
set_target_properties(icarus PROPERTIES LINK_SEARCH_START_STATIC 1)
set_target_properties(icarus PROPERTIES LINK_SEARCH_END_STATIC 1)
set(CMAKE_FIND_LIBRARY_SUFFIXES ".a")
#Static Libs
...
#Set Linker flags
set(CMAKE_EXE_LINKER_FLAGS "-static-libgcc -static-libstdc++")
This works without passing a -static which creates other big issues, and can essentially mix static and dynamic libraries.
As long as the order of static libraries is correct, and as long as dependencies of static libraries are satisfied, I get an ELF using some dynamic libraries (i.e. in my case mysqlclient, libmysql++) and the rest are all static libraries (crypto++, PocoNet, PocoUtil, PocoXML, PocoFoundation).
Bear in mind that static linked libraries have their own dependencies. Examining my debug application using readelf -d app, I see:
Dynamic section at offset 0x508f88 contains 28 entries:
Tag Type Name/Value
0x0000000000000001 (NEEDED) Shared library: [libmysqlpp.so.3]
0x0000000000000001 (NEEDED) Shared library: [libmysqlclient.so.18]
0x0000000000000001 (NEEDED) Shared library: [libm.so.6]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
0x0000000000000001 (NEEDED) Shared library: [ld-linux-x86-64.so.2]
0x0000000000000001 (NEEDED) Shared library: [libpthread.so.0]
I know that pthread is imported by Poco::Runnable, libm is for math operations, etc.
I am still unaware if this is the right way to use CMake for partial static linking.
In the case of Debian packaged libraries, such as crypto++, mysql++, mysqlclient, simply finding the *.a library worked, but in case of Poco Libraries, which only got me the full path and name of the library, but not a flag, -Bdynamic could only be turned off by using the above lines.
Note: Poco could not be linked statically, without -static-libstdc++
I hope this helps anyone stuck at something similar.
How do I setup for static linkage using CMake
Well... you don't :) That's not how CMake works: in CMake, you first find the absolute path of a library, then link to it with target_link_libraries.
So, if you want to link to a static library, you need to search for that static library:
find_library(SOMELIB libsomelib.a)
instead of:
find_library(SOMELIB somelib)
For recently verions of cmake, it smart enough to link lib as static if you specify full file name of .a file.
for example, the following line
TARGET_LINK_LIBRARIES(pfs ... liburing.a ...)
it will generate compile cmd line:
ld ... -Wl,-Bstatic -luring -Wl,-Bdynamic ...
while cmd line the syntax is GCC's syntax to link against static lib. https://stackoverflow.com/a/4500201/388354