When I enable Link Time Optimization in GCC, the binaries produced become much larger. I expected GCC to be able to remove a lot of reduntant functions, and perform other optimizations otherwise impossible, so how come this makes the generated output grow?
Compiler flags:
-Os -c -fno-builtin -ffunction-sections -fdata-sections -flto -mcpu=cortex-m0 -mthumb
Linker flags:
-nostdlib -s -Xlinker --gc-sections -flto -mcpu=cortex-m0 -mthumb -T
Did you strip the binary after linking with -flto (it should be provided both at compilation and at link time)?
BTW, notice that the same optimization flags (-flto -Os) should be passed both at compile time and at link time. If you forgot -like you did- any of them at linking phase, LTO won't work! (When using make you want CC=gcc -flto -Os not CFLAGS= -flto -Os).
You did forgot -Os at link time, in addition of -flto; just passing -flto at link time without any optimizations is wrong: the LTO phase would "de-optimize" at most
I know that -flto adds a lot of sections in the ELF object files and executables (those sections contain the serialization of GCC internal representations like Gimple....). I guess (but have not checked) that the linking does not remove them.
Also, the main point of LTO is to inline accross several compilation units and this is expected to grow the code. So perhaps you should not use LTO in your particular case.
Most redundant functions are already removed (by the linker "GC" on sections) even without LTO.
BTW, you could use objdumpor readelfto find out.
Important thing for LTO is the working linker plugin support. Only with that GCC can make agressive unreachable code removal and other optimizaitons. Be sure you have plugin enabled linker (just try if LTO linking works with -fuse-linker-plugin)
Related
I am utilizing yocto (dunfell) to cross-compile a project for multiple different architectures. Specifically, the targets I have are a 64-bit RaspberryPi4 (aarch64) and a 32-bit Orange Pi (armhf). My project that I am cross-compiling compiles and runs without issue when building for the raspi target; the runtime linker is properly set and things run without issue. However, whenever I build for the Orange Pi target, the program appears to compile without issue, but when I try to execute it on the platform, I get a "File not found" error.
This appears to be because the interpreter (runtime linker) is set to /usr/lib/ld.so which is not actually on the system. See below:
$ file my-exec
my-exec: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /usr/lib/ld.so.1, with debug_info, not stripped
In contrast, when I build the same program for the raspi target, the interpreter seems to be set properly for the system:
$ file my-exec
my-exec: ELF 64-bit LSB executable, ARM aarch64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-aarch64.so.1, with debug_info, not stripped
This issue seems to be isolated to just this project on the Orange Pi target. In comparison, other projects on the Orange Pi target look like the following and run without issue:
$ file other-exec
other-exec: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 3.2.0, stripped
What I am trying to understand is the following:
Which things could possibly be influencing the interpreter which is chosen at compile time? I have done some digging into this and nothing is explicitly set in my Makefile which could be causing this. Additionally, when I build a simple hello world application, the correct interpreter is chosen. It seems to be some set of flags in my compiling/linking which is causing this. Below is the compiling/linking output (I have excluded all the libraries it is also building which use the same arguments):
NOTE: make -j 24 i2c-core VERBOSE=1 CC=arm-poky-linux-gnueabi-gcc -march=armv7ve -mthumb -mfpu=neon -mfloat-abi=hard -fstack-protector-strong -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security -Werror=format-security --sysroot=/home/my-yocto-os/build/tmp/work/armv7vet2hf-neon-poky-linux-gnueabi/my-exec/1.0+git999-r0/recipe-sysroot LD=arm-poky-linux-gnueabi-ld --sysroot=/home/my-yocto-os/build/tmp/work/armv7vet2hf-neon-poky-linux-gnueabi/my-exec/1.0+git999-r0/recipe-sysroot
arm-poky-linux-gnueabi-gcc -march=armv7ve -mthumb -mfpu=neon -mfloat-abi=hard -fstack-protector-strong -D_FORTIFY_SOURCE=2 -Wformat -Wformat-security -Werror=format-security --sysroot=/home/my-yocto-os/build/tmp/work/armv7vet2hf-neon-poky-linux-gnueabi/my-exec/1.0+git999-r0/recipe-sysroot -I.. -I. -g -std=gnu99 -Wall -Wextra -pedantic -Wshadow -Wpointer-arith -Wcast-align -Wwrite-strings -Wmissing-prototypes -Wmissing-declarations -Wredundant-decls -Wnested-externs -Winline -Wno-long-long -Wconversion -Wstrict-prototypes -DBUILD_CORE -c -o build/other.o ../other.c
...
arm-poky-linux-gnueabi-ld --sysroot=/home/my-yocto-os/build/tmp/work/armv7vet2hf-neon-poky-linux-gnueabi/my-exec/1.0+git999-r0/recipe-sysroot build/my-exec.o build/other.o build/other1.o -luv -lsystemd -lc --entry main -o build/exe/my-exec
The linker which is being set (/usr/lib/ld.so.1) seems to be a really common linker to use, so why doesn't it actually end up on my system?
Am I fundamentally misunderstanding something about the ARM architecture which is resulting in this outcome? Are there different flags I could set at linking which would resolve this?
One further piece of information I have, which may or may not be relevant is the following:
An issue I hit while building for the Raspi target (showed up on both targets, actually) was the compiler wasn't able to find the entry symbol. This is quite a mature project and cross-compilers have been used to compile it, historically, but in a way that isn't yocto. This has never been an issue in the past. It wasn't until I added --entry main to the linker flags that this issue was resolved. I am wondering if there is a more fundamental problem which is resulting in these odd outcomes.
Thank you for any help you can provide on this. Please let me know if you have any additional questions about my environment.
After a few days of debugging, I figured out there problem. If anyone with more knowledge than I on linking would like to chime in to add things, please do. Ultimately, this was resolved by using gcc as the linker as opposed to using ld (the ones provided by yocto's cross compiler; i.e. aarch64-poky-linux-gcc).
In order to do this, I modified my recipe to pass in LD=${CC} LDFLAGS=${LDFLAGS} to my Makefile. Now, it builds and executes properly for both the RPi and OrangePi targets.
I believe this is mainly the case because the LDFLAGS provided by yocto actually can't be parsed by ld. From my research, it looks like ld is typically invoked by gcc. However, the flags still need to get to the complier. So, originally, LDFLAGS that needed to be passed into linking, weren't being passed in at all because I just assumed there was an error with doing it that way. So, be sure you're passing your LDFLAGS that yocto gives you into gcc.
I'm doing this in a makefile and it results on the first run creating the .gcda files in that dir; but as soon as i do the second, if find that the executable is almost as slow (and this is surely related), is still writing new files to the dir after compiled. From my understanding this shouldn't occur. Removing -fprofile-arcs (or -lgcov for that matter) makes the second compile complain about missing symbols. What amd i missing? I make clean in between both of these executions btw.
I also tried some variations besides -lgcovbut i reached this one from reading the manual and realizing -fprofile-use opens a lot of optimizations, including -fprofile-arcs and no easy alternative i tried was working.
PROFILE_DIR=/tmp/pgo/${PN}
ifeq ($(wildcard $(PROFILE_DIR)),)
all:
CXXFLAGS += -O3 -march=native -fprofile-generate=${PROFILE_DIR} -fprofile-correction
CFLAGS += -O3 -march=native -fprofile-generate=${PROFILE_DIR} -fprofile-correction
LDFLAGS += -fprofile-arcs
$(info profile-sampling build)
else
all:
CXXFLAGS += -O3 -march=native -fprofile-use=${PROFILE_DIR} -fprofile-correction
CFLAGS += -O3 -march=native -fprofile-use=${PROFILE_DIR} -fprofile-correction
LDFLAGS += -fprofile-arcs
$(info profile-guided build)
endif
gcc version is gcc (Ubuntu 9.2.1-9ubuntu2) 9.2.1 20191008
edit: the final LDFLAGS of the case with the files already is : -lpthread -ldl -lrt -fPIC -shared -Wl,--no-undefined -Wl,--version-script=link.T -fprofile-arcs
If i'm not mistaken,-Wl,--no-undefined is sabotaging not linking gcov or not using -fprofile-arcs because the symbols for gcov are somewhere in the files used even if they're no longer important and that option explicitly fails the compile on all missing symbols. So the solution might be omitting -fprofile-arcs in the second compile and (somehow) allowing just this single library gcov to have uninitialized symbols. I don't know how to try the second.
edit 2: unfortunately, no, it crashes at runtime without that flag and without the linking to -gcov (direct or indirect). So something in the .gcda files is forcing -lgcov and that forces the resulting executable to start writing .gcda files, but if you try to remove the -lgcov you either get a failed build or a crash at runtime. I don't understand how this is supposed to work, or why those files are not just a compile time artifact....
This was caused by 'make clean' not removing all created .o files but a outdated partial hardcoded list. The second compilation 'reused' some .o files from the first (instrumented) compilation, and thus required the -lgcov LDFLAG which naturally started the second compilation to profile again.
I was using the options:
-O3
-march=armv7-a
-mtune=cortex-a8
-ftree-vectorize
-mfloat-abi=softfp
-fsigned-char
-Wall
-save-temps
for cross compiling for arm-v7 (32 bit) using ArmLinuxToolChain.
What is the equivalent compiler options for gcc-linaro-aarch64-linux-gnu-4.8-2014.04_linux to build for armv8? I tried using those same options with -march=armv8-a
-mtune=cortex-a53 and got:
aarch64-linux-gnu-g++: error: unrecognized command line option '-mfloat-abi=softfp'
The GCC options beginning with -m are machine-dependent options, so the availability of -m* options varies between targets. This is one such case. There is no soft float ABI defined for Aarch64, so GCC does not provide the ARM-specific -mfloat-abi option.
If you simply remove -mfloat-abi=softfp then your problem should be solved.
I would suggest you investigate whether you need -save-temps, which is normally only used for debugging or reporting compiler problems.
Compiling / linking with -nostdlib seems to prevent static initialization, even if I add my own crti.s and crtn.s with .init/.fini sections.
Are there workarounds to make g++ generate static initialization code that is inserted in .init or that I can call manually?
This is what I tried:
g++ -o test.o -c -fno-use-cxa-atexit test.cc # has _start (entry point)
# that calls _init and _main
as -o crti.o crti.s # has _init in section .init
as -o crtn.o crtn.s
g++ -o test ./crti.o test.o -nodefaultlibs -nostartfiles ./crtn.o
-nodefaultlibs alone includes static initialization code and call, but forces use of libc-_start/_init.
-nodefaultlibs -nostartfiles allows me to use my own _start / _init, but does not include code or call to static initialization.
From gcc linker docs,
-nostdlib
Do not use the standard system startup files or libraries when
linking. No startup files and only the libraries you specify will be
passed to the linker, and options specifying linkage of the system
libraries, such as -static-libgcc or -shared-libgcc, are ignored.
Hence use,
-nodefaultlibs
Do not use the standard system libraries when linking. Only the libraries you specify will be passed to the linker, options specifying linkage of the system libraries, such as -static-libgcc or -shared-libgcc, will be ignored. The standard startup files are used normally, unless -nostartfiles is used. The compiler may generate calls to memcmp, memset, memcpy and memmove. These entries are usually resolved by entries in libc. These entry points should be supplied through some other mechanism when this option is specified.
Also try,
g++ -Wl, -static
-Wl passes the next command on to the linker
-static On systems that support dynamic linking, this prevents linking with
the shared libraries. On other systems, this option has no effect.
I've been trying to find the proper .a's and related flags for statically linking an app or SO under Linux. I know -static exists, but I can't use it as there's one specific SO I must link to.
To put it another way, I'm looking for the appropriate flags to statically link everything, except for a specific SO.
Thanks.
At my workplace we use -Bstatic and -Bdynamic but they are options to the linker ld. You can specify them with gcc using the -Wl option.
g++ -o app -Wl,-Bstatic -llib1 -llib2 -llib3 -Wl,-Bdynamic -llib4 app.o
Above shows command line for linking with lib1, lib2, and lib3 as static libraries and lib4 as a shared object library.