To be able to debug and fuzz a whole Linux distribution, I would like to set ASAN (AddressSanitizer, https://en.wikipedia.org/wiki/AddressSanitizer) as default option to gcc. So normally to achieve what I want, generally, I set the following variables before to compile a linux package:
CFLAGS="-fsanitize=address,undefined -Wformat -Werror=format-security -Werror=array-bounds -g"
CXXFLAGS="-fsanitize=address,undefined -Wformat -Werror=format-security -Werror=array-bounds -g"
LDFLAGS="-fsanitize=address,undefined"
and try to compile and run my code. I would like to have it default to gcc.
One option to do it is using spec files: https://gcc.gnu.org/onlinedocs/gcc/Spec-Files.html. However I didn't find a way to set a "catch all rules" to compile and link all my c/c++ code with AddressSanitizer.
My questions are:
Any example how to do it using spec files?
Is that the best approach to do it?
Any other alternative approach?
First of all, be sure to take a look at existing whole-distro Asan enablings in Tizen (also here) and Gentoo.
In general there are two main approaches:
customize your build system to enable Asan by default, usually using CFLAGS and CXXFLAGS; this won't always work because many packages ignore them (I think that's what Hanno Boeck did in Gentoo)
replace /usr/bin/gcc, /usr/bin/g++ and /usr/bin/cc (and may x86_64-linux-gnu-gcc, x86_64-linux-gnu-g++) with wrappers which would add Asan flags and redirect calls to original executables (this is the approach we eventually took in Tizen and found it very successful)
As a side note, I'd suggest to add the following options
CFLAGS += -fsanitize-recover=address,undefined
otherwise boot will fail at too early stages. Also look at suggested settings ASAN_OPTIONS in above links, it took people long time to figure them out.
Related
I am now a sophomore in university and just started learning compiler few months ago. I am wondering what tool will make it easier if I want to enable -O3 as default option instead of -O1.
To change the global compiler defaults for all projects, you will have to install a new spec file in the place where your GCC installation expects it (for example, /usr/lib/gcc/x86_64-linux-gnu/6/specs). See Spec Files in the GCC documentation. This might work:
*cc1_options:
+ %{!O*:-O3}
*cc1plus_options:
+ %{!O*:-O3}
It enables -O3 if no -O option is set.
Note that this risks breaking quite a few build systems because few of them will expect that -O3 is enabled by default.
I just got this error when compiling some plugins for my new toy (on gcc/g++ on Linux):
relocation R_X86_64_32 can not be used when making a shared object; recompile with -fPIC
I basically understand why PIC is needed but, within the CMake system, the solution seems to be this:
IF (CMAKE_SYSTEM_PROCESSOR STREQUAL "x86_64")
SET_TARGET_PROPERTIES (${PLUGIN_BASE_LIB} PROPERTIES COMPILE_FLAGS "-fPIC")
ENDIF (CMAKE_SYSTEM_PROCESSOR STREQUAL "x86_64")
I don't understand why this solution is conditional.
The follow-up seems to suggest that -fPIC should be used basically everywhere except 32-bit Linux, which suggests that the above is not portable.
Should I always use -fPIC? Will there be any adverse effects?
${PLUGIN_BASE_LIB} needs to be statically linked to both the main executable, and statically the various shared libraries which are the plugins.
Ideally you want to build two versions of code: one for the main executable and one for the library. The first will need to be compiled with -fPIE (which is default in modern distros) and the second with -fPIC. As you point out this does not depend on target architecture.
You can compile only one version with -fPIC but then main executable will be suboptimal because -fPIC forces compiler to obey symbol interposition rules which significantly limits it's ability to optimize code e.g. inline and clone functions.
I'm interested in answers, approaches, and ideas out of the box. At a high level, the main page is pretty sparse and they mainly list -g, with one level, suggesting that -O0 is also either very helpful or essential.
But I'm wondering what other clang flags can be given to give maximum debugging. Is there an equivalent to gcc's -ggdb3 which includes some of the source or annotations directly in the object output? Or could there be? Is it possible and helpful to recompile the OS and its original libraries to have debugging (and if so, if I'm using Debian, can I have it write the debugging into the main .deb package instead of putting a separate debugged .deb package which stores debugging data in /usr/lib/debug?)? Will a static build of a binary affect the ability to see a good stacktrace? And is there anything that needs to be done to ensure that addr2line works well? Is it needed to compile all libraries (even glibc) with clang to get the maximum debugging benefit? I note that there is a project to recompile Debian with clang, and otherwise am open to a distribution that does so or otherwise places emphasis on debugging.
On Linux there are also options like an LD_PRELOAD set to /lib/libSegFault.so, or a set of LD_LIBRARY_PATH reassignments to /usr/lib/debug instead of the usual /usr/lib location (including redirecting libc itself to the debugged version). Is there a central place or external sources for answers to this question of how to enhance debuggability of a binary? The bigger mystery is clang, since I see in the long gcc man page that there are various options which can increase debugging (or reduce optimisations), but on the other hand the documentation for clang only shows a smaller set. It's possible that clang will accept more options than given, including gcc flags (which may either translate to a no-op or to more debugging - hard to tell without a canonical source of information).
Also from a package build perspective, since an external package may not respect CFLAGS, I've redirected /usr/bin/strip to be a no-op command that always succeeds, but other ideas on ensuring compliance are suggested (I believe that pkgsrc does a good job of wrapping gcc and the linker in shell scripts - useful to insert mandatory flags). Also there may be various ld options that can be passed to increase debugging of the outputted target. Also, it's quite possible that BSD (including FreeBSD 10, based upon clang) may have a different linking architecture which could make it easier to request and find debugged symbols in the generated libraries and executables.
To take debugging more broadly defined, I've set LD_WARN=yes, LD_DEBUG=unused, SEGFAULT_SIGNALS="all", LD_PRELOAD=.../libSegFault.so (as mentioned above), and LD_BIND_NOW=yes. Also I believe I can prefer that gcc search for libraries in /usr/lib/debug - above the standard search paths using strategic -Bs. Also, using --whole-archive for a static build might ensure that more objects are included in the linked output. There's also ulimit -c unlimited, and on Linux a nice way to differentiate core files like:
sysctl -w kernel.core_pattern="core.%t.SIG-%s.PID-%p.ID-%g-%u.%h.%E"
For gcc I've used and seen flags like: -O0 -fno-omit-frame-pointer -fverbose-asm -ggdb3 -mno-omit-leaf-frame-pointer -mtune=generic -fvar-tracking -D_GLIBCXX_DEBUG=1 -frecord-gcc-switches -femit-class-debug-always -fmath-errno -fno-eliminate-unused-debug-symbols -fno-eliminate-unused-debug-types -fno-merge-debug-strings -mieee-fp -mtune=generic -static-libgcc -fexceptions -fvar-tracking -fbounds-check -rdynamic -UNDEBUG -DDEBUG=1 (-ffreestanding -static-libgcc -pass-exit-codes) -fno-stack-check (since I believe I've read that the latter can interfere with debugging)
Other flags are there for other reasons but the emphasis is to be on maximum debugging. With all or most of the above, it's unclear to what extent clang would support or use there, or whether there are other options.
Clang does not support the -ggdb3 flag, only -g, as you have noticed. If you try to use it, you'll get the message:
clang: warning: argument unused during compilation: '-ggdb3'
so you can run your entire command line through Clang and it will tell you which of those GCC flags it supports and which it does not, some will print warnings, others may error out, but Clang will not silently ignore them. Here are the ones that Clang rejected when I tried your long command: -static-libgcc and -pass-exit-codes.
As pointed out in another SO answer, clang -cc1 --help can be used to list supported compilation flags, where we see the following which may be of interest to you:
-disable-llvm-optzns: Don't run LLVM optimization passes
-fno-elide-constructors: Disable C++ copy constructor elision
-mdisable-fp-elim: Disable frame pointer elimination optimization
The code I'm working on is supposed to be possible to build for both hosted and freestanding environments, providing private implementations for some stdlib functions for the latter case.
Can I reliably test this with just GCC on a normal workstation/build server? Compile for freestanding environment with GCC
The "-ffreestanding" option looked promising, but it seems that it "only" disables built-ins and sets the STDC_HOSTED macro properly, it still provides all system headers.
The option "-nostdinc" is too restrictive; I still want to use the headers required for a freestanding implementation (in particular stddef.h and limits.h).
What am I missing here?
Oh, and I'm using GCC 4.4.3 for the moment, will upgrade to 4.5.0 "soon".
Well, since no answer is given yet I'd might as well describe how I made this work. It's pretty simple although depending on the target system it can be tedious.
Using "-nostdinc" means that the standard system include paths will be skipped; other include-paths given with "-I" will of course still be searched for headers.
So, for the freestanding build target I create a folder 'include-freestanding-c89' and link the relevant system headers -- float.h, iso646.h, limits.h, stdarg.h and stddef.h -- there. Other headers might be included in these, depending on your platform, so you might have to do some research and set up more links (hence the tediousness if you need to do this for several target platforms).
The C89 directory can then be used as base for 'include-freestanding-c99', the extra headers to link are stdbool.h and stdint.h
The command-line to use is then
gcc -std=c89 -nostdinc -nostdlib -ffreestanding -I include-freestanding-c89
or
gcc -std=c99 -nostdinc -nostdlib -ffreestanding -I include-freestanding-c99
This Xen Makefile uses gcc -print-search-dirs to get the directory with stddef.h and similar, adds it with -isystem, then uses -nostdinc to build:
https://github.com/mirage/xen/blob/2676bc915157ab474ee478d929b0928cf696b385/stubdom/Makefile#L35
I am trying to build a pass using llvm and I have finished building llvm and its associated components. However, when I run make after following all the steps to build a pass including the makefile, I get the following
relocation R_X86_64_32 against `a local symbol' can not be used when making a shared object; recompile with -fPIC
After tyring to find a fix by googling the error message, I came to know that this is not specific to llvm. A few solutions suggested that I should use "--enable-shared" while running configure but that didn't help my case. Now I want to re-build llvm using fPIC, as the error says. But how do I do this using the makefile?
Looks like you could add the -fPIC (for position-independent code, something you want for a shared library that could be loaded at any address) by setting shell variables:
export CFLAGS="$CFLAGS -fPIC"
export CXXFLAGS="$CXXFLAGS -fPIC"
Looking at Makefile.rules, these will be picked up and used. Seems strange that it wasn't there to begin with.
EDIT:
Actually, reading more in the makefiles, I found this link to the LLVM Makefile Guide. From Makefile.rules, setting either SHARED_LIBRARY=1 or LOADABLE_MODULE=1 (which implies SHARED_LIBRARY) in Makefile will put -fPIC in the compiler flags.
If you are moderately convinced that you should use '-fPIC' everywhere (or '-m32' or '-m64', which I need more frequently), then you can use the 'trick':
CC="gcc -fPIC" ./configure ...
This assumes a Bourne/Korn/POSIX/Bash shell and sets the environment variable CC to 'gcc -fPIC' before running the configure script. This (usually) ensures that all compilations are done with the specified flags. For setting the correct 'bittiness' of the compilation, this sometimes works better than the various other mechanisms you find - it is hard for a compilation to wriggle around it except by completely ignoring the fact you specified the C compiler to use.
Another option is to pass -fPIC directly to make in the following way:
make CFLAGS='-fPIC' CXXFLAGS='-fPIC'