Configure an autotools project with Clang sanitizers in a static lib configuration? - static-libraries

Question

EDIT: If its TLDR, just skip to the bottom. Its where I ask: How do I configure an autotools project to use a static library?
I'm working with a couple of open source libraries, and I'm trying to run their test suite under Clang's sanitizers. To run under Clang sanitizers, we need to (1) specify some options, and (2) link static libraries from Clang's Compiler-RT as required. Note: there are no dynamic libraries or shared objects.
Setting the options is easy:
export DYLD_FALLBACK_LIBRARY_PATH=/usr/local/lib/clang/3.3/lib/darwin/
export CC=/usr/local/bin/clang
export CXX=/usr/local/bin/clang++
export CFLAGS="-g3 -fsanitize=address -fsanitize=undefined"
export CXXFLAGS="-g3 -fsanitize=address -fsanitize=undefined -fno-sanitize=vptr"
./configure
However, this will generate some archive warnings (when AR is run) and link errors (when LD is run) with undefined symbols. The message will be similar to:
libjpeg.a(jmemmgr.o): In function `do_sarray_io':
/home/jwalton/jpeg-6b/jmemmgr.c:695: undefined reference to
`__ubsan_handle_type_mismatch'
/home/jwalton/jpeg-6b/jmemmgr.c:695: undefined reference to
`__ubsan_handle_type_mismatch'
/home/jwalton/jpeg-6b/jmemmgr.c:696: undefined reference to
`__ubsan_handle_type_mismatch'
I know the libraries that need to be linked. For the sanitizers that I use, they are libclang_rt.asan_osx.a and libclang_rt.ubsan_osx.a (or libclang_rt.full-x86_64.a and libclang_rt.ubsan-x86_64.a on Linux).
To supply the libraries, I then export the following. Note: it is LIBS, and not LDLIBS as expected by most other make related tools.
export LIBS="/usr/local/lib/clang/3.3/lib/darwin/libclang_rt.asan_osx.a \
/usr/local/lib/clang/3.3/lib/darwin/libclang_rt.ubsan_osx.a"
This results in a configure problem of:
configure: error: cannot run C compiled programs.
If you meant to cross compile, use `--host'.
...
Looking at config.log, it looks like two problems are occurring. First, the path is being butchered by changing from /usr/local/... to /Users/jwalton/.... And second, the filename is being butchered by changing from a static lib to a dynamic lib:
configure:3346: ./conftest
dyld: Library not loaded: /Users/jwalton/clang-llvm/llvm-3.3.src/Release+Asserts/lib/clang/3.3/lib/darwin/libclang_rt.asan_osx_dynamic.dylib
Referenced from: /Users/jwalton/libpng-1.6.7/./conftest
Reason: image not found
In another attempt, I tried using LDFLAGS:
export LDFLAGS="-L/usr/local/lib/clang/3.3/lib/darwin/"
export LIBS="libclang_rt.asan_osx.a libclang_rt.ubsan_osx.a"
That results in a similar error:
configure: error: in `/Users/jwalton/libpng-1.6.7':
configure: error: C compiler cannot create executables
And config.log:
configure:3209: /usr/local/bin/clang -g3 -fsanitize=address -fsanitize=undefined -L/usr/local/lib/clang/3.3/lib/darwin/ conftest.c libclang_rt.asan_osx.a libclang_rt.ubsan_osx.a >&5
clang: error: no such file or directory: 'libclang_rt.asan_osx.a'
clang: error: no such file or directory: 'libclang_rt.ubsan_osx.a'
And dropping the lib prefix and .a suffix from the LIBS results in:
configure:3209: /usr/local/bin/clang -g3 -fsanitize=address -fsanitize=undefined -L/usr/local/lib/clang/3.3/lib/darwin/ conftest.c clang_rt.asan_osx clang_rt.ubsan_osx >&5
clang: error: no such file or directory: 'clang_rt.asan_osx'
clang: error: no such file or directory: 'clang_rt.ubsan_osx'
And adding the -l to LIBS results in:
configure:3335: /usr/local/bin/clang -o conftest -g3 -fsanitize=address -fsanitize=undefined
-L/usr/local/lib/clang/3.3/lib/darwin/ conftest.c -lclang_rt.asan_osx -lclang_rt.ubsan_osx >&5
configure:3339: $? = 0
configure:3346: ./conftest
dyld: could not load inserted library: /Users/jwalton/libpng-1.6.7/./conftest
./configure: line 3348: 38224 Trace/BPT trap: 5 ./conftest$ac_cv_exeext
Finally, the -L argument is valid:
$ ls /usr/local/lib/clang/3.3/lib/darwin/
libclang_rt.10.4.a libclang_rt.ios.a
libclang_rt.asan_osx.a libclang_rt.osx.a
libclang_rt.asan_osx_dynamic.dylib libclang_rt.profile_ios.a
libclang_rt.cc_kext.a libclang_rt.profile_osx.a
libclang_rt.cc_kext_ios5.a libclang_rt.ubsan_osx.a
libclang_rt.eprintf.a
After all the background: how do I configure an autotools project to use a static library?
Bonus points: why has something so easy been made so difficult?

Answer 1

You need to add -fsanitize flags to LDFLAGS as well.

Answer 2

Shipping versions of GNU libtool do not pass -fsanitize=... arguments to the linker. You will need to update libtool with the patch from http://savannah.gnu.org/patch/?8775 Specifically:
diff --git a/build-aux/ltmain.in b/build-aux/ltmain.in
index 0c40da0..b99b0cd 100644
--- a/build-aux/ltmain.in
+++ b/build-aux/ltmain.in
## -5362,10 +5362,11 ## func_mode_link ()
# -O*, -g*, -flto*, -fwhopr*, -fuse-linker-plugin GCC link-time optimization
# -specs=* GCC specs files
# -stdlib=* select c++ std lib with clang
+ # -fsanitize=* Clang memory and address sanitizer
-64|-mips[0-9]|-r[0-9][0-9]*|-xarch=*|-xtarget=*|+DA*|+DD*|-q*|-m*| \
-t[45]*|-txscale*|-p|-pg|--coverage|-fprofile-*|-F*|#*|-tp=*|--sysroot=*| \
-O*|-g*|-flto*|-fwhopr*|-fuse-linker-plugin|-fstack-protector*|-stdlib=*| \
- -specs=*)
+ -specs=*|-fsanitize=*)
func_quote_for_eval "$arg"
arg=$func_quote_for_eval_result
func_append compile_command " $arg"
As for getting libtool to accept a static library, you should be able to just include the full path to the static library on the command line, or you can use options like -L/usr/local/lib/clang/3.3/lib/darwin/ -lclang_rt.ubsan_osx. However, cfe should take care of the library magic for you once libtool tells it to use -fsanitize=... for linking.

Answer 3

As for the undefined symbol issue, if you are linking C++ objects, then you must use clang++ instead of clang. Otherwise you can get error such as:
/bin/bash libtool --tag=CC --mode=link clang-3.6 ... -fsanitize=undefined -o freeswitch ...
libtool: link: clang-3.6 ... -fsanitize=undefined -o .libs/freeswitch ./.libs/libfreeswitch.so ...
./.libs/libfreeswitch.so: undefined reference to `__ubsan_vptr_type_cache'
./.libs/libfreeswitch.so: undefined reference to `__ubsan_handle_dynamic_type_cache_miss'
clang: error: linker command failed with exit code 1 (use -v to see invocation)
In the above case, the atrocious libtool used clang instead of clang++ because automake thought that you wanted to link a set of C objects (it does not understand that linking a libtool archive (libfreeswitch.la) using C++ means that a C++ linker is needed (note the --tag=CC). See also https://lists.debian.org/debian-mentors/2003/06/msg00004.html.
To work around this issue, follow the instructions at https://www.gnu.org/software/automake/manual/html_node/Libtool-Convenience-Libraries.html and add a (dummy/real) C++ source to your sources in your Makefile.am:
SUBDIRS = sub1 sub2 …
lib_LTLIBRARIES = libtop.la
libtop_la_SOURCES =
# Dummy C++ source to cause C++ linking.
nodist_EXTRA_libtop_la_SOURCES = dummy.cxx
libtop_la_LIBADD = \
sub1/libsub1.la \
sub2/libsub2.la \
…