Using -Os/-Oz with Android NDK and cmake for release builds - gradle

Using: Andorid Studio 3.1.3, NDKr17b, gradle plugin 3.1.3, gradle-4.5.1-all.zip
I am not able to produce any release build of a shared library using cmake it fails with the following:
arm-linux-androideabi/bin/ld: fatal error: Optimization level must be between 0 and 3
Here is cmake release configuration I use
release {
externalNativeBuild {
cmake {
arguments "-DCMAKE_BUILD_TYPE=Release",
"-DANDROID_CPP_FEATURES=rtti exceptions",
"-DANDROID_STL=c++_static",
"-DCMAKE_VERBOSE_MAKEFILE:BOOL=ON"
cppFlags "-ffunction-sections -fdata-sections -fvisibility=hidden -fvisibility-inlines-hidden -Os"
cFlags "-ffunction-sections -fdata-sections -fvisibility=hidden -fvisibility-inlines-hidden -Os"
}
}
consumerProguardFiles 'proguard-project.txt'
}
I found that ld is called with the following options:
-plugin /home/myhome/Android/Sdk/ndk-bundle/toolchains/llvm/prebuilt/linux-x86_64/bin/../lib64/LLVMgold.so -plugin-opt=mcpu=generic -plugin-opt=Os -plugin-opt=-function-sections -plugin-opt=-data-sections
And the problem is existence of "-plugin-opt=Os", when I run the command without this option it links, even that all source is compiled with the proper optimization level.
Using the same configuration with ndk-build works fine (ld call has no such option, it just loads the LLVMgold.so plugin, with no --plugin-opt=Os).
So my question is why this option "-plugin-opt=Os" is applied and how can I remove it? Is this comes from cmake or its from ninja?

I see you're also the filer of this bug, but for anyone else landing here, this is a bug in the Clang LTO plugin: https://github.com/android-ndk/ndk/issues/721.

Even this is a bug in Clang LTO plugin (https://reviews.llvm.org/D30920) - it can be avoided - note that ndk-build system is using the same toolchain, but this problem does not exist.
The solution is to not pass Os option to Clang LTO plugin ("-plugin-opt=Os")
and this can be done by removing -Os option from generic ANDROID_COMPILER_FLAGS_RELEASE, instead pass this option directly to your makefiles, this way it will not be added to Clang LTO plugin. But since ANDROID_COMPILER_FLAGS_RELEASE is not user variable, the only way to do this is to comment two lines (510, 512) in ndk-bundle/build/cmake/android.toolchain.cmake inside Android Sdk folder

Related

Using gcc plugins with cross compiler, undefined symbol

I'm trying to see if it's possible to use a gcc plugin in an ARM cross compiler (arm-none-eabi-gcc). I'm running into compiler errors however, and am questioning whether what I'm trying to do is possible.
The plugin I'm trying to set up is: https://github.com/vanhauser-thc/AFLplusplus/tree/master/gcc_plugin
I'm compiling the plugin on x86-64 linux using the -m32 flag, since the cross compiler is a 32-bit application. However when I try to use the plugin in the cross compiler using -fplugin , I get an undefined symbol compiler error:
cc1plus: error: cannot load plugin ../afl-gcc-pass.so
../afl-gcc-pass.so: undefined symbol: _Z13build_int_cstP9tree_nodel
I looked through the plugin's symbols using nm and discovered that the majority of symbols are undefined, including ones like exit and random. I'm new to most of this and am unsure of what that really means. Some searching online suggested that it may have had something to do with incorrect library paths, but setting LIBRARY_PATH and LD_LIBRARY_PATH and rebuilding did not seem to help.
The gcc version set-ups I have tried:
1: x86: 5.4.0 , arm: 5.4.1 on ubuntu 16.04
2: x86: 5.2.0 , arm: 5.2.1 on CentOS 6.8
Is it possible to use a gcc plugin in a different gcc than it was compiled with or am I wasting my time?
Yes, it is possible to build a gcc plugin with a given compiler and then use the plugin in another compiler (including a cross-compiler), but you have to make sure you include the right header files when building the plugin.
Specifically, you have to include the plugin development header files of the target compiler, instead of those of the host compiler. The directory where plugin development files for your target compiler are located can be obtained with the following command:
$(TARGET_CC) -print-file-name=plugin
where $(TARGET_CC) is your target compiler. So a concise way to specify the relevant include directory in the compiler flags when building the plugin would be something like -I"$(shell $(TARGET_CC) -print-file-name=plugin)/include".
For the specific plugin you are trying to use (instrumentation for afl-fuzz), in order to build the plugin for your cross-compiler you could modify the Makefile in the gcc_plugin folder; more specifically, you could define a TARGET_CC variable containing the path to your cross-compiler, and then replace $(CC) with $(TARGET_CC) in the definition of PLUGIN_FLAGS, as in:
PLUGIN_FLAGS = -fPIC -fno-rtti -I"$(shell $(TARGET_CC) -print-file-name=plugin)/include"
You will also have to comment out the commands executed in the test_build Makefile target, because those commands would try to use the plugin with the native compiler and so would fail.
Then, you will be able to use the plugin with your cross-compiler, as in:
arm-none-eabi-gcc -fplugin=../afl-gcc-pass.so --specs=nosys.specs my_source_file.c

GCC gprof complaining GLIBC_2.16 is not found

I have a code running on a PowerPC e500v2 embedded Linux and I want to measure its performance since it is running in an infinite loop. I tried gcc's gprof which was simply by adding -pg option to gcc. When I run the binary on the target device I get this:
./main: /lib/libc.so.6: version GLIBC_2.16 not found (required by ./main)
I am using ELDK 5.6 toolchain with the default CFLAGS and LDFLAGS and these flags: -Wall -lrt -pthread -D_GNU_SOURCE nothing else. Some article suggested defining FORTIFY_SOURCE along with an optimization level but it did not work. I searched for some gcc's feature test macros and tried defining some GLIBC 2.16 specific macros but it did not work.
I faced similar issue with GLIBC 2.17 when I used some structures and functions from <sched.h>, adding _GNU_SOURCE resolved it. Any idea on how to resolve it?
When I run the binary on the target device I get this
Your tool chain targets a version of GLIBC that is newer than what is installed on the target.
This doesn't bite you in non-pg compiles only by accident. An "innocent" change to your source can cause the same problem.
You need to upgrade your target to the version of GLIBC which your toolchain actually builds for.

cmake keeps adding the std=gnu++11 option

I'm trying to compile a project in C++ using cmake, and in the page of the project they tell me that it will crash if I don't add the standard 98. (I'm on a mac)
I've tried all I found on the internet and I could manage to make the cmake use the option -std=c++98 but it also adds -DNDEBUG -std=gnu++11. (I saw it using the make VERBOSE=1 option)
I would like to get rid of that. Using the --trace option I could see that the option is set in a file which is in the cellar folder, that is, is something that has to do with cmake itself and not in the CMakeList.txt file im using.
How can I solve this problem?
If it can help the code I'm trying to compile is this:
SAMoS
Thank you.
UPDATE:
with the --trace option I was able to see that the -std=gnu++11 option was selected in the file:
/usr/local/Cellar/cmake/3.9.4.1/share/cmake/Modules/Compiler/GNU-CXX.cmake
which can be seen here GNU-CXX.cmake
If I eddit that file in a way that every if sets the option to -std=c++98 then, the cmake complains giving me the next error:
CMake Error in src/CMakeLists.txt:
The compiler feature "cxx_nullptr" is not known to CXX compiler
"GNU"
version 7.2.0.
I don't know what else can I try...
You need to set the language standard:
set(CMAKE_CXX_STANDARD 98)
Depending on the compiler, it may enable extensions as well. To disable the GNU extensions also add:
set(CMAKE_CXX_EXTENSIONS OFF)
Note that setting this options does so only for the specified target and dependent targets.
Have take a look at this section of the CMake manual for more information on compiler features. Do note however, using this
The inclusion of VTK is polluting SAMoS's CMake scope with the C++11 requirement. You can test this by disabling VTK on your cmake command line.
$ cd ~SAMoS
$ mkdir build; cd build
$ cmake -DVTK_FOUND=FALSE ../
[...]
$ make VERBOSE=1
[...]
Scanning dependencies of target samos
/Applications/Xcode.app/Contents/Developer/usr/bin/make -f src/CMakeFiles/samos.dir/build.make src/CMakeFiles/samos.dir/build
[ 1%] Building CXX object src/CMakeFiles/samos.dir/samos.cpp.o
cd /Users/nega/SAMoS/build/src && /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -DCGAL_USE_GMP -DCGAL_USE_MPFR -DHAS_CGAL -isystem /usr/local/include -I/include -I/Users/nega/SAMoS/src/constraints -I/Users/nega/SAMoS/src/dump -I/Users/nega/SAMoS/src/log -I/Users/nega/SAMoS/src/integrators -I/Users/nega/SAMoS/src/messenger -I/Users/nega/SAMoS/src/parser -I/Users/nega/SAMoS/src/potentials -I/Users/nega/SAMoS/src/potentials/external -I/Users/nega/SAMoS/src/potentials/pair -I/Users/nega/SAMoS/src/potentials/bond -I/Users/nega/SAMoS/src/potentials/angle -I/Users/nega/SAMoS/src/system -I/Users/nega/SAMoS/src/utils -I/Users/nega/SAMoS/src/aligner -I/Users/nega/SAMoS/src/aligner/pair -I/Users/nega/SAMoS/src/aligner/external -I/Users/nega/SAMoS/src/population -I/Users/nega/SAMoS/src -I/Users/nega/SAMoS/build -DNDEBUG -o CMakeFiles/samos.dir/samos.cpp.o -c /Users/nega/SAMoS/src/samos.cpp
You'll notice there's no -std=gnu++11 flag anymore. Of course, since it looks like you're GCC version 7.2, you'll still want your set CMAKE_CXX_STANDARD to 98 since gcc-7.2 uses C++11 by default. (Or maybe it's C++14 now...) You can do this on your cmake command line.
$ cmake -DUSE_VTK=FALSE -DCMAKE_CXX_STANDARD=98 ..
CMake will then add -std=gnu++98 to its compile commands.
If you can't live without VTK, then you'll need to send a bug report upstream asking the SAMoS folks to clarify their documentation, or fix how they're including VTK.

"undefined reference" errors when trying to use address sanitizer with GCC

I'm trying to build my project with
g++ -O0 -g -fsanitize=address -fno-omit-frame-pointer
but get lots of errors like:
/home/user/libs/opencv/include/opencv2/core/mat.hpp:715: undefined reference to `__asan_report_load8'
How to compile project with AddressSanitize support?
My gcc version is 4.8.4.
You need to add -fsanitize=address to compiler flags (both CFLAGS and CXXFLAGS) and linker flags (LDFLAGS). You've probably added it to your compiler flags only.
Note that using explicit -lasan option has been widely discouraged by ASan developers (e.g. here) as it misses some other important linker flags. The only recommended way to link is to use -fsanitize=address.
As a side note, for more aggressive verification flags check Asan FAQ (look for "more aggressive diagnostics").
Make sure you have libasan installed. For example, in Fedora:
dnf install libasan libasan-static
You need to add the switch -lasan -fsanitize=address to your both your compile and link command line to link the correct library.
Note: the original answer -lasan is outdated and should not be used, as per comments

Compiling library in g++ using C++11

I'm been attempting to compile an open-source C++ library (QuantLib-1.7) on my mac for several days but I seem to be encountering some kind of C++11 compatibility issue.
When I run make && sudo make install from the terminal the compilation seems to work except for a bunch of errors of the form
Making all in BermudanSwaption
g++ -DHAVE_CONFIG_H -I. -I../../ql -I../.. -I../.. -I/opt/local/include -g -O2 -MT BermudanSwaption.o -MD -MP -MF .deps/BermudanSwaption.Tpo -c -o BermudanSwaption.o BermudanSwaption.cpp
In file included from BermudanSwaption.cpp:22:
In file included from ../../ql/quantlib.hpp:43:
In file included from ../../ql/experimental/all.hpp:25:
In file included from ../../ql/experimental/volatility/all.hpp:21:
In file included from ../../ql/experimental/volatility/zabr.hpp:31:
In file included from ../../ql/math/statistics/incrementalstatistics.hpp:35:
In file included from /opt/local/include/boost/accumulators/statistics/stats.hpp:14:
In file included from /opt/local/include/boost/accumulators/statistics_fwd.hpp:12:
/opt/local/include/boost/mpl/print.hpp:50:19: warning: in-class initialization
of non-static data member is a C++11 extension [-Wc++11-extensions]
const int m_x = 1 / (sizeof(T) - sizeof(T));
^
1 warning generated.
I'm guessing this has something to do with g++ not being correctly configured for C++11. I'm familiar with the fact that C++11 can be invoked by compiling with g++ -std=c++11. However, despite a lot of googling I can't find a way to modify the makefile such that -std=c++11 is called when I run make && sudo make install.
Any help would be greatly appreciated.
Here is the section of the makefile which I believe is relevant:
BOOST_INCLUDE = -I/opt/local/include
BOOST_LIB = -L/opt/local/lib
BOOST_THREAD_LIB =
BOOST_UNIT_TEST_DEFINE = -DQL_WORKING_BOOST_STREAMS
BOOST_UNIT_TEST_LIB = boost_unit_test_framework-mt
BOOST_UNIT_TEST_MAIN_CXXFLAGS = -DBOOST_TEST_DYN_LINK
CC = gcc
CCDEPMODE = depmode=gcc3
CFLAGS = -g -O2
CPP = gcc -E
CPPFLAGS = -I/opt/local/include
CXX = g++
CXXCPP = g++ -E
CXXDEPMODE = depmode=gcc3
CXXFLAGS = -g -O2
Here is the output from running "g++ -v":
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 7.0.0 (clang-700.1.76)
Target: x86_64-apple-darwin14.5.0
Thread model: posix
Makefile.am: https://www.dropbox.com/s/v5j7qohwfup81od/Makefile.am?dl=0
Makefile.in: https://www.dropbox.com/s/t92hft9ea2ar1zw/Makefile.in?dl=0
QuantLib-1.7 directory: https://www.dropbox.com/sh/ulj0y68m8x35zg8/AAA-w7L2_YWIP8_KnwURErzYa?dl=0
Full error log: https://www.dropbox.com/s/g09lcnma8skipv7/errors.txt?dl=0
Add something like
CXXFLAGS += -std=c++11
to your Makefile. This will work regardless of the Darwin-specific munging of the g++ executable---it's really clang++.
References:
https://gcc.gnu.org/onlinedocs/gcc-5.2.0/gcc/C_002b_002b-Dialect-Options.html#C_002b_002b-Dialect-Options
https://gcc.gnu.org/projects/cxx0x.html
http://clang.llvm.org/cxx_status.html
https://www.gnu.org/software/make/manual/html_node/Implicit-Variables.html
As you already have, and are familiar with homebrew, my suggestion would be to use that to install and manage quantlib like this:
brew install quantlib
That will then build and put all the files in /usr/local/Cellar/quantlib under some version number that is not of importance. The important thing is the the tools are then linked into /usr/local/bin so all you need to do is make sure that /usr/local/bin is in your PATH.
That gives you access to the tool quantlib-config which is always linked to the latest version and it knows which version that is. So, if you run:
quantlib-config --cflags
it will tell you what the correct path is for your includes like this:
-I/usr/local/Cellar/quantlib/1.6.1/include
Likewise, if you run:
quantlib-config --libs
it will tell you the correct linking directories and libraries for your latest version.
In short, all you need to do to compile is:
g++ $(quantlib-config --cflags --libs)
and it will always pull in the version you are using.
Note that if you use a Makefile, you will need to double the dollar signs.
This is how I eventually managed to compile the Quantlib library for future reference. It is probably not the most efficient/elegant method but it appears to work.
I followed the steps given in http://quantlib.org/install/macosx.shtml and found that running make && sudo make install led to the error reported in the OP.
Create a new static library C++ project in Eclipse called 'Quantlib'
Copy the ql directory located in the .tar file to the Quantlib Eclipse workspace
Right-click Quantlib > Properties > C/C++ Build > Settings > Cross G++ Compiler: Change the Language standard to ISO C++ 11 (-std=c++0x)
Right-click Quantlib > C/C++ General > Paths and Symbols: Add the following include directories for GNU C++
opt/local/include
/Quantlib (check "Is a workspace directory")
/opt/local/include/boost.
Build the Quantlib project (around 34 min on MacBook Air 1.8 GHz Intel Core i7)
Create a new C++ executable project (e.g. BermudanSwaption) and copy the BermudanSwaption.cpp into the BermudanSwaption Eclipse workspace
Repeat steps 4. and 5. for the BermudanSwaption Eclipse project
Right-click BermudanSwaption > Properties > C/C++ General > Paths and Symbols > References: check Quantlib (the Library Paths tab should now contain the entry '/Quantlib/Debug')
Build and run the BermudanSwaption executable project
QuantLib-1.7
OSX Yosemite 10.10.5
Eclipse C/C++ Development Tools Version: 8.8.0.201509131935
Xcode Version 7.1 (7B91b)
xcode-select version 2339.

Resources