DSO library symbol seems to have DEFAULT visibility although should be HIDDEN - c++11

I have a problem with setting up a visibility of my shared library. I would like to ask for help in resolving my issue as described below:
I have a bunch of source files which I would like to build as shared library:
+Base
|_Parameter
| |_Parameter.h
| |_Parameter_Exception.h
| |_Parameter_Exception.cpp
|_Data
| |_DataInput.h
| |_DataInput_Exception.h
| |_DataInput_Exception.cpp
...
Well there are some more files in there but I think it does not influence my problem description.
I am using CMake in order to build this SHARED library. Here is part of my CMakeLists.txt which deals with building the library.
project( Base )
cmake_minimum_required(VERSION 3.5
set( Base_HEADERS
Parameter/Parameter.h
Parameter/Parameter_Exception.h
Data/DataInput.h
Data/DataInput_Exception.h
)
set( Base_SOURCES
Parameter/Parameter_Exception.cpp
Data/DataInput_Exception.cpp
)
set( LIBRARY_OUTPUT_PATH ${CMAKE_RUNTIME_OUTPUT_DIRECTORY} )
add_library( Base SHARED ${Base_SOURCES} ${Base_HEADERS} )
generate_export_header( ${PROJECT_NAME} EXPORT_FILE_NAME ${PROJECT_NAME}_Export.h )
This setup builds libBase.so shared library without any problem. It is important to mention that I have set some visibility related flags to be used during linking:
set( CMAKE_SHARED_LINKER_FLAGS "-Wl,--no-undefined -fvisibility=hidden -fvisibility-inlines-hidden" )
As configured in Base library CMakeLists.txt partly listed above:
Base_Export.h header containig GCC visibility attributes macros is
generated by CMake. It is then included in my sources listed in the
tree up there.
By default all the symbols shall be hidden as set in
linker flags (-fvisibility=hidden)
For example a piece of Parameter_Exception.h:
#include "Base_Export.h"
class SomeException : public std::exception
{
public:
SomeException( void ) {}
...
};
Please notice I have not set any visibility attribute here (by a macro defined in Base_Export). So I would assume, the symbol of SomeException class shall be hidden to all the DSO "users" (due to -fvisibility=hidden)
But it doesn't seem so. Unfortunately:
Once I use the library, it the exception is usable although the symbol should be hidden. I would expect link failure cause by this code snippet "outside the library":
#include "Parameter/Parameter_Exception.h"
...
try
{
...
}
/* THIS SHOULD FAIL IN LINKING DUE TO UNDEFINED SYMBOL, RIGHT? */
catch( const SomeException & e )
{
...
}
I have also tried to look for the symbol in the libBase.so using the command:
readelf -Ws libBase.so
it lists enormously long text:
Num: Value Size Type Bind Vis Ndx Name
0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND
1: 000000000008b7e0 0 SECTION LOCAL DEFAULT 9
....
but what I have noticed here is that in "Vis" column up there there is always DEFAULT which implies a default visibility ( __attribute__((visibility("default"))) ) to me BUT I would expect to see HIDDEN in there ( __attribute__((visibility("hidden"))) )
So what am I doing wrong? Or my understanding is not correct? I know all the linked flags are propagated correctly but seem to have no effect.
My toolchain configuration as listed by CMake configuration script (GCC version that supports visibility attributes):
...
-- The CXX compiler identification is GNU 5.3.1
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
...
-- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY
-- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY - Success
-- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY
-- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY - Success
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR - Success
-- Configuring done
-- Generating done
Many thanks in advance to anybody willing to help me and possibly others having the same trouble as I do.
Thanks again, Martin

CMake has direct support for symbol visibility handling, so you shouldn't need to touch CMAKE_SHARED_LINKER_FLAGS or CMAKE_CXX_FLAGS at all. You can set the visibility on a per-target basis like this:
set_target_properties(Base PROPERTIES
C_VISIBILITY_PRESET hidden
CXX_VISIBILITY_PRESET hidden
VISIBILITY_INLINES_HIDDEN YES
)
CMake will then ensure the relevant flags are added for you. Rather than having to set these properties for each target, it is usually more convenient to set the associated variables to set the defaults for all subsequently created targets (and this is what I generally recommend and use in projects I work on):
cmake_minimum_required(VERSION 3.5)
project(Base)
set(CMAKE_C_VISIBILITY_PRESET hidden)
set(CMAKE_CXX_VISIBILITY_PRESET hidden)
set(CMAKE_VISIBILITY_INLINES_HIDDEN YES)
add_library(Base SHARED ...)
When used in conjunction with the GenerateExportHeader module (which you already appear to be using), you get support not just for gcc/clang, but also Visual Studio, making it a nice cross-platform way of controlling symbol visibility. You don't need to know the relevant compiler flags or attributes, CMake will handle them for you and give you equivalent behavior across all the compilers.

-fvisibility=hidden and -fvisibility-inlines-hidden are compiler options,
not linker options. Set them in CMAKE_CXX_FLAGS

Related

Unexpected path to libprotocd.lib after add gRPC::grpc++ library dependence in CMake

I have a gRPC 1.23.0 compiled by conan(and all dependencies also resolved by conan).
And I have a CMake project, that use grpc library:
set (LIB_DEPS
protobuf::libprotobuf
gRPC::grpc++
)
target_link_libraries(${PROJECT_NAME} PUBLIC ${LIB_DEPS})
When I generate a VS project from this CMakeLists.txt in Linker - Input - Addition Dependencies property, I have there next value:
C:\.conan\data\protobuf\3.9.1\kpa_conan\stable\package\b786e9ece960c3a76378ca4d5b0d0e922f4cedc1\lib\libprotobufd.lib <-- (1)
C:\.conan\data\grpc\1.23.0\kpa_conan\stable\package\d85cccdf40588ac852bd1445d45838487543194f\lib\grpc++.lib
libprotocd.lib <-- (2)
libprotobufd.lib <-- (3)
C:\.conan\data\grpc\1.23.0\kpa_conan\stable\package\d85cccdf40588ac852bd1445d45838487543194f\lib\grpc.lib
C:\.conan\data\grpc\1.23.0\kpa_conan\stable\package\d85cccdf40588ac852bd1445d45838487543194f\lib\gpr.lib
C:\.conan\data\c-ares\1.15.0\kpa_conan\stable\package\b786e9ece960c3a76378ca4d5b0d0e922f4cedc1\lib\cares.lib
C:\.conan\data\grpc\1.23.0\kpa_conan\stable\package\d85cccdf40588ac852bd1445d45838487543194f\lib\address_sorting.lib
wsock32.lib
kernel32.lib
(1) - expected valid path, that I add in CMakeLists.txt
(2),(3) - unexpected and invalid path, that added by gRPC::grpc++ in CMakeLists.txt.
If I change gRPC::grpc++ to gRPC::grpc, lines (2) and (3) will disappear, but path to grpc++.lib will disappear too, but I need it.
How to avoid this strange invalid path to libprotocd.lib and libprotobufd.lib?
To properly link against grpc targets you can use:
target_link_libraries(${PROJECT_NAME} PUBLIC CONAN_PKG::grpc)
This should contain everything required.
More details on this approach here: https://docs.conan.io/en/latest/integrations/build_system/cmake/cmake_generator.html#targets-approach
I have found workaround solution:
find_library(GRPC_GRPC++_LIBRARY NAMES grpc++)
set (LIB_DEPS
protobuf::libprotobuf
gRPC::grpc
${GRPC_GRPC++_LIBRARY NAMES}
)
target_link_libraries(${PROJECT_NAME} PUBLIC ${LIB_DEPS})
And opened the issue about it in gRPC repository: https://github.com/grpc/grpc/issues/20578

Error Building Pin Tool that Uses External Library

I declared a function foo() in a header file imp.h and implemented it in imp.c. Then I generated a shared library named libimp.so and in my Pin tool I called foo().
In order to link the tool with this new library I added the following definitions to makefile.rules in its directory:
TOOL_CXXFLAGS += -I/path/to/imp.h
TOOL_LPATHS += -L/path/to/libimp.so
TOOL_LIBS += -limp
I also set LD_LIBRARY_PATH to the /path/to/libimp.so. But, at runtime, if I use foo(), the following error will be received:
dlopen failed. library "libimp.so" not found.
The library is OK when I call it from a simple test program. Any ideas?
I also set LD_LIBRARY_PATH to the /path/to/libimp.so
If the full path to libimp.so is literally /path/to/libimp.so, then the correct value for LD_LIBRARY_PATH is /path/to, and not /path/to/libimp.so.
(It isn't clear from your question whether you understand this.)
You may wish to link your pintool with -Wl,--rpath=/path/to so you don't have to set LD_LIBRARY_PATH at all.

Converting static framework to dynamic

I want to convert static iOS framework (https://github.com/comScore/ComScore-iOS-watchOS-tvOS/tree/master/ComScore/iOS) into dynamic.
> clang -arch x86_64 -dynamiclib -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator10.3.sdk -lc++ -F . -framework ComScore -ObjC -o ComScoreDynamic
This command is successful, however, there is a problem with symbol visibility.
When I check symbols in the original framework it is around 4k of public symbols:
> nm -gU ComScore.framework/ComScore | wc -l
4387
In dynamic version only very few of them:
nm -gU ComScoreDynamic
0000000000114af8 S _OBJC_CLASS_$_SCORCommonUtils
0000000000114940 S _OBJC_CLASS_$_SCORCrossPublisherIdSourceValue
0000000000114a08 S _OBJC_CLASS_$_SCORHTTP
0000000000114990 S _OBJC_CLASS_$_SCORHelper
0000000000114aa8 S _OBJC_CLASS_$_SCORObfuscation
0000000000114a80 S _OBJC_CLASS_$_SCORReachability
0000000000114918 S _OBJC_CLASS_$_SCORUniqueId
0000000000114b20 S _OBJC_METACLASS_$_SCORCommonUtils
00000000001149e0 S _OBJC_METACLASS_$_SCORCrossPublisherIdSourceValue
0000000000114a30 S _OBJC_METACLASS_$_SCORHTTP
0000000000114968 S _OBJC_METACLASS_$_SCORHelper
0000000000114ad0 S _OBJC_METACLASS_$_SCORObfuscation
0000000000114a58 S _OBJC_METACLASS_$_SCORReachability
00000000001149b8 S _OBJC_METACLASS_$_SCORUniqueId
00000000001166b0 D __ZTINSt3__117bad_function_callE
00000000000d5d60 S __ZTSNSt3__117bad_function_callE
All other symbols marked as internal (t and s markings).
How can I keep symbols external?
UPDATE:
Looks like this can be similar problem: Export an `OBJC_CLASS` from one static lib as part of another
Problem is that symbols in static lib are exported as private_extern and there is no way to preserve them in dynamic library.
The public symbols in the comScore framework are marked private external. This can be seen with the nm utility and looking for the SCORAnalytics class:
nm -m ComScore/iOS/ComScore.framework/Versions/A/ComScore |grep _OBJC_CLASS_\$_SCORAnalytics
Shows:
---------------- (LTO,DATA) private external _OBJC_CLASS_$_SCORAnalytics
This means the symbol can only be linked against once. When Cocoapods performs a pre-link for secondary ('transitive') dependencies, these symbols lose their extern attribute. The idea here is to prevent public symbols from a dependency from leaking into those of another library. The problem is that, for Swift projects, they aren't fully resolved until the final app link; by that point they're no longer available.
The real issue is that the comScore library is a static framework. The best solution would be for comScore to release it as a dynamic framework, but these are only supported by iOS 8 and above; comScore insists on support all the way back to iOS 6. I know.
For now, my solution is to include the comScore framework directly in our Cocoapod and vend it within the Podspec, allowing it to work with both Obj-C and Swift projects. The downside is I have to manually update our Cocoapod every time comScore releases a new version. There would also be symbol conflicts if another pod included comScore, but since our pod is a metrics aggregator that logs to several backends, it's likely to be the only metrics component in use. YMMV.

How to run gcov on test application with Nuttx OS on STM discovery board?

Setup:
Toolchain: gcc-arm-none-eabi-5_2-2015q4-20151219
Target: STM429i-disco board
I want to run gcov and get real time report generated in target as per below link:
https://mcuoneclipse.com/2014/12/26/code-coverage-for-embedded-target-with-eclipse-gcc-and-gcov/
First, sucessfully have compiled my code (POSIX compliant NUTTX OS) with -fprofile-arcs & -ftest-coverage flags & got generated the .gcno files for my src files.
second, sucesfully have linked with -fprofile-arcs flags enabled and using the libgcov.a file (part of the toolchain) and the final binary is generated.
Now, I dont know what changes are needed in my test application to invoke gcov, generate report & dump report.
Another problem is, gcov functions are with HIDDEN attribute in libgcov.a as below.
9: 00000000 4 FUNC GLOBAL HIDDEN 1 __gcov_flush
9: 00000000 4 FUNC GLOBAL HIDDEN 1 __gcov_init
so, I could not invoke as I need.
Any inputs in getting the .gcda file generated would be of great help.
Can you look for gcov_exit instead? It is similar to __gcov_flush. Typically, it's one of gcov_exit and __gcov_flush that would be there and you can use any.
In case this is not there or is also hidden, you can use this approach which I tried for one of my projects. I picked (and modified for various reasons) the implementation of gcov_exit from gcc source code (of version matching my toolchain) (available at https://github.com/reeteshranjan/libgcov-embedded) and plugged it in my project. With everything else remaining the same (the compiler flags etc.), I was able to then break into gcov_exit and follow the rest of the approach in the blog link you have mentioned.

undefined reference to '__gthrw___pthread_key_create(unsigned int*, void (*)(void*))

I'm using 64-bit gcc-4.8.2 to generate a 32-bit target, and my machine is 64-bit. I'm using c++11 concurrency features such as thread, mutex, conditiona_variables and etc.
The linker gave the above error message when trying to link the executable. libMyLib is also part of the project.
libMyLib.so: undefined reference to '__gthrw___pthread_key_create(unsigned int*, void (*)(void*))
nm libMyLib.so | grep pthread_key_create shows:
U _ZL28__gthrw___pthread_key_createPjPFvPvE
w __pthread_key_create##GLIBC_2.0
where is the symbol 'ghtrw___pthread_key_create' from? I tried adding '-lpthread(-pthread)' as compiler flag, but it does not help.
More information. nm libMyLib.so | grep pthread shows other symbols such as _ZL20__gthread_mutex_lockP15pthread_mutex_t is defined
where is the symbol 'ghtrw___pthread_key_create' from?
It is defined in GCC's "gthreads" abstraction layer for thread primitives, in the gthr-posix.h header.
#if SUPPORTS_WEAK && GTHREAD_USE_WEAK
# ifndef __gthrw_pragma
# define __gthrw_pragma(pragma)
# endif
# define __gthrw2(name,name2,type) \
static __typeof(type) name __attribute__ ((__weakref__(#name2))); \
__gthrw_pragma(weak type)
# define __gthrw_(name) __gthrw_ ## name
#else
# define __gthrw2(name,name2,type)
# define __gthrw_(name) name
#endif
/* Typically, __gthrw_foo is a weak reference to symbol foo. */
#define __gthrw(name) __gthrw2(__gthrw_ ## name,name,name)
...
#ifdef __GLIBC__
__gthrw2(__gthrw_(__pthread_key_create),
__pthread_key_create,
pthread_key_create)
After preprocessing that expands to:
static __typeof(pthread_key_create) __gthrw___pthread_key_create __attribute__ ((__weakref__("__pthread_key_create")));
It is supposed to be a weak reference to __pthread_key_create, so it should never have a definition, because it is just a reference to glibc's internal __pthread_key_create symbol.
So it looks like something has gone wrong with how you build you library. You should not have an undefined weak symbol.
I recently stumbled into this error, and not because of a missing -pthread.
The situation happened in a somewhat unusual setup: I was compiling a software written in C++14 under Centos7. Since C++14 requires a recent version of GCC, I relied on devtoolset 6 for it.
Given the specific setup, I opened a thread on the Centos mailing list, so I'm directly referencing the relevant thread. See https://lists.centos.org/pipermail/centos-devel/2018-May/016701.html and https://lists.centos.org/pipermail/centos-devel/2018-June/016727.html
In short, it might be caused by some bug in the preprocessor macros, either of glibc or libgcc. It can be fixed by placing #include <thread> in the beginning of the source code which gives problems once compiled. Yes, even if you don't use std::thread in it.
I don't claim this will work for everyone, but it might do the job in some particular situations.

Resources