Linking a static boost build into a static library under VS2012 & Win32/64 - visual-studio

I'm trying to build a static library to be released as an API for a network device. I can successfully compile and link the library to produce .lib output files, and I relocate them into a directory structure as follows:
EyeLib
L-Include
| L-PublicInterface.h
L-Lib
| L-debug
| | L-MyLib.lib
| | L-MyLib.pdb
| L-release
| L-MyLib.lib
L-MyLibConfig.cmake
Where the MyLibConfig.cmake file is extremely simple, and contains:
# the header file is relative to this cmake file, so get the path.
GET_FILENAME_COMPONENT( MyLib_TOPLEVEL_DIR ${CMAKE_CURRENT_LIST_FILE} PATH )
SET( MyLib_INCLUDE_DIR ${MyLib_TOPLEVEL_DIR}/include )
IF( WIN32 )
FIND_LIBRARY( MyLib_DEBUG_LIBRARY MyLib ${MyLib_TOPLEVEL_DIR}/lib/debug )
FIND_LIBRARY( MyLib_RELEASE_LIBRARY MyLib ${MyLib_TOPLEVEL_DIR}/lib/release )
SET( MyLib_LIBRARIES optimized ${MyLib_RELEASE_LIBRARY} debug ${MyLib_DEBUG_LIBRARY} )
ENDIF( WIN32 )
IF( UNIX )
FIND_LIBRARY( MyLib_LIBRARY MyLib ${MyLib_TOPLEVEL_DIR}/lib )
SET( MyLib_LIBRARIES "${MyLib_LIBRARY}" )
MARK_AS_ADVANCED( MyLib_LIBRARY )
ENDIF( UNIX )
# handle the QUIETLY and REQUIRED arguments
INCLUDE(FindPackageHandleStandardArgs)
FIND_PACKAGE_HANDLE_STANDARD_ARGS(MyLib DEFAULT_MSG MyLib_LIBRARIES MyLib_INCLUDE_DIR)
MARK_AS_ADVANCED( MyLib_INCLUDE_DIR )
This build structure has worked for some test libraries I've built in the past, but I'm getting a link error when I try and use it to build a simple test app saying "error LNK1104: cannot open file 'libboost_thread-vc110-mt-s-1_54.lib'"
I can get the test app to build and run successfully if I add it to the same project as the library build. I assume this is because the library build is finding the boost libs to link against, so it propagates through to the executables in the project.
I built boost 1.54 with b2 link=static runtime-link=static threading=multi variant=debug,release --layout=tagged and linked both the library build and the test app build to the static MSVC runtime (/MT).
Can anyone offer some help/advice/further tests with this one? I need to make sure that all the boost stuff is compiled-in to the API library, so our clients don't have to install boost themselves.
Additional Info
In-case it's helpful, here's the cmakelists.txt file from the library build:
set(LIBRARY_OUTPUT_PATH "${CMAKE_BINARY_DIR}/lib")
set(Boost_USE_STATIC_LIBS ON)
set(Boost_USE_MULTITHREADED ON)
find_package(Boost REQUIRED COMPONENTS system date_time regex thread chrono)
if(NOT WIN32)
list(APPEND Boost_LIBRARIES pthread)
endif()
include_directories(${Boost_INCLUDE_DIRS})
FILE(GLOB srcs *.cpp)
FILE(GLOB headers *.h)
set(libname MyLib)
set(deps ${Boost_LIBRARIES})
#To allow compilation. std=c++0x is for accepting the access to enums, which usually is just accepted with Visual Studio
IF( NOT WIN32 )
set (CMAKE_CXX_FLAGS "-fpermissive -std=c++0x")
ENDIF( NOT WIN32 )
SOURCE_GROUP( ${libname} FILES ${srcs} )
SOURCE_GROUP( "${libname}\\Hdr" FILES ${headers} )
add_library(${libname} ${srcs} ${headers})
target_link_libraries( ${libname} ${deps} )

This is by design.
When building a static library, any dependencies to that library will not get linked into the library directly. Instead when building an executable all library dependencies (direct and indirect) will be linked directly to that executable.
This is also the way most compiler handle static libraries. While VS does offer a special option to link dependencies into static libs, this is not possible on e.g. gcc without resorting to dirty file hacks. Since CMake only supports features that can be used on all supported generators, CMake will not allow to do this even on VS builds.
You have a couple of options now:
Use a dll instead of a static library (add_library(${libname} SHARED ...)). While a static library is basically a bunch of object files wrapped together, a dll is more or less the same as an executable. In particular, all static library dependencies get linked directly into the dll. The disadvantage here is that you have to deal with the usual dll mess: You have to specify which functions to export and the usual issues with passing stuff across dll boundaries apply.
Have your find script also search for all the dependencies. You should be able to restructure your library's dependency handling in a way that the amount of code duplication is minimal. The disadvantage is that configuring a third-party application becomes more difficult (especially on Windows) since you now not only need to find the library itself, but also all of its dependencies.
Use exported targets. This approach makes most sense if the library is built on the same machine as the final executable. Upon building the library, CMake auto-generates the config files for using that library. Your application then just needs to find an include that script and you're good to go. Disadvantage is that the export mechanism is not the most straight-forward feature of CMake, so you will have to spend some time to familiarize yourself with it.
Pull in the library sources directly into each executable. Basically each executable does an add_subdirectory on the library source dir. You still have to configure the dependencies for each executable and on top of that you also have to build the library seperately for each executable. You probably don't want to do this.

Related

cmake: using shared library while building when only headers exist

I would like to
build a binary that is relying on a shared library at runtime.
the library does not exist while compiling the binary.
the headers for the library are available
I would like to create a CMake project under those circumstances. Therefore I tried the following, but it will complain about the missing lib for sure:
cmake_minimum_required (VERSION 3.8)
project (example)
add_executable(${PROJECT_NAME}
main.c
)
target_link_libraries(${PROJECT_NAME}
PRIVATE
# system libs
lib_that_does_not_yet_exist
)
When removing the link instruction to lib_that_does_not_yet_exist, then I will get a undefined reference error.
Question: Is there any way to accomplish what I need?
Background: I am crosscompiling the binary, the lib is therefore having the wrong architecture to install it on the host. I know there may be some ways around that issue, but I am mainly interested in the above question.

missing .lib file when creating shared library with cmake and visual studio 2019 generator

I have created a personnal C++ library and I try to use it in an other program. To create and compile my library, I use the commands
cmake -G "Visual Studio 16 2019" -A x64 ..
cmake --build . --config Release --target INSTALL
I have no problem with te compilation and the installation, but in the Target-release.cmake install look like this :
#----------------------------------------------------------------
# Generated CMake target import file for configuration "Release".
#----------------------------------------------------------------
# Commands may need to know the format version.
set(CMAKE_IMPORT_FILE_VERSION 1)
# Import target "containers" for configuration "Release"
set_property(TARGET containers APPEND PROPERTY IMPORTED_CONFIGURATIONS RELEASE)
set_target_properties(containers PROPERTIES
IMPORTED_IMPLIB_RELEASE "C:/Program Files (x86)/containers/lib/containers.lib"
IMPORTED_LOCATION_RELEASE "C:/Program Files (x86)/containers/bin/containers.dll"
)
list(APPEND _IMPORT_CHECK_TARGETS containers )
list(APPEND _IMPORT_CHECK_FILES_FOR_containers "C:/Program Files (x86)/containers/lib/containers.lib" "C:/Program Files (x86)/containers/bin/containers.dll" )
# Commands beyond this point should not need to know the version.
set(CMAKE_IMPORT_FILE_VERSION)
Nothing abnormal, as far as I can see. My problem is : I have no file C:/Program Files (x86)/containers/lib/containers.lib generated during the compilation and the installation. So every time I try to use a find package, I have an error. (It's only with visual studio generator, it works with mingw). Here is my CMakeLists.txt :
cmake_minimum_required(VERSION 3.1)
set(VERSION 1.0.0)
project (containers
VERSION ${VERSION}
DESCRIPTION "Library adding some features to containers of the stl."
LANGUAGES CXX)
option(BUILD_TESTING "Build test programs" OFF)
include(CTest)
# Set the possible values of build type for cmake-gui
set(CMAKE_BUILD_TYPE Release CACHE STRING "Choose the type of build.")
set_property(CACHE CMAKE_BUILD_TYPE PROPERTY STRINGS "Debug" "Release" "MinSizeRel" "RelWithDebInfo")
message( STATUS "Sources path : ${PROJECT_SOURCE_DIR}")
message( STATUS "Binary path : ${PROJECT_BINARY_DIR}")
message( STATUS "install path : ${CMAKE_INSTALL_PREFIX}")
message( STATUS "Version : ${PROJECT_VERSION}")
message( STATUS "Version : ${PROJECT_VERSION}")
message( STATUS "Compiler : ${CMAKE_CXX_COMPILER_ID}")
set(SOURCES ${PROJECT_SOURCE_DIR}/src/containers.cpp)
set(HEADERS ${PROJECT_SOURCE_DIR}/include/containers/containers_check.h
${PROJECT_SOURCE_DIR}/include/containers/containers.h
${PROJECT_SOURCE_DIR}/include/containers/append.h
${PROJECT_SOURCE_DIR}/include/containers/contains.h
${PROJECT_SOURCE_DIR}/include/containers/remove.h
${PROJECT_SOURCE_DIR}/include/containers/print.h
)
add_library(containers SHARED ${SOURCES} ${HEADERS})
#add_library(containers INTERFACE)
target_compile_features(containers PRIVATE cxx_std_17)
set_target_properties(containers
PROPERTIES
MSVC_RUNTIME_LIBRARY "MultiThreadedDLL"
CXX_STANDARD_REQUIRED YES
CXX_EXTENSIONS OFF)
target_include_directories(containers
PUBLIC
$<INSTALL_INTERFACE:include>
$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include>)
target_compile_options(containers PRIVATE
# g++
#$<$<CXX_COMPILER_ID:GNU>:$<BUILD_INTERFACE:-Wall>>
#$<$<CXX_COMPILER_ID:GNU>:$<BUILD_INTERFACE:-Wextra>>
#$<$<CXX_COMPILER_ID:GNU>:$<BUILD_INTERFACE:-pedantic>>
#$<$<CXX_COMPILER_ID:GNU>:$<BUILD_INTERFACE:-Werror>>
#$<$<CXX_COMPILER_ID:GNU>:-Wno-reorder>
## Clang
#$<$<CXX_COMPILER_ID:Clang>:$<BUILD_INTERFACE:-Wall>>
##TODO
## MSVC
#$<$<CXX_COMPILER_ID:MSVC>:$<BUILD_INTERFACE:/W4>>
#$<$<CXX_COMPILER_ID:MSVC>:$<BUILD_INTERFACE:/WX>>
)
install(TARGETS containers
EXPORT containersTarget
ARCHIVE DESTINATION ${CMAKE_INSTALL_PREFIX}/lib/ COMPONENT Development
LIBRARY DESTINATION ${CMAKE_INSTALL_PREFIX}/lib/ COMPONENT Library NAMELINK_COMPONENT Development
RUNTIME DESTINATION ${CMAKE_INSTALL_PREFIX}/bin/ COMPONENT Runtimes
)
install(FILES ${HEADERS}
DESTINATION ${CMAKE_INSTALL_PREFIX}/include/containers/
COMPONENT headers)
include(CMakePackageConfigHelpers)
# For moteur_de_calculConfig.cmake
set(INCLUDE_INSTALL_DIR ${CMAKE_INSTALL_PREFIX}/include CACHE PATH "install path for include files")
set(LIBRARY_INSTALL_DIR ${CMAKE_INSTALL_PREFIX}/lib CACHE PATH "install path for libraries")
#------------------------------------------------------------------------------
# Configure <export_config_name>ConfigVersion.cmake common to build and install tree
set(config_version_file ${PROJECT_BINARY_DIR}/containersConfigVersion.cmake)
write_basic_package_version_file(
${config_version_file}
VERSION "${CMAKE_PROJECT_VERSION}"
COMPATIBILITY ExactVersion
)
#------------------------------------------------------------------------------
# Export '<export_config_name>Target.cmake' for a build tree
export(TARGETS
containers
FILE "${CMAKE_CURRENT_BINARY_DIR}/${PROJECT_NAME}Target.cmake"
)
#------------------------------------------------------------------------------
# Configure '<export_config_name>Config.cmake' for a build tree
set(build_config ${CMAKE_BINARY_DIR}/containersConfig.cmake)
configure_package_config_file(
"containersConfig.cmake.in"
${build_config}
INSTALL_DESTINATION "${PROJECT_BINARY_DIR}"
PATH_VARS INCLUDE_INSTALL_DIR LIBRARY_INSTALL_DIR VERSION
)
#------------------------------------------------------------------------------
# Export '<export_config_name>Target.cmake' for an install tree
install(EXPORT
containersTarget
DESTINATION "${CMAKE_INSTALL_PREFIX}/lib/cmake/${PROJECT_NAME}"
)
#------------------------------------------------------------------------------
# Configure '<export_config_name>Config.cmake' for a install tree
set(install_config ${PROJECT_BINARY_DIR}/CMakeFiles/containersConfig.cmake)
configure_package_config_file(
containersConfig.cmake.in
${install_config}
INSTALL_DESTINATION "${CMAKE_INSTALL_PREFIX}/lib/cmake/${PROJECT_NAME}"
PATH_VARS INCLUDE_INSTALL_DIR LIBRARY_INSTALL_DIR VERSION
)
# Install config files
install(
FILES ${config_version_file} ${install_config}
DESTINATION "${CMAKE_INSTALL_PREFIX}/lib/cmake/${PROJECT_NAME}"
)
# test
if(BUILD_TESTING)
add_subdirectory(test)
endif()
# uninstall target
# use : make uninstall
if(NOT TARGET uninstall)
configure_file(
"${CMAKE_CURRENT_SOURCE_DIR}/cmake_uninstall.cmake.in"
"${CMAKE_CURRENT_BINARY_DIR}/cmake_uninstall.cmake"
IMMEDIATE #ONLY)
add_custom_target(uninstall
COMMAND ${CMAKE_COMMAND} -P ${CMAKE_CURRENT_BINARY_DIR}/cmake_uninstall.cmake)
endif()
My cmake version :
cmake version 3.18.0
What is missing in my CMakeLists.txt for generate the container.lib file ?
Faced the same problem, I found the solution here: for Visual Studio to export symbols in a .lib file besides the .dll library, you need to set this in your CMake (version>= 3.4) script:
set(CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS ON)
Note that the .lib file created this way is a small size file and is not a static library.
CMake manual
I'd like to add to the accepted answer here, for others who might find it: although CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS will solve the problem, it could well be a sticking plaster masking a deeper issue going on with your code.
I'm not an expert on the Microsoft compilers, but I do know that if you build a shared library with MSVC, the .lib file should still be produced. However, rather than containing all of your compiled code as it would for a static library, it basically provides the compiler with information about the exported symbols in your shared library. This means that the compiler can automatically link any other targets to your shared library, without you needing to manually load the library from your code using the Windows API functions. If you link an executable to a shared library this way, the Microsoft C runtime will basically call LoadLibrary() for you automatically when your application starts. Useful, huh?
If the compiler does not produce a .lib alongside your shared library, this basically means that there are no exported symbols in the shared library. This is why CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS solves the problem - it forces all the symbols to be exported anyway, thereby causing the compiler to generate the .lib file detailing these exports. However, as you may be able to work out, this is very much the nuclear option. There's probably a lot of stuff in your shared library that really doesn't need to be visible from the outside! So the pertinent question becomes: why is nothing being exported from my library?
In the linked answer referred to previously, underneath all the CMake technicals, the issue was that the OP was not properly marking their symbols for export. It turns out that this was my issue too, but in a more round-about way: I had decided that for my particular library, which could previously be built in shared or static configurations, I now wanted to force it only to be built in a shared configuration. Because of this, I had removed a particular preprocessor definition from my project in CMake which specified whether the build mode was shared or static; this meant that all of the export annotations on my functions compiled down to nothing (as they should do under a static configuration where they are not needed). The upshot was that I accidentally ended up building the shared library with no exported symbols whatsoever, and MSVC just said "Well I guess there's no point building a .lib then", and didn't produce one. This caused the build issues stating that the .lib could not be found on disk.
When I encountered the answer above, I was suspicious as I wondered why I'd not had to set this value before, despite having used CMake to build Windows shared library projects for years. The correct solution in my case was not to switch CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS on - it was actually to remove the C++ preprocessor condition that checked for my "this is a shared build" preprocessor definition. This re-enabled all of the export annotations on my functions, and everything built as it should.
Before you use CMAKE_WINDOWS_EXPORT_ALL_SYMBOLS, do check that there isn't some subtle bug in your scripts that is preventing your symbols from being exported!

Library's CMake generates DLL. Application's CMake wants LIB

My library has minimal, straightforward CMake code with the pertinent lines
add_library(MyLib <sources>)
install(
TARGETS MyLib
LIBRARY DESTINATION ${destination}/lib
RUNTIME DESTINATION ${destination}/lib
COMPONENT Libraries)
install(
FILES mylib.h
DESTINATION ${destination}/include
COMPONENT Headers)
When executed under Windows, the system generates mylib.dll in ...\build\Release, and mylib.lib and mylib.exp (what's that?) in ...\build\lib\Release. It only installs mylib.dll.
My application has minimal, straightforward CMake code to search for my library:
find_path(MyLib_INCLUDE_DIR mylib.h)
find_library(MyLib_LIBRARIES NAMES MyLib)
include(FindPackageHandleStandardArgs)
find_package_handle_standard_args(MyLib DEFAULT_MSG MyLib_LIBRARIES MyLib_INCLUDE_DIR)
Which works under Linux, but under Windows results in
-- Could NOT find MyLib (missing: MyLib_LIBRARIES)
From experimentation I know that this error occurs whenever there is only a .DLL file, and no associated .LIB import library.
Shall I correct MyLib to install mylib.lib? How?
Or is it possible to modify my application so that it is satisfied with mylib.dll only? How?
Research done so far
This is not about static vs dynamic linking (DLL and LIB files - what and why?, cmake link against dll/lib): I want dynamic linking; if a .LIB file is required, it has nothing to do with static linking.
This link cmake : shared library : how to get the .lib name instead of .dll name? may be pertinent, but is not explicit enough. Two other questions CMake generated VS project expecting lib instead of dll, Linking dll/lib to a cmake project seem related, but have no answer.
Command install classifies .lib file for a shared library as ARCHIVE. This is explicitly stated in the documentation:
For DLL platforms (all Windows-based systems including Cygwin), the DLL import library is treated as an ARCHIVE target.
So you need to add ARCHIVE clause to install() for install .lib file as well:
install(
TARGETS MyLib
ARCHIVE DESTINATION ${destination}/lib
LIBRARY DESTINATION ${destination}/lib
RUNTIME DESTINATION ${destination}/bin
COMPONENT Libraries)
Not also, that RUNTIME DESTINATION is usually specified as bin, the same as destination for executables. This helps the executables on Windows to locate the shared libraries (.dll).

Add compile option if shared lib is linked

I have a special compiler that need an extra flag -non-static when linking an executable to with an shared lib:
gcc main.cpp -non-static -lshared -o main
(Note: Its the compiler for vxworks)
How can i do this in CMake?
Solutions i have used so far (but are not satisfying):
Always set -non-static in the toolchain file. Works, but has unwanted side effects when you dont need any shared lib.
set(CMAKE_EXE_LINKER_FLAGS ${CMAKE_EXE_LINKER_FLAGS} "-non-static")
Create a INTERFACE library with -non-static compile option set. However, each new project must rewrite this and and each executable must link this library (if it links any shared lib):
add_library(allwaysLinkMe INTERFACE)
if(CMAKE_SYSTEM_NAME STREQUALS "VxWorks")
target_link_libraries(allwaysLinkMe INTERFACE "-non-static")
endif()
add_library(shared SHARED a.cpp)
add_executatble(main main.cpp)
target_link_libraries(main shared allwaysLinkMe)
Or i link this library to all other shared libraries:
add_library(shared SHARED a.cpp)
target_link_libraries(shared allwaysLinkMe)
add_executatble(main main.cpp)
target_link_libraries(main shared)
In both cases, I must use ifs to prevent this from failing with other target platforms. Also both are pretty ugly and error prone.
I am looking for a solution, that is only in the toolchain/platform/compiler file(s) and no change to the project must be made.

Boost is using pthread but should use win32

I used Cmake gui and the FindBoost module to add Boost as a dependency in my visual studio 2010 c++ project. I set the parameter that tells FindBoost to use the win32 thread library instead of pthread (because I'm running on windows). When i build my code, all the calls to boost libraries compile fine, but boost itself is having a compile error. its failing on
boost\thread\pthread\mutex.hpp line 11
cannot open include file: 'pthread.h': No such file or directory
If I could make it use the win32 library, the mutex.hpp (and all the other files) dont call pthread.h
How do I build so that it uses boost\thread\win32 ?
here is the section of my cmakelists.txt
# include boost
set(BOOST_ROOT "E:/boost_1_58_0/boost_1_58_0")
add_definitions( -DBOOST_ALL_NO_LIB )
find_package(Boost COMPONENTS thread system)
if(Boost_FOUND)
message(STATUS "Boost was found, Success!")
# telling it to use win32 threads not pthreads
set(Boost_THREADAPI "win32")
INCLUDE_DIRECTORIES(${Boost_INCLUDE_DIRS})
LINK_DIRECTORIES(${Boost_LIBRARY_DIRS})
endif()
CMake comes with a module called FindBoost which i used, and it correctly found the boost directory and set the following
Boost_INCLUDE_DIR
E:/boost_1_58_0/boost_1_58_0/
Boost_LIBRARY_DIR
E:/boost_1_58_0/boost_1_58_0/stage/lib
Boost_SYSTEM_LIBRARY_DEBUG
E:/boost_1_58_0/boost_1_58_0/stage/lib/boost_system-vc100-mt-gd-1_58.lib
Boost_SYSTEM_LIBRARY_RELEASE
E:/boost_1_58_0/boost_1_58_0/stage/lib/boost_system-vc100-mt-1_58.lib
Boost_THREAD_LIBRARY_DEBUG
E:/boost_1_58_0/boost_1_58_0/stage/lib/boost_system-vc100-mt-gd-1_58.lib
Boost_THREAD_LIBRARY_RELEASE
E:/boost_1_58_0/boost_1_58_0/stage/lib/boost_thread-vc100-mt-1_58.lib
visual studio project settings
C/C++ -> General -> Additional Include Directories:
...bunch of my other dependencies
E:\boost_1_58_0\boost_1_58_0
Linker -> General -> Additional Library Directories
E:/boost_1_58_0/boost_1_58_0/stage/lib;E:/boost_1_58_0/boost_1_58_0/stage/lib/$(Configuration);%(AdditionalLibraryDirectories)
Linker -> General -> Link Library Dependencies - No
Linker -> General -> Use LIbrary Dependency Inputs - No
Linker -> Input -> Additional Dependencies:
...bunch of my other .libs
boost_thread.lib
boost_system.lib
I fixed the error but I had try what I call the "dumb" way. The "smart" way didnt work. Smart way was reviewing several times the cmakelists code and the visual studio include library settings. i even did a clean and rebuild boost using b2 and manually set threading=multi and link=static because i thought maybe the library just wasnt built correct.
Finally i gave up and tried the dumb way: I cut and pasted the pthread folder and moved it somewhere else, so that it was no longer in the boost/thread folder, only the win32 folder.
I wish i tried it so long ago, because i didnt think that the compilation error would be due to my project's code, but rather an environment setting in the include and linking. But sure enough i got an error showing that in one of my project's files i was directly trying to include that mutex.hpp from the pthread folder. i didnt write this code (its open source) so i couldnt have exactly known, but still i want to slap myself
#include <boost/thread.hpp>
#include <boost/thread/pthread/mutex.hpp>
changing pthread to win32 made the error go away.

Resources