Conan boost dlls not installed on Windows -> runtime exception - boost

I have a conan recipe of a package, named Package, that requires boost as a shared library:
def requirements(self):
self.requires("boost/1.79.0#")
self.options["boost"].shared = True
self.options["boost"].bzip2 = False
self.options["boost"].without_stacktrace = True
The used generators are CMakeDeps and CMakeToolchain.
def generate(self):
tc = CMakeToolchain(self)
tc.variables['BUILD_SHARED_LIBS'] = "ON" if self.options.shared == True else "OFF"
tc.variables['CMAKE_FIND_ROOT_PATH_MODE_PACKAGE'] = 'NEVER'
tc.variables['CMAKE_POSITION_INDEPENDENT_CODE'] = 'ON'
tc.generate()
The unit tests of this conan package use a CMakeLists.txt which defines a PackageTests target that links against Package and boost::boost.
Building the Package and PackageTests works fine for both Ubuntu and Windows but only on Ubuntu the tests run without issues. On Windows I get exceptions for all the tests because the boost dlls are not found. Using ldd PackageTests and readelf -d PackageTests on Ubuntu shows that boost so files are used from the conan cache.
Using conans VirtualRunEnv generator and then activating the generated environement helps to also run the PackageTests.exe on Windows but I would like to know if there is another way using for example pure CMake to install/copy the required boost dlls to the bin/PackageTests.exe folder? Or is there a way to extend the conan recipe to install the dlls on Windows?
Why are boost shared libraries found correctly in the conan cache but not on Windows? Is there some extra manual work needed or shouldn't this be handeled by conan as well?
Edit:
Trying to use the following to copy the dlls results in a cmake command usage error because the TARGET_RUNTIME_DLLS generator expression is empty.
add_custom_command(TARGET PackageTest POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different $<TARGET_RUNTIME_DLLS:PackageTest > $<TARGET_FILE_DIR:PackageTest>
COMMAND_EXPAND_LISTS
)
Also IMPORTED_LOCATION property for the following targets are *-NOTFOUND:
get_target_property(VAR-boost boost::boost IMPORTED_LOCATION)
message(${VAR-boost})
> VAR-boost-NOTFOUND
get_target_property(VAR-Package Package IMPORTED_LOCATION)
message(${VAR-Package})
> VAR-Package-NOTFOUND
get_target_property(VAR-Package PackageTest IMPORTED_LOCATION)
message(${VAR-Package})
> VAR-PackageTest-NOTFOUND
From the boost conan recipe package_info() I can see that the CMakeDeps generator will only create the BoostConfig.cmake and BoostTargets.cmake scripts, which is also the case for Package. There is no FindBoost.cmake generated. By default CMakeDeps only creates config scripts, except the recipes define cmake_find_mode to be both. Not sure though if adding both to the recipe could help. Even if it would help, this is no immediate solution as this is not directly in my control (hosted on conan-center-index repo). I am still not able to see the reason why everything works fine on Ubuntu but on Windows the dlls are not found/copied at all by conan.

Related

How can i setup meson and ninja on Ubuntu-Linux to produce the expected .a file by use of MakeFile?

Some years ago on Ubuntu 16.0.4 I've used this library: git clone https://github.com/Beckhoff/ADS and using only the make command I got build, compile and finally on the main directory I found a file called AdsLib-Linux.a and maybe nothing more than this.
Now I'm on Ubuntu 20.04 I need this library once again but this times make dosn't produce the same output and looking forth to the ReadMe instructions I finally used that instead of make:
meson build
ninja -C build
That now create a new directory build but no .a file as before on the root directory. Instead a new file in the build directory libADSLib.a is there. The same thing happens using right the make command.
Maybe the author changed over the years something on the config files or the behavior of the tools have changed, but I cannot get the former file anymore and I need it for other referencing code that now is not executing anymore.
Looking to the MakeFile I found that in the example folder, differently from the one on the parent directory, the MakeFile has something like that:
$(warning ATTENTION make is deprecated and superseeded by meson)
...
${PROGRAM}: LIB_NAME = ../AdsLib-${OS_NAME}.a
...
But all i've tried reading the guides on meson and ninja about setup, configure, build, and so on, did not produce anymore that file.
I've tried also to first build and then copy all files form the example folder to the parent directory and then build again, but again no .a file there.
How's the right way to configure the build process corectly so that this -Linux.a file is created. Or if not possibile anymore, what does it now produce I can use instead of what produced before?
Meson is a build system generator, similar to CMake or somewhat like ./configure, you need to run meson, then run ninja to actually build something.
You need to run both meson and ninja:
meson setup builddir
ninja -C builddir
Once you do that successfully, there will be a libAdsLib.a inside the builddir directory.
Let me correct a bit #dcbaker, according to their README you should setup build as build directory:
# configure meson to build the library into "build" dir
meson build
# let ninja build the library
ninja -C build
Of course, in general, it shouldn't be specific, but their example code is written in a weird way so this path is hard-coded. So, to use the example:
# configure meson to build example into "build" dir
meson example/build example
# let ninja build the example
ninja -C example/build
# and run the example
./example/build/example
About the library: it's now libAdsLib.a and produced in build directory. The name is set here and it's now in linux naming style, the old one - not. So, you have options:
Update your configuration/build files (Makefile?) where you use it
Copy or make symbolic link, e.g.
$ ln -s <>/build/libAdsLib.a <target_path>/AdsLib-Linux.a
Above it's very dependent on your development environment, do you have installation or setup scripts for it? do you permissions to modify/configure parameters for target application? do you need to support both old and new names? - many questions not related to original question about meson.

How to correctly build cabal project using hmatrix under Windows 10?

Using Windows 10 64-bit, Cabal-3.4.0.0, ghc-8.10.7.
I installed OpenBLAS in MSYS2 environment with command
pacman -S mingw-w64-x86_64-openblas.
Than, I successfully installed hmatrix-0.20.2 with command
cabal install --lib hmatrix --flags=openblas --extra-include-dirs="C:\\ghcup\\msys64\\mingw64\\include\\OpenBLAS" --extra-lib-dirs="C:\\ghcup\\msys64\\mingw64\\bin" --extra-lib-dirs="C:\\ghcup\\msys64\\mingw64\\lib"
I am trying to build simple test project using cabal build cabalhmatrix with Main
module Main where
import Numeric.LinearAlgebra
main :: IO ()
main = do
putStrLn $ show $ vector [1,2,3] * vector [3,0,-2]
But now I am getting output
Resolving dependencies...
Build profile: -w ghc-8.10.7 -O1
In order, the following will be built (use -v for more details):
- hmatrix-0.20.2 (lib) (requires build)
- cabalhmatrix-0.1.0.0 (exe:cabalhmatrix) (first run)
Starting hmatrix-0.20.2 (lib)
Failed to build hmatrix-0.20.2. The failure occurred during the configure
step.
Build log (
C:\cabal\logs\ghc-8.10.7\hmatrix-0.20.2-6dd2e8f2795550e4dd624770ac98c326dacc0cac.log
):
Warning: hmatrix.cabal:21:28: Packages with 'cabal-version: 1.12' or later
should specify a specific version of the Cabal spec of the form
'cabal-version: x.y'. Use 'cabal-version: 1.18'.
Configuring library for hmatrix-0.20.2..
cabal-3.4.0.0.exe: Missing dependencies on foreign libraries:
* Missing (or bad) C libraries: blas, lapack
This problem can usually be solved by installing the system packages that
provide these libraries (you may need the "-dev" versions). If the libraries
are already installed but in a non-standard location then you can use the
flags --extra-include-dirs= and --extra-lib-dirs= to specify where they are.If
the library files do exist, it may contain errors that are caught by the C
compiler at the preprocessing stage. In this case you can re-run configure
with the verbosity flag -v3 to see the error messages.
cabal-3.4.0.0.exe: Failed to build hmatrix-0.20.2 (which is required by
exe:cabalhmatrix from cabalhmatrix-0.1.0.0). See the build log above for
details.
What should I do to correctly build that package?
I guess I need to somehow pass arguments --flags=openblas --extra-include-dirs="C:\\ghcup\\msys64\\mingw64\\include\\OpenBLAS" --extra-lib-dirs="C:\\ghcup\\msys64\\mingw64\\bin" --extra-lib-dirs="C:\\ghcup\\msys64\\mingw64\\lib" to hmatrix during compilation, but don't know how to do that. To be honest, I don't understand for what program exactly are those arguments (cabal, ghc, ghc-pkg or something else) and why cabal is trying to install hmatrix again. I see hmatrix in directory "C:\cabal\store\ghc-8.10.7\hmatrix-0.20.2-e917eca0fc7690010007a19f4f2a3602d86df0f0".
Created cabal.project file:
packages: .
package hmatrix
flags: +openblas
extra-include-dirs: C:\\ghcup\\msys64\\mingw64\\include\\OpenBLAS
extra-lib-dirs: C:\\ghcup\\msys64\\mingw64\\bin, C:\\ghcup\\msys64\\mingw64\\libenter code here
After adding libopenblas.dll location to PATH variable cabal project is working.
Even though there is the --lib flag, it's generally best to work under the assumption that Cabal doesn't do library installs. Never install a library, instead just depend on it – and have Cabal install, update etc. it whenever necessary.
But then how can you pass the necessary flags? With a cabal.project file.
packages: .
package hmatrix
flags: openblas
extra-include-dirs: C:\\ghcup\\msys64\\mingw64\\include\\OpenBLAS
...
Put this file in the working directory of your own project, together with cabalhmatrix.cabal. Then running cabal build in that directory will use a hmatrix install with the suitable library etc. flags.

How to get CMake/CPack to generate multiple packages for different OSs and different architectures within each OS?

Say I have the following basic CMakeLists.txt file.
target_include_directories( addnum
PUBLIC
"${CMAKE_CURRENT_SOURCE_DIR}/include"
)
add_executable(addnumapp src/main.cpp)
target_link_libraries(addnumapp addnum)
SET(CPACK_GENERATOR "DEB")
set(CPACK_DEBIAN_PACKAGE_DEPENDS "g++ (>= 7), libffi-dev, libncurses5-dev, libsqlite3-dev, mcpp, zlib1g-dev")
SET(CPACK_DEBIAN_PACKAGE_MAINTAINER "Some person")
INCLUDE(CPack)
To create a "deb" package, I create a build directory and run the following command to build my project:
cmake -S . -B ./build
I then run the cpack command from within the build directory and a ".deb" package is generated. This is great to generate a single ".deb" package; however, I would like to generate multiple packages using cmake and cpack and I'm unsure what the best way to go about this is.
For example, how do I generate ".deb" packages for all the supported Debian architectures?
From my research, I'm aware of the following command, which allows you to specify an architecture:
set(CPACK_DEBIAN_PACKAGE_ARCHITECTURE, "i386")
However, this only works if you're generating a single Debian package through the method I've outlined above. How does one generate multiple Debian packages for all the different architectures?
My second question is about generating multiple packages for different Operating Systems. For example, the above CMakeLists.txt file just generates a Debian package; however, I would also like to generate packages for MacOS and Windows.
From my research, I'm aware that the following edits to my CMakeLists.txt file should theoretically be the minimal changes required for me to generate a ".dmg" package for MacOS in addition to the Debian package I also want generated.
...
SET(CPACK_GENERATOR "DEB;DragNDrop") # modification here
set(CPACK_DEBIAN_PACKAGE_DEPENDS "g++ (>= 7), libffi-dev, libncurses5-dev, libsqlite3-dev, mcpp, zlib1g-dev")
SET(CPACK_DEBIAN_PACKAGE_ARCHITECTURE i386)
SET(CPACK_DEBIAN_PACKAGE_MAINTAINER "Some person") #required
INCLUDE(CPack)
INCLUDE(CPackDMG) # addition here
However, when I run cmake -S . -B ./build on this modified CMakeLists.txt file (from an Ubuntu OS), I get the following error:
CMake Error at CMakeLists.txt:28 (INCLUDE): INCLUDE could not find
requested file:
CPackDMG
-- Configuring incomplete, errors occurred!
Is this error because I'm trying to generate a MacOS "dmg" package from Ubuntu or is it because I'm missing some cpack dependency?
Further, is this the best way to generate different packages for different platforms?
To summarise, my two questions are (1) how to generate multiple packages for different architectures and (2) how to generate multiple packages for different platforms? (Please bear in mind that I'm very new to C++, CMake and CPack.)
(1) how to generate multiple packages for different architectures
Just use different build environments (hardware, docker containers, virtual machines, ...) or use cross-compilation and toolchain files.
(2) how to generate multiple packages for different platforms?
CPackDMG is an internal CPack module. You don't need to include it. Your CMakeLists.txt should use only CPack and set desired variables before. Set CPACK_GENERATOR to the list of package types you want and the generator-specific variables. However, some generators use external tools -- e.g., to build RPM package your environment requires working rmbuild tool... meaning that it'll be hard to build RPM package within a Debian distro %) In other words, you most likely fail to build DMG when building your package in Linux/Windows :)

boost::filesystem::current_path() returns empty path

I have a C++ program where I need the current path to later create a folder. The location of my executable is, let's say /home/me/foo/bin. This is what I run:
//Here I expect '/home/me/foo/bin/', but get ''
auto currentPath = boost::filesystem::current_path();
//Here I expect '/home/me/foo/', but get ''
auto parentPath = currentPath.parent_path();
//Here I expect '/home/me/foo/foo2/', but get 'foo2/'
string subFolder = "foo2";
string folderPath = parentPath.string() + "/" + subFolder + "/";
//Here I expect to create '/home/me/foo/foo2/', but get a core dump
boost::filesystem::path boostPath{ folderPath};
boost::filesystem::create_directories( boostPath);
I am running on Ubuntu 16.04, using Boost 1.66 installed with the package manager Conan.
I used to run this successfully with a previous version of Boost (1.45 I believe) without using Conan. Boost was just normally installed on my machine. I now get a core dump when running create_directories( boostPath);.
Two questions:
Why isn't current_path() providing me with the actual path, and returns and empty path instead?
Even if current_path() returned nothing why would I still have a core dump even if I run it with sudo? Wouldn't I simply create the folder(s) at root?
Edit:
Running the compiled program, having some cout outputs of the above variables in between the lines rather than using debug mode, normally gives me the following output:
currentPath: ""
parentPath: ""
folderPath: /foo2/
Segmentation fault (core dumped)
But sometimes (about 20% of the times) gives me the following output:
currentPath: "/"
parentPath: "/home/me/fooA�[oFw�[oFw#"
folderPath: /home/me/fooA�[oFw�[oFw#/foo2/
terminate called after throwing an instance of 'boost::filesystem::filesystem_error'
what(): boost::filesystem::create_directories: Invalid argument
Aborted (core dumped)
Edit 2:
Running conan profile show default I get:
[settings]
os=Linux
os_build=Linux
arch=x86_64
arch_build=x86_64
compiler=gcc
compiler.version=5
compiler.libcxx=libstdc++
build_type=Release
[options]
[build_requires]
[env]
There is some discrepance between the libcxx used in the dependencies, and the one that you are using to build your application.
In g++ (linux) there are 2 standard library modes you can use, libstdc++, built without C++11 enabled, and libstdc++11, built with C++11 enabled. When you are building an executable (application or shared library), all the individual libraries linked together must link with the same libcxx.
libstdc++11 was made the default for g++ >= 5, but this also depends on the linux distro. It happens that even if you install a g++ >=5 in older distros like Ubuntu 14, the default libcxx will still be libstdc++, apparently it is not easy to upgrade it without breaking. It also happens that very popular CI services used in open-source, like travis-ci, used older linux distros, and thus libstdc++ linkage was the most popular.
libstdc++ was the default for g++ < 5.
For historical and backwards compatibility reasons, conan default profile always use libstdc++, even for modern compilers in modern distros. You can read your default profile the first time conan is executed, but also find it as a file in .conan/profiles/default, or show it with conan profile show default. This will likely change in conan 2.0 (or even sooner), and the correct libcxx will be detected for each compiler if possible.
So, if you are not changing the default profile (using your own profiles is recommended for production), then when you execute conan install, the depedencies which are installed are built against libstdc++. Note that this conan install is independent on the build in most cases, it just downloads, unzip and configure the dependencies you want, with the requested configuration (from the default profile).
Then, when you are building, if you are not changing _GLIBCXX_USE_CXX11_ABI, then you can be using your system compiler default, in this case, libstdc++11. In most cases, a linking error appears that shows this discrepance. But in your case you were unlucky, and your application managed to link, but then crashed at runtime.
There are a couple of approaches to solve this:
Build your application with libstdc++ too. Make sure to define _GLIBCXX_USE_CXX11_ABI=0.
Install your dependencies for libstdc++11. Edit your default profile to use libstdc++11, then issue a new conan install and rebuild your app.

Get travisCI build environment working for netcdf-dependent Fortran built in R package

The title should have two subtitles:
What is the pathname to any installed libraries in the Travis CI environment?
or
How do I get my Makevars file portable for NetCDF libraries??
Background:
I am developing and R package that is supposed to work with a shared library I have written in Fortran. I want to check my builds with TravisCI, so my package is currently on GitHub.
So upon package installation, the Fortran source code should be compiled. I can do this locally, but TravisCI errors with the following message:
gfortran -fdefault-real-8 -c HANDLE_ERR.f90
HANDLE_ERR.f90:4: Error: Can't open included file 'netcdf.inc'
make: *** [HANDLE_ERR.o] Error 1
Which I understand as the compiler not finding the NetCDF library, which I made sure I installed by adding this to my .travis.yml:
before_install:
- sudo apt-get install libnetcdf-dev -y
Example
I have created a minimal working example, which is failing in TravisCI, with the error message (above) I am getting on my big project.
See here for the travis build https://travis-ci.org/teatree1212/nctest
you can access my minimal working example repository from there, but here is the link as well: https://github.com/teatree1212/nctest/tree/master
The compilation works when I do it locally, as I can specify the NetCDF library directories. I don't know where these are installed in the Travis build environment, so I think this is where my problem lies at the moment.
However, I would like to make this package portable and not only make it work in the travis container. Therefore these two questions:
What is the pathname to any installed libraries in the Travis CI environment?
and more importantly:
How do I make my Makevars file portable for compiling with NetCDF libraries?

Resources