I need to compile boost from sources, it is a requirement. But I only need one library regex for a benchmark.
I have success if I clone the entire boost project with the snippet below. However downloading the entire boost every time is very expensive for the type of usage we have in mind.
However when I uncomment the line # GIT_SUBMODULES I get many errors, in particular complaining about boost-install.jam not existing.
ExternalProject_Add(
libboost
GIT_REPOSITORY "https://github.com/boostorg/boost.git"
# GIT_SUBMODULES tools/build tools/bcp libs/regex libs/config tools/boostdep libs/predef libs/core libs/detail libs/io libs/iterator libs/predef libs/preprocessor libs/smart_ptr libs/throw_exception libs/system libs/filesystem libs/integer tools/build tools/bcp
GIT_SHALLOW ON
GIT_SUBMODULES_RECURSE OFF
BUILD_IN_SOURCE ON
PREFIX ${CMAKE_CURRENT_SOURCE_DIR}
SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/boost
TMP_DIR ${PROJECT_BINARY_DIR}/boost-tmp
STAMP_DIR ${PROJECT_BINARY_DIR}/boost-stamp
#BINARY_DIR ${PROJECT_BINARY_DIR}/boost-build
DOWNLOAD_DIR ${PROJECT_BINARY_DIR}/boost-down
CONFIGURE_COMMAND
cd ${CMAKE_CURRENT_SOURCE_DIR}/boost &&
./bootstrap.sh
--with-toolset=clang
BUILD_COMMAND
cd ${CMAKE_CURRENT_SOURCE_DIR}/boost &&
./b2 headers &&
./b2 install -q
--prefix=${CMAKE_CURRENT_SOURCE_DIR}/local
--build-type=minimal
--layout=system
--disable-icu
--with-regex
--with-system
variant=release link=static runtime-link=static
threading=single address-model=64 architecture=x86
toolset=clang
INSTALL_COMMAND ""
)
So the question is: what is the minimal number of submodules that I need to download/clone in order to make boost::regex available?
Thank you for your help!
[6/54] Performing build step for 'libboost'
FAILED: boost-stamp/libboost-build /home/hbucher/git/regex-performance/build/boost-stamp/libboost-build
cd /home/hbucher/git/regex-performance/vendor/boost && cd /home/hbucher/git/regex-performance/vendor/boost && ./b2 headers && ./b2 install -q --prefix=/home/hbucher/git/regex-performance/vendor/local --build-type=minimal --layout=system --disable-icu --with-regex variant=release link=static runtime-link=static threading=single address-model=64 architecture=x86 toolset=clang && /usr/local/bin/cmake -E touch /home/hbucher/git/regex-performance/build/boost-stamp/libboost-build
[errno 2] boost-install.jam (No such file or directory)
Jamroot:308: in boost-install
ERROR: rule "boost-install.boost-install" unknown in module "Jamfile</home/hbucher/git/regex-performance/vendor/boost>".
libs/filesystem/build/Jamfile.v2:171: in modules.load
Related
I'm trying to build libxc-4.3.4 in an MSYS2 shell on Windows 10. I've installed the latest version of MSYS2 (msys2-x86_64-20220319.exe) and followed the installation instructions. I've installed build tools using
pacman -S --needed base-devel mingw-w64-x86_64-toolchain autoconf
I've installed libxc dozens of time on Linux machines. The first step is
./configure --prefix /somewhere
But in MSYS2 I get
$ ./configure --prefix $PWD/../libxc
bash: ./configure: No such file or directory
How can I make this work?
MSYS2 prerequisites
First of all make sure MSYS2 has all programs that are needed.
In the MSYS2 shell first update the package manager information:
pacman -Syu --noconfirm
Then install the packages you need. I would recommend at least these:
pacman -S --noconfirm autoconf autoconf-archive automake make libtool pkg-config
Project sources
Next you should make sure the folder you are in actually has a configure script:
ls -l configure
A lot of projects these days are switching to more efficient build systems like CMake or Meson.
I usually use the following command in the projects source folder to check for several build systems:
ls -ld configure* m4 CMakeLists.txt cmake Makefile GNUmakefile setup.py scons SConscript SConstruct meson.build meson_options.txt *.pro *.proj *.sln BUILD.gn .gn 2> /dev/null
building libxc
For the libxc project I see there is a CMakeLists.txt file and also a configure.ac file.
So either you should look into using CMake or generate the configure file with:
touch README ChangeLog
autoreconf -f -i -I m4
I have just tried to build libxc in MSYS2 with CMake and Ninja and this worked:
# set the line below to the desired install location
INSTALLPREFIX=D:\Prog\changeme
# build static library
cmake -Wno-dev -GNinja -DCMAKE_INSTALL_PREFIX:PATH=$INSTALLPREFIX -DCMAKE_BUILD_TYPE:STRING=Release -DBUILD_SHARED_LIBS:BOOL=OFF -DENABLE_PYTHON:BOOL=OFF -DBUILD_TESTING:BOOL=OFF -S. -Bbuild_static &&
ninja -Cbuild_static install/strip &&
echo SUCCESS
# build shared library
cmake -Wno-dev -GNinja -DCMAKE_INSTALL_PREFIX:PATH=$INSTALLPREFIX -DCMAKE_BUILD_TYPE:STRING=Release -DBUILD_SHARED_LIBS:BOOL=ON -DENABLE_PYTHON:BOOL=OFF -DBUILD_TESTING:BOOL=OFF -S. -Bbuild_shared &&
ninja -Cbuild_shared install/strip &&
echo SUCCESS
I need to compile boost-python 1.73.0 with MinGW64 on Fedora's rawhide, so that I can build Python bindings for Windows 10 64bit (no, I can't use Visual Studio unfortunately). And unfortunately the only library missing from Fedora's mingw64-boost package is boost-python, argggh!
Actually all dependencies are packaged on Fedora for MinGW64, which is quite nice. So I'm trying to build myself with:
wget https://dl.bintray.com/boostorg/release/1.73.0/source/boost_1_73_0.tar.bz2
tar -xjf boost_1_73_0.tar.bz2 && cd boost_1_73_0
./bootstrap.sh
# in a docker container, so /root
./b2 --with-python address-model=64 target-os=windows toolset=gcc-mingw64 \
variant=release link=shared --user-config=/root/user-config.jam
And a ~/user-config.jam:
using gcc : mingw64 : x86_64-w64-mingw32-g++ ;
using python
: 3.9
: /usr/x86_64-w64-mingw32/sys-root/mingw/bin/python3.9.exe
: /usr/x86_64-w64-mingw32/sys-root/mingw/include/python3.9
: /usr/x86_64-w64-mingw32/sys-root/mingw/lib/python3.9 ;
It runs for a while successfully, but then errors out on
/usr/lib/gcc/x86_64-w64-mingw32/10.2.1/../../../../x86_64-w64-mingw32/bin/ld: cannot find -lutil
collect2: error: ld returned 1 exit status
So libutil is missing somehow. What a super specific name! I tried (re-)installing all glibc packages (where libutil.so apparently usually is in), to no avail. Also found a /usr/x86_64-w64-mingw32/sys-root/mingw/lib/libutildll.a, tried including it via ./b2 library=..., again no luck.
What's going on here? BTW, this gist has the full command which is failing.
So, this needed a patch to the boost sources to get rid of the libutil linking. To compile fully, a batch of other patch files had to be applied. Here's my full command chain in fedora:rawhide:
Get the sources
wget https://dl.bintray.com/boostorg/release/1.73.0/source/boost_1_73_0.tar.bz2
tar -xjf boost_1_73_0.tar.bz2
cd boost_1_73_0
Setup ~/user-config.jam
cat >> ~/user-config.jam <<EOL
using gcc : mingw64 : x86_64-w64-mingw32-g++ ;
using python
: 3.9
: /usr/x86_64-w64-mingw32/sys-root/mingw/bin/python3.9
: /usr/x86_64-w64-mingw32/sys-root/mingw/include/python3.9
: /usr/x86_64-w64-mingw32/sys-root/mingw/lib/python3.9/config-3.9 ;
EOL
Bootstrap boost
./bootstrap.sh --with-icu=/usr/x86_64-w64-mingw32/sys-root/mingw/ --with-toolset=gcc
Patch all files in ./boost_1_73_0_patches from
the single patch in this repo (git clone for source): https://aur.archlinux.org/packages/mingw-w64-boost-python/
all patch files from this repo: https://github.com/msys2/MINGW-packages/tree/master/mingw-w64-boost
Mostly apply like this:
patch -p1 -i patches/boost-1.63.0-python-test-PyImport_AppendInittab.patch
Build boost-python
./b2 address-model=64 link=shared runtime-link=shared threading=multi threadapi=win32 toolset=gcc variant=release python=3.9 --with-python
I'm using libtool (version 2.4.6) for creating a simple shared library
under GNU/Linux.
In my specific case, portability is not as important as simplicity: I don't
want to use the whole Autotools suite, nor CMake. I just want a simple
Makefile which can compile a shared library and install it properly.
I've got a couple of variables definitions, which follow the
conventions of GNU Make
prefix ?= /usr/local
exec_prefix ?= $(prefix)
libdir ?= $(exec_prefix)/lib
I want to support an install target, and I believe it should be roughly
like this:
LIBTOOL_LIB := libxyz.la
install: target = $(abspath $(DESTDIR))
install: all
mkdir -p $(target)/$(libdir)
libtool --mode=install install $(LIBTOOL_LIB) $(target)/$(libdir)
libtool --mode=finish $(target)/$(libdir)
If I run make install DESTDIR=./test I get the following message:
mkdir -p /yadayada/src/test//usr/local/lib
libtool --mode=install install libtrullallero.la /yadayada/src/test//usr/local/lib
libtool: install: install .libs/libtrullallero.so.0.0.0 /yadayada/src.so.0.0.0
libtool: install: (cd /yadayada/src.so.0; }; })
libtool: install: (cd /yadayada/src.so; }; })
libtool: install: install .libs/libtrullallero.lai /yadayada/src.la
libtool: install: install .libs/libtrullallero.a /yadayada/src.a
libtool: install: chmod 644 /yadayada/src.a
libtool: install: ranlib /yadayada/src.a
libtool: warning: remember to run 'libtool --finish /usr/local/lib'
libtool --mode=finish /yadayada/src/test//usr/local/lib
libtool: finish: PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/sbin" ldconfig -n /yadayada/src/test//usr/local/lib
----------------------------------------------------------------------
Libraries have been installed in:
/yadayada/src/test//usr/local/lib
If you ever happen to want to link against installed libraries
in a given directory, LIBDIR, you must either use libtool, and
specify the full pathname of the library, or use the '-LLIBDIR'
flag during linking and do at least one of the following:
- add LIBDIR to the 'LD_LIBRARY_PATH' environment variable
during execution
- add LIBDIR to the 'LD_RUN_PATH' environment variable
during linking
- use the '-Wl,-rpath -Wl,LIBDIR' linker flag
- have your system administrator add LIBDIR to '/etc/ld.so.conf'
See any operating system documentation about shared libraries for
more information, such as the ld(1) and ld.so(8) manual pages.
----------------------------------------------------------------------
I guess I deserve this notification, since I'm installing outside the
regular library path. This is by the way the same message I would get in a
Autotools-based project, should I set a --prefix in some arbitrary path at
./configure.
There are however some weird things:
If I run make install as root (within a container, of course), without
defining DESTDIR, I get the same kind of message. This does not feel
correct: this time I did not deserve it.
With or without DESTDIR, a call to ldconfig makes in my opinion no
sense at this point. And when I'm not running it as root, it should be
also ineffective, yet the program succeeds.
Lurking in the libtool documentation, I've found a
reference to the -inst-prefix-dir option,
which should allow me to have a DESTDIR-like behaviour.
Perhaps this is what I do wrong. However it behaves weirdly!
For starters, it seems to work only if specified at the end of the
--mode=install line. In fact, if I do this:
install: target = $(abspath $(DESTDIR))
install: all
mkdir -p $(target)/$(libdir)
libtool --mode=install install -inst-prefix-dir $(target) $(LIBTOOL_LIB) $(target)/$(libdir)
libtool --mode=finish $(target)/$(libdir)
…I clearly get a messy command line:
libtool: install: install -inst-prefix-dir /yadayada/libtrullallero/test/usr/local/lib/-inst-prefix-dir
install: invalid option -- 'i'
If I move -inst-prefix-dir to the end of the line, I just get another
wrong call to install:
mkdir -p /yadayada/src/libtrullallero/test//usr/local/lib
libtool --mode=install install libtrullallero.la /yadayada/src/libtrullallero/test
libtool: install: install .libs/libtrullallero.so.0.0.0 /yadayada/src/libtrullallero/test/libtrullallero.so.0.0.0
libtool: install: (cd /yadayada/src/libtrullallero/test && { ln -s -f libtrullallero.so.0.0.0 libtrullallero.so.0 || { rm -f libtrullallero.so.0 && ln -s libtrullallero.so.0.0.0 libtrullallero.so.0; }; })
libtool: install: (cd /yadayada/src/libtrullallero/test && { ln -s -f libtrullallero.so.0.0.0 libtrullallero.so || { rm -f libtrullallero.so && ln -s libtrullallero.so.0.0.0 libtrullallero.so; }; })
libtool: install: install .libs/libtrullallero.lai /yadayada/src/libtrullallero/test/libtrullallero.la
libtool: install: install /yadayada/src/libtrullallero/test/lib
install: omitting directory '/yadayada/src/libtrullallero/test//usr/local/lib'
Makefile:22: recipe for target 'install' failed
make: *** [install] Error 1
I must be doing something wrong, but I cannot spot it. Any hint?
I know how to install Boost with the b2 command and specify only to install the filesystem library. The Boost directory has many contents that I won't need, and it takes a long time to clone all submodules. I have this cmake file to download, build and install Boost. But I don't know how to specify which submodules I want to clone in the CMakeLists.txt. I'm using ExternalProject_add to download, build, and install Boost.
cmake_minimum_required(VERSION 2.8)
include(ExternalProject)
# Download boost from git
SET (BOOST_INCLUDE_DIRS ${CMAKE_CURRENT_BINARY_DIR}/boost/src/boost/lib/filesystem/include/boost/)
SET (BOOST_URL https://github.com/boostorg/boost.git )
get_filename_component(BOOST_STATIC_LIBRARIES ${CMAKE_CURRENT_BINARY_DIR}/lib/libboost_filesystem.a ABSOLUTE)
if ( UNIX )
SET (BOOST_STATIC_LIBRARIES ${BOOST_BUILD}/libboost_filesystem.a)
endif ()
ExternalProject_Add(
boost
PREFIX ${CMAKE_CURRENT_BINARY_DIR}/boost
GIT_REPOSITORY ${BOOST_URL}
UPDATE_COMMAND ./bootstrap.sh --with-libraries=filesystem,system
CONFIGURE_COMMAND ""
BUILD_COMMAND ./b2 link=static install --exec-prefix=${CMAKE_BINARY_DIR}/lib/ --includedir=${CMAKE_BINARY_DIR}/include/boost/
BUILD_IN_SOURCE 1
INSTALL_COMMAND ""
)
set(BOOST_LIBRARIES ${CMAKE_CURRENT_BINARY_DIR}/lib)
message ("${BOOST_STATIC_LIBRARIES}")
EDIT
I now have the following CMakeLists.txt file:
cmake_minimum_required(VERSION 2.8)
include(ExternalProject)
# Download boost from git
SET (BOOST_INCLUDE_DIRS ${CMAKE_CURRENT_BINARY_DIR}/boost/src/boost/lib/filesystem/include/boost/)
SET (BOOST_URL https://github.com/boostorg/boost.git )
get_filename_component(BOOST_STATIC_LIBRARIES ${CMAKE_CURRENT_BINARY_DIR}/lib/libboost_filesystem.a ABSOLUTE)
if ( UNIX )
SET (BOOST_STATIC_LIBRARIES ${BOOST_BUILD}/libboost_filesystem.a)
endif ()
ExternalProject_Add(
boost
PREFIX ${CMAKE_CURRENT_BINARY_DIR}/boost
GIT_REPOSITORY ${BOOST_URL}
GIT_SUBMODULES libs/assert libs/utility libs/config libs/predef libs/system libs/detail libs/filesystem tools/boostdep tools/build tools/bcp tools/boostbook tools/inspect tools/litre tools/quickbook tools/auto_index
UPDATE_COMMAND ./bootstrap.sh --with-libraries=filesystem,system
CONFIGURE_COMMAND ./b2 headers
BUILD_COMMAND ./b2 link=static install
CMAKE_ARGS -DCMAKE_INSTALL_PREFIX=${CMAKE_BINARY_DIR}/lib
BUILD_IN_SOURCE 1
INSTALL_COMMAND ""
)
set(BOOST_LIBRARIES ${CMAKE_CURRENT_BINARY_DIR}/lib)
message ("${BOOST_STATIC_LIBRARIES}")
But I get this error message:
...patience...
...found 925 targets...
...updating 337 targets...
common.mkdir /usr/local/include/boost
mkdir: Unable to create directory "/usr/local/include/boost": Permission denied.
mkdir -p "/usr/local/include/boost"
...failed common.mkdir /usr/local/include/boost...
But in the cmake file I use this:
CMAKE_ARGS -DCMAKE_INSTALL_PREFIX=${CMAKE_BINARY_DIR}/lib
To install in a different directory than root.
But it does not seem to be working.
EDIT 2:
I got how to install in a different directory than root. But it seems that has another problem, because the same problem than before happens. What can I do?
I added this:
UPDATE_COMMAND ./bootstrap.sh --with-libraries=filesystem,system --includedir=${CMAKE_BINARY_DIR}/include/
cmake_minimum_required(VERSION 2.8)
include(ExternalProject)
# Download boost from git
SET (BOOST_INCLUDE_DIRS ${CMAKE_CURRENT_BINARY_DIR}/boost/src/boost/lib/filesystem/include/boost/)
SET (BOOST_URL https://github.com/boostorg/boost.git )
get_filename_component(BOOST_STATIC_LIBRARIES ${CMAKE_CURRENT_BINARY_DIR}/lib/libboost_filesystem.a ABSOLUTE)
if ( UNIX )
SET (BOOST_STATIC_LIBRARIES ${BOOST_BUILD}/libboost_filesystem.a)
endif ()
ExternalProject_Add(
boost
PREFIX ${CMAKE_CURRENT_BINARY_DIR}/boost
GIT_REPOSITORY ${BOOST_URL}
GIT_SUBMODULES libs/assert libs/utility libs/config libs/predef libs/system libs/detail libs/filesystem tools/boostdep tools/build tools/bcp tools/boostbook tools/inspect tools/litre tools/quickbook tools/auto_index
UPDATE_COMMAND ./bootstrap.sh --with-libraries=filesystem,system --includedir=${CMAKE_BINARY_DIR}/include/
CONFIGURE_COMMAND ./b2 headers
BUILD_COMMAND ./b2 link=static install
CMAKE_ARGS -DCMAKE_INSTALL_PREFIX=${CMAKE_BINARY_DIR}/lib
BUILD_IN_SOURCE 1
INSTALL_COMMAND ""
)
set(BOOST_LIBRARIES ${CMAKE_CURRENT_BINARY_DIR}/lib)
message ("${BOOST_STATIC_LIBRARIES}")
How I solved my problem:
Download the specifics submodules using the
GIT_SUBMODULES option
Install the boost in a different directory using and only install the statically version
BUILD_COMMAND ./b2 link=static install --libdir=${CMAKE_BINARY_DIR}/lib
And also: installed the specifics binaries using:
UPDATE_COMMAND ./bootstrap.sh --with-libraries=filesystem,system,regex,program_options --includedir=${CMAKE_BINARY_DIR}/include
cmake_minimum_required ( VERSION 2.8.7 )
# Download boost from git
include(ExternalProject)
if ( UNIX )
if (NOT Boost)
SET (Boost_INCLUDE_DIRS ${CMAKE_CURRENT_BINARY_DIR}/include/boost/)
SET (Boost_LIBRARIES ${CMAKE_CURRENT_BINARY_DIR}/lib )
SET (BOOST_URL https://github.com/boostorg/boost.git )
SET (BOOST_BUILD ${CMAKE_CURRENT_BINARY_DIR})
SET (FILESYSTEM_STATIC_LIBRARIES ${BOOST_BUILD}/lib/libboost_filesystem.a)
SET (SYSTEM_STATIC_LIBRARIES ${BOOST_BUILD}/lib/libboost_system.a)
SET (REGEX_STATIC_LIBRARIES ${BOOST_BUILD}/lib/libboost_regex.a)
SET (PROGRAM_OPTIONS_STATIC_LIBRARIES ${BOOST_BUILD}/lib/libboost_program_options.a)
ExternalProject_Add( Boost
PREFIX Boost
GIT_REPOSITORY ${BOOST_URL}
GIT_SUBMODULES libs/asio libs/date_time libs/config libs/core libs/detail libs/io libs/iterator libs/predef libs/preprocessor libs/smart_ptr libs/throw_exception libs/system libs/filesystem libs/integer tools/build tools/bcp libs/serialization libs/interprocess libs/tokenizer libs/algorithm libs/program_options libs/regex libs/static_assert libs/thread libs/utility libs/numeric libs/range libs/array libs/bind libs/concept_check libs/lexical_cast libs/math libs/functional libs/assert libs/type_traits libs/mpl libs/move libs/container libs/any libs/type_index libs/function
UPDATE_COMMAND ./bootstrap.sh --with-libraries=filesystem,system,regex,program_options --includedir=${CMAKE_BINARY_DIR}/include/
CONFIGURE_COMMAND ./b2 headers
BUILD_COMMAND ./b2 link=static install --libdir=${CMAKE_BINARY_DIR}/lib
BUILD_IN_SOURCE 1
INSTALL_COMMAND ""
)
ADD_LIBRARY (Boost_LIB STATIC IMPORTED DEPENDS Boost)
SET_TARGET_PROPERTIES (Boost_LIB PROPERTIES IMPORTED_LOCATION ${FILESYSTEM_STATIC_LIBRARIES})
SET_TARGET_PROPERTIES (Boost_LIB PROPERTIES IMPORTED_LOCATION ${SYSTEM_STATIC_LIBRARIES})
SET_TARGET_PROPERTIES (Boost_LIB PROPERTIES IMPORTED_LOCATION ${REGEX_STATIC_LIBRARIES})
SET_TARGET_PROPERTIES (Boost_LIB PROPERTIES IMPORTED_LOCATION ${PROGRAM_OPTIONS_STATIC_LIBRARIES})
endif()
endif()
I'm trying to do a boost source build, via:
git clone --recursive https://github.com/boostorg/boost.git
cd boost
./bootstrap
./b2 link=shared threading=multi variant=release --without-mpi
This chokes with the following error message:
error: Name clash for '<pstage/lib>libboost_system.so.1.58.0'
error:
error: Tried to build the target twice, with property sets having
error: these incompabile properties:
error:
error: - none
error: - <address-model>64 <architecture>x86
error:
error: Please make sure to have consistent requirements for these
error: properties everywhere in your project, especially for install
error: targets.
This occurs on both the develop and master branch. What can be done to fix this error? Thanks in advance.
From the Boost 1.58 beta release notes:
Important Note
There is a bug with the build scripts; you have to specify the
address-mode and architecture to b2. I used:
./b2 address-model=64 architecture=x86
to test this.
Adding these flags to the b2 command solves the problem without having to exclude the context and coroutine libraries (handy if, say, you actually use these libraries, like I do!).
Naturally, if you're building 32-bit libraries, you want to add address-model=32 instead.
This is a bug in the current git master.
As a workaround, explicitly state the address-model and architecture options on the command line:
./b2 link=shared threading=multi variant=release --without-mpi address-model=64 architecture=x86
I was able to build using the suggestion at https://stackoverflow.com/a/27885628/200985 . I'm compiling branch boost-1.57.0, and I started compiling branch boost-1.56.0, and it got past this point, too. To sum up, I ran
git co boost-1.57.0;
./bootstrap.sh --prefix=/home/me/builds/development;
./b2 --prefix=/home/me/builds/development -j9 --without-context --without-coroutine;