How do I properly invoke MSYS2 Bash commands using CMake execute_process()? - bash

Problem Description
I am having trouble setting up a CMake external_process() command that executes a MSYS2 bash command. When I am in the MSYS2 shell if I run the command $ bash -v ./bootstrap.sh the command works correctly. But if I run the CMake script in the MSYS2 shell using $ cmake -P Run_bash_command.cmake the command errors out part way through the process. An important piece of information I found in the CMake Documentation makes me think that I am not invoking bash correctly or missing an environmental variable:
CMake executes the child process using operating system APIs directly. All arguments are passed VERBATIM to the child process. No intermediate shell is used, so shell operators such as > are treated as normal arguments.
I would like to be able to do this command using CMake if possible as this problem is part of a much larger CMake superbuild project. If there is another approach to solving the problem I am open to suggestions as long as I can include it into the automation of the superbuild project. Any help will be appreciated.
Run_bash_command.cmake contents:
SET( ENV{MSYSTEM} MINGW64 )
SET( DIR_CONTAINING_BOOTSTRAP_SH C:/bash_test )
SET( BASH_COMMAND_TO_RUN bash -v ./bootstrap.sh )
EXECUTE_PROCESS( COMMAND ${BASH_COMMAND_TO_RUN}
WORKING_DIRECTORY ${DIR_CONTAINING_BOOTSTRAP_SH} RESULT_VARIABLE command_result )
IF( NOT "${command_result}" STREQUAL "0" )
MESSAGE( FATAL_ERROR "Error: command_result='${command_result}'" )
ENDIF()
Environment Setup
I followed the directions to setup MSYS2 64bit and added the mingw-w64 toolchain as well as cmake using the command pacman -S base-devel git mingw-w64-x86_64-cmake mingw-w64-x86_64-toolchain
To run the commands I use the MinGW-w64 Win64 Shell that is installed with MSYS2
The file bootstrap.sh comes from the libusb github repository and the folder c:/bash_test contains a clone of the master branch
Output
$ bash -v ./bootstrap.sh output:
$ bash -v ./bootstrap.sh
#!/bin/sh
if ! test -d m4 ; then
mkdir m4
fi
autoreconf -ivf || exit 1
autoreconf: Entering directory `.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal --force -I m4
autoreconf: configure.ac: tracing
autoreconf: running: libtoolize --copy --force
libtoolize: putting auxiliary files in '.'.
libtoolize: copying file './ltmain.sh''
...<clipped output due to length>...
configure.ac:29: installing './install-sh'
configure.ac:29: installing './missing'
examples/Makefile.am: installing './depcomp'
autoreconf: Leaving directory `.'
$ cmake -P Run_bash_command.cmake output:
$ cmake -P Run_bash_command.cmake
#!/bin/sh
if ! test -d m4 ; then
mkdir m4
fi
autoreconf -ivf || exit 1
autoreconf: Entering directory `.'
autoreconf: configure.ac: not using Gettext
autoreconf: running: aclocal --force -I m4
aclocal-1.15: error: aclocal: file '/msys64/usr/share/aclocal/xsize.m4' does not exist
autoreconf: aclocal failed with exit status: 1
CMake Error at Run_bash_command.cmake:10 (MESSAGE):
Error: command_result='1'
Things I have tried:
Substituting bash -l -c but this causes the shell to default to the home directory and then it is unable to find the file bootstrap.sh
Verified the correct version of bash is found by inspecting my environmental PATH varaiable
Verified MSYS2 and its packages are up to date
Using sh instead of bash
Calling autoreconf -ivf directly, but the same issue occurs
Using Unix style paths instead of Windows style

I was able to fix the issue using the code below.
Run_bash_command.cmake contents:
SET( DIR_CONTAINING_BOOTSTRAP_SH /C/bash_test )
SET( BASH_COMMAND_TO_RUN bash -l -c "cd ${DIR_CONTAINING_BOOTSTRAP_SH} && sh ./bootstrap.sh" )
EXECUTE_PROCESS( COMMAND ${BASH_COMMAND_TO_RUN}
WORKING_DIRECTORY ${DIR_CONTAINING_BOOTSTRAP_SH} RESULT_VARIABLE command_result )
IF( NOT "${command_result}" STREQUAL "0" )
MESSAGE( FATAL_ERROR "Error: command_result='${command_result}'" )
ENDIF()
Important Notes:
Using bash -l causes the shell to default to the home directory, to get around this I added a change directory cd <path> command to get us back to the directory where we want to issue the bash command.
Using -c causes bash to read a command from a string. Since we want to issue two commands, one for change directory and one for running the shell script, we need to use && to chain the commands together as well as use "quotation marks" to make sure the entire command is read properly as a string.

Related

How to extract all file in a .tar.gz to specific folder with cmake add_custom_target?

To decrease library size, used cmake -E tar "zcvf" "lib.tar.gz" out.lib in cmake to automatically create .tar.gz then lib.tar.gz is moved to folder f, ie. f/lib.tar.gz.
However, to use this library in another project, it's needed to extract to specific folder. The command without cmake is clear: tar -xzf f/lib.tar.gz -C target-folder.
The problem is that, in cmake, how to combine cmake -E tar "xzf" f/lib.tar.gz with -C option? Commands such as cmake -E tar "xzf" f/lib.tar.gz -C f/ will result error.
Ideal script would be sth. like
add_custom_target(PrepareLib
COMMAND ${CMAKE_COMMAND} -E tar "xzf" command....
WORKING_DIRECTORY ${CMAKE_SOURCE_DIR} ## note, this line must not be modified
)
Tools version: Visual Studio 2017, Windows 10 and CMake 3.17.1.

libtoolize: error: cannot list files

I was installing libtool 2.4.6 and ran into the following problem:
libtool: Version mismatch error. This is libtool 2.4.6, revision 2.4.6,
libtool: but the definition of this LT_INIT comes from revision .
libtool: You should recreate aclocal.m4 with macros from revision 2.4.6
libtool: of libtool 2.4.6 and run autoconf again.
Makefile:1261: recipe for target 'libltdl/loaders/libltdl_libltdl_la-preopen.lo' failed
In an effort to fix this issue and after looking around on google, I executed the following command to recreate aclocal.m4. however, I got "cannot list files" error:
[server]# autoreconf --install
libtoolize: error: cannot list files: '/var/tmp/work/libtool-2.4.6.i386/share/libtool/build-aux'
autoreconf: libtoolize failed with exit status: 1
the directory was there and can be listed with ls command:
[server]# ls -l /var/tmp/work/libtool-2.4.6.i386/share/libtool/build-aux
total 0
Any idea?
this was cause by the line in libtoolize script:
test -n "`{ cd $my_dir && ls; } 2>/dev/null`" \
|| func_fatal_error "cannot list files: '$my_dir'"
I don't know why it interpret a empty directory as "cannot list files"
the issue can be overcome by replacing the above line with:
cd $my_dir && ls || func_fatal_error "cannot list files: '$my_dir'"

How to create symlinks in a specific directory

I'm working on an automated installation of a openSUSE system using AutoYAST, and I'm stumped on a small detail. In order to setup relevant applications in the user's environment, I try to symlink to all applications located in /usr/local/bin in ~/bin (so say /usr/local/bin has the addr2line utility, then I want to have a symlink to that in ~/bin).
I've tried to execute the following snipped to accomplish this:
su -c "for program in `ls /usr/local/bin`; do ln -s /usr/local/bin/$program ~/bin/$program; done" <user>
This snippet executes in the post-script phase of the automatic installation, which is executed as root (and seeing as I want the owner of the symlinks to be the user, this command is executed using su).
However, this does not work, and gives the following output:
++ ls /usr/local/bin
+ su -c 'for program in addr2line
ar
as
c++
c++filt
cpp
elfedit
g++
gcc
gcc-ar
gcc-nm
gcc-ranlib
gcov
gprof
i686-pc-linux-gnu-c++
i686-pc-linux-gnu-g++
i686-pc-linux-gnu-gcc
i686-pc-linux-gnu-gcc-4.9.3
i686-pc-linux-gnu-gcc-ar
i686-pc-linux-gnu-gcc-nm
i686-pc-linux-gnu-gcc-ranlib
ld
ld.bfd
nm
objcopy
objdump
ranlib
readelf
size
strings
strip; do ln -s /usr/local/bin/ ~/bin/; done' <user>
bash: -c: line 1: syntax error near unexpected token `ar'
bash: -c: line 1: `ar'
I've tried several variations of the command, but all seem to not exactly do what I want.
For example, I've also tried:
su -c "for program in /usr/local/bin/*; do ln -s $program ~/bin/; done" <user>
But this only created a symlink to /usr/local/bin in ~/bin.
So I'm a bit stuck on this one... Does anybody have an idea?
You're using double quotes to define your su command, so $program is being evaluated immediately. You want it evaluated when su executes the command. Use single quotes instead:
su -c 'for program in `ls /usr/local/bin`; do ln -s /usr/local/bin/$program ~/bin/$program; done' <user>
You can also use cp -s to create symlinks on a system with GNU cp (like your suse system), which gives you the ability to use recursion and the other fun options of cp.
In the end, I decided to go with the command posted by pacholik to fix this, as my original attempt was over-engineered and thus not necessary.
ln -s /usr/local/bin/* ~/bin

Use shell command as INSTALL_COMMAND to ExternalProject_Add

Is it possible to use any shell command for the INSTALL_COMMAND phase of cmake's ExternalProject_Add? e.g.
ExternalProject_Add(leveldb
GIT_REPOSITORY git#github.com:google/leveldb.git
GIT_TAG v1.18
CONFIGURE_COMMAND ./build_detect_platform build.settings .
BUILD_COMMAND make -j 8
BUILD_IN_SOURCE 1
INSTALL_COMMAND ""
)
# INSTALL_COMMAND "mkdir -p ${CMAKE_BINARY_DIR}/lib/ \
# && find . \( -name \"*${CMAKE_SHARED_LIBRARY_SUFFIX}\" -or -name \"*${CMAKE_STATIC_LIBRARY_SUFFIX}\" \
# -exec cp {} ${CMAKE_BINARY_DIR}/lib/\;\) \
# && cp -r ./include ${CMAKE_BINARY_DIR}")
I commented out the INSTALL_COMMAND I want to use, find and cp don't seem to be allowed, "No such file or directory, error 127" is the result of using this.
Direct Answer:
install(CODE "execute_process(...)")
The SCRIPT and CODE signature:
install([[SCRIPT <file>] [CODE <code>]] [...])
The SCRIPT form will invoke the given CMake script files during installation. If the script file name is a relative path it will be interpreted with respect to the current source directory. The CODE form will invoke the given CMake code during installation. Code is specified as a single argument inside a double-quoted string. For example, the code
install(CODE "MESSAGE(\"Sample install message.\")")
will print a message during installation.
Food for thought:
You might be in a hurry to get something done quickly. Do consider this if you have time or you can get back to it and fix it.
One of the main reasons why people love cmake is because of it's cross platform nature. Any project when properly coded with this aspect in mind will work with either on Linux or Windows or any other operating supported system. The various generators will work happily if the developer had this in mind. My suggestion is to convert the shell commands into cmake in a cross platform way, put them in a separate *.cmake file and execute them using cmake -E option.
Here is an extract from a working project that I had worked on in the past.
project_build_steps.cmake
message(VAR1=${VAR1}) # These variables can be passed from the invocation place
message(VAR2=${VAR2}) # You can use them in the build steps
if("${BUILD_STEP}" STREQUAL "patch")
message("BUILD_STEP: patch")
# Put your patch steps using cmake
endif()
if("${BUILD_STEP}" STREQUAL "configure")
message("BUILD_STEP: configure")
# Put your configure steps using cmake
endif()
if("${BUILD_STEP}" STREQUAL "build")
message("BUILD_STEP: build")
# Put your build steps using cmake
endif()
if("${BUILD_STEP}" STREQUAL "install")
message("BUILD_STEP: install")
# Put your install steps using cmake
endif()
CMakeLists.txt (Option 1)
set(CMAKE_COMMAND /usr/bin/cmake)
set(PROJECT_BUILD_STEPS_FILE project_build_steps.cmake)
ExternalProject_Add(
project_name
SOURCE_DIR /path/to/project/source
PATCH_COMMAND ${CMAKE_COMMAND} -DBUILD_STEP=patch -P ${PROJECT_BUILD_STEPS_FILE}
CONFIGURE_COMMAND ${CMAKE_COMMAND} -DBUILD_STEP=configure -P ${PROJECT_BUILD_STEPS_FILE}
BUILD_COMMAND ${CMAKE_COMMAND} -DBUILD_STEP=build -P ${PROJECT_BUILD_STEPS_FILE}
INSTALL_COMMAND ${CMAKE_COMMAND} -DBUILD_STEP=install -P ${PROJECT_BUILD_STEPS_FILE}
)
If you do not want to use ExternalProject_Add you can use something like following. This will also give you individual build targets like make project_patch, make project_configure, make project_build, make project_install.
CMakeLists.txt (Option 2)
set(CMAKE_COMMAND /usr/bin/cmake)
set(PROJECT_BUILD_STEPS_FILE project_build_steps.cmake)
set(STAMP_FILE_PROJECT_PATCH .project_patch_done)
add_custom_command(
OUTPUT ${STAMP_FILE_PROJECT_PATCH}
COMMAND ${CMAKE_COMMAND} -DVAR1=value1 -DSTEP=patch -P ${PROJECT_BUILD_STEPS_FILE}
COMMAND ${CMAKE_COMMAND} -E touch ${STAMP_FILE_PROJECT_PATCH}
)
add_custom_target(project_patch DEPENDS ${STAMP_FILE_PROJECT_PATCH})
set(STAMP_FILE_PROJECT_CONFIGURE .project_configure_done)
add_custom_command(
OUTPUT ${STAMP_FILE_PROJECT_CONFIGURE}
COMMAND ${CMAKE_COMMAND} -DSTEP=configure -P ${PROJECT_BUILD_STEPS_FILE}
COMMAND ${CMAKE_COMMAND} -E touch ${STAMP_FILE_PROJECT_CONFIGURE}
)
add_custom_target(project_configure DEPENDS project_patch ${STAMP_FILE_PROJECT_CONFIGURE})
set(STAMP_FILE_PROJECT_BUILD .project_build_done)
add_custom_command(
OUTPUT ${STAMP_FILE_PROJECT_BUILD}
COMMAND ${CMAKE_COMMAND} -DSTEP=build -P ${PROJECT_BUILD_STEPS_FILE}
COMMAND ${CMAKE_COMMAND} -E touch ${STAMP_FILE_PROJECT_BUILD}
VERBATIM
)
add_custom_target(project_build DEPENDS project_configure ${STAMP_FILE_PROJECT_BUILD})
set(STAMP_FILE_PROJECT_INSTALL .project_install_done)
add_custom_command(
OUTPUT ${STAMP_FILE_PROJECT_INSTALL}
COMMAND ${CMAKE_COMMAND} -DSTEP=install -P ${PROJECT_INSTALL_STEPS_FILE}
COMMAND ${CMAKE_COMMAND} -E touch ${STAMP_FILE_PROJECT_INSTALL}
VERBATIM
)
add_custom_target(project_install DEPENDS project_build ${STAMP_FILE_PROJECT_INSTALL})

Installing readline 6.0 on OS X

I'm trying to install readline 6 from source but run into an error during 'make install'.
Here is the end of the output after executing 'sudo make install'
( cd shlib ; make DESTDIR= install )
/bin/sh ../support/mkdirs /usr/local/lib
/bin/sh ../support/shlib-install -O darwin9.7.0 -d /usr/local/lib -b /usr/local/bin -i "/usr/bin/install -c -m 644" libhistory.6.0.dylib
/bin/sh ../support/shlib-install -O darwin9.7.0 -d /usr/local/lib -b /usr/local/bin -i "/usr/bin/install -c -m 644" libreadline.6.0.dylib
install: you may need to run ldconfig
I know that ldconfig isn't installed by default on OS X, and I read somewhere that it shouldn't be needed to fix this issue. I believe it has something to do with dynamic libraries, but I haven't been able to find out how to fix the issue, anyone have any insight?
FYI, I'm running OS X on an intel 2.4ghz macbook
thanks
P.S. I also applied the 3 available readline 6 patches before running configure and make
Actually, this isn't an error at all... it's just a notice message at the end of the install. It get this too, and my readline 6 is happily installed.
If you check /usr/local/lib and see readline there, you're done :-) No need to run any equivalent of ldconfig.
$ ls /usr/local/lib | grep readline
libreadline.6.0.dylib
libreadline.6.dylib
libreadline.a
libreadline.dylib

Resources