I am facing this problem for a while:
make [1]: *** read jobs pipe: Resource temporarily unavailable. Stop.
I have a top-level makefile that "calls" additional cmake and make based on the rules.
target A is a relatively small code (C/C++). target B is much bigger.
The issue happens mainly during the building target B but not always. I use make 3.82.
I have tried with make 4.3 and it occurs always.
Here is a skeleton of my build system:
MAKEFLAGS = --jobs=10 --max-load=20
<target A>:
+. /usr/share/Modules/init/bash &&
module load <target> &&
cmake --build <project buildsystem> --target <target A>
<target B>: <target A>
+. /usr/share/Modules/init/bash &&
module load <target> &&
make -C <path> <target B> &&
cmake --build <project buildsystem> --target <target B>
/usr/share/Modules/init/bash
The Modules package and the module command are initialized when a shell-specific initialization script is sourced into the shell. The script creates the module command as either an alias or function and creates Modules environment variables.
https://modules.readthedocs.io/en/latest/module.html
(--max-load is equal the number of cores on the build sever. --jobs is half of it.)
I have verified the MAKEFLAGS passes correctly (I have checked the number of gcc/g++ processes during build) and I don't have this warning:
warning: -jN forced in submake: disabling jobserver mode.
Host OS: RedHat 7.6
cmake: 3.16.4
How can I solve this issue?
I had this exact same error on nested makes. The issue ended up being hardware. I added a CPU temperature throttle background job to keep the CPU cores below 85C and the errors went away.
My system is a core-i9 10850K with a very small water cooler. During the heavy parts of the compile, the core temps were throttled from the stock 5GHz down to about 3.8GHz. Without the throttle, core temps actually hit 100C.
Related
I am trying to create custom snapshot from some Javascript file. I was able to create a snapshot using the command
mksnapshot.exe snapshot11.js --startup_blob snap.bin
but when I was trying to create an Isolate with this snap.bin file I got this message
The Isolate is incompatible with the embedded blob. This is usually caused by incorrect usage of mksnapshot. When generating custom snapshots, embedders must ensure they pass the same flags as during the V8 build process (e.g.: --turbo-instruction-scheduling).
I am guessing that I need recreate the snapshot with the proper flags but I couldn't find which flags I need to use.
My args.gn
is_component_build=true
v8_static_library=false
is_official_build=false
is_debug=true
use_custom_libcxx=false
use_custom_libcxx_for_host=false
target_cpu="x64"
use_goma=false
v8_use_external_startup_data=false
v8_enable_i18n_support = false
symbol_level=2
v8_enable_fast_mksnapshot=true
Any lead will be helpful.
10x
You can invoke ninja with -v to have it print all the commands it executes; e.g. if you compile V8 with:
ninja -v -C out/... v8_monolith
then you'll find a line for the mksnapshot invocation in the output, and can copy the flags from there. (If you have already compiled V8, ninja will say "nothing to do"; in that case you can either clean out everything, or just delete snapshot_blob.bin and libv8_monolith.so.)
In the steps of setting up CI builds on our gitlab server, I can't seem to find information on how to set up the detection of compiler warnings. Example build output:
[100%] Building CXX object somefile.cpp.o
/home/gitlab-runner/builds/XXXXXXX/0/group/project/src/somefile.cpp:14:2: warning: #warning ("This is a warning to test gitlab") [-Wcpp]
#warning("This is a warning to test gitlab")
^
However the build result is success instead of warning or something similar. Ideally the results wuold also be visible on the merge request on the feature (and block the merge if possible).
I can't imagine I'm the only one trying to achieve this, so I am probably looking in the wrong direction. The 'best' solution I found is to somehow manually parse the build output and generate a JUnit report.
How would I go about doing this without allowing the build job to fail, since I would like the job to fail when compiler errors occur.
Update
For anyone stumbling across this question later, and in lieu of a best practice, this is how I solved it:
stages:
- build
- check-warnings
shellinspector:
stage: build
script:
- cmake -Bcmake-build -S.
- make -C cmake-build > >(tee make.output) 2> >(tee make.error)
artifacts:
paths:
- make.output
- make.error
expire_in: 1 week
analyse build:
stage: check-warnings
script:
- "if [[ $(cat make.error | grep warning -i) ]]; then cat make.error; exit 1; fi"
allow_failure: true
This stores the build output errors in make.error in the first stage, the next stage then queries that file for warnings and fails that stage with allow_failure: true to create the passed with warning pipeline status I was looking for.
It seems that the solution to such need (e.g., see the issue "Add new CI state: has-warnings" https://gitlab.com/gitlab-org/gitlab-runner/issues/1224) has been to introduce the allow_failure option so that one job can be the compilation itself, which is not allowed to fail (if it does, then the pipeline fails) and another job can be the detection of such warnings which is allowed to fail (if one is found then the pipeline will not fail).
Also the possibility of defining warning regex in the .gitlab-ci.yml has been requested but it does not exist yet.
I'm using Buildroot 2018.02.7 to build a simple Linux system for i386 PC, as a precursor to doing the same thing for an embedded ARM system. I keep running into problems like this one, in building the util-linux module:
CC lib/libcommon_la-path.lo
lib/pager.c:11:17: fatal error: err.h: No such file or directory
#include <err.h>
^
compilation terminated.
Makefile:8596: recipe for target 'lib/libcommon_la-pager.lo' failed
make[3]: *** [lib/libcommon_la-pager.lo] Error 1
When I look in the linux-headers source tree in .../output/build/linux-headers-4.13.8, the file is found. But Buildroot is pointing to a different set of headers, the one built into its GCC:
devuser#3faf730b4a1b:~/pc/buildroot-2018.02.7/output/build/util-linux-2.31.1$ ../../host/bin/i686-buildroot-linux-uclibc-gcc -print-sysroot
/home/devuser/pc/buildroot-2018.02.7/output/host/i686-buildroot-linux-uclibc/sysroot
devuser#3faf730b4a1b:~/pc/buildroot-2018.02.7/output/build/util-linux-2.31.1$
And the two /usr/include subdirectories are significantly different.
Have I missed a configuration setting somewhere in the defconfig file? This is the defconfig:
BR2_x86_core2=y
BR2_SSP_REGULAR=y
BR2_PACKAGE_HOST_LINUX_HEADERS_CUSTOM_4_13=y
BR2_UCLIBC_CONFIG="board/pc/dsa_pc_i386_uclibc.config"
BR2_TOOLCHAIN_BUILDROOT_USE_SSP=y
BR2_TOOLCHAIN_BUILDROOT_CXX=y
BR2_TARGET_GENERIC_GETTY_PORT="tty1"
BR2_ROOTFS_POST_IMAGE_SCRIPT="board/pc/post-image.sh support/scripts/genimage.sh"
BR2_ROOTFS_POST_SCRIPT_ARGS="-c board/pc/genimage-bios.cfg"
BR2_LINUX_KERNEL=y
BR2_LINUX_KERNEL_CUSTOM_VERSION=y
BR2_LINUX_KERNEL_CUSTOM_VERSION_VALUE="4.13.8"
BR2_LINUX_KERNEL_USE_CUSTOM_CONFIG=y
BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE="board/pc/linux.config"
BR2_LINUX_KERNEL_INSTALL_TARGET=y
BR2_LINUX_KERNEL_EXT_XENOMAI=y
BR2_PACKAGE_BUSYBOX_CONFIG="board/pc/dsa_pc_i386_busybox.config"
BR2_PACKAGE_LINUX_FIRMWARE=y
BR2_PACKAGE_LINUX_FIRMWARE_ATHEROS_9170=y
BR2_PACKAGE_LINUX_FIRMWARE_ATHEROS_9271=y
BR2_PACKAGE_LINUX_FIRMWARE_IWLWIFI_3160=y
BR2_PACKAGE_LINUX_FIRMWARE_IWLWIFI_3168=y
BR2_PACKAGE_LINUX_FIRMWARE_IWLWIFI_5000=y
BR2_PACKAGE_LINUX_FIRMWARE_IWLWIFI_6000G2A=y
BR2_PACKAGE_LINUX_FIRMWARE_IWLWIFI_6000G2B=y
BR2_PACKAGE_LINUX_FIRMWARE_IWLWIFI_7260=y
BR2_PACKAGE_LINUX_FIRMWARE_IWLWIFI_7265D=y
BR2_PACKAGE_LINUX_FIRMWARE_IWLWIFI_8000C=y
BR2_PACKAGE_LINUX_FIRMWARE_IWLWIFI_8265=y
BR2_PACKAGE_LINUX_FIRMWARE_RALINK_RT73=y
BR2_PACKAGE_LINUX_FIRMWARE_RALINK_RT2XX=y
BR2_PACKAGE_LINUX_FIRMWARE_RTL_81XX=y
BR2_PACKAGE_LINUX_FIRMWARE_RTL_87XX=y
BR2_PACKAGE_LINUX_FIRMWARE_RTL_88XX=y
BR2_PACKAGE_LINUX_FIRMWARE_RTL_8169=y
BR2_PACKAGE_ACPID=y
BR2_PACKAGE_DBUS=y
BR2_PACKAGE_ZLIB=y
BR2_PACKAGE_LIBFFI=y
BR2_PACKAGE_PCRE=y
BR2_PACKAGE_PCRE_UCP=y
BR2_PACKAGE_READLINE=y
BR2_PACKAGE_WPA_SUPPLICANT=y
BR2_PACKAGE_WPA_SUPPLICANT_DBUS_NEW=y
BR2_PACKAGE_KMOD=y
BR2_PACKAGE_UTIL_LINUX=y
BR2_PACKAGE_UTIL_LINUX_LIBMOUNT=y
BR2_TARGET_ROOTFS_EXT2=y
BR2_TARGET_ROOTFS_EXT2_4=y
BR2_TARGET_ROOTFS_EXT2_SIZE="120M"
# BR2_TARGET_ROOTFS_TAR is not set
BR2_PACKAGE_HOST_GENIMAGE=y
Enable UCLIBC_HAS_BSD_ERR in uClibc.
You have a custom uClibc configuration. That falls squarely in the "you know what you are doing" category, since it allows you to remove features that other packages rely on.
In this case, util-linux relies on the non-Posix err.h include. This is only installed if UCLIBC_HAS_BSD_ERR is enabled in the uClibc configuration.
There is a high risk of running into similar issues with a custom uClibc configuration.
I am using travis-ci and biicode to build my project who is depending on boost log. But boost log times are longer than 10 min so I get this message:
No output has been received in the last 10 minutes, this potentially indicates a
stalled build or something wrong with the build itself.
The build has been terminated
The build is working correctly, it's just that boost log is really long to compile with limited resources (I tried to compile it on a VM with 1 CPU and 2GB of RAM and it took almost more than 15 min)
I know this is happening because there is not enough verbose going on so I tried the following flags:
>bii cpp:build -- VERBOSE=1
In the CMakeList.txt, set BII_BOOST_VERBOSE ON as mentionnened here
Set BOOST_LOG_COMPILE_FAST_ON as explained here
Using travis_wait
Actually travis_wait seems to be the correct solution but when I put it in my .travis.yml like this
script: travis_wait bii cpp:build
It does actually doesn't output logs like usually and just time out after 20 min. I don't think the actual building is taking place
What is the correct way to handle this problem?
This is a known issue, Boost.Log takes a long time to compile.
You can use travis_wait to call bii cpp:configure, but I'm with you, I need log feedback (No pun intended). However, I have tried that too and leaded to >50min build, which means travis aborts build on free accounts :( Of course my repo does not build Boost.Log only.
Just as a note, here's part of the settings.py file from the boost-biicode repo:
#Boost.Log takes so much time to compile, leads to timeouts on Travis CI
#It was tested on Windows and linux, works 'ok' (Be careful with linking settings)
if args.ci: del packages['examples/boost-log']
I'm currently working on a solution, launching asynchronous builds while printing progress. Check this issue. It will be ready for this week :)
To speed-up your build, try to play with BII_BOOST_BUILD_J variable to set the number of threads you want for building Boost components. Here's an example:
script:
- bii cpp:configure -DBII_BOOST_BUILD_J=4
Be careful, more threads means more RAM needed to compile at a time. Be sure you don't make the travis job VM go out of memory.
What configure options a should use to compile mpich2 (ver 1.1.1p1 or 1.2.1p1) with sctp ?
In my try there is a error when linking cpi.c (small example).
/home/op02/mpiopt/sctp/lib/libmpich.a(ch3u_rma_sync.o)(.text+0x20a7): In functio
n `MPIDI_Win_post':
: undefined reference to `PMPI_Group_translate_ranks'
/home/op02/mpiopt/sctp/lib/libmpich.a(ch3u_rma_sync.o)(.text+0x21bd): In functio
n `MPIDI_Win_post':
: undefined reference to `PMPI_Group_free'
/home/op02/mpiopt/sctp/lib/libmpich.a(ch3u_rma_sync.o)(.text+0x25c4): In functio
n `MPIDI_Win_complete':
: undefined reference to `PMPI_Group_translate_ranks'
....
My options was
../mpich2-1.1.1p1/configure --enable-fast=O1 \
--host=x86_64-unknown-linux-gnu \
--target=x86_64-secret-linux-gnu \
--with-device=ch3:sctp --with-pm=hydra \
--with-cross=x8664secret.cross --disable-f77 --disable-f90 \
>conf.log 2>&1
with x8664secret.cross being an output of getcross.c program. Host, target, and this file are here to force a cross-compilation. (it is a requirement for this build)
Is sctp in mpich2 in active state and can it be compiled?
Does sctp network module support cross-building?
Try 1.3.1 instead. I see that Brad Penoff committed a couple of small changes to the build system since 1.2.1p1 was released, so it may be in better shape now. Alternatively, try using (the rather old) MPICH2 1.0.8, where I believe things were still working.
If the cross-compile step is what's really causing the problem and you still need to solve this problem, you can get more interactive support from mpich-discuss#lists.anl.gov. We can dig in over there instead.