I am trying build cuda-enabled JAX from source on a cluster with CentOS version7. In the jax home directory, I run:
python build/build.py --enable_cuda --cuda_path=$CUDA_HOME --cudnn_path=$CUDNN_HOME
Here are my specs:
cuda version: 11.6
cudnn version 7.5
gcc version: 11.2.0
Bazel binary path: ./bazel-5.1.1-linux-x86_64
Bazel version: 5.1.1
Python binary path: ~/.conda/envs/JAX/bin/python
Python version: 3.9
NumPy version: 1.21.5
MKL-DNN enabled: yes
Target CPU: x86_64
Target CPU features: release
CUDA enabled: yes
Here are my environment variables:
CC=/usr/local/app/compiler/gcc/11.2.0/bin/gcc
CXX=/usr/local/app/compiler/gcc/11.2.0/bin/g++
INCLUDE=/usr/local/app/compiler/gcc/11.2.0/include
LIBRARY_PATH=/usr/local/app/compiler/gcc/11.2.0/lib64:/usr/local/app/lib/nvidia/cuda/11.6.1/lib64:/usr/local/app/lib/nvidia/driver/510.47/lib64/nvidia
LD_LIBRARY_PATH=/usr/local/app/compiler/gcc/11.2.0/lib64:/usr/local/app/lib/nvidia/cuda/11.6.1/lib64:/usr/local/app/lib/nvidia/driver/510.47/lib64/nvidia
The Bazel build throws this error:
ERROR: /lustre1/home/rice_cake/.cache/bazel/_bazel_rice_cake/8366d0a62cbb3b115627233e356374ab/external/zlib/BUILD.bazel:5:11: Compiling uncompr.c failed: undeclared inclusion(s) in rule '#zlib//:zlib':
this rule is missing dependency declarations for the following files included by 'uncompr.c':
'/usr/local/app/compiler/gcc/11.2.0/lib/gcc/x86_64-redhat-linux/11.2.0/include/stddef.h'
'/usr/local/app/compiler/gcc/11.2.0/lib/gcc/x86_64-redhat-linux/11.2.0/include-fixed/limits.h'
'/usr/local/app/compiler/gcc/11.2.0/lib/gcc/x86_64-redhat-linux/11.2.0/include-fixed/syslimits.h'
'/usr/local/app/compiler/gcc/11.2.0/lib/gcc/x86_64-redhat-linux/11.2.0/include/stdarg.h'
It seems that Bazel is complaining that I do not declare dependency for the C++ library files. Is there a way to solve this?
The trouble is that these Bazel build files are automatically generated by jax/build/build.py, and I don't know how to fix them manually.
P.S. Building JAX from source is successful on my local machine..
Thanks very much for any help,
Rice
I went into the Bazel cache files and checked the BUILD file, and it's the same as the on my local machine, so I'm a bit stuck. I am expecting that maybe setting environment variables might help Bazel build my JAX.
Related
I have already used the Aravis library on a host running on Linux,
Aravis is a C open-source library for camera acquisition. You could find the GitHub repo in the link below:
https://github.com/AravisProject/aravis
I want to cross-compile the library for my ARM target running on a Mendel Linux machine.
The thing is I am not very familiar with the meson build and this library does not offer any makefile.
Do you have any idea or sample of the makefile that could help me cross-compile, please?
Thank you in advance
S.Tarik
I tried to change the configuration in Linux to call the g++ cross compiler first by changing the priority between the default gcc and aarch64-linux-gnu-g++.
https://github.com/highperformancecoder
the error message thrown by the compiler is the following:
www#www-virtual-machine:~/Downloads/aravis-main$ meson build
The Meson build system
Version: 0.63.3
Source dir: /home/www/Downloads/aravis-main
Build dir: /home/www/Downloads/aravis-main/build
Build type: native build
Project name: aravis
Project version: 0.8.23
meson.build:1:0: ERROR: Could not invoke sanity test executable: [Errno 8] Exec format error: '/home/www/Downloads/aravis-main/build/meson-private/sanitycheckc.exe'.
A full log can be found at /home/www/Downloads/aravis-main/build/meson-logs/meson-log.txt
tar
Building the example application skeleton or l2fwd or l3fwd via DPDK meson with option -Dexamples=, creates the binary executable without any warnings or error. But executing the same example as standalone build leads to DPDK build error
C:\dpdk\dpdk-21.02\examples\skeleton>gcc basicfwd.c -include rte_config.h -march=native -IC:/include -Wl,--as-needed -LC:/lib -lrte_cfgfile -lrte_hash -lrte_cmdline -lrte_pci -lrte_ethdev -lrte_meter -lrte_net -lrte_mbuf -lrte_mempool -lrte_rcu -lrte_ring -lrte_eal -lrte_telemetry -lrte_kvargs
In file included from C:/include/rte_eal.h:21,
from basicfwd.c:7:
C:/include/rte_bus.h:22:10: fatal error: sys/queue.h: No such file or directory
#include <sys/queue.h>
^~~~~~~~~~~~~
compilation terminated.
Expectation:
C:\dpdk\dpdk-21.02\examples\skeleton>dir
Directory of C:\dpdk\dpdk-21.02\examples\skeleton
<DIR> .
<DIR> ..
163,348 a.exe
5,458 basicfwd.c
1,511 Makefile
322 meson.build
Steps to reproduce the error
Platform: x86_64, Intel E5 2680 Xeon
OS: windows server 2019
DPDK version: 21.02
Meson version: 0.57.1
Ninja version: 1.10.2
Pkg-config version: 0.26
Steps followed: http://doc.dpdk.org/guides/windows_gsg/index.html
compiler flags: execute pkg-config --cflags --libs libdpdk
There is a difference in the CFLAGS and LDFLAGS when one uses meson meson -Dexamples=l2fwd build and meson build. In the former scenario, the dependency is pulled from DPDK root folder directly and direct link to lib/librte_eal/windows/include. But when DPDK pkg has installed the files under lib/librte_eal/windows/include are not copied over which causes build failure.
Checking meson.build in lib/librte_eal/windows/ reveals the only install is for 'rte_os.h', 'rte_virt2phys.h', and 'rte_windows.h'. Folder 'netinet' and 'sys' are skipped. Based on the update from DPDK maintainer, since DPDK on windows is experimental only validation done is with meson -Dexamples=l2fwd build. Hence as temporary work around is to use -I[DPDK folder]\lib\librte_eal\windows\include\
Note:
Thanks to #stackinside for the comment
working on DPDK patch to plug the missing files.
OS: MacOS 11
Platform: Apple M1 architecture (Darwin ARM64)
According to
https://chromium.googlesource.com/external/github.com/grpc/grpc/+/HEAD/BUILDING.md
using make is outdated.
I downloaded and installed bazel Darwin x64 (MacOS Intel) binary latest version, bazel build :all completed without errors.
bazel test --config=dbg //test/... however prints out a lot of errors after a few minutes of compiling.
There's also the option to run cmake, which also completes.
So, my question is, am I correct if cmake or bazel (with build :all, of course) does not install the binaries, but rather make and (sudo) make install do?
If the use of make is "outdated", is bazel / cmake used to install the compiled binaries?
I am trying to install latest version glib version 2.64.3 library which uses meson build system to compile glib code.
While installing glib i am trying to give custom options to compiler to compile with specific gcc version...etc.
Here is my command :-
/test/common/pkgs/python/v3.6.1/bin/meson setup glib_install -Dbuild.cpp_args="-fPIC" -Dbuild.c_args='-fPIC -O2' -Dc_link_args='/test/common/pkgs/gcc/v6.3.0/lib64' -Ddefault_library=both -Db_staticpic=true -Dprefix='/home/user/test/glib-2.64.3/glib-2.64.3/glib_install'
I am getting following issue :-
The Meson build system
Version: 0.54.2
Source dir: /home/user/extlibs/glib-2.64.3/glib-2.64.3
Build dir: /home/user/extlibs/glib-2.64.3/glib-2.64.3/glib_install
Build type: native build
WARNING: Unknown options: "c_args, cpp_args"
The value of new options can be set with:
meson setup <builddir> --reconfigure -Dnew_option=new_value ...
Project name: glib
Project version: 2.64.3
meson.build:1:0: ERROR: Compiler cc can not compile programs.
I am not sure if i am missing something here. I could not find much info about like how can we provide custom arguments to use specific compiler flags while running meson. Let me know if i am missing something here.
Check in your glib_install/meson-logs/meson-log.txt file to see why the configuration is failing, but as a start it seems you need to pass -Dcpp_args="-fPIC" -Dc_args='-fPIC -O2' (without build.) on the command line.
I'm trying out a new setup. I'm on a 64-bit Windows 10. I've meson example project and clang compiler stack over Visual Studio 2017. Both of those are in my PATH.
[0/1] Regenerating build files.
The Meson build system Version: 0.49.0
Source dir: C:\WORK\cpp-example\wx-example
Build dir: C:\WORK\cpp-example\wx-example\builddir
Build type: native build
Project name: wx-example
Project version: undefined
Native C++ compiler: clang++ (clang 7.0.0 "clang version 7.0.0 (tags/RELEASE_700/final)")
Build machine cpu family: x86_64
Build machine cpu: x86_64
Found wx-config '>=3.0.0' NO
Dependency WxWidgets found: NO (tried config-tool)
meson.build:8:2: ERROR: Dependency "wxwidgets" not found, tried config-tool
A full log can be found at C:\WORK\cpp-example\wx-example\builddir\meson-logs\meson-log.txt
FAILED: build.ninja
My meson.build is
project('wx-example', 'cpp')
#if build_machine.system() == 'windows'
# cpp = meson.get_compiler('cpp')
# add_project_link_arguments(['C:\WORK\wxWidgets-3.1.2\include'], language : 'cpp')
# wx_dep = cpp.find_library('wxwidgets', dirs : ['C:\WORK\wxWidgets-3.1.2\lib\vc_x64_dll'])
#else
wx_dep = dependency('wxwidgets', version : '>=3.0.0', required : true)
#endif
executable('wx-example.exe', ['main.cpp'], dependencies : [wx_dep])
Do u have any idea how to compile my example?
Maybe I should use MinGW wxWitgets package?
Meson's dependency for wxWidgets only supports wx-config tool, which was originally intended for *nix only. So you need to get wx-config Windows native port first to make it work with VC++.
BTW. Maybe even the easiest thing to do is to write it on your own, as it's just a regular console application, which parses the command line and spits to stdout the corresponding compiler/linker flags.
Maybe I should use MinGW wxWitgets package?
Well, that's, of course, possible but then you will also have to switch to gcc/g++. Moreover, Meson's wxWidgets dependency is still broken under MSYS2/MinGW. The problem is that Meson erroneously tries to execute wx-config directly, while under Windows it must be prefixed with env/sh/bash or whatever. Not a big problem to fix it, but you'll still have some work to do.
UPD.: The issue appears to be fixed in Meson 0.51.0