I'm getting started with bazel and trying to generate the protobuf code for golang for an RPC service.
When I try to build it I get the following error:
bazel-out/k8-fastbuild/bin/examples/grpc/protos/helloworld_go_proto_/examples/grpc/protos/helloworld.pb.go:229:7: undefined: grpc.ClientConnInterface
bazel-out/k8-fastbuild/bin/examples/grpc/protos/helloworld_go_proto_/examples/grpc/protos/helloworld.pb.go:233:11: undefined: grpc.SupportPackageIsVersion6
bazel-out/k8-fastbuild/bin/examples/grpc/protos/helloworld_go_proto_/examples/grpc/protos/helloworld.pb.go:243:5: undefined: grpc.ClientConnInterface
bazel-out/k8-fastbuild/bin/examples/grpc/protos/helloworld_go_proto_/examples/grpc/protos/helloworld.pb.go:246:26: undefined: grpc.ClientConnInterface
Full build log: https://app.buildbuddy.io/invocation/c3773978-22dd-44c8-b977-13967a1953b7
Here is the code: https://github.com/juanique/example-go-grpc. I'm trying to include the least possible amount of code to make that target work.
Since the BUILD file was generated by gazelle I suspect the issue is in the WORKSPACE file: https://raw.githubusercontent.com/juanique/example-go-grpc/main/WORKSPACE. I'm just doing what I found in https://github.com/bazelbuild/rules_go
UPDATE: it looks surely a version issue: https://github.com/grpc/grpc-go#compiling-error-undefined-grpcsupportpackageisversion
Not a bazel user, but after hours' test, I found your rpc version should be higher:
go_repository(
name = "org_golang_google_grpc",
- build_file_proto_mode = "disable",
importpath = "google.golang.org/grpc",
- sum = "h1:J0UbZOIrCAl+fpTOf8YLs4dJo8L/owV4LYVtAXQoPkw=",
- version = "v1.22.0",
+ sum = "h1:f+PlOh7QV4iIJkPrx5NQ7qaNGFQ3OTse67yaDHfju4E=",
+ version = "v1.41.0",
)
Then you can build it:
~/code/example-go-grpc (main*) [09:45:55]
p1gd0g$ bazel build //examples/grpc/protos:helloworld_go_proto
INFO: Analyzed target //examples/grpc/protos:helloworld_go_proto (49 packages loaded, 306 targets configured).
INFO: Found 1 target...
Target //examples/grpc/protos:helloworld_go_proto up-to-date:
bazel-bin/examples/grpc/protos/helloworld_go_proto.a
INFO: Elapsed time: 41.180s, Critical Path: 1.20s
INFO: 29 processes: 1 internal, 28 linux-sandbox.
INFO: Build completed successfully, 29 total actions
And I recommend using gazelle with go.mod to generate WORKSPACE. A demo: https://github.com/p1gd0g/helloworld
Related
I have a (1) Dockerfile and a (2) C++ project with Bazel
I want to create a docker image that has Bazel targets pre-built within the image, so as to when I power up new containers the Bazel targets are pre-built and I just do Bazel run //hello:hello_world from the container bash.
Dockerfile
# Copy my project with Bazel files to a Docker image, and the
...
RUN bazel --output_user_root=/tmp/hello_project/bazel build //...
...
Within the execution of the Dockerfile, I get the following output which is expected
Loading:
Loading: 0 packages loaded
Analyzing: 2 targets (1 packages loaded, 0 targets configured)
Analyzing: 2 targets (11 packages loaded, 18 targets configured)
INFO: Analyzed 2 targets (15 packages loaded, 60 targets configured).
INFO: Found 2 targets...
[0 / 11] [Prepa] BazelWorkspaceStatusAction stable-status.txt
INFO: Elapsed time: 6.333s, Critical Path: 0.37s
INFO: 11 processes: 6 internal, 5 processwrapper-sandbox.
INFO: Build completed successfully, 11 total actions
INFO: Build completed successfully, 11 total actions
When I run a new container form the Docker Image built previously and then on the container I run
bazel run //hello:hello_world
Instead of using existing pre-built targets it re-builds the targets, which is not required.
Result I expect (not get): Everything is pre-built and just needs to run
INFO: Analyzed target //hello:hello_world (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //hello:hello_world up-to-date:
bazel-bin/hello/hello_world
INFO: Elapsed time: 0.163s, Critical Path: 0.01s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Build completed successfully, 1 total action
Hello World!
Result I get: I re-builds the binaries
[root#4a6bdb57fd79 test-rc]# bazel run //hello:hello_world
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
INFO: Analyzed target //hello:hello_world (15 packages loaded, 60 targets configured).
INFO: Found 1 target...
Target //hello:hello_world up-to-date:
bazel-bin/hello/hello_world
INFO: Elapsed time: 6.255s, Critical Path: 0.38s
INFO: 7 processes: 4 internal, 3 processwrapper-sandbox.
INFO: Build completed successfully, 7 total actions
INFO: Build completed successfully, 7 total actions
Hello World!
How can I make sure, the the bazel run uses same pre-built targets and not build them again before run.
This sounds like non-determinism - in the second execution of Bazel, something is different causing a cache miss.
Some things that can cause it:
different options passed to bazel build vs. bazel test - check your .bazelrc file
actions that include VCS info or a timestamp - make sure you have rules_docker 0.22.0 or later to pick up https://github.com/bazelbuild/rules_docker/commit/2b35b2dd56f0be6cc6b8df957332a31435f6b3ce
One step to diagnose is to use the --explain=log.txt and --verbose_explanations flags to Bazel, then the log file will say why it was rebuilt. However it doesn't have much detail, just something like "a source file has changed".
If you want the power tool for this, there is a way to find out exactly why Bazel didn't get a cache hit - read https://georgi.hristozov.net/til/2020/04/20/compare-bazel-execlogs-to-find-non-deterministic-parts-of-the-build
I'm following the readthedocs instructions for the meta-raspberrypi layer and trying to build the rpi-test-image image for the raspberrypi3-64 machine contained in the meta-raspberrypi layer using the zeus release of yocto.
I added this to conf/local.conf:
MACHINE ??= "raspberrypi3-64"
ENABLE_UART = "1"
My bblayers.conf file looks like this:
/opt/yocto/workspace/rpi3-64-build/conf[master]☢ ☠$ cat bblayers.conf
# POKY_BBLAYERS_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
POKY_BBLAYERS_CONF_VERSION = "2"
BBPATH = "${TOPDIR}"
BBFILES ?= ""
BBLAYERS ?= " \
/opt/yocto/workspace/sources/poky/meta \
/opt/yocto/workspace/sources/poky/meta-poky \
/opt/yocto/workspace/sources/poky/meta-yocto-bsp \
/opt/yocto/workspace/sources/meta-openembedded/meta-oe \
/opt/yocto/workspace/sources/meta-openembedded/meta-multimedia \
/opt/yocto/workspace/sources/meta-openembedded/meta-networking \
/opt/yocto/workspace/sources/meta-openembedded/meta-python \
/opt/yocto/workspace/sources/meta-raspberrypi \
"
I added the 4 items from meta-openembeeded as a work-around for ERROR: Nothing RPROVIDES 'bigbuckbunny-480p':
This enables the build to run but it exits with the following errors:
/opt/yocto/workspace/rpi3-64-build/conf[master]☢ ☠$ bitbake rpi-test-image
Parsing recipes: 100% |#########################################################################################################################################| Time: 0:02:49
Parsing of 2340 .bb files complete (0 cached, 2340 parsed). 3464 targets, 133 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies
Build Configuration:
BB_VERSION = "1.44.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "ubuntu-16.04"
TARGET_SYS = "aarch64-poky-linux"
MACHINE = "raspberrypi3-64"
DISTRO = "poky"
DISTRO_VERSION = "3.0.2"
TUNE_FEATURES = "aarch64 cortexa53 crc"
TARGET_FPU = ""
meta
meta-poky
meta-yocto-bsp = "zeus:74f229160c7f4037107c1dad8f0d02128c080a7e"
meta-oe
meta-multimedia
meta-networking
meta-python = "zeus:9e60d30669a2ad0598e9abf0cd15ee06b523986b"
meta-raspberrypi = "zeus:0e05098853eea77032bff9cf81955679edd2f35d"
Initialising tasks: 100% |######################################################################################################################################| Time: 0:00:06
Sstate summary: Wanted 1338 Found 0 Missed 1338 Current 0 (0% match, 0% complete)
NOTE: Executing Tasks
NOTE: Setscene tasks completed
WARNING: icu-native-64.2-r0 do_fetch: Checksum mismatch for local file /opt/yocto/cache/downloads/icu4c-64_2-src.tgz
Cleaning and trying again.
WARNING: icu-native-64.2-r0 do_fetch: Renaming /opt/yocto/cache/downloads/icu4c-64_2-src.tgz to /opt/yocto/cache/downloads/icu4c-64_2-src.tgz_bad-checksum_abb12cb25a05198ad8f4c1e6f668fa05
WARNING: icu-native-64.2-r0 do_fetch: Checksum failure encountered with download of http://download.icu-project.org/files/icu4c/64.2/icu4c-64_2-src.tgz - will attempt other sources if available
ERROR: rpi-test-image-1.0-r0 do_rootfs: Could not invoke dnf. Command '/opt/yocto/workspace/rpi3-64-build/tmp/work/raspberrypi3_64-poky-linux/rpi-test-image/1.0-r0/recipe-sysroot-native/usr/bin/dnf -v --rpmverbosity=info -y -c /opt/yocto/workspace/rpi3-64-build/tmp/work/raspberrypi3_64-poky-linux/rpi-test-image/1.0-r0/rootfs/etc/dnf/dnf.conf --setopt=reposdir=/opt/yocto/workspace/rpi3-64-build/tmp/work/raspberrypi3_64-poky-linux/rpi-test-image/1.0-r0/rootfs/etc/yum.repos.d --installroot=/opt/yocto/workspace/rpi3-64-build/tmp/work/raspberrypi3_64-poky-linux/rpi-test-image/1.0-r0/rootfs --setopt=logdir=/opt/yocto/workspace/rpi3-64-build/tmp/work/raspberrypi3_64-poky-linux/rpi-test-image/1.0-r0/temp --repofrompath=oe-repo,/opt/yocto/workspace/rpi3-64-build/tmp/work/raspberrypi3_64-poky-linux/rpi-test-image/1.0-r0/oe-rootfs-repo --nogpgcheck install psplash-raspberrypi packagegroup-core-boot packagegroup-base-extended run-postinsts packagegroup-rpi-test locale-base-en-us locale-base-en-gb' returned 1:
DNF version: 4.2.2
cachedir: /opt/yocto/workspace/rpi3-64-build/tmp/work/raspberrypi3_64-poky-linux/rpi-test-image/1.0-r0/rootfs/var/cache/dnf
Added oe-repo repo from /opt/yocto/workspace/rpi3-64-build/tmp/work/raspberrypi3_64-poky-linux/rpi-test-image/1.0-r0/oe-rootfs-repo
repo: using cache for: oe-repo
not found other for:
not found modules for:
not found deltainfo for:
not found updateinfo for:
oe-repo: using metadata from Thu 30 Apr 2020 09:25:08 PM UTC.
Last metadata expiration check: 0:00:02 ago on Thu 30 Apr 2020 09:25:11 PM UTC.
No module defaults found
--> Starting dependency resolution
--> Finished dependency resolution
Error:
Problem: package packagegroup-base-wifi-1.0-r83.raspberrypi3_64 requires wireless-regdb-static, but none of the providers can be installed
- package wireless-regdb-2019.06.03-r0.noarch conflicts with wireless-regdb-static provided by wireless-regdb-static-2019.06.03-r0.noarch
- package packagegroup-base-1.0-r83.raspberrypi3_64 requires packagegroup-base-wifi, but none of the providers can be installed
- package packagegroup-rpi-test-1.0-r0.noarch requires wireless-regdb, but none of the providers can be installed
- package packagegroup-base-extended-1.0-r83.raspberrypi3_64 requires packagegroup-base, but none of the providers can be installed
- conflicting requests
(try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)
ERROR: Logfile of failure stored in: /opt/yocto/workspace/rpi3-64-build/tmp/work/raspberrypi3_64-poky-linux/rpi-test-image/1.0-r0/temp/log.do_rootfs.23419
ERROR: Task (/opt/yocto/workspace/sources/meta-raspberrypi/recipes-core/images/rpi-test-image.bb:do_rootfs) failed with exit code '1'
NOTE: Tasks Summary: Attempted 3511 tasks of which 1 didn't need to be rerun and 1 failed.
Summary: 1 task failed:
/opt/yocto/workspace/sources/meta-raspberrypi/recipes-core/images/rpi-test-image.bb:do_rootfs
Summary: There were 3 WARNING messages shown.
Summary: There was 1 ERROR message shown, returning a non-zero exit code.
/opt/yocto/workspace/rpi3-64-build/conf[master]☢ ☠$
I'm not even sure how to go about debugging it.
I've tried googling the warnings and error messages but was not able to find anything the might be causing the problem.
2 questions:
1: What might the problem be in this particular instance?
2: What is a good general procedure for debugging failed builds like this?
UPDATE: "bitbake core-image-base" does successfully build the image, but it is still unclear to me why "bitbake rpi-test-image" failed.
I am trying to include an external library (glm) in my bazel build, but I get undeclared inclusion the thing is I don't understand how to fix it and why, Because if I use the same BUILD script when I extract manually the library it builds fine.
I managed to make a very small repro of this problem, both the working case and the not working.
The not working example
My folder structure
glmExtBuild
WORKSPACE
externals
glm.BUILD
This is my WORKSPACE file
new_http_archive(
name = "glmExt",
url = "https://github.com/g-truc/glm/archive/0.9.9.0.tar.gz",
sha256 = "514dea9ac0099dc389cf293cf1ab3d97aff080abad55bf79d4ab7ff6895ee69c",
strip_prefix = "glm-0.9.9.0",
build_file = "externals/glm.BUILD",
)
This is my glm.BUILD file
package(default_visibility = ["//visibility:public"])
cc_library(
name= "glm",
defines= ["GLM_ENABLE_EXPERIMENTAL", "GLM_FORCE_RADIANS",
"GLM_FORCE_DEPTH_ZERO_TO_ONE"],
srcs=glob(["glm/**/*.cpp"]) + ["glm/detail/_fixes.hpp"],
hdrs=glob(["glm/**/*.hpp"])+glob(["glm/**/*.h"]),
includes = ["glm"],
textual_hdrs = glob(["glm/**/*.inl"]),
visibility = ["//visibility:public"],
)
And this is the errors that I get when I run
bazel build #glmExt//:glm --verbose_failures --sandbox_debug
ERROR: C:/users/...../external/glmExt/BUILD.bazel:121:1: undeclared inclusion(s) in rule '#glmExt//:glm':
this rule is missing dependency declarations for the following files included by 'external/glmExt/glm/detail/glm.cpp':
'external/glmext/glm/detail/_fixes.hpp'
'external/glmext/glm/detail/setup.hpp'
'external/glmext/glm/simd/platform.h'
'external/glmext/glm/fwd.hpp'
'external/glmext/glm/detail/setup.hpp'
'external/glmext/glm/detail/type_int.hpp'
'external/glmext/glm/detail/setup.hpp'
'external/glmext/glm/detail/type_float.hpp'
...
The working example:
My folder structure
glmBuild2
WORKSPACE
BUILD
glmSrc
https://github.com/g-truc/glm/archive/0.9.9.0.tar.gz extracted here
My WORKSPACE file is empty
And this is my BUILD file
package(default_visibility = ["//visibility:public"])
cc_library(
name= "glm",
defines= ["GLM_ENABLE_EXPERIMENTAL", "GLM_FORCE_RADIANS", "GLM_FORCE_DEPTH_ZERO_TO_ONE"],
srcs=glob(["glmSrc/glm/**/*.cpp"]) + ["glmSrc/glm/detail/_fixes.hpp"],
hdrs=glob(["glmSrc/glm/**/*.hpp"])+glob(["glmSrc/glm/**/*.h"]),
includes = ["glmSrc"],
textual_hdrs = glob(["glmSrc/glm/**/*.inl"]),
visibility = ["//visibility:public"],
)
Then when running the build command it finished successfully.
PS D:\......\glmBuild2> bazel build :glm --verbose_failures --sandbox_debug
INFO: Analysed target //:glm (1 packages loaded).
INFO: Found 1 target...
INFO: From Compiling glmSrc/glm/detail/glm.cpp:
glmSrc/glm/detail/glm.cpp(4): warning C4005: 'GLM_ENABLE_EXPERIMENTAL': macro redefinition
glmSrc/glm/detail/glm.cpp(4): note: command-line arguments: see previous definition of 'GLM_ENABLE_EXPERIMENTAL'
Target //:glm up-to-date:
C:/users/.../execroot/__main__/bazel-out/x64_windows-fastbuild/bin/libglm.a
INFO: Elapsed time: 1.830s, Critical Path: 1.30s
INFO: 3 processes, local.
INFO: Build completed successfully, 4 total actions
Does anybody have an idea?
Thanks
Update:
bazel version
Build label: 0.14.0
Build target: bazel-out/x64_windows-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Fri Jun 1 13:06:29 2018 (1527858389)
Build timestamp: 1527858389
Build timestamp as int: 1527858389
Bazel issue:
https://github.com/bazelbuild/bazel/issues/5485
I tried the first example on linux, with bazel 0.15.0, and all worked once I added "." into includes attribute of glm. Do you have a possibility to test it on linux? Is it windows-only issue then? Which bazel version do you use?
Thanks for clarifications.
I am trying to build Boost library locally and want to use it as a local_repository in my own project. I am referring to this (https://github.com/nelhage/rules_boost) for help.
I have a directory tree like this:
boost/ // root of the boost project
|
bazel/
| |
| boost.bzl // contains "boost_library" function from https://github.com/nelhage/rules_boost/blob/master/boost/boost.bzl
| |
| BUILD // empty
boost/ // the original boost headers folder from original boost dist
|
lib/ // the original boost sources folder from original boost dist
|
BUILD
|
WORKSPACE
The BUILD file looks like this one:
https://github.com/nelhage/rules_boost/blob/master/BUILD.boost
with properly loading boost.bzl
The WORKSPACE is just:
workspace( name = "boost" )
The issue:
Now I am trying to build individual components (bazel build //:<component>).
Some of the components (Boost.Container, Boost.Test) are failing to build for a similar reason (not finding header files).
bazel build //:container 1 ↵
INFO: Found 1 target...
ERROR: /home/spyder/codebase/boost/BUILD:103:1: C++ compilation of rule '//:container' failed: Process exited with status 1 [sandboxed].
libs/container/src/global_resource.cpp:12:51: fatal error: boost/container/pmr/memory_resource.hpp: No such file or directory
compilation terminated.
Use --strategy=CppCompile=standalone to disable sandboxing for the failing actions.
Target //:container failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 0.519s, Critical Path: 0.18s
bazel build //:test 1 ↵
INFO: Found 1 target...
ERROR: /home/spyder/codebase/boost/BUILD:581:1: C++ compilation of rule '//:test' failed: Process exited with status 1 [sandboxed].
libs/test/src/junit_log_formatter.cpp:11:51: fatal error: boost/test/impl/junit_log_formatter.ipp: No such file or directory
compilation terminated.
Use --strategy=CppCompile=standalone to disable sandboxing for the failing actions.
Target //:test failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 1.027s, Critical Path: 0.82s
Can anyone guide me on fixing it ?
EDIT:
I also tried to query the expanded cc_library rule by bazel query --output=build //:container and got this
cc_library(
name = "container",
visibility = ["//visibility:public"],
generator_name = "container",
generator_function = "boost_library",
generator_location = "/home/spyder/codebase/boost/BUILD:103",
licenses = ["notice"],
deps = ["//:config", "//:core", "//:intrusive", "//:move"],
defines = [],
includes = ["boost/container/"],
copts = ["-Wno-unused-value"],
srcs = ["//:libs/container/src/alloc_lib.c", "//:libs/container/src/dlmalloc.cpp", "//:libs/container/src/dlmalloc_2_8_6.c", "//:libs/container/src/dlmalloc_ext_2_8_6.c", "//:libs/container/src/global_resource.cpp", "//:libs/container/src/monotonic_buffer_resource.cpp", "//:libs/container/src/pool_resource.cpp", "//:libs/container/src/synchronized_pool_resource.cpp", "//:libs/container/src/unsynchronized_pool_resource.cpp"],
hdrs = [ ....... , "//:boost/container/pmr/map.hpp", "//:boost/container/pmr/memory_resource.hpp", "//:boost/container/pmr/monotonic_buffer_resource.hpp", "//:boost/container/pmr/polymorphic_allocator.hpp", ........ ],
)
Very shockingly, the source file which is complaining (libs/container/src/global_resource.cpp) and the header that it is complaining about (boost/container/pmr/memory_resource.hpp) are both correctly included in the srcs and hdrs list.
I played with your rules a bit and the biggest problem I see is incomplete dependencies. For example I tried :algorithm component, and that needs to depend on :tuple. I guess you just needs to make sure all dependencies are set correctly.
I played more with container, and the problem I encountered was incorrect include dirs. If you run your build with -s flag, bazel will output all the command lines so you can inspect them and check that -isystem flags contain the directory that contains boost/container/... . What can also help you is the bazel flag --sandbox_debug, which will prevent the sandbox directory from being deleted so you can see which files are where and also you can reproduce the compiler invocation without running through bazel.
Last, bazel does not understand .ipp header extension yet (tbh I've never heard of it, but I also know next to nothing about boost). I have a change in flight that will introduce this.
I can't get SyntaxNet compiled on my MacBook Pro.
I followed the SyntaxNet installation steps here: https://github.com/tensorflow/models/tree/master/syntaxnet#installation
the bazel test --linkopt=-headerpad_max_install_names syntaxnet/... util/utf8/... line fails with this output:
Joachims-MacBook-Pro:syntaxnet joachim$ bazel test --linkopt=-headerpad_max_install_names syntaxnet/... util/utf8/...
ERROR: /private/var/tmp/_bazel_joachim/2d6c6b79fd5d2f10f2fd8d8e3457de30/external/org_tensorflow/tensorflow/core/platform/default/build_config/BUILD:108:1: error loading package '#jpeg//': Extension file not found. Unable to load package for '//third_party:common.bzl': BUILD file not found on package path and referenced by '#org_tensorflow//tensorflow/core/platform/default/build_config:jpeg'.
ERROR: Analysis of target '//syntaxnet:test_main' failed; build aborted.
INFO: Elapsed time: 4.823s
ERROR: Couldn't start the build. Unable to run tests.
Here's some system info:
Joachims-MacBook-Pro:syntaxnet joachim$ bazel version
Build label: 0.4.3-homebrew
Build target: bazel-out/local-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Thu Dec 22 15:20:22 2016 (1482420022)
Build timestamp: 1482420022
Build timestamp as int: 1482420022
Joachims-MacBook-Pro:syntaxnet joachim$ python --version
Python 2.7.11
I pulled the sources as follows:
Joachims-MacBook-Pro:workspace joachim$ git clone --recurse-submodules https://github.com/tensorflow/models.git
Cloning into 'models'
...
'aab099711d7e04034cf742ddb9b00dd15edbe99c'
I encountered a similar problem when compiling on Ubuntu.
Find the file tensorflow/third_party/jpeg/jpeg.BUILD
Then, change the code load("#//third_party:common.bzl", "template_rule")
to
load("#org_tensorflow//third_party:common.bzl", "template_rule")