Golang Bazel empty coverage_report.dat - go

I am working on a small Golang monorepo and I am using Bazel + gazelle to manage deps along with tests, builds, coverage,...
I am having an issue when it comes to coverage. I have followed a few examples but for some reason the coverage_report.dat file (that then will be processed by genhtml) is always empty.
This is the command that I am running with relative output
$ bazel coverage --combined_report=lcov --cache_test_results=no --test_output=all //...
INFO: Using default value for --instrumentation_filter: "^//service/hello[/:],^//service/hellotwo[/:]".
INFO: Override the above default with --instrumentation_filter
INFO: Analyzed 4 targets (0 packages loaded, 0 targets configured).
INFO: Found 2 targets and 2 test targets...
INFO: From Testing //service/hello:hello_test:
==================== Test output for //service/hello:hello_test:
PASS
coverage: 100.0% of statements
================================================================================
INFO: From Testing //service/hellotwo:hellotwo_test:
==================== Test output for //service/hellotwo:hellotwo_test:
PASS
coverage: 100.0% of statements
================================================================================
INFO: LCOV coverage report is located at /private/var/tmp/_bazel_andrea/292e77b095155ea362773b482c2c4621/execroot/__main__/bazel-out/_coverage/_coverage_report.dat
and execpath is bazel-out/_coverage/_coverage_report.dat
INFO: Elapsed time: 0.217s, Critical Path: 0.09s
INFO: 3 processes: 1 internal, 2 darwin-sandbox.
INFO: Build completed successfully, 3 total actions
//service/hello:hello_test PASSED in 0.1s
/private/var/tmp/_bazel_andrea/292e77b095155ea362773b482c2c4621/execroot/__main__/bazel-out/darwin-fastbuild/testlogs/service/hello/hello_test/coverage.dat
//service/hellotwo:hellotwo_test PASSED in 0.1s
/private/var/tmp/_bazel_andrea/292e77b095155ea362773b482c2c4621/execroot/__main__/bazel-out/darwin-fastbuild/testlogs/service/hellotwo/hellotwo_test/coverage.dat
Executed 2 out of 2 tests: 2 tests pass.
INFO: Build completed successfully, 3 total actions
The coverage is calculated but bazel-out/_coverage/_coverage_report.dat is empty. I was wondering if anyone has had this issue before and managed to find a solution to it.
I am working on a MacBook Pro with Bazel (v4.1.0) installed via Homebrew.

Related

Build Docker Images with Bazel targets Built

I have a (1) Dockerfile and a (2) C++ project with Bazel
I want to create a docker image that has Bazel targets pre-built within the image, so as to when I power up new containers the Bazel targets are pre-built and I just do Bazel run //hello:hello_world from the container bash.
Dockerfile
# Copy my project with Bazel files to a Docker image, and the
...
RUN bazel --output_user_root=/tmp/hello_project/bazel build //...
...
Within the execution of the Dockerfile, I get the following output which is expected
Loading:
Loading: 0 packages loaded
Analyzing: 2 targets (1 packages loaded, 0 targets configured)
Analyzing: 2 targets (11 packages loaded, 18 targets configured)
INFO: Analyzed 2 targets (15 packages loaded, 60 targets configured).
INFO: Found 2 targets...
[0 / 11] [Prepa] BazelWorkspaceStatusAction stable-status.txt
INFO: Elapsed time: 6.333s, Critical Path: 0.37s
INFO: 11 processes: 6 internal, 5 processwrapper-sandbox.
INFO: Build completed successfully, 11 total actions
INFO: Build completed successfully, 11 total actions
When I run a new container form the Docker Image built previously and then on the container I run
bazel run //hello:hello_world
Instead of using existing pre-built targets it re-builds the targets, which is not required.
Result I expect (not get): Everything is pre-built and just needs to run
INFO: Analyzed target //hello:hello_world (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //hello:hello_world up-to-date:
bazel-bin/hello/hello_world
INFO: Elapsed time: 0.163s, Critical Path: 0.01s
INFO: 1 process: 1 internal.
INFO: Build completed successfully, 1 total action
INFO: Build completed successfully, 1 total action
Hello World!
Result I get: I re-builds the binaries
[root#4a6bdb57fd79 test-rc]# bazel run //hello:hello_world
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
INFO: Analyzed target //hello:hello_world (15 packages loaded, 60 targets configured).
INFO: Found 1 target...
Target //hello:hello_world up-to-date:
bazel-bin/hello/hello_world
INFO: Elapsed time: 6.255s, Critical Path: 0.38s
INFO: 7 processes: 4 internal, 3 processwrapper-sandbox.
INFO: Build completed successfully, 7 total actions
INFO: Build completed successfully, 7 total actions
Hello World!
How can I make sure, the the bazel run uses same pre-built targets and not build them again before run.
This sounds like non-determinism - in the second execution of Bazel, something is different causing a cache miss.
Some things that can cause it:
different options passed to bazel build vs. bazel test - check your .bazelrc file
actions that include VCS info or a timestamp - make sure you have rules_docker 0.22.0 or later to pick up https://github.com/bazelbuild/rules_docker/commit/2b35b2dd56f0be6cc6b8df957332a31435f6b3ce
One step to diagnose is to use the --explain=log.txt and --verbose_explanations flags to Bazel, then the log file will say why it was rebuilt. However it doesn't have much detail, just something like "a source file has changed".
If you want the power tool for this, there is a way to find out exactly why Bazel didn't get a cache hit - read https://georgi.hristozov.net/til/2020/04/20/compare-bazel-execlogs-to-find-non-deterministic-parts-of-the-build

Cannot build go_proto_library with gRPC

I'm getting started with bazel and trying to generate the protobuf code for golang for an RPC service.
When I try to build it I get the following error:
bazel-out/k8-fastbuild/bin/examples/grpc/protos/helloworld_go_proto_/examples/grpc/protos/helloworld.pb.go:229:7: undefined: grpc.ClientConnInterface
bazel-out/k8-fastbuild/bin/examples/grpc/protos/helloworld_go_proto_/examples/grpc/protos/helloworld.pb.go:233:11: undefined: grpc.SupportPackageIsVersion6
bazel-out/k8-fastbuild/bin/examples/grpc/protos/helloworld_go_proto_/examples/grpc/protos/helloworld.pb.go:243:5: undefined: grpc.ClientConnInterface
bazel-out/k8-fastbuild/bin/examples/grpc/protos/helloworld_go_proto_/examples/grpc/protos/helloworld.pb.go:246:26: undefined: grpc.ClientConnInterface
Full build log: https://app.buildbuddy.io/invocation/c3773978-22dd-44c8-b977-13967a1953b7
Here is the code: https://github.com/juanique/example-go-grpc. I'm trying to include the least possible amount of code to make that target work.
Since the BUILD file was generated by gazelle I suspect the issue is in the WORKSPACE file: https://raw.githubusercontent.com/juanique/example-go-grpc/main/WORKSPACE. I'm just doing what I found in https://github.com/bazelbuild/rules_go
UPDATE: it looks surely a version issue: https://github.com/grpc/grpc-go#compiling-error-undefined-grpcsupportpackageisversion
Not a bazel user, but after hours' test, I found your rpc version should be higher:
go_repository(
name = "org_golang_google_grpc",
- build_file_proto_mode = "disable",
importpath = "google.golang.org/grpc",
- sum = "h1:J0UbZOIrCAl+fpTOf8YLs4dJo8L/owV4LYVtAXQoPkw=",
- version = "v1.22.0",
+ sum = "h1:f+PlOh7QV4iIJkPrx5NQ7qaNGFQ3OTse67yaDHfju4E=",
+ version = "v1.41.0",
)
Then you can build it:
~/code/example-go-grpc (main*) [09:45:55]
p1gd0g$ bazel build //examples/grpc/protos:helloworld_go_proto
INFO: Analyzed target //examples/grpc/protos:helloworld_go_proto (49 packages loaded, 306 targets configured).
INFO: Found 1 target...
Target //examples/grpc/protos:helloworld_go_proto up-to-date:
bazel-bin/examples/grpc/protos/helloworld_go_proto.a
INFO: Elapsed time: 41.180s, Critical Path: 1.20s
INFO: 29 processes: 1 internal, 28 linux-sandbox.
INFO: Build completed successfully, 29 total actions
And I recommend using gazelle with go.mod to generate WORKSPACE. A demo: https://github.com/p1gd0g/helloworld

bitbake rpi-test-image with MACHINE=raspberrypi3-64 failing to build on the zeus release of yocto

I'm following the readthedocs instructions for the meta-raspberrypi layer and trying to build the rpi-test-image image for the raspberrypi3-64 machine contained in the meta-raspberrypi layer using the zeus release of yocto.
I added this to conf/local.conf:
MACHINE ??= "raspberrypi3-64"
ENABLE_UART = "1"
My bblayers.conf file looks like this:
/opt/yocto/workspace/rpi3-64-build/conf[master]☢ ☠$ cat bblayers.conf
# POKY_BBLAYERS_CONF_VERSION is increased each time build/conf/bblayers.conf
# changes incompatibly
POKY_BBLAYERS_CONF_VERSION = "2"
BBPATH = "${TOPDIR}"
BBFILES ?= ""
BBLAYERS ?= " \
/opt/yocto/workspace/sources/poky/meta \
/opt/yocto/workspace/sources/poky/meta-poky \
/opt/yocto/workspace/sources/poky/meta-yocto-bsp \
/opt/yocto/workspace/sources/meta-openembedded/meta-oe \
/opt/yocto/workspace/sources/meta-openembedded/meta-multimedia \
/opt/yocto/workspace/sources/meta-openembedded/meta-networking \
/opt/yocto/workspace/sources/meta-openembedded/meta-python \
/opt/yocto/workspace/sources/meta-raspberrypi \
"
I added the 4 items from meta-openembeeded as a work-around for ERROR: Nothing RPROVIDES 'bigbuckbunny-480p':
This enables the build to run but it exits with the following errors:
/opt/yocto/workspace/rpi3-64-build/conf[master]☢ ☠$ bitbake rpi-test-image
Parsing recipes: 100% |#########################################################################################################################################| Time: 0:02:49
Parsing of 2340 .bb files complete (0 cached, 2340 parsed). 3464 targets, 133 skipped, 0 masked, 0 errors.
NOTE: Resolving any missing task queue dependencies
Build Configuration:
BB_VERSION = "1.44.0"
BUILD_SYS = "x86_64-linux"
NATIVELSBSTRING = "ubuntu-16.04"
TARGET_SYS = "aarch64-poky-linux"
MACHINE = "raspberrypi3-64"
DISTRO = "poky"
DISTRO_VERSION = "3.0.2"
TUNE_FEATURES = "aarch64 cortexa53 crc"
TARGET_FPU = ""
meta
meta-poky
meta-yocto-bsp = "zeus:74f229160c7f4037107c1dad8f0d02128c080a7e"
meta-oe
meta-multimedia
meta-networking
meta-python = "zeus:9e60d30669a2ad0598e9abf0cd15ee06b523986b"
meta-raspberrypi = "zeus:0e05098853eea77032bff9cf81955679edd2f35d"
Initialising tasks: 100% |######################################################################################################################################| Time: 0:00:06
Sstate summary: Wanted 1338 Found 0 Missed 1338 Current 0 (0% match, 0% complete)
NOTE: Executing Tasks
NOTE: Setscene tasks completed
WARNING: icu-native-64.2-r0 do_fetch: Checksum mismatch for local file /opt/yocto/cache/downloads/icu4c-64_2-src.tgz
Cleaning and trying again.
WARNING: icu-native-64.2-r0 do_fetch: Renaming /opt/yocto/cache/downloads/icu4c-64_2-src.tgz to /opt/yocto/cache/downloads/icu4c-64_2-src.tgz_bad-checksum_abb12cb25a05198ad8f4c1e6f668fa05
WARNING: icu-native-64.2-r0 do_fetch: Checksum failure encountered with download of http://download.icu-project.org/files/icu4c/64.2/icu4c-64_2-src.tgz - will attempt other sources if available
ERROR: rpi-test-image-1.0-r0 do_rootfs: Could not invoke dnf. Command '/opt/yocto/workspace/rpi3-64-build/tmp/work/raspberrypi3_64-poky-linux/rpi-test-image/1.0-r0/recipe-sysroot-native/usr/bin/dnf -v --rpmverbosity=info -y -c /opt/yocto/workspace/rpi3-64-build/tmp/work/raspberrypi3_64-poky-linux/rpi-test-image/1.0-r0/rootfs/etc/dnf/dnf.conf --setopt=reposdir=/opt/yocto/workspace/rpi3-64-build/tmp/work/raspberrypi3_64-poky-linux/rpi-test-image/1.0-r0/rootfs/etc/yum.repos.d --installroot=/opt/yocto/workspace/rpi3-64-build/tmp/work/raspberrypi3_64-poky-linux/rpi-test-image/1.0-r0/rootfs --setopt=logdir=/opt/yocto/workspace/rpi3-64-build/tmp/work/raspberrypi3_64-poky-linux/rpi-test-image/1.0-r0/temp --repofrompath=oe-repo,/opt/yocto/workspace/rpi3-64-build/tmp/work/raspberrypi3_64-poky-linux/rpi-test-image/1.0-r0/oe-rootfs-repo --nogpgcheck install psplash-raspberrypi packagegroup-core-boot packagegroup-base-extended run-postinsts packagegroup-rpi-test locale-base-en-us locale-base-en-gb' returned 1:
DNF version: 4.2.2
cachedir: /opt/yocto/workspace/rpi3-64-build/tmp/work/raspberrypi3_64-poky-linux/rpi-test-image/1.0-r0/rootfs/var/cache/dnf
Added oe-repo repo from /opt/yocto/workspace/rpi3-64-build/tmp/work/raspberrypi3_64-poky-linux/rpi-test-image/1.0-r0/oe-rootfs-repo
repo: using cache for: oe-repo
not found other for:
not found modules for:
not found deltainfo for:
not found updateinfo for:
oe-repo: using metadata from Thu 30 Apr 2020 09:25:08 PM UTC.
Last metadata expiration check: 0:00:02 ago on Thu 30 Apr 2020 09:25:11 PM UTC.
No module defaults found
--> Starting dependency resolution
--> Finished dependency resolution
Error:
Problem: package packagegroup-base-wifi-1.0-r83.raspberrypi3_64 requires wireless-regdb-static, but none of the providers can be installed
- package wireless-regdb-2019.06.03-r0.noarch conflicts with wireless-regdb-static provided by wireless-regdb-static-2019.06.03-r0.noarch
- package packagegroup-base-1.0-r83.raspberrypi3_64 requires packagegroup-base-wifi, but none of the providers can be installed
- package packagegroup-rpi-test-1.0-r0.noarch requires wireless-regdb, but none of the providers can be installed
- package packagegroup-base-extended-1.0-r83.raspberrypi3_64 requires packagegroup-base, but none of the providers can be installed
- conflicting requests
(try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)
ERROR: Logfile of failure stored in: /opt/yocto/workspace/rpi3-64-build/tmp/work/raspberrypi3_64-poky-linux/rpi-test-image/1.0-r0/temp/log.do_rootfs.23419
ERROR: Task (/opt/yocto/workspace/sources/meta-raspberrypi/recipes-core/images/rpi-test-image.bb:do_rootfs) failed with exit code '1'
NOTE: Tasks Summary: Attempted 3511 tasks of which 1 didn't need to be rerun and 1 failed.
Summary: 1 task failed:
/opt/yocto/workspace/sources/meta-raspberrypi/recipes-core/images/rpi-test-image.bb:do_rootfs
Summary: There were 3 WARNING messages shown.
Summary: There was 1 ERROR message shown, returning a non-zero exit code.
/opt/yocto/workspace/rpi3-64-build/conf[master]☢ ☠$
I'm not even sure how to go about debugging it.
I've tried googling the warnings and error messages but was not able to find anything the might be causing the problem.
2 questions:
1: What might the problem be in this particular instance?
2: What is a good general procedure for debugging failed builds like this?
UPDATE: "bitbake core-image-base" does successfully build the image, but it is still unclear to me why "bitbake rpi-test-image" failed.

Flink Error: Could not find or load main class

I'm trying to run those Flink Benchmarks:
https://github.com/dataArtisans/flink-benchmarks
I've generated the jar file using maven with that command:
mvn clean package -Pbuild-jar
Then I'm trying to run the benchmark on a Flink Cluster with that command:
./bin/flink run -c org.apache.flink.benchmark.WindowBenchmarks ~/flinkBenchmarks/target/flink-hackathon-benchmarks-0.1.jar
I've used the -c option to add to the classpath the Main of the benchmark (WindowBenchmarks) I want to run.
Finally, I get that error:
# JMH version: 1.19
# VM version: JDK 1.8.0_151, VM 25.151-b12
# VM invoker: /usr/lib/jvm/java-8-oracle/jre/bin/java
# VM options: -Dlog.file=/home/user/flink-1.3.2/flink-dist/target/flink-1.3.2-bin/flink-1.3.2/log/flink-user-client-mypc.log -Dlog4j.configuration=file:/home/user/flink-1.3.2/flink-dist/target/flink-1.3.2-bin/flink-1.3.2/conf/log4j-cli.properties -Dlogback.configurationFile=file:/home/user/flink-1.3.2/flink-dist/target/flink-1.3.2-bin/flink-1.3.2/conf/logback.xml -Djava.rmi.server.hostname=127.0.0.1 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false
# Warmup: 10 iterations, 1 s each
# Measurement: 10 iterations, 1 s each
# Timeout: 10 min per iteration
# Threads: 1 thread, will synchronize iterations
# Benchmark mode: Throughput, ops/time
# Benchmark: org.apache.flink.benchmark.WindowBenchmarks.sessionWindow
# Run progress: 0.00% complete, ETA 00:04:00
# Fork: 1 of 3
Error: Could not find or load main class org.openjdk.jmh.runner.ForkedMain
<forked VM failed with exit code 1>
<stdout last='20 lines'>
</stdout>
<stderr last='20 lines'>
Error: Could not find or load main class org.openjdk.jmh.runner.ForkedMain
</stderr>
# Run complete. Total time: 00:00:00
Benchmark Mode Cnt Score Error Units
The program didn't contain a Flink job. Perhaps you forgot to call execute() on the execution environment.
I don't have any previous experience with Flink and Maven so I find out what is missing. My first thought was that it's a missing dependencies error, but they look fine. Any suggestions?
Thank you in advance!
flink-benchmarks is a repository that contains sets of micro benchmarks designed to run on single machine, not on the cluster. The main functions defined in the various classes (test cases) are 'JMH' runners, not Flink programs. As such you can either execute whole benchmark suite (which takes ~1hour):
mvn -Dflink.version=1.5.0 clean install exec:exec
or if you want to execute just one benchmark, the best approach is to execute selected main function manually. For example from your IDE (don't forget about selecting flink.version, default value for the property is defined in pom.xml).
There is also a possibility to execute single benchmark from console, but I haven't tried it for very long time.

Pex run in loop after reporting?

i run pex by command line on a class of about 2200 methods.
After 2 days of elaboration, I think to have the test suite and report html, pex finished this morning at 8.05 AM and the list file modified is at 08.40 AM.
The last line of output are :
08:05:08.3> [finished] execution time 2.08:05:08.3014942.
-- 0 critical errors, 0 errors, 0 warnings.
-- 83635 generated tests, 1 failing, 83635 new, 0 inconclusive.
[coverage] generating coverage reports...
[reports] generating reports...
[reports] report path: reports\121005.001302.3848.pex
The process pex.exe is running continuos, But what is happened ? After about 3.5 hours ( now for me is 12.15 pm ) its maybe is going to loop or what is the operation is it doing ?
CPU is full and memory too ( process pex 1.5G of RAM)
Do you think I have to stop it?
Thanks best regards.
After 2 days and about 14h I stop the process :
[coverage] generating coverage reports...
[reports] generating reports...
[reports] report path: reports\121005.001302.3848.pex
!error! [reports] rendering of coverage reports failed: exit code 1 - 0x1
Microsoft Pex v0.94.51023.0 -- http://research.microsoft.com/pex -- v4.0.30319
Copyright (c) Microsoft Corporation 2007-2010. All rights reserved.
[tests] could not resolve test framework, using default (Visual Studio Unit Test
)
[reports] html report: reports\121005.001302.3848.pex\report.html
LAUNCHER FAILED
unexpected failure: exit code 1 - 0x1
Maybe I have few resource ( 2G RAM and a dual core ) and pex doesn't do the report.html

Resources