I'm on MacOS 10.13.2.
Go 1.10.
bazel 0.11.1
I need to compile a repo that has 2 projects (project1 and project2).
project1 has 2 subpacakges. p1lib and dep1
p1lib uses dep1.
I generate BUILD files with gazelle, files look ok.
gazelle -go_prefix=github.com/BazelBuildForGo
But when I run build I get an error that says that I'm missing direct dependency.
bazel build //project1
INFO: Analysed target //project1:project1 (3 packages loaded).
INFO: Found 1 target...
ERROR: /private/var/tmp/_bazel_user1/df78026a5ee0c7ed3d23dd05c3a3b1f7/external/com_github_wix_private_bazelbuildforgo/project1/p1lib/BUILD.bazel:3:1: GoCompile external/com_github_wix_private_bazelbuildforgo/project1/p1lib/darwin_amd64_stripped/go_default_library~/github.com/BazelBuildForGo/project1/p1lib.a failed (Exit 1)
2018/03/25 18:02:55 missing strict dependencies:
external/com_github_wix_private_bazelbuildforgo/project1/p1lib/p1lib.go: import of github.com/wix-private/BazelBuildForGo/project1/dep1, which is not a direct dependency
Target //project1:project1 failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 0.358s, Critical Path: 0.12s
FAILED: Build did NOT complete successfully
My project can be found here
https://github.com/wix-playground/BazelBuildForGo
I think the problem is that the import prefix you passed to Gazelle on the command line (github.com/BazelBuildForGo) is different from the imports in the .go files (github.com/wix-private/BazelBuildForGo). When Gazelle sees imports that are outside the current prefix, it will generate external dependencies for those imports, and those dependencies will be missing:
go_library(
name = "go_default_library",
srcs = ["p1lib.go"],
importpath = "github.com/BazelBuildForGo/project1/p1lib",
visibility = ["//visibility:public"],
deps = ["#com_github_wix_private_bazelbuildforgo//project1/dep1:go_default_library"],
)
The fix for this is pretty easy though. Just run Gazelle with the prefix github.com/wix-private/BazelBuildForGo. You actually already have this in //:gazelle, so just run that, then rebuild.
$ bazel run //:gazelle
$ bazel build //...
That will change the go_library rule above to this:
go_library(
name = "go_default_library",
srcs = ["p1lib.go"],
importpath = "github.com/wix-private/BazelBuildForGo/project1/p1lib",
visibility = ["//visibility:public"],
deps = ["//project1/dep1:go_default_library"],
)
Related
I have a .proto protobuf definition file in a dir and I'm building a go library from it with Bazel like so (BUILD.bazel file below generated using gazelle):
load("#rules_proto//proto:defs.bzl", "proto_library")
load("#io_bazel_rules_go//go:def.bzl", "go_library")
load("#io_bazel_rules_go//proto:def.bzl", "go_proto_library")
proto_library(
name = "events_proto",
srcs = ["events.proto"],
visibility = ["//visibility:public"],
deps = ["#com_google_protobuf//:timestamp_proto"],
)
go_proto_library(
name = "proto_go_proto",
importpath = "github.com/acme/icoyote/proto",
proto = ":events_proto",
visibility = ["//visibility:public"],
)
go_library(
name = "proto",
embed = [":proto_go_proto"],
importpath = "github.com/acme/icoyote/proto",
visibility = ["//visibility:public"],
)
Some other code depends on //icoyote/proto:proto, and when I run go mod tidy in my module, it complains that it can't find the package github.com/acme/icoyote/proto:
go: finding module for package github.com/acme/icoyote/proto
github.com/acme/icoyote/cmd/icoyote imports
github.com/acme/icoyote/proto: no matching versions for query "latest"
Any IDE that doesn't have Bazel integration (e.g. VSCode, GoLand/IntelliJ without the Bazel plugin) complains as well
What do I do?
This is happening of course because because Bazel does generate .go files using protoc under the covers for the go_proto_library rule in the BUILD file, but only writes them out to a temp dir under bazel-bin to be used by the go_library rule, and go mod tidy doesn't seem look into bazel-bin (probably because it's a symlink but also if it did, the path of those files relative to the location of go.mod is all wrong)
One option is to manually generate the go files by calling protoc on your own, and remove the proto_library and go_proto_library rules in the BUILD file, then change the go_library rule to build your generated files. This is suboptimal because you have to manually rerun protoc every time you make changes to the .proto file (and if you put it into a //go:generate directive, you have to rerun go generate).
Instead, we can do the following:
Add a file empty.go to the dir that contains the .proto file. It should look like this:
package proto
Then tell gazelle to ignore empty.go (so it doesn't try to add a go_library rule to the BUILD file when you run gazelle --fix). We do that by adding the following to the BUILD file:
# gazelle:exclude empty.go
That's enough to make go mod tidy shut up.
This will also make the IDE stop complaining about the import, although you'll still get errors when referring to anything that's supposed to be in that package. If you don't want to abandon your IDE for an excellent GoLand or IntelliJ IDEA with a Bazel plugin, you might have to resort to the manual protoc method. Perhaps there's a way to create a symlink to wherever Bazel writes out the generated .go files under bazel-bin and force go mod tidy to follow it, but I haven't tried that. If you do and it works, do share!
I'm trying to run some grpc tests with bazel.
I'm using "google.golang.org/grpc/credentials/insecure" to dial insecurely.
When running bazel test ..., I get the following error:
no such package '#org_golang_google_grpc//credentials/insecure': BUILD file not found in directory 'credentials/insecure' of external repository #org_golang_google_grpc. Add a BUILD file to a directory to mark it as a package. and referenced by '//go/internal/handlers/helloworld:helloworld_test'
I am generating my BUILD files with gazelle which outputs this for the go_test
go_test(
name = "helloworld_test",
srcs = ["helloworld_test.go"],
deps = [
":helloworld",
"//protos/helloworld",
"#com_github_stretchr_testify//assert",
"#org_golang_google_grpc//:go_default_library",
"#org_golang_google_grpc//credentials/insecure",
"#org_golang_google_grpc//test/bufconn",
"#org_uber_go_zap//:zap",
"#org_uber_go_zap//zaptest",
],
)
My go.mod file contains the dep:
google.golang.org/grpc v1.47.0
My deps.bzl is auto generated by gazelle:
go_repository(
name = "org_golang_google_grpc",
importpath = "google.golang.org/grpc",
sum = "h1:9n77onPX5F3qfFCqjy9dhn8PbNQsIKeVU04J9G7umt8=",
version = "v1.47.0",
)
What am I missing?
In the WORKSPACE, the local go_repositories() generated by gazelle must be called before gazelle_dependencies(), which will
define an ancient version of org_golang_google_grpc if it doesn't
exist yet and the local go_repository for the newer version will be
silently ignored. Reference
I'm using bazel to run a go test on my CI system. The go test will need to use a helper binary built from another go file.
The dir structure looks like the following:
some/of/my/path
├── BUILD.bazel
├── my_utils.go
├── my_test.go
└── my_tool
├── BUILD.bazel
└── my_tool.go
I.e. the code in my_test.go is going to use the executable binary built from dir my_tool.
In a non-bazel case, I can just go build ./mytool to get the executable binary that my_test.go needs. To accommodate this the bazel world, I write the following in the bazel files:
In some/of/my/path/BUILD.bazel:
load("#io_bazel_rules_go//go:def.bzl", "go_library", "go_test")
go_library(
name = "testutils",
srcs = ["my_utils.go"],
importpath = "some/of/my/path",
visibility = ["//visibility:public"],
deps = [
"#org_golang_x_net//context",
... (some other dependencies, not related)
],
)
go_test(
name = "my_test",
size = "enormous",
srcs = ["mytest.go"],
embed = [":testutils"],
deps = [
"//some/of/my/path/my_tool:my_tool",
],
)
In some/of/my/path/my_tool/BUILD.bazel
load("#io_bazel_rules_go//go:def.bzl", "go_binary", "go_library")
go_library(
name = "my_tool_lib",
srcs = ["my_tool.go"],
importpath = "some/of/my/path/my_tool",
visibility = ["//visibility:private"],
deps = ["some dependency"],
)
go_binary(
name = "my_tool",
embed = [":my_tool_lib"],
visibility = ["//visibility:public"],
)
So the idea is when I hit bazel run, it will build the my_tool go binary, which the go test my_test can use (see the deps field of the go test).
While in the CI I hit this error:
ERROR: /home/agent/work/.go/src/some/of/my/path/BUILD.bazel:20:8: in go_test rule //some/of/my/path:my_test:
Traceback (most recent call last):
File "/home/agent/.cache/bazel/_bazel_agent/40f5e2a2e18a7cdb4cd075f919d8072b/external/io_bazel_rules_go/go/private/rules/test.bzl", line 66, column 43, in _go_test_impl
internal_source = go.library_to_source(go, ctx.attr, internal_library, ctx.coverage_instrumented())
File "/home/agent/.cache/bazel/_bazel_agent/40f5e2a2e18a7cdb4cd075f919d8072b/external/io_bazel_rules_go/go/private/context.bzl", line 247, column 26, in _library_to_source
_check_binary_dep(go, dep, "deps")
File "/home/agent/.cache/bazel/_bazel_agent/40f5e2a2e18a7cdb4cd075f919d8072b/external/io_bazel_rules_go/go/private/context.bzl", line 295, column 13, in _check_binary_dep
fail("rule {rule} depends on executable {dep} via {edge}. This is not safe for cross-compilation. Depend on go_library instead.".format(
Error in fail: rule //some/of/my/path:my_test depends on executable //some/of/my/path/my_tool:my_tool via deps. This is not safe for cross-compilation. Depend on go_library instead.
ERROR: Analysis of target '//some/of/my/path:my_test' failed; build aborted: Analysis of target '//some/of/my/path:my_test' failed
INFO: Elapsed time: 79.552s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (595 packages loaded, 12182 targets configured)
ERROR: Build failed. Not running target
FAILED: Build did NOT complete successfully (595 packages loaded, 12182 targets configured)
Process exited with code 1
But my goal is to use the binary. How should I change the configs to achieve that?
You want data, not deps. deps is for things linked into a binary, data is for things the binary uses at runtime.
Once your binary is in data, use runfiles.go to run it. Add #io_bazel_rules_go//go/tools/bazel:go_default_library to deps (it's a library you want linked in), and then use FindBinary to find the path.
I have a workspace where I use gazelle to generate my BUILD files and for some reason the com_github_ipfs_go_merkledag dependency is breaking the build when trying to resolve it's github.com/gogo/protobuf/gogoproto dependency. The setup and run is below.
Workspace:
# Declare indirect dependencies and init toolchains.
go_rules_dependencies()
go_register_toolchains(version = "1.16")
go_embed_data_dependencies()
load("#rules_proto//proto:repositories.bzl", "rules_proto_dependencies", "rules_proto_toolchains")
rules_proto_dependencies()
rules_proto_toolchains()
load("#bazel_gazelle//:deps.bzl", "gazelle_dependencies", "go_repository")
gazelle_dependencies()
Commands:
bazel run //:gazelle
bazel run //:gazelle -- update-repos -from_file=go.mod
bazel build //...
Output:
...
no such package '#com_github_ipfs_go_merkledag//github.com/gogo/protobuf/gogoproto':
BUILD file not found in directory 'github.com/gogo/protobuf/gogoproto' of external repository #com_github_ipfs_go_merkledag.
Add a BUILD file to a directory to mark it as a package. and referenced by '#com_github_ipfs_go_merkledag//pb:merkledag_pb_go_proto'
...
Not sure what the solution for this is?
Add build directives to the go_repository to ignore
go_repository(
name = "com_github_ipfs_go_merkledag",
importpath = "github.com/ipfs/go-merkledag",
sum = "h1:ixNu/5MJSaT/Qs073T0/HsWKwnOoBgqSq1g+GaJIen0=",
version = "v0.4.0",
build_directives = [
"gazelle:proto disable",
],
)
Consider a repo, github.com/myacct/sandbox, with a ji directory at the top:
$ ls -CFR
.:
BUILD.bazel WORKSPACE one/
./one:
oneonone.go
$ cat BUILD.bazel
load("#bazel_gazelle//:def.bzl", "gazelle")
# gazelle:prefix github.com/myacct/sandbox/ji
gazelle(name = "gazelle")
Pretty standard stuff, straight out of the docs. I have already run bazel clean --expunge. Now:
$ bazel run //:gazelle
Starting local Bazel server and connecting to it...
INFO: Analyzed target //:gazelle (66 packages loaded, 7036 targets configured).
INFO: Found 1 target...
Target //:gazelle up-to-date:
bazel-bin/gazelle-runner.bash
bazel-bin/gazelle
INFO: Elapsed time: 23.191s, Critical Path: 5.81s
INFO: 49 processes: 13 internal, 36 linux-sandbox.
INFO: Build completed successfully, 49 total actions
INFO: Build completed successfully, 49 total actions
Let's see what it did:
$ cat one/BUILD.bazel
load("#io_bazel_rules_go//go:def.bzl", "go_binary", "go_library")
go_library(
name = "one_lib",
srcs = ["one.go"],
importpath = "github.com/symbiont-io/sandbox/ji/one",
visibility = ["//visibility:private"],
deps = [
"#org_golang_google_api//calendar/v3:go_default_library",
"#org_golang_google_api//people/v1:go_default_library",
"#org_golang_x_net//context:go_default_library",
"#org_golang_x_oauth2//:go_default_library",
"#org_golang_x_oauth2//google:go_default_library",
],
)
go_binary(
name = "one",
embed = [":one_lib"],
visibility = ["//visibility:public"],
)
The deps look good, that's what I'm importing in one.go.
Here is how it fails:
$ bazel build //one:one
ERROR: /home/ji/sandbox/ji/one/BUILD.bazel:3:11: no such package '#org_golang_google_api//people/v1': The repository '#org_golang_google_api' could not be resolved and referenced by '//oneonone:oneonone_lib'
ERROR: Analysis of target '//one:one' failed; build aborted: Analysis failed
INFO: Elapsed time: 0.087s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded, 0 targets configured)
I've found the documentation to actually be very thorough.
Anyway, you need to run update-repos. You can either run a one-off command to import the specific repo you need:
bazel run //:gazelle -- update-repos google.golang.org/api/people/v1
Or you can create a Go module and just import dependencies from go.mod:
bazel run //:gazelle -- update-repos -from_file=go.mod
I recommend doing the latter. Create a module with go mod init and then you can have a script you run periodically to update your dependencies:
# update go.mod
go get -d ./...
# update repos with Gazelle
bazel run //:gazelle -- update-repos -from_file=go.mod