Gazelle not creating correct dependencies? - go

Consider a repo, github.com/myacct/sandbox, with a ji directory at the top:
$ ls -CFR
.:
BUILD.bazel WORKSPACE one/
./one:
oneonone.go
$ cat BUILD.bazel
load("#bazel_gazelle//:def.bzl", "gazelle")
# gazelle:prefix github.com/myacct/sandbox/ji
gazelle(name = "gazelle")
Pretty standard stuff, straight out of the docs. I have already run bazel clean --expunge. Now:
$ bazel run //:gazelle
Starting local Bazel server and connecting to it...
INFO: Analyzed target //:gazelle (66 packages loaded, 7036 targets configured).
INFO: Found 1 target...
Target //:gazelle up-to-date:
bazel-bin/gazelle-runner.bash
bazel-bin/gazelle
INFO: Elapsed time: 23.191s, Critical Path: 5.81s
INFO: 49 processes: 13 internal, 36 linux-sandbox.
INFO: Build completed successfully, 49 total actions
INFO: Build completed successfully, 49 total actions
Let's see what it did:
$ cat one/BUILD.bazel
load("#io_bazel_rules_go//go:def.bzl", "go_binary", "go_library")
go_library(
name = "one_lib",
srcs = ["one.go"],
importpath = "github.com/symbiont-io/sandbox/ji/one",
visibility = ["//visibility:private"],
deps = [
"#org_golang_google_api//calendar/v3:go_default_library",
"#org_golang_google_api//people/v1:go_default_library",
"#org_golang_x_net//context:go_default_library",
"#org_golang_x_oauth2//:go_default_library",
"#org_golang_x_oauth2//google:go_default_library",
],
)
go_binary(
name = "one",
embed = [":one_lib"],
visibility = ["//visibility:public"],
)
The deps look good, that's what I'm importing in one.go.
Here is how it fails:
$ bazel build //one:one
ERROR: /home/ji/sandbox/ji/one/BUILD.bazel:3:11: no such package '#org_golang_google_api//people/v1': The repository '#org_golang_google_api' could not be resolved and referenced by '//oneonone:oneonone_lib'
ERROR: Analysis of target '//one:one' failed; build aborted: Analysis failed
INFO: Elapsed time: 0.087s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded, 0 targets configured)

I've found the documentation to actually be very thorough.
Anyway, you need to run update-repos. You can either run a one-off command to import the specific repo you need:
bazel run //:gazelle -- update-repos google.golang.org/api/people/v1
Or you can create a Go module and just import dependencies from go.mod:
bazel run //:gazelle -- update-repos -from_file=go.mod
I recommend doing the latter. Create a module with go mod init and then you can have a script you run periodically to update your dependencies:
# update go.mod
go get -d ./...
# update repos with Gazelle
bazel run //:gazelle -- update-repos -from_file=go.mod

Related

NIX - go: modules disabled by GO111MODULE=off;

I'm trying to download and build a project written in golang using nix
My nix file:
buildGoPackage {
name = "site";
goPackagePath = "git#github.com:username/rep.git";
src = fetchGit{
url = "git#github.com:username/rep.git";
};
}
run:
nix build -f test3.nix
error log:
last 8 log lines:
> unpacking sources
> unpacking source archive /nix/store/l1gsllgg693s46sk7b7qiwbnjysnbbz6-source
> source root is source
> patching sources
> configuring
> building
> Building subPackage git#github.com:user/rep.git
> go: modules disabled by GO111MODULE=off; see 'go help modules'
I think you should be using buildGoModule instead of buildGoPackage, as buildGoPackage is deprecated, according to the nixpkgs documentation for go.
Additionally, buildGoModule doesn't seem to set GO111MODULE=off.

Bazel - BUILD not referencing external dependency

I'm trying to run some grpc tests with bazel.
I'm using "google.golang.org/grpc/credentials/insecure" to dial insecurely.
When running bazel test ..., I get the following error:
no such package '#org_golang_google_grpc//credentials/insecure': BUILD file not found in directory 'credentials/insecure' of external repository #org_golang_google_grpc. Add a BUILD file to a directory to mark it as a package. and referenced by '//go/internal/handlers/helloworld:helloworld_test'
I am generating my BUILD files with gazelle which outputs this for the go_test
go_test(
name = "helloworld_test",
srcs = ["helloworld_test.go"],
deps = [
":helloworld",
"//protos/helloworld",
"#com_github_stretchr_testify//assert",
"#org_golang_google_grpc//:go_default_library",
"#org_golang_google_grpc//credentials/insecure",
"#org_golang_google_grpc//test/bufconn",
"#org_uber_go_zap//:zap",
"#org_uber_go_zap//zaptest",
],
)
My go.mod file contains the dep:
google.golang.org/grpc v1.47.0
My deps.bzl is auto generated by gazelle:
go_repository(
name = "org_golang_google_grpc",
importpath = "google.golang.org/grpc",
sum = "h1:9n77onPX5F3qfFCqjy9dhn8PbNQsIKeVU04J9G7umt8=",
version = "v1.47.0",
)
What am I missing?
In the WORKSPACE, the local go_repositories() generated by gazelle must be called before gazelle_dependencies(), which will
define an ancient version of org_golang_google_grpc if it doesn't
exist yet and the local go_repository for the newer version will be
silently ignored. Reference

Bazel: how to import/build a go_binary for a go_test

I'm using bazel to run a go test on my CI system. The go test will need to use a helper binary built from another go file.
The dir structure looks like the following:
some/of/my/path
├── BUILD.bazel
├── my_utils.go
├── my_test.go
└── my_tool
├── BUILD.bazel
└── my_tool.go
I.e. the code in my_test.go is going to use the executable binary built from dir my_tool.
In a non-bazel case, I can just go build ./mytool to get the executable binary that my_test.go needs. To accommodate this the bazel world, I write the following in the bazel files:
In some/of/my/path/BUILD.bazel:
load("#io_bazel_rules_go//go:def.bzl", "go_library", "go_test")
go_library(
name = "testutils",
srcs = ["my_utils.go"],
importpath = "some/of/my/path",
visibility = ["//visibility:public"],
deps = [
"#org_golang_x_net//context",
... (some other dependencies, not related)
],
)
go_test(
name = "my_test",
size = "enormous",
srcs = ["mytest.go"],
embed = [":testutils"],
deps = [
"//some/of/my/path/my_tool:my_tool",
],
)
In some/of/my/path/my_tool/BUILD.bazel
load("#io_bazel_rules_go//go:def.bzl", "go_binary", "go_library")
go_library(
name = "my_tool_lib",
srcs = ["my_tool.go"],
importpath = "some/of/my/path/my_tool",
visibility = ["//visibility:private"],
deps = ["some dependency"],
)
go_binary(
name = "my_tool",
embed = [":my_tool_lib"],
visibility = ["//visibility:public"],
)
So the idea is when I hit bazel run, it will build the my_tool go binary, which the go test my_test can use (see the deps field of the go test).
While in the CI I hit this error:
ERROR: /home/agent/work/.go/src/some/of/my/path/BUILD.bazel:20:8: in go_test rule //some/of/my/path:my_test:
Traceback (most recent call last):
File "/home/agent/.cache/bazel/_bazel_agent/40f5e2a2e18a7cdb4cd075f919d8072b/external/io_bazel_rules_go/go/private/rules/test.bzl", line 66, column 43, in _go_test_impl
internal_source = go.library_to_source(go, ctx.attr, internal_library, ctx.coverage_instrumented())
File "/home/agent/.cache/bazel/_bazel_agent/40f5e2a2e18a7cdb4cd075f919d8072b/external/io_bazel_rules_go/go/private/context.bzl", line 247, column 26, in _library_to_source
_check_binary_dep(go, dep, "deps")
File "/home/agent/.cache/bazel/_bazel_agent/40f5e2a2e18a7cdb4cd075f919d8072b/external/io_bazel_rules_go/go/private/context.bzl", line 295, column 13, in _check_binary_dep
fail("rule {rule} depends on executable {dep} via {edge}. This is not safe for cross-compilation. Depend on go_library instead.".format(
Error in fail: rule //some/of/my/path:my_test depends on executable //some/of/my/path/my_tool:my_tool via deps. This is not safe for cross-compilation. Depend on go_library instead.
ERROR: Analysis of target '//some/of/my/path:my_test' failed; build aborted: Analysis of target '//some/of/my/path:my_test' failed
INFO: Elapsed time: 79.552s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (595 packages loaded, 12182 targets configured)
ERROR: Build failed. Not running target
FAILED: Build did NOT complete successfully (595 packages loaded, 12182 targets configured)
Process exited with code 1
But my goal is to use the binary. How should I change the configs to achieve that?
You want data, not deps. deps is for things linked into a binary, data is for things the binary uses at runtime.
Once your binary is in data, use runfiles.go to run it. Add #io_bazel_rules_go//go/tools/bazel:go_default_library to deps (it's a library you want linked in), and then use FindBinary to find the path.

Bazel not adding BUILD file to external dependency

I have a workspace where I use gazelle to generate my BUILD files and for some reason the com_github_ipfs_go_merkledag dependency is breaking the build when trying to resolve it's github.com/gogo/protobuf/gogoproto dependency. The setup and run is below.
Workspace:
# Declare indirect dependencies and init toolchains.
go_rules_dependencies()
go_register_toolchains(version = "1.16")
go_embed_data_dependencies()
load("#rules_proto//proto:repositories.bzl", "rules_proto_dependencies", "rules_proto_toolchains")
rules_proto_dependencies()
rules_proto_toolchains()
load("#bazel_gazelle//:deps.bzl", "gazelle_dependencies", "go_repository")
gazelle_dependencies()
Commands:
bazel run //:gazelle
bazel run //:gazelle -- update-repos -from_file=go.mod
bazel build //...
Output:
...
no such package '#com_github_ipfs_go_merkledag//github.com/gogo/protobuf/gogoproto':
BUILD file not found in directory 'github.com/gogo/protobuf/gogoproto' of external repository #com_github_ipfs_go_merkledag.
Add a BUILD file to a directory to mark it as a package. and referenced by '#com_github_ipfs_go_merkledag//pb:merkledag_pb_go_proto'
...
Not sure what the solution for this is?
Add build directives to the go_repository to ignore
go_repository(
name = "com_github_ipfs_go_merkledag",
importpath = "github.com/ipfs/go-merkledag",
sum = "h1:ixNu/5MJSaT/Qs073T0/HsWKwnOoBgqSq1g+GaJIen0=",
version = "v0.4.0",
build_directives = [
"gazelle:proto disable",
],
)

bazel golang build fails when project has internal dependency

I'm on MacOS 10.13.2.
Go 1.10.
bazel 0.11.1
I need to compile a repo that has 2 projects (project1 and project2).
project1 has 2 subpacakges. p1lib and dep1
p1lib uses dep1.
I generate BUILD files with gazelle, files look ok.
gazelle -go_prefix=github.com/BazelBuildForGo
But when I run build I get an error that says that I'm missing direct dependency.
bazel build //project1
INFO: Analysed target //project1:project1 (3 packages loaded).
INFO: Found 1 target...
ERROR: /private/var/tmp/_bazel_user1/df78026a5ee0c7ed3d23dd05c3a3b1f7/external/com_github_wix_private_bazelbuildforgo/project1/p1lib/BUILD.bazel:3:1: GoCompile external/com_github_wix_private_bazelbuildforgo/project1/p1lib/darwin_amd64_stripped/go_default_library~/github.com/BazelBuildForGo/project1/p1lib.a failed (Exit 1)
2018/03/25 18:02:55 missing strict dependencies:
external/com_github_wix_private_bazelbuildforgo/project1/p1lib/p1lib.go: import of github.com/wix-private/BazelBuildForGo/project1/dep1, which is not a direct dependency
Target //project1:project1 failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 0.358s, Critical Path: 0.12s
FAILED: Build did NOT complete successfully
My project can be found here
https://github.com/wix-playground/BazelBuildForGo
I think the problem is that the import prefix you passed to Gazelle on the command line (github.com/BazelBuildForGo) is different from the imports in the .go files (github.com/wix-private/BazelBuildForGo). When Gazelle sees imports that are outside the current prefix, it will generate external dependencies for those imports, and those dependencies will be missing:
go_library(
name = "go_default_library",
srcs = ["p1lib.go"],
importpath = "github.com/BazelBuildForGo/project1/p1lib",
visibility = ["//visibility:public"],
deps = ["#com_github_wix_private_bazelbuildforgo//project1/dep1:go_default_library"],
)
The fix for this is pretty easy though. Just run Gazelle with the prefix github.com/wix-private/BazelBuildForGo. You actually already have this in //:gazelle, so just run that, then rebuild.
$ bazel run //:gazelle
$ bazel build //...
That will change the go_library rule above to this:
go_library(
name = "go_default_library",
srcs = ["p1lib.go"],
importpath = "github.com/wix-private/BazelBuildForGo/project1/p1lib",
visibility = ["//visibility:public"],
deps = ["//project1/dep1:go_default_library"],
)

Resources