School me on Go packages, imports and protos - go

Question

I'm mostly familiar with Python and a bazel build environment. I'm trying to rewrite a portion of code in Go and I'm struggling to get the proto imports to align properly.
In github.com/djhedges/exit_speed/gps.proto I have set go_package like so
option go_package = "github.com/djhedges/exit_speed/gps_go_proto";
I've added a reflector.proto which imports gps.proto and reuses some of the messages defined in there.
import "gps.proto";
I compile gps.proto with
protoc -I ./ --go_out=./ --go_opt=paths=source_relative --python_out ./ gps.proto
And compile reflector.proto with
LD_PRELOAD=/usr/lib/arm-linux-gnueabihf/libatomic.so.1 python3 -m grpc_tools.protoc -I ./ --go_out=./ --go_opt=paths=source_relative --python_out ./ --grpc_python_out ./ reflector.proto
And finally in exit_speed/reflector.go I try to import gps_go_proto with
import gpspb "github.com/djhedges/exit_speed/gps_go_proto"
Which errors with
go: finding github.com/djhedges/exit_speed/gps_go_proto latest
reflector.go:21:2: unknown import path "github.com/djhedges/exit_speed/gps_go_proto": cannot find module providing package github.com/djhedges/exit_speed/gps_go_proto
I would prefer to keep gps.proto in the root directory if possible because I have protos logged into data files and I believe moving this breaks the proto parsing. I suppose I could write a migration script.
I'm also confused as to how setup multiple protos with different go packages and imports. Do go packages have to have a dedicated folder?

Answer 1

If I'm understanding go packages correctly the package name must map to a directory. By changing --go_out=./ to --go_out=./gps_go_proto seems to solve my issue without having to move the proto files. It does mean I have some additional directories but that seems okay.

Answer 2

As the Go Specification import section says:
Import declarations
An import declaration states that the source file containing the declaration depends on functionality of the imported package (§Program initialization and execution) and enables access to exported identifiers of that package. The import names an identifier (PackageName) to be used for access and an ImportPath that specifies the package to be imported.
ImportDecl = "import" ( ImportSpec | "(" { ImportSpec ";" } ")" ) .
ImportSpec = [ "." | PackageName ] ImportPath .
ImportPath = string_lit .
(snipped here, and I broke some links; use the first link to get the text with the links). The string literal is the familiar:
import "github.com/..."
part; the PackageName, or a dot, are optional. Returning to the text:
The interpretation of the ImportPath is implementation-dependent but it is typically a substring of the full file name of the compiled package and may be relative to a repository of installed packages.
In your own answer, you add:
If I'm understanding go packages correctly the package name must map to a directory.
That's not necessarily so. It's up to the implementation to decide what the string literal means. In general, though, the Go build system will find a directory full of files, using this path. It may also use a version control system, such as Git, to fill a directory full of files, so that the compiler can find them.
Something that I found a bit surprising, but in the end, makes perfect sense (and is described in the spec), is this: the package name found in the package clause:
package somename
statement at the top of some imported file, is sometimes irrelevant. A directory full of files that implement package x will have them all say package x, of course. But it's the y identifier in the line:
import y "path/to/x"
in your package that supplies the name you use to access exported identifiers. The name you use here, with the import having y as the package name, is y, even if the package itself says package x. The package clause in package x is, in effect, just recommending the name x, so that you can write:
import "path/to/x"
if you'd like to refer to these as x.Thing.
There's no need for the string literal to mention x either: perhaps all the files are in something found via path/to/z, and you write:
import "path/to/z"
and then (still) write x.Thing because files in path/to/z/*.go recommend x as the package name.
How protoc uses this
Regardless of the above, the protobuf Go generators will print, in the generated name*.pb.go* Go code, an import directive. That import directive will read:
import "some/path/here"
You can specify the string literal using option go_package. The protobuf compiler doesn't need to interpret this itself. But, as the documentation notes, these name.pb.go files themselves will be dropped into a directory somewhere. They will also contain a package pkg line, of course.
The location of each output file depends on whether you use --go_opt=paths=source_relative. If you do, the go_package option string becomes irrelevant here.
The .pb.go file generator takes the go_package option string, strips off everything up to and including the last slash, and uses the remaining string for the package line.
The second point means that regardless of what the Go system itself does, how it finds these files, and how you write your own imports, the recommended package name in those files will be the last component of whatever was in the go_package option. So in a sense, all you really need is the final component. But if your Go implementation needs more than that, you can supply more.
The first point means that if you don't use --go_opt=paths=source_relative, the rest of the text in the go_package option matters: the Go generator will try to use that string to produce output files. So if your Go implementation needs more, and you do supply more, and it doesn't match what you want for output file names, you'll definitely need the --go-opt.