Is it possible to use go doc to view all sub-packages defined under a specific package?
Say, I want to view all sub-packages under crypto.
go doc crypto only lists what crypto defines, but no information about its sub-packages, like crypto/aes and crypto/cipher:
go doc crypto
package crypto // import "crypto"
Package crypto collects common cryptographic constants.
func RegisterHash(h Hash, f func() hash.Hash)
type Decrypter interface{ ... }
type DecrypterOpts interface{}
type Hash uint
const MD4 Hash = 1 + iota ...
...
If you want to see all sub-packages under a specific package you can use go list command:
go list crypto/...
crypto
crypto/aes
crypto/cipher
crypto/des
crypto/dsa
crypto/ecdsa
crypto/ed25519
crypto/ed25519/internal/edwards25519
crypto/elliptic
crypto/hmac
crypto/internal/randutil
crypto/internal/subtle
crypto/md5
crypto/rand
crypto/rc4
crypto/rsa
crypto/sha1
crypto/sha256
crypto/sha512
crypto/subtle
crypto/tls
crypto/x509
crypto/x509/pkix
Finally, for each package you can get the doc with go doc command.
go doc crypto/x509
...
You can write a script if you need to iterate over the results returned by go list.
Honestly, I think that the best way to consume the doc of std library is the Go website: https://golang.org/pkg/.
You can also start a local godoc web server to read the doc of your Go code:
godoc -http=:6060
*open your browser and visit localhost:6060*
Related
I want to define an ASN.1 structure with a numeric field. I use a custom type for that number, to make it clear what kind of value one deals with and let the compiler catch some type-related errors (e.g., attempts to use other numerical values that don't semantically fit into the field).
However, the serializer doesn't know how to deal with this field, because it is not a Go primitive type.
Here is a minimal example that illustrates the problem:
package main
import (
"encoding/asn1"
"fmt"
)
type MessageType uint
const (
Signal MessageType = 1
Payload = 2
Control = 3
)
type Packet struct {
Type MessageType
// more stuff here ...
}
func main() {
fmt.Println(Signal)
var packet Packet
packet.Type = Payload
fmt.Println(asn1.Marshal(packet))
}
When I run it, the error is [] asn1: structure error: unknown Go type: main.MessageType. It works if I use a plain integer instead of the custom MessageType, but in this case I don't get the benefit of using specific types.
What is the proper way of handling this scenario in Go? The documentation of the ASN1 module doesn't provide any examples that address this question.
golang encoding/asn1 is a low level library ... it is not an ASN.1 compiler
So it's normal you can only encode basic types
You will need to write more code before it calls encoding/asn1 library
You can do it manually if the problem you want to solve is very simple.
However, the normal process when you use ASN.1 is:
Write an ASN.1 specification
Use a tool to compile it and produce stubs in your preferred language
Write some code that will use these stubs
I think you won't find free tools to do that in Go
Handling errors in a gRPC service commonly requires both a status message and error codes. Both have two definitions:
Google APIs definition (googleapis/go-genproto) - the generated Go packages for common protocol buffer types, and the generated gRPC code for Google's gRPC APIs
gRPC definition (grpc/grpc-go) - the Go implementation of gRPC
The Go packages for both definitions of Status and Codes are:
Google APIs
Status: google.golang.org/genproto/googleapis/rpc/status
Codes: google.golang.org/genproto/googleapis/rpc/code
gRPC
Status: google.golang.org/grpc/status
Codes: google.golang.org/grpc/codes
Since I'm a client of my own gRPC service and not a client of an existing Google gRPC API, I want to use the gRPC definitions of Status and Code.
However, the gRPC proto file for Status is actually copied from Google APIs definition. See https://github.com/grpc/grpc/tree/master/src/proto/grpc/status. The go_package of status.proto is also unchanged, so both the Google API and gRPC definitions use the following Go package
option go_package = "google.golang.org/genproto/googleapis/rpc/status;status";
The upshot is the only way to use Status when defining an API is by importing it with
import "google/rpc/status.proto";
...and importing the language bindings in Go with
import (
"google.golang.org/genproto/googleapis/rpc/status"
)
// Go server code...
But as stated earlier, this is wrong since I'm not a client of a Google API, but rather my own gRPC service. Therefore the language bindings should be imported with
import (
"google.golang.org/grpc/status"
)
// Go server code...
As expected if I switch to importing the gRPC language bindings and try and return a Status message to the API client, I get a compile error
cannot use &(status.Status literal)
(value of type *"google.golang.org/grpc/internal/status".Status) as
*"google.golang.org/genproto/googleapis/rpc/status".Status value
This is caused by my .proto file using the Google API definition of Status while the server implementation (in Go) uses the gRPC definition.
The problem impacts error codes since Google APIs uses signed 32 bit integers (int32) whereas gRPC uses unsigned 32 bit integers (uint32).
Questions
Is my assertion that I should be using the gRPC definition of Status and Codes correct?
If my assertion is correct, how can I use the gRPC definition of Status when it's packaged for Google APIs?
We need to distinguish a few cases. Some of them are obvious, some are not.
Just returning Status from a gRPC handler
If your proto schema (.proto files) doesn't define messages that use Status or Code directly, then the gRPC handlers can satisfy the return type error simply with "google.golang.org/grpc/status".Error(), or Newf().Err(). And that's about it.
Example:
// implements SomeServiceServer unary RPC GetFoo
func (s *SomeService) GetFoo(ctx context.Context, req *grpc.FooRequest) (*grpc.FooResponse, error) {
// status is "google.golang.org/grpc/status"
return nil, status.Error(codes.Unimplemented, "coming soon")
Using Status in your .proto files
In this case, you are forced to use the googleapis implementation. As you already have seen, the status.proto Go package is defined as:
option go_package = "google.golang.org/genproto/googleapis/rpc/status;status";
So let's say you have the following .proto file, where the imported status.proto is just a copy-paste of the gRPC status.proto as per this question:
syntax = "proto3";
package test;
import "status.proto";
option go_package = ".;main";
message Foo {
string a = 1;
google.rpc.Status status = 2;
}
with directory structure as:
/protos
|_ status.proto
|_ test.proto
and you compile the above with:
cd protos && protoc -I=. --go_out=. test.proto
breathe ...then the generated Go code will have the following import
import (
status "google.golang.org/genproto/googleapis/rpc/status"
)
and you must satisfy that by go get google.golang.org/genproto.
So about your first question, you can only use Status from googleapis in proto files, because that's how status.proto declares its Go package.
Using generated googleapis Status in Go code
Since the imported Go package is from googleapis that is what you must use in your Go code in order to initialize such messages:
package main
import (
"fmt"
googleapis "google.golang.org/genproto/googleapis/rpc/status"
)
func main() {
foo := &Foo{
A: "foo",
Status: &googleapis.Status{
Code: int32(code.Code_OK),
Message: "all good",
},
}
// fmt.Println(foo)
}
Yes but I must use grpc-go Status in my Go code
You can't. protoc generates code with the packages described above. If you absolutely NEED to construct these Status fields using grpc-go, you can use Status.Proto:
package main
import (
"fmt"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
)
func main() {
foo := &Foo{
A: "foo",
Status: status.New(codes.OK, "all good").Proto(),
}
fmt.Println(foo)
}
Just for the record, the opposite is also possible with status.FromProto:
package main
import (
"fmt"
googleapis_codes "google.golang.org/genproto/googleapis/rpc/code"
googleapis "google.golang.org/genproto/googleapis/rpc/status"
"google.golang.org/grpc/status"
)
func main() {
gapisStatus := &googleapis.Status{
Code: int32(googleapis_codes.Code_OK),
Message: "all good",
}
grpcStatus := status.FromProto(gapisStatus)
fmt.Println(grpcStatus)
}
As a less well-behaved alternative, you can simply copy-paste the status.proto sources into your project and manually change the go_package:
option go_package = "google.golang.org/grpc/status;status";
This way protoc will generate the Go code with this import, and your own sources will be able to follow suit. Of course this means you now have your own fork of status.proto.
Hi I am trying to generate the simple protobuf file in Go language
syntax = "proto3";
package gen;
message EvtKeepAlive
{
string SvcName = 2;
}
In the header I see that the package uses two different proto go implementations, one from github.com and one from google.golang.org. As far as I understand the latter supersedes the former, so is this file generation valid?
// versions:
// protoc-gen-go v1.25.0-devel
// protoc v3.13.0
// source: common.proto
package gen
import (
proto "github.com/golang/protobuf/proto"
protoreflect "google.golang.org/protobuf/reflect/protoreflect"
protoimpl "google.golang.org/protobuf/runtime/protoimpl"
reflect "reflect"
sync "sync"
)
The file is valid; see the comments from dsnet in response to this issue:
The only reason the newly generated .pb.go files depend on the
deprecated proto package is to enforce a weak dependency on a
sufficiently new version of the legacy package. This is necessary
because not everyone is using Go modules such that the Go toolchain
would enforce this dependency constraint. I wasn't fond of adding it,
but I think it's probably necessary to keep at least for a few months.
I am using wgo for dependency management in Golang (although I think wgo has little to do with this), wgo has a folder structure like this
project/
.gocfg/
gopaths
vendor.json
vendor/
src/
github.com_or_whatever/
I have a library I coded myself which uses an nsq-go type in one of the exported methods:
func AddNsqSubscription(
topic, channel string,
handler nsq.Handler,
config *nsq.Config) error { }
The library is called messi and I import the nsq-go like so "messi/vendor/src/github.com/bitly/go-nsq"
The problem comes when I try to use this library in another project. For instance, in a project called scribe I have the following code (notice the imports):
import (
"scribe/vendor/src/github.com/bitly/go-nsq"
"scribe/vendor/src/messi"
)
//...
nsqHandler := nsq.HandlerFunc(func(message *nsq.Message) error {
msgHandler(MessiMessage{message})
return nil
})
return messi.AddNsqSubscription(destination, subdestination, nsqHandler, nsq.NewConfig())
When I go build the following error is returned:
cannot use nsqHandler (type "scribe/vendor/src/github.com/bitly/go-nsq".HandlerFunc) as type "messi/vendor/src/github.com/bitly/go-nsq".Handler in argument to messi.AddNsqSubscription:
"scribe/vendor/src/github.com/bitly/go-nsq".HandlerFunc does not implement "messi/vendor/src/github.com/bitly/go-nsq".Handler (wrong type for HandleMessage method)
have HandleMessage("scribe/vendor/src/github.com/bitly/go-nsq".Message) error
want HandleMessage("messi/vendor/src/github.com/bitly/go-nsq".Message) error
Why? I do not really know what is going on. The code go-nsq imported is exactly the same, yet golang wants that this code comes from the same folder?
What am I doing wrong?
Packages in Go are identified by full import path, not by name.
For example in the standard library there are two different packages with the same name template but different import paths: text/template and html/template.
You should make sure that go-nsq package is imported using the same path.
For a contrived example:
I have 2 packages, repo.com/alpha/A & repo.net/beta/B. package A uses package B, both structured as example.
A:
main.go
B:
b.go
templates \
1.tmpl
2.tmpl
In main.go of package A, I'd need to access the templates directory of package B.
b.go
var templates string
templates = templatepath
func init(){
templatepath, _ = filepath.Abs("./templates")
}
main.go
import(
repo.net/beta/B
)
func main(){
fmt.Printf("%s", B.templates)
}
So the problem being in my more complex use case & the contrived example here is that B.templates will be in the directory for package A, where I need to establish and reference the directory of the imported path. This is part of learning and navigating the Go way of doing things, and my understanding is probably basic, so I need to understand how to do this in a Go context.
My use case is a package that uses other packages that do things for the base package, and these external packages may use standard web resources files(css, html, js) the problem being I'm having immediate trouble packaging and referencing them abstractly enough for what I want to do.
You can't, you have to either use something like go-bindata or so, or simply embed the templates in your B package as consts.
tmpl1.go:
const tmpl1 = `........`