I have a user message as such :
message User {
string uid = 1;
}
Protobuff generates a struct as such
type User struct {
Uid string
}
Is there a way to enforce destination case so that my user struct in Go would be like :
type User struct {
UID string
}
So far, I can use string uID in my proto definition but it feels a bit hacky.
This does not seem to be supported.
As listed in golang/protobuf issue 555
By convention, naming styles are different:
Protos uses CamelCase for messages, groups, enums, services, and methods
Protos uses snake_case for fields and oneof fields
Protos uses SCREAMING_SNAKE_CASE for enum values
Go uses CamelCase for everything
So for now, uID is the best workaround.
Related
I have database and JSON models which use struct field annotation for various purposes, namely specifying enumerations, which values are acceptable for validations, etc.
A database model example:
type QRCode struct {
Algorithm string `json:"algorithm" gorm:"type:enum('hmac-sha3-256-v1')" validate:"oneof=hmac-sha3-256-v1"`
PublicCode []byte `json:"token" gorm:"size:32" validate:"len=32"`
UserType string `json:"user_type" gorm:"type:enum('admin','member')" validate:"one_of=admin member"`
gorm.Model
}
So in this case, there are a few different constants:
public key size, which I have in constants.QRCodePublicCodeLength
algorithm, which I have in constants.QRCodeAlgorithmV1
user type, which I have in constants.UserTypeAdmin and constants.UserTypeMember
It would be very nice to be able to embed these constants in the field tags so that there is truly one source of truth for everything, but I don't know if this is possible in Go.
Can I use constants in struct field tag definitions?
Can I use constants in struct field tag definitions?
No, this is not possible.
I would like to define an object in a shared libary,
type Common struct {
field_a string
custom interface{}
}
where custom is expected to hold additional fields that the user of this common object might define in their file, e.g.,
// module-1
type Mod1Customs struct {
abc string
}
From here, I would like to be able to set Common.custom to Mod1Customs so that I can access both field_a and abc in the same way from within module-1. I would like to do the same thing for other modules that may define an entirely different struct to assign to custom though.
The general idea is to create an object of pre-defined defaults and allowing a bucket of sorts for the users of the object to add their own custom fields.
Is this possible?
What you seem to be looking for is embedding:
type Common struct {
FieldA string
}
type Mod1Customs struct {
Common
Abc string
}
Then Mod1Customs gets access to all of Common fields directly:
c := Mod1Customs{}
fmt.Println(c.FieldA)
fmt.Println(c.Abc)
See Embedding in Effective Go for more details.
If you have some functions that deal with Common fields (no matter what the struct), then you would implement that with an interface that declares the fields you need. Mod1Customs would automatically conform to that interface.
I am running into a bit of a problem with Protoc and my existing struct that contains nullable string fields.
The struct I am trying to serialize for transmission contains a bunch of fields that are nullable in json (so we can distinguish between null, "" and a set value).
type Message struct {
Path *string `json:"path"`
}
So if a user sends a empty json string {} the Path will be nil and not "", whereas {"path":""} is also valid and a different case from {"path": null}.
The proto3 declaration I came up with obviously looks like this (and is optional as required and optional got dropped from proto3:
syntax = "proto3";
message Message {
string Path = 1;
}
After running Protoc I end up with a struct that looks like this and all values are string and no way to declare them as *string:
type Message struct {
Path string `protobuf:"bytes,1,opt,name=Path,proto3" json:"Path,omitempty"`
}
Obviously I can't assign to this array from my existing struct. But even if I were to write the tedious mapping code with target.Path = *source.Path with the appropriate null pointer checks etc I'd loose the triple meaning of my source struct (nil, "", "value").
Any suggestions on how to proceed here or if there is an extension to the Go Protobuf to do this? Or how would one go about describing this proto declaration?
Proto3 returns the Zero Value even if a field isn't set. Currently there is no way to distinguish if a field has been set or not.
See Github issue #15.
Possible solutions:
Use proto2 instead of proto3.
Use gogoproto's nullable extension.
Use the google.protobuf.FieldMask extension, see Common Design Patterns
from the Google API Design Guide: Partial Responses and Output Fields.
https://stackoverflow.com/a/62566052
proto3 support optional with flag --experimental_allow_proto3_optional
In my case, I solved this problem with several packages:
https://github.com/gogo/protobuf
https://github.com/golang/protobuf
My proto file looks like this:
syntax = "proto3";
import "google/protobuf/wrappers.proto";
import "github.com/gogo/protobuf/gogoproto/gogo.proto";
message Message {
google.protobuf.StringValue path = 1 [(gogoproto.wktpointer) = true];
}
The command for generating the go code, which I used look like this:
protoc -I. -I%GOPATH%/src --gogofaster_out=plugins=grpc:. proto/*.proto
The generated go file looks like this:
type Message struct {
Path *string `protobuf:"bytes,1,opt,name=path,json=path,proto3,wktptr" json:"path,omitempty"`
}
I have two structs, whose types are as follows:
type UserStruct struct {
UserID string `bson:"user_id" json:"user_id"`
Address string `bson:"address" json:"address"`
Email string `bson:"email" json:"email"`
CreatedAt time.Time `bson:"created_at" json:"created_at"`
PhoneNumber string `bson:"phone_number" json:"phone_number"`
PanCard string `bson:"pancard" json:"pancard"`
Details map[string]string `json:"details"`
}
type SecretsStruct struct {
UserID string `r:"user_id" json:"user_id"`
Secrets []string `r:"secrets" json:secrets`
Address string `r:"address" json:"address"`
Email string `r:"email"json:"email"`
CreatedAt time.Time `r:"created_at"json:"created_at"`
PhoneNumber string `r:"phone_number" json:"phone_number"`
PanCard string `r:"pancard" json:"pancard"`
}
I already have an instance of UserStruct. I want to copy the fields common to both structs from UserStruct to a new instance of SecretStruct, without using reflection.
Go is a statically typed language (and is not Python). If you want to copy fields between the structs, you must either cause code to be supplied at compile time which knows how to do this, or use the reflect library to perform the operation at runtime.
Note that I said "cause code to be supplied at compile time" because you don't have to explicitly write that code. You could use code generation to produce the copy code from the struct definitions, or from a higher-level definition (e.g. XML) which generates both the struct definition and the copying code.
However, good Go programmers prefer clear code over clever solutions. If this is a single localized requirement, writing a code generator to avoid "boilerplate" code is almost certainly overkill; its implementation will take longer than the code to copy the structs, and the associated complexity will introduce a risk of more bugs. Similarly, reflect-based solutions are complicated, not clear, and only recommended in cases where you require a generic or extensible solution, and where this cannot be fulfilled at compile time.
I recommend simply write the copying code, and add appropriate comments to the struct definitions and copy methods to ensure future maintainers are aware of their obligation to maintain the copy methods.
Example
// Define your types - bodies elided for brevity
// NOTE TO MAINTAINERS: if editing the fields in these structs, ensure
// the methods defined in source file <filename>.go are updated to
// ensure common fields are copied between structs on instantiation.
type UserStruct struct { ... }
type SecretStruct struct { ... }
// NewSecretStructFromUserStruct populates and returns a SecretStruct
// from the elements common to the two types. This method must be
// updated if the set of fields common to both structs is changed in
// future.
func NewSecretStructFromUserStruct(us *UserStruct) *SecretStruct {
// You should take care to deep copy where necessary,
// e.g. for any maps shared between the structs (not
// currently the case).
ss := new(SecretStruct)
ss.UserID = us.UserID
ss.Address = us.Address
ss.Email = us.Email
ss.CreatedAt = us.CreatedAt
ss.PhoneNumber = us.PhoneNumber
ss.PanCard = us.PanCard
return ss
}
// You may also consider this function to be better suited as
// a receiver method on UserStruct.
I just dived in Go programming using protobuf and I'm at the point where I need to validate data in a struct. I found govalidator, which seems to do the perfect job for what I need. It does validate structs based on the field tags, something like
type Contact struct {
firstName string `valid:"alpha,required"`
lastName string `valid:"alpha,required"`
email string `valid:"email,required"`
}
jdoe := &Contact{
firstName: "John",
lastName: "Doe",
email: "jdoe#mail.com"
}
ok, err = govalidator.ValidateStruct(jdoe)
And my protobuf definition would look like
message Contact {
string firstName = 1;
string lastName = 2;
string email = 3;
}
Now my question would be, is there a way to define the field tags in the proto message. From what I've seen in the generated go code, the compiler adds tags to the fields anyway, but could I "sneak" the ones that I need too? Also, I would imagine that unmarshalling could be one possible solution, but it somehow seems inefficient to me to unmarshal just to copy the field values to an equivalent struct which would have the necessary field tags.
Having the same structure for the data encapsulation and the input coming from the client was just a pure coincidence. As it has been suggested not only in the comments, but also by co-workers more experienced (than me) with protobuf I've just mapped (1:1 in this particular case) the fields from the structure generated by proto to the data encapsulation structure I have defined.