Whats the correct way in go to distinguish between when a value in a struct was never set, or is just empty, for example, given the following:
type Organisation struct {
Category string
Code string
Name string
}
I need to know (for example) if the category was never set, or was saved as blank by the user, should I be doing this:
type Organisation struct {
Category *string
Code *string
Name *string
}
I also need to ensure I correctly persist either null or an empty string to the database
I'm still learning GO so it is entirely possible my question needs more info.
The zero value for a string is an empty string, and you can't distinguish between the two.
If you are using the database/sql package, and need to distinguish between NULL and empty strings, consider using the sql.NullString type. It is a simple struct that keeps track of the NULL state:
type NullString struct {
String string
Valid bool // Valid is true if String is not NULL
}
You can scan into this type and use it as a query parameter, and the package will handle the NULL state for you.
Google's protocol buffers (https://code.google.com/p/goprotobuf/) use pointers to describe optional fields.
The generated objects provide GetFoo methods which take the pain away from testing for nil (a.GetFoo() returns an empty string if a.Foo is nil, otherwise it returns *a.Foo).
It introduces a nuisance when you want to write literal structs (in tests, for example), because &"something" is not valid syntax to generate a pointer to a string, so you need a helper function (see, for example, the source code of the protocol buffer library for proto.String).
// String is a helper routine that allocates a new string value
// to store v and returns a pointer to it.
func String(v string) *string {
return &v
}
Overall, using pointers to represent optional fields is not without drawbacks, but it's certainly a viable design choice.
The standard database/sql package provides a NullString struct (members are just String string and Valid bool). To take care of some of the repetitive work of persistence, you could look at an object-relational manager like gorp.
I looked into whether there was some way to distinguish two kinds of empty string just out of curiosity, and couldn't find one. With []bytes, []byte{} == []byte(nil) currently returns false, but I'm not sure if the spec guarantees that to always remain true. In any case, it seems like the most practical thing to do is to go with the flow and use NullString.
Related
I need to print all form requests to log, for later debugging. I need to have something like this, where formInput are different kind of structs. (All defined in my code.)
type SomeFormInput {
Name string
Age int
}
func handleForm(formInput interface{}) {
...
logger.Logf("%+v", formInput);
}
However, some of the forms contain "secret" data that needs to be sanitized before logging - specifically, passwords.
What is the best and easiest way to do this, without losing the generality of handleForm?
I still need it to accept interface{}.
I can use tags if needed.
type SomeFormInput struct {
Name string
Password string `hide:"true"` // something like this
}
There is no completely general solution to this. There are only specific solutions with different trade-offs.
Perhaps the most general solution would be to write a function that inspects each form type with reflection, and omits any sensitive fields in the output.
logger.Logf("%s", stripPassword(formInput));
This has the side effect of no longer using the %+v verb, which often won't matter, but if your type implements the fmt.Formatter interface, it will change behavior. But then again, your goal is to change behavior, so...
Which leads to a second option:
You could implement a custom Format() method on your type, to cause %+v (or %v or %s) to output whatever you desire (omitting private fields).
Along the same lines, but easier, if you're logging (or willing to log) JSON output, just impliment your own json.Marshaler for each type. Example:
func (i *SomeFormInput) MarshalJSON() ([]byte, error) {
intermediate := struct{
*SomeFormInput
Password struct{} `json:"-"` // Occlude Password field in the embedded struct
MarshalJSON struct{} `json:"-"` // Occlude MarshalJSON method on embedded struct, to avoid infinite loop
}{
SomeFormInput: i,
}
return json.Marshal(intermediate)
}
A similar option is to use the json:"-" tag for fields that should not be output. Example:
type SomeFormInput struct {
Name string
Password string `json:"-"`
}
And as a final option, you could filter the log output, prior to sending it to the logger. If you're using JSON, you could parse the data using the tokenizing json.Decoder, to omit only the keys you want. For arbitrary text, perhaps a regular expression could be used.
For most applications I work on, where I practically always use JSON logging, I would probably prefer option #4, followed by #5.
I am new to golang and I have one issue which I think community can help me to solve it.
I have one data structure like below
type ParentIDInfo struct {
PCOrderID string `json:"PCorderId",omitempty"`
TableVarieties TableVarietyDC `json:"tableVariety",omitempty"`
ProduceID string `json:"PRID",omitempty"`
}
type PCDCOrderAsset struct {
PcID string `json:"PCID",omitempty"`
DcID string `json:"DCID",omitempty"`
RequiredDate string `json:"requiredDate",omitempty"`
Qty uint64 `json:"QTY",omitempty"`
OrderID string `json:"ORDERID",omitempty"`
Status string `json:"STATUS",omitempty"`
Produce string `json:"Produce",omitempty"`
Variety string `json:"VARIETY",omitempty"`
Transports []TransportaionPCDC `json:"Transportaion",omitempty"`
ParentInfo []ParentIDInfo `json:"ParentInfo",omitempty"`
So I have issue to access the PCOrderID which inside the []ParentIDInfo . I have tried below however I getting error as "pcdcorder.ParentInfo.PCOrderID undefined (type []ParentIDInfo has no field or method PCOrderID)"
keyfarmercas = append(keyfarmercas, pcdcorder.ParentInfo.PCOrderID)
Any help will be very good
Thanks in advance
PCDCOrderAsset.ParentInfo is not a struct, it does not have a PCOrderID field. It's a slice (of element type ParentIDInfo), so its elements do, e.g. pcdcorder.ParentInfo[0].PCOrderID.
Whether this is what you want we can't tell. pcdcorder.ParentInfo[0].PCOrderID gives you the PCOrderID field of the first element of the slice. Based on your question this may or may not be what you want. You may want to append all IDs (one from each element). Also note that if the slice is empty (its length is 0), then pcdcorder.ParentInfo[0] would result in a runtime panic. You could avoid that by first checking its length and only index it if its not empty.
In case you'd want to add ids of all elements, you could use a for loop to do that, e.g.:
for i := range pcdorder.ParentInfo {
keyfarmercas = append(keyfarmercas, pcdcorder.ParentInfo[i].PCOrderID)
}
I have two structs, whose types are as follows:
type UserStruct struct {
UserID string `bson:"user_id" json:"user_id"`
Address string `bson:"address" json:"address"`
Email string `bson:"email" json:"email"`
CreatedAt time.Time `bson:"created_at" json:"created_at"`
PhoneNumber string `bson:"phone_number" json:"phone_number"`
PanCard string `bson:"pancard" json:"pancard"`
Details map[string]string `json:"details"`
}
type SecretsStruct struct {
UserID string `r:"user_id" json:"user_id"`
Secrets []string `r:"secrets" json:secrets`
Address string `r:"address" json:"address"`
Email string `r:"email"json:"email"`
CreatedAt time.Time `r:"created_at"json:"created_at"`
PhoneNumber string `r:"phone_number" json:"phone_number"`
PanCard string `r:"pancard" json:"pancard"`
}
I already have an instance of UserStruct. I want to copy the fields common to both structs from UserStruct to a new instance of SecretStruct, without using reflection.
Go is a statically typed language (and is not Python). If you want to copy fields between the structs, you must either cause code to be supplied at compile time which knows how to do this, or use the reflect library to perform the operation at runtime.
Note that I said "cause code to be supplied at compile time" because you don't have to explicitly write that code. You could use code generation to produce the copy code from the struct definitions, or from a higher-level definition (e.g. XML) which generates both the struct definition and the copying code.
However, good Go programmers prefer clear code over clever solutions. If this is a single localized requirement, writing a code generator to avoid "boilerplate" code is almost certainly overkill; its implementation will take longer than the code to copy the structs, and the associated complexity will introduce a risk of more bugs. Similarly, reflect-based solutions are complicated, not clear, and only recommended in cases where you require a generic or extensible solution, and where this cannot be fulfilled at compile time.
I recommend simply write the copying code, and add appropriate comments to the struct definitions and copy methods to ensure future maintainers are aware of their obligation to maintain the copy methods.
Example
// Define your types - bodies elided for brevity
// NOTE TO MAINTAINERS: if editing the fields in these structs, ensure
// the methods defined in source file <filename>.go are updated to
// ensure common fields are copied between structs on instantiation.
type UserStruct struct { ... }
type SecretStruct struct { ... }
// NewSecretStructFromUserStruct populates and returns a SecretStruct
// from the elements common to the two types. This method must be
// updated if the set of fields common to both structs is changed in
// future.
func NewSecretStructFromUserStruct(us *UserStruct) *SecretStruct {
// You should take care to deep copy where necessary,
// e.g. for any maps shared between the structs (not
// currently the case).
ss := new(SecretStruct)
ss.UserID = us.UserID
ss.Address = us.Address
ss.Email = us.Email
ss.CreatedAt = us.CreatedAt
ss.PhoneNumber = us.PhoneNumber
ss.PanCard = us.PanCard
return ss
}
// You may also consider this function to be better suited as
// a receiver method on UserStruct.
Originally I figured I'd only use pointers for optional struct fields which could potentionally be nil in cases which it was initially built for.
As my code evolved I was writing different layers upon my models - for xml and json (un)marshalling. In these cases even the fields I thought would always be a requirement (Id, Name etc) actually turned out to be optional for some layers.
In the end I had put a * in front of all the fields including so int became *int, string became *string etc.
Now I'm wondering if I had been better of not generalising my code so much? I could have duplicated the code instead, which I find rather ugly - but perhaps more efficient than using pointers for all struct fields?
So my question is whether this is turning into an anti-pattern and just a bad habbit, or if this added flexibility does not come at a cost from a performance point of view?
Eg. can you come up with good arguments for sticking with option A:
type MyStruct struct {
Id int
Name string
ParentId *int
// etc.. only pointers where NULL columns in db might occur
}
over this option B:
type MyStruct struct {
Id *int
Name *string
ParentId *int
// etc... using *pointers for all fields
}
Would the best practice way of modelling your structs be from a purely database/column perspective, or eg if you had:
func (m *MyStruct) UnmarshalXML(d *xml.Decoder, start xml.StartElement) error {
var v struct {
XMLName xml.Name `xml:"myStruct"`
Name string `xml:"name"`
Parent string `xml:"parent"`
Children []*MyStruct `xml:"children,omitempty"`
}
err := d.DecodeElement(&v, &start)
if err != nil {
return err
}
m.Id = nil // adding to db from xml, there's initially no Id, until after the insert
m.Name = v.Name // a parent might be referenced by name or alias
m.ParentId = nil // not by parentId, since it's not created yet, but maybe by nesting elements like you see above in the V struct (Children []*ContentType)
// etc..
return nil
}
This example could be part of the scenario where you want to add elements from XML to the database. Here ids would generally not make sense, so instead we use nesting and references on name or other aliases. An Id for the structs would not be set until we got the id, after the INSERT query. Then using that ID we could traverse down the hierachy to the child elements etc.
This would allow us to have just 1 MyStruct, and use eg. different POST http request handler functions, depending if the call came from form input, or xml importing where a nested hierarchy and different relations might need come different handling.
In the end I guess what I'm asking is:
Would you be better off separating struct models for db, xml- and json operations (or whatever scenario that you can think of), than using struct field pointers all the way, so we can reuse the model for different, yet related stuff?
Apart from possible performance (more pointers = more things for the GC to scan), safety (nil pointer dereference), convenience (s.a = 2 vs s.a = new(int); *s.a = 42), and memory penalties (a bool is one byte, a *bool is four to eight), there is one thing that really bothers me in the all-pointer approach. It violates the Single responsibility principle.
Is the MyStruct you get from XML or DB same as MyStruct? What if the DB schema will change? What if the XML changes format? What if you'll also need to unmarshal it into JSON, but in a slightly different manner? And what if you need to support all that (and in multiple versions!) at the same time?
A lot of pain comes to you when you try to make one thing do many things. Is having one do-it-all type instead of N specialised types really worth it?
I have two structs that represent models that will be inserted into a mongodb database. One struct (Investment) has the other struct (Group) as one of its fields.
type Group struct {
Base
Name string `json:"name" bson"name"`
}
type Investment struct {
Base
Symbol string `json:"symbol" bson:"symbol" binding:"required"`
Group Group `json:"group" bson:"group"`
Fields bson.M `json:"fields" bson:"fields"`
}
The problem I'm having is that in the Investment model, Group is not required. If there is no group, I think its better for it to not be inserted in the db. Whats the best way to handle a db model such as this in Go?
tl;dr: Use ,omitempty, and if you need to worry about the difference between a zero value and null/not specified, do what the GitHub API does and use a pointer.
Both json and bson support the ,omitempty tag. For json, "empty values are false, 0, any nil pointer or interface value, and any array, slice, map, or string of length zero" (json docs). For bson, ,omitempty means "Only include the field if it's not set to the zero value for the type or to empty slices or maps", and zero values include empty strings and nil pointers (bson docs).
So if you really need a Group struct, you can put a *Group in instead, and it won't be stored when the pointer is nil. If Investment only needs to hold the group's name, it's even simpler: "" as group name keeps a group key from being stored.
bson defaults to using the lowercased field name already so you can omit that from the struct tag when they match. json will default to the Capitalized name, so specify the lowercase name in a tag if you need lowercase.
So, best case, maybe you can just use:
type Investment struct {
Base
Symbol string `json:"symbol" binding:"required"`
Group string `json:"group,omitempty" bson:",omitempty"`
Fields bson.M `json:"fields"`
}
If you ever run into fields where the zero value for the type ("", 0, false, etc.) is distinct from "not specified", you can do what the GitHub API does and put pointers in your structures--essentially an extension of the *Group trick.
Avoid strut fields to marshal if they are empty -
A struct field may be primitive type(string, int, bool etc) or even an another struct type.
So sometimes we don't want a struct's field to
go in json data(may to database insertion or in external api call) if they are empty
Example:
type Investment struct {
Base
Symbol string `json:"symbol" bson:"symbol" binding:"required"`
Group Group `json:"group" bson:"group"`
Fields bson.M `json:"fields" bson:"fields"`
}
If we want that Symbol and Group might contain empty values(0, false, nil pointer, zero size interface/struct) then we can avoid them in json marshaling like below.
type Investment struct {
Base
Symbol string `json:"symbol,omitempty" bson:"symbol,omitempty" binding:"required"`
Group *Group `json:"group,omitempty" bson:"group,omitempty"`
Fields bson.M `json:"fields" bson:"fields"`
}
Her "Group" field is pointer to Group struct and whenever it will point to nil pointer it will be omitted from json marshaling.
And obviously we would be filling values in Group field like below.
// declared investment variable of type Investment struct
investment.Group = &groupData