I have model like:
type Service struct {
Id uint64
Name string
Secret string
Disabled bool
}
And want to use annotations like form, valid and orm. And I can't find how I should declare these annotations. Should it be one or many? If many, what separator should I use?
Quoting from reflect.StructTag:
By convention, tag strings are a concatenation of optionally space-separated key:"value" pairs.
So you may specify multiple key-value pairs separated by space, like:
type Service struct {
Id uint64 `form:"id" valid:"Range(1, 999)" orm:"auto"`
}
See more about tags in this answer: What are the use(s) for tags in Go?
Related
I have database and JSON models which use struct field annotation for various purposes, namely specifying enumerations, which values are acceptable for validations, etc.
A database model example:
type QRCode struct {
Algorithm string `json:"algorithm" gorm:"type:enum('hmac-sha3-256-v1')" validate:"oneof=hmac-sha3-256-v1"`
PublicCode []byte `json:"token" gorm:"size:32" validate:"len=32"`
UserType string `json:"user_type" gorm:"type:enum('admin','member')" validate:"one_of=admin member"`
gorm.Model
}
So in this case, there are a few different constants:
public key size, which I have in constants.QRCodePublicCodeLength
algorithm, which I have in constants.QRCodeAlgorithmV1
user type, which I have in constants.UserTypeAdmin and constants.UserTypeMember
It would be very nice to be able to embed these constants in the field tags so that there is truly one source of truth for everything, but I don't know if this is possible in Go.
Can I use constants in struct field tag definitions?
Can I use constants in struct field tag definitions?
No, this is not possible.
I'm introducing a package from a third party that has this struct with mapstructure tag.
I want the instance of this struct to be json with mapstructure specified value.What should I do?
I can add json tag, but in doing so,I modify package files,I think this is a bad way.
type ServiceConfig struct {
// name of the service
Name string `mapstructure:"name"`
// set of endpoint definitions
Endpoints string `mapstructure:"end_points"`
// defafult timeout
Timeout time.Duration `mapstructure:"timeout"`
}
I want to get:
{"name":"sss", "end_points" :"xxx", "timeout" : "120"}
If you do not want to modify the package files, you can create another struct with the same field names, but with JSON tags, and copy:
type JSONServiceConfig struct {
Name string `json:"name"`
Endpoints string `json:"end_points"`
Timeout time.Duration `json:"timeout"`
}
Then:
x := JSONServiceConfig(serviceConfig)
You cannot do what you want without modifying the mapstructure source, and it would probably get a little bit hairy if you want to specify options, such as json's omitempty. However, you can simply add a second struct tag for this
type ServiceConfig struct {
// name of the service
Name string `mapstructure:"name" json:"name"`
// set of endpoint definitions
Endpoints string `mapstructure:"end_points" json:"end_points"`
// defafult timeout
Timeout time.Duration `mapstructure:"timeout" json:"timeout"`
}
From the documentation of reflect
By convention, tag strings are a concatenation of optionally
space-separated key:"value" pairs. Each key is a non-empty string
consisting of non-control characters other than space (U+0020 ' '),
quote (U+0022 '"'), and colon (U+003A ':'). Each value is quoted using
U+0022 '"' characters and Go string literal syntax.
Here's a simple example on the playground
type Config struct {
CommitIndex uint64 `json:"commitIndex"`
// TODO decide what we need to store in peer struct
Peers []*Peer `json:"peers"`
}
I understand what the first two columns are,but what is json:"commitIndex"?
It's called a struct tag, they can be parsed using the reflect package at runtime.
From https://golang.org/ref/spec#Struct_types:
A field declaration may be followed by an optional string literal tag, which becomes an attribute for all the fields in the corresponding field declaration.
The tags are made visible through a reflection interface and take part in type identity for structs but are otherwise ignored.
Some packages that use reflection like json and xml use tags to handle special cases better.
What you are referring to is called a tag, and the Go specification states:
A field declaration may be followed by an optional string literal tag,
which becomes an attribute for all the fields in the corresponding
field declaration. The tags are made visible through a reflection
interface and take part in type identity for structs but are otherwise
ignored.
// A struct corresponding to the TimeStamp protocol buffer.
// The tag strings define the protocol buffer field numbers.
struct {
microsec uint64 "field 1"
serverIP6 uint64 "field 2"
process string "field 3"
}
This does nothing at compile time, but is used by different packages when doing runtime reflection on the struct. As Amit already pointed out, the encoding/json package is using it to specify marshalling/unmarshalling behaviour. The same goes with encoding/xml, gopkg.in/mgo.v2/bson, etc.
The tag string is by convention a space separated string. As stated in the reflect package:
By convention, tag strings are a concatenation of optionally
space-separated key:"value" pairs. Each key is a non-empty string
consisting of non-control characters other than space (U+0020 ' '),
quote (U+0022 '"'), and colon (U+003A ':'). Each value is quoted using
U+0022 '"' characters and Go string literal syntax.
I have two structs that represent models that will be inserted into a mongodb database. One struct (Investment) has the other struct (Group) as one of its fields.
type Group struct {
Base
Name string `json:"name" bson"name"`
}
type Investment struct {
Base
Symbol string `json:"symbol" bson:"symbol" binding:"required"`
Group Group `json:"group" bson:"group"`
Fields bson.M `json:"fields" bson:"fields"`
}
The problem I'm having is that in the Investment model, Group is not required. If there is no group, I think its better for it to not be inserted in the db. Whats the best way to handle a db model such as this in Go?
tl;dr: Use ,omitempty, and if you need to worry about the difference between a zero value and null/not specified, do what the GitHub API does and use a pointer.
Both json and bson support the ,omitempty tag. For json, "empty values are false, 0, any nil pointer or interface value, and any array, slice, map, or string of length zero" (json docs). For bson, ,omitempty means "Only include the field if it's not set to the zero value for the type or to empty slices or maps", and zero values include empty strings and nil pointers (bson docs).
So if you really need a Group struct, you can put a *Group in instead, and it won't be stored when the pointer is nil. If Investment only needs to hold the group's name, it's even simpler: "" as group name keeps a group key from being stored.
bson defaults to using the lowercased field name already so you can omit that from the struct tag when they match. json will default to the Capitalized name, so specify the lowercase name in a tag if you need lowercase.
So, best case, maybe you can just use:
type Investment struct {
Base
Symbol string `json:"symbol" binding:"required"`
Group string `json:"group,omitempty" bson:",omitempty"`
Fields bson.M `json:"fields"`
}
If you ever run into fields where the zero value for the type ("", 0, false, etc.) is distinct from "not specified", you can do what the GitHub API does and put pointers in your structures--essentially an extension of the *Group trick.
Avoid strut fields to marshal if they are empty -
A struct field may be primitive type(string, int, bool etc) or even an another struct type.
So sometimes we don't want a struct's field to
go in json data(may to database insertion or in external api call) if they are empty
Example:
type Investment struct {
Base
Symbol string `json:"symbol" bson:"symbol" binding:"required"`
Group Group `json:"group" bson:"group"`
Fields bson.M `json:"fields" bson:"fields"`
}
If we want that Symbol and Group might contain empty values(0, false, nil pointer, zero size interface/struct) then we can avoid them in json marshaling like below.
type Investment struct {
Base
Symbol string `json:"symbol,omitempty" bson:"symbol,omitempty" binding:"required"`
Group *Group `json:"group,omitempty" bson:"group,omitempty"`
Fields bson.M `json:"fields" bson:"fields"`
}
Her "Group" field is pointer to Group struct and whenever it will point to nil pointer it will be omitted from json marshaling.
And obviously we would be filling values in Group field like below.
// declared investment variable of type Investment struct
investment.Group = &groupData
In the Go Language Specification, it mentions a brief overview of tags:
A field declaration may be followed by an optional string literal tag,
which becomes an attribute for all the fields in the corresponding
field declaration. The tags are made visible through a reflection
interface but are otherwise ignored.
// A struct corresponding to the TimeStamp protocol buffer.
// The tag strings define the protocol buffer field numbers.
struct {
microsec uint64 "field 1"
serverIP6 uint64 "field 2"
process string "field 3"
}
This is a very short explanation IMO, and I was wondering if anyone could provide me with what use these tags would be?
A tag for a field allows you to attach meta-information to the field which can be acquired using reflection. Usually it is used to provide transformation info on how a struct field is encoded to or decoded from another format (or stored/retrieved from a database), but you can use it to store whatever meta-info you want to, either intended for another package or for your own use.
As mentioned in the documentation of reflect.StructTag, by convention the value of a tag string is a space-separated list of key:"value" pairs, for example:
type User struct {
Name string `json:"name" xml:"name"`
}
The key usually denotes the package that the subsequent "value" is for, for example json keys are processed/used by the encoding/json package.
If multiple information is to be passed in the "value", usually it is specified by separating it with a comma (','), e.g.
Name string `json:"name,omitempty" xml:"name"`
Usually a dash value ('-') for the "value" means to exclude the field from the process (e.g. in case of json it means not to marshal or unmarshal that field).
Example of accessing your custom tags using reflection
We can use reflection (reflect package) to access the tag values of struct fields. Basically we need to acquire the Type of our struct, and then we can query fields e.g. with Type.Field(i int) or Type.FieldByName(name string). These methods return a value of StructField which describes / represents a struct field; and StructField.Tag is a value of type StructTag which describes / represents a tag value.
Previously we talked about "convention". This convention means that if you follow it, you may use the StructTag.Get(key string) method which parses the value of a tag and returns you the "value" of the key you specify. The convention is implemented / built into this Get() method. If you don't follow the convention, Get() will not be able to parse key:"value" pairs and find what you're looking for. That's also not a problem, but then you need to implement your own parsing logic.
Also there is StructTag.Lookup() (was added in Go 1.7) which is "like Get() but distinguishes the tag not containing the given key from the tag associating an empty string with the given key".
So let's see a simple example:
type User struct {
Name string `mytag:"MyName"`
Email string `mytag:"MyEmail"`
}
u := User{"Bob", "bob#mycompany.com"}
t := reflect.TypeOf(u)
for _, fieldName := range []string{"Name", "Email"} {
field, found := t.FieldByName(fieldName)
if !found {
continue
}
fmt.Printf("\nField: User.%s\n", fieldName)
fmt.Printf("\tWhole tag value : %q\n", field.Tag)
fmt.Printf("\tValue of 'mytag': %q\n", field.Tag.Get("mytag"))
}
Output (try it on the Go Playground):
Field: User.Name
Whole tag value : "mytag:\"MyName\""
Value of 'mytag': "MyName"
Field: User.Email
Whole tag value : "mytag:\"MyEmail\""
Value of 'mytag': "MyEmail"
GopherCon 2015 had a presentation about struct tags called:
The Many Faces of Struct Tags (slide) (and a video)
Here is a list of commonly used tag keys:
json - used by the encoding/json package, detailed at json.Marshal()
xml - used by the encoding/xml package, detailed at xml.Marshal()
bson - used by gobson, detailed at bson.Marshal(); also by the mongo-go driver, detailed at bson package doc
protobuf - used by github.com/golang/protobuf/proto, detailed in the package doc
yaml - used by the gopkg.in/yaml.v2 package, detailed at yaml.Marshal()
db - used by the github.com/jmoiron/sqlx package; also used by github.com/go-gorp/gorp package
orm - used by the github.com/astaxie/beego/orm package, detailed at Models – Beego ORM
gorm - used by gorm.io/gorm, examples can be found in their docs
valid - used by the github.com/asaskevich/govalidator package, examples can be found in the project page
datastore - used by appengine/datastore (Google App Engine platform, Datastore service), detailed at Properties
schema - used by github.com/gorilla/schema to fill a struct with HTML form values, detailed in the package doc
asn - used by the encoding/asn1 package, detailed at asn1.Marshal() and asn1.Unmarshal()
csv - used by the github.com/gocarina/gocsv package
env - used by the github.com/caarlos0/env package
Here is a really simple example of tags being used with the encoding/json package to control how fields are interpreted during encoding and decoding:
Try live: http://play.golang.org/p/BMeR8p1cKf
package main
import (
"fmt"
"encoding/json"
)
type Person struct {
FirstName string `json:"first_name"`
LastName string `json:"last_name"`
MiddleName string `json:"middle_name,omitempty"`
}
func main() {
json_string := `
{
"first_name": "John",
"last_name": "Smith"
}`
person := new(Person)
json.Unmarshal([]byte(json_string), person)
fmt.Println(person)
new_json, _ := json.Marshal(person)
fmt.Printf("%s\n", new_json)
}
// *Output*
// &{John Smith }
// {"first_name":"John","last_name":"Smith"}
The json package can look at the tags for the field and be told how to map json <=> struct field, and also extra options like whether it should ignore empty fields when serializing back to json.
Basically, any package can use reflection on the fields to look at tag values and act on those values. There is a little more info about them in the reflect package
http://golang.org/pkg/reflect/#StructTag :
By convention, tag strings are a concatenation of optionally
space-separated key:"value" pairs. Each key is a non-empty string
consisting of non-control characters other than space (U+0020 ' '),
quote (U+0022 '"'), and colon (U+003A ':'). Each value is quoted using
U+0022 '"' characters and Go string literal syntax.
It's some sort of specifications that specifies how packages treat with a field that is tagged.
for example:
type User struct {
FirstName string `json:"first_name"`
LastName string `json:"last_name"`
}
json tag informs json package that marshalled output of following user
u := User{
FirstName: "some first name",
LastName: "some last name",
}
would be like this:
{"first_name":"some first name","last_name":"some last name"}
other example is gorm package tags declares how database migrations must be done:
type User struct {
gorm.Model
Name string
Age sql.NullInt64
Birthday *time.Time
Email string `gorm:"type:varchar(100);unique_index"`
Role string `gorm:"size:255"` // set field size to 255
MemberNumber *string `gorm:"unique;not null"` // set member number to unique and not null
Num int `gorm:"AUTO_INCREMENT"` // set num to auto incrementable
Address string `gorm:"index:addr"` // create index with name `addr` for address
IgnoreMe int `gorm:"-"` // ignore this field
}
In this example for the field Email with gorm tag we declare that corresponding column in database for the field email must be of type varchar and 100 maximum length and it also must have unique index.
other example is binding tags that are used very mostly in gin package.
type Login struct {
User string `form:"user" json:"user" xml:"user" binding:"required"`
Password string `form:"password" json:"password" xml:"password" binding:"required"`
}
var json Login
if err := c.ShouldBindJSON(&json); err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()})
return
}
the binding tag in this example gives hint to gin package that the data sent to API must have user and password fields cause these fields are tagged as required.
So generraly tags are data that packages require to know how should they treat with data of type different structs and best way to get familiar with the tags a package needs is READING A PACKAGE DOCUMENTATION COMPLETELY.