type A struct {
Id int64
Email sql.NullString
Phone sql.NullString
}
Assume I have one record in the database
A{1, "x#x.com", "1112223333"}
Send an update request via PUT
curl -X PUT -d '{"Email": "y#y.com", "Phone": null}' http://localhost:3000/a/1
Here is the psuedo algorithm that would work with a full PUT request (i.e. update all fields of the record A - but it will cause difficulties with the PATCH request semantics - delta update)
-- Unmarshal json into empty record
a := A{}
json.Unmarshal([]byte(request.body), &a)
-- Load the record from the database
aFromDb = <assume we got record from db> //A{1, "x#x.com", "1112223333"}
-- Compare a and aFromDB
-- Notice the email change and set it on aFromDb - ok
-- Notice the phone number change -- but wait! Was it set to NULL in the JSON explicitly or was it not even included in the JSON? i.e. was the json request - {"Email": "y#y.com", "Phone": null} or was it {"Email": "y#y.com"}?
How can we tell by just looking at the unmarshaled json into the struct a?
Is there another method to do the update via rest (with patch semantics)? I am looking for a generic way to do it (not tied to a particular struct).
I created a separate datatype for this purpose. This example is for an int64 (actually string-encoded int64), but you can easily change it to a string as well. The idea behind it is, that the UnmarshalJSON method will only be called if the value is present in the JSON. The type will implement the Marshaler and the Unmarshaler.
// Used as a JSON type
//
// The Set flag indicates whether an unmarshaling actually happened on the type
type RequiredInt64 struct {
Set bool
Int64 int64
}
func (r RequiredInt64) MarshalJSON() ([]byte, error) {
lit := strconv.FormatInt(r.Int64, 10)
return json.Marshal(lit)
}
func (r *RequiredInt64) UnmarshalJSON(raw []byte) error {
var lit string
var err error
if err = json.Unmarshal(raw, &lit); err != nil {
return err
}
r.Int64, err = strconv.ParseInt(lit, 10, 64)
if err != nil {
return err
}
r.Set = true
return nil
}
So, if Set is false, you know that the value was not present in the JSON.
Try adding this tags to the struct:
type A struct {
Id int64 `json:"Id,omitempty"`
Email sql.NullString `json:"Email,omitempty"`
Phone sql.NullString `json:"Phone,omitempty"`
}
In this way if you are serializing and the field is empty then the json will not contain the field.
When deserializing though the field will have a either a value or it will have the default value for the type (Nil for the pointer or empty string for strings).
You could potentially write your own marshalling/uinmarshalling of your struct and react to the raw response within, although it might be non-obvious witgh inspection what that those functions are manipulating.
Alternatively, you could not omitempty within your fields and force null populated fields.
Or, maybe leveraging a different flavor of patching, perhaps http://jsonpatch.com/, which is more explicit in the nature of your modifications. This would require the client to be more understanding of the state of changes than say for a put.
Related
This question already has answers here:
preserve int64 values when parsing json in Go
(3 answers)
Closed 1 year ago.
I’m working on a rest api but for some reason can't type assert an interface{} to its underlying type - int.
I send data via post request, to create an ad. It looks like so:
POST http://localhost:8080/api/ads
{
"Title": "Example Title",
"Section": "machinery",
"CostPerDayInCent": 34500,
"Description": "A description",
"User": 4,
"AllowMobileContact": true,
"AllowEmailContact": true,
"IsActive": false
}
The passed values are decoded into a map[string]interface{} like so:
var adToValidate map[string]interface{}
err = json.NewDecoder(r.Body).Decode(&adToValidate)
if err != nil {
api.errorLog.Printf("Error decoding ad object: %v", err)
http.Error(w, http.StatusText(http.StatusInternalServerError), http.StatusInternalServerError)
return
}
My problem happens when I type assert the costperdayincent and user interface values to int.
localScopeAd.CostPerDayInCent, ok = adMapToValidate[“CostPerDayInCent”].(int)
if !ok {
fieldErrors[structFieldName] = fmt.Sprintf("%v value is not of int type", structFieldName)
}
The if statement executes, indicating it cant type assert.
Why is this? Is it because the passed json treats every value as a string?
How would I resolve this issue?
FOLLOW UP
I resolved this issue with the help of #DanielFarrell 's answer.
As I couldn't fit a response in the comment section, below is my reason for decoding into a map and not a struct:
I understand it would make a lot more sense to decode into a struct.
I had initially been decoding into a struct. However I had run into some issues when trying to validate the bool values.
i.e
"AllowMobileContact": true,
"AllowEmailContact": true,
"IsActive": false
If a user was to make a post request to create an ad leaving out the above values. When the request body was decoded the above fields would default to false in the struct (bools 0 value).
If i was then going to validate the passed in values I wouldn't know if the user had entered false or had left out a whole key value pair.
As I wanted to ensure the user had entered these values I first decoded into a map so that I could check if the key of the bool key value pair was present.
Then I could send the relevant response if there was some missing required bool data.
If you know of a simpler way of going about the above I'd be interested in hearing it. There's some 3rd party packages that have 3 value boolean types which may have worked but I decided to go with the above instead.
I would prefer to decode into a struct and let json/encoding do the work of handling proper type.
type Input struct {
Title,
Selection string
CostPerDayInCent int64
Description string
User int64
AllowMobileContact,
AllowEmailContact,
IsActive bool
}
This approach is quite common and useful for several reasons. First, you get to choose the type. Second, you are defining a type, so you get to attach functions, which I find quite convenient as a method to organize my code and avoid having to pass structures as explicit function arguments. Third, you can attach annotations to the struct's fields, further adjusting unmarshalling.
Finally, and to me, most importantly, you're thinking about data the way Go handles it. In many of the popular modern languages such as Python, the code - and the documentation - doesn't usually attach types to functions, or attempt to explicitly enumerate attributes of objects. Go is different. Its removal of inherent language reflection is one of the reasons it is so performant. This can provide vast benefits to the programmer - when you know all the types, you know exactly how things will behave, and can precisely identify what you must pass the functions you call. I suggest you embrace the explicit typing, and avoid decoding json into interface-oriented types whenever possible. You'll know what you're handling, and you'll be able to define it explicitly for your consumers as well.
https://play.golang.org/p/egaequIT8ET contrasts your approach, in which you don't need to bother defining the types - but you also don't have the opportunity to choose them.
package main
import (
"bytes"
"encoding/json"
"fmt"
)
type Input struct {
Title,
Selection string
CostPerDayInCent int64
Description string
User int64
AllowMobileContact,
AllowEmailContact,
IsActive bool
}
func main() {
var input = `{
"Title": "Example Title",
"Section": "machinery",
"CostPerDayInCent": 34500,
"Description": "A description",
"User": 4,
"AllowMobileContact": true,
"AllowEmailContact": true,
"IsActive": false
}`
// map[string]interface
data := make(map[string]interface{})
if err := json.NewDecoder(bytes.NewBufferString(input)).Decode(&data); err != nil {
panic(err)
}
fmt.Printf("%f %T\n", data["CostPerDayInCent"], data["CostPerDayInCent"])
// structure
obj := new(Input)
if err := json.NewDecoder(bytes.NewBufferString(input)).Decode(obj); err != nil {
panic(err)
}
fmt.Printf("%d %T", obj.CostPerDayInCent, obj.CostPerDayInCent)
}
I made a MySQL query using GORM inside a little go app.
I have declared my domain struct
type Domain struct {
gorm.Model
Name string
...
}
Then when I send a query to MySQL with GORM with this method.
func DomainInfos(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
w.WriteHeader(http.StatusOK)
var d Domain
config.DbConnection.Where("name = ?", vars["domain"]).Find(&d)
json.NewEncoder(w).Encode(d)
}
When the domain is not found it return an default object from my struct definition
{
{0 0001-01-01 00:00:00 +0000 UTC 0001-01-01 00:00:00 +0000 UTC <nil>}
0
0
0
}
So I wrote a little condition to manually return an empty object
if d.ID == 0 {
json.NewEncoder(w).Encode(make(map[string]string))
return
}
Is it possible GORM return an empty object directly when the query return nothing, to avoid this manual checking ?
GORM is returning an empty object; when it comes to Go values, "empty" and "default" are the same, and are actually called the zero value. In your situation, you're needing to control the JSON output, not the GORM return value.
You can add the omitempty tag to your fields to have them exluded from JSON output if they contain the zero value for their type:
type Domain struct {
gorm.Model
Name string `json:",omitempty"`
...
}
For each field with this tag, when you call Encode or Marshal, if the field contains its zero value (e.g. for Name, which is a string, if it is equal to ""), the field will not be included in the output. If you tag all the exported fields this way, and they all contain their zero values, the output will be an empty JSON object {}.
Also note that this:
json.NewEncoder(w).Encode(make(map[string]string))
Is equivalent to, but significantly less efficient than:
w.Write([]byte("{}"))
Your other option would be a custom marshal func, something like this:
func (d Domain) MarshalJSON() ([]byte, error) {
if t.ID == 0 {
return []byte("{}"), nil
}
// Wrap the type to avoid infinite recursion on MarshalJSON
type dd Domain
return json.Marshal(dd(d))
}
You can see a working sample here: https://play.golang.org/p/mIRfRKXeyyW
I created map[string]interface{} and i want to pass multiple types through the map between 2 reset services.
every time i marshal i get empty map in the field that should contain reflect.Type.
ServiceType:map[]
is it even possible?
code for testing: (the KeyValuePair will represent a single value of the map) and the MessageService can be replaced with any type of course
data := GenericHandlers.KeyValuePair{Key:"ServiceType",Value:reflect.TypeOf(MessageService.MessageService{})}
Json , _ := json.Marshal(data)
resKvp := GenericHandlers.KeyValuePair{}
err := json.Unmarshal(Json,&resKvp)
if(err != nil){
log.Println(err.Error())
}
fmt.Println(resKvp)
type KeyValuePair struct{
Key string
Value interface{}
}
Short answer: No, it's not possible.
Slightly longer answer: Since relfect.Type, as returned by TypeOf, is an interface type implemented by the unexported type rtype, none of whose fields are exported and whose method set does not contain the MarshalJSON/UnmarshalJSON methods, you cannot marshal or unmarshal a reflect.Type value as is.
I am trying to understand the code that is used at my company. I am new to go lang, and I have already gone through the tutorial on their official website. However, I am having a hard time wrapping my head around empty interfaces, i.e. interface{}. From various sources online, I figured out that the empty interface can hold any type. But, I am having a hard time figuring out the codebase, especially some of the functions. I will not be posting the entire thing here, but just the minimal functions in which it has been used. Please bear with me!
Function (I am trying to understand):
func (this *RequestHandler) CreateAppHandler(rw http.ResponseWriter, r *http.Request) *foo.ResponseError {
var data *views.Data = &views.Data{Attributes: &domain.Application{}}
var request *views.Request = &views.Request{Data: data}
if err := json.NewDecoder(r.Body).Decode(request); err != nil {
logrus.Error(err)
return foo.NewResponsePropogateError(foo.STATUS_400, err)
}
requestApp := request.Data.Attributes.(*domain.Application)
requestApp.CreatedBy = user
Setting some context, RequestHandler is a struct defined in the same package as this code. domain and views are seperate packages. Application is a struct in the package domain. The following two structs are part of the package views:
type Data struct {
Id string `json:"id"`
Type string `json:"type"`
Attributes interface{} `json:"attributes"`
}
type Request struct {
Data *Data `json:"data"`
}
The following are part of the package json:
func NewDecoder(r io.Reader) *Decoder {
return &Decoder{r: r}
}
func (dec *Decoder) Decode(v interface{}) error {
if dec.err != nil {
return dec.err
}
if err := dec.tokenPrepareForDecode(); err != nil {
return err
}
if !dec.tokenValueAllowed() {
return &SyntaxError{msg: "not at beginning of value"}
}
// Read whole value into buffer.
n, err := dec.readValue()
if err != nil {
return err
}
dec.d.init(dec.buf[dec.scanp : dec.scanp+n])
dec.scanp += n
// Don't save err from unmarshal into dec.err:
// the connection is still usable since we read a complete JSON
// object from it before the error happened.
err = dec.d.unmarshal(v)
// fixup token streaming state
dec.tokenValueEnd()
return err
}
type Decoder struct {
r io.Reader
buf []byte
d decodeState
scanp int // start of unread data in buf
scan scanner
err error
tokenState int
tokenStack []int
}
Now, I understood that, in the struct Data in package views, Application is being set as a type for the empty interface. After that, a pointer to Request in the same package is created which points to the variable data.
I have the following doubts:
What exactly does this keyword mean in Go? What is the purpose of writing this * RequestHandler?
Initialization of a structure in Go can be done while assigning it to a variable by specifying the values of all it's members. However, here, for the struct Data, only the empty interface value is assigned and the values for the other two fields are not assigned?
What is the advantage of assigning the Application struct to an empty interface? Does it mean I can use the struct members using the interface variable directly?
Can someone help me figure out the meaning of this statement? json.NewDecoder(r.Body).Decode(request)?
While I know this is too much, but I am having a hard time figuring out the meaning of interfaces in Go. Please help!
this is not a keyword in go; any variable name can be used there. That is called the receiver. A function declared in that way must be called like thing.func(params), where "thing" is an expression of the type of the receiver. Within the function, the receiver is set to the value of thing.
A struct literal does not have to contain values for all the fields (or any of them). Any fields not explicitly set will have the zero value for their types.
As you said, an empty interface can take on a value of any type. To use a value of type interface{}, you would use type assertion or a type switch to determine the type of the value, or you could use reflection to use the value without having to have code for the specific type.
What specifically about that statement do you not understand? json is the name of a package in which the function NewDecoder is declared. That function is called, and then the Decode function (which is implemented by the type of the return value of NewDecoder) is called on that return value.
You may want to take a look at Effective Go and/or The Go Programming Language Specification for more information.
If there any clean way to check if any field value of struct is nil?
Suppose i have
type test_struct struct {
Name string `json:"name" binding:"required"`
Email string `json:"email" binding:"required"`
Message string `json:"message" binding:"required"`
}
And with Gin i am filling the struct with values as
var temp test_struct
c.Bind(&temp)
And everything works fine, but i want to check if any of temp.Name, temp.Email, temp.Message is nil, sure we can check it by simply comparing each field with nil: if temp.Name == nil and so on, but i am looking for a cleaner version of that, is there any?
UPD: Due to a lack of knowledge in Go language i didn't know that the Bind function of gin package return an error if we pass a strucure with binding:"required" fields. Thanks to #matt.s i get it. So the answer would be to check the err:
var temp test_struct
err := c.Bind(&temp)
if err != nil {
// Do my stuff(some of fields are not set)
}
You should first check that Bind doesn't return an error. If it doesn't then all fields will be set to their appropriate values or initialized to zero values if not. This means that the strings are guaranteed to not be nil (though they will be set to "" if they did not have a value).