Transform values when umarshaling - go

Given the code below, is it possible to transform first name while it's being unmarshaled? Say I want to always have it be lowercase whether it is in the actual json or not.
type Person struct {
FirstName string `json:"first_name"`
LastName string `json:"last_name"`
}
func main() {
jsonText := GetJsonFromSomewhere()
var v Person
json.Unmarshal(jsonText, &v)
}

One way to do this is to create a custom type that implements the Unmarshaler interface from the encoding/json package. Here's a link to this interface. Any type which implements Unmarshaler can be used as the type of a struct field when doing JSON unmarshalling. When doing the unmarshalling, encoding/json will use your implementation of the interface's UnmarshalJSON function to convert the JSON bytes to the field type.
So you could write an UnmarshalJSON function which includes changing the string values to lowercase.
Here's an example of what that could look like:
type LowerCaseString string
func (l *LowerCaseString) UnmarshalJSON(bytes []byte) error {
lowerCasedString := strings.ToLower(string(bytes))
*l = LowerCaseString(lowerCasedString)
return nil
}
Then, in your struct for JSON mapping, you can use your custom type instead of string:
type Person struct {
FirstName LowerCaseString `json:"first_name"`
LastName LowerCaseString `json:"last_name"`
}
If you unmarshal into this struct, the values of FirstName and LastName will be lowercased (also note that you'll need to type convert them back to string to use them as strings).
testJSON := `{"first_name" : "TestFirstNAME", "last_name": "TestLastNAME"}`
var result Person
err := json.Unmarshal([]byte(testJSON), &result)
if err != nil { /*handle the error*/ }
fmt.Println(result.FirstName) // prints "testfirstname"
var stringLastName string
stringLastName = string(result.LastName) // need to type convert from LowerCaseString to string
fmt.Println(stringLastName) // prints "testlastname"
Here is the above code running in Go Playground.

Related

How to trim white space for query and json in go gin?

I have a struct like this
type Data struct {
Foo string `json:"foo" binding:"required"`
}
And I use ShouldBind to bind query or json body to the struct.
data := Data{}
err := ctx.ShouldBind(&data)
I was wondering what is the best practice to trim white space for the string field?
transform {"foo": " bar "} to struct {"foo": "bar"}
I have tried using custom string type, and add custom UnmarshalJSON function, but it won't work for ctx.shouldBind if it is query.
type Data struct {
Foo TrimSpaceString `json:"foo" binding:"required"`
}
type TrimSpaceString string
func (t *TrimSpaceString) UnmarshalJSON(data []byte) error {
data = bytes.Trim(data, "\"")
data = bytes.Trim(data, " ")
*t = TrimSpaceString(strings.TrimSpace(string(data)))
return nil
}
I also tried to use conform and add tag for struct. But I have to add conform.Strings(data) after bind it and it is not convinence.
type Data struct {
Foo TrimSpaceString `json:"foo" binding:"required" conform:"trim"`
}
err := ctx.ShouldBind(&data)
conform.Strings(&data)
Should I custom a Binding and trim string inside Binding?
I got this working by modifying UnmarshalJSON function a bit. I don't have anything to explain as I'm also new, but will try, what's different is that we are converting byte data into a string by json.Unmarshal.Then we trim the space on string and appends it to the field or the original string
type trim string
func (t *trim) UnmarshalJSON(data []byte) error {
var s string
if err := json.Unmarshal(data, &s); err != nil {
return err
}
*t = trim(strings.TrimSpace(s))
fmt.Printf("these are the strings: '%s'\n", *t)
return nil
}
Here is the playground if you want to try it yourself
https://go.dev/play/p/BuxvMQibVsn

Unable to use type string as sql.NullString

I am creating a gorm model
// Day is a corresponding day entry
type Day struct {
gorm.Model
Dateday string `json:"dateday" gorm:"type:date;NOT NULL"`
Nameday string `json:"nameday" gorm:"type:varchar(100);NOT NULL"`
Something sql.NullString `json:"salad"`
Holyday bool `json:"holyday"`
}
I am using sql.NullString for the field Something cause it may be NULL.
So when I try to execute a typical gorm example to validate my setup works:
db.Create(&Day{
Nameday: "Monday",
Dateday: "23-10-2019",
Something: "a string goes here",
Holyday: false,
})
I get:
cannot use "a string goes here", (type string) as type sql.NullString in field value
What type should I use for the Something field given it may be NULL?
The sql.NullString type is not actually a string type but a struct type. It's defined as:
type NullString struct {
String string
Valid bool // Valid is true if String is not NULL
}
Therefore you need to initialize it as such:
db.Create(&Day{
Nameday: "Monday",
Dateday: "23-10-2019",
Something: sql.NullString{String: "a string goes here", Valid: true},
Holyday: false,
})
As an alternative, if you want to keep using the simpler syntax when initializing a nullable string, you could declare your own nullable string type, have it implement the sql.Scanner and driver.Valuer interfaces, and leverage the null byte to signal a NULL value.
type MyString string
const MyStringNull MyString = "\x00"
// implements driver.Valuer, will be invoked automatically when written to the db
func (s MyString) Value() (driver.Value, error) {
if s == MyStringNull {
return nil, nil
}
return []byte(s), nil
}
// implements sql.Scanner, will be invoked automatically when read from the db
func (s *String) Scan(src interface{}) error {
switch v := src.(type) {
case string:
*s = String(v)
case []byte:
*s = String(v)
case nil:
*s = StringNull
}
return nil
}
With this, if you declare the field Something to be of type MyString you can initialize it as you originally intended.
db.Create(&Day{
Nameday: "Monday",
Dateday: "23-10-2019",
// here the string expression is an *untyped* string constant
// that will be implicitly converted to MyString because
// both `string` and `MyString` have the same *underlying* type.
Something: "a string goes here",
Holyday: false,
})
Just keep in mind that this works only with untyped constants, once you have a constant or variable of type string, to be able to assign that to a MyString you'll need to use an explicit conversion.
var s string
var ms MyString
s = "a string goes here"
ms = s // won't compile because s is not an untyped constant
ms = MyString(s) // you have to explicitly convert
package main
import (
"github.com/guregu/null"
)
func main() {
db.Create(&Day{
Nameday: null.StringFrom("Monday"),
})
}

How to unmarshall Go struct field of type map[string]interface

I am trying to unmarshall multidimensional JSON. My JSON is contains dynamic key therefore I can't do it.
JSON
{
"id":"3",
"datetime":"2019-06-08",
"metadata":[{"a":"A"},{"b":"B"}]
}
Go file
type Chats struct {
Id string json:"id"
Datetime string json:"date"
Metadata string json:"metadata"
}
chat := models.Chats{}
err := c.BindJSON(&chat)
if err != nil {
c.Error(err)
return
}
fmt.Println(chat)
If metadata is dynamic then treat is as an interface{}. If you know it's always going to be a JSON container then you could do map[string]interface{} for convenience. There is also json.RawMessage if you don't necessarily want to use type assertions to see what's inside of it, but just want to preserve the JSON (I'm guessing this is what you were hoping to do by setting it to string).
type Chats struct {
Id string `json:"id"`
Datetime string `json:"date"`
Metadata interface{} `json:"metadata"`
}
type Chats struct {
Id string `json:"id"`
Datetime string `json:"date"`
Metadata map[string]interface{} `json:"metadata"`
}
type Chats struct {
Id string `json:"id"`
Datetime string `json:"date"`
Metadata json.RawMessage `json:"metadata"`
}

Go unmarshal JSON to struct with nil values [duplicate]

Since empty string is the zero/default value for Go string, I decided to define all such fields as interface{} instead. for example
type student struct {
FirstName interface{} `json:"first_name"`
MiddleName interface{} `json:"middle_name"`
LastName interface{} `json:"last_name"`
}
The application I am sending my data expect a null instead of an empty string if value is not available for that specific field.
Is this the correct approach or can someone please point me to something better than this.
In json package documentation :
Pointer values encode as the value pointed to. A nil pointer encodes as the null JSON object.
So you can store a pointer to a string which will be encoded as a string if not nil and will be encoded as "null" if nil
type student struct {
FirstName *string `json:"first_name"`
MiddleName *string `json:"middle_name"`
LastName *string `json:"last_name"`
}
For the case of a json object with null strings, its easiest to use the omitempty decorator on the field.
type student struct {
FirstName string `json:"first_name,omitempty"`
MiddleName string `json:"middle_name,omitempty"`
LastName string `json:"last_name"`
}
With the above declaration, only if first_name was assigned will that key show up in the resultant json. last_name, on the otherhand, will always show up in the result with a value of "" if not assigned.
Now when you start including numeric fields where 0 could be a value, using omitempty doesn't do what one would expect. A 0 value always drops the field, and we need to be able to differentiate between a 0 value and an unassigned value. Here use of library such as https://github.com/guregu/null may be advisable.
More discussion here: https://www.sohamkamani.com/blog/golang/2018-07-19-golang-omitempty/
Can be used https://github.com/guregu/null
type student struct {
FirstName null.String `json:"first_name"`
MiddleName null.String `json:"middle_name"`
LastName null.String `json:"last_name"`}
Another way actually is a workaround with using the MarhshalJSON and UnmarshalJSON interface method offered by the json lib of golang. The code is as below:
type MyType string
type MyStruct struct {
A MyType `json:"my_type"`
}
func (c MyType) MarshalJSON() ([]byte, error) {
var buf bytes.Buffer
if len(string(c)) == 0 {
buf.WriteString(`null`)
} else {
buf.WriteString(`"` + string(c) + `"`) // add double quation mark as json format required
}
return buf.Bytes(), nil
}
func (c *MyType)UnmarshalJSON(in []byte) error {
str := string(in)
if str == `null` {
*c = ""
return nil
}
res := MyType(str)
if len(res) >= 2 {
res = res[1:len(res)-1] // remove the wrapped qutation
}
*c = res
return nil
}
then when using json.Marshal, the MyType value will be marshaled as null.
You could use something like sql.NullString,use 'Valid' to check if it is a nil value.
NullString represents a string that may be null. NullString implements
the Scanner interface so it can be used as a scan destination:
type NullString struct {
String string
Valid bool // Valid is true if String is not NULL
}
var s NullString
err := db.QueryRow("SELECT name FROM foo WHERE id=?", id).Scan(&s)
...
if s.Valid {
// use s.String
} else {
// NULL value
}
Please DO NOT use pointers like below:
type student struct {
FirstName *string `json:"first_name"`
MiddleName *string `json:"middle_name"`
LastName *string `json:"last_name"`
}
Because you need check nil value and dereference like below everywhere,and maybe cause unexpected crash somewhere:
if obj.FirstName != nil {
fmt.Print("%s", *obj.FirstName)
}
I have compared two solution and choose former method in my extensive production code... it works ok.

Assigning null to JSON fields instead of empty strings

Since empty string is the zero/default value for Go string, I decided to define all such fields as interface{} instead. for example
type student struct {
FirstName interface{} `json:"first_name"`
MiddleName interface{} `json:"middle_name"`
LastName interface{} `json:"last_name"`
}
The application I am sending my data expect a null instead of an empty string if value is not available for that specific field.
Is this the correct approach or can someone please point me to something better than this.
In json package documentation :
Pointer values encode as the value pointed to. A nil pointer encodes as the null JSON object.
So you can store a pointer to a string which will be encoded as a string if not nil and will be encoded as "null" if nil
type student struct {
FirstName *string `json:"first_name"`
MiddleName *string `json:"middle_name"`
LastName *string `json:"last_name"`
}
For the case of a json object with null strings, its easiest to use the omitempty decorator on the field.
type student struct {
FirstName string `json:"first_name,omitempty"`
MiddleName string `json:"middle_name,omitempty"`
LastName string `json:"last_name"`
}
With the above declaration, only if first_name was assigned will that key show up in the resultant json. last_name, on the otherhand, will always show up in the result with a value of "" if not assigned.
Now when you start including numeric fields where 0 could be a value, using omitempty doesn't do what one would expect. A 0 value always drops the field, and we need to be able to differentiate between a 0 value and an unassigned value. Here use of library such as https://github.com/guregu/null may be advisable.
More discussion here: https://www.sohamkamani.com/blog/golang/2018-07-19-golang-omitempty/
Can be used https://github.com/guregu/null
type student struct {
FirstName null.String `json:"first_name"`
MiddleName null.String `json:"middle_name"`
LastName null.String `json:"last_name"`}
Another way actually is a workaround with using the MarhshalJSON and UnmarshalJSON interface method offered by the json lib of golang. The code is as below:
type MyType string
type MyStruct struct {
A MyType `json:"my_type"`
}
func (c MyType) MarshalJSON() ([]byte, error) {
var buf bytes.Buffer
if len(string(c)) == 0 {
buf.WriteString(`null`)
} else {
buf.WriteString(`"` + string(c) + `"`) // add double quation mark as json format required
}
return buf.Bytes(), nil
}
func (c *MyType)UnmarshalJSON(in []byte) error {
str := string(in)
if str == `null` {
*c = ""
return nil
}
res := MyType(str)
if len(res) >= 2 {
res = res[1:len(res)-1] // remove the wrapped qutation
}
*c = res
return nil
}
then when using json.Marshal, the MyType value will be marshaled as null.
You could use something like sql.NullString,use 'Valid' to check if it is a nil value.
NullString represents a string that may be null. NullString implements
the Scanner interface so it can be used as a scan destination:
type NullString struct {
String string
Valid bool // Valid is true if String is not NULL
}
var s NullString
err := db.QueryRow("SELECT name FROM foo WHERE id=?", id).Scan(&s)
...
if s.Valid {
// use s.String
} else {
// NULL value
}
Please DO NOT use pointers like below:
type student struct {
FirstName *string `json:"first_name"`
MiddleName *string `json:"middle_name"`
LastName *string `json:"last_name"`
}
Because you need check nil value and dereference like below everywhere,and maybe cause unexpected crash somewhere:
if obj.FirstName != nil {
fmt.Print("%s", *obj.FirstName)
}
I have compared two solution and choose former method in my extensive production code... it works ok.

Resources