I'm stuck when I'm updating an empty string values in a struct for updating dynamodb table.
Currently I have this struct
type Client struct {
ClientID *string `json:"client_key,omitempty"`
Name *string `json:"client_name,omitempty"`
Address *string `json:"address,omitempty"`
Country *string `json:"country,omitempty"`
City *string `json:"city,omitempty"`
State *string `json:"state,omitempty"`
PostCode *string `json:"post_code,omitempty"`
CreatedAt *time.Time `json:"created_at,omitempty"`
}
And this code when updating an item
keyAttr, err := dynamodbattribute.MarshalMap(key)
if err != nil {
return nil, err
}
valAttr, err := dynamodbattribute.MarshalMap(attributes)
if err != nil {
return nil, err
}
keyAttr will be used for the Key field and valAttr will be used in ExpressionAttributeValues field. Note that I didn't include the complete updating fields function to save space. But I will do that if you ask for it.
Currently the function is running fine except when I updated one of the field with empty string. E.g. client.Address = aws.String(""). While I'm fine with dynamodb converting my empty string to null, I can't seem to find a way to update it because of the omitempty tag.
I need the omitempty tag to ignore all nil values. However, I just researched that the omitempty tag also omits empty string values. Currently I have to make a struct in my function like this.
type client struct {
Name *string `json:"client_name"`
Address *string `json:"address"`
Country *string `json:"country"`
City *string `json:"city"`
State *string `json:"state"`
PostCode *string `json:"post_code"`
}
But i'm not a fan of repeating things. So, the question is: is there any better way of doing this? How do you guys use structs with dynamodb?
EDIT
Based on #Peter's comment, it seems that json.Encode() does print the empty string if it's not nil.
{"client_key":"test","username":"test","email":"","first_name":"test","last_name":"","phone":"","title":"","email_verified":false,"phone_verified":false,"updated_at":"2018-12-06T14:04:56.2743841+11:00"}
The problem seems to be in dynamodbattribute.MarshalMap function
After several trials, I finally got it. I didn't test it so I don't know whether it's buggy or not. But it seems to work for me right now.
So what I did was encode the struct with json.Marshal first then use json.Unmarshal with a map[string]interface{}. Then, I use dynamodbattribute.Marshal to convert it to map[string]*AttributeValue
Here's the code:
var temp map[string]interface{}
json.Unmarshal(tempStr, &temp)
valAttr, err := dynamodbattribute.MarshalMap(temp)
if err != nil {
return nil, err
}
Related
I'm facing this issue in golang that I haven't been able to figure out.
I have a client api which returns a pointer to a struct and inside that struct there's another pointer to another struct that contains the information I want to access, the problem is, I haven't been able to successfully obtain the value behind the two pointers.
The problem looks like this:
The function that returns the first struct:
func (c *ClientWithResponses) TriggerPipelineWithBodyWithResponse(ctx context.Context, projectSlug string, contentType string, body io.Reader, reqEditors ...RequestEditorFn) (*TriggerPipelineResponse, error) {
rsp, err := c.TriggerPipelineWithBody(ctx, projectSlug, contentType, body, reqEditors...)
if err != nil {
return nil, err
}
return ParseTriggerPipelineResponse(rsp)
}
Then, the returned struct TriggerPipelineResponse looks like this:
type TriggerPipelineResponse struct {
Body []byte
HTTPResponse *http.Response
JSON201 *PipelineCreation
JSONDefault *ListContextsDefaultResponse
}
Finally, PipelineCreation looks like this:
type PipelineCreation struct {
CreatedAt time.Time `json:"created_at"`
Id openapi_types.UUID `json:"id"`
Number int64 `json:"number"`
State PipelineCreationState `json:"state"`
}
When I execute the function TriggerPipelineWithBodyWithResponse I get an object back but impossible for me to access the field Id down the stack, I have tried a lot of combinations using pointer dereference with no success
I executed the function like this
response, err := c.client.TriggerPipelineWithBodyWithResponse(
...
)
Then I tried to access the value I need:
fmt.Println("%#v", *response) // This seems fine it shows me the `TriggerPipelineResponse` struct
fmt.Println("%#v", *response.JSON201) // invalid memory address or nil pointer dereference
fmt.Println("%#v", *(*response).JSON201) // invalid memory address or nil pointer dereference
Since empty string is the zero/default value for Go string, I decided to define all such fields as interface{} instead. for example
type student struct {
FirstName interface{} `json:"first_name"`
MiddleName interface{} `json:"middle_name"`
LastName interface{} `json:"last_name"`
}
The application I am sending my data expect a null instead of an empty string if value is not available for that specific field.
Is this the correct approach or can someone please point me to something better than this.
In json package documentation :
Pointer values encode as the value pointed to. A nil pointer encodes as the null JSON object.
So you can store a pointer to a string which will be encoded as a string if not nil and will be encoded as "null" if nil
type student struct {
FirstName *string `json:"first_name"`
MiddleName *string `json:"middle_name"`
LastName *string `json:"last_name"`
}
For the case of a json object with null strings, its easiest to use the omitempty decorator on the field.
type student struct {
FirstName string `json:"first_name,omitempty"`
MiddleName string `json:"middle_name,omitempty"`
LastName string `json:"last_name"`
}
With the above declaration, only if first_name was assigned will that key show up in the resultant json. last_name, on the otherhand, will always show up in the result with a value of "" if not assigned.
Now when you start including numeric fields where 0 could be a value, using omitempty doesn't do what one would expect. A 0 value always drops the field, and we need to be able to differentiate between a 0 value and an unassigned value. Here use of library such as https://github.com/guregu/null may be advisable.
More discussion here: https://www.sohamkamani.com/blog/golang/2018-07-19-golang-omitempty/
Can be used https://github.com/guregu/null
type student struct {
FirstName null.String `json:"first_name"`
MiddleName null.String `json:"middle_name"`
LastName null.String `json:"last_name"`}
Another way actually is a workaround with using the MarhshalJSON and UnmarshalJSON interface method offered by the json lib of golang. The code is as below:
type MyType string
type MyStruct struct {
A MyType `json:"my_type"`
}
func (c MyType) MarshalJSON() ([]byte, error) {
var buf bytes.Buffer
if len(string(c)) == 0 {
buf.WriteString(`null`)
} else {
buf.WriteString(`"` + string(c) + `"`) // add double quation mark as json format required
}
return buf.Bytes(), nil
}
func (c *MyType)UnmarshalJSON(in []byte) error {
str := string(in)
if str == `null` {
*c = ""
return nil
}
res := MyType(str)
if len(res) >= 2 {
res = res[1:len(res)-1] // remove the wrapped qutation
}
*c = res
return nil
}
then when using json.Marshal, the MyType value will be marshaled as null.
You could use something like sql.NullString,use 'Valid' to check if it is a nil value.
NullString represents a string that may be null. NullString implements
the Scanner interface so it can be used as a scan destination:
type NullString struct {
String string
Valid bool // Valid is true if String is not NULL
}
var s NullString
err := db.QueryRow("SELECT name FROM foo WHERE id=?", id).Scan(&s)
...
if s.Valid {
// use s.String
} else {
// NULL value
}
Please DO NOT use pointers like below:
type student struct {
FirstName *string `json:"first_name"`
MiddleName *string `json:"middle_name"`
LastName *string `json:"last_name"`
}
Because you need check nil value and dereference like below everywhere,and maybe cause unexpected crash somewhere:
if obj.FirstName != nil {
fmt.Print("%s", *obj.FirstName)
}
I have compared two solution and choose former method in my extensive production code... it works ok.
I have a code:
values, err := redis.Values(c.Do("hgetall", value))
if err != nil {
fmt.Println("HGETALL", err)
}
/*
type UD struct {
created_at string
B time.Time
ended_at string
data string
status string
}
*/
if err := redis.ScanStruct(values, &UD); err != nil {
fmt.Println(err)
}
The error I got is
redigo.ScanStruct: cannot assign field B: cannot convert from Redis
bulk string to time.Time
How do I resolve this? Any examples of ScanStruct in detail for a variety of field types for Struct for reference?
The documentation for ScanStruct is quite clear:
Integer, float, boolean, string and []byte fields are supported.
Other field types are not supported time.Time included.
To resolve this, I'd go and make my own version of ScanStruct that can deal with the conversion between Redis' and whatever types I need to throw at it.
You can simply add an ignore tag to make time field avoid being marshaled.
type UD struct {
created_at string
B time.Time `redis:"-"`
ended_at string
data string
status string
}
I'm POSTing a JSON user object to my Golang application where I decode the 'req.body' into a 'User' struct.
err := json.NewDecoder(req.Body).Decode(user)
//handle err if there is one
and the 'User' struct:
type User struct {
Name string `json:"name,omitempty"`
Username string `json:"username,omitempty"`
Email string `json:"email,omitempty"`
Town string `json:"town,omitempty"`
//more fields here
}
While I don't need help with the actual validation, I would like know how to validate usernames only if it is included as part of the JSON object. At the moment, if a username isn't included then User.Username will still exist but be empty i.e. ""
How can I check to see if '"username"' was included as part of the POSTed object?
You can use a pointer to a string:
type User struct {
Name string `json:"name,omitempty"`
Username *string `json:"username,omitempty"`
Email string `json:"email,omitempty"`
Town string `json:"town,omitempty"`
//more fields here
}
func main() {
var u, u2 User
json.Unmarshal([]byte(`{"username":"hi"}`), &u)
fmt.Println("username set:", u.Username != nil, *u.Username)
json.Unmarshal([]byte(`{}`), &u2)
fmt.Println("username set:", u2.Username != nil)
fmt.Println("Hello, playground")
}
playground
To add to the above answer. Note that the order of the validate functions is important. I was getting the error because I was placing the UUIDV4 validation tag before the omitempty:
ParentID *string `json:"parent_id" validate:"uuid4,omitempty"`
Correct way is:
ParentID *string `json:"parent_id" validate:"omitempty,uuid4"`
Since empty string is the zero/default value for Go string, I decided to define all such fields as interface{} instead. for example
type student struct {
FirstName interface{} `json:"first_name"`
MiddleName interface{} `json:"middle_name"`
LastName interface{} `json:"last_name"`
}
The application I am sending my data expect a null instead of an empty string if value is not available for that specific field.
Is this the correct approach or can someone please point me to something better than this.
In json package documentation :
Pointer values encode as the value pointed to. A nil pointer encodes as the null JSON object.
So you can store a pointer to a string which will be encoded as a string if not nil and will be encoded as "null" if nil
type student struct {
FirstName *string `json:"first_name"`
MiddleName *string `json:"middle_name"`
LastName *string `json:"last_name"`
}
For the case of a json object with null strings, its easiest to use the omitempty decorator on the field.
type student struct {
FirstName string `json:"first_name,omitempty"`
MiddleName string `json:"middle_name,omitempty"`
LastName string `json:"last_name"`
}
With the above declaration, only if first_name was assigned will that key show up in the resultant json. last_name, on the otherhand, will always show up in the result with a value of "" if not assigned.
Now when you start including numeric fields where 0 could be a value, using omitempty doesn't do what one would expect. A 0 value always drops the field, and we need to be able to differentiate between a 0 value and an unassigned value. Here use of library such as https://github.com/guregu/null may be advisable.
More discussion here: https://www.sohamkamani.com/blog/golang/2018-07-19-golang-omitempty/
Can be used https://github.com/guregu/null
type student struct {
FirstName null.String `json:"first_name"`
MiddleName null.String `json:"middle_name"`
LastName null.String `json:"last_name"`}
Another way actually is a workaround with using the MarhshalJSON and UnmarshalJSON interface method offered by the json lib of golang. The code is as below:
type MyType string
type MyStruct struct {
A MyType `json:"my_type"`
}
func (c MyType) MarshalJSON() ([]byte, error) {
var buf bytes.Buffer
if len(string(c)) == 0 {
buf.WriteString(`null`)
} else {
buf.WriteString(`"` + string(c) + `"`) // add double quation mark as json format required
}
return buf.Bytes(), nil
}
func (c *MyType)UnmarshalJSON(in []byte) error {
str := string(in)
if str == `null` {
*c = ""
return nil
}
res := MyType(str)
if len(res) >= 2 {
res = res[1:len(res)-1] // remove the wrapped qutation
}
*c = res
return nil
}
then when using json.Marshal, the MyType value will be marshaled as null.
You could use something like sql.NullString,use 'Valid' to check if it is a nil value.
NullString represents a string that may be null. NullString implements
the Scanner interface so it can be used as a scan destination:
type NullString struct {
String string
Valid bool // Valid is true if String is not NULL
}
var s NullString
err := db.QueryRow("SELECT name FROM foo WHERE id=?", id).Scan(&s)
...
if s.Valid {
// use s.String
} else {
// NULL value
}
Please DO NOT use pointers like below:
type student struct {
FirstName *string `json:"first_name"`
MiddleName *string `json:"middle_name"`
LastName *string `json:"last_name"`
}
Because you need check nil value and dereference like below everywhere,and maybe cause unexpected crash somewhere:
if obj.FirstName != nil {
fmt.Print("%s", *obj.FirstName)
}
I have compared two solution and choose former method in my extensive production code... it works ok.