Golang Json Not returning expected values - go

I have a code # http://play.golang.org/p/HDlJJ54YqW
I wanted to print the Phone and email of a person.
It can be of multiple entries.
But getting the error undefined.
Can anyone help out.
Thanks.

Small details: you are referencing twice: you give the address of the address of the object to json.Unmarshal. Just give the address.
` allows for multiline, no need to split your json input.
I don't know what you where trying to achieve with u.Details[Phone:"1111"].Email, but this is no Go syntax. your Details member is a slice off Detail. A slice is similar to an array and can be accessed by index.
Also, your json does not match your object structure. If you want to have multiple details in one content, then it needs to be embed in an array ([ ])
You could do something like this: (http://play.golang.org/p/OP1zbPW_wk)
package main
import (
"encoding/json"
"fmt"
)
type Content struct {
Owner string
Details []*Detail
}
type Detail struct {
Phone string
Email string
}
func (c *Content) SearchPhone(phone string) *Detail {
for _, elem := range c.Details {
if elem.Phone == phone {
return elem
}
}
return nil
}
func (c *Content) SearchEmail(email string) *Detail {
for _, elem := range c.Details {
if elem.Email == email {
return elem
}
}
return nil
}
func main() {
encoded := `{
"Owner": "abc",
"Details": [
{
"Phone": "1111",
"Email": "#gmail"
},
{
"Phone": "2222",
"Email": "#yahoo"
}
]
}`
// Decode the json object
u := &Content{}
if err := json.Unmarshal([]byte(encoded), u); err != nil {
panic(err)
}
// Print out Email and Phone
fmt.Printf("Email: %s\n", u.SearchPhone("1111").Email)
fmt.Printf("Phone: %s\n", u.SearchEmail("#yahoo").Phone)
}

Related

govalidator ValidateMap validate map of array

I want to validate my input map of array with govalidator.ValidateMap.
Please can someone suggest for Sample mapTemplate for map of array.
Please find below the code snippet.
Thanks in Advance
package main
import (
"fmt"
"github.com/asaskevich/govalidator"
)
func main() {
var mapTemplate = map[string]interface{}{
"name": "required,alpha",
"categories": []interface{}{",alpha"}, //error: map validator has to be either map[string]interface{} or string; got []interface {}
}
var inputMap = map[string]interface{}{
"name": "Prabhu",
"categories": []interface{}{"category1", "category2"},
}
result, err := govalidator.ValidateMap(inputMap, mapTemplate)
if err != nil {
fmt.Println("error: " + err.Error())
}
fmt.Printf("result : %v\n", result)
for _, v := range inputMap["categories"].([]interface{}) {
fmt.Printf("category : %v\n", v)
}
}
It seems validation of slices has not yet been implemented. There is no check in the What to contribute list for slices/arrays.
You can however use the function ValidateArray to iterate over a slice and validate its members.
govalidator.ValidateArray(inputMap["categories"], func(val interface{}, i int) bool {
valStr, ok := val.(string)
if !ok {
return false
}
return govalidator.IsAlpha(valStr)
})

Unmarshal unknown JSON fields without structs

I am trying to unmarshal a JSON object which has an optional array, I am doing this without an array and this is what I got so far:
import (
"encoding/json"
"fmt"
)
func main() {
jo := `
{
"given_name": "Akshay Raj",
"name": "Akshay",
"country": "New Zealand",
"family_name": "Gollahalli",
"emails": [
"name#example.com"
]
}
`
var raw map[string]interface{}
err := json.Unmarshal([]byte(jo), &raw)
if err != nil {
panic(err)
}
fmt.Println(raw["emails"][0])
}
The emails field might or might not come sometime. I know I can use struct and unmarshal it twice for with and without array. When I try to get the index 0 of raw["emails"][0] I get the following error
invalid operation: raw["emails"][0] (type interface {} does not support indexing)
Is there a way to get the index of the emails field?
Update 1
I can do something like this fmt.Println(raw["emails"].([]interface{})[0]) and it works. Is this the only way?
The easiest way is with a struct. There's no need to unmarshal twice.
type MyStruct struct {
// ... other fields
Emails []string `json:"emails"`
}
This will work, regardless of whether the JSON input contains the emails field. When it is missing, your resulting struct will just have an uninitialized Emails field.
You can use type assertions. The Go tutorial on type assertions is here.
A Go playground link applying type assertions to your problem is here. For ease of reading, that code is replicated below:
package main
import (
"encoding/json"
"fmt"
)
func main() {
jo := `
{
"given_name": "Akshay Raj",
"name": "Akshay",
"country": "New Zealand",
"family_name": "Gollahalli",
"emails": [
"name#example.com"
]
}
`
var raw map[string]interface{}
err := json.Unmarshal([]byte(jo), &raw)
if err != nil {
panic(err)
}
emails, ok := raw["emails"]
if !ok {
panic("do this when no 'emails' key")
}
emailsSlice, ok := emails.([]interface{})
if !ok {
panic("do this when 'emails' value is not a slice")
}
if len(emailsSlice) == 0 {
panic("do this when 'emails' slice is empty")
}
email, ok := (emailsSlice[0]).(string)
if !ok {
panic("do this when 'emails' slice contains non-string")
}
fmt.Println(email)
}
As always you can use additional libraries for work with your json data. For example with gojsonq package it will like so:
package main
import (
"fmt"
"github.com/thedevsaddam/gojsonq"
)
func main() {
json := `
{
"given_name": "Akshay Raj",
"name": "Akshay",
"country": "New Zealand",
"family_name": "Gollahalli",
"emails": [
"name#example.com"
]
}
`
first := gojsonq.New().JSONString(json).Find("emails.[0]")
if first != nil {
fmt.Println(first.(string))
} else {
fmt.Println("There isn't emails")
}
}

How to return custom error if validation failed with Gin [duplicate]

This question already has an answer here:
Return custom error message from struct tag validation
(1 answer)
Closed 1 year ago.
For example I've got the following struct
type Address struct {
City string `json:"city" binding:"required"`
AddressLine string `json:"address_line" binding:"required"`
}
and I've got the following function to handle request from users
func AddressCreate(c *gin.Context) {
var address Address
if err := c.BindJSON(&address); err == nil {
// if everything is good save to database
// and return success message
db.Create(&address)
c.JSON(http.StatusOK, gin.H {"status":"success"})
} else {
c.JSON(http.StatusBadRequest, err)
}
}
Expected behavior is to return JSON, formatted this way
[
{
"city":"required"
}
{
"address_line":"required"
}
]
But I'm getting an error formatted like this
"Address.City": {
"FieldNamespace": "Address.City",
"NameNamespace": "City",
"Field": "City",
"Name": "City",
"Tag": "required",
"ActualTag": "required",
"Kind": 24,
"Type": {},
"Param": "",
"Value": ""
},
"Address.AddressLine": {
"FieldNamespace": "AddressLine",
"NameNamespace": "AddressLine",
"Field": "AddressLine",
"Name": "AddressLine",
"Tag": "required",
"ActualTag": "required",
"Kind": 24,
"Type": {},
"Param": "",
"Value": ""
}
What I tried:
I created function which casts error to ValidationErrors and iterates through all FieldError's in it
func ListOfErrors(e error) []map[string]string {
ve := e.(validator.ValidationErrors)
InvalidFields := make([]map[string]string, 0)
for _, e := range ve {
errors := map[string]string{}
// field := reflect.TypeOf(e.NameNamespace)
errors[e.Name] = e.Tag
InvalidFields = append(InvalidFields, errors)
}
return InvalidFields
}
The output look's much better
[
{
"City":"required"
},
{
"AddressLine":"required"
}
]
But I cannot solve the problem with the name of the fields. I cannot swap Name into name which I noted in structs tag json:"city". So my question is did I choose correct way to solve the problem if the answer is yes how to get structs JSON tag for field?
If you want it to be same as defined in your json tag, then you should use reflection to pull that tag from your data type.
I don't have your libraries, so can't compile and check it. But I believe what you are after should go along those lines:
func ListOfErrors(address *Address, e error) []map[string]string {
ve := e.(validator.ValidationErrors)
InvalidFields := make([]map[string]string, 0)
for _, e := range ve {
errors := map[string]string{}
// field := reflect.TypeOf(e.NameNamespace)
field, _ := reflect.TypeOf(address).Elem().FieldByName(e.Name)
jsonTag := string(field.Tag.Get("json"))
errors[jsonTag] = e.Tag
InvalidFields = append(InvalidFields, errors)
}
return InvalidFields
}
Note that it is a bit contrived as type of address parameter is essentially known. So, not strictly required as a function parameter. But you can change address *Address to address interface{} and use it for other types too.
Disclaimer: I skipped error checking for brevity, but you certainly should check for errors in your code (e.g. no such field error or no json tag on that field).
You can use ToSnake to snake case the names:
import (
"unicode"
)
// ToSnake convert the given string to snake case following the Golang format:
// acronyms are converted to lower-case and preceded by an underscore.
func ToSnake(in string) string {
runes := []rune(in)
length := len(runes)
var out []rune
for i := 0; i < length; i++ {
if i > 0 && unicode.IsUpper(runes[i]) && ((i+1 < length && unicode.IsLower(runes[i+1])) || unicode.IsLower(runes[i-1])) {
out = append(out, '_')
}
out = append(out, unicode.ToLower(runes[i]))
}
return string(out)
}
func ListOfErrors(e error) []map[string]string {
ve := e.(validator.ValidationErrors)
invalidFields := make([]map[string]string, 0)
for _, e := range ve {
errors := map[string]string{}
errors[ToSnake(e.Name)] = e.Tag
invalidFields = append(InvalidFields, errors)
}
return invalidFields
}

Unmarshal map[string]DynamoDBAttributeValue into a struct

I'm trying to set-up an AWS-lambda using aws-sdk-go that is triggered whenever a new user is added to a certain dynamodb table.
Everything is working just fine but I can't find a way to unmarshal a map map[string]DynamoDBAttributeValue like:
{
"name": {
"S" : "John"
},
"residence_address": {
"M": {
"address": {
"S": "some place"
}
}
}
}
To a given struct, for instance, a User struct. Here is shown an example of unsmarhaling a map[string]*dynamodb.AttributeValue into a given interface, but I can't find a way to do the same thing with map[string]DynamoDBAttributeValue even though these types seem to fit the same purposes.
map[string]DynamoDBAttributeValue is returned by a events.DynamoDBEvents from package github.com/aws/aws-lambda-go/events. This is my code:
package handler
import (
"context"
"github.com/aws/aws-lambda-go/events"
"github.com/aws/aws-sdk-go/service/dynamodb/dynamodbattribute"
"github.com/aws/aws-sdk-go/service/dynamodb"
)
func HandleDynamoDBRequest(ctx context.Context, e events.DynamoDBEvent) {
for _, record := range e.Records {
if record.EventName == "INSERT" {
// User Struct
var dynamoUser model.DynamoDBUser
// Of course this can't be done for incompatible types
_ := dynamodbattribute.UnmarshalMap(record.Change.NewImage, &dynamoUser)
}
}
}
Of course, I can marshal record.Change.NewImage to JSON and unmarshal it back to a given struct, but then, I would have to manually initialize dynamoUser attributes starting from the latter ones.
Or I could even write a function that parses map[string]DynamoDBAttributeValue to map[string]*dynamodb.AttributeValue like:
func getAttributeValueMapFromDynamoDBStreamRecord(e events.DynamoDBStreamRecord) map[string]*dynamodb.AttributeValue {
image := e.NewImage
m := make(map[string]*dynamodb.AttributeValue)
for k, v := range image {
if v.DataType() == events.DataTypeString {
s := v.String()
m[k] = &dynamodb.AttributeValue{
S : &s,
}
}
if v.DataType() == events.DataTypeBoolean {
b := v.Boolean()
m[k] = &dynamodb.AttributeValue{
BOOL : &b,
}
}
// . . .
if v.DataType() == events.DataTypeMap {
// ?
}
}
return m
}
And then simply use dynamodbattribute.UnmarshalMap, but on events.DataTypeMap it would be quite a tricky process.
Is there a way through which I can unmarshal a DynamoDB record coming from a events.DynamoDBEvent into a struct with a similar method shown for map[string]*dynamodb.AttributeValue?
I tried the function you provided, and I met some problems with events.DataTypeList, so I managed to write the following function that does the trick:
// UnmarshalStreamImage converts events.DynamoDBAttributeValue to struct
func UnmarshalStreamImage(attribute map[string]events.DynamoDBAttributeValue, out interface{}) error {
dbAttrMap := make(map[string]*dynamodb.AttributeValue)
for k, v := range attribute {
var dbAttr dynamodb.AttributeValue
bytes, marshalErr := v.MarshalJSON(); if marshalErr != nil {
return marshalErr
}
json.Unmarshal(bytes, &dbAttr)
dbAttrMap[k] = &dbAttr
}
return dynamodbattribute.UnmarshalMap(dbAttrMap, out)
}
I was frustrated that the type of NewImage from the record wasn't map[string]*dynamodb.AttributeValue so I could use the dynamodbattribute package.
The JSON representation of events.DynamoDBAttributeValue seems to be the same as the JSON represenation of dynamodb.AttributeValue.
So I tried creating my own DynamoDBEvent type and changed the type of OldImage and NewImage, so it would be marshalled into map[string]*dynamodb.AttributeValue instead of map[string]events.DynamoDBAttributeValue
It is a little bit ugly but it works for me.
package main
import (
"github.com/aws/aws-lambda-go/events"
"github.com/aws/aws-lambda-go/lambda"
"github.com/aws/aws-sdk-go/service/dynamodb"
"github.com/aws/aws-sdk-go/service/dynamodb/dynamodbattribute"
"fmt"
)
func main() {
lambda.Start(lambdaHandler)
}
// changed type of event from: events.DynamoDBEvent to DynamoDBEvent (see below)
func lambdaHandler(event DynamoDBEvent) error {
for _, record := range event.Records {
change := record.Change
newImage := change.NewImage // now of type: map[string]*dynamodb.AttributeValue
var item IdOnly
err := dynamodbattribute.UnmarshalMap(newImage, &item)
if err != nil {
return err
}
fmt.Println(item.Id)
}
return nil
}
type IdOnly struct {
Id string `json:"id"`
}
type DynamoDBEvent struct {
Records []DynamoDBEventRecord `json:"Records"`
}
type DynamoDBEventRecord struct {
AWSRegion string `json:"awsRegion"`
Change DynamoDBStreamRecord `json:"dynamodb"`
EventID string `json:"eventID"`
EventName string `json:"eventName"`
EventSource string `json:"eventSource"`
EventVersion string `json:"eventVersion"`
EventSourceArn string `json:"eventSourceARN"`
UserIdentity *events.DynamoDBUserIdentity `json:"userIdentity,omitempty"`
}
type DynamoDBStreamRecord struct {
ApproximateCreationDateTime events.SecondsEpochTime `json:"ApproximateCreationDateTime,omitempty"`
// changed to map[string]*dynamodb.AttributeValue
Keys map[string]*dynamodb.AttributeValue `json:"Keys,omitempty"`
// changed to map[string]*dynamodb.AttributeValue
NewImage map[string]*dynamodb.AttributeValue `json:"NewImage,omitempty"`
// changed to map[string]*dynamodb.AttributeValue
OldImage map[string]*dynamodb.AttributeValue `json:"OldImage,omitempty"`
SequenceNumber string `json:"SequenceNumber"`
SizeBytes int64 `json:"SizeBytes"`
StreamViewType string `json:"StreamViewType"`
}
I have found the same problem and the solution is to perform a simple conversion of types. This is possible because in the end the type received by lambda events events.DynamoDBAttributeValue and the type used by the SDK V2 of AWS DynamoDB types.AttributeValue are the same. Next I show you the conversion code.
package aws_lambda
import (
"github.com/aws/aws-lambda-go/events"
"github.com/aws/aws-sdk-go-v2/feature/dynamodb/attributevalue"
"github.com/aws/aws-sdk-go-v2/service/dynamodb/types"
)
func UnmarshalDynamoEventsMap(
record map[string]events.DynamoDBAttributeValue, out interface{}) error {
asTypesMap := DynamoDbEventsMapToTypesMap(record)
err := attributevalue.UnmarshalMap(asTypesMap, out)
if err != nil {
return err
}
return nil
}
func DynamoDbEventsMapToTypesMap(
record map[string]events.DynamoDBAttributeValue) map[string]types.AttributeValue {
resultMap := make(map[string]types.AttributeValue)
for key, rec := range record {
resultMap[key] = DynamoDbEventsToTypes(rec)
}
return resultMap
}
// DynamoDbEventsToTypes relates the dynamo event received by AWS Lambda with the data type that is
// used in the Amazon SDK V2 to deal with DynamoDB data.
// This function is necessary because Amazon does not provide any kind of solution to make this
// relationship between types of data.
func DynamoDbEventsToTypes(record events.DynamoDBAttributeValue) types.AttributeValue {
var val types.AttributeValue
switch record.DataType() {
case events.DataTypeBinary:
val = &types.AttributeValueMemberB{
Value: record.Binary(),
}
case events.DataTypeBinarySet:
val = &types.AttributeValueMemberBS{
Value: record.BinarySet(),
}
case events.DataTypeBoolean:
val = &types.AttributeValueMemberBOOL{
Value: record.Boolean(),
}
case events.DataTypeList:
var items []types.AttributeValue
for _, value := range record.List() {
items = append(items, DynamoDbEventsToTypes(value))
}
val = &types.AttributeValueMemberL{
Value: items,
}
case events.DataTypeMap:
items := make(map[string]types.AttributeValue)
for k, v := range record.Map() {
items[k] = DynamoDbEventsToTypes(v)
}
val = &types.AttributeValueMemberM{
Value: items,
}
case events.DataTypeNull:
val = nil
case events.DataTypeNumber:
val = &types.AttributeValueMemberN{
Value: record.Number(),
}
case events.DataTypeNumberSet:
val = &types.AttributeValueMemberNS{
Value: record.NumberSet(),
}
case events.DataTypeString:
val = &types.AttributeValueMemberS{
Value: record.String(),
}
case events.DataTypeStringSet:
val = &types.AttributeValueMemberSS{
Value: record.StringSet(),
}
}
return val
}
There is a package that allows conversion from events.DynamoDBAttributeValue to dynamodb.AttributeValue
https://pkg.go.dev/github.com/aereal/go-dynamodb-attribute-conversions/v2
From there one can unmarshal AttributeValue into struct
func Unmarshal(attribute map[string]events.DynamoDBAttributeValue, out interface{}) error {
av := ddbconversions.AttributeValueMapFrom(attribute)
return attributevalue.UnmarshalMap(av, out)
}

Traversing a Golang Map

I initialized a variable named data like this:
var data interface{}
Then I unmarshalled raw json into.
err = json.Unmarshal(raw, &data)
I've run these two functions on it:
fmt.Println(reflect.TypeOf(data))
fmt.Println(data)
and those return this:
map[string]interface {}
map[tasks:[map[payload:map[key:36A6D454-FEEE-46EB-9D64-A85ABEABBCB7] code_name:image_resize]]]
and I need to access the "key". I have tried these approaches and a few more:
data["tasks"][0]["payload"]["key"]
data[0][0][0][0]
Those have all given me an error similar to this one:
./resize.go:44: invalid operation: data["tasks"] (index of type interface {})
Any advice on how to grab the "key" value out of this interface? Thanks in advance.
Since you already know your schema, the best way to do this is to unmarshal directly into structs you can use.
http://play.golang.org/p/aInZp8IZQA
package main
import (
"encoding/json"
"fmt"
)
type msg struct {
Tasks []task `json:"tasks"`
}
type task struct {
CodeName string `json:"code_name"`
Payload payload `json:"payload"`
}
type payload struct {
Key string `json:"key"`
}
func main() {
var data msg
raw := `{ "tasks": [ { "code_name": "image_resize", "payload": { "key": "36A6D454-FEEE-46EB-9D64-A85ABEABBCB7" } } ] }`
err := json.Unmarshal([]byte(raw), &data)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(data.Tasks[0].Payload.Key)
}
If you insist on doing things the hard way using your original code, you need to do type assertions. I highly recommend avoiding this route when possible. It is not fun. Every step needs to be checked to ensure it matches the data structure you expect.
http://play.golang.org/p/fI5sqKV19J
if m, ok := data.(map[string]interface{}); ok {
if a, ok := m["tasks"].([]interface{}); ok && len(a) > 0 {
if e, ok := a[0].(map[string]interface{}); ok {
if p, ok := e["payload"].(map[string]interface{}); ok {
if k, ok := p["key"].(string); ok {
fmt.Println("The key is:", k)
}
}
}
}
}
In response to Goodwine's question: You can read further about how to marshal and unmarshal by reading the encoding/json godoc. I suggest starting here:
http://golang.org/pkg/encoding/json/#Marshal
http://golang.org/pkg/encoding/json/#Unmarshal
Another solution is to use 3rd-party package like https://github.com/Jeffail/gabs
With gabs, The example above by #stephen-weinberg can be written as:
raw := `{ "tasks": [ { "code_name": "image_resize", "payload": { "key": "36A6D454-FEEE-46EB-9D64-A85ABEABBCB7" } } ] }`
j, _ := gabs.ParseJSON([]byte(raw))
fmt.Println("The key is", j.S("tasks").Index(0).S("payload", "key").Data().(string))
other popular json handling packages I've stumbled upon are: https://github.com/bitly/go-simplejson and https://github.com/antonholmquist/jason.

Resources