I'm trying to figure out how to create a slice I can more easily manipulate and use JUST the values from to later iterate over to make a number of API requests. The slice of integers are API IDs. I am successfully making a struct with custom types after making a GET to retrieve the JSON Array of IDs, but I now need to pull only the values from that JSON array and dump them into a slice without the key "id" (which will likely need to change over time in size).
This is my JSON:
{
"data": [
{
"id": 38926
},
{
"id": 38927
}
],
"meta": {
"pagination": {
"total": 163795,
"current_page": 3,
"total_pages": 81898
}
}
}
And I would like this from it:
{38926, 38927}
If you want custom Unmarshaling behavior, you need a custom type with its own json.Unmarshaler e.g.
type ID int
func (i *ID) UnmarshalJSON(data []byte) error {
id := struct {
ID int `json:"id"`
}{}
err := json.Unmarshal(data, &id)
if err != nil {
return err
}
*i = ID(id.ID)
return nil
}
To use this, reference this type in your struct e.g.
type data struct {
IDs []ID `json:"data"`
}
var d data
working example: https://go.dev/play/p/i3MAy85nr4X
Related
Let's say I have the first struct as
type Person struct {
Name string `json:"person_name"`
Age int `json:"person_age"`
Data map[string]interface{} `json:"data"`
}
and I am trying to marshal an array of the above struct
Things work well till here and a sample response I receive is
[
{
"person_name":"name",
"person_age":12,"data":{}
},
{
"person_name":"name2",
"person_age":12,"data":{}
}
]
Now, I need to append another struct over here and the final response should like
[
{
"person_name":"name",
"person_age":12,"data":{}
},
{
"person_name":"name2",
"person_age":12,"data":{}
},
{
"newData":"value"
}
]
So can someone help on this and how i can achieve this ?
I tried by creating an []interface{} and then iterating over person to append each data, but the issue in this approach is that it makes the Data as null if in case it's an empty string.
I would need it be an empty map only.
Let me prefix this by saying this looks to me very much like you might be dealing with an X-Y problem. I can't really think of many valid use-cases where one would end up with a defined data-type that has to somehow be marshalled alongside a completely different, potentially arbitrary/freeform data structure. It's possible, though, and this is how you could do it:
So you just want to append a completely different struct to the data-set, then marshal it and return the result as JSON? You'll need to create a new slice for that:
personData := []Person{} // person 1 and 2 here
more := map[string]string{ // or some other struct
"newdata": "value",
}
allData := make([]any, 0, len(personData) + 1) // the +1 is for the more, set cap to however many objects you need to marshal
for _, p := range personData {
allData = append(allData, p) // copy over to this slice, because []Person is not compatible with []any
}
allData = append(allData, more)
bJSON, err := json.Marshal(allData)
if err != nil {
// handle
}
fmt.Println(string(bJSON))
Essentially, because you're trying to marshal a slice containing multiple different types, you have to add all objects to a slice of type any (short for interface{}) before marshalling it all in one go
Cleaner approaches
There are much, much cleaner approaches that allow you to unmarshal the data, too, assuming the different data-types involved are known beforehand. Consider using a wrapper type like so:
type Person struct {} // the one you have
type NewData {
NewData string `json:"newdata"`
}
type MixedData struct {
*Person
*NewData
}
In this MixedData type, both Person and NewData are embedded, so MixedData will essentially act as a merged version of all embedded types (fields with the same name should be overridden at this level). With this type, you can marshal and unmarshal the JSON accordingly:
allData := []MixedData{
{
Person: &person1,
},
{
Person: &person2,
},
{
NewData: &newData,
},
}
Similarly, when you have a JSON []byte input, you can unmarshal it same as you would any other type:
data := []MixedData{}
if err := json.Unmarshal(&data, in); err != nil {
// handle
}
fmt.Printf("%#v\n", data) // it'll be there
It pays to add some functions/getters to the MixedData type, though:
func (m MixedData) IsPerson() bool { return m.Person != nil }
func (m MixedData) Person() *Person {
if m.Person == nil {
return nil
}
cpy := *m.Person // create a copy to avoid shared pointers
return &cpy // return pointer to the copy
}
Do the same for all embedded types and this works like a charm.
As mentioned before, should your embedded types contain fields with the same name, then you should override them in the MixedData type. Say you have a Person and Address type, and both have an ID field:
type MixedData struct {
ID string `json:"id"`
*Person
*Address
}
This will set the ID value on the MixedData type, and all other (non-shared) fields on the corresponding embedded struct. You can then use the getters to set the ID where needed, or use a custom unmarshaller, but I'll leave that to you to implement
I want to append a value to the returned response of my code, this is what I already have:
publisherShare := 25 // I also have this variable that want to append to the returned response.
c.JSON(http.StatusOK, nf) // nf is a row found and returned from database.
This returns such json:
{
"id": 105324,
"title": "test title",
"last_update": "2021-03-10T12:50:37+03:30",
"created_at": "2021-03-10T12:50:36+03:30",
"updated_at": "2021-05-05T05:46:39.352859604Z"
}
I need to have a json result like this:
{
"id": 105324,
"title": "test title",
"last_update": "2021-03-10T12:50:37+03:30",
"created_at": "2021-03-10T12:50:36+03:30",
"updated_at": "2021-05-05T05:46:39.352859604Z",
"publisher_share": 25 // I want this to be added.
}
This is what I have tried so far, but it changes the schema and will not be backward compatible anymore:
c.Json(http.StatusOK, gin.H{"book": nf, "publisher_share": publisherShare})
but this is not the json result I want. I want publisher_share alongside other fields just like the json result I mentioned above.
You can add publisher share to the same type as
type nfType struct {
// All of the db row fields
PublisherShare int `json:"publisher_share,omitempty" db:"-"`
}
Simply create a struct that matches the structure of your json responce. Key is to add aditional fild to it like
type Responce struct {
// other fields. They have to be written in same way to keep conventions
// (Go-ish name and needed name as tag)
PublisherShare int `json:"publisher_share"`
}
Then when you handle responce you can just unmarshal and marshal json.
responce := Responce{PublisherShare: 25}
err := json.Unmarshal(bytes, &responce)
if err != nil {
//handle error
}
newResponce, err := jsom.Marshal(responce)
if err != nil {
//handle error
}
// newResponce is now in form you want
Define a new type. It embeds the type of the db row, and it has a field for the extra value. E.G: if the type of the db row is DatabaseRow:
type CompleteReturn struct {
DatabaseRow
PublisherShare int `json:"publisher_share"`
}
Later use it as:
complete := CompleteReturn{
DatabaseRow: nf, // Serialize in json as before
PublisherShare: 25, // Serialize in json as publisher_share
}
c.Json(http.StatusOk, complete)
I'm trying to get values/index/key of a JSON but no success. I found some answers and tutorials but I couldn't get them working for me.
I have the following JSON:
{
"id": "3479",
"product": "Camera",
"price": "",
"creation": 04032020,
"products": [
{
"camera": "Nikon",
"available": true,
"freeshipping": false,
"price": "1,813",
"color": "black"
},
{
"camera": "Sony",
"available": true,
"freeshipping": true,
"price": "931",
"color": "black"
}
],
"category": "eletronics",
"type": "camera"
}
I have tried several examples but none worked for this type of Json.
The error I'm getting:
panic: interface conversion: interface {} is nil, not
map[string]interface {}
I believe it's because of "products[]" I tried map[string]interface{} and []interface{} it compiles but gives me that error afterwards.
Could you give me an example on how I can extract these values?
The code I'm using:
//gets the json
product,_:=conn.GetProductData(shop.Info.Id)
// assing a variable
productInformation:=<-product
//prints the json
fmt.Printf(productInformation)
//try to get json values
type Product struct {
Id string
Product string
}
var product Product
json.Unmarshal([]byte(productInformation), &product)
fmt.Printf("Id: %s, Product: %s", product.Id, product.Product)
This code does not panic but it doesn't print all results either so I tried
this one below (which was supposed to give me all results) but it panics
var result map[string]interface{}
json.Unmarshal([]byte(productInformation), &result)
// The object stored in the "birds" key is also stored as
// a map[string]interface{} type, and its type is asserted from
// the interface{} type
products := result["products"].(map[string]interface{})
for key, value := range products {
// Each value is an interface{} type, that is type asserted as a string
fmt.Println(key, value.(string))
}
You need to add json tag to specify the field name in json as it is in lowercase
type Product struct {
Id string `json:"id"`
Product string `json:"product"`
}
And in the second case according to json it is a slice not a map so you need to cast it to []interface{}
var result map[string]interface{}
json.Unmarshal([]byte(productInformation), &result)
// The object stored in the "birds" key is also stored as
// a map[string]interface{} type, and its type is asserted from
// the interface{} type
products := result["products"].([]interface{})
for key, value := range products {
// Each value is an interface{} type, that is type asserted as a string
fmt.Println(key, value)
}
I am getting nested data from mongo and I want to flatten that out in a structure to store it in a csv file.
The data looks like this:
{
"_id" : "bec7bfaa-7a47-4f61-a463-5966a2b5c8ce",
"data" : {
"driver" : {
"etaToStore" : 156
},
"createdAt" : 1532590052,
"_id" : "07703a33-a3c3-4ad5-9e06-d05063474d8c"
}
}
And the structure I want to eventually get should be something like this
type EventStruct struct {
Id string `bson:"_id"`
DataId string `bson:"data._id"`
EtaToStore string `bson:"data.driver.etaToStore"`
CreatedAt int `bson:"data.createdAt"`
}
This doesn't work, so following some SO answers I broke it down into multiple structures:
// Creating a structure for the inner struct that I will receive from the query
type DriverStruct struct {
EtaToStore int `bson:"etaToStore"`
}
type DataStruct struct {
Id string `bson:"_id"`
Driver DriverStruct `bson:"driver"`
CreatedAt int `bson:"createdAt"`
}
// Flattenning out the structure & getting the fields we need only
type EventStruct struct {
Id string `bson:"_id"`
Data DataStruct `bson:"data"`
}
This gets all the data from the Mongo query result but it's nested:
{
"Id": "bec7bfaa-7a47-4f61-a463-5966a2b5c8ce",
"Data": {
"Id": a33-a3c3-4ad5-9e06-d05063474d8c,
"Driver": {
"EtaToStore": 156
},
"CreatedAt": 1532590052
}
}
What I want to end up with is:
{
"Id": "bec7bfaa-7a47-4f61-a463-5966a2b5c8ce",
"DataId": "a33-a3c3-4ad5-9e06-d05063474d8c",
"EtaToStore": 156,
"CreatedAt": 1532590052
}
I'm sure there's an easy way to do this but I can't figure it out, help!
You can implement the json.Unmarshaler interface to add a custom method to unmarshal the json. Then in that method, you can use the nested struct format, but return the flattened one at the end.
func (es *EventStruct) UnmarshalJSON(data []byte) error {
// define private models for the data format
type driverInner struct {
EtaToStore int `bson:"etaToStore"`
}
type dataInner struct {
ID string `bson:"_id" json:"_id"`
Driver driverInner `bson:"driver"`
CreatedAt int `bson:"createdAt"`
}
type nestedEvent struct {
ID string `bson:"_id"`
Data dataInner `bson:"data"`
}
var ne nestedEvent
if err := json.Unmarshal(data, &ne); err != nil {
return err
}
// create the struct in desired format
tmp := &EventStruct{
ID: ne.ID,
DataID: ne.Data.ID,
EtaToStore: ne.Data.Driver.EtaToStore,
CreatedAt: ne.Data.CreatedAt,
}
// reassign the method receiver pointer
// to store the values in the struct
*es = *tmp
return nil
}
Runnable example: https://play.golang.org/p/83VHShfE5rI
This question is a year and a half old, but I ran into it today while reacting to an API update which put me in the same situation, so here's my solution (which, admittedly, I haven't tested with bson, but I'm assuming the json and bson field tag reader implementations handle them the same way)
Embedded (sometimes referred to as anonymous) fields can capture JSON, so you can compose several structs into a compound one which behaves like a single structure.
{
"_id" : "bec7bfaa-7a47-4f61-a463-5966a2b5c8ce",
"data" : {
"driver" : {
"etaToStore" : 156
},
"createdAt" : 1532590052,
"_id" : "07703a33-a3c3-4ad5-9e06-d05063474d8c"
}
}
type DriverStruct struct {
EtaToStore string `bson:"etaToStore"`
type DataStruct struct {
DriverStruct `bson:"driver"`
DataId string `bson:"_id"`
CreatedAt int `bson:"createdAt"`
}
type EventStruct struct {
DataStruct `bson:"data"`
Id string `bson:"_id"`
}
You can access the nested fields of an embedded struct as though the parent struct contained an equivalent field, so e.g. EventStructInstance.EtaToStore is a valid way to get at them.
Benefits:
You don't have to implement the Marshaller or Unmarshaller interfaces, which is a little overkill for this problem
Doesn't require any copying fields between intermediate structs
Handles both marshalling and unmarshalling for free
Read more about embedded fields here.
You can use basically the same logic as:
package utils
// FlattenIntegers flattens nested slices of integers
func FlattenIntegers(slice []interface{}) []int {
var flat []int
for _, element := range slice {
switch element.(type) {
case []interface{}:
flat = append(flat, FlattenIntegers(element.([]interface{}))...)
case []int:
flat = append(flat, element.([]int)...)
case int:
flat = append(flat, element.(int))
}
}
return flat
}
(Source: https://gist.github.com/Ullaakut/cb1305ede48f2391090d57cde355074f)
By adapting it for what's in your JSON. If you want it to be generic, then you'll need to support all of the types it can contain.
I have a code # http://play.golang.org/p/HDlJJ54YqW
I wanted to print the Phone and email of a person.
It can be of multiple entries.
But getting the error undefined.
Can anyone help out.
Thanks.
Small details: you are referencing twice: you give the address of the address of the object to json.Unmarshal. Just give the address.
` allows for multiline, no need to split your json input.
I don't know what you where trying to achieve with u.Details[Phone:"1111"].Email, but this is no Go syntax. your Details member is a slice off Detail. A slice is similar to an array and can be accessed by index.
Also, your json does not match your object structure. If you want to have multiple details in one content, then it needs to be embed in an array ([ ])
You could do something like this: (http://play.golang.org/p/OP1zbPW_wk)
package main
import (
"encoding/json"
"fmt"
)
type Content struct {
Owner string
Details []*Detail
}
type Detail struct {
Phone string
Email string
}
func (c *Content) SearchPhone(phone string) *Detail {
for _, elem := range c.Details {
if elem.Phone == phone {
return elem
}
}
return nil
}
func (c *Content) SearchEmail(email string) *Detail {
for _, elem := range c.Details {
if elem.Email == email {
return elem
}
}
return nil
}
func main() {
encoded := `{
"Owner": "abc",
"Details": [
{
"Phone": "1111",
"Email": "#gmail"
},
{
"Phone": "2222",
"Email": "#yahoo"
}
]
}`
// Decode the json object
u := &Content{}
if err := json.Unmarshal([]byte(encoded), u); err != nil {
panic(err)
}
// Print out Email and Phone
fmt.Printf("Email: %s\n", u.SearchPhone("1111").Email)
fmt.Printf("Phone: %s\n", u.SearchEmail("#yahoo").Phone)
}