Transform a golang map into another structure of map - go

I need help for transforming this input map into the output map. I try with switch/case and for but I didn't succeed it. Thanks a lot !
Input :
Values{
"toto_voiture_brand": Ad{
"CITROEN": "CITROEN",
},
"toto_voiture_model": Ad{
"CITROEN_toto": "C3",
},
"toto_moto_brand": Ad{
"KAWASAKI": "KAWASAKI",
},
"toto_moto_model": Ad{
"KAWASAKI_tata": "Ninja 1000 SX",
},
"toto_camion_brand": Ad{
"RENAULT": "RENAULT",
"PEUGEOT": "PEUGEOT",
},
"toto_camion_model": Ad{
"RENAULT_toto": "J5",
"PEUGEOT_tata": "255",
},
},
}
Output
Values{
"toto_voiture_model": {
"Citroen": {
{Value: "C3"},
},
},
"toto_moto_model": {
"Kawasaki": {
{Value: "Ninja 1000 SX"},
},
},
"toto_camion_model": {
"RENAULT": {
{Value: "J5"},
},
"PEUGEOT": {
{Value: "255"},
},
},
}
I've tried with switch case and loop for and map. But I don't have the result attendee, I didn't found how to match every map, key and value. Thanks a lot

I should have managed what you need with the following code:
package main
import (
"encoding/json"
"fmt"
"strings"
)
type Output struct {
Value string `json:"Value"`
}
func main() {
// declare output
output := make(map[string]map[string]Output, 0)
// input
input := make(map[string]map[string]string, 0)
input["toto_voiture_brand"] = map[string]string{
"CITROEN": "CITROEN",
}
input["toto_voiture_model"] = map[string]string{
"CITROEN_toto": "C3",
}
input["toto_moto_model"] = map[string]string{
"KAWASAKI_tata": "Ninja 1000 SX",
}
input["toto_camion_model"] = map[string]string{
"RENAULT_toto": "J5",
"PEUGEOT_tata": "255",
}
// transformation
for k, v := range input {
if strings.HasSuffix(k, "_model") {
tempMap := make(map[string]Output, len(v))
for kk, vv := range v {
key := strings.Split(kk, "_")[0]
tempMap[key] = Output{
Value: vv,
}
}
output[k] = tempMap
}
}
data, _ := json.MarshalIndent(&output, "", "\t")
fmt.Println(string(data))
}
I put some comments within the code just to separate sections. The first two parts are only supposed to define your input and output variables.
The section starting with // transformation is a good candidate to become a function but I preferred to leave it within the main function for demo purposes. Let me recap what's happens in the loop:
You range over the entries of your input variable
If the key has the suffix _model, you take it into consideration
You define a locally-scoped map (called tempMap) of type map[string]Output with the right number of elements that we're gonna add
You range over the v variable (that's why we're dealing with nested maps)
For each item, you're gonna add an entry to the tempMap
At the end of the nested loop, you add an entry to the parent map (output)
The last part is only for printing a beautiful JSON that can be easily read and checked.
Note that this code is simplified just to show off how to achieve your goal, adjust it before putting it into production.
Let me know if this helps, thanks!

Related

mapstructure json into a structure with nested dictionary

I am new to golang. I have a json file with nested structures that I want to parse and populate.
I am trying to use mapstructure to try and populate. I am able to do it for simple structures. But when it comes to array for dictionaries(key:struct). The map[string]interface{} seems to fail with runtime error: index out of range.
I tried to do the following for the json example below.
type Window struct {
loc []int
wrtc string
label string
}
type View struct {
windows []Window
}
type Views struct {
views []View
}
type Desktop struct {
views []Views `mapstructure:views`
rotation_speed string `mapstructure:"rotationSpeed" json:rotationSpeed"`
}
func main() {
file, _ := ioutil.ReadFile("test.json")
data := Desktop{}
_ = json.Unmarshal([]byte(file), &data)
fmt.Println("data: ", data.views[0])
}
{
"desktop": {
"view": [{// configs for view1
"random_id1": {
"loc": [0,0,640,360],
"wrtc": "some string",
"label": "window 1"
},
"random_id213443": {
"loc": [640,360,1280,720],
"wrtc": "some string blah",
"label": "window 2"
},
// more windows with random ids....
},
{
// configs for view2...
}
],
"rotationSpeed": 30
}
Since the window id is random I am not able to define it in a struct.
I tried using mapstructure:",squash" but that seems to fail as well.
I appreciate any help with this.
#Burak Serdar is right
You don't need mapstructure. JSON unmarshaling can deal with this.
you code have many error, like struct, uppercase, 'views' etc..
follow is a demo:
package main
import (
"encoding/json"
"fmt"
)
var data = `
{
"desktop":{
"view":[
{
"random_id1_1":{
"loc":[
0,
0,
640,
360
],
"wrtc":"some string",
"label":"window 1"
},
"random_id1_2":{
"loc":[
640,
360,
1280,
720
],
"wrtc":"some string blah",
"label":"window 2"
}
},
{
"random_id2_1":{
"loc":[
0,
0,
640,
360
],
"wrtc":"some string",
"label":"window 1"
},
"random_id2_2":{
"loc":[
640,
360,
1280,
720
],
"wrtc":"some string blah",
"label":"window 2"
}
}
],
"rotationSpeed":30
}
}
`
type Window struct {
Loc []int
Wrtc string
Label string
}
type Desktop struct {
View []map[string]Window
Rotation_speed int `json:"rotationSpeed" mapstructure:"rotationSpeed"`
}
type Config struct {
Desktop Desktop
}
func main() {
c := Config{}
json.Unmarshal([]byte(data), &c)
fmt.Println("json.Unmarshal: ", c)
}
json.Unmarshal: {{[map[random_id1_1:{[0 0 640 360] some string window 1} random_id1_2:{[640 360 1280 720] some s
tring blah window 2}] map[random_id2_1:{[0 0 640 360] some string window 1} random_id2_2:{[640 360 1280 720] some
string blah window 2}]] 30}}
also you can use mapstructure by "remain", if you want View struct
type Window struct {
Loc []int
Wrtc string
Label string
}
type View struct {
Windows map[string]Window `mapstructure:",remain"`
}
type Desktop struct {
View []View
Rotation_speed int `json:"rotationSpeed" mapstructure:"rotationSpeed"`
}
type Config struct {
Desktop Desktop
}
func main() {
c2 := Config{}
m := map[string]interface{}{}
_ = json.Unmarshal([]byte(data), &m)
mapstructure.Decode(m, &c2)
fmt.Println("mapstructure: ", c2)
}

How to build an array of structs in Go and gRPC?

I am trying to add an array of strings into the content part of a struct with the Id as the array index. I have the code working for one element, but get various errors when I try and add the loop. Any ideas are welcome. I am using Go.
func buildRequest(s []string) []*storepb.LongStoreRequest {
// ss:= []storepb.LongStoreRequest
// int32 i =0 stringv := s[0]
// for i := 0; i < len(s); i++ {
// println(i, apps[i])
ss := []*storepb.LongStoreRequest{
&storepb.LongStoreRequest {
Msg: &storepb.StoreMessage{
Content: stringv,
Account: "trevor3",
Parent: "parentrec",
Id: 0,
},
},
} // }
return ss
}
If I understand your description correctly, you want to build an array of LongStoreRequests, where each element corresponding to an item in the string array, with Id giving the array index. If that's really what you need, something like this should work:
ss := []*storepb.LongStoreRequest{}
for i,str:=range s {
ss=append(ss,&storepb.LongStoreRequest {
Msg: &storepb.StoreMessage{
Content: str,
Account: "trevor3",
Parent: "parentrec",
Id: i,
}})
}

Remove Element From Struct But Only For This One Function

So I have an Struct that holds data that has a AddedByUser which links to my User Struct.
What I want to be able to do it remove the UserLevel from the AddedByUser
Now I want to be able to do it from this function only, so using the json:"-" is not an option. That would remove it from all json output. I only want to remove it form this one function.
I should also say that these are Gorm models and when I have been trying to remove the 10 option (UserLevels) it only removes the outer data set not the UserLevel from all of the data.
{
"ID": 1,
"CreatedAt": "2019-01-08T16:33:09.514711Z",
"UpdatedAt": "2019-01-08T16:33:09.514711Z",
"DeletedAt": null,
"UUID": "00000000-0000-0000-0000-000000000000",
"Title": "title000",
"Information": "info999",
"EventDate": "2006-01-02T15:04:05Z",
"AddedByUser": {
"ID": 2,
"CreatedAt": "2019-01-08T15:27:52.435397Z",
"UpdatedAt": "2019-01-08T15:27:52.435397Z",
"DeletedAt": null,
"UUID": "b019df80-a7e4-4397-814a-795e7e84b4ca",
"Firstname": "Me",
"Surname": "admin",
"Password": "....",
"Email": "admin#email.co.uk",
"UserLevel": {
"ID": 0,
"CreatedAt": "0001-01-01T00:00:00Z",
"UpdatedAt": "0001-01-01T00:00:00Z",
"DeletedAt": null,
"LevelTitle": "",
"UserLevel": null
},
So this is what I have tried,
data := []models.MyData{}
data = append(data[0:2])
I have about 14 results, with out the append it loads all the results but with this is only loads two results. The idea was to remove either UpdateAt or Title. As I am not sure if the gorm model information is all 0 or if the slice sees them as 0,1,2,3,4 etc.
I have also tried to range over the slice of models, while I can access each of the sections, I can not seem to find a simple method to remove data by name from a struct? Maps seem to have that but not structs which I am not sure why?
Thanks.
UPDATE
This is the model I am using:
//Model
type MyData struct {
gorm.Model
UUID uuid.UUID
Title string
Information string
EventDate time.Time
AddedByUser Users `gorm:"ForeignKey:added_by_user_fk"`
AddedByUserFK uint
}
//Users Model
type Users struct {
gorm.Model
UUID uuid.UUID
Firstname string
Surname string
Password string
Email string
UserLevel UserLevels `gorm:"ForeignKey:user_level_fk" json:",omitempty"`
UserLevelFK uint
}
As mentioned in the comments, you cannot remove fields from a struct value, because that would yield a value of a different type.
However, you can set fields to their zero value. Combined with the omitempty JSON tag, you can exclude fields from the JSON encoding. To make this work properly, you have to change the UserLevel field to a pointer type (otherwise you end up with empty objects in the JSON document).
Types shortened for brevity:
package main
import (
"encoding/json"
"fmt"
)
type MyData struct {
Title string
AddedByUser Users
}
type Users struct {
ID int
UserLevel *UserLevels `json:",omitempty"` // pointer type with omitempty
}
type UserLevels struct {
LevelTitle string
}
func main() {
var x MyData
x.Title = "foo"
x.AddedByUser.ID = 2
x.AddedByUser.UserLevel = &UserLevels{}
f(x)
b, _ := json.MarshalIndent(x, "", " ")
fmt.Println("main:\n" + string(b))
}
func f(x MyData) {
// "unset" UserLevel. Since we are receiving a copy of MyData, this is
// invisible to the caller.
x.AddedByUser.UserLevel = nil
b, _ := json.MarshalIndent(x, "", " ")
fmt.Println("f:\n" + string(b))
}
// Output:
// f:
// {
// "Title": "foo",
// "AddedByUser": {
// "ID": 2
// }
// }
// main:
// {
// "Title": "foo",
// "AddedByUser": {
// "ID": 2,
// "UserLevel": {
// "LevelTitle": ""
// }
// }
// }
Try it on the playground: https://play.golang.org/p/trUgnYamVOA
Alternatively, you can define new types that exclude the AddedByUser field. However, since this field isn't at the top level, this is a lot of work, and it's easy to forget to update those types when new fields are added to the original types.
If the field were at the top level, the compiler would do most of the work for you, because types that only differ in their field tags can be directly converted to one another:
type MyData struct {
ID int
Title string
}
func main() {
var x MyData
x.ID = 1
x.Title = "foo"
f(x)
}
func f(x MyData) {
type data struct { // same as MyData, except the field tags
ID int
Title string `json:"-"`
}
b, _ := json.MarshalIndent(data(x), "", " ")
fmt.Println("main:\n" + string(b))
}

Golang implementing pagination on map[string]interface{} data

I have a json file (nested json) that I am unmarshalling its content into a map[string]interface. Now I have to implement pagination as the data is large. The client side will send as a query parameter the desired page, how can I slice the data I have?
This is a snippet of the data I am dealing with:
"packages":{
"pkg1": {
"meta": {
"description": "description1",
"name": "pkg1.1"
},
"name": "pkg1.1"
},
"pkg2": {
"meta": {
"description": "description2",
"name": "pkg2.2"
},
"name": "pkg2.2"
},
}
So what I did is that I recursively iterated through the data and created an array of a custom type containing the data I need (name, description) for each entry so that I can use it for pagination. Here is the code I used:
type Object struct {
name string
description string
}
func iterate(aMap map[string]interface{}, result *[]Object){
for key, val := range aMap {
switch val.(type) {
case map[string]interface{}:
if(key == "meta"){
switch reflect.TypeOf(val).Kind() {
case reflect.Map:
s := reflect.ValueOf(val)
var tmpData Object
if(s.MapIndex(reflect.ValueOf("name")).IsValid()){
tmpData.name = s.MapIndex(reflect.ValueOf("name")).Interface().(string)
}
if(s.MapIndex(reflect.ValueOf("description")).IsValid()){
tmpData.description = s.MapIndex(reflect.ValueOf("description")).Interface().(string)
}
*result = append(*result, tmpData)
}
}
iterate(val.(map[string]interface{}), result)
default: //DO NOTHING!!
}
}
}
If you're doing pagination, somewhere the data must be represented as a list instead of an object? I assume at some place in your JSON, you have a list of items, otherwise pagination doesn't make sense.
It shouldn't be very hard, something simple like this should work:
const (
itemsPerPage = 10
)
var data []map[string]interface{}
// pages start at 1, can't be 0 or less.
func GetDataPage(page int) []map[string]interface{} {
start := (page - 1) * itemsPerPage
stop := start + itemsPerPage
if start > len(data) {
return nil
}
if stop > len(data) {
stop = len(data)
}
return data[start:stop]
}
You are unmarshalling your json into a map which has no order by itself. In order to be able to paginate your results you need to order them in some way.
One way of doing it is to sort your data and then store it into an array. But in order to paginate you need to have ordered data and that is not possible with a map.

golang got empty slice after initialize

I have 3 struct: Queue,Config,Tasker
type Queue struct {
Name string
Concurrent int
Connections []*redis.Client
}
type Config struct {
Queues []Queue
RedisAddr string
RedisDB int
}
type Tasker struct {
Config Config
}
The problem happend in this method, I initialize queue.Connections in for-loop, but I got zero length of queue.Connections outside the for-loop
func (t *Tasker) StartListening() {
for j := 0; j < len(t.Config.Queues); j++ {
queue := t.Config.Queues[j]
queue.Connections = make([]*redis.Client, queue.Concurrent)
fmt.Println(len(queue.Connections)) //here print correct length, 1 for default queue, 2 for mail queue
}
fmt.Println(len(t.Config.Queues[0].Connections)) //but why here print 0?
}
This is my test code
func main() {
config := Config{
RedisAddr: "10.1.1.59:6379",
RedisDB: 8,
Queues: []Queue{
Queue{Name: "default", Concurrent: 1},
Queue{Name: "mail", Concurrent: 2},
},
}
daemon := Tasker{Config: config}
daemon.StartListening()
}
why fmt.Println(len(t.Config.Queues[0].Connections)) is 0 outside the for-loop?
You are creating a new Queue instead of accessing the one in the Config structure, and this new value prevents modification to the Queue in Config.Queues. Try direct assignment:
// ...
t.Config.Queues[j].Connections = make([]*redis.Client, queue.Concurrent)
// ...
Or if you want to use an auxillary variable, change Config.Queues type to []*Queue:
type Config struct {
Queues []*Queue
RedisAddr string
RedisDB int
}
// ...
config := Config{
RedisAddr: "10.1.1.59:6379",
RedisDB: 8,
Queues: []*Queue{
&Queue{Name: "default", Concurrent: 1},
&Queue{Name: "mail", Concurrent: 2},
},
}
Now your original code should work.

Resources