I'm trying same my nested struct to binary files.
In future there will be a lots of "Rooms" records so serialized struct in binary file is the best approach I think.
package main
import (
"bytes"
"encoding/binary"
"log"
"time"
)
type House struct {
ID int
Floors int
Rooms []Room
}
type Room struct {
Width int
Height int
Description string
CreatedAt time.Time
}
func main() {
var house House = House{
ID: 1,
Floors: 3,
}
house.Rooms = append(house.Rooms, Room{Width: 20, Height: 30, CreatedAt: time.Now(), Description: "This is description"})
house.Rooms = append(house.Rooms, Room{Width: 14, Height: 21, CreatedAt: time.Now(), Description: "This is other description"})
house.Rooms = append(house.Rooms, Room{Width: 12, Height: 43, CreatedAt: time.Now(), Description: "This is other desc"})
log.Println(house)
buf := new(bytes.Buffer)
err := binary.Write(buf, binary.LittleEndian, house)
if err != nil {
log.Println(err)
}
}
But I have error:
- Binary.Write: invalid type main.House
Could someone help me because I cant find solution.
According to the binary.Write documentation:
Data must be a fixed-size value or a slice of fixed-size values, or a pointer to such data.
Your House structure is not a fixed size value.
You might consider writing/reading House and Room separately. The House you use to store house structs must not contain a slice, so you can declare another House struct that you use to read/write from your file.
Instead of a binary file you can store your objects as JSON, and then you wouldn't need to deal with this problem.
Related
I am trying to create the new API with Go and Fiber framework. I am using MongoDB.
Please refer to my below structure
type Product struct {
Name string `json:"name"`
Code string `json:"code"`
Description string `json:"description"`
Brand string `json:"brand"`
Variants []map[string]string `json:"variants"`
Categories []int `json:"categories"`
}
The Variants field will have another set array of objects. But the fields will be dynamic. So I can't define the structure. Now this one I solved with []map[string][string].
My task here I have to form the CSV file with the below format.
[{
Name: "Rock",
Code: "RRR"
Description: "This is rock"
Brand: "R"
...variants
},
{
Name: "Rock",
Code: "RRR"
Description: "This is rock"
Brand: "R"
...variants
},
{
Name: "Rambo",
Code: "RAM"
Description: "This is Rambo"
Brand: "RA"
...variants
},
{
Name: "Rambo",
Code: "RAM"
Description: "This is Rambo"
Brand: "RA"
...variants
}]
Only variants will change for every row. Others will be remaining same
I formed the other fields except for the Variants. I can't create the struct because the data will be dynamic.
I have a lot of confusion
How to create the struct for dynamic fields
How to spread the Variants at the root level
I am from javascript. So, we used the spread operator. Here I am confused about how to do that in Golang.
Help me out, guys
#mkopriva is right
However you can use reflection to dynamically construct a new struct if you like
follow is a example:
package main
import (
"encoding/json"
"fmt"
"reflect"
)
type DynamicMap map[string]interface{}
type Product struct {
Name string `json:"name"`
Code string `json:"code"`
Description string `json:"description"`
Brand string `json:"brand"`
Variants map[string]string `json:"variants"`
Categories []int `json:"categories"`
}
func (j Product) MarshalJSON() ([]byte, error) {
m := DynamicMap{}
t := reflect.TypeOf(j)
v := reflect.ValueOf(j)
for i := 0; i < t.NumField(); i++ {
if t.Field(i).Tag.Get("json") == "variants" {
dyn := v.Field(i).Interface().(map[string]string)
for key, val := range dyn {
m[key] = val
}
} else if t.Field(i).Type.Kind() == reflect.String {
m[t.Field(i).Tag.Get("json")] = v.Field(i).String()
} else {
m[t.Field(i).Tag.Get("json")] = v.Field(i)
}
}
return json.Marshal(m)
}
func (j Product) UnmarshalJSON(data []byte) error {
fmt.Println("Unmarshal...")
return json.Unmarshal(data, &j)
}
func main() {
p := Product{Name: "aa", Variants: map[string]string{"a": "a", "b": "b"}}
marshal, err := json.Marshal(p)
if err != nil {
fmt.Printf("%v\n", err)
}
fmt.Printf("%s\n", marshal)
}
{"a":"a","b":"b","brand":"","categories":{},"code":"","description":"","name":"aa"}
I understand that usage of unsafe package of golang is unsafe, but I'm doing it only for education purposes.
The idea is to copy a slice field of structure, copy it with reflect package and unsafe.pointer, modify and replace the original slice with new one.
And it looks like new slice is created and has correct capacity/length, and it contains an instance of GoodForSale type. But all fields of that instance (Name, Price and Qnty) have wrong values. So I suppose that I'm doing something wrong with pointers and getting garbade data:
package main
import (
"fmt"
"reflect"
"unsafe"
)
type GoodForSale struct {
Name string
Price int
Qnty int
}
type Lead struct {
ID int
Name string
ContactName string
Budget int
Items []GoodForSale
DateTime string
Status int
Win bool
}
func main() {
lead := &Lead{
ID: 41,
Name: "Phone",
ContactName: "John Smith",
Budget: 30000,
Items: []GoodForSale{
{
Name: "Iphone 6",
Price: 100,
Qnty: 3,
},
},
DateTime: "12.08.2020 11:23:10",
Status: 150,
Win: false,
}
//Change Items
pt1 := unsafe.Pointer(&lead.Items)
copyItems := &reflect.SliceHeader{
Data: uintptr(pt1),
Cap: cap(lead.Items),
Len: len(lead.Items),
}
items := *(*[]GoodForSale)(unsafe.Pointer(copyItems))
fmt.Println(items[0].Name)
}
It looks like I'm missing something about how pointers work here. But how can I make this idea work correct?
Here is a go playground url: https://play.golang.org/p/SKtJWe9RVEU
The problem is here:
pt1 := unsafe.Pointer(&lead.Items) // pointer the slice, a.k.a "slice header"
copyItems := &reflect.SliceHeader{
Data: uintptr(pt1), // trying to use it as pointer to the actual data.
"Data" needs a pointer to where the actual data starts, but you're giving a pointer to the slice itself.
This is the correct way to get a pointer to the slice's data:
pt1 := unsafe.Pointer(&lead.Items[0]) // pointer to the first element referenced by the slice
Note that this will panic if len(lead.Items) == 0. In that case, you should use unsafe.Pointer(nil).
You could also get the data pointer (along with len and cap) from the original slice by casting it as a reflect.SliceHeader like this:
copyHeader := *(*reflect.SliceHeader)(unsafe.Pointer(&lead.Items))
items := *(*[]GoodForSale)(unsafe.Pointer(©Header))
but at this point we've essentially just made a convoluted and "unsafe" version of this:
items := lead.Items
Based on the inputs from Hymns For Disco modifying your code by providing the element of slice instead of whole slice to the pointer.
package main
import (
"fmt"
"reflect"
"unsafe"
)
type GoodForSale struct {
Name string
Price int
Qnty int
}
type Lead struct {
ID int
Name string
ContactName string
Budget int
Items []GoodForSale
DateTime string
Status int
Win bool
}
func main() {
lead := &Lead{
ID: 41,
Name: "Phone",
ContactName: "John Smith",
Budget: 30000,
Items: []GoodForSale{
{
Name: "Iphone 6",
Price: 100,
Qnty: 3,
},
},
DateTime: "12.08.2020 11:23:10",
Status: 150,
Win: false,
}
pt1 := unsafe.Pointer(&lead.Items[0])
copyHeader := *(*reflect.SliceHeader)(unsafe.Pointer(&lead.Items))
items := *(*[]GoodForSale)(unsafe.Pointer(©Header))
//items := *(*[]GoodForSale)(unsafe.Pointer(copyItems))
fmt.Println(pt1)
fmt.Println(items[0].Name)
}
Output:
0xc00000c080
Iphone 6
I'm getting the following output when running https://golangci-lint.run/:
rangeValCopy: each iteration copies 128 bytes (consider pointers or indexing) (gocritic)
for _, v := range products {
Here is a cut down version of the code I am running:
package main
import (
"fmt"
"encoding/json"
)
type Application struct {
ProductData []ProductDatum
}
type ProductDatum struct {
Name string
ProductBrand string
ProductType string
}
type Item struct {
ProductBrand string
ProductName string
ProductType string
}
func main() {
appl := Application{
ProductData: []ProductDatum{
{
Name: "Baz",
ProductBrand: "Foo",
ProductType: "Bar",
},
},
}
products := appl.ProductData
var orderLinesItem []Item
for _, v := range products {
item := []Item{
{
ProductBrand: v.ProductBrand,
ProductName: v.Name,
ProductType: v.ProductType,
},
}
orderLinesItem = append(orderLinesItem, item...)
}
body, _ := json.Marshal(orderLinesItem)
fmt.Println(string(body))
}
Here it is in go playground.
What does this output mean and how can I do what it's asking? I tried to use a pointer on each item but that didn't seem to make a difference.
What the linter is trying to tell you is that, by using range the way you are using it, each time you get a new element v it is not returning an element directly from the collection, rather it's a new copy of that element. The linter suggest two approaches:
Either change the slice to be a slice of pointers to struct, that way each iteration of the for loop will get a reference to an element instead of a full struct copy
var products []*ProductDatum
//fill products slice
var orderLinesItem []Item
for _, v := range products{
//here v is a pointer instead of a full copy of a struct.
//Go dereferences the pointer automatically therefore you don't have to use *v
item := []Item{
{
ProductBrand: v.ProductBrand,
ProductName: v.Name,
ProductType: v.ProductType,
},
}
}
The other suggestion from the linter is using the index value that the range returns on each iteration
for i := range products{
item := []Item{
{
//access elements by index directly
ProductBrand: products[i].ProductBrand,
ProductName: products[i].Name,
ProductType: products[i].ProductType,
},
}
}
I saw several questions asking on how to merge unique structs and how to merge identical structs.
But how would I merge structs that have some overlap? and which fields get taken & when?
e.g.:
type structOne struct {
id string `json:id`
date string `json:date`
desc string `json:desc`
}
and
type structTwo struct {
id string `json:id`
date string `json:date`
name string `json:name`
}
how would I merge it such that I get
{
id string `json:id`
date string `json:date`
desc string `json:desc`
name string `json:name`
}
also, what happens if in this case the two id's are the same (assuming a join over id's) but the names are different?
In javascript, doing something like Object.assign(structOne, structTwo).
Go is a strongly typed language, unlike javascript you can't merge two struct into one combined struct because all type are determined at compile-time. You have two solution here :
Using embedded struct:
One great solution is to use embedded struct because you don't have to merge anything anymore.
package main
import (
"bytes"
"encoding/json"
"fmt"
"log"
)
// Shared field
type common struct {
ID string `json:id`
Date string `json:date`
}
type merged struct {
// Common field is embedded
common
Name string `json:name`
Desc string `json:desc`
}
func main() {
buf := bytes.Buffer{}
buf.WriteString("{ \"id\": \"1\", \"date\": \"27/07/2020\", \"desc\": \"the decription...\" }")
merged := &merged{}
err := json.Unmarshal(buf.Bytes(), merged)
if err != nil {
log.Fatal(err)
}
// Look how you can easily access field from
// embedded struct
fmt.Println("ID:", merged.ID)
fmt.Println("Date:", merged.Date)
fmt.Println("Name:", merged.Name)
fmt.Println("Desc:", merged.Desc)
// Output:
// ID: 1
// Date: 27/07/2020
// Name:
// Desc: the decription...
}
If you want to read more about struct embedding:
golangbyexample.com
travix.io
Using Maps
Another solution is to use maps but you will loose the benefits of struct and methods. This example is not the simplest but there is some great example in the other responses.
In this example I'm using Mergo. Mergo is library that can merge structs and map. Here it is used for creating maps object in the Map methods but you can totally write your own methods.
package main
import (
"fmt"
"log"
"github.com/imdario/mergo"
)
type tOne struct {
ID string
Date string
Desc string
}
// Map build a map object from the struct tOne
func (t1 tOne) Map() map[string]interface{} {
m := make(map[string]interface{}, 3)
if err := mergo.Map(&m, t1); err != nil {
log.Fatal(err)
}
return m
}
type tTwo struct {
ID string
Date string
Name string
}
// Map build a map object from the struct tTwo
func (t2 tTwo) Map() map[string]interface{} {
m := make(map[string]interface{}, 3)
if err := mergo.Map(&m, t2); err != nil {
log.Fatal(err)
}
return m
}
func main() {
dst := tOne{
ID: "destination",
Date: "26/07/2020",
Desc: "destination object",
}.Map()
src := tTwo{
ID: "src",
Date: "26/07/1010",
Name: "source name",
}.Map()
if err := mergo.Merge(&dst, src); err != nil {
log.Fatal(err)
}
fmt.Printf("Destination:\n%+v", dst)
// Output:
// Destination:
// map[date:26/07/2020 desc:destination object iD:destination name:object name
}
Go structs and JavaScript objects are very different. Go structs do not have dynamic fields.
If you want dynamic key/value sets that you can easily iterate over and merge, and are very JSON friendly, why not a map[string]interface{}?
$ go run t.go
map[a:1 b:4]
map[a:1 b:4 c:3]
$ cat t.go
package main
import(
"fmt"
)
type MyObj map[string]interface{}
func (mo MyObj)Merge(omo MyObj){
for k, v := range omo {
mo[k] = v
}
}
func main() {
a := MyObj{"a": 1, "b": 4}
b := MyObj{"b": 2, "c": 3}
b.Merge(a)
fmt.Printf("%+v\n%+v\n", a, b)
}
you can use github.com/fatih/structs to convert your struct to map. Then iterate over that map and choose which fields need to copy over. I have a snippet of code which illustrates this solution.
func MergeStruct (a structOne,b structTwo) map[string]interface{}{
a1:=structs.Map(a)
b1:=structs.Map(b)
/* values of structTwo over writes values of structOne */
var myMap=make(map[string]interface{})
for val,key:=range(a1){
myMap[key]=val
}
for val,key:=range(b1){
myMap[key]=val
}
return myMap
}
You can use the reflect package to do this. Try iterating through the two structs and then you can either use another struct type to store the values or maybe use a map.
Check out this question to find out how you can iterate over a struct.
Check out this question to find out how to get the name of the fields.
Remember to use exported field names for the reflect package to work.
Here is an example which works.
For example, I have a struct that I take from the outside. I do not know the struct in fields and field values. I want to copy and use the same struct.
With reflection I find the fields and types in it. But how do I create this struct in the runtime?
Edit : I just want to create a struct in the same name as the runtime. Imagine I do not know my person type. I just want to create the same struct by reflection with interface.
I only know one interface. Person struct I just created it for instance. When a person creates a struct and sends it out, I will create it. instead of person, customer, student etc. You can send.
consider the following code as a 3rd party library.
package main
import(
"fmt"
"reflect"
)
type Person struct {
Id int
Name string
Surname string
}
func main(){
person := NewSomething()
newPerson := typeReflection(person)
ChangePerson(newPerson)
fmt.Println("Success")
}
func typeReflection(_person interface{}){
val := reflect.ValueOf(_person)
//How to create same struct
}
The github.com/mitchellh/copystructure library handles this operation, which is known as a deep copy. After performing a deep copy, the original and the copy contain the same data but modifications to either one do not affect the other.
package main
import (
"fmt"
"github.com/mitchellh/copystructure"
)
type Person struct {
Id int
Name string
Surname string
}
func main() {
original := Person{Id: 0, Name: "n", Surname: "s"}
copy := deepCopy(original)
// Change fields of the original Person.
original.Id = 9
fmt.Printf("original: %#v\n", original)
// The copy of the Person has not change, still has Id:0.
fmt.Printf("copy: %#v\n", copy)
}
func deepCopy(original interface{}) interface{} {
copy, err := copystructure.Copy(original)
if err != nil {
panic(err)
}
return copy
}