This question already has answers here:
Iterate through the fields of a struct in Go
(8 answers)
Closed 5 years ago.
I have a struct representing a dataset that I need to write to a CSV file as a time-series data. This is what I have so far.
type DataFields struct {
Field1 int,
Field2 string,
...
Fieldn int
}
func (d DataFields) String() string {
return fmt.Sprintf("%v,%v,...,%v", Field1, Field2,..., Fieldn)
}
Is there a way I can iterate through the members of the struct and construct a string object using it?
Performance is not really an issue here and I was wondering if there was a way I could generate the string without having to modify the String() function if the structure changed in the future.
EDITED to add my change below:
This is what I ended up with after looking at the answers below.
func (d DataFields) String() string {
v := reflect.ValueOf(d)
var csvString string
for i := 0; i < v.NumField(); i++ {
csvString = fmt.Sprintf("%v%v,", csvString, v.Field(i).Interface())
}
return csvString
}
What you are looking for is called reflection. This answer explains how to use it to loop though a struct and get the values.
This is the example the author uses on the other answer:
package main
import (
"fmt"
"reflect"
)
func main() {
x := struct{Foo string; Bar int }{"foo", 2}
v := reflect.ValueOf(x)
values := make([]interface{}, v.NumField())
for i := 0; i < v.NumField(); i++ {
values[i] = v.Field(i).Interface()
}
fmt.Println(values)
}
You can see it working on the go playground.
One way would be to use the reflect package. There is a Value.Field(int) Value method that might be usefull to you. You would essentially first call ValueOf(interface{}) Value with your DataFields, and then have a simple loop calling Field(int) Value on the Value.
Another approach is to use code generation which would generate a serializer code for you.
The trade-offs are:
Codegen is more compilcated in that it, most of the time, relies
on running an external program (though this could be made simpler by
employing go run as it's supposed to be always available).
Every time you make a change to your data type adding or removing
a field which has to be serialized, you need to run go generate
to regenerate the serializer code.
On the flip side the resulting code is fast and robust,
and the changes to the data type are usually seldom enough.
Reflection is simpler in that it does not require thinking about
regenerating the code.
On the flip side, the code which uses reflect is usually ugly and quite hard to understand. And of course it incurs runtime performance penalty.
Related
I've been struggling with the reflect package. This code below does what I expect:
package main
import (
"reflect"
"log"
)
type Car struct {
Model string
}
type Person struct {
Name string
Cars []Car
}
func ModifyIt(parent interface{},fieldName string, val interface{}) {
slice := reflect.ValueOf(parent).Elem()
nth := slice.Index(0)
//row := nth.Interface() // this line causes errors
row := nth.Interface().(Person)
elem := reflect.ValueOf(&row).Elem()
field := elem.FieldByName(fieldName)
log.Println(field.CanSet())
}
func main() {
p := []Person{Person{Name:"john"}}
c := []Car{Car{"corolla"},Car{"jetta"}}
ModifyIt(&p,"Cars",&c)
}
However, if I replace the line row := nth.Interface().(Person) with row := nth.Interface(), that is I remove the type assertion, then I get the error:
panic: reflect: call of reflect.Value.FieldByName on interface Value
on line "field := elem.FieldByName(fieldName)
I've tried a bunch of other things the last few hours like trying to do reflect.TypeOf(), reflect.Indirect() etc... on some of the other variables but with no success.
I've read some other questions like these:
reflect: call of reflect.Value.FieldByName on ptr Value
Set a struct field with field type of a interface
Golang reflection: Can't set fields of interface wrapping a struct
They seem to suggest that I don't have a good understanding of how pointers or interfaces work.
So my question is, how do I go about setting the field of a struct when the struct is typed as an interface?
UPDATE
I posted a solution as an answer, but I have no confidence in whether it is the proper or safe way of doing things. I hope someone can explain, or post a better solution.
Try this:
func ModifyIt(slice interface{}, fieldName string, newVal interface{}) {
// Create a value for the slice.
v := reflect.ValueOf(slice)
// Get the first element of the slice.
e := v.Index(0)
// Get the field of the slice element that we want to set.
f := e.FieldByName(fieldName)
// Set the value!
f.Set(reflect.ValueOf(newVal))
}
Call it like this:
p := []Person{Person{Name: "john"}}
c := []Car{Car{"corolla"}, Car{"jetta"}}
ModifyIt(p, "Cars", c)
Note that the call passes the slices directly instead of using pointers to slices. The pointers are not needed and add extra complexity.
Run it on the Playground.
Out of sheer luck, I finally got something to work.
I pieced together a bunch of random things I read with very little rhyme or reason. I even tried reading the Laws of Reflection on the Golang site, but I don't think I have a good grasp of how it relates to why I couldn't set variables typed as interface{}. In general, I still don't understand what I did.
My solution below is littered with comments to indicate my confusion, and lack of confidence in whether I did things properly or safely.
package main
import (
"reflect"
"log"
)
type Car struct {
Model string
}
type Person struct {
Name string
Cars []Car
}
func ModifyIt(parent interface{},fieldName string, val interface{}) {
log.Println(parent)
slice := reflect.ValueOf(parent).Elem()
nth := slice.Index(0)
row := nth.Interface()
log.Println(nth.CanSet()) // I can set this nth item
// I think I have a to make a copy, don't fully understand why this is necessary
newitem := reflect.New(reflect.ValueOf(row).Type())
newelem := newitem.Elem()
field := newelem.FieldByName(fieldName)
// I need to copy the values over from the old nth row to this new item
for c:=0; c<nth.NumField(); c++ {
newelem.Field(c).Set(reflect.Indirect(nth.Field(c)))
}
// now I can finally set the field for some reason I don't understand
field.Set(reflect.ValueOf(val).Elem())
// now that newitem has new contents in the field object, I need to overwrite the nth item with new item
// I don't know why I'm doing it, but I'll do it
// I also don't fully understand why I have to use Indirect sometimes, and not other times...it seems interchangeable with ValueOf(something).Elem(), I'm confused....
nth.Set(reflect.Indirect(newitem))
}
func main() {
p := []Person{Person{Name:"john"}}
c := []Car{Car{"corolla"},Car{"jetta"}}
ModifyIt(&p,"Cars",&c)
// now parent is up to date, although I have no idea how I got here.
log.Println(p)
}
If anyone can post a better answer that clears up my confusion, that will be great. I've been having a really hard time learning golang.
This question already has answers here:
Instance new Type (Golang)
(2 answers)
Closed 5 years ago.
I am new to Go and I have this problem. I need to make a kind of "dispatcher" that will receive a string and return a type to be instantiated based on the string. For example:
AnimalType := mymap["animal"]
newAnimal := new(AnimalType)
Is there a way to do so?
Thanks in advance.
You can do this with the reflect package, though it should be noted that eventually you have to know the concrete type to really do much with it.
EDIT: LET IT BE KNOWN. This is a very BAD idea in the first place, and if you are doing this, you should probably rethink things. Go is a statically typed language, and unless you REALLY need to use the reflect package, you should stay away from it if possible. Even then, in most cases, this has already been done for you. Take, for instance, the JSON Marshal/Unmarshaller. At its core, they do some nasty reflection stuff, but it is already taken care of for you, just use it.
It is important to note that the type assertions (the .(*Thing1) lines) will panic if it isn't the right type. See https://tour.golang.org/methods/15
Test on playground: https://play.golang.org/p/DhiTnCVJi1
package main
import (
"fmt"
"reflect"
)
type Thing1 bool
type Thing2 int
type Thing3 struct {
Item string
}
func main() {
m := map[string]reflect.Type{}
var t1 Thing1
var t2 Thing2
var t3 Thing3
m["thing1"] = reflect.TypeOf(t1)
m["thing2"] = reflect.TypeOf(t2)
m["thing3"] = reflect.TypeOf(t3)
// later on
// thing1
newT1Value := reflect.New(m["thing1"])
// you use * here because a pointer to a boolean type isn't useful
newT1 := *newT1Value.Interface().(*Thing1) // cast to concrete type
fmt.Printf("T1: %v\n", newT1)
// thing2
newT2Value := reflect.New(m["thing2"])
// you use * here because a pointer to an int type isn't useful
newT2 := *newT2Value.Interface().(*Thing2)
fmt.Printf("T2: %v\n", newT2)
// thing3
newT3Value := reflect.New(m["thing3"])
// you can choose to use * or not here. Pointers to structs are actually useful
newT3 := newT3Value.Interface().(*Thing3)
newT3.Item = "Hello world"
fmt.Printf("T3: %#v\n", newT3)
}
Is it a good idea to create own type from a slice in Golang?
Example:
type Trip struct {
From string
To string
Length int
}
type Trips []Trip // <-- is this a good idea?
func (trips *Trips) TotalLength() int {
ret := 0
for _, i := range *trips {
ret += i.Length
}
return ret
}
Is it somehow a convention in Golang to create types like Trips in my example? Or it is better to use []Trip in the whole project? Any pros and cons?
There's no convention, as far as I am aware of. It's OK to create a slice type if you really need it. In fact, if you ever want to sort your data, this is pretty much the only way: create a type and define the sort.Interface methods on it.
Also, in your example there is no need to take the address of Trips since slice is already a "fat pointer" of a kind. So you can simplify your method to:
func (trips Trips) TotalLength() (tl int) {
for _, l := range trips {
tl += l.Length
}
return tl
}
If this is what your type is (a slice), it's just fine. It gives you an easy access to underlying elements (and allows for range iteration) while providing additional methods.
Of course you probably should only keep essential set of methods on this type and not bloating it with everything that would take []Trip as an argument. (For example I would suggest having DrawTripsOnTheGlobe(t Trips) rather than having it as a Trips' method.)
To calm your mind there are plenty of such slice-types in standard packages:
http://golang.org/pkg/net/#IP
http://golang.org/pkg/sort/#Float64Slice
http://golang.org/pkg/sort/#IntSlice
http://golang.org/pkg/encoding/json/#RawMessage
This is a variant of the venerable "why is my map printing out of order" question.
I have a (fairly large) number of maps of the form map[MyKey]MyValue, where MyKey and MyValue are (usually) structs. I've got "less" functions for all the key types.
I need to iterate over the maps in order. (Specifically, the order defined by the less function on that type.) Right now, my code looks like this:
type PairKeyValue struct {
MyKey
MyValue
}
type PairKeyValueSlice []Pair
func (ps PairKeyValueSlice) Len() int {
return len(ps)
}
func (ps PairKeyValueSlice) Swap(i,j int) {
ps[i], ps[j] = ps[j], ps[i]
}
func (ps PairKeyValueSlice) Less(i,j int) {
return LessKey(ps[i].MyKey, ps[j].MyKey)
}
func NewPairKeyValueSlice(m map[MyKey]MyValue) (ps PairKeyValueSlice) {
ps = make(PairKeyValueSlice, len(m))
i := 0
for k,v := range m {
ps[i] = PairKeyValue{k,v}
i++
}
sort.Sort(ps)
}
And then, any time I want an in-order iteration, it looks like:
var m map[MyKey]MyValue
m = GetMapFromSomewhereUseful()
for _, kv := range NewPairKeyValueSlice(m) {
key := kv.MyKey
value := kv.MyValue
DoUsefulWork(key, value)
}
And this appears to largely work. The problem is that it is terribly verbose. Particularly since the problem at hand really has very little to do with implmenting ordered maps and is really about the useful work in the loop.
Also, I have several different keys and value types. So, every time I want to iterate over a map in order, I copy/paste all that code and do find/replace MyKey with the new key and MyValue with the new value. Copy/paste on that magnitude is... "smelly". It has already become a hassle, since I've already made a few errors that I had to fix several times.
This technique also has the downside that it requires making a full copy of all the keys and values. That is undesirable, but I don't see a way around it. (I could reduce it to just the keys, but it doesn't change the primary nature of the problem.)
This question is attempting the same thing with strings. This question does it with strings and ints. This question implies that you need to use reflection and will have to have a switch statement that switches on every possible type, including all user-defined types.
But with the people who are puzzled that maps don't iterate deterministically, it seems that there has got to be a better solution to this problem. I'm from an OO background, so I'm probably missing something fundamental.
So, is there a reasonable way to iterate over a map in order?
Update: Editing the question to have more information about the source, in case there's a better solution than this.
I have a lot of things I need to group for output. Each grouping level is in a structure that looks like these:
type ObjTypeTree struct {
Children map[Type]*ObjKindTree
TotalCount uint
}
type ObjKindTree struct {
Children map[Kind]*ObjAreaTree
TotalCount uint
}
type ObjAreaTree struct {
Children map[Area]*ObjAreaTree
TotalCount uint
Objs []*Obj
}
Then, I'd iterate over the children in the ObjTypeTree to print the Type groupings. For each of those, I iterate over the ObjKindTree to print the Kind groupings. The iterations are done with methods on the types, and each kind of type needs a little different way of printing its grouping level. Groups need to be printed in order, which causes the problem.
Don't use a map if key collating is required. Use a B-tree or any other/similar ordered container.
I second jnml's answer. But if you want something shorter than you have and are willing to give up compile time type safety, then my library might work for you. (It's built on top of reflect.) Here's a full working example:
package main
import (
"fmt"
"github.com/BurntSushi/ty/fun"
)
type OrderedKey struct {
L1 rune
L2 rune
}
func (k1 OrderedKey) Less(k2 OrderedKey) bool {
return k1.L1 < k2.L1 || (k1.L1 == k2.L1 && k1.L2 < k2.L2)
}
func main() {
m := map[OrderedKey]string{
OrderedKey{'b', 'a'}: "second",
OrderedKey{'x', 'y'}: "fourth",
OrderedKey{'x', 'x'}: "third",
OrderedKey{'a', 'b'}: "first",
OrderedKey{'x', 'z'}: "fifth",
}
for k, v := range m {
fmt.Printf("(%c, %c): %s\n", k.L1, k.L2, v)
}
fmt.Println("-----------------------------")
keys := fun.QuickSort(OrderedKey.Less, fun.Keys(m)).([]OrderedKey)
for _, k := range keys {
v := m[k]
fmt.Printf("(%c, %c): %s\n", k.L1, k.L2, v)
}
}
Note that such a method will be slower, so if you need performance, this is not a good choice.
This question already has answers here:
Generic variadic argument in Go?
(3 answers)
Closed 8 months ago.
Let's say I want to make the equivalent of the JavaScript Array.splice function in Go, for Slices. I have the following code:
func splice(slice []int, index, amount int, elements ...int) []int {
newslice := make([]int, 0)
for i := 0; i < index; i++ {
newslice = append(newslice, slice[i])
}
for i := index + amount; i < len(slice); i++ {
newslice = append(newslice, slice[i])
}
for _, el := range elements {
newslice = append(newslice, el)
}
return newslice
}
This example will work, but only for arguments of type int. I want to make it generic, and I know that I should give the variadic argument elements the type interface{}, but how do I create a new slice with the type of that interface from inside the function?
In other words, how can I specify the type of the slice dynamically depending on the type of the arguments in the first line of the function, where newslice is created?
Using reflection
If you really want to do generic stuff, reflection is the ultimate answer.
See the MakeSlice documentation
in the reflection package for details on your problem.
You just need to retrieve the type of the incoming slice (using TypeOf(...))
and applying MakeSlice correctly.
Example of using reflection to create a slice:
y := []int{1,2,3}
t := reflect.TypeOf(y)
slice := reflect.MakeSlice(t, 0, 10)
slice = reflect.Append(slice, reflect.ValueOf(2))
fmt.Println(slice.Interface())
Run it here.
Using []interface{}
Another way to work with, is []interface{}, which can store any value
but may lead to runtime panics as you omit compiler type checking completely
(this is a bad thing).
Here is an example for using []interface{}
as storage for arbitrary values. With this you don't need to know the type in
your splice implementation, you just splice and use []interface{} for new slices.
This method has the drawback, that you can't convert some slice to []interface{} easily. You have to copy it manually, as described in posts before.
Conclusion
Regardless of which version you use, you will never get back type safety without
knowing the type and converting it back manually. There's no such thing in Go
which will do that for you. That means, that you'll have something like this
in your code to regain type safety:
x := []int{1,2,3,4}
y := splice(x, ...)
yn := []int(y)
Instead of emulating JavaScript in Go (why ???) I would like to suggest to compose simmilar required operations from the building blocks of SliceTricks.
They are:
Completely type agnostic (think "generics" for free).
Quite probably pretty faster compared to packing/unpacking whatsoever in/from a []interface{}.