Recursive Struct Reflect in Golang - go

I have a nested structure definition flatened into a slice (this hypothesis is not negociable, I have to deal with it) :
type element struct {
Name string
Type string // can be basic type string (eg "uint8")
// or a definition.ID for nested struct
}
type definition struct {
ID string
Elements []element
}
type definitions []definition
allDefs := definitions{
{
ID: "root",
Elements: []element{
{
Name: "SubStruct",
Type: "sub1", // this is reference to another definition
},
{
Name: "VarU32",
Type: "uint32", // this is a basic type string representation
},
},
},
{
ID: "sub1",
Elements: []element{
{
Name: "VarU8",
Type: "uint8",
},
{
Name: "VarU16",
Type: "uint16",
},
},
},
}
And I would like to build the corresponding nested struct using a recursive method (don't know the real depth) :
func (defs definitions) getStruct(id string) interface{} {
for _, def := range defs {
if def.ID == id { // found a reference to definition
fields := []reflect.StructField{}
for _, e := range def.Elements {
s := defs.getStruct(e.Type)
fields = append(fields, reflect.StructField{
Name: e.Name,
Type: reflect.TypeOf(s),
Tag: "",
})
}
return reflect.New(reflect.StructOf(fields)).Interface()
}
}
// didn't found a reference to a definition, it should be a basic type
if id == "uint8" {
return reflect.ValueOf(uint8(0)).Interface()
}
if id == "uint16" {
return reflect.ValueOf(uint16(0)).Interface()
}
if id == "uint32" {
return reflect.ValueOf(uint32(0)).Interface()
}
// ignore the fact id could be anything else for this example
return nil
}
root := allDefs.getStruct("root")
// using spew to inspect variable : github.com/davecgh/go-spew/spew
spew.Dump(root)
// (*struct { SubStruct *struct { VarU8 uint8; VarU16 uint16 }; VarU32 uint32 })(0xc00004e400)({
// SubStruct: (*struct { VarU8 uint8; VarU16 uint16 })(<nil>),
// VarU32: (uint32) 0
// })
Then I want to be able to assign some variable's values :
rootValElem := reflect.ValueOf(root).Elem()
rootValElem.FieldByName("VarU32").SetUint(1)
spew.Dump(root)
// (*struct { SubStruct *struct { VarU8 uint8; VarU16 uint16 }; VarU32 uint32 })(0xc00004e400)({
// SubStruct: (*struct { VarU8 uint8; VarU16 uint16 })(<nil>),
// VarU32: (uint32) 1
// })
Setting root level variable value is OK, but I am unable to entering Sub level and assign any variable, no matter how I play with reflect ValueOf()/Elem()/Addr()..., here are some examples :
fieldSub := rootValElem.FieldByName("SubStruct")
// fieldSub.Kind() : ptr
// fieldSub.CanSet() : true
subVal := reflect.ValueOf(fieldSub)
// subVal.Kind() : struct
// subVal.CanSet() : false
fieldU16 := subVal.FieldByName("VarU16")
// fieldU16.Kind() : invalid
// fieldU16.CanSet() : false
fieldSubElem := fieldSub.Elem()
// fieldSubElem.Kind() : invalid
// fieldSubElem.CanSet() : false
fieldU16 := fieldSubElem.FieldByName("VarU16")
// panic: reflect: call of reflect.Value.FieldByName on zero Value

Thanks to mkopriva comment above, I understand now my mistake : fieldSub is a pointer and I should check if nil, then allocate the struct value before trying to get Elem() then Field :
fieldSub := rootValElem.FieldByName("SubStruct")
// fieldSub.Kind() : ptr
// fieldSub.CanSet() : true
// fieldSub.IsNil() : true
if fieldSub.IsNil() && fieldSub.CanSet() {
fieldSub.Set(reflect.New(fieldSub.Type().Elem()))
}
fieldU8 := fieldSub.Elem().FieldByName("VarU8")
// fieldSub.Kind() : uint8
// fieldSub.CanSet() : true
if fieldU8.CanSet() {
fieldU8.SetUint(8)
}
spew.Dump(root)
// (*struct { SubStruct *struct { VarU8 uint8; VarU16 uint16 }; VarU32 uint32 })(0xc00008a3f0)({
// SubStruct: (*struct { VarU8 uint8; VarU16 uint16 })(0xc0000ac430)({
// VarU8: (uint8) 8,
// VarU16: (uint16) 0
// }),
// VarU32: (uint32) 1
// })

Related

mapstructure json into a structure with nested dictionary

I am new to golang. I have a json file with nested structures that I want to parse and populate.
I am trying to use mapstructure to try and populate. I am able to do it for simple structures. But when it comes to array for dictionaries(key:struct). The map[string]interface{} seems to fail with runtime error: index out of range.
I tried to do the following for the json example below.
type Window struct {
loc []int
wrtc string
label string
}
type View struct {
windows []Window
}
type Views struct {
views []View
}
type Desktop struct {
views []Views `mapstructure:views`
rotation_speed string `mapstructure:"rotationSpeed" json:rotationSpeed"`
}
func main() {
file, _ := ioutil.ReadFile("test.json")
data := Desktop{}
_ = json.Unmarshal([]byte(file), &data)
fmt.Println("data: ", data.views[0])
}
{
"desktop": {
"view": [{// configs for view1
"random_id1": {
"loc": [0,0,640,360],
"wrtc": "some string",
"label": "window 1"
},
"random_id213443": {
"loc": [640,360,1280,720],
"wrtc": "some string blah",
"label": "window 2"
},
// more windows with random ids....
},
{
// configs for view2...
}
],
"rotationSpeed": 30
}
Since the window id is random I am not able to define it in a struct.
I tried using mapstructure:",squash" but that seems to fail as well.
I appreciate any help with this.
#Burak Serdar is right
You don't need mapstructure. JSON unmarshaling can deal with this.
you code have many error, like struct, uppercase, 'views' etc..
follow is a demo:
package main
import (
"encoding/json"
"fmt"
)
var data = `
{
"desktop":{
"view":[
{
"random_id1_1":{
"loc":[
0,
0,
640,
360
],
"wrtc":"some string",
"label":"window 1"
},
"random_id1_2":{
"loc":[
640,
360,
1280,
720
],
"wrtc":"some string blah",
"label":"window 2"
}
},
{
"random_id2_1":{
"loc":[
0,
0,
640,
360
],
"wrtc":"some string",
"label":"window 1"
},
"random_id2_2":{
"loc":[
640,
360,
1280,
720
],
"wrtc":"some string blah",
"label":"window 2"
}
}
],
"rotationSpeed":30
}
}
`
type Window struct {
Loc []int
Wrtc string
Label string
}
type Desktop struct {
View []map[string]Window
Rotation_speed int `json:"rotationSpeed" mapstructure:"rotationSpeed"`
}
type Config struct {
Desktop Desktop
}
func main() {
c := Config{}
json.Unmarshal([]byte(data), &c)
fmt.Println("json.Unmarshal: ", c)
}
json.Unmarshal: {{[map[random_id1_1:{[0 0 640 360] some string window 1} random_id1_2:{[640 360 1280 720] some s
tring blah window 2}] map[random_id2_1:{[0 0 640 360] some string window 1} random_id2_2:{[640 360 1280 720] some
string blah window 2}]] 30}}
also you can use mapstructure by "remain", if you want View struct
type Window struct {
Loc []int
Wrtc string
Label string
}
type View struct {
Windows map[string]Window `mapstructure:",remain"`
}
type Desktop struct {
View []View
Rotation_speed int `json:"rotationSpeed" mapstructure:"rotationSpeed"`
}
type Config struct {
Desktop Desktop
}
func main() {
c2 := Config{}
m := map[string]interface{}{}
_ = json.Unmarshal([]byte(data), &m)
mapstructure.Decode(m, &c2)
fmt.Println("mapstructure: ", c2)
}

How to build an array of structs in Go and gRPC?

I am trying to add an array of strings into the content part of a struct with the Id as the array index. I have the code working for one element, but get various errors when I try and add the loop. Any ideas are welcome. I am using Go.
func buildRequest(s []string) []*storepb.LongStoreRequest {
// ss:= []storepb.LongStoreRequest
// int32 i =0 stringv := s[0]
// for i := 0; i < len(s); i++ {
// println(i, apps[i])
ss := []*storepb.LongStoreRequest{
&storepb.LongStoreRequest {
Msg: &storepb.StoreMessage{
Content: stringv,
Account: "trevor3",
Parent: "parentrec",
Id: 0,
},
},
} // }
return ss
}
If I understand your description correctly, you want to build an array of LongStoreRequests, where each element corresponding to an item in the string array, with Id giving the array index. If that's really what you need, something like this should work:
ss := []*storepb.LongStoreRequest{}
for i,str:=range s {
ss=append(ss,&storepb.LongStoreRequest {
Msg: &storepb.StoreMessage{
Content: str,
Account: "trevor3",
Parent: "parentrec",
Id: i,
}})
}

Is there a way to directly access a field value in an enum struct without pattern matching?

I wish that enums in Rust can be used like Haskell's productive type. I want to
access a field's value directly
assign a field's value directly or make a clone with the changing value.
Directly means that not using too long pattern matching code, but just could access like let a_size = a.size.
In Haskell:
data TypeAB = A {size::Int, name::String} | B {size::Int, switch::Bool} deriving Show
main = do
let a = A 1 "abc"
let b = B 1 True
print (size a) -- could access a field's value directly
print (name a) -- could access a field's value directly
print (switch b) -- could access a field's value directly
let aa = a{size=2} -- could make a clone directly with the changing value
print aa
I tried two styles of Rust enum definition like
Style A:
#[derive(Debug)]
enum EntryType {
A(TypeA),
B(TypeB),
}
#[derive(Debug)]
struct TypeA {
size: u32,
name: String,
}
#[derive(Debug)]
struct TypeB {
size: u32,
switch: bool,
}
fn main() {
let mut ta = TypeA {
size: 3,
name: "TAB".to_string(),
};
println!("{:?}", &ta);
ta.size = 2;
ta.name = "TCD".to_string();
println!("{:?}", &ta);
let mut ea = EntryType::A(TypeA {
size: 1,
name: "abc".to_string(),
});
let mut eb = EntryType::B(TypeB {
size: 1,
switch: true,
});
let vec_ab = vec![&ea, &eb];
println!("{:?}", &ea);
println!("{:?}", &eb);
println!("{:?}", &vec_ab);
// Want to do like `ta.size = 2` for ea
// Want to do like `ta.name = "bcd".to_string()` for ea
// Want to do like `tb.switch = false` for eb
// ????
println!("{:?}", &ea);
println!("{:?}", &eb);
println!("{:?}", &vec_ab);
}
Style B:
#[derive(Debug)]
enum TypeCD {
TypeC { size: u32, name: String },
TypeD { size: u32, switch: bool },
}
fn main() {
// NOTE: Rust requires representative struct name before each constructor
// TODO: Check constructor name can be duplicated
let mut c = TypeCD::TypeC {
size: 1,
name: "abc".to_string(),
};
let mut d = TypeCD::TypeD {
size: 1,
switch: true,
};
let vec_cd = vec![&c, &d];
println!("{:?}", &c);
println!("{:?}", &d);
println!("{:?}", &vec_cd);
// Can't access a field's value like
// let c_size = c.size
let c_size = c.size; // [ERROR]: No field `size` on `TypeCD`
let c_name = c.name; // [ERROR]: No field `name` on `TypeCD`
let d_switch = d.switch; // [ERROR]: No field `switch` on `TypeCD`
// Can't change a field's value like
// c.size = 2;
// c.name = "cde".to_string();
// d.switch = false;
println!("{:?}", &c);
println!("{:?}", &d);
println!("{:?}", &vec_cd);
}
I couldn't access/assign values directly in any style. Do I have to implement functions or a trait just to access a field's value? Is there some way of deriving things to help this situation?
What about style C:
#[derive(Debug)]
enum Color {
Green { name: String },
Blue { switch: bool },
}
#[derive(Debug)]
struct Something {
size: u32,
color: Color,
}
fn main() {
let c = Something {
size: 1,
color: Color::Green {
name: "green".to_string(),
},
};
let d = Something {
size: 2,
color: Color::Blue { switch: true },
};
let vec_cd = vec![&c, &d];
println!("{:?}", &c);
println!("{:?}", &d);
println!("{:?}", &vec_cd);
let _ = c.size;
}
If all variant have something in common, why separate them?
Of course, I need to access not common field too.
This would imply that Rust should define what to do when the actual type at runtime doesn't contain the field you required. So, I don't think Rust would add this one day.
You could do it yourself. It will require some lines of code, but that matches the behavior of your Haskell code. However, I don't think this is the best thing to do. Haskell is Haskell, I think you should code in Rust and not try to code Haskell by using Rust. That a general rule, some feature of Rust come directly from Haskell, but what you want here is very odd in my opinion for Rust code.
#[derive(Debug)]
enum Something {
A { size: u32, name: String },
B { size: u32, switch: bool },
}
impl Something {
fn size(&self) -> u32 {
match self {
Something::A { size, .. } => *size,
Something::B { size, .. } => *size,
}
}
fn name(&self) -> &String {
match self {
Something::A { name, .. } => name,
Something::B { .. } => panic!("Something::B doesn't have name field"),
}
}
fn switch(&self) -> bool {
match self {
Something::A { .. } => panic!("Something::A doesn't have switch field"),
Something::B { switch, .. } => *switch,
}
}
fn new_size(&self, size: u32) -> Something {
match self {
Something::A { name, .. } => Something::A {
size,
name: name.clone(),
},
Something::B { switch, .. } => Something::B {
size,
switch: *switch,
},
}
}
// etc...
}
fn main() {
let a = Something::A {
size: 1,
name: "Rust is not haskell".to_string(),
};
println!("{:?}", a.size());
println!("{:?}", a.name());
let b = Something::B {
size: 1,
switch: true,
};
println!("{:?}", b.switch());
let aa = a.new_size(2);
println!("{:?}", aa);
}
I think there is currently no built-in way of accessing size directly on the enum type. Until then, enum_dispatch or a macro-based solution may help you.

golang got empty slice after initialize

I have 3 struct: Queue,Config,Tasker
type Queue struct {
Name string
Concurrent int
Connections []*redis.Client
}
type Config struct {
Queues []Queue
RedisAddr string
RedisDB int
}
type Tasker struct {
Config Config
}
The problem happend in this method, I initialize queue.Connections in for-loop, but I got zero length of queue.Connections outside the for-loop
func (t *Tasker) StartListening() {
for j := 0; j < len(t.Config.Queues); j++ {
queue := t.Config.Queues[j]
queue.Connections = make([]*redis.Client, queue.Concurrent)
fmt.Println(len(queue.Connections)) //here print correct length, 1 for default queue, 2 for mail queue
}
fmt.Println(len(t.Config.Queues[0].Connections)) //but why here print 0?
}
This is my test code
func main() {
config := Config{
RedisAddr: "10.1.1.59:6379",
RedisDB: 8,
Queues: []Queue{
Queue{Name: "default", Concurrent: 1},
Queue{Name: "mail", Concurrent: 2},
},
}
daemon := Tasker{Config: config}
daemon.StartListening()
}
why fmt.Println(len(t.Config.Queues[0].Connections)) is 0 outside the for-loop?
You are creating a new Queue instead of accessing the one in the Config structure, and this new value prevents modification to the Queue in Config.Queues. Try direct assignment:
// ...
t.Config.Queues[j].Connections = make([]*redis.Client, queue.Concurrent)
// ...
Or if you want to use an auxillary variable, change Config.Queues type to []*Queue:
type Config struct {
Queues []*Queue
RedisAddr string
RedisDB int
}
// ...
config := Config{
RedisAddr: "10.1.1.59:6379",
RedisDB: 8,
Queues: []*Queue{
&Queue{Name: "default", Concurrent: 1},
&Queue{Name: "mail", Concurrent: 2},
},
}
Now your original code should work.

Swift 3 reading from network socket using 'withUnsafeBufferPointer' - 'init' unavailable use 'withMemoryRebound'

I have a function in Swift 2.3 which reads Int value from a network socket
open func readInt() -> Int32 {
var intValueBuffer = [UInt8](repeating: 0, count: MemoryLayout<Int32>.size)
self.stream!.read (&intValueBuffer, maxLength: MemoryLayout<Int32>.size)
let intValue = intValueBuffer.withUnsafeBufferPointer({
UnsafePointer<Int32>($0.baseAddress!).pointee
})
return Int32(bigEndian: intValue)
}
After migrating to Swift 3 I'm hitting this error. I haven't been able to figure out what actually needs to change, any help would be really appreciated
The conversion from a pointer to UInt8 to a pointer to Int32
must be made explicit with withMemoryRebound():
func readInt() -> Int32 {
var intValueBuffer = [UInt8](repeating: 0, count: MemoryLayout<Int32>.size)
self.stream!.read(&intValueBuffer, maxLength: MemoryLayout<Int32>.size)
let intValue = intValueBuffer.withUnsafeBufferPointer({
$0.baseAddress!.withMemoryRebound(to: Int32.self, capacity: 1) {
$0.pointee
}
})
return Int32(bigEndian: intValue)
}
Alternatively, read into a Data object instead of an array:
func readInt() -> Int32 {
var intValueBuffer = Data(count: MemoryLayout<Int32>.size)
intValueBuffer.withUnsafeMutableBytes {
_ = self.stream!.read($0, maxLength: MemoryLayout<Int32>.size)
}
return Int32(bigEndian: intValueBuffer.withUnsafeBytes { $0.pointee })
}
or directly into an integer variable:
func readInt() -> Int32 {
var intValue: Int32 = 0
withUnsafePointer(to: &intValue) {
$0.withMemoryRebound(to: UInt8.self, capacity: MemoryLayout<Int32>.size) {
_ = self.stream!.read($0, maxLength: MemoryLayout<Int32>.size)
}
}
return Int32(bigEndian: intValue)
}
For more information see UnsafeRawPointer Migration on swift.org.
You may also want to check the return value of the read method
which is the number of bytes actually read, or -1 if the read operation failed.

Resources