Input Yaml:
params:
- aws.region: ["us-west-1","us-west-2"]
- aws.s3path: ["s3-path-1", "s3-path-2"]
Code:
package main
import (
"fmt"
"io/ioutil"
"os"
"gopkg.in/yaml.v2"
)
func main() {
var f File
wfyaml, _ := os.Open("temp.yaml")
byteValue, _ := ioutil.ReadAll(wfyaml)
yaml.Unmarshal(byteValue, &f)
fmt.Print(f)
}
type File struct {
Params Params `yaml:"params"`
}
type Params []struct { // <<< This [] behavior is confusing
AwsRegion []string `yaml:"aws.region"`
S3path []string `yaml:"aws.s3path"`
}
Output:
{[{[us-west-1 us-west-2] []} {[] [s3-path-1 s3-path-2]}]}%
If I use
type Params struct { // <<< This [] behavior is confusing
AwsRegion []string `yaml:"aws.region"`
S3path []string `yaml:"aws.s3path"`
}
Then output is {{[] []}}%
I am not sure if I am missing something here.
This is because params from your input yaml is an array.
If you change your input yaml to the following, then you will find your code with type Params struct works as you would expect, because params is no longer an array in this case.
params:
aws.region: ["us-west-1","us-west-2"]
aws.s3path: ["s3-path-1", "s3-path-2"]
The behavior is consistent with the YAML input:
params:
- aws.region: ["us-west-1","us-west-2"]
- aws.s3path: ["s3-path-1", "s3-path-2"]
Above, params is an array, because its elements start with a -. Each - is an array element, and since each element has a key, param is an array of objects. The first element is an object with key aws.region, and the second element is an object with key aws.s3path. These objects are themselves string arrays.
Related
I want to remove elements with empty value in struct. My script is below. Output of this script is {"keyA":{}}. I used omitempty to KeyA and KeyB. But an element with empty value is left. On the other hand KeyB is not shown. I want to show KeyA when it has values. I don't want to show KeyA when it has no values. Is there way to do this?
Script
package main
import (
"encoding/json"
"fmt"
)
type sample struct {
KeyA struct {
Key1 string `json:"keyA1,omitempty"`
Key2 string `json:"keyA2,omitempty"`
} `json:"keyA,omitempty"`
KeyB string `json:"keyB,omitempty"`
}
func main() {
var s sample
response, _ := json.Marshal(s)
fmt.Println(string(response)) // {"keyA":{}}
}
Thank you so much for your time. And I'm sorry for my immature question.
Try this:
package main
import (
"encoding/json"
"fmt"
)
type KeyA struct {
Key1 string `json:"keyA1,omitempty"`
Key2 string `json:"keyA2,omitempty"`
}
type sample struct {
KeyA *KeyA `json:"keyA,omitempty"`
KeyB string `json:"keyB,omitempty"`
}
func main() {
var s sample
response, _ := json.Marshal(s)
fmt.Println(string(response)) // {}
}
output:
{}
I try parse JSON data include integer array. But, I can't get integer array.
package main
import (
"encoding/json"
"fmt"
)
type Anything struct {
A []int `json:"a"`
}
func main() {
s := "{a:[1,2,3]}"
var a Anything
json.Unmarshal([]byte(s), &a)
fmt.Println(a.A)
}
I got empty array.
[]
How can I get [1, 2, 3]?
{a:[1,2,3]} is not valid JSON. Object keys must be double-quoted. Changing it like this works as expected:
s := "{\"a\":[1,2,3]}"
https://play.golang.org/p/qExZAeiRJy
You have an invalid JSON. You should replace it, for example like this: s := [{"a":[1,2,3]}] or maybe like this s := "[{\"a\":[1,2,3]}]".
You can edit your code to something like this:
package main
import (
"encoding/json"
"fmt"
)
type Anything struct {
A []int `json:"a"`
}
func main() {
// note here: `[{"a":[1,2,3]}]`
// or: s := "[{\"a\":[1,2,3]}]"
s := `[{"a":[1,2,3]}]`
var a []Anything
json.Unmarshal([]byte(s), &a)
fmt.Println(a)
}
Output:
[{[1 2 3]}]
You can run it on https://play.golang.org/p/H4GupGFpfP
package main
import (
"fmt"
"encoding/json"
"reflect"
)
type GeneralConfig map[string]interface{}
var data string = `
{
"key":"value",
"important_key":
{"foo":"bar"}
}`
func main() {
jsonData := &GeneralConfig{}
json.Unmarshal([]byte(data), jsonData)
fmt.Println(reflect.TypeOf(jsonData)) //main.GeneralConfig
jsonTemp := (*jsonData)["important_key"]
fmt.Println(reflect.TypeOf(jsonTemp)) //map[string]interface {}
//newGeneralConfig := GeneralConfig(jsonTemp)
//cannot convert jsonTemp (type interface {}) to type GeneralConfig:
//need type assertion
newGeneralConfig := jsonTemp.(GeneralConfig)
//fmt.Println(reflect.TypeOf(newGeneralConfig))
//panic: interface conversion: interface {} is map[string]interface {},
//not main.GeneralConfig
}
Available at the playground
I understand that I can use a nested struct in lieu of GeneralConfig, but that would require me knowing the exact structure of the payload, ie it wouldn't work for different keys (I would be locked into "important_key").
Is there a golang workaround for when I don't know what the value of "important_key" is? I say golang, because if possible, one could require all "important_keys" to have a constant parent key, which could resolve this issue.
To summarize, given an arbitrary json object, there must be a way that I can traverse its keys, and if a value is a custom type, convert the value to that type. Right now it seems that if I use type conversion, it tells me that the type is interface{} and I need to use type assertion; however, if I use type assertion, it tells me that interface{} is map[string]interface{} not main.GeneralConfig.
I agree the comments about trying to utilise the expected structure of the incoming JSON in order to write well-defined Structs, but I'll attempt to answer the question anyway.
The thing to take away from what you're seeing printed versus the error messages that you're seeing is that the compiler knows less about the type than the runtime because the runtime can look at the actual value. To bring the compiler up-to-speed we must (i) assert (*jsonData)["important_key"] is a map[string]interface{} -- the compiler only knows it to be an interface{} -- and then (ii) type-cast that to a GeneralConfig type. See:
package main
import (
"fmt"
"encoding/json"
)
type GeneralConfig map[string]interface{}
func main() {
jsonStruct := new(GeneralConfig)
json.Unmarshal([]byte(`{"parent_key": {"foo": "bar"}}`), jsonStruct)
fmt.Printf("%#v\n", jsonStruct)
// => &main.GeneralConfig{"parent_key":map[string]interface {}{"foo":"bar"}}
nestedStruct := (*jsonStruct)["parent_key"]
fmt.Printf("%#v\n", nestedStruct)
// => map[string]interface {}{"foo":"bar"}
// Whilst this shows the runtime knows its actual type is
// map[string]interface, the compiler only knows it to be an interface{}.
// First we assert for the compiler that it is indeed a
// map[string]interface{} we are working with. You can imagine the issues
// that might arrise if we has passed in `{"parent_key": 123}`.
mapConfig, ok := nestedStruct.(map[string]interface{})
if !ok {
// TODO: Error-handling.
}
// Now that the compiler can be sure mapConfig is a map[string]interface{}
// we can type-cast it to GeneralConfig:
config := GeneralConfig(mapConfig)
fmt.Printf("%#v\n", config)
// => main.GeneralConfig{"foo":"bar"}
}
You are looking for json.RawMessage.
You can delay unmarshalling based upon some other value and then force it to unmarshal to a specific type.
This is not a good idea, but might be closer to what you are looking for.
http://play.golang.org/p/PWwAUDySE0
This is a standard "workaround" if get what you're after. When handling unknown data you can implement this pattern (modified from your example) of switching on the type recursively to get to the concrete values in an unknown body of json data.
package main
import (
"encoding/json"
"fmt"
"reflect"
)
var data = `
{
"key":"value",
"important_key":
{"foo":"bar"}
}`
func main() {
var jsonData interface{}
json.Unmarshal([]byte(data), &jsonData)
fmt.Println(reflect.TypeOf(jsonData))
parseArbitraryJSON(jsonData.(map[string]interface{}))
}
func parseArbitraryJSON(data map[string]interface{}) {
for k, v := range data {
switch a := v.(type) {
case string:
fmt.Printf("%v:%v\n", k, a)
case map[string]interface{}:
fmt.Printf("%v:%v\n", k, a)
parseArbitraryJSON(a)
}
}
}
The resulting output is:
map[string]interface {}
key:value
important_key:map[foo:bar]
foo:bar
This example only accounts for the base data being a string type but you can switch on any type that you expect to receive, and like any switch you can group your cases, so you can treat all numbers similarly for example.
I have some code that looks to be working but does nothing in the end:
http://play.golang.org/p/TfAWWy4-R8
Have a struct that has fields of type struct. The inner struct has all string fields.
Using reflect in a loop, want to get all struct fields from outer struct. Next, populate all string values in inner struct. the example code is getting text from the tags and parsing it on "," to get strings values for inner loop.
This is the main part that should create the inner struct and add the parsed data to the string values.
t := reflect.TypeOf(Alias{})
alias = reflect.New(t)
for i := 0; i < alias.Elem().NumField(); i++ {
p := alias.Elem().Field(i)
p.SetString(params[i])
}
It looks like it is working when you look at the output from example, but after printing a value from outer struct it seems to be empty:
fmt.Println("Final01 = ", Attr.Final01.Command) // prints empty string
So not sure how to get values into the inner struct Alias{}
Thanks for any help.
Here's the working program. Explanation below.
package main
import "fmt"
import "strings"
import "reflect"
type Alias struct {
Name string
DevicePath string
GuiPath string
Setpoint string
Command string
Status string
FunctionCmds string
}
type Manifold struct {
Final01 Alias "Final01,/Gaspanel/Shared/Final01,,,wOpen,rIsOpen,"
Dump01 Alias "Dump01,/Gaspanel/Shared/Dump01,,,wOpen,rIsOpen,"
N2Vent01 Alias "N2Vent01,/Gaspanel/Shared/N2Vent01,,,wOpen,rIsOpen,"
N2Vent201 Alias "N2Vent201,/Gaspanel/Shared/N2Vent201,,,wOpen,rIsOpen,"
PurgeValve Alias "PurgeValve,/Gaspanel/Shared/Purge01,,,wOpen,rIsOpen,"
}
func MapTagedAliasToChamber(chamber string, struc interface{}) []string {
attributeStruct := reflect.ValueOf(struc).Elem()
typeAttributeStruct := attributeStruct.Type()
attributes := make([]string, attributeStruct.NumField(), attributeStruct.NumField())
for i := 0; i < attributeStruct.NumField(); i++ {
alias := attributeStruct.Field(i)
tag := string(typeAttributeStruct.Field(i).Tag)
name := typeAttributeStruct.Field(i).Name
params := strings.Split(tag, ",")
alias = reflect.New(reflect.TypeOf(Alias{})).Elem()
for i := 0; i < alias.NumField(); i++ {
alias.Field(i).SetString(params[i])
}
attributeStruct.Field(i).Set(alias)
fmt.Printf("%d: %s %s = %v\n", i, name, alias.Type(), alias.Interface())
}
return attributes
}
func main() {
Attr := Manifold{}
MapTagedAliasToChamber("A", &Attr)
fmt.Println("Final01 = ", Attr.Final01.Command)
}
The problem was on line 38 of your original program, where you created a new reflect.Value named alias representing a value of type *Alias, then filled it with your information but never wrote it back into your Manifold struct.
Additionally I suggest that you stick to the standard struct-tag format which can be parsed and used more easily through (reflect.StructTag).Get(key string). And don't use strings where you don't need them e.g. rIsOpen sounds like a boolean value to me.
having a rough time working with struct fields using reflect package. in particular, have not figured out how to set the field value.
type t struct { fi int; fs string }
var r t = t{ 123, "jblow" }
var i64 int64 = 456
getting Name of field i - this seems to work
var field = reflect.TypeOf(r).Field(i).Name
getting value of field i as a) interface{}, b) int - this seems to work
var iface interface{} = reflect.ValueOf(r).Field(i).Interface()
var i int = int(reflect.ValueOf(r).Field(i).Int())
setting value of field i - try one - panic
reflect.ValueOf(r).Field(i).SetInt( i64 )
panic: reflect.Value·SetInt using value obtained using unexported field
assuming it did not like field names "id" and "name", so renamed to "Id" and "Name"
a) is this assumption correct?
b) if correct, thought not necessary since in same file / package
setting value of field i - try two (with field names capitalized ) - panic
reflect.ValueOf(r).Field(i).SetInt( 465 )
reflect.ValueOf(r).Field(i).SetInt( i64 )
panic: reflect.Value·SetInt using unaddressable value
Instructions below by #peterSO are thorough and high quality
Four. this works:
reflect.ValueOf(&r).Elem().Field(i).SetInt( i64 )
he documents as well that the field names must be exportable (begin with capital letter)
The Go json package marshals and unmarshals JSON from and to Go structures.
Here's a step-by-step example which sets the value of a struct field while carefully avoiding errors.
The Go reflect package has a CanAddr function.
func (v Value) CanAddr() bool
CanAddr returns true if the value's
address can be obtained with Addr.
Such values are called addressable. A
value is addressable if it is an
element of a slice, an element of an
addressable array, a field of an
addressable struct, or the result of
dereferencing a pointer. If CanAddr
returns false, calling Addr will
panic.
The Go reflect package has a CanSet function, which, if true, implies that CanAddr is also true.
func (v Value) CanSet() bool
CanSet returns true if the value of v
can be changed. A Value can be changed
only if it is addressable and was not
obtained by the use of unexported
struct fields. If CanSet returns
false, calling Set or any
type-specific setter (e.g., SetBool,
SetInt64) will panic.
We need to make sure we can Set the struct field. For example,
package main
import (
"fmt"
"reflect"
)
func main() {
type t struct {
N int
}
var n = t{42}
// N at start
fmt.Println(n.N)
// pointer to struct - addressable
ps := reflect.ValueOf(&n)
// struct
s := ps.Elem()
if s.Kind() == reflect.Struct {
// exported field
f := s.FieldByName("N")
if f.IsValid() {
// A Value can be changed only if it is
// addressable and was not obtained by
// the use of unexported struct fields.
if f.CanSet() {
// change value of N
if f.Kind() == reflect.Int {
x := int64(7)
if !f.OverflowInt(x) {
f.SetInt(x)
}
}
}
}
}
// N at end
fmt.Println(n.N)
}
Output:
42
7
If we can be certain that all the error checks are unnecessary, the example simplifies to,
package main
import (
"fmt"
"reflect"
)
func main() {
type t struct {
N int
}
var n = t{42}
fmt.Println(n.N)
reflect.ValueOf(&n).Elem().FieldByName("N").SetInt(7)
fmt.Println(n.N)
}
BTW, Go is available as open source code. A good way to learn about reflection is to see how the core Go developers use it. For example, the Go fmt and json packages. The package documentation has links to the source code files under the heading Package files.
This seems to work:
package main
import (
"fmt"
"reflect"
)
type Foo struct {
Number int
Text string
}
func main() {
foo := Foo{123, "Hello"}
fmt.Println(int(reflect.ValueOf(foo).Field(0).Int()))
reflect.ValueOf(&foo).Elem().Field(0).SetInt(321)
fmt.Println(int(reflect.ValueOf(foo).Field(0).Int()))
}
Prints:
123
321