I'd like to use amino marshal when I have private variables in the structure.
In test2 package, test.go :
type Lnt struct {
Neg bool
abs string // this variable
}
func NewLnt() Lnt {
return Lnt{
Neg: true,
abs: "abcdefef",
}
}
testing go file :
func TestAbc(t *testing.T) {
s := test2.NewLnt()
t.Log("s=", s)
cdc := amino.NewCodec()
b, err := cdc.MarshalBinary(s)
assert.Nil(t, err)
t.Log("b=",b)
var s2 test2.Lnt
err = cdc.UnmarshalBinary(b, &s2)
assert.Nil(t, err)
assert.Equal(t, s, s2)
t.Log("s2=", s2)
}
result :
encoding_test.go:39: s= {true abcdefef}
encoding_test.go:55:
Error Trace: encoding_test.go:55
Error: Not equal:
expected: test2.Lnt{Neg:true, abs:"abcdefef"}
actual : test2.Lnt{Neg:true, abs:""} // error
Diff:
--- Expected
+++ Actual
## -2,3 +2,3 ##
Neg: (bool) true,
- abs: (string) (len=8) "abcdefef"
+ abs: (string) ""
}
Test: TestAbc
encoding_test.go:57: s2= {true }
Private variable "abs" is lost..
Is it not supported, or is there another way to use it in this case?
The short answer is you can't.
What's happening here is you are marshalling all the exported values into the binary format, but the unexported values are not included because the marshaler doesn't have access to them.
The binary data is the unmarshalled into a new struct, and because the unexported field was not in the binary data, it's impossible for the struct to get initialised with that value. (also, it cannot have a value set because it's unexported).
You will need to export the struct field (make public) if you want this test to pass, or accept that that data is lost on marshalling.
Related
I'm pretty new to Golang. I have a YAML file with dynamic keys but only one is known, and it's not necessarily the first one (after config). There is no other key at the level of config.
The yaml file :
config:
foo:bar: baz
bar:foo: baz
abs:getit: myvalue
I want to retrieve myvalue from the nested key config:abs:getit. This nested key name will never change, it will always be config:abs:getit. All other keys can be whatever, we don't care, with different types of content (arrays, int, strings, array of array).
What is the best way to recover the value ?
I worked with yaml package, but I have to fix every field in a struct to unmarshall it, but I don't know how many nested keys there can be so I cannot write a struct which works all the time.
I worked with a map, but I can figure out which map I have to use, because if I can have a field with 6 nested keys or 3 nested keys with array in it before the value I'm searching and it will fails.
I am pretty lost with those kind of things in a dynamic context.
Ideally, I want to do a cat myFile.yaml | yq '.config."abs:getit"', but in Golang...
Any ideas and best practices to do that ?
You can do:
func main() {
var obj struct {
Config struct {
AbsGetit string `yaml:"abs:getit"`
} `yaml:"config"`
}
err := yaml.Unmarshal(data, &obj)
if err != nil {
panic(err)
}
fmt.Printf("%q\n", obj.Config.AbsGetit)
}
https://go.dev/play/p/KJ_lzZxaZBy
I think you need this code. Just put the correct path to your "myFile.yaml" file. In main() function, you will see two different examples of how to use the code according to your needs.
getConfVal finds a node of a YAML tree with an arbitrary sequence in N-dimensional depth. If the node does not exist, the value will be nil.
myFile.yaml
config:
foo:bar: baz
bar:foo: baz
abs:getit: myvalue
foo:
bar: "conf-foo-bar"
bar:
foo: "conf-bar-foo"
abs:
getit: "conf-abs-getit"
one:
two:
three:
four:
five: 5
five2: [4, 7]
package main
import (
"fmt"
"os"
yaml "gopkg.in/yaml.v3"
)
func main() {
if err := readTConf("./path/to/myFile.yaml", &cfg); err != nil {
fmt.Printf("Read YAML Conf: %v\n", err)
return
}
// e.g.
getConfVal(cfg, []string{"foo:bar"})
// getConfVal(cfg, []string{"foo", "bar"})
// getConfVal(cfg, []string{"four", "five"})
// getConfVal(cfg, []string{"four", "five2"})
fmt.Printf("\nThis is the result you are looking for. (%v)\n", needleRes)
}
var needleRes interface{}
var cfg map[string]interface{}
func getConfVal(o map[string]interface{}, ns []string) (map[string]interface{}, bool) {
nsCnt := len(ns)
for kn, vn := range ns {
for ko, vo := range o {
if fmt.Sprintf("%T", vo) == "map[string]interface {}" {
res, ok := getConfVal(vo.(map[string]interface{}), ns)
if ok {
return res, true
break
}
}
if fmt.Sprintf("%T", vo) == "string" {
if ko == vn {
if kn+1 == nsCnt {
needleRes = vo
return map[string]interface{}{}, true
}
}
}
}
}
return map[string]interface{}{}, false
}
func readTConf(f string, c *map[string]interface{}) error {
yamlFile, err := os.ReadFile(f)
if err != nil {
return err
}
if err := yaml.Unmarshal([]byte(yamlFile), &c); err != nil {
return err
}
return nil
}
Thank you for your precise answer. I'm sorry but there is an error in the question, and I apologize for the mistake.
It's not a single flow scalar Yaml but a map, since there is space before the value :
config:
foo:bar: baz
bar:foo: baz
abs:getit: myvalue
The code above logically returns a conversion error like this :
panic: yaml: unmarshal errors:
line 2: cannot unmarshal !!map into string
My whole code is here. The file I read is a Pulumi config Yaml, which will be different for all projects, except for one common key ("abs:getit:"), only the value is different.
The original question file has been modified. Really sorry for that...
Imagine I have the following struct:
type MyGeneric[T string | int] struct {
}
I want to check whether the generic used to instantiate that struct was a string or a int when creating a new MyGeneric.
myGenericString := MyGeneric[string]{}
myGenericString.canHandle("hello") -> should return true
myGenericString.canHandle(8) -> should return false
func (mG MyGeneric[T]) canHandle(value any) bool {
// how to get what T is the same type as value
}
Just instantiate the T directly to get its value.
type MyGeneric[T any] struct {
}
func (mG MyGeneric[T]) canHandle(value any) bool {
var t T
tt := reflect.TypeOf(t)
vt := reflect.TypeOf(value)
fmt.Printf("-> %v == %v\n", tt, vt)
return tt == vt
}
type empty struct{}
func main() {
fmt.Printf("%v\n", MyGeneric[string]{}.canHandle(""))
fmt.Printf("%v\n", MyGeneric[string]{}.canHandle(1))
fmt.Printf("%v\n", MyGeneric[MyGeneric[struct{}]]{}.canHandle(MyGeneric[struct{}]{}))
fmt.Printf("%v\n", MyGeneric[MyGeneric[struct{}]]{}.canHandle(MyGeneric[empty]{}))
}
Output:
-> string == string
true
-> string == int
false
-> main.MyGeneric[struct {}] == main.MyGeneric[struct {}]
true
-> main.MyGeneric[struct {}] == main.MyGeneric[main.empty]
false
If you are worried about T allocating too much unused stack, make it an array instead:
var tArr [0]T
tt := reflect.TypeOf(tArr).Elem()
It hasn't been implemented yet. There is an open proposal about adding the necessary methods to reflect.Type.
The current workaround as of Go 1.19 is to parse the string obtained from TypeOf. Something like this:
var r = regexp.MustCompile("[A-Za-z0-9_]+\\.[A-Za-z0-9_]+\\[(.*)\\]")
func (mG MyGeneric[T]) typeParam(value any) {
tname := reflect.TypeOf(mG).String() // this is `main.MyGeneric[string]`
match := r.FindStringSubmatch(tname)
fmt.Println(match[1]) // string
}
This if the goal is just to obtain the name of the type parameter. It's not great, as it depends on the type's string representation. On the bright side, it doesn't force you to think about what happens if you instantiate T with interfaces.
If you need to do further computations with the type of T, e.g. compare it to other types etc. #SOFe’s answer provides a solution that doesn’t depend on arbitrary string representations.
However watch out for T instantiated with interfaces: see also In Golang, how to compare interface as generics type to nil?
Another solution is to add a placeHolder field to your struct, that can be used to get its type via type switch to avoid reflect:
type MyGeneric[T string | int] struct {
placeHolder T
}
func (mG MyGeneric[T]) canHandle(value any) bool {
switch t1 := any(p.placeHolder).(type) {
case any:
switch t2 := value.(type) {
case any:
return t1 == t2
}
}
return false
}
I am trying to understand generics implementation in go1.18. In my test example, I am storing a series of test cases and attempting to call a function variable. Unfortunately, I am getting an error in the EvalCases function when I try to use the variable tc.input, I receive the following error:
cannot use input (variable of type T constrained by Inputer) as type
string in argument to fn
Why am I receiving that error, and how do I fix it?
import (
"fmt"
"strconv"
)
type BoolCase func(string) bool
type Inputer interface {
int | float64 | ~string
}
type Wanter interface {
Inputer | bool
}
type TestCase[T Inputer, U Wanter] struct {
input T
want U
}
type TestConditions[T Inputer, U Wanter] map[string]TestCase[T, U]
// IsNumeric validates that a string is either a valid int64 or float64
func IsNumeric(s string) bool {
_, err := strconv.ParseFloat(s, 64)
return err == nil
}
func EvalCases[T Inputer, U Wanter](cases TestConditions[T, U], fn BoolCase) {
for name, tc := range cases {
input := T(tc.input)
want := tc.want
// Error: cannot use input (variable of type T constrained by Inputer) as type string in argument to fn
got := fn(input)
fmt.Printf("name: %-20s | input: %-10v | want: %-10v | got: %v\n", name, input, want, got)
}
}
func main() {
var cases = TestConditions[string, bool]{
"empty": {input: "", want: false},
"integer": {input: "123", want: true},
"float": {input: "123.456", want: true},
}
fn := IsNumeric
EvalCases(cases, fn)
}
Why am I receiving that error
Because fn is a BoolFunc, which is a func(string) bool, thus requires a string as parameter, but input is of type T. Furthermore, by your definition, T satisfies the Inputer constraint, and thus can also assume type int, float64 or any non-string type which has string as its underlying type (~string) none of which implicitly cast to string.
how do I fix it?
You need to change the definition of BoolCase to itself have a generic type parameter for its one parameter. You could constraint it to Inputer, but you could also use any (interface{}).
type BoolCase[T any] func(T) bool
Then make sure to serve this generic type parameter in the signature of function EvalCases:
func EvalCases[T Inputer, U Wanter](cases TestConditions[T, U], fn BoolCase[T])
https://go.dev/play/p/RdjQXJ0WpDh
Is there a way to use reflect to access unexported fields in Go 1.8?
This no longer seems to work: https://stackoverflow.com/a/17982725/555493
Note that reflect.DeepEqual works just fine (that is, it can access unexported fields) but I can't make heads or tails of that function. Here's a go playarea that shows it in action: https://play.golang.org/p/vyEvay6eVG. The src code is below
import (
"fmt"
"reflect"
)
type Foo struct {
private string
}
func main() {
x := Foo{"hello"}
y := Foo{"goodbye"}
z := Foo{"hello"}
fmt.Println(reflect.DeepEqual(x,y)) //false
fmt.Println(reflect.DeepEqual(x,z)) //true
}
If the struct is addressable, you can use unsafe.Pointer to access the field (read or write) it, like this:
rs := reflect.ValueOf(&MyStruct).Elem()
rf := rs.Field(n)
// rf can't be read or set.
rf = reflect.NewAt(rf.Type(), unsafe.Pointer(rf.UnsafeAddr())).Elem()
// Now rf can be read and set.
See full example on the playground.
This use of unsafe.Pointer is valid according to the documentation and running go vet returns no errors.
If the struct is not addressable this trick won't work, but you can create an addressable copy like this:
rs = reflect.ValueOf(MyStruct)
rs2 := reflect.New(rs.Type()).Elem()
rs2.Set(rs)
rf = rs2.Field(0)
rf = reflect.NewAt(rf.Type(), unsafe.Pointer(rf.UnsafeAddr())).Elem()
// Now rf can be read. Setting will succeed but only affects the temporary copy.
See full example on the playground.
Based on cpcallen's work:
import (
"reflect"
"unsafe"
)
func GetUnexportedField(field reflect.Value) interface{} {
return reflect.NewAt(field.Type(), unsafe.Pointer(field.UnsafeAddr())).Elem().Interface()
}
func SetUnexportedField(field reflect.Value, value interface{}) {
reflect.NewAt(field.Type(), unsafe.Pointer(field.UnsafeAddr())).
Elem().
Set(reflect.ValueOf(value))
}
reflect.NewAt might be confusing to read at first. It returns a reflect.Value representing a pointer to a value of the specified field.Type(), using unsafe.Pointer(field.UnsafeAddr()) as that pointer. In this context reflect.NewAt is different than reflect.New, which would return a pointer to a freshly initialized value.
Example:
type Foo struct {
unexportedField string
}
GetUnexportedField(reflect.ValueOf(&Foo{}).Elem().FieldByName("unexportedField"))
https://play.golang.org/p/IgjlQPYdKFR
reflect.DeepEqual() can do it because it has access to unexported features of the reflect package, in this case namely for the valueInterface() function, which takes a safe argument, which denies access to unexported field values via the Value.Interface() method if safe=true. reflect.DeepEqual() will (might) call that passing safe=false.
You can still do it, but you cannot use Value.Interface() for unexported fields. Instead you have to use type-specific methods, such as Value.String() for string, Value.Float() for floats, Value.Int() for ints etc. These will return you a copy of the value (which is enough to inspect it), but will not allow you to modify the field's value (which might be "partly" possible if Value.Interface() would work and the field type would be a pointer type).
If a field happens to be an interface type, you may use Value.Elem() to get to the value contained / wrapped by the interface value.
To demonstrate:
type Foo struct {
s string
i int
j interface{}
}
func main() {
x := Foo{"hello", 2, 3.0}
v := reflect.ValueOf(x)
s := v.FieldByName("s")
fmt.Printf("%T %v\n", s.String(), s.String())
i := v.FieldByName("i")
fmt.Printf("%T %v\n", i.Int(), i.Int())
j := v.FieldByName("j").Elem()
fmt.Printf("%T %v\n", j.Float(), j.Float())
}
Output (try it on the Go Playground):
string hello
int64 2
float64 3
package main
import (
"fmt"
"reflect"
"strings"
"unsafe"
)
type Person1 struct {
W3ID string
Name string
}
type Address1 struct {
city string
country string
}
type User1 struct {
name string
age int
address Address1
manager Person1
developer Person1
tech Person1
}
func showDetails(load, email interface{}) {
if reflect.ValueOf(load).Kind() == reflect.Struct {
typ := reflect.TypeOf(load)
value := reflect.ValueOf(load)
//#1 For struct, not addressable create a copy With Element.
value2 := reflect.New(value.Type()).Elem()
//#2 Value2 is addressable and can be set
value2.Set(value)
for i := 0; i < typ.NumField(); i++ {
if value.Field(i).Kind() == reflect.Struct {
rf := value2.Field(i)
/* #nosec G103 */
rf = reflect.NewAt(rf.Type(), unsafe.Pointer(rf.UnsafeAddr())).Elem()
irf := rf.Interface()
typrf := reflect.TypeOf(irf)
nameP := typrf.String()
if strings.Contains(nameP, "Person") {
//fmt.Println(nameP, "FOUND !!!!!!! ")
for j := 0; j < typrf.NumField(); j++ {
re := rf.Field(j)
nameW := typrf.Field(j).Name
if strings.Contains(nameW, "W3ID") {
valueW := re.Interface()
fetchEmail := valueW.(string)
if fetchEmail == email {
fmt.Println(fetchEmail, " MATCH!!!!")
}
}
}
}
showDetails(irf, email)
} else {
// fmt.Printf("%d.Type:%T || Value:%#v\n",
// (i + 1), value.Field(i), value.Field(i))
}
}
}
}
func main() {
iD := "tsumi#in.org.com"
load := User1{
name: "John Doe",
age: 34,
address: Address1{
city: "New York",
country: "USA",
},
manager: Person1{
W3ID: "jBult#in.org.com",
Name: "Bualt",
},
developer: Person1{
W3ID: "tsumi#in.org.com",
Name: "Sumi",
},
tech: Person1{
W3ID: "lPaul#in.org.com",
Name: "Paul",
},
}
showDetails(load, iD)
}
I have a function that solves the problem of Go not allowing for the setting of default values in method declarations. I want to make it just a little bit better by allowing for a variable number of return variables. I understand that I can allow for an array of interfaces as a return type and then create an interface array with all the variables to return, like this:
func SetParams(params []interface{}, args ...interface{}) (...[]interface{}) {
var values []interface{}
for i := range params {
var value interface{}
paramType := reflect.TypeOf(params[i])
if len(args) < (i + 1) {
value = params[i]
} else {
argType := reflect.TypeOf(args[i])
if paramType != argType {
value = params[i]
}
value = args[i]
}
values = append(values, value)
}
return values
}
This is an example of a method you want to define default values for. You build it as a variadic function (allowing a variable number of parameters) and then define the default values of the specific params you are looking for inside the function instead of in the declaration line.
func DoSomething(args ...interface{}) {
//setup default values
str := "default string 1"
num := 1
str2 := "default string 2"
//this is fine
values := SetParams([]interface{str, num, str2}, args)
str = values[0].(string)
num = values[1].(int)
str = values[2].(string)
//I'd rather support this
str, num, str2 = SetParams(params, args)
}
I understand that
[]interface{str, num, str2}
in the above example is not syntactically correct. I did it that way to simplify my post. But, it represents another function that builds the array of interfaces.
I would like to support this:
str, num, str2 = SetParams(params, args)
instead of having to do this:
values := SetParams([]interface{str, num, str2}, args)
str = values[0].(string)
num = values[1].(int)
str = values[2].(string)
Any advice? Help?
Please don't write horrible (and ineffective due to reflect) code to solve nonexistent problem.
As was indicated in comments, turning a language into
one of your previous languages is indeed compelling
after a switch, but this is counterproductive.
Instead, it's better to work with the idioms and approaches
and best practices the language provides --
even if you don't like them (yet, maybe).
For this particular case you can roll like this:
Make the function which wants to accept
a list of parameters with default values
accept a single value of a custom struct type.
For a start, any variable of such type, when allocated,
has all its fields initialized with the so-called "zero values"
appropriate to their respective types.
If that's enough, you can stop there: you will be able
to pass values of your struct type to your functions
by producing them via literals right at the call site --
initializing only the fields you need.
Otherwise have pre-/post- processing code which
would provide your own "zero values" for the fields
you need.
Update on 2016-08-29:
Using a struct type to simulate optional parameters
using its fields being assigned default values which happen
to be Go's native zero values for their respective data types:
package main
import (
"fmt"
)
type params struct {
foo string
bar int
baz float32
}
func myfun(params params) {
fmt.Printf("%#v\n", params)
}
func main() {
myfun(params{})
myfun(params{bar: 42})
myfun(params{foo: "xyzzy", baz: 0.3e-2})
}
outputs:
main.params{foo:"", bar:0, baz:0}
main.params{foo:"", bar:42, baz:0}
main.params{foo:"xyzzy", bar:0, baz:0.003}
As you can see, Go initializes the fields of our params type
with the zero values appropriate to their respective types
unless we specify our own values when we define our literals.
Playground link.
Providing default values which are not Go-native zero values for
the fields of our custom type can be done by either pre-
or post-processing the user-submitted value of a compound type.
Post-processing:
package main
import (
"fmt"
)
type params struct {
foo string
bar int
baz float32
}
func (pp *params) setDefaults() {
if pp.foo == "" {
pp.foo = "ahem"
}
if pp.bar == 0 {
pp.bar = -3
}
if pp.baz == 0 { // Can't really do this to FP numbers; for demonstration purposes only
pp.baz = 0.5
}
}
func myfun(params params) {
params.setDefaults()
fmt.Printf("%#v\n", params)
}
func main() {
myfun(params{})
myfun(params{bar: 42})
myfun(params{foo: "xyzzy", baz: 0.3e-2})
}
outputs:
main.params{foo:"ahem", bar:-3, baz:0.5}
main.params{foo:"ahem", bar:42, baz:0.5}
main.params{foo:"xyzzy", bar:-3, baz:0.003}
Playground link.
Pre-processing amounts to creating a "constructor" function
which would return a value of the required type pre-filled
with the default values your choice for its fields—something
like this:
func newParams() params {
return params{
foo: "ahem",
bar: -3,
baz: 0.5,
}
}
so that the callers of your function could call newParams(),
tweak its fields if they need and then pass the resulting value
to your function:
myfunc(newParams())
ps := newParams()
ps.foo = "xyzzy"
myfunc(ps)
This approach is maybe a bit more robust than post-processing but
it precludes using of literals to construct the values to pass to
your function right at the call site which is less "neat".
Recently I was playing with anonymous functions in Go and implemented an example which accepts and returns undefined parameters:
func functions() (funcArray []func(args ... interface{}) (interface{}, error)) {
type ret struct{
first int
second string
third bool
}
f1 := func(args ... interface{}) (interface{}, error){
a := args[0].(int)
b := args[1].(int)
return (a < b), nil
}
funcArray = append(funcArray , f1)
f2 := func(args ... interface{}) (interface{}, error){
return (args[0].(string) + args[1].(string)), nil
}
funcArray = append(funcArray , f2)
f3 := func(args ... interface{}) (interface{}, error){
return []int{1,2,3}, nil
}
funcArray = append(funcArray , f3)
f4 := func(args ... interface{}) (interface{}, error){
return ret{first: 1, second: "2", third: true} , nil
}
funcArray = append(funcArray , f4)
return funcArray
}
func main() {
myFirst_Function := functions()[0]
mySecond_Function := functions()[1]
myThird_Function := functions()[2]
myFourth_Function := functions()[3]
fmt.Println(myFirst_Function(1,2))
fmt.Println(mySecond_Function("1","2"))
fmt.Println(myThird_Function())
fmt.Println(myFourth_Function ())
}
I hope it helps you.
https://play.golang.org/p/d6dSYLwbUB9