Appending to struct slice in Go - go

I have two structs, like so:
// init a struct for a single item
type Cluster struct {
Name string
Path string
}
// init a grouping struct
type Clusters struct {
Cluster []Cluster
}
What I want to do is append to new items to the clusters struct. So I wrote a method, like so:
func (c *Clusters) AddItem(item Cluster) []Cluster {
c.Cluster = append(c.Cluster, item)
return c.Cluster
}
The way my app works, I loop through some directories then append the name of the final directory and it's path. I have a function, that is called:
func getClusters(searchDir string) Clusters {
fileList := make([]string, 0)
//clusterName := make([]string, 0)
//pathName := make([]string, 0)
e := filepath.Walk(searchDir, func(path string, f os.FileInfo, err error) error {
fileList = append(fileList, path)
return err
})
if e != nil {
log.Fatal("Error building cluster list: ", e)
}
for _, file := range fileList {
splitFile := strings.Split(file, "/")
// get the filename
fileName := splitFile[len(splitFile)-1]
if fileName == "cluster.jsonnet" {
entry := Cluster{Name: splitFile[len(splitFile)-2], Path: strings.Join(splitFile[:len(splitFile)-1], "/")}
c.AddItem(entry)
}
}
Cluster := []Cluster{}
c := Clusters{Cluster}
return c
}
The problem here is that I don't know the correct way to do this.
Currently, I'm getting:
cmd/directories.go:41:4: undefined: c
So I tried moving this:
Cluster := []Cluster{}
c := Clusters{Cluster}
Above the for loop - range. The error I get is:
cmd/directories.go:43:20: Cluster is not a type
What am I doing wrong here?

The error is in the loop where you are calling AddItem function on Cluster method receiver which is not defined inside getClusters function. Define Cluster struct before for loop and then call the function c.AddItem as defined below:
func getClusters(searchDir string) Clusters {
fileList := make([]string, 0)
fileList = append(fileList, "f1", "f2", "f3")
ClusterData := []Cluster{}
c := Clusters{Cluster: ClusterData} // change the struct name passed to Clusters struct
for _, file := range fileList {
entry := Cluster{Name: "name" + file, Path: "path" + file}
c.AddItem(entry)
}
return c
}
you have defined the same struct name to Clusters struct that's why the error
cmd/directories.go:43:20: Cluster is not a type
Checkout working code on Go playground
In Golang Composite literal is defined as:
Composite literals construct values for structs, arrays, slices, and maps and create a new value each time they are evaluated. They
consist of the type of the literal followed by a brace-bound list of
elements. Each element may optionally be preceded by a corresponding
key.
Also Have a look on struct literals section defined in above link for Compositeliterals to get more description.

You need to define c before entering the loop in which you use it.
The Cluster is not a type error is due to using the same Cluster name as the type and the variable, try using a different variable name.
clusterArr := []Cluster{}
c := Clusters{clusterArr}
for _, file := range fileList {
....
}

Related

Get the type of value using cty in hclwrite

am looking for a way to find the type of variable using go-cty package in hclwrite.
My aim is to generate a variables file like below
variable "test_var" {
val1 = bool
val2 = string
val3 = number
}
reference: https://developer.hashicorp.com/terraform/language/values/variables
I am using the below code to generate this.
vars := hclwrite.NewEmptyFile()
vars_root_body := vars.Body()
vars_file, vars_create_err := os.Create("variables.tf")
logErrors(vars_create_err)
vars_block := vars_root_body.AppendNewBlock("variable",[]string{"test_var"})
vars_block_body := vars_block.Body()
vars_block_body.SetAttributeValue("val", cty.Value{})
_, vars_write_err := vars_file.Write(vars.Bytes())
logErrors(vars_write_err)
defer vars_file.Close()
the above code generates this
variable "test_var" {
val = null
}
I want to fetch the type of that variable and set the attribute value based on that type, as show in the reference link above. I tried lot of ways but didn't get anything. Can someone please help me on this?
I tried the above code and lot of other ways like
cty.SetValEmpty(cty.Bool)
but it didn't work.
The expected syntax for a variable block in Terraform includes an argument named type, not an argument named val. From your example I assume that you are intending to populate type.
The type constraint syntax that Terraform uses is not directly part of HCL and so there isn't any built-in way to generate that syntax in only one step. However, type constraint are built from HCL's identifier and function call syntaxes, and hclwrite does have some functions for helping to generate those as individual parts:
TokensForIdentifier
TokensForFunctionCall
f := hclwrite.NewEmptyFile()
rootBody := f.Body()
varBlock := rootBody.AppendNewBlock("variable", []string{"example"})
varBody := varBlock.Body()
varBody.SetAttributeRaw(
"type",
hclwrite.TokensForFunctionCall(
"set",
hclwrite.TokensForIdentifier("string"),
),
)
fmt.Printf("%s", f.Bytes())
The above will generate the following:
variable "example" {
type = set(string)
}
If you already have a cty.Value value then you can obtain its type using the Type method. However, as mentioned above there isn't any ready-to-use function for converting a type into a type expression, so if you want to be able to generate a type constraint for any value then you'd need to write a function for this yourself, wrapping the TokensForFunctionCall and TokensForIdentifier functions. For example:
package main
import (
"fmt"
"sort"
"github.com/hashicorp/hcl/v2/hclwrite"
"github.com/zclconf/go-cty/cty"
)
func main() {
f := hclwrite.NewEmptyFile()
rootBody := f.Body()
varBlock := rootBody.AppendNewBlock("variable", []string{"example"})
varBody := varBlock.Body()
varBody.SetAttributeRaw(
"type",
typeExprTokens(cty.Set(cty.String)),
)
fmt.Printf("%s", f.Bytes())
}
func typeExprTokens(ty cty.Type) hclwrite.Tokens {
switch ty {
case cty.String:
return hclwrite.TokensForIdentifier("string")
case cty.Bool:
return hclwrite.TokensForIdentifier("bool")
case cty.Number:
return hclwrite.TokensForIdentifier("number")
case cty.DynamicPseudoType:
return hclwrite.TokensForIdentifier("any")
}
if ty.IsCollectionType() {
etyTokens := typeExprTokens(ty.ElementType())
switch {
case ty.IsListType():
return hclwrite.TokensForFunctionCall("list", etyTokens)
case ty.IsSetType():
return hclwrite.TokensForFunctionCall("set", etyTokens)
case ty.IsMapType():
return hclwrite.TokensForFunctionCall("map", etyTokens)
default:
// Should never happen because the above is exhaustive
panic("unsupported collection type")
}
}
if ty.IsObjectType() {
atys := ty.AttributeTypes()
names := make([]string, 0, len(atys))
for name := range atys {
names = append(names, name)
}
sort.Strings(names)
items := make([]hclwrite.ObjectAttrTokens, len(names))
for i, name := range names {
items[i] = hclwrite.ObjectAttrTokens{
Name: hclwrite.TokensForIdentifier(name),
Value: typeExprTokens(atys[name]),
}
}
return hclwrite.TokensForObject(items)
}
if ty.IsTupleType() {
etys := ty.TupleElementTypes()
items := make([]hclwrite.Tokens, len(etys))
for i, ety := range etys {
items[i] = typeExprTokens(ety)
}
return hclwrite.TokensForTuple(items)
}
panic(fmt.Errorf("unsupported type %#v", ty))
}
This program will generate the same output as the previous example. You can change func main to pass a different type to typeExprTokens to see how it behaves with some different types.

Examining fields of a struct by reference (via static analysis)

I'm trying to write a parser for golang code to examine the fields of a referenced struct. For example, given:
type Hello struct {
id int64
}
func Test(ref Hello) {}
I would like to be able to statically analyze this code and go from the args of Test and inspect Hello's fields.
I'm currently using the analysis package. I know how to inspect the struct definition itself in the ast, and also how to parse the function's args for its types. But is there a way to go from reference to parsing the struct? What if the struct is defined in a different file?
If you're doing static analysis and you'd like to better understand how the packages go/ast, go/types, etc. work together then you should definitely check out Alan Donovan's go types document.
You can use the golang.org/x/tools/go/packages package to get the syntax tree and the type info. There may be better, less involved, approaches to achieve the same but this one's the one I'm familiar with.
To get the go/types representation of Hello you can do the following:
func main() {
cfg := new(packages.Config)
cfg.Mode = packages.NeedSyntax | packages.NeedTypes | packages.NeedTypesInfo
cfg.Fset = token.NewFileSet()
// "." specifies the current directory.
// You should replace it with a pattern that
// will match the package you want to analyse.
pkgs, err := packages.Load(cfg, ".")
if err != nil {
panic(err)
}
for _, pkg := range pkgs {
// Loop over the list of files in the package.
for _, syn := range pkg.Syntax {
// Loop over the top-level declarations in the file.
for _, dec := range syn.Decls {
// Look for the func declaration
// of your Help function.
fd, ok := dec.(*ast.FuncDecl)
if !ok || fd.Name.Name != "Test" {
continue
}
// Get the expression node that
// represents the identifier of
// the parameter's type i.e. Hello.
p := fd.Type.Params.List[0].Type
// NOTE: if the type is not a named
// package-local type, e.g. a pointer,
// a slice, or an imported type, then
// you'll have have to "dig deeper"
// to get to the *ast.Ident.
id, ok := p.(*ast.Ident)
if !ok {
continue
}
// With the packages.NeedTypesInfo mode set
// the package will also include the result
// of the complete type-check of the package's
// syntax trees.
//
// The TypeInfo.Types field maps ast expressions
// to their types, this allows you to get the type
// information using the identifier.
typ := pkg.TypesInfo.Types[id]
named := typ.Type.(*types.Named)
fmt.Println(named) // Hello's *types.Named
fmt.Println(named.Underlying().(*types.Struct)) // Hello's *types.Struct
}
}
}
}
To get the go/ast representation of the Hello type's definition you can do the following:
func main() {
// You'll need to repeat the steps above
// to load the packages as well as finding
// the *types.Named instance which will be
// used to determine the position of the
// type's definition ast.
pos := named.Obj().Pos() // the source position of the type's name
for _, pkg := range pkgs {
// Loop over the files in the package.
for _, syn := range pkg.Syntax {
// Use the position to determine whether
// or not the type is declared in this
// file, if not then go to the next one.
if syn.Pos() >= pos || pos >= syn.End() {
continue
}
// Loop over the top-level declarations in the file.
for _, dec := range syn.Decls {
// If the declaration is something
// other than a type declaration then
// continue to the next one.
gd, ok := dec.(*ast.GenDecl)
if !ok || gd.Tok != token.TYPE {
continue
}
// Loop over the specs in the declaration.
for _, spec := range gd.Specs {
// Look for the type spec whose name matches
// the name of the *types.Named instance.
ts, ok := spec.(*ast.TypeSpec)
if !ok || ts.Name.Name != named.Obj().Name() {
continue
}
fmt.Println(ts) // Hello's *ast.TypeSpec
fmt.Println(ts.Type.(*ast.StructType)) // Hello's *ast.StructType
}
}
}
}
}

Append to golang slice passed as empty interface

How to append to empty interface (that has been verified to be a *[]struct)?
func main() {
var mySlice []myStruct // myStruct can be any struct (dynamic)
decode(&mySlice, "...")
}
func decode(dest interface{}, src string) {
// assume dest has been verified to be *[]struct
var modelType reflect.Type = getStructType(dest)
rows, fields := getRows(src)
for _, row := range rows {
// create new struct of type modelType and assign all fields
model := reflect.New(modelType)
for field := fields {
fieldValue := getRowValue(row, field)
model.Elem().FieldByName(field).Set(fieldValue)
}
castedModelRow := model.Elem().Interface()
// append model to dest; how to do this?
// dest = append(dest, castedModelRow)
}
}
Things I've tried:
This simply panics: reflect: call of reflect.Append on ptr Value (as we pass &mySlice instead of mySlice)
dest = reflect.Append(reflect.ValueOf(dest), reflect.ValueOf(castedModelRow))
This works but doesn't set the value back to dest... in main func, len(mySlice) remains 0 after decode function is called.
func decode(dest interface{}, src string) {
...
result := reflect.MakeSlice(reflect.SliceOf(modelType), rowCount, rowCount)
for _, row : range rows {
...
result = reflect.Append(result, reflect.ValueOf(castedModelRow))
}
dest = reflect.ValueOf(result)
}
Here's how to fix the second decode function shown in the question. The statement
dest = reflect.ValueOf(result)
modifies local variable dest, not the caller's value. Use the following statement to modify the caller's slice:
reflect.ValueOf(dest).Elem().Set(result)
The code in the question appends decoded elements after the elements created in reflect.MakeSlice. The resulting slice has len(rows) zero values followed by len(rows) decoded values. Fix by changing
result = reflect.Append(result, reflect.ValueOf(castedModelRow))
to:
result.Index(i).Set(model)
Here's the update version of the second decode function in the question:
func decode(dest interface{}, src string) {
var modelType reflect.Type = getStructType(dest)
rows, fields := getRows(src)
result := reflect.MakeSlice(reflect.SliceOf(modelType), len(rows), len(rows))
for i, row := range rows {
model := reflect.New(modelType).Elem()
for _, field := range fields {
fieldValue := getRowValue(row, field)
model.FieldByName(field).Set(fieldValue)
}
result.Index(i).Set(model)
}
reflect.ValueOf(dest).Elem().Set(result)
}
Run it on the Playground.
You were very close with your original solution. You had to de-reference the pointer before calling the append operation. This solution would be helpful if your dest already had some existing elements and you don't want to lose them by creating a newSlice.
tempDest := reflect.ValueOf(dest).Elem()
tempDest = reflect.Append(tempDest, reflect.ValueOf(model.Interface()))
Similar to how #I Love Reflection pointed out, you finally need to set the new slice back to the pointer.
reflect.ValueOf(dest).Elem().Set(tempDest)
Overall Decode:
var modelType reflect.Type = getStructType(dest)
rows, fields := getRows(src)
tempDest := reflect.ValueOf(dest).Elem()
for _, row := range rows {
model := reflect.New(modelType).Elem()
for _, field := range fields {
fieldValue := getRowValue(row, field)
model.FieldByName(field).Set(fieldValue)
}
tempDest = reflect.Append(tempDest, reflect.ValueOf(model.Interface()))
}
reflect.ValueOf(dest).Elem().Set(tempDest)

Initialize structs from list of strings

I'm trying to initialize structs from a list of strings, but the compiler is throwing the following error. I'm still learning the language so excuse my ignorance, but is this solved by utilizing type assertion?
ERROR: v.UberX undefined (type string has no field method UberX)
type Galaxy struct {
UberX int64
UberY int64
}
func main() {
galaxies := []string{"andromeda", "milkyway", "maffei"}
for _, v := range galaxies {
v := &Galaxy{}
}
for _, v := range galaxies {
v.UberX += 1000
v.UberY += 750
}
}
Your Galaxy struct doesn't even store the name, in your attempt there isn't any connection between the names and the struct values. Add the name to the struct:
type Galaxy struct {
Name string
UberX int64
UberY int64
}
Next, in your first loop you create a *Galaxy value, but you only store it in a local variable v which by the way shadows the loop variable v:
for _, v := range galaxies {
v := &Galaxy{}
}
You need a slice of Galaxy or a slice of *Galaxy which you can populate:
gs := make([]*Galaxy, len(galaxies))
Then 1 loop is enough to loop over the galaxy names and populate the gs slice:
for i, v := range galaxies {
gs[i] = &Galaxy{
Name: v,
UberX: 1000,
UberY: 750,
}
}
Verifying the result:
for _, v := range gs {
fmt.Printf("%+v\n", v)
}
Output (try it on the Go Playground):
&{Name:andromeda UberX:1000 UberY:750}
&{Name:milkyway UberX:1000 UberY:750}
&{Name:maffei UberX:1000 UberY:750}
Recommended to go through the Golang Tour first to learn the basics.

Correct use of XML annotations, fields and structs in custom UnmarshalXML function

Consider the following struct:
type MyStruct struct {
Name string
Meta map[string]interface{}
}
Which has the following UnmarshalXML function:
func (m *MyStruct) UnmarshalXML(d *xml.Decoder, start xml.StartElement) error {
var v struct {
XMLName xml.Name //`xml:"myStruct"`
Name string `xml:"name"`
Meta struct {
Inner []byte `xml:",innerxml"`
} `xml:"meta"`
}
err := d.DecodeElement(&v, &start)
if err != nil {
return err
}
m.Name = v.Name
myMap := make(map[string]interface{})
// ... do the mxj magic here ... -
temp := v.Meta.Inner
prefix := "<meta>"
postfix := "</meta>"
str := prefix + string(temp) + postfix
//fmt.Println(str)
myMxjMap, err := mxj.NewMapXml([]byte(str))
myMap = myMxjMap
// fill myMap
//m.Meta = myMap
m.Meta = myMap["meta"].(map[string]interface{})
return nil
}
My problem with this code is these lines:
prefix := "<meta>"
postfix := "</meta>"
str := prefix + string(temp) + postfix
myMxjMap, err := mxj.NewMapXml([]byte(str))
myMap = myMxjMap
//m.Meta = myMap
m.Meta = myMap["meta"].(map[string]interface{})
My question is how I make the correct use of the xml annotations (,innerxml etc), fields and structs, so I don't have to manually pre-/append the <meta></meta> tags afterwards to get the whole Meta field as a single map.
The full code example is here: http://play.golang.org/p/Q4_tryubO6
xml package doesn't provide a way to unmarshal XML into map[string]interface{} because there is no single way to do it and in some cases it is not possible. A map doesn't preserve order of the elements (that is important in XML) and doesn't allow elements with duplicate keys.
mxj package that you used in your example has some rules how to unmarshal arbitrary XML into Go map. If your requirements do not conflict with these rules you can use mxj package to do all parsing and do not use xml package at all:
// I am skipping error handling here
m, _ := mxj.NewMapXml([]byte(s))
mm := m["myStruct"].(map[string]interface{})
myStruct.Name = mm["name"].(string)
myStruct.Meta = mm["meta"].(map[string]interface{})
Full example: http://play.golang.org/p/AcPUAS0QMj

Resources