Using go/ast for iota declarations - go

I've been working with go/ast to parse go source code and copy it into another file as part of a vendoring exercise. I've got most things handled - functions, types etc - but I'm struggling with const declarations which use iota. I'm iterating through the items in ast.File.Scope.Objects and copying over the source for objects with Scope.Outer == nil and their Decl == ast.ValueSpec, basically implying top level variables and constants.
In a block of type:
const (
a = iota
b
c
d
)
...each one of them registers as a separate object, which is fair enough. However, I'm struggling to assign them with values because the objects can also be out of order when I'm iterating through them. I'm able to see the value as ast.Object.Data for these, but it also seems off when it's set to 1 << iota and so on. Does anyone have any thoughts on how I can get a grouped const declaration with the correct iota values assigned?
Thank you!

I was working on this problem for the exhaustive analyzer, which, during its enum discovery phase, needs to find constant values.
Playground: https://play.golang.org/p/nZLmgE4rJZH
Consider the following ConstDecl. It comprises of 3 ConstSpec, and each ConstSpec has 2 names. (This example uses iota, but the approach below should work generally for any ConstDecl.)
package example
const (
A, B = iota, iota * 100 // 0, 0
_, D // 1, 100
E, F // 2, 200
)
With go/ast and go/types (or x/tools/go/packages), we can obtain a *ast.GenDecl representing the above ConstDecl and a *types.Info for the package.
var decl *ast.GenDecl
var info *types.Info
The following will be true of decl.
decl.Tok == token.CONST
To obtain the constant values, we can do:
func printValuesConst(decl *ast.GenDecl, info *types.Info) {
for _, s := range decl.Specs {
v := s.(*ast.ValueSpec) // safe because decl.Tok == token.CONST
for _, name := range v.Names {
c := info.ObjectOf(name).(*types.Const)
fmt.Println(name, c.Val().ExactString())
}
}
}
This will, as expected, print:
A 0
B 0
_ 1
D 100
E 2
F 200
Side note: var instead of const
Note that the code above works for const blocks; for a var block we will have to obtain the value using v.Values[i] (assuming a value exists at the index i).
Playground: https://play.golang.org/p/f4mYjXvsvHB
decl.Tok == token.VAR
func printValuesVar(decl *ast.GenDecl, info *types.Info) {
for _, s := range decl.Specs {
v := s.(*ast.ValueSpec) // safe because decl.Tok == token.VAR
for i, name := range v.Names {
if len(v.Values) <= i {
fmt.Println(name, "(no AST value)")
continue
}
tv := info.Types[v.Values[i]]
if tv.Value == nil {
fmt.Println(name, "(not constant value)")
continue
}
fmt.Println(name, tv.Value.ExactString())
}
}
}

Related

Remove all map key in Golang [duplicate]

I'm looking for something like the c++ function .clear() for the primitive type map.
Or should I just create a new map instead?
Update: Thank you for your answers. By looking at the answers I just realized that sometimes creating a new map may lead to some inconsistency that we don't want. Consider the following example:
var a map[string]string
var b map[string]string
func main() {
a = make(map[string]string)
b=a
a["hello"]="world"
a = nil
fmt.Println(b["hello"])
}
I mean, this is still different from the .clear() function in c++, which will clear the content in the object.
You should probably just create a new map. There's no real reason to bother trying to clear an existing one, unless the same map is being referred to by multiple pieces of code and one piece explicitly needs to clear out the values such that this change is visible to the other pieces of code.
So yeah, you should probably just say
mymap = make(map[keytype]valtype)
If you do really need to clear the existing map for whatever reason, this is simple enough:
for k := range m {
delete(m, k)
}
Unlike C++, Go is a garbage collected language. You need to think things a bit differently.
When you make a new map
a := map[string]string{"hello": "world"}
a = make(map[string]string)
the original map will be garbage-collected eventually; you don't need to clear it manually. But remember that maps (and slices) are reference types; you create them with make(). The underlying map will be garbage-collected only when there are no references to it.
Thus, when you do
a := map[string]string{"hello": "world"}
b := a
a = make(map[string]string)
the original array will not be garbage collected (until b is garbage-collected or b refers to something else).
// Method - I , say book is name of map
for k := range book {
delete(book, k)
}
// Method - II
book = make(map[string]int)
// Method - III
book = map[string]int{}
Go 1.18 and above
You can use maps.Clear. The function belongs to the package golang.org/x/exp/maps (experimental and not covered by the compatibility guarantee)
Clear removes all entries from m, leaving it empty.
Example usage:
func main() {
testMap := map[string]int{"gopher": 1, "badger": 2}
maps.Clear(testMap)
fmt.Println(testMap)
testMap["zebra"] = 2000
fmt.Println(testMap)
}
Playground: https://go.dev/play/p/qIdnGrd0CYs?v=gotip
If you don't want to depend on experimental packages, you can copy-paste the source, which is actually extremely simple:
func Clear[M ~map[K]V, K comparable, V any](m M) {
for k := range m {
delete(m, k)
}
}
IMPORTANT NOTE: just as with the builtin delete — which the implementation of maps.Clear uses —, this does not remove irreflexive keys from the map. The reason is that for irreflexive keys, by definition, x == x is false. Irreflexive keys are NaN floats and every other type that supports comparison operators but contains NaN floats somewhere.
See this code to understand what this entails:
func main() {
m := map[float64]string{}
m[1.0] = "foo"
k := math.NaN()
fmt.Println(k == k) // false
m[k] = "bar"
maps.Clear(m)
fmt.Printf("len: %d, content: %v\n", len(m), m)
// len: 1, content: map[NaN:bar]
a := map[[2]float64]string{}
a[[2]float64{1.0, 2.0}] = "foo"
h := [2]float64{1.0, math.NaN()}
fmt.Println(h == h) // false
a[h] = "bar"
maps.Clear(a)
fmt.Printf("len: %d, content: %v\n", len(a), a)
// len: 1, content: map[[1 NaN]:bar]
}
Playground: https://go.dev/play/p/LWfiD3iPA8Q
A clear builtin is being currently discussed (Autumn 2022) that, if added to next Go releases, will delete also irreflexive keys.
For the method of clearing a map in Go
for k := range m {
delete(m, k)
}
It only works if m contains no key values containing NaN.
delete(m, k) doesn't work for any irreflexive key (such as math.NaN()), but also structs or other comparable types with any NaN float in it. Given struct{ val float64 } with NaN val is also irreflexive (Quote by blackgreen comment)
To resolve this issue and support clearing a map in Go, one buildin clear(x) function could be available in the new release, for more details, please refer to add clear(x) builtin, to clear map, zero content of slice, ptr-to-array
If you are trying to do this in a loop, you can take advantage of the initialization to clear out the map for you. For example:
for i:=0; i<2; i++ {
animalNames := make(map[string]string)
switch i {
case 0:
animalNames["cat"] = "Patches"
case 1:
animalNames["dog"] = "Spot";
}
fmt.Println("For map instance", i)
for key, value := range animalNames {
fmt.Println(key, value)
}
fmt.Println("-----------\n")
}
When you execute this, it clears out the previous map and starts with an empty map. This is verified by the output:
$ go run maptests.go
For map instance 0
cat Patches
-----------
For map instance 1
dog Spot
-----------

golang: Preferred way to tag a struct with aribtrary data

tl;dr
I have an arbitrary directed graph defined by a Node struct.
I now want to be able to provide a way to write functions that walk this graph and "tag" each Node with metadata specific to that function.
For example, consider a function to count the number of nodes:
type Node struct {
Nexts []*Node
}
func CountNodes(root *Node) int {
m := make(map[*Node]bool)
return countNodesHelper(root, m)
}
func countNodesHelper(root *Node, m map[*Node]bool) int {
_, seen := m[root]
if seen {
return 0
}
m[root] = true
c := 1
for _, child := range root.Nexts {
c += countNodesHelper(child, m)
}
return c
}
func main() {
n1 := &Node{make([]*Node, 0, 1)}
n2 := &Node{[]*Node{n1}}
n1.Nexts = append(n1.Nexts, n2)
fmt.Println(CountNodes(n1))
}
I could rewrite this if I added a "seen" tag inside the struct:
type NodeWithTag struct {
Nexts []*NodeWithTag
Seen bool
}
func CountNodesWithTag(root *NodeWithTag) int {
if root.Seen {
return 0
}
root.Seen = true
c := 1
for _, child := range root.Nexts {
c += CountNodesWithTag(child)
}
return c
}
func main() {
n1 := &NodeWithTag{make([]*NodeWithTag, 0, 1), false}
n2 := &NodeWithTag{[]*NodeWithTag{n1}, false}
n1.Nexts = append(n1.Nexts, n2)
fmt.Println(CountNodesWithTag(n1))
}
But the Seen tag isn't enough for, say, a DFS on a tree where I also want to find backwards edges (you need to count up to 2 -- never seen, seen, seen a second time along a a single path). So, I want some way to allow the function's implementation to use it's own type to tag the struct with. A rough equivalent of:
type Node struct {
...
// Not valid golang
void* tag
}
but safer that a void* -- The function should be able to statically verify that the tag is the current type that it expects. Is there a way to do this / an alternative approach.
The reason I want to associate the tag with the Node (rather than a separate map / store of the tags) is to allow easy parallelization of the functions that use such tags, farming out the nodes to different goroutines. In the first approach, the map would have to be shared between the goroutines, and this would quickly become a bottleneck because it will require synchronized access.
If you need to support arbitrary data types, you'll need to use an empty interface:
type NodeWithTag struct {
Nexts []*NodeWithTag
Tag interface{}
}
You can assign any value to the Tag field. If you want to verify that the value is a certain type, say MyType, you can use a type assertion:
myVal, ok := node.Tag.(MyType)
If the value is of that type, ok will be true and myVal will contain the typed value.

Golang: declaring maps and slices with iota values

I have this Go code:
package main
import "fmt"
type baseGroup int
const (
fooGroup baseGroup = iota + 1
barGroup
)
var groups = [...]string{
fooGroup: "foo",
barGroup: "bar",
}
var xGroups = map[baseGroup]string{
fooGroup: "foo",
barGroup: "bar",
}
func main() {
fmt.Println("groups")
for k, v := range groups {
fmt.Println(k, v)
}
fmt.Println("xGroups")
for k, v := range xGroups {
fmt.Println(k, v)
}
}
If i run the code i get:
groups
0
1 foo
2 bar
xGroups
1 foo
2 bar
I'm wondering why the different outputs?
You're expecting range to behave the same on both types but in the first case it's ranging over an array and you just have an empty index 0. The value being stored in k is the current index; 0, 1, 2. In your second example you're ranging over the map and getting the key stored in k which only contains 1 and 2.
You might be wondering how is this happening? It's because this;
var groups = [...]string{
fooGroup: "foo",
barGroup: "bar",
}
Little confusing (imo very bad) code is giving you this;
var groups = [...]string{
1: "foo",
2: "bar",
}
Which is causing groups to be allocated/initialized with an empty string in index 0. Of course, the map doesn't need a key 0 to let you put something there in key 1 so it doesn't suffer from the same problems. If this is anything more than an exercise to demonstrate the features of the language I would highly recommend you get rid of the intentionally obfuscated code and use more clear constructs for constants, types and initialization.
If your skeptical about that mechanism add the following lines to your main and you can clearly see
fmt.Printf("Lenght of groups is %d and index 0 is %s\n", len(groups), groups[0])
fmt.Printf("Lenght of xGroups is %d\n", len(xGroups))

How to clear a map in Go?

I'm looking for something like the c++ function .clear() for the primitive type map.
Or should I just create a new map instead?
Update: Thank you for your answers. By looking at the answers I just realized that sometimes creating a new map may lead to some inconsistency that we don't want. Consider the following example:
var a map[string]string
var b map[string]string
func main() {
a = make(map[string]string)
b=a
a["hello"]="world"
a = nil
fmt.Println(b["hello"])
}
I mean, this is still different from the .clear() function in c++, which will clear the content in the object.
You should probably just create a new map. There's no real reason to bother trying to clear an existing one, unless the same map is being referred to by multiple pieces of code and one piece explicitly needs to clear out the values such that this change is visible to the other pieces of code.
So yeah, you should probably just say
mymap = make(map[keytype]valtype)
If you do really need to clear the existing map for whatever reason, this is simple enough:
for k := range m {
delete(m, k)
}
Unlike C++, Go is a garbage collected language. You need to think things a bit differently.
When you make a new map
a := map[string]string{"hello": "world"}
a = make(map[string]string)
the original map will be garbage-collected eventually; you don't need to clear it manually. But remember that maps (and slices) are reference types; you create them with make(). The underlying map will be garbage-collected only when there are no references to it.
Thus, when you do
a := map[string]string{"hello": "world"}
b := a
a = make(map[string]string)
the original array will not be garbage collected (until b is garbage-collected or b refers to something else).
// Method - I , say book is name of map
for k := range book {
delete(book, k)
}
// Method - II
book = make(map[string]int)
// Method - III
book = map[string]int{}
Go 1.18 and above
You can use maps.Clear. The function belongs to the package golang.org/x/exp/maps (experimental and not covered by the compatibility guarantee)
Clear removes all entries from m, leaving it empty.
Example usage:
func main() {
testMap := map[string]int{"gopher": 1, "badger": 2}
maps.Clear(testMap)
fmt.Println(testMap)
testMap["zebra"] = 2000
fmt.Println(testMap)
}
Playground: https://go.dev/play/p/qIdnGrd0CYs?v=gotip
If you don't want to depend on experimental packages, you can copy-paste the source, which is actually extremely simple:
func Clear[M ~map[K]V, K comparable, V any](m M) {
for k := range m {
delete(m, k)
}
}
IMPORTANT NOTE: just as with the builtin delete — which the implementation of maps.Clear uses —, this does not remove irreflexive keys from the map. The reason is that for irreflexive keys, by definition, x == x is false. Irreflexive keys are NaN floats and every other type that supports comparison operators but contains NaN floats somewhere.
See this code to understand what this entails:
func main() {
m := map[float64]string{}
m[1.0] = "foo"
k := math.NaN()
fmt.Println(k == k) // false
m[k] = "bar"
maps.Clear(m)
fmt.Printf("len: %d, content: %v\n", len(m), m)
// len: 1, content: map[NaN:bar]
a := map[[2]float64]string{}
a[[2]float64{1.0, 2.0}] = "foo"
h := [2]float64{1.0, math.NaN()}
fmt.Println(h == h) // false
a[h] = "bar"
maps.Clear(a)
fmt.Printf("len: %d, content: %v\n", len(a), a)
// len: 1, content: map[[1 NaN]:bar]
}
Playground: https://go.dev/play/p/LWfiD3iPA8Q
A clear builtin is being currently discussed (Autumn 2022) that, if added to next Go releases, will delete also irreflexive keys.
For the method of clearing a map in Go
for k := range m {
delete(m, k)
}
It only works if m contains no key values containing NaN.
delete(m, k) doesn't work for any irreflexive key (such as math.NaN()), but also structs or other comparable types with any NaN float in it. Given struct{ val float64 } with NaN val is also irreflexive (Quote by blackgreen comment)
To resolve this issue and support clearing a map in Go, one buildin clear(x) function could be available in the new release, for more details, please refer to add clear(x) builtin, to clear map, zero content of slice, ptr-to-array
If you are trying to do this in a loop, you can take advantage of the initialization to clear out the map for you. For example:
for i:=0; i<2; i++ {
animalNames := make(map[string]string)
switch i {
case 0:
animalNames["cat"] = "Patches"
case 1:
animalNames["dog"] = "Spot";
}
fmt.Println("For map instance", i)
for key, value := range animalNames {
fmt.Println(key, value)
}
fmt.Println("-----------\n")
}
When you execute this, it clears out the previous map and starts with an empty map. This is verified by the output:
$ go run maptests.go
For map instance 0
cat Patches
-----------
For map instance 1
dog Spot
-----------

Are there any go libraries that provide associative array capability?

I'm looking for a go language capability similar to the "dictionary" in python to facilitate the conversion of some python code.
EDIT: Maps worked quite well for this de-dupe application. I was able to condense 1.3e6 duplicated items down to 2.5e5 unique items using a map with a 16 byte string index in just a few seconds. The map-related code was simple so I've included it below. Worth noting that pre-allocation of map with 1.3e6 elements sped it up by only a few percent:
var m = make(map[string]int, 1300000) // map with initial space for 1.3e6 elements
ct, ok := m[ax_hash]
if ok {
m[ax_hash] = ct + 1
} else {
m[ax_hash] = 1
}
To expand a little on answers already given:
A Go map is a typed hash map data structure. A map's type signature is of the form map[keyType]valueType where keyType and valueType are the types of the keys and values respectively.
To initialize a map, you must use the make function:
m := make(map[string]int)
An uninitialized map is equal to nil, and if read from or written a panic will occur at runtime.
The syntax for storing values is much the same as doing so with arrays or slices:
m["Alice"] = 21
m["Bob"] = 17
Similarly, retrieving values from a map is done like so:
a := m["Alice"]
b := m["Bob"]
You can use the range keyword to iterate over a map with a for loop:
for k, v := range m {
fmt.Println(k, v)
}
This code will print:
Alice 21
Bob 17
Retrieving a value for a key that is not in the map will return the value type's zero value:
c := m["Charlie"]
// c == 0
By reading multiple values from a map, you can test for a key's presence. The second value will be a boolean indicating the key's presence:
a, ok := m["Alice"]
// a == 21, ok == true
c, ok := m["Charlie"]
// c == 0, ok == false
To remove a key/value entry from a map, you flip it around and assign false as the second value:
m["Bob"] = 0, false
b, ok := m["Bob"]
// b == 0, ok == false
You can store arbitrary types in a map by using the empty interface type interface{}:
n := make(map[string]interface{})
n["One"] = 1
n["Two"] = "Two"
The only proviso is that when retrieving those values you must perform a type assertion to use them in their original form:
a := n["One"].(int)
b := n["Two"].(string)
You can use a type switch to determine the types of the values you're pulling out, and deal with them appropriately:
for k, v := range n {
switch u := v.(type) {
case int:
fmt.Printf("Key %q is an int with the value %v.\n", k, u)
case string:
fmt.Printf("Key %q is a string with the value %q.\n", k, u)
}
}
Inside each of those case blocks, u will be of the type specified in the case statement; no explicit type assertion is necessary.
This code will print:
Key "One" is an int with the value 1.
Key "Two" is a string with the value "Two".
The key can be of any type for which the equality operator is defined, such as integers, floats, strings, and pointers. Interface types can also be used, as long as the underlying type supports equality. (Structs, arrays and slices cannot be used as map keys, because equality is not defined on those types.)
For example, the map o can take keys of any of the above types:
o := make(map[interface{}]int)
o[1] = 1
o["Two"] = 2
And that's maps in a nutshell.
The map type. http://golang.org/doc/effective_go.html#maps
There is some difference from python in that the keys have to be typed, so you can't mix numeric and string keys (for some reason I forgot you can), but they're pretty easy to use.
dict := make(map[string]string)
dict["user"] = "so_user"
dict["pass"] = "l33t_pass1"
You're probably looking for a map.

Resources