Using variable-length array as a map key in golang - go

I can't find a good way to do this. I want to have a map from a list of sorted key-value pairs.
type Tag struct {
key string
value string
}
type SortedTag []Tag // sorted list of tags.
map[SortedTags]T // cannot do.
I can solve this problem by joining all the key-value pairs with a delimiter, but I feel like this is inefficient and error prone in many ways. Converting back to the key-value pair is cumbersome because we need to split the input. moreover, if the key value pair can be anything, that means we have to escape it.
If it was python, I would've stored Tag as N-tuple of sorted 2-tupless.
If it was java, I would've created a composite object with Map<String,String> with equals() checking against the other hash map, hashCode() returning the xor of all the hashes of the map (xor since it is commutative thus we can iterate the map in any order to compute this value).
In go, I can't think of any other good way.

For example,
package main
import "fmt"
type Tag struct {
Key string
Value string
}
type Tags []Tag
type TagsValue struct {
// some type used as Tags value
}
type TagsMapValue struct {
Tags
TagsValue
}
type TagsMapKey string
type TagsMap map[TagsMapKey]TagsMapValue
func NewTagsMapKey(tags Tags) TagsMapKey {
b := []byte{}
for _, tag := range tags {
b = append(b, tag.Key...)
b = append(b, tag.Value...)
}
return TagsMapKey(b[:len(b)])
}
func (m *TagsMap) AddElement(tags Tags, tagsValue TagsValue) {
mapKey := NewTagsMapKey(tags)
mapValue := TagsMapValue{Tags: make(Tags, 0, len(tags)), TagsValue: tagsValue}
i := 0
for _, tag := range tags {
key := string(mapKey[i : i+len(tag.Key)])
i += len(tag.Key)
value := string(mapKey[i : i+len(tag.Value)])
i += len(tag.Value)
mapValue.Tags = append(mapValue.Tags, Tag{Key: key, Value: value})
}
(*m)[mapKey] = mapValue
return
}
func main() {
m := make(TagsMap)
sortedTags := Tags{
{Key: "key1", Value: "value1"},
{Key: "key7", Value: "value7"},
{Key: "key7", Value: "value49"},
{Key: "key42", Value: "value42"},
}
m.AddElement(sortedTags, TagsValue{})
for k, v := range m {
fmt.Println("Tags Key:", k)
fmt.Println(" Tags: ", v.Tags)
fmt.Println(" Tags Value:", v.TagsValue)
}
}
Output:
Tags Key: key1value1key7value7key7value49key42value42
Tags: [{key1 value1} {key7 value7} {key7 value49} {key42 value42}]
Tags Value: {}
If you are simply trying to test for Tags set membership,
package main
import "fmt"
type Tag struct {
Key string
Value string
}
type Tags []Tag
type TagsSetKey string
type TagsSet map[TagsSetKey]Tags
func NewTagsSetKey(tags Tags) TagsSetKey {
b := []byte{}
for _, tag := range tags {
b = append(b, tag.Key...)
b = append(b, tag.Value...)
}
return TagsSetKey(b[:len(b)])
}
func (m *TagsSet) AddElement(tags Tags) {
setKey := NewTagsSetKey(tags)
setValue := make(Tags, 0, len(tags))
i := 0
for _, tag := range tags {
key := string(setKey[i : i+len(tag.Key)])
i += len(tag.Key)
value := string(setKey[i : i+len(tag.Value)])
i += len(tag.Value)
setValue = append(setValue, Tag{Key: key, Value: value})
}
(*m)[setKey] = setValue
return
}
func (m *TagsSet) IsMember(tags Tags) bool {
return (*m)[NewTagsSetKey(tags)] != nil
}
func main() {
m := make(TagsSet)
sortedTags := Tags{
{Key: "key1", Value: "value1"},
{Key: "key7", Value: "value7"},
{Key: "key7", Value: "value49"},
{Key: "key42", Value: "value42"},
}
m.AddElement(sortedTags)
for k, v := range m {
fmt.Println("Tags Key:", k)
fmt.Println(" Tags: ", v)
}
// In set
fmt.Println(m.IsMember(sortedTags))
// Not in set
sortedTags[0].Key = "key0"
fmt.Println(m.IsMember(sortedTags))
}
Output:
Tags Key: key1value1key7value7key7value49key42value42
Tags: [{key1 value1} {key7 value7} {key7 value49} {key42 value42}]
true
false

if you are after (sorted) tuples, you can check out kmanley/golang-tuple
It does have examples of sorting tuples.
This is different from deckarep/golang-set, which can also be helpful for managing those Tag.

Related

how to get intersection between 2 lists faster

I have 2 lists, one list element type is structA, the other list element type is structB, there's common field string name between structA and structB. how to get intersection element which has same name between 2 lists and avoid o(n^2) time complexity using golang.
type structA struct {
name string
....
}
type structB struct {
name string
..
}
noted: name field in each list is not unique, so convert map way is not a solution
Having lists that are not unique does not prevent you from using a map; you can do something like the following (playground):
package main
import (
"fmt"
)
func main() {
type structA struct {
name string
otherfield int
}
type structB struct {
name string
differentField bool
}
aSlice := []structA{
{name: "foo", otherfield: 1},
{name: "foo", otherfield: 2},
{name: "unique", otherfield: 3},
{name: "one", otherfield: 4},
}
bSlice := []structB{
{name: "foo", differentField: true},
{name: "foo", differentField: false},
{name: "noIntersection", differentField: true},
{name: "one", differentField: false},
}
inA := make(map[string][]interface{})
for _, a := range aSlice {
inA[a.name] = append(inA[a.name], a)
}
intersect := make(map[string][]interface{})
for _, b := range bSlice {
if _, ok := intersect[b.name]; ok {
intersect[b.name] = append(intersect[b.name], b)
continue
}
if a, ok := inA[b.name]; ok {
intersect[b.name] = append(a, b)
continue
}
}
fmt.Println(intersect)
}
I think you can try to use Maps in Golang. The average complexity time is O(n) because a Map in golang base on a Hash table.
Example Code:
aMap := make(map[string][]*structA)
for _,a := range aList {
aMap[a.name] = append(aMap[a.name], a)
}
bMap := make(map[string][]*structB)
for _,b := range bList {
bMap[b.name] = append(bMap[b.name], b)
}
//get an intersection of aList and bList
itersections := make([]*structB, 0, len(aList))
for k,v := range aMap {
if b, ok := bMap[k];ok {
itersections = append(intersections, b...)
}
}

Not getting expected results with waitgroup [duplicate]

Let's suppose I have these types:
type Attribute struct {
Key, Val string
}
type Node struct {
Attr []Attribute
}
and that I want to iterate on my node's attributes to change them.
I would have loved to be able to do:
for _, attr := range n.Attr {
if attr.Key == "href" {
attr.Val = "something"
}
}
but as attr isn't a pointer, this wouldn't work and I have to do:
for i, attr := range n.Attr {
if attr.Key == "href" {
n.Attr[i].Val = "something"
}
}
Is there a simpler or faster way? Is it possible to directly get pointers from range?
Obviously I don't want to change the structures just for the iteration and more verbose solutions are no solutions.
No, the abbreviation you want is not possible.
The reason for this is that range copies the values from the slice you're iterating over.
The specification about range says:
Range expression 1st value 2nd value (if 2nd variable is present)
array or slice a [n]E, *[n]E, or []E index i int a[i] E
So, range uses a[i] as its second value for arrays/slices, which effectively means that the
value is copied, making the original value untouchable.
This behavior is demonstrated by the following code:
x := make([]int, 3)
x[0], x[1], x[2] = 1, 2, 3
for i, val := range x {
println(&x[i], "vs.", &val)
}
The code prints you completely different memory locations for the value from range and the actual
value in the slice:
0xf84000f010 vs. 0x7f095ed0bf68
0xf84000f014 vs. 0x7f095ed0bf68
0xf84000f018 vs. 0x7f095ed0bf68
So the only thing you can do is to either use pointers or the index, as already proposed by jnml and peterSO.
You seem to be asking for something equivalent to this:
package main
import "fmt"
type Attribute struct {
Key, Val string
}
type Node struct {
Attr []Attribute
}
func main() {
n := Node{
[]Attribute{
{"key", "value"},
{"href", "http://www.google.com"},
},
}
fmt.Println(n)
for i := 0; i < len(n.Attr); i++ {
attr := &n.Attr[i]
if attr.Key == "href" {
attr.Val = "something"
}
}
fmt.Println(n)
}
Output:
{[{key value} {href http://www.google.com}]}
{[{key value} {href something}]}
This avoids creating a--possibly large--copy of type Attribute values, at the expense of slice bounds checks. In your example, type Attribute is relatively small, two string slice references: 2 * 3 * 8 = 48 bytes on a 64-bit architecture machine.
You could also simply write:
for i := 0; i < len(n.Attr); i++ {
if n.Attr[i].Key == "href" {
n.Attr[i].Val = "something"
}
}
But, the way to get an equivalent result with a range clause, which creates a copy but minimizes slice bounds checks, is:
for i, attr := range n.Attr {
if attr.Key == "href" {
n.Attr[i].Val = "something"
}
}
I'd adapt your last suggestion and use the index-only version of range.
for i := range n.Attr {
if n.Attr[i].Key == "href" {
n.Attr[i].Val = "something"
}
}
It seems simpler to me to refer to n.Attr[i] explicitly in both the line that tests Key and the line that sets Val, rather than using attr for one and n.Attr[i] for the other.
For example:
package main
import "fmt"
type Attribute struct {
Key, Val string
}
type Node struct {
Attr []*Attribute
}
func main() {
n := Node{[]*Attribute{
&Attribute{"foo", ""},
&Attribute{"href", ""},
&Attribute{"bar", ""},
}}
for _, attr := range n.Attr {
if attr.Key == "href" {
attr.Val = "something"
}
}
for _, v := range n.Attr {
fmt.Printf("%#v\n", *v)
}
}
Playground
Output
main.Attribute{Key:"foo", Val:""}
main.Attribute{Key:"href", Val:"something"}
main.Attribute{Key:"bar", Val:""}
Alternative approach:
package main
import "fmt"
type Attribute struct {
Key, Val string
}
type Node struct {
Attr []Attribute
}
func main() {
n := Node{[]Attribute{
{"foo", ""},
{"href", ""},
{"bar", ""},
}}
for i := range n.Attr {
attr := &n.Attr[i]
if attr.Key == "href" {
attr.Val = "something"
}
}
for _, v := range n.Attr {
fmt.Printf("%#v\n", v)
}
}
Playground
Output:
main.Attribute{Key:"foo", Val:""}
main.Attribute{Key:"href", Val:"something"}
main.Attribute{Key:"bar", Val:""}

Write a function to get a slice of string keys from a map no matter what value type the map is

I want to write a function to get all keys from a map as a slice of string which key type is string and the value could be any other type.
Like this but can have any kind of map[string]... as input.
func mapLowCaseKeys(v map[string]string) []string {
keys := make([]string, len(v))
i := 0
for key := range v {
keys[i] = strings.ToLower(key)
i++
}
return keys
}
Actually I want achive Object.keys() in Javascript.
I've tried use map[string]interface{} as the function's paramter type but it can't just pass any specific map to that function, is this possible in golang?
You can make use MapKeys in reflect package to do that(ref: https://golang.org/pkg/reflect/#Value.MapKeys).
MapKeys returns a slice containing all the keys present in the map, in unspecified order. It panics if v's Kind is not Map. It returns an empty slice if v represents a nil map.
An example is given below (playground link: https://play.golang.org/p/xhDtmbGUyz0):
package main
import (
"fmt"
"reflect"
)
func main() {
fmt.Println(mapLowCaseKeys(map[string]float64{
"key1" : 1.2,
}))
fmt.Println(mapLowCaseKeys(map[string]interface{} {
"key1" : 1.2,
"key2" : map[string]string{"kk": "3"},
}))
fmt.Println(mapLowCaseKeys(map[int]float64{
11 : 1.2,
}))
fmt.Println(mapLowCaseKeys(nil))
}
func mapLowCaseKeys(v interface{}) []string {
keys := []string{}
value := reflect.ValueOf(v)
if value.Kind() == reflect.Map {
for _, v := range value.MapKeys() {
if v.Kind() == reflect.String {
keys = append(keys, v.String())
}
}
return keys
} else {
fmt.Println("it is not a map!!")
return keys
}
}

How to merge two Go values of same struct type?

I want to create a function called merge() that takes in two values of the same struct, but of any struct, and returns the merged values of the two structs.
I want the first value to take precedence. For example, if there are two structs a and b, after calling merge(a,b), if there are fields that both a and b contain, I want it to have a's value for that given field.
What would be the best way to implement this?
https://play.golang.org/p/7s9PWx26gfz
type cat struct {
name string
color string
age int
}
type book struct {
title string
author string
}
func main() {
c1 := cat{
name: "Oscar",
color: "",
age: 3,
}
c2 := cat{
name: "",
color: "orange",
age: 2,
}
c3 := merge(c1, c2)
// want: c3 = cat{
// name: "Oscar",
// color: "orange",
// age: 3,
// }
// another case...
b1 := book{
title: "Lord of the Rings",
author: "John Smith",
}
b2 := book{
title: "Harry Potter",
author: "",
}
b3 := merge(b1, b2)
// want: b3 = book{
// title: "Lord of the Rings",
// author: "John Smith",
// }
}
This is what I have so far:
// merges two structs, where a's values take precendence over b's values (a's values will be kept over b's if each field has a value)
func merge(a, b interface{}) (*interface{}, error) {
var result interface{}
aFields := reflect.Fields(a)
bFields := reflect.Fields(b)
if !reflect.DeepEqual(aFields, bFields) {
return &result, errors.New("cannot merge structs of different struct types")
}
aValOf := reflect.ValueOf(a)
bValOf := reflect.ValueOf(b)
resultValOf := reflect.ValueOf(result)
aValues := make([]interface{}, aValOf.NumField())
resultValues := make([]interface{}, resultValOf.NumField())
for i := 0; i < aValOf.NumField(); i++ {
if reflect.ValueOf(aValues[i]).IsNil() {
resultValues[i] = bValOf.Field(i).Interface()
break
}
resultValues[i] = aValOf.Field(i).Interface()
}
return &result, nil
}
Check this package https://github.com/imdario/mergo
Sample code:
package main
import (
"fmt"
"github.com/imdario/mergo"
)
type Foo struct {
A string
B int64
}
func main() {
src := Foo{
A: "one",
B: 2,
}
dest := Foo{
A: "two",
}
mergo.Merge(&dest, src)
fmt.Println(dest)
// Will print
// {two 2}
}
See it at playground: https://play.golang.org/p/9KWTK5mSZ6Q
Use custom types for fields in your target structs.
type firstString string
type firstInt int
type cat struct {
Name firstString
Color firstString
Age firstInt
}
type book struct {
Title firstString
Author firstString
}
Implement UnMarshalJSON for each custom type such that they only unmarshal for target values that are empty.
func (fs *firstString) UnmarshalJSON(bytes []byte) error {
if len(*fs) > 0 {
return nil
}
var s string
err := json.Unmarshal(bytes, &s)
if err != nil {
return err
}
*fs = firstString(s)
return nil
}
func (fi *firstInt) UnmarshalJSON(bytes []byte) error {
if *fi != 0 {
return nil
}
var i int
err := json.Unmarshal(bytes, &i)
if err != nil {
return err
}
*fi = firstInt(i)
return nil
}
If your data is coming via JSON, you can avoid the use of the merge function; just keep unMarshalling incoming JSON to the same struct. If your data is already in separate structs, you can use JSON as an intermediary in your merge function to abstract away all the reflect-ing you have in your example.
// merges two structs, where a's values take precendence over b's values (a's values will be kept over b's if each field has a value)
func merge(a, b interface{}) interface{} {
jb, err := json.Marshal(b)
if err != nil {
fmt.Println("Marshal error b:", err)
}
err = json.Unmarshal(jb, &a)
if err != nil {
fmt.Println("Unmarshal error b-a:", err)
}
return a
}
All together in a working example: https://play.golang.org/p/5YO2HCi8f0N

Contains method for a slice

Is there anything similar to a slice.contains(object) method in Go without having to do a search through each element in a slice?
Mostafa has already pointed out that such a method is trivial to write, and mkb gave you a hint to use the binary search from the sort package. But if you are going to do a lot of such contains checks, you might also consider using a map instead.
It's trivial to check if a specific map key exists by using the value, ok := yourmap[key] idiom. Since you aren't interested in the value, you might also create a map[string]struct{} for example. Using an empty struct{} here has the advantage that it doesn't require any additional space and Go's internal map type is optimized for that kind of values. Therefore, map[string] struct{} is a popular choice for sets in the Go world.
No, such method does not exist, but is trivial to write:
func contains(s []int, e int) bool {
for _, a := range s {
if a == e {
return true
}
}
return false
}
You can use a map if that lookup is an important part of your code, but maps have cost too.
Starting with Go 1.18, you can use the slices package – specifically the generic Contains function:
https://pkg.go.dev/golang.org/x/exp/slices#Contains.
go get golang.org/x/exp/slices
import "golang.org/x/exp/slices"
things := []string{"foo", "bar", "baz"}
slices.Contains(things, "foo") // true
Note that since this is outside the stdlib as an experimental package, it is not bound to the Go 1 Compatibility Promise™ and may change before being formally added to the stdlib.
With Go 1.18+ we could use generics.
func Contains[T comparable](s []T, e T) bool {
for _, v := range s {
if v == e {
return true
}
}
return false
}
The sort package provides the building blocks if your slice is sorted or you are willing to sort it.
input := []string{"bird", "apple", "ocean", "fork", "anchor"}
sort.Strings(input)
fmt.Println(contains(input, "apple")) // true
fmt.Println(contains(input, "grow")) // false
...
func contains(s []string, searchterm string) bool {
i := sort.SearchStrings(s, searchterm)
return i < len(s) && s[i] == searchterm
}
SearchString promises to return the index to insert x if x is not present (it could be len(a)), so a check of that reveals whether the string is contained the sorted slice.
Instead of using a slice, map may be a better solution.
simple example:
package main
import "fmt"
func contains(slice []string, item string) bool {
set := make(map[string]struct{}, len(slice))
for _, s := range slice {
set[s] = struct{}{}
}
_, ok := set[item]
return ok
}
func main() {
s := []string{"a", "b"}
s1 := "a"
fmt.Println(contains(s, s1))
}
http://play.golang.org/p/CEG6cu4JTf
If the slice is sorted, there is a binary search implemented in the sort package.
func Contain(target interface{}, list interface{}) (bool, int) {
if reflect.TypeOf(list).Kind() == reflect.Slice || reflect.TypeOf(list).Kind() == reflect.Array {
listvalue := reflect.ValueOf(list)
for i := 0; i < listvalue.Len(); i++ {
if target == listvalue.Index(i).Interface() {
return true, i
}
}
}
if reflect.TypeOf(target).Kind() == reflect.String && reflect.TypeOf(list).Kind() == reflect.String {
return strings.Contains(list.(string), target.(string)), strings.Index(list.(string), target.(string))
}
return false, -1
}
I think map[x]bool is more useful than map[x]struct{}.
Indexing the map for an item that isn't present will return false. so instead of _, ok := m[X], you can just say m[X].
This makes it easy to nest inclusion tests in expressions.
You can use the reflect package to iterate over an interface whose concrete type is a slice:
func HasElem(s interface{}, elem interface{}) bool {
arrV := reflect.ValueOf(s)
if arrV.Kind() == reflect.Slice {
for i := 0; i < arrV.Len(); i++ {
// XXX - panics if slice element points to an unexported struct field
// see https://golang.org/pkg/reflect/#Value.Interface
if arrV.Index(i).Interface() == elem {
return true
}
}
}
return false
}
https://play.golang.org/p/jL5UD7yCNq
Not sure generics are needed here. You just need a contract for your desired behavior. Doing the following is no more than what you would have to do in other languages if you wanted your own objects to behave themselves in collections, by overriding Equals() and GetHashCode() for instance.
type Identifiable interface{
GetIdentity() string
}
func IsIdentical(this Identifiable, that Identifiable) bool{
return (&this == &that) || (this.GetIdentity() == that.GetIdentity())
}
func contains(s []Identifiable, e Identifiable) bool {
for _, a := range s {
if IsIdentical(a,e) {
return true
}
}
return false
}
If it is not feasable to use a map for finding items based on a key, you can consider the goderive tool. Goderive generates a type specific implementation of a contains method, making your code both readable and efficient.
Example;
type Foo struct {
Field1 string
Field2 int
}
func Test(m Foo) bool {
var allItems []Foo
return deriveContainsFoo(allItems, m)
}
To generate the deriveContainsFoo method:
Install goderive with go get -u github.com/awalterschulze/goderive
Run goderive ./... in your workspace folder
This method will be generated for deriveContains:
func deriveContainsFoo(list []Foo, item Foo) bool {
for _, v := range list {
if v == item {
return true
}
}
return false
}
Goderive has support for quite some other useful helper methods to apply a functional programming style in go.
The go style:
func Contains(n int, match func(i int) bool) bool {
for i := 0; i < n; i++ {
if match(i) {
return true
}
}
return false
}
s := []string{"a", "b", "c", "o"}
// test if s contains "o"
ok := Contains(len(s), func(i int) bool {
return s[i] == "o"
})
If you have a byte slice, you can use bytes package:
package main
import "bytes"
func contains(b []byte, sub byte) bool {
return bytes.Contains(b, []byte{sub})
}
func main() {
b := contains([]byte{10, 11, 12, 13, 14}, 13)
println(b)
}
Or suffixarray package:
package main
import "index/suffixarray"
func contains(b []byte, sub byte) bool {
return suffixarray.New(b).Lookup([]byte{sub}, 1) != nil
}
func main() {
b := contains([]byte{10, 11, 12, 13, 14}, 13)
println(b)
}
If you have an int slice, you can use intsets package:
package main
import "golang.org/x/tools/container/intsets"
func main() {
var s intsets.Sparse
for n := 10; n < 20; n++ {
s.Insert(n)
}
b := s.Has(16)
println(b)
}
https://golang.org/pkg/bytes
https://golang.org/pkg/index/suffixarray
https://pkg.go.dev/golang.org/x/tools/container/intsets
I created the following Contains function using reflect package.
This function can be used for various types like int32 or struct etc.
// Contains returns true if an element is present in a slice
func Contains(list interface{}, elem interface{}) bool {
listV := reflect.ValueOf(list)
if listV.Kind() == reflect.Slice {
for i := 0; i < listV.Len(); i++ {
item := listV.Index(i).Interface()
target := reflect.ValueOf(elem).Convert(reflect.TypeOf(item)).Interface()
if ok := reflect.DeepEqual(item, target); ok {
return true
}
}
}
return false
}
Usage of contains function is below
// slice of int32
containsInt32 := Contains([]int32{1, 2, 3, 4, 5}, 3)
fmt.Println("contains int32:", containsInt32)
// slice of float64
containsFloat64 := Contains([]float64{1.1, 2.2, 3.3, 4.4, 5.5}, 4.4)
fmt.Println("contains float64:", containsFloat64)
// slice of struct
type item struct {
ID string
Name string
}
list := []item{
item{
ID: "1",
Name: "test1",
},
item{
ID: "2",
Name: "test2",
},
item{
ID: "3",
Name: "test3",
},
}
target := item{
ID: "2",
Name: "test2",
}
containsStruct := Contains(list, target)
fmt.Println("contains struct:", containsStruct)
// Output:
// contains int32: true
// contains float64: true
// contains struct: true
Please see here for more details:
https://github.com/glassonion1/xgo/blob/main/contains.go
There are several packages that can help, but this one seems promising:
https://github.com/wesovilabs/koazee
var numbers = []int{1, 5, 4, 3, 2, 7, 1, 8, 2, 3}
contains, _ := stream.Contains(7)
fmt.Printf("stream.Contains(7): %v\n", contains)
It might be considered a bit 'hacky' but depending the size and contents of the slice, you can join the slice together and do a string search.
For example you have a slice containing single word values (e.g. "yes", "no", "maybe"). These results are appended to a slice. If you want to check if this slice contains any "maybe" results, you may use
exSlice := ["yes", "no", "yes", "maybe"]
if strings.Contains(strings.Join(exSlice, ","), "maybe") {
fmt.Println("We have a maybe!")
}
How suitable this is really depends on the size of the slice and length of its members. There may be performance or suitability issues for large slices or long values, but for smaller slices of finite size and simple values it is a valid one-liner to achieve the desired result.

Resources