Problem:
Want to read data from CSV file and group it based on this combination warehouse[item[batch, qty]]
[Batch, Qty] pair should be inserted in sequence
Approach:
I thought best way to make it is to use a combination of map and pair/tuple
Code:
package main
import "fmt"
type Pair struct {
Key string
Value float64
}
func main() {
fmt.Println("Hello, 世界")
var inventory = map[string]map[string][]Pair{} //warehouse[item[batch, qty]]
fmt.Printf("%T\n", inventory)
inventory["DMM"] = map[string][]Pair{} // map[string]map[string][]main.Pair
fmt.Printf("%T\n", inventory["DMM"])
inventory["DMM"]["Helmet"] = []Pair{}
fmt.Printf("%T = %v\n", inventory["DMM"]["Helmet"])
inventory["DMM"]["Helmet"] = append(inventory["DMM"]["Helmet"], Pair{"Jan", 10})
fmt.Printf("%T = %v\n", inventory["DMM"]["Helmet"][0])
}
Code error
Hello, 世界
map[string]map[string][]main.Pair
map[string][]main.Pair
[]main.Pair = %!v(MISSING)
main.Pair = %!v(MISSING)
Notes
It looks I was able to enter the warehouse and item combination correctly, but something not correct for entering/inserting/adding/appending the Pair to this combination!
I don't think there is anything wrong with the way you are building up your data structure, just the way you are using Printf(). You have 2 format specs in the format string (%T and %v) but only pass one value. You can recycle the parameter like this:
fmt.Printf("%T = %[1]v\n", inventory["DMM"]["Helmet"])
Also you can just build your data structure with a single literal:
inventory := map[string]map[string][]Pair{"DMM" : {"Helmet": {{Key: "Jan", Value: 10}}}}
Related
This question already has answers here:
Assign a new value to a struct field
(2 answers)
Closed 10 months ago.
The output of the following code surprises me:
package main
import (
"fmt"
)
type Thing struct {
mappings map[string]int
orderings []string
}
func NewThing() Thing {
t := Thing{}
t.mappings = make(map[string]int)
return t
}
func (t Thing) Add(s string) {
t.mappings[s] = 1
t.orderings = append(t.orderings, s)
}
func main() {
t := NewThing()
t.Add("foo")
if len(t.mappings) == len(t.orderings) {
fmt.Printf("Equal lengths: %v versus %v", t.mappings, t.orderings)
} else {
fmt.Printf("Unequal lengths: %v versus %v", t.mappings, t.orderings)
}
}
When run on the playground (https://play.golang.org/p/Ph67tHOt2Z_I) the output is this:
Unequal lengths: map[foo:1] versus []
I believe I'm treating the slice correctly; from my understanding it is initialized to nil in NewThing(), and is appended to in Add() (ensuring that the value returned from append is only assigned to its first argument).
Am I missing something incredibly obvious?
I looked at the following resources for an explanation:
https://gobyexample.com/slices - only uses either slice literals (i.e. not a struct field) or slices with set capacities, and I will not know the final size of t.orderings. It's my understanding that append should perform the extension and allocation automatically.
https://go.dev/blog/slices-intro - again, all demonstrations use slice literals. If the fields are moved out of the struct things work as expected. It's only once in the struct that this behavior occurs.
https://yourbasic.org/golang/gotcha-append/ - while it does describe behavior where append does not work as expected, the explanation involves append reusing memory when the slice has enough capacity for a new element, causing unexpected behavior when attempts to append the same array to two different copies. In my case, there is no reassignment of slice operations such as the one in this article, which is discouraged (some_var = append(some_other_var, elem)).
And I looked at the following questions for inspiration:
Go - append to slice in struct: the solution to this question was to assign the result of append back to the field, which I have done.
Correct way to initialize empty slice: the explanation is that slices don't have to be initialized, and can be left as nil and "appended to with allocation", so I believe I'm fine not initializing Thing.orderings.
Incase you don't want to use a pointer ,you can declare a global variable for Thing struct and assign it with the value of t from add function.Here is the code for the same logic :
package main
import (
"fmt"
)
var thing Thing
type Thing struct {
mappings map[string]int
orderings []string
}
func NewThing() Thing {
t := Thing{}
t.mappings = make(map[string]int)
return t
}
func (t Thing) Add(s string) {
t.mappings[s] = 1
t.orderings = append(t.orderings, s)
thing = t
}
func main() {
t := NewThing()
t.Add("foo")
if len(thing.mappings) == len(thing.orderings) {
fmt.Printf("Equal lengths: %v versus %v", thing.mappings, thing.orderings)
} else {
fmt.Printf("Unequal lengths: %v versus %v", thing.mappings, thing.orderings)
}
}
Output:
Equal lengths: map[foo:1] versus [foo]
I want to Field sort based on multiple fields. I have a slice of elastic.FieldSort (say, sorter) and I want to do something like:
searchSource := elastic.NewSearchSource()
// searchSource.SortBy(sorter...) // Unpacking of slice isn't working
I know that this SortBy accepts single elements as:
searchSource.SortBy(sorter[0], sorter[1], sorter[2])
but the issue is that I have a slice of dynamic size. Please help me how this can be implemented, or is there any better to do this?
The slice unpack would work if the type is right.
package main
import (
"fmt"
)
func print(data ...string) {
fmt.Println(data)
}
func main() {
d := []string{"hello", "world", "now"}
print(d...)
}
For this specific example how if for any reason the unpack is not working still, there is a work around to this.
Since the SortBy function just appends to the underlying sorters slice https://github.com/olivere/elastic/blob/v7.0.22/search_source.go#L160
func (s *SearchSource) SortBy(sorter ...Sorter) *SearchSource {
s.sorters = append(s.sorters, sorter...)
return s
}
You can do
for _, sorter := range sorters {
searchSource.SortBy(sorter)
}
I figured out the issue. There were two problems with my code.
First, I needed to initialize the slice (sorter) as type Sorter:
var sorter []elastic.Sorter
Second issue was on Elastic side. I needed to pass suffix keyword with keyword type fields like,
var fieldName string = field.FieldName
if field.Keyword {
fieldName = field.FieldName + ".keyword"
}
And then use this fieldName to create NewFieldSort. This is why I was getting elastic error 404.
sorter = append(sorter, elastic.NewFieldSort(fieldName).Order(true)
Rest of the code was same, I was passing the sorter slice to the sortBy method:
searchSource.SortBy(sorter...)
I want to tag my maps with a timestamp, there will be various maps - dynamically generated. The keys will be repeated, so if the same map appears I want to updated its values
I'm getting the maps from external sources and want to registered when they are read
Ever so often I want to iterate the map and find those maps that didn't update for a while
a = map[string]string{"a":1, "b":2}
b = map[map[string]string]time.Time{}
b[a] = 1
I understand it's not supported but I was wondering what are the some ways of doing it?
The first thing that comes to mind is: serialize the map into string and use it as key,,?
Make a struct that will keep the info you need:
type MyMap struct {
timestamp time.Time
m map[string]string
}
And lookup by iterating over a slice of such structs:
func lookup(needle map[string]string, haystack []MyMap) (MyMap, bool) {
for _, myMap := range haystack {
if reflect.DeepEqual(needle, myMap.m) {
return myMap, true
}
}
return nil, false
}
Alternatively, convert your map to JSON and use the result as the key:
b, _ := json.Marshal(myMap)
key := string(b)
m, ok := myMaps[key] // myMaps is map[string]MyMap
if !ok {
myMaps[key] = myMap
}
Apparently, json.Marshal sorts the map's keys before marshaling, so this should be a reliable solution.
I have a struct that looks like this:
type guitaristT struct {
Surname string `required=true`
Year int64 `required=false`
American bool // example of missing tag
Rating float32 `required=true`
Styles []string `required=true,minsize=1`
}
I have an environment variable that looks like the following, and I'm using reflection to fill the struct based on the keys.
jimiEnvvar :="surname=Hendrix|year=1942|american=true|rating=9.99
|styles=blues|styles=rock|styles=psychedelic"
I'm able to set the string, int64, bool and float32 fields using reflection, but I'm stuck on how to append to the slice field Styles. For example, based on the above jimiEnvvar I would like the field jimi.Styles to have the values ["blues","rock", "psychedelic"].
I have the following (simplified) code:
result := guitaristT{}
// result.Styles = make([]string, 10) // XXX causes 10 empty strings at start
result.Styles = make([]string, 0) // EDIT: Alessandro's solution
...
v := reflect.ValueOf(&result).Elem()
...
field := v.FieldByName(key) // eg key = "styles"
...
switch field.Kind() {
case reflect.Slice:
// this is where I get stuck
//
// reflect.Append() has signature:
// func Append(s Value, x ...Value) Value
// so I convert my value to a reflect.Value
stringValue := reflect.ValueOf(value) // eg value = "blues"
// field = reflect.Append(field, stringValue) // XXX doesn't append
field.Set(reflect.Append(field, stringValue)) // EDIT: Alessandro's solution
EDIT:
A second part (which I solved) was filling a map in the struct. For example:
type guitaristT struct {
...
Styles []string `required=true,minsize=1`
Cities map[string]int
}
Where jimiEnvvar looks like:
jimiEnvvar += "|cities=New York^17|cities=Los Angeles^14"
I wrote in this manner:
case reflect.Map:
fmt.Println("keyAsString", keyAsString, "is Map, has value:", valueAsString)
mapKV := strings.Split(valueAsString, "^")
if len(mapKV) != 2 {
log.Fatalln("malformed map key/value:", mapKV)
}
mapK := mapKV[0]
mapV := mapKV[1]
thisMap := fieldAsValue.Interface().(map[string]int)
thisMap[mapK] = atoi(mapV)
thisMapAsValue := reflect.ValueOf(thisMap)
fieldAsValue.Set(thisMapAsValue)
The final result was:
main.guitaristT{
Surname: "Hendrix",
Year: 1942,
American: true,
Rating: 9.989999771118164,
Styles: {"blues", "rock", "psychedelic"},
Cities: {"London":11, "Bay Area":9, "New York":17, "Los Angeles":14},
}
If you're interested the full code is at https://github.com/soniah/reflect/blob/master/structs.go. Code is just some notes/exercises I'm writing.
EDIT2:
Chapter 11 of "Go in Practice" (Butcher and Farina) has detailed explanations of reflection, structs and tags.
You were not too far off. Just replace with
field.Set(reflect.Append(field, stringValue))
and you are done. Also, make sure you that you initialise the slice using
result.Styles = make([]string, 0)
or you will end up having 10 blank string at the top of the Styles array.
Hope this helps and good luck with your project.
I really want a way to print the string representation of a field name in go. It has several use cases, but here is an example:
lets say I have a struct
type Test struct {
Field string `bson:"Field" json:"field"`
OtherField int `bson:"OtherField" json:"otherField"`
}
and, for example, I want to do a mongo find:
collection.Find(bson.M{"OtherField": someValue})
I don't like that I have to put the string "OtherField" in there. It seems brittle and easy to either misstype or have the struct change and then my query fails without me knowing it.
Is there any way to get the string "OtherField" without having to either declare a const or something like that? I know I can use reflection to a get a list of field names from a struct, but I'd really like to do something along the lines of
fieldName := nameOf(Test{}.OtherField)
collection.Find(bson.M{fieldName: someValue})
is there any way to do this in Go?? C# 6 has the built in nameof, but digging through reflection I can't find any way to do this in Go.
I don't really think there is. You may be able to load a set of types via reflection and generate a set of constants for the field names. So:
type Test struct {
Field string `bson:"Field" json:"field"`
OtherField int `bson:"OtherField" json:"otherField"`
}
Could generate something like:
var TestFields = struct{
Field string
OtherField string
}{"Field","OtherField"}
and you could use TestFields.Field as a constant.
Unfortunately, I don't know of any existing tool that does anything like that. Would be fairly simple to do, and wire up to go generate though.
EDIT:
How I'd generate it:
Make a package that accepts an array of reflect.Type or interface{} and spits out a code file.
Make a generate.go somewhere in my repo with main function:
func main(){
var text = mygenerator.Gen(Test{}, OtherStruct{}, ...)
// write text to constants.go or something
}
Add //go:generate go run scripts/generate.go to my main app and run go generate
Here is a function that will return a []string with the struct field names. I think it comes in the order they are defined.
WARNING: Reordering the fields in the struct definition will change the order in which they appear
https://play.golang.org/p/dNATzNn47S
package main
import (
"fmt"
"strings"
"regexp"
)
type Test struct {
Field string `bson:"Field" json:"field"`
OtherField int `bson:"OtherField" json:"otherField"`
}
func main() {
fields, err := GetFieldNames(Test{})
if err != nil {
fmt.Println(err)
return
}
fmt.Println(fields)
}
func GetFieldNames(i interface{}) ([]string, error) {
// regular expression to find the unquoted json
reg := regexp.MustCompile(`(\s*?{\s*?|\s*?,\s*?)(['"])?(?P<Field>[a-zA-Z0-9]+)(['"])?:`)
// print struct in almost json form (fields unquoted)
raw := fmt.Sprintf("%#v", i)
// remove the struct name so string begins with "{"
fjs := raw[strings.Index(raw,"{"):]
// find and grab submatch 3
matches := reg.FindAllStringSubmatch(fjs,-1)
// collect
fields := []string{}
for _, v := range matches {
if len(v) >= 3 && v[3] != "" {
fields = append(fields, v[3])
}
}
return fields, nil
}