I am trying to manipulate a golang sync.Map of sync.Map, but I have some issues with the casting.
I have the following code:
func (cluster *Cluster) action(object1, object2 MyObject) {
value, _ := cluster.globalMap.LoadOrStore(object1.name, sync.Map{})
localMap := value.(sync.Map)
localMap.Store(object2.Name, object2)
value2, _ := cluster.resourceInflight.Load(node.Name)
forComparison := value2.(sync.Map)
fmt.Println(localMap.Load(object2.Name))
fmt.Println(forComparison.Load(object2.Name))
}
{myObject map[] map[]} true
<nil> false
I am doing this since I wish to keep the content of localMap thread safe.
The problem is I am expecting to have the same result for my two print, as "forComparison" should be pointing to the same object than "localMap". But second result is nil.
I am suspecting that the problem is coming from the casting of the interface "value" into an actual "sync.Map". But I am not sure how I can actually call the .Store method with inline casting.
I thought about Storing localMap inside cluster.globalMap, but this seems incorrect to me as it would break the whole point of using a localSyncMap and create concurrency issues.
Any input on what I should do ?
As per the comments the issue was that you were copying a sync.Map; the following code will fail (output "Not found" - playground):
var sm sync.Map
var x interface{}
x = sm
sm2 := x.(sync.Map)
sm2.Store("test", "test")
result, ok := sm.Load("test")
if ok {
fmt.Printf("Found: %s\n", result)
} else {
fmt.Printf("Not found\n")
}
Whereas using a pointer works as expected:
var sm sync.Map
var x interface{}
x = &sm
sm2 := x.(*sync.Map)
sm2.Store("test", "test")
result, ok := sm.Load("test")
if ok {
fmt.Printf("Found: %s\n", result)
} else {
fmt.Printf("Not found\n")
}
Running go vet would probably have warned you about other issues (sync.Map contains a sync.Mutex and these "must not be copied after first use").
Note that the docs for Sync.Map state:
The Map type is specialized. Most code should use a plain Go map instead, with separate locking or coordination, for better type safety and to make it easier to maintain other invariants along with the map content.
Related
I'm trying to test around an SQL query wherein one of the arguments is a gosnowflake.Array (essentially a wrapper to a slice) using the go-sqlmock package. Normally, something like this requires me to create a value converter, which I have included:
func (opt arrayConverterOption[T]) ConvertValue(v any) (driver.Value, error) {
casted, ok := v.(*[]T)
if ok {
Expect(*casted).Should(HaveLen(len(opt.Expected)))
for i, c := range *casted {
Expect(c).Should(Equal(opt.Expected[i]))
}
} else {
fmt.Printf("Type: %T\n", v)
return v, nil
}
return "TEST_RESULT", nil
}
Now, this function is called for every argument submitted to the query. I use it to test the correctness of the values in the slice or pass the argument through if it isn't one. The problem I'm having is that, when I create a arrayConverterOption[string] and give it a gosnowflake.Array(["A", "B", "C"]) as an argument, the type assertion fails because gosnowflake.Array returns an internal dynamic type, *stringArray, which is defined as a *[]string.
So you can see my dilemma here. On the one hand, I can't convert v because it's an interface{} and I can't alias v because the inner type is not *[]string, but *stringArray. So then, what should I do here?
I didn't find a way to do this without resulting to reflection. However, with reflction I did manage it:
var casted []T
var ok bool
value := reflect.ValueOf(v)
if value.Kind() == reflect.Pointer {
if inner := value.Elem(); inner.Kind() == reflect.Slice {
r := inner.Convert(reflect.TypeOf([]T{})).Interface()
casted, ok = r.([]T)
}
}
So, this code checks specifically for anything that is a pointer to a slice, which my dynamic type is. Then it uses reflection to convert the inner object to the slice type I was expecting. After that, I call Interface() on the result to get the interface{} from the reflected value and then cast it to a []T. This succeeds. If it doesn't then I'm not working with one of those dynamically typed slices and I can handle the type normally.
I'm trying to write a wrap around a function that uses an interface{} parameter to return data, by adding cache.
My problem is that once I have a valid interface{} I don't know how to assign it to be returned in the parameter. The wrapped call is (github.Client) .Do in github API client and the problem hit me when I tried to add caching with go-cache
This somewhat my function
func (c *cachedClient) requestAPI(url string, v interface{}) error {
x, found := c.cache.Get(url)
if found { // Found in cache code
log.Printf("cached: %v", x)
v = x // HERE: this do not work. x contains a valid copy of what I want but how do I store it in v?
return nil
}
req, _ := c.githubClient.NewRequest("GET", url, nil) // not found I cache, request it
res, err := c.githubClient.Do(*c.context, req, v)
if err != nil {
return err
}
if res.StatusCode != 200 {
return fmt.Errorf("Error Getting %v: %v", url, res.Status)
}
c.cache.Add(url, v, cache.DefaultExpiration) // store in cache
return nil // once here v works as expected and contain a valid item
}
It fails when has to return a cached value when I try to use it like this:
// Some init code c is a cachedClient
i := new(github.Issue)
c.requestAPI(anAPIValidURLforAnIssue, i)
log.Printf("%+v", i) // var i correctly contains an issue from the github api
o := new(github.Issue)
c.requestAPI(anAPIValidURLforAnIssue, o)
log.Printf("%+v", o) // var o should have been get from the cache but here is empty
So basically my problem is that when I correctly recover a cached item it is good but I can not store it in the parameter meant to be used to store it. I can not work with subclasses because the call I'm wrapping is using an interface{} already. And I can not move it to return values because you can't return a generic interface. How do I make the interface{} x be stored in v to have it available outside?
To archive what you want you need to use a bit of reflection magic.
Please try to replace v = x with next code snippet:
reflect.ValueOf(v).Elem().Set(reflect.ValueOf(x).Elem())
Note from OP: I had to add the last .Elem() to make this work.
NOTE: in the call of the requestAPI method you should use a pointer to the value:
let's say the cached value is of type int. Then you should call requestAPI like:
var dst int // destination of the cached value or a newly retrieved result
cc.requestAPI(url, &dst)
With certain assumptions like you are storing json data in your cache below is how I will try. Errors not handled.
package main
import (
"encoding/json"
"fmt"
)
type Data struct {
Name string
}
func main() {
var d Data
requestAPI(&d)
fmt.Println(d)
}
func requestAPI(v interface{}) {
var cache_res interface{} = []byte("{\"Name\":\"CCC\"}")
//assume you got cache_res from cache
x, _ := cache_res.([]byte)
_ = json.Unmarshal(x, &v)
}
Actually above is what githubClient.Do is also doing. It checks whether v satisfies io.Writer interface, if yes write data to v. If not then it unmarshals json into v as shown above. So same can be done from cache.
Check here:
https://github.com/google/go-github/blob/v32.1.0/github/github.go#L586
If the cache object is specific then below can be used. You don't deal with empty interface{} because you should be able to pass your specific type to c.githubClient.Do as v. Since it uses json package, it will detect the type information and accordingly fill the values into it.
Lets say you store type Data struct
In below code other details eliminated like condition checking whether to cache & error handling
package main
import (
"fmt"
)
type Data struct {
Name string
}
func main() {
var d Data
requestAPI(&d)
fmt.Println(d)
}
func requestAPI(v *Data) {
var cache_res interface{} = Data{"CCC"}
//assume you got cache_res from cache
x, _ := cache_res.(Data)
*v = x
//in case you did not find it in cache then githubClient.Do should unmarshal
//contents of response body into v *Data if Data fields match that of json
//res, err := c.githubClient.Do(*c.context, req, v)
}
I'm new to golang; however based on my current knowledge I understand that a value-type and a reference-type can both fulfill an interface. But it seems in regards to type assertion, how you return a struct does matter. See the following:
package main
import (
"fmt"
)
type SomeError interface {
Error() string
}
type ConcreteError struct{}
func (ConcreteError) Error() string {
return "?"
}
func returnPointer() SomeError {
return &ConcreteError{}
}
func returnVal() SomeError {
return ConcreteError{}
}
func main() {
pointer := returnPointer()
value := returnVal()
_, pointerWithPointer := pointer.(*ConcreteError);
_, pointerWithValue := pointer.(ConcreteError);
_, valueWithValue := value.(ConcreteError);
_, valueWithPointer := value.(*ConcreteError)
fmt.Printf("Returning a pointer, assert using (*ConcreteError): %v\n", pointerWithPointer); // true
fmt.Printf("Returning a pointer, assert using (ConcreteError): %v\n", pointerWithValue); // false
fmt.Printf("Returning a value, assert using (ConcreteError): %v\n", valueWithValue); // true
fmt.Printf("Returning a value, assert using (*ConcreteError): %v\n", valueWithPointer); // false
}
So if my understanding is correct, the user needs to know how the struct is returned to correctly assert its type?
I'm going to guess and assume the standard practice in golang is to always return a pointer to a struct(i.e like *PathError)?
link to play: here
So if my understanding is correct, the user needs to know how the struct is returned to correctly assert its type?
It depends. If you need the type assertion to pass - you surely need to know the exact type of the value. a and *a are different types.
I'm going to guess and assume the standard practice in golang is to always return a pointer to a struct(i.e like *PathError)?
No.
In go there are functions which return two values or more values, commonly one is an error. Suppose that I want to store the first return value into an already initialized variable, but I would like to initialize the variable to contain the error inline. Is there a way to do this?
For example, say I had this code
var a int
//This code doesn't compile because err doesn't exist
a, err = SomeFuncWithTwoReturnValues()
//This code doesn't compile either
a, err := SomeFuncWithTwoReturnValues()
I know you could do this, but I was hoping there was a way to do it all inline
var a int
var err error
a, err = SomeFuncWithTwoReturnValues()
or
a, err := SomeFuncWithTwoReturnValues()
EDIT: The code above actually compiles, so I looked back at my code to drill down more and have created a quick sample that actually replicates the problem (not just in my mind...).
package main
func myfunc() (int, int) {
return 1, 1
}
func main() {
a := make([]int, 1)
a[0], b := myfunc()
a[0] = b
}
Compiler says main.go|9| non-name a[0] on left side of :=. If I make it = instead of := though then b is never created. I get the feeling that there is not shorthand way to do it though.
As you've mentioned in the comments, you'll need to use the = operator in order to assign to a variable you've already declared. The := operator is used to simultaneously declare and assign a variable. The two are the same:
var x int
x = 5
//is the same as
x := 5
This solution will at least compile:
package main
func myfunc() (int, int) {
return 1, 1
}
func main() {
var b int
a := make([]int, 1)
a[0], b = myfunc()
a[0] = b
}
To answer your question, I don't think there is a way to simultaneously use an undeclared and a declared variable when returning multiple values. That would be trying to use two different operators simultaneously.
Edit: just saw your example from the code that compiles, so it appears you're already familiar with go's assignment operators. I'll leave the example up anyway.
Golang is not a very consistent language. This is a good example. At the beginning I was confused and it would be much simpler if they would always allow the := operator. The compiler is smart enough to detect already declared variables:
package main
import "fmt"
func testFunc() (int,error) {
return 42,fmt.Errorf("Test Error")
}
func main() {
number1,err := testFunc() // OK
number2,err := testFunc() // OK, even if err is already defined
number1,err = testFunc() // OK
// number1,err := testFunc() // ERROR: no new variables on left side of :=
fmt.Println(number1,number2,err)
}
Playground Link: https://play.golang.org/p/eZVB-kG6RtX
It's not consistent, because golang allows you to use := for already declared variables if you assign to them while also introducing a new variable. So the compiler can detect that variables already exists and skip their declaration. But the golang developers decided to allow that only if you introduce at least one new value. The last example shows that.
I ran into this situation like this:
package main
import "os"
func main() {
var cache struct { dir string }
// undefined: err
cache.dir, err = os.UserCacheDir()
// non-name cache.dir on left side of :=
cache.dir, err := os.UserCacheDir()
if err != nil {
panic(err)
}
println(cache.dir)
}
as you discovered, this issue does not have a clean solution. You can declare
an extra variable:
dir, err := os.UserCacheDir()
if err != nil {
panic(err)
}
cache := userCache{dir}
Or, while more verbose, you can declare the error beforehand. This can save
memory, as Go does not use a Rust ownership model:
var (
cache struct { dir string }
err error
)
cache.dir, err = os.UserCacheDir()
As mention in the spec, while using:=, if one of the variables is new, then the old one will just be assigned with the new data.
Unlike regular variable declarations, a short variable declaration may redeclare variables provided they were originally declared earlier in the same block (or the parameter lists if the block is the function body) with the same type, and at least one of the non-blank variables is new. As a consequence, redeclaration can only appear in a multi-variable short declaration. Redeclaration does not introduce a new variable; it just assigns a new value to the original.
field1, offset := nextField(str, 0)
field2, offset := nextField(str, offset) // redeclares offset
As mentioned by the other answers you cannot use assignment and declaration in the same return statement. You have to use either.
However I guess the main reason for your question is cleaning up the code so you don't have to declare an extra err variable above the method or function statement.
You can solve this in two ways:
Declare a global var err error variable and use it in the assignment:
var err error
func MyFunc(someInput string) {
var a int
a, err = someOtherFunction()
}
If your method or function returns an error you can use the declared return variable
func MyFunc(someInput string) (err error) {
var a int
a, err = someOtherFunction()
return
}
I mainly have the problem in methods when I want to assign something to a struct member, e.g.:
type MyStruct struct {
so string
}
func (m *MyStruct) SomeMethod() (err error) {
m.so, err = SomeFunction()
// handle error and continue or return it
return
}
I'm writing code that allows data access from a database. However, I find myself repeating the same code for similar types and fields. How can I write generic functions for the same?
e.g. what I want to achieve ...
type Person{FirstName string}
type Company{Industry string}
getItems(typ string, field string, val string) ([]interface{}) {
...
}
var persons []Person
persons = getItems("Person", "FirstName", "John")
var companies []Company
cs = getItems("Company", "Industry", "Software")
So you're definitely on the right track with the idea of returning a slice of nil interface types. However, you're going to run into problems when you try accessing specific members or calling specific methods, because you're not going to know what type you're looking for. This is where type assertions are going to come in very handy. To extend your code a bit:
getPerson(typ string, field string, val string) []Person {
slice := getItems(typ, field, val)
output := make([]Person, 0)
i := 0
for _, item := range slice {
// Type assertion!
thing, ok := item.(Person)
if ok {
output = append(output, thing)
i++
}
}
return output
}
So what that does is it performs a generic search, and then weeds out only those items which are of the correct type. Specifically, the type assertion:
thing, ok := item.(Person)
checks to see if the variable item is of type Person, and if it is, it returns the value and true, otherwise it returns nil and false (thus checking ok tells us if the assertion succeeded).
You can actually, if you want, take this a step further, and define the getItems() function in terms of another boolean function. Basically the idea would be to have getItems() run the function pass it on each element in the database and only add that element to the results if running the function on the element returns true:
getItem(critera func(interface{})bool) []interface{} {
output := make([]interface{}, 0)
foreach _, item := range database {
if criteria(item) {
output = append(output, item)
}
}
}
(honestly, if it were me, I'd do a hybrid of the two which accepts a criteria function but also accepts the field and value strings)
joshlf13 has a great answer. I'd expand a little on it though to maintain some additional type safety. instead of a critera function I would use a collector function.
// typed output array no interfaces
output := []string{}
// collector that populates our output array as needed
func collect(i interface{}) {
// The only non typesafe part of the program is limited to this function
if val, ok := i.(string); ok {
output = append(output, val)
}
}
// getItem uses the collector
func getItem(collect func(interface{})) {
foreach _, item := range database {
collect(item)
}
}
getItem(collect) // perform our get and populate the output array from above.
This has the benefit of not requiring you to loop through your interface{} slice after a call to getItems and do yet another cast.