I have a struct with a slice member, and a method to expose this slice. But I don't want the caller being able to change the content of the slice. If I do this:
type A struct {
slice []int
}
func (a *A) list() []int {
return a.slice
}
it is not safe, as the content can be easily modified:
a := A{[]int{1, 2, 3}}
_ = append(a.list()[:2], 4)
fmt.Println(a.list()) // [1 2 4]
Obviously I can let list() return a copy of the slice to avoid this:
func (a *A) list() []int {
return append([]int{}, a.slice...)
}
but that means every time when I just want to iterate through the slice I created a copy, which seems wasteful. Is there a way to do this without unnecessary copying?
As soon as you provide this slice to an external caller by returning it, it can be modified. If copying isn't acceptable for performance reasons, you can implement a visitor:
func (a *A) Visit(f func(int)) {
for _, v := range a.slice {
f(v)
}
}
This doesn't expose the slice at all, and allows client code to see all items in the slice once. If the items aren't pointers, or other mutable types, this is effectively read-only as the visitor callback will receive a copy of the value.
Optionally, the visitor could return a boolean in case you want to stop the iteration early.
func (a *A) Visit(f func(int) bool) {
for _, v := range a.slice {
if !f(v) {
return
}
}
}
Related
Is there a way to doing this automatically ?
package main
import "fmt"
func main() {
var a []string
a = append(a, "this", "this2", "this3")
increaseArguments(a)
a = append(a, "this4")
increaseArguments(a)
}
func increaseArguments(b []string) {
// I want, when i add new element to slice i want this function act as this
// fmt.Println(b[0],b[1], b[2], b[3])
fmt.Println(b[0], b[1], b[2])
}
Instead of adding b[3] as argument to fmt.Println is there a way to add it automatically ?
Note that if b would be of type []any, you could pass it as the value of the variadic parameter of fmt.Println():
fmt.Println(b...)
But since b is of type []string, you can't.
But if you transform b into a []any slice, you can. You can use this helper function to do it:
func convert[T any](x []T) []any {
r := make([]any, len(x))
for i, v := range x {
r[i] = v
}
return r
}
And then:
func increaseArguments(b []string) {
fmt.Println(convert(b)...)
}
This will output (try it on the Go Playground):
this this2 this3
this this2 this3 this4
Note: creating a new slice in convert() will not make this solution any slower, because passing values explicitly (like fmt.Println(b[0], b[1], b[2])) also implicitly creates a slice.
See related question: How to pass multiple return values to a variadic function?
What cast / assertion need I do in Go in order to pass to a function expecting a generic function like func(interface{}) interface{}, a more specific function like func(int) int instead?
For example, in code like this, fooA can be passed to MakeExclamer, but not fooB:
func MakeExclamer(foo func (interface{}) interface{}, n int) func () {
return func() {
fmt.Printf("%v!!!", foo(n))
}
}
func fooA(x interface{}) interface{} {
return x.(int)*2
}
func fooB(x int) int {
return x * 10
}
func main() {
exclamerA := MakeExclamer(fooA, 12)
exclamerA()
exclamerB := MakeExclamer(fooB, 66)
// >> cannot use fooB (type func(int) int) as type func(interface {}) interface {} in argument to MakeExclamer
exclamerB()
}
(Go Playground link: https://play.golang.org/p/xGzfco0IAG)
I'm not interested much in alternative code structure patterns, since this is how I want it to work: a specific function should be passed to a general function transformer (accepting function of type Any -> Any) that will return another general function (Any -> Any). This may not be idiomatic in Go, but it is the pattern that I want my code to follow.
To use type assertions, every possible type must be enumerated in MakeExclamer:
func MakeExclamer(fn interface{}, arg interface{}) func() {
switch fn := fn.(type) {
case func(int) int:
return func() {
fmt.Printf("%v!!!\n", fn(arg.(int)))
}
case func(interface{}) interface{}:
return func() {
fmt.Printf("%v!!!\n", fn(arg))
}
default:
panic("not supported")
}
}
To accept a function of any type, the fn argument is declared as type interface{}. The code uses a type switch to handle the different function types.
playground example
Reflection can be used to write a more general function.
func MakeExclamer(fn interface{}, arg interface{}) func() {
fnr := reflect.ValueOf(fn)
argr := reflect.ValueOf(arg)
return func() {
resultr := fnr.Call([]reflect.Value{argr})
fmt.Printf("%v!!!\n", resultr[0].Interface())
}
}
playground example
First things first : When it comes to typing in Go, everything is theoretically possible. That's because even though the compiler does a lot of checks at compile-time, it is possible to change the runtime... at runtime. So-called runtime hacks, where you dynamically manipulate runtime structs that you're NOT supposed to handle.
Now, you have an interesting question, whose answer doesn't include the need to use the 'unsafe' package. However, the way I found of generalizing a function involves heavy reflection.
How to call a function (via reflection) ?
The documentation for the reflect package can be found here.
So, like all elements in Golang, functions have a Type. Without going through all fields, functions do take an array of arguments and produce an array of results. It is possible to investigate the Type of arguments and results through the In(int) and Out(int) method.
func investigate(fn interface{}) {
fnType := reflect.TypeOf(fn)
for idx := 0; idx < fnType.NumIn(); idx ++ {
fmt.Printf("Input arg %d has type %v\n", idx, fnType.In(idx))
}
for idx := 0; idx < fnType.NumOut(); idx ++ {
fmt.Printf("Output arg %d has type %v\n", idx, fnType.Out(idx))
}
}
We won't use this code. However, two important things are to be noted at this point :
The generic type under which a function can be passed around without caring about its type is interface{}. Something like "func(interface{}) interface{}" is not a generalization of a function, it is already a concrete type. Hence, "func(interface{}) interface{}" is not a generalization of "func(int) int", those are two different function types entirely. This is why you can't use type assertions/cast to convert from one function type to another.
A function can be represented as something that takes an input array and produces and output array.
Now, in order to call a function, you have to get not its Type, but its Value. Once you get its value, you can call it using an array of arguments, which must all be Values.
The prototype is:
func (v Value) Call(in []Value) []Value
Using this method, it is possible to call any function.
The code
So, the only thing you need is to convert whichever arguments array you have to an array of Values, then you will be able to call your function.
Here is your code:
package main
import (
"fmt"
"reflect"
)
func MakeExclamer(foo interface{}, n int) func() {
exclamer := generalize(foo, n)
return func() {
fmt.Printf("%v!!!\n", exclamer())
}
}
func fooA(x interface{}) interface{} {
return x.(int) * 2
}
func fooB(x int) int {
return x * 10
}
func generalize(implem interface{}, args ...interface{}) func() interface{} {
valIn := make([]reflect.Value, len(args), len(args))
fnVal := reflect.ValueOf(implem)
for idx, elt := range args {
valIn[idx] = reflect.ValueOf(elt)
}
ret := func() interface{} {
res := fnVal.Call(valIn)
// We assume the function produces exactly one result
return res[0].Interface()
}
return ret
}
func main() {
exclamerA := MakeExclamer(fooA, 12)
exclamerA()
exclamerB := MakeExclamer(fooB, 66)
exclamerB()
}
Playground
The important bit is the generalize function which makes the translation between your arguments and an array of Values, then returns a new function whith all parameters already filled.
Do not hesitate if you need any precision !
Let's say I have many goroutines doing something like this:
func (o *Obj) Reader() {
data := o.data;
for i, value := range data {
log.Printf("got data[%v] = %v", i, value)
}
}
And one doing this:
func (o *Obj) Writer() {
o.data = append(o.data, 1234)
}
If data := o.data means the internal structure of the slice is copied, this looks like it could be safe, because I'm never modifying anything in the accessible range of the copy. I'm either setting one element outside of the range and increasing the length, or allocating a completely new pointer, but the reader would be operating on the original one.
Are my assumptions correct and this is safe to do?
I'm aware that slices are not meant to be "thread-safe" in general, the question is more about how much does slice1 := slice2 actually copy.
The code in the question is unsafe because it reads a variable in one goroutine and modifies the variable in another goroutine without synchronization.
Here's one way to make the code safe:
type Obj struct {
mu sync.Mutex // add mutex
... // other fields as before
}
func (o *Obj) Reader() {
o.mu.Lock()
data := o.data
o.mu.Unlock()
for i, value := range data {
log.Printf("got data[%v] = %v", i, value)
}
}
func (o *Obj) Writer() {
o.mu.Lock()
o.data = append(o.data, 1234)
o.mu.Unlock()
}
It's safe for Reader to range over the local slice variable data because the Writer does not modify the local variable data or the backing array visible through the local variable data.
A bit late to the party, but if your use-case is frequent reads and infrequent writes, atomic.Value is designed to solve this:
type Obj struct {
data atomic.Value // []int
mu sync.Mutex
}
func (o *Obj) Reader() {
data := o.data.Load().([]int);
for i, value := range data {
log.Printf("got data[%v] = %v", i, value)
}
}
func (o *Obj) Writer() {
o.mu.Lock()
data := o.data.Load().([]int);
data = append(o.data, 1234)
o.data.Store(data)
o.mu.Unlock()
}
This will generally be much faster than either a Mutex or an RWMutex.
Note that this will only work with data this is effectively a copy, which it is in this case because you can safely maintain a reference to the previous slice when appending, as append() creates a new copy if it extends. If you're mutating the elements of the slice, or using another data structure, this approach is not safe.
I want to know is there a generic way to write code to judge whether a slice contains an element, I find it will frequently useful since there is a lot of logic to fist judge whether specific elem is already in a slice and then decide what to do next. But there seemed not a built-in method for that(For God's sake, why?)
I try to use interface{} to do that like:
func sliceContains(slice []interface{}, elem interface{}) bool {
for _, item := range slice {
if item == elem {
return true
}
}
return false
}
I thought interface{} is sort of like Object of Java, but apparently, I was wrong. Should I write this every time meet with a new struct of slice? Isn't there a generic way to do this?
You can do it with reflect, but it will be MUCH SLOWER than a non-generic equivalent function:
func Contains(slice, elem interface{}) bool {
sv := reflect.ValueOf(slice)
// Check that slice is actually a slice/array.
// you might want to return an error here
if sv.Kind() != reflect.Slice && sv.Kind() != reflect.Array {
return false
}
// iterate the slice
for i := 0; i < sv.Len(); i++ {
// compare elem to the current slice element
if elem == sv.Index(i).Interface() {
return true
}
}
// nothing found
return false
}
func main(){
si := []int {3, 4, 5, 10, 11}
ss := []string {"hello", "world", "foo", "bar"}
fmt.Println(Contains(si, 3))
fmt.Println(Contains(si, 100))
fmt.Println(Contains(ss, "hello"))
fmt.Println(Contains(ss, "baz"))
}
How much slower? about x50-x60 slower:
Benchmarking against a non generic function of the form:
func ContainsNonGeneic(slice []int, elem int) bool {
for _, i := range slice {
if i == elem {
return true
}
}
return false
}
I'm getting:
Generic: N=100000, running time: 73.023214ms 730.23214 ns/op
Non Generic: N=100000, running time: 1.315262ms 13.15262 ns/op
You can make it using the reflect package like that:
func In(s, e interface{}) bool {
slice, elem := reflect.ValueOf(s), reflect.ValueOf(e)
for i := 0; i < slice.Len(); i++ {
if reflect.DeepEqual(slice.Index(i).Interface(), elem.Interface()) {
return true
}
}
return false
}
Playground examples: http://play.golang.org/p/TQrmwIk6B4
Alternatively, you can:
define an interface and make your slices implement it
use maps instead of slices
just write a simple for loop
What way to choose depends on the problem you are solving.
I'm not sure what your specific context is, but you'll probably want to use a map to check if something already exists.
package main
import "fmt"
type PublicClassObjectBuilderFactoryStructure struct {
Tee string
Hee string
}
func main() {
// Empty structs occupy zero bytes.
mymap := map[interface{}]struct{}{}
one := PublicClassObjectBuilderFactoryStructure{Tee: "hi", Hee: "hey"}
two := PublicClassObjectBuilderFactoryStructure{Tee: "hola", Hee: "oye"}
three := PublicClassObjectBuilderFactoryStructure{Tee: "hi", Hee: "again"}
mymap[one] = struct{}{}
mymap[two] = struct{}{}
// The underscore is ignoring the value, which is an empty struct.
if _, exists := mymap[one]; exists {
fmt.Println("one exists")
}
if _, exists := mymap[two]; exists {
fmt.Println("two exists")
}
if _, exists := mymap[three]; exists {
fmt.Println("three exists")
}
}
Another advantage of using maps instead of a slice is that there is a built-in delete function for maps. https://play.golang.org/p/dmSyyryyS8
If you want a rather different solution, you might try the code-generator approach offered by tools such as Gen. Gen writes source code for each concrete class you want to hold in a slice, so it supports type-safe slices that let you search for the first match of an element.
(Gen also offers a few other kinds of collection and allows you to write your own.)
Go has stumped me again. Hopefully someone can help. I've created a slice (mySlice) that contains pointers to structs (myStruct).
The problem is the "Remove" method. When we're inside "Remove" everything is fine, but once we return, the slice size hasn't changed, and so we see the last element listed twice.
I originally tried writing "Remove" using the same pattern used in the "Add" method, but it wouldn't compile and has been commented out.
I can get it to work by returning the newly created slice to the calling function, but I don't want to do this because mySlice (ms) is a singleton.
And if I hadn't asked enough already...
The code for the "Add" method is working, although I'm not sure how. From what I can gather "Add" is receiving a pointer to the slice header (the 3 item "struct"). From what I've read, the length and capacity of an slice don't get passed to methods (when passing by value), so perhaps passing a pointer to the slice allows the method to see and use the length and capacity thereby allowing us to "append". If this is true, then why doesn't the same pattern work in "Remove"?
Thanks very much for everyone's insights and help!
package main
import (
"fmt"
)
type myStruct struct {
a int
}
type mySlice []*myStruct
func (slc *mySlice) Add(str *myStruct) {
*slc = append(*slc, str)
}
//does not compile with reason: cannot slice slc (type *mySlice)
//func (slc *mySlice) Remove1(item int) {
// *slc = append(*slc[:item], *slc[item+1:]...)
//}
func (slc mySlice) Remove(item int) {
slc = append(slc[:item], slc[item+1:]...)
fmt.Printf("Inside Remove = %s\n", slc)
}
func main() {
ms := make(mySlice, 0)
ms.Add(&myStruct{0})
ms.Add(&myStruct{1})
ms.Add(&myStruct{2})
fmt.Printf("Before Remove: Len=%d, Cap=%d, Data=%s\n", len(ms), cap(ms), ms)
ms.Remove(1) //remove element 1 (which also has a value of 1)
fmt.Printf("After Remove: Len=%d, Cap=%d, Data=%s\n", len(ms), cap(ms), ms)
}
and the results...
Before Remove: Len=3, Cap=4, Data=[%!s(*main.myStruct=&{0}) %!s(*main.myStruct=&{1}) %!s(*main.myStruct=&{2})]
Inside Remove = [%!s(*main.myStruct=&{0}) %!s(*main.myStruct=&{2})]
After Remove: Len=3, Cap=4, Data=[%!s(*main.myStruct=&{0}) %!s(*main.myStruct=&{2}) %!s(*main.myStruct=&{2})]
You were right the first time with Remove1(). Remove gets a copy of the slice and therefore cannot change the length of the slice.
The issue in your remove function is that according to order of operations in Go, slicing comes before dereferencing.
The fix is to change *slc = append(*slc[:item], *slc[item+1:]...) to *slc = append((*slc)[:item], (*slc)[item+1:]...).
However I would recommend the following for readability and maintainability:
func (slc *mySlice) Remove1(item int) {
s := *slc
s = append(s[:item], s[item+1:]...)
*slc = s
}
Because append would not necessarily return the same address of reference to the slice, as Stephen Weinberg has pointed out.
Another way to workaround with this limitation is defining a struct that wraps the slice.
for example:
package main
import "fmt"
type IntList struct {
intlist []int
}
func (il *IntList) Pop() {
if len(il.intlist) == 0 { return }
il.intlist = il.intlist[:len(il.intlist)-1]
}
func (il *IntList) Add(i... int) {
il.intlist = append(il.intlist, i...)
}
func (il *IntList) String() string {
return fmt.Sprintf("%#v",il.intlist)
}
func main() {
intlist := &IntList{[]int{1,2,3}}
fmt.Println(intlist)
intlist.Pop()
fmt.Println(intlist)
intlist.Add([]int{4,5,6}...)
fmt.Println(intlist)
}
output:
[]int{1, 2, 3}
[]int{1, 2}
[]int{1, 2, 4, 5, 6}