I'm wondering why it's not possible to do the following in go:
func main() {
stuff := []string{"baz", "bla"}
foo("bar", stuff...)
}
func foo(s ...string) {
fmt.Println(s)
}
In my understanding, slice... "explodes" the slice so it can be used for multi argument function calls. So the above example should actually expand to foo("bar", "baz", "bla").
foo(stuff...) works as expected, no surprises here, but in the example above, the compiler complains about too many arguments.
Is this a desired limitation? I'm coming from a ruby background where a foo("bar", *stuff) is perfectly fine (and is, at least in my book, the same thing), that's why this surprises me.
The value for a variadic argument can be specified either by enumerating the elements, or using an existing slice, specified by its name followed by ....
You want to mix the 2 possible ways which is not permitted by the Go Language Specification (Passing arguments to ... parameters).
If the first form is used (enumerating the elements):
The value passed [as the variadic parameter] is a new slice of type []T with a new underlying array whose successive elements are the actual arguments.
If the latter is used (passing an existing slice followed by ...) no new slice is created, the one you pass is used as is. And the passed slice can only be used to specify the value of one – the final – variadic parameter. Attempting to pass both a single element and a slice will not match the signature (the parameter list in this case) of your function and you'll get an error:
too many arguments in call to foo
There is no actual "exploding" involved in Go, the term is just used in other languages to help visualize that the passed array or slice will not be an element of the variadic parameter but will be the value of variadic parameter itself.
Mixing the 2 would require to allocate a new slice because obviously the existing slice cannot be used.
The ugly way to get this to work is make it into a new variadic.
foo(append([]string{"bar"}, stuff...)...)
And if the order doesn't matter:
foo(append(stuff, "bar")...)
https://play.golang.org/p/mY6y0vScfPB
The specification on this is at the "Passing arguments to ... parameters":
If f is variadic with a final parameter p of type ...T, then within f the type of p is equivalent to type []T.
If f is invoked with no actual arguments for p, the value passed to p is nil.
Otherwise, the value passed is a new slice of type []T with a new underlying array whose successive elements are the actual arguments, which all must be assignable to T.
In your case, where stuff... works:
If the final argument is assignable to a slice type []T, it may be passed unchanged as the value for a ...T parameter if the argument is followed by .... In this case no new slice is created.
But "bar", stuff... doesn't match either case specified above.
T, []T doesn't match f([]T).
I ran into this situation when preparing arguments to feed to external commands. If possible, just build an one argument slice, then you don't have to worry about combining scalars with slices when it's time to call the function:
package main
import "os/exec"
func main() {
stuff := []string{"bar"}
stuff = append(stuff, "baz", "bla")
exec.Command("name", stuff...).Run()
}
Related
Coming from Nodejs, I could do something like:
// given an array `list` of objects with a field `fruit`:
fruits = list.map(el => el.fruit) # which will return an array of fruit strings
Any way to do that in an elegant one liner in golang?
I know I can do it with a range loop, but I am looking for the possibility of a one liner solution
In Go, arrays are inflexible (because their length is encoded in their type) and costly to pass to functions (because a function operates on copies of its array arguments). I'm assuming you'd like to operate on slices rather than on arrays.
Because methods cannot take additional type arguments, you cannot simply declare a generic Map method in Go. However, you can define Map as a generic top-level function:
func Map[T, U any](ts []T, f func(T) U) []U {
us := make([]U, len(ts))
for i := range ts {
us[i] = f(ts[i])
}
return us
}
Then you can write the following code,
names := []string{"Alice", "Bob", "Carol"}
fmt.Println(Map(names, utf8.RuneCountInString))
which prints [5 3 5] to stdout (try it out in this Playground).
Go 1.18 saw the addition of a golang.org/x/exp/slices package, which provides many convenient operations on slices, but a Map function is noticeably absent from it. The omission of that function was the result of a long discussion in the GitHub issue dedicated to the golang.org/x/exp/slices proposal; concerns included the following:
hidden cost (O(n)) of operations behind a one-liner
uncertainty about error handling inside Map
risk of encouraging a style that strays too far from Go's traditional style
Russ Cox ultimately elected to drop Map from the proposal because it's
probably better as part of a more comprehensive streams API somewhere else.
Researching the interface value in go - I found a great (maybe outdated) article by Russ Cox.
According to it:
The itable begins with some metadata about the types involved and then becomes a list of function pointers.
The implementation for this itable should be the one from src/runtime/runtime2.go:
type itab struct {
inter *interfacetype
_type *_type
hash uint32 // copy of _type.hash. Used for type switches.
_ [4]byte
fun [1]uintptr // variable sized. fun[0]==0 means _type does not implement inter.
}
First confusing thing is - how is an array - variable sized?
Second, assuming that we have a function pointer at index 0 for a method that satisfies the interface, where could we store a second/third/... function pointer?
The compiled code and runtime access fun as if the field is declared fun [n]uintpr where n is the number of methods in the interface. The second method is stored at fun[1], the third at fun[2] and so on. The Go Language does not have a variable size array feature like this, but unsafe shenanigans can be used to simulate the feature.
Here's how itab is allocated:
m = (*itab)(persistentalloc(unsafe.Sizeof(itab{})+uintptr(len(inter.mhdr)-1)*goarch.PtrSize, 0, &memstats.other_sys))
The function persistentalloc allocates memory. The first argument to the function is the size to allocate. The expression inter.mhdr is the number of methods in the interface.
Here's code that creates a slice on the variable size array:
methods := (*[1 << 16]unsafe.Pointer)(unsafe.Pointer(&m.fun[0]))[:ni:ni]
The expression methods[i] refers to the same element as m.fun[i] in a hypothetical world where m.fun is a variable size array with length > i. Later code uses normal slice syntax with methods to access the variable size array m.fun.
I am implementing a bit-vector in Go:
// A bit vector uses a slice of unsigned integer values or “words,”
// each bit of which represents an element of the set.
// The set contains i if the ith bit is set.
// The following program demonstrates a simple bit vector type with these methods.
type IntSet struct {
words []uint64 //uint64 is important because we need control over number and value of bits
}
I have defined several methods (e.g. membership test, adding or removing elements, set operations like union, intersection etc.) on it which all have a pointer receiver. Here is one such method:
// Has returns true if the given integer is in the set, false otherwise
func (this *IntSet) Has(m int) bool {
// details omitted for brevity
}
Now, I need to return an empty set that is a true constant, so that I can use the same constant every time I need to refer to an IntSet that contains no elements. One way is to return something like &IntSet{}, but I see two disadvantages:
Every time an empty set is to be returned, a new value needs to be allocated.
The returned value is not really constant since it can be modified by the callers.
How do you define a null set that does not have these limitations?
If you read https://golang.org/ref/spec#Constants you see that constants are limited to basic types. A struct or a slice or array will not work as a constant.
I think that the best you can do is to make a function that returns a copy of an internal empty set. If callers modify it, that isn't something you can fix.
Actually modifying it would be difficult for them since the words inside the IntSet are lowercase and therefore private. If you added a value next to words like mut bool you could add a if mut check to every method that changes the IntSet. If it isn't mutable, return an error or panic.
With that, you could keep users from modifying constant, non-mutable IntSet values.
What does ... mean in this context in Go?
ids = append(ids[:index], ids[index+1:]...)
I have read this great Q&A:
Do three dots (which is called wildcard?) contain multiple meanings?
about what ... means in some situations, but I don't understand what it means in the situation above.
Some languages (C, Python, ...) accept variadic arguments. Basically, you allow the client of a function to pass a number of arguments, without specifying how many. Since the function will still need to process these arguments one way or another, they are usually converted to a collection of some sort. For instance, in Python:
def foo(x, *args): # * is used for variadic arguments
return len(args)
>>> foo(1) # Passed the required argument, but no varargs
0
>>> foo(1, 2, 3) # Passed the required, plus two variadic arguments
2
>>> foo(1, 2, 3, 4, 5, 6) # required + 5 args, etc...
5
Now one obvious problem of that approach is that a number of arguments is quite a fuzzy concept as far as types are concerned. C uses pointers, Python doesn't really care about types that much in the first place, and Go takes the decision of restricting it to a specified case: you pass a slice of a given type.
This is nice because it lets the type-system do its thing, while still being quite flexible (in particular, the type in question can be an interface, so you can pass different 'actual types' as long as the function knows how to process these.
The typical example would be the Command function, which executes a program, passing it some arguments:
func Command(name string, arg ...string) *Cmd
It makes a lot of sense here, but recall that variadic arguments are just a convenient way to pass slices. You could have the exact same API with:
func Command(name string, args []string) *Cmd
The only advantage of the first is that it lets you pass no argument, one argument, several... without having to build the slice yourself.
What about your question then ?
Sometimes, you do have a slice, but need to call a variadic function. But if you do it naively:
my_args_slice := []string{"foo", "bar"}
cmd := Command("myprogram", my_args_slice)
The compiler will complain that it expects string(s), but got a slice instead ! What you want to tell it is that it doesn't have to 'build the slice in the back', because you have a slice already. You do that by using this ellipsis:
my_args_slice := []string{"foo", "bar"}
cmd := Command("myprogram", my_args_slice...) // <- Interpret that as strings
The append function, despite being built-in and special, follows the same rules. You can append zero, one, or more elements to the slice. If you want to concatenate slices (i.e. you have a 'slice of arguments' already), you similarly use the ellipsis to make it use it directly.
It unpacks slice.
ids[:index] is a short form of ids[0:index]
ids[index+1:] is a short form of ids[index+1:len(ids)-1]
So, your example ids = append(ids[:index], ids[index+1:]...) translates to
//pseudocode
ids = append(ids[0:index], ids[index+1], ids[index+2], ids[index+3], ..., ids[len(ids)-2], ids[len(ids)-1])
I've been trying to implement a function which can randomly select an element from any type of slice (like python's random.choice function)
func RandomChoice(a []interface{}, r *rand.Rand) interface{} {
i := r.Int()%len(a)
return a[i]
}
However, when I try to pass in a slice of type []float32 into the first argument this error occurs:
cannot use my_array (type []float32) as type []interface {} in function argument
is this a fundemental misuse of interface{}? is there some better way to do this?
Re: is there some better way to do this?
IMO there is. The OP approach is inefficient wrt to simple:
var v []T
...
// Select a random element of 'v'
e := v[r.Intn(len(v))]
...
Note that both of the approaches will panic for len(v) == 0 until a pre-check for this is made.
Using reflection:
func RandomChoice(slice interface{}, r *rand.Rand) interface{} {
x := reflect.ValueOf(slice)
return x.Index(r.Intn(x.Len())).Interface()
}
From the language specification:
Two types are either identical or different.
Two named types are identical if their type names originate in the
same TypeSpec. A named and an unnamed type are always different. Two
unnamed types are identical if the corresponding type literals are
identical, that is, if they have the same literal structure and
corresponding components have identical types. In detail:
Two array types are identical if they have identical element types and
the same array length.
Two slice types are identical if they have
identical element types.
Two struct types are identical if they have the same sequence of fields, and if corresponding fields have the same names, and identical
types, and identical tags. Two anonymous fields are considered to have
the same name. Lower-case field names from different packages are
always different.
Two pointer types are identical if they have identical base types.
Two function types are identical if they have the same number of parameters and result values, corresponding parameter and result types
are identical, and either both functions are variadic or neither is.
Parameter and result names are not required to match.
Two interface types are identical if they have the same set of methods with the same names and identical function types. Lower-case
method names from different packages are always different. The order
of the methods is irrelevant.
Two map types are identical if they have identical key and value types.
Two channel types are identical if they have identical value types and the same direction.
And:
A value x is assignable to a variable of type T ("x is assignable to
T") in any of these cases:
x's type is identical to T.
x's type V and T have identical underlying types and at least one of V or T is not a named type.
T is an interface type and x implements T.
x is a bidirectional channel value, T is a channel type, x's type V and T have identical element types, and at least one of V or T is not
a named type.
x is the predeclared identifier nil and T is a pointer, function, slice, map, channel, or interface type.
x is an untyped constant representable by a value of type T.
Any value may be assigned to the blank identifier.
The combination of these two results that you can't assign a []MyType to an []interface{}.
Well, I can hardly believe after all the searching, that the first listed related question was: Type converting slices of interfaces in go which pretty much contains the answer.
Changing RandomChoice to use the InterfaceSlice function described in the answer to that question yields:
func RandomChoice(slice interface{}, r *rand.Rand) interface{} {
islice := InterfaceSlice(slice)
i := r.Int()%len(islice)
return islice[i]
}
although apparently this answer is not very well performing, because it requires the entire slice to be converted to []interface{}...