Adding a func never called improves behavior? - go

The code below produces undesirable
[20010101 20010102].
When uncommenting the String func it produces better (but not my implementation):
[{20010101 1.5} {20010102 2.5}]
However that String func is never called.
I see that Date in DateValue is anonymous and therefore func (Date) String is being used by DateValue.
So my questions are:
1) Is this a language issue, a fmt.Println implementation issue, or
something else? Note: if I switch from:
func (*DateValue) String() string
to
func (DateValue) String() string
my function is at least called and panic ensues. So if I really want my method called I could do that, but assume DateValue is really a very large object which I only want to pass by reference.
2) What is a good strategy for mixing anonymous fields with
functionality like Stringer and json encoding that use reflection
under the covers? For example adding a String or MarshalJSON method
for a type that happens to be used as an anonymous field can cause
strange behavior (like you only print or encode part of the whole).
package main
import (
"fmt"
"time"
)
type Date int64
func (d Date) String() string {
t := time.Unix(int64(d),0).UTC()
return fmt.Sprintf("%04d%02d%02d", t.Year(), int(t.Month()), t.Day())
}
type DateValue struct {
Date
Value float64
}
type OrderedValues []DateValue
/*
// ADD THIS BACK and note that this is never called but both pieces of
// DateValue are printed, whereas, without this only the date is printed
func (dv *DateValue) String() string {
panic("Oops")
return fmt.Sprintf("DV(%s,%f)", dv.Date, dv.Value )
}
*/
func main() {
d1, d2 := Date(978307200),Date(978307200+24*60*60)
ov1 := OrderedValues{{ d1, 1.5 }, { d2, 2.5 }}
fmt.Println(ov1)
}

It's because you've passed in a slice of DateValues and not DateValue pointers. Since you've defined the String method for *DataValue, *DateValue is what fulfills the Stringer interface. This also prevents DateValue from fulfilling the Stringer interface via its anonymous Date member, because only one of either the value type (DateValue) or the pointer type (*DateValue) can be used to fulfill an interface. So, when fmt.Println is printing the contents of the slice, it sees that the elements are not Stringers, and uses the default struct formatting instead of the method you defined, giving [{20010101 1.5} {20010102 2.5}].
You can either make OrderedValues a []*DateValue or define func (dv DateValue) String() string instead of the pointer version.

Based on what #SteveM said, I distilled it to a simpler test case:
package main
import "fmt"
type Fooable interface {
Foo()
}
type A int
func (a A) Foo() { }
type B struct {
A
}
// Uncomment the following method and it will print false
//func (b *B) Foo() { }
func main() {
var x interface{} = B{}
_, ok := x.(Fooable)
fmt.Println(ok) // prints true
}
In other words, the Foo method is not part of the method set of B when the Foo method for *B is defined.
From reading the spec, I don't see a clear explanation of what is happening. The closest part seems to be in the section on selectors:
For a value x of type T or *T where T is not an interface type, x.f
denotes the field or method at the shallowest depth in T where there
is such an f.
The only way I can see this explaining what is going on is if when it is looking for a method Foo at shallowest depth in B, it takes into consideration the methods for *B too, for some reason (even though we are considering type B not *B); and the Foo in *B is indeed shallower than the Foo in A, so it takes that one as the candidate; and then it sees that that Foo doesn't work, since it's in *B and not B, so it gets rid of Foo altogether (even though there is a valid one inherited from A).
If this is indeed what is going on, then I agree with the OP in that this is very counter-intuitive that adding a method to *B would have the reverse consequence of removing a method from B.
Maybe someone more familiar with Go can clarify this.

Related

go generics: how to declare a type parameter compatible with another type parameter

I'm looking for a way to declare type compatibility between type parameters in Go generics constraints.
More specifically, I need to say some type T is compatible with another type U. For instance, T is a pointer to a struct that implements the interface U.
Below is a concrete example of what I want to accomplish:
NOTE: Please, do not answer with alternative ways to implement "array prepend". I've only used it as a concrete application of the problem I'm looking to solve. Focusing on the specific example digresses the conversation.
func Prepend[T any](array []T, values ...T) []T {
if len(values) < 1 { return array }
result := make([]T, len(values) + len(array))
copy(result, values)
copy(result[len(values):], array)
return result
}
The above function can be called to append elements of a given type T to an array of the same type, so the code below works just fine:
type Foo struct{ x int }
func (self *Foo) String() string { return fmt.Sprintf("foo#%d", self.x) }
func grow(array []*Foo) []*Foo {
return Prepend(array, &Foo{x: len(array)})
}
If the array type is different than the elements being added (say, an interface implemented by the elements' type), the code fails to compile (as expected) with type *Foo of &Foo{…} does not match inferred type Base for T:
type Base interface { fmt.Stringer }
type Foo struct{ x int }
func (self *Foo) String() string { return fmt.Sprintf("foo#%d", self.x) }
func grow(array []Base) []Base {
return Prepend(array, &Foo{x: len(array)})
}
The intuitive solution to that is to change the type parameters for Prepend so that array and values have different, but compatible types. That's the part I don't know how to express in Go.
For instance, the code below doesn't work (as expected) because the types of array and values are independent of each other. Similar code would work with C++ templates since the compatibility is validated after template instantiation (similar to duck typing). The Go compiler gives out the error invalid argument: arguments to copy result (variable of type []A) and values (variable of type []T) have different element types A and T:
func Prepend[A any, T any](array []A, values ...T) []A {
if len(values) < 1 { return array }
result := make([]A, len(values) + len(array))
copy(result, values)
copy(result[len(values):], array)
return result
}
I've tried making the type T compatible with A with the constraint ~A, but Go doesn't like a type parameter used as type of a constraint, giving out the error type in term ~A cannot be a type parameter:
func Prepend[A any, T ~A](array []A, values ...T) []A {
What's the proper way to declare this type compatibility as generics constraints without resorting to reflection?
This is a limitation of Go's type parameter inference, which is the system that tries to automatically insert type parameters in cases where you don't define them explicitly. Try adding in the type parameter explicitly, and you'll see that it works. For example:
// This works.
func grow(array []Base) []Base {
return Prepend[Base](array, &Foo{x: len(array)})
}
You can also try explicitly converting the *Foo value to a Base interface. For example:
// This works too.
func grow(array []Base) []Base {
return Prepend(array, Base(&Foo{x: len(array)}))
}
Explanation
First, you should bear in mind that the "proper" use of type parameters is to always include them explicitly. The option to omit the type parameter list is considered a "nice to have", but not intended to cover all use cases.
From the blog post An Introduction To Generics:
Type inference in practice
The exact details of how type inference works are complicated, but using it is not: type inference either succeeds or fails. If it succeeds, type arguments can be omitted, and calling generic functions looks no different than calling ordinary functions. If type inference fails, the compiler will give an error message, and in those cases we can just provide the necessary type arguments.
In adding type inference to the language we’ve tried to strike a balance between inference power and complexity. We want to ensure that when the compiler infers types, those types are never surprising. We’ve tried to be careful to err on the side of failing to infer a type rather than on the side of inferring the wrong type. We probably have not gotten it entirely right, and we may continue to refine it in future releases. The effect will be that more programs can be written without explicit type arguments. Programs that don’t need type arguments today won’t need them tomorrow either.
In other words, type inference may improve over time, but you should expect it to be limited.
In this case:
// This works.
func grow(array []*Foo) []*Foo {
return Prepend(array, &Foo{x: len(array)})
}
It is relatively simple for the compiler to match that the argument types of []*Foo and *Foo match the pattern []T and ...T by substitutingT = *Foo.
So why does the plain solution you gave first not work?
// Why does this not work?
func grow(array []Base) []Base {
return Prepend(array, &Foo{x: len(array)})
}
To make []Base and *Foo match the pattern []T and ...T, just substituting T = *Foo or T = Base provides no apparent match. You have to apply the rule that *Foo is assignable to the type Base to see that T = Base works. Apparently the inference system doesn't go the extra mile to try to figure that out, so it fails here.

Passing pointer to string when the function takes an interface?

I am passing a pointer to a string, to a method which takes an interface (I have multiple versions of the method, with different receivers, so I am trying to work with empty interfaces, so that I don't end up with a ton of boilerplate madness. Essentially, I want to populate the string with the first value in the slice. I am able to see the value get populated inside the function, but then for some reason, in my application which calls it, tha value doesn't change. I suspect this is some kind of pointer arithmetic problem, but could really use some help!
I have the following interface :
type HeadInterface interface{
Head(interface{})
}
And then I have the following functions :
func Head(slice HeadInterface, result interface{}){
slice.Head(result)
}
func (slice StringSlice) Head(result interface{}){
result = reflect.ValueOf(slice[0])
fmt.Println(result)
}
and... here is my call to the function from an application which calls the mehtod...
func main(){
test := x.StringSlice{"Phil", "Jessica", "Andrea"}
// empty result string for population within the function
var result string = ""
// Calling the function (it is a call to 'x.Head' because I lazily just called th import 'x')
x.Head(test, &result)
// I would have thought I would have gotten "Phil" here, but instead, it is still empty, despite the Println in the function, calling it "phil.
fmt.Println(result)
}
*NOTE : I am aware that getting the first element doesn't need to be this complicated, and could be slice[0] as a straight assertion, but this is more of an exercise in reusable code, and also in trying to get a grasp of pointers, so please don't point out that solution - I would get much more use out of a solution to my actual problem here * :)
As you said in your NOTE, I'm pretty sure this doesn't have to be this complicated, but to make it work in your context:
package main
import (
"fmt"
"reflect"
)
type HeadInterface interface {
Head(interface{})
}
func Head(slice HeadInterface, result interface{}) {
slice.Head(result)
}
type StringSlice []string
func (slice StringSlice) Head(result interface{}) {
switch result := result.(type) {
case *string:
*result = reflect.ValueOf(slice[0]).String()
fmt.Println("inside Head:", *result)
default:
panic("can't handle this type!")
}
}
func main() {
test := StringSlice{"Phil", "Jessica", "Andrea"}
// empty result string for population within the function
var result string = ""
// Calling the function (it is a call to 'x.Head' because I lazily just called th import 'x')
Head(test, &result)
// I would have thought I would have gotten "Phil" here, but instead, it is still empty, despite the Println in the function, calling it "phil.
fmt.Println("outside:", result)
}
The hard part about working with interface{} is that it's hard to be specific about a type's behavior given that interface{} is the most un-specific type. To modify a variable that you pass as a pointer to a function, you have to use the asterisk (dereference) (for example *result) on the variable in order to change the value it points to, not the pointer itself. But to use the asterisk, you have to know it's actually a pointer (something interface{} doesn't tell you) so that's why I used the type switch to be sure it's a pointer to a string.

What exactly does .(data_type) method called/do?

I came a cross a piece of code that used .(string) method. Not knowing what this is called I had difficulties searching for it.
Here is my try to understand it:
package main
import "fmt"
import "reflect"
func main(){
var b interface{}
b = "silly"
fmt.Println(reflect.TypeOf(b.(string))) // we know that b
// is a string
// at compile time
fmt.Println(reflect.TypeOf(b)) // we do not
}
Result:
string
string
However, I think that reflect.TypeOf takes place at run time, while .(string) would tell the compiler that b is indeed a string, and this could be used to tell the compiler that a variable is of certain type. Is my understanding right?
goplayground
b.(string) is called a type assertion. As written in Effective Go:
A type assertion takes an interface value and extracts from it a value of the specified explicit type.
So, yes, the value you get from a type assertion is not an interface value, but is of the explicit type. You can also test if the type assertion was successful by adding an untyped boolean value:
s, ok := b.(string) // s is of type string
if !ok {
// b did not contain a value of type string!
}
Edit:
To explain further to clear out any possible misunderstanding:
A type assertion doesn't "tell Go that b is a string" as you suggested. What it does is that it will, in run time, try to extract a string from b, and panic if b contains some other type (unless assigning the optional bool value).
The value that you get from the assertion will indeed be of type string, allowing you to do things like slicing (you cannot slice an interface value) or checking its len.
The previous answer is correct. But I submit this as more like what happens in practice. The .(type) syntax is usually used with type names in the cases of a switch. In this example, I (integer expr), B (bool expr), and Bop (binary op) are type names.
func commute (e Expr) (r Expr, d bool) {
switch exp:= e.(type) {
case I: r,d = exp,false
case B: r,d = exp,false
case Bop: r,d = Bop{exp.op, exp.right, exp.left},true
default: r,d = e,false
}
return
}
This isn't unsafe like a C cast, because inside the case statement you are guaranteed to have the matching type. I see this quite a bit when reading a channel, where the type of the channel is an interface that all the cases implement.

what exactly is interface(struc) and interface(struc).function

Trying to do go koan, i got stuck in understanding the interface(struct) syntax, what exactly
does it do ?
I came up with following fun program, which has further confused me on how is interface casting working :
package main
import "fmt"
type foo interface{ fn() }
type t struct { }
type q struct { }
func (_i t ) fn() { fmt.Print("t","\n") }
func (_i q ) fn() { fmt.Print("q","\n")}
func main() {
_j := t{}
_q := q{}
// This is alright ..
fmt.Print( _j.fn,"\n") //0x4015e0
fmt.Print( _q.fn,"\n") //0x401610
_j.fn() //t
_q.fn() //q
// both pointers same .. why ?
fmt.Print( foo(_j).fn,"\n") //0x401640
fmt.Print( foo(_q).fn,"\n") //0x401640
// but correct fns called .. how ?
foo(_j).fn() //t
foo(_q).fn() //q
// same thing again ...
_fj := foo(_j).fn
_fq := foo(_q).fn
// both pointers same .. as above
fmt.Print( _fj,"\n") //0x401640
fmt.Print( _fq,"\n") //0x401640
// correct fns called .. HOW !
_fj() //t
_fq() //q
}
The pointer are what i'm getting my machin, YMMV.
My question is .. what exactly does interface(struct) returns ?
and how does interface(struct).func , finds the original struct ...
is there some thunk/stub magic going on here?
From here: http://research.swtch.com/interfaces
what exactly does interface(struct) return?
It creates a new interface value (like the one you see on top in the graphic), wrapping a concrete struct value.
how does interface(struct).func find the original struct?
See the data field in the graphic. Most of the time this will be a pointer to an existing value. Sometimes it will contain the value itself if it fits, though.
In the itable you'll see a function table (where fun[0] is).
I assume that on your machine 0x401640 is the address of the respective pointers to fn, which is in that table for foo. Although this is best verified by someone working on the GC compiler suite.
Note that the behaviour you discovered is not strictly defined to be so. Compiler builders can take other approaches to implementing Go interfaces if they like to, as long as the language semantics are preserved.
Edit to answer questions in the comments:
package main
import "fmt"
type foo interface {
fn()
}
type t struct{}
type q struct{}
func (_i t) fn() { fmt.Print("t", "\n") }
func (_i q) fn() { fmt.Print("q", "\n") }
func main() {
_j := t{}
_j1 := t{}
fmt.Println(foo(_j) == foo(_j)) // true
fmt.Println(foo(_j) == foo(_j1)) // true
}
On the diagram you see 3 blocks:
The one on the left side labeled Binary is a concrete type instance, like your struct instances _j and _j1.
The one on the top center is an interface value, this one wraps (read: points to) a concrete value.
The block on the right lower side is the interface definition for Binary underlyings. This is where the jump table / call forwarding table is (itable).
_j and _j1 are two instances of the concrete type t. So there are two of the lower-left blocks somewhere in memory.
Now you decide to wrap both _j and _j1 in interfaces values of type foo; now you have 2 of the top-center blocks somewhere in memory, pointing back at _j and _j1.
In order for the interface value to remember what its underlying type is and where the methods of those types are it keeps a single instance of the lower-right block in memory, to which both interface values for _j and _j1 respectively point to.
In that block you have a jump table to forward method calls made on the interface values to the concrete underlying type's implementation. That's why both are the same.
It's worth mentioning that unlike Java and C++ (not sure about Python), all Go methods are static and the dot-call notation is only syntactic sugar. So _j and _j1 don't have different fn methods, it's the same exact method called with another implicit first parameter which is the receiver on which the method is called.

Why isn't my Stringer interface method getting invoked? When using fmt.Println

Suppose I have the following code:
package main
import "fmt"
type Car struct{
year int
make string
}
func (c *Car)String() string{
return fmt.Sprintf("{make:%s, year:%d}", c.make, c.year)
}
func main() {
myCar := Car{year:1996, make:"Toyota"}
fmt.Println(myCar)
}
When I call fmt.Println(myCar) and the object in question is a pointer, my String() method gets called properly. If, however the object is a value, my output is formatted using the default formatting built into Go and my code to format the said object is not called.
The interesting thing is in either case if I call myCar.String() manually it works properly whether my object is either a pointer or value.
How can I get my object formatted the way I want no matter if the object is value-based or pointer-based when used with Println?
I don't want to use a value method for String because then that means every time it's invoked the object is copied which seams unreasonable. And I don't want to have to always manually called .String() either because I'm trying to let the duck-typing system do it's work.
When calling fmt.Println, myCar is implicitly converted to a value of type interface{} as you can see from the function signature. The code from the fmt package then does a type switch to figure out how to print this value, looking something like this:
switch v := v.(type) {
case string:
os.Stdout.WriteString(v)
case fmt.Stringer:
os.Stdout.WriteString(v.String())
// ...
}
However, the fmt.Stringer case fails because Car doesn't implement String (as it is defined on *Car). Calling String manually works because the compiler sees that String needs a *Car and thus automatically converts myCar.String() to (&myCar).String(). For anything regarding interfaces, you have to do it manually. So you either have to implement String on Car or always pass a pointer to fmt.Println:
fmt.Println(&myCar)
Methods
Pointers vs. Values
The rule about pointers vs. values for receivers is that value methods
can be invoked on pointers and values, but pointer methods can only be
invoked on pointers. This is because pointer methods can modify the
receiver; invoking them on a copy of the value would cause those
modifications to be discarded.
Therefore, for your String method to work when invoked on both pointers and values, use a value receiver. For example,
package main
import "fmt"
type Car struct {
year int
make string
}
func (c Car) String() string {
return fmt.Sprintf("{make:%s, year:%d}", c.make, c.year)
}
func main() {
myCar := Car{year: 1996, make: "Toyota"}
fmt.Println(myCar)
fmt.Println(&myCar)
}
Output:
{make:Toyota, year:1996}
{make:Toyota, year:1996}
Define your fmt.Stringer on a pointer receiver:
package main
import "fmt"
type Car struct {
year int
make string
}
func (c *Car) String() string {
return fmt.Sprintf("{maker:%s, produced:%d}", c.make, c.year)
}
func main() {
myCar := Car{year: 1996, make: "Toyota"}
myOtherCar := &Car{year: 2013, make: "Honda"}
fmt.Println(&myCar)
fmt.Println(myOtherCar)
}
Playground
Output:
{maker:Toyota, produced:1996}
{maker:Honda, produced:2013}
Then, always pass a pointer to instances of Car to fmt.Println. This way a potentially expensive value copy is avoided under your control.
The OP further asked:
OP: [when a value receiver is used] "Does this basically mean that if I have a large struct, then every time it goes through Println it will be copied?"
The following experiment is evidence that the answer is "yes" (when a value receiver is used). Note that the String() method increments the year in this experiment, and check how this affects the printed output.
type Car struct {
year int
make string
}
func (c Car) String() string {
s := fmt.Sprintf("{ptr:%p, make:%s, year:%d}", c, c.make, c.year)
// increment the year to prove: is c a copy or a reference?
c.year += 1
return s
}
func main() {
myCar := Car{year: 1996, make: "Toyota"}
fmt.Println(&myCar)
fmt.Println(&myCar)
fmt.Println(myCar)
fmt.Println(myCar)
}
With a value receiver (c Car), the following printed output shows that Go makes value copies of the Car struct, because the year increment is not reflected in subsequent calls to Println:
{ptr:%!p(main.Car={1996 Toyota}), make:Toyota, year:1996}
{ptr:%!p(main.Car={1996 Toyota}), make:Toyota, year:1996}
{ptr:%!p(main.Car={1996 Toyota}), make:Toyota, year:1996}
{ptr:%!p(main.Car={1996 Toyota}), make:Toyota, year:1996}
Changing the receiver to a pointer (c *Car) but changing nothing else, the printed output becomes:
{ptr:0xc420094020, make:Toyota, year:1996}
{ptr:0xc420094020, make:Toyota, year:1997}
{1998 Toyota}
{1998 Toyota}
Even when a pointer is provided as argument in a call to Println, i.e. fmt.Println(&myCar), Go still makes a value copy of the Car struct when a value receiver is used. The OP wants to avoid value copies being made, and my conclusion is that only pointer receivers satisfy that requirement.
It's only related to implementation of fmt instead of Go however.
String() with pointer receiver would be invoked by https://github.com/davecgh/go-spew since spew print things in this way:
v = reflect.ValueOf(arg)
...
switch iface := v.Interface().(type) {
case fmt.Stringer:
defer catchPanic(w, v)
if cs.ContinueOnMethod {
w.Write(openParenBytes)
w.Write([]byte(iface.String()))
w.Write(closeParenBytes)
w.Write(spaceBytes)
return false
}
w.Write([]byte(iface.String()))
return true
}
Generally speaking, it's best to avoid assigning values to variables via static initializers, i.e.
f := Foo{bar:1,baz:"2"}
This is because it can create exactly the complaint you're talking about, if you forget to pass foo as a pointer via &foo or you decide to use value receivers you end up making a lot of clones of your values.
Instead, try to assign pointers to static initializers by default, i.e.
f := &Foo{bar:1,baz:"2"}
This way f will always be a pointer and the only time you'll get a value copy is if you explicitly use value receivers.
(There are of course times when you want to store the value from a static initializer, but those should be edge cases)

Resources