Here is a snippet that I am looking at:
var t txn
t.c = c
err := c.read(&t.req)
Why do you have to write &t and not just t.req?
You don't always have to use the ampersand. In your example it depends on the signature of c.read, which asks for a pointer (* before the type, such as *MyStruct). & returns the address of a value, giving you a pointer to it, so &t.req satisfies read's signature.
For further reading, see the FAQ on pointers and the spec on Address operators.
Related
I have the following code snippet which "go vet" complains about with the warning "possible misuse of reflect.SliceHeader". I can not find very much information about this warning other then this. After reading that it is not very clear to me what is needed to do this in a way that makes go vet happy - and without possible gc issues.
The goal of the snippet is to have a go function copy data to memory which is managed by an opaque C library. The Go function expects a []byte as a parameter.
func Callback(ptr unsafe.Pointer, buffer unsafe.Pointer, size C.longlong) C.longlong {
...
sh := &reflect.SliceHeader{
Data: uintptr(buffer),
Len: int(size),
Cap: int(size),
}
buf := *(*[]byte)(unsafe.Pointer(sh))
err := CopyToSlice(buf)
if err != nil {
log.Fatal("failed to copy to slice")
}
...
}
https://pkg.go.dev/unsafe#go1.19.4#Pointer
Pointer represents a pointer to an arbitrary type. There are four
special operations available for type Pointer that are not available
for other types:
A pointer value of any type can be converted to a Pointer.
A Pointer can be converted to a pointer value of any type.
A uintptr can be converted to a Pointer.
A Pointer can be converted to a uintptr.
Pointer therefore allows a program to defeat the type system and read
and write arbitrary memory. It should be used with extreme care.
The following patterns involving Pointer are valid. Code not using
these patterns is likely to be invalid today or to become invalid in
the future. Even the valid patterns below come with important caveats.
Running "go vet" can help find uses of Pointer that do not conform to
these patterns, but silence from "go vet" is not a guarantee that the
code is valid.
(6) Conversion of a reflect.SliceHeader or reflect.StringHeader Data
field to or from Pointer.
As in the previous case, the reflect data structures SliceHeader and
StringHeader declare the field Data as a uintptr to keep callers from
changing the result to an arbitrary type without first importing
"unsafe". However, this means that SliceHeader and StringHeader are
only valid when interpreting the content of an actual slice or string
value.
var s string
hdr := (*reflect.StringHeader)(unsafe.Pointer(&s)) // case 1
hdr.Data = uintptr(unsafe.Pointer(p)) // case 6 (this case)
hdr.Len = n
In this usage hdr.Data is really an alternate way to refer to the
underlying pointer in the string header, not a uintptr variable
itself.
In general, reflect.SliceHeader and reflect.StringHeader should be used only as *reflect.SliceHeader and *reflect.StringHeader pointing at actual slices or strings, never as plain structs. A program should not declare or allocate variables of these struct types.
// INVALID: a directly-declared header will not hold Data as a reference.
var hdr reflect.StringHeader
hdr.Data = uintptr(unsafe.Pointer(p))
hdr.Len = n
s := *(*string)(unsafe.Pointer(&hdr)) // p possibly already lost
It looks like JimB (from the comments) hinted upon the most correct answer, though he didn't post it as an answer and he didn't include an example. The following passes go vet, staticcheck, and golangci-lint - and doesn't segfault so I think it is the correct answer.
func Callback(ptr unsafe.Pointer, buffer unsafe.Pointer, size C.longlong) C.longlong {
...
buf := unsafe.Slice((*byte)(buffer), size)
err := CopyToSlice(buf)
if err != nil {
log.Fatal("failed to copy to slice")
}
...
}
In Go, suppose I have a []byte of UTF-8 that I want to return as a string.
func example() *string {
byteArray := someFunction()
text := string(byteArray)
return &text
}
I would like to eliminate the text variable, but Go doesn't support the following:
func example() *string {
byteArray := someFunction()
return &string(byteArray)
}
Is this second example syntax correct? And if so, why doesn't Go support it?
Because the spec defines is that way:
For an operand x of type T, the address operation &x generates a pointer of type *T to x. The operand must be addressable, that is, either a variable, pointer indirection, or slice indexing operation; or a field selector of an addressable struct operand; or an array indexing operation of an addressable array. As an exception to the addressability requirement, x may also be a (possibly parenthesized) composite literal.
Notice that type conversions (what you are trying to do with string(byteArray)) are not included in this list.
See Marc's answer for an official citation, but here's an intuitive reason for why Go doesn't support this.
Suppose the following code
var myString string
stringPointer := &myString
*stringPointer = "some new value"
Hopefully you know, this code will write some new value into myString. This is a basic use of pointers. Now consider the modified example (pretending that it is valid code):
var myBytes []byte
// modify myBytes...
stringPointer := &string(myString)
*stringPointer = "some new value"
The question is, where in the world (or computer) are we writing to?? Where is some new value going?
In order for the language to handle this correctly, the compiler would need some internal process to "promote" the temporary value to an invisible variable, and then take the address of that. This would be adding needless complexity to make some code slightly shorter, but create this confusing situation where we have pointers with no well defined location in the program. Instead of creating these confusing ghost-variables, the language delegates to the programmer to use their own variable as usual.
What's the cleanest way to handle a case such as this:
func a() string {
/* doesn't matter */
}
b *string = &a()
This generates the error:
cannot take the address of a()
My understanding is that Go automatically promotes a local variable to the heap if its address is taken. Here it's clear that the address of the return value is to be taken. What's an idiomatic way to handle this?
The address operator returns a pointer to something having a "home", e.g. a variable. The value of the expression in your code is "homeless". if you really need a *string, you'll have to do it in 2 steps:
tmp := a(); b := &tmp
Note that while there are completely valid use cases for *string, many times it's a mistake to use them. In Go string is a value type, but a cheap one to pass around (a pointer and an int). String's value is immutable, changing a *string changes where the "home" points to, not the string value, so in most cases *string is not needed at all.
See the relevant section of the Go language spec. & can only be used on:
Something that is addressable: variable, pointer indirection, slice indexing operation, field selector of an addressable struct, array indexing operation of an addressable array; OR
A composite literal
What you have is neither of those, so it doesn't work.
I'm not even sure what it would mean even if you could do it. Taking the address of the result of a function call? Usually, you pass a pointer of something to someone because you want them to be able to assign to the thing pointed to, and see the changes in the original variable. But the result of a function call is temporary; nobody else "sees" it unless you assign it to something first.
If the purpose of creating the pointer is to create something with a dynamic lifetime, similar to new() or taking the address of a composite literal, then you can assign the result of the function call to a variable and take the address of that.
In the end you are proposing that Go should allow you to take the address of any expression, for example:
i,j := 1,2
var p *int = &(i+j)
println(*p)
The current Go compiler prints the error: cannot take the address of i + j
In my opinion, allowing the programmer to take the address of any expression:
Doesn't seem to be very useful (that is: it seems to have very small probability of occurrence in actual Go programs).
It would complicate the compiler and the language spec.
It seems counterproductive to complicate the compiler and the spec for little gain.
I recently was tied up in knots about something similar.
First talking about strings in your example is a distraction, use a struct instead, re-writing it to something like:
func a() MyStruct {
/* doesn't matter */
}
var b *MyStruct = &a()
This won't compile because you can't take the address of a(). So do this:
func a() MyStruct {
/* doesn't matter */
}
tmpA := a()
var b *MyStruct = &tmpA
This will compile, but you've returned a MyStruct on the stack, allocated sufficient space on the heap to store a MyStruct, then copied the contents from the stack to the heap. If you want to avoid this, then write it like this:
func a2() *MyStruct {
/* doesn't matter as long as MyStruct is created on the heap (e.g. use 'new') */
}
var a *MyStruct = a2()
Copying is normally inexpensive, but those structs might be big. Even worse when you want to modify the struct and have it 'stick' you can't be copying then modifying the copies.
Anyway, it gets all the more fun when you're using a return type of interface{}. The interface{} can be the struct or a pointer to a struct. The same copying issue comes up.
You can't get the reference of the result directly when assigning to a new variable, but you have idiomatic way to do this without the use of a temporary variable (it's useless) by simply pre-declaring your "b" pointer - this is the real step you missed:
func a() string {
return "doesn't matter"
}
b := new(string) // b is a pointer to a blank string (the "zeroed" value)
*b = a() // b is now a pointer to the result of `a()`
*b is used to dereference the pointer and directly access the memory area which hold your data (on the heap, of course).
Play with the code: https://play.golang.org/p/VDhycPwRjK9
Yeah, it can be annoying when APIs require the use of *string inputs even though you’ll often want to pass literal strings to them.
For this I make a very tiny function:
// Return pointer version of string
func p(s string) *string {
return &s
}
and then instead of trying to call foo("hi") and getting the dreaded cannot use "hi" (type string) as type *string in argument to foo, I just wrap the argument in a call to to p():
foo(p("hi"))
a() doesn't point to a variable as it is on the stack. You can't point to the stack (why would you ?).
You can do that if you want
va := a()
b := &va
But what your really want to achieve is somewhat unclear.
At the time of writing this, none of the answers really explain the rationale for why this is the case.
Consider the following:
func main() {
m := map[int]int{}
val := 1
m[0] = val
v := &m[0] // won't compile, but let's assume it does
delete(m, 0)
fmt.Println(v)
}
If this code snippet actually compiled, what would v point to!? It's a dangling pointer since the underlying object has been deleted.
Given this, it seems like a reasonable restriction to disallow addressing temporaries
guess you need help from More effective Cpp ;-)
Temp obj and rvalue
“True temporary objects in C++ are invisible - they don't appear in your source code. They arise whenever a non-heap object is created but not named. Such unnamed objects usually arise in one of two situations: when implicit type conversions are applied to make function calls succeed and when functions return objects.”
And from Primer Plus
lvalue is a data object that can be referenced by address through user (named object). Non-lvalues include literal constants (aside from the quoted strings, which are represented by their addresses), expressions with multiple terms, such as (a + b).
In Go lang, string literal will be converted into StrucType object, which will be a non-addressable temp struct object. In this case, string literal cannot be referenced by address in Go.
Well, the last but not the least, one exception in go, you can take the address of the composite literal. OMG, what a mess.
This question already has answers here:
What does an underscore and interface name after keyword var mean?
(2 answers)
Closed 2 years ago.
When I read a copy of the docker/distribution source code, I find there are variables declared which make me quite confused.
The code is:
var _ FileInfo = FileInfoInternal{}
var _ FileInfo = &FileInfoInternal{}
I don't know what the declare mean, and hope to get some help.
From the FAQ:
You can ask the compiler to check that the type T implements the
interface I by attempting an assignment:
type T struct{}
var _ I = T{} // Verify that T implements I.
In this case the blank identifier _ stands for the variable name which is not needed here (and thus prevents a "declared but not used" error).
And more general from the spec:
The blank identifier provides a way to ignore right-hand side values
in an assignment:
_ = x // evaluate x but ignore it
x, _ = f() // evaluate f() but ignore second result value
By testing both FileInfoInternal{} and &FileInfoInternal{} you check if the interface is implemented with a value receiver. A value receiver will accept both a value and a pointer whereas the pointer receiver will only work with a pointer and the first assignment by value will fail.
The second test with &FileInfoInternal{} is not actually needed (as confirmed by the author in the comments) since the first test will pass with a value receiver and fail with a pointer received. Thus the second test is redundant.
This is an excellent article that explains the difference between value and pointer receivers and how they are used very well.
FileInfo is an interface and the code checks whether FileInfoInternal implements this interface.
I'm currently reading the source code of the https://github.com/codegangsta/inject go package to understand how does this package works.
I have some questions concerning the file https://github.com/codegangsta/inject/blob/master/inject.go file thats use some element of the Go language I don't understand and don't find precise explanations in the documentation.
// InterfaceOf dereferences a pointer to an Interface type.
// It panics if value is not an pointer to an interface.
func InterfaceOf(value interface{}) reflect.Type {
t := reflect.TypeOf(value)
for t.Kind() == reflect.Ptr {
t = t.Elem()
}
if t.Kind() != reflect.Interface {
panic("Called inject.InterfaceOf with a value that is not a pointer to an interface. (*MyInterface)(nil)")
}
return t
}
My first question is concerning the for loop. Why does it uses a for loop with a test expression ?
The second relates to the message in the panic function. "A pointer to an interface" is mentioned with the (*MyInterface)(nil). I only encounter a similar construction in the go documentation concerning 'compile time checking structure' when you check that a type implements a structure :
var _ SomeType = (*SomeInterface)(nil)
I did not find any informations about a statement with (*Interface)(nil) and pointer to interface.
How should we interpret this statement ? What is the relation with a pointer to interface and where could I find informations about pointer to interface ?
To summarize both answers:
The for loop
for t.Kind() == reflect.Ptr {
t = t.Elem()
}
t.Elem() is the reflection equivalent to *t, so what this loop does it dereferencing t as long as it holds another pointer value. At the end of the loop, t will hold the value that the last pointer pointed to, not a pointer anymore.
The message
Called [...] with a value that is not a pointer to an interface. (*MyInterface)(nil)
The expression (*MyInterface)(nil) is just an (poorly phrased) example of what is expected as parameter.
The syntax is that of a conversion. A conversion will attempt to convert a value (in this case nil) to a given type (*MyInterface) in this case. So,
(*MyInterface)(nil)
will give you a zero value of a *MyInterface whose interface type would be MyInterface (play):
x := (*MyInterface)(nil)
InterfaceOf(x) // MyInterface
Of course, this value does not point somewhere meaningful.
Compile time checking of interface implementation
To avoid confusion, the construct you showed
var _ SomeType = (*SomeInterface)(nil)
is probably not what you wanted. I guess you wanted this:
var _ SomeInterface = (*SomeType)(nil)
This construct enables compile time checking of interface implementation for certain types.
So in case you're writing a library of some sort and you want to satisfy an interface without
using it, you can use this to make sure that your struct implements the interface.
Why this works
First of all, var _ someType is a variable that is going to be checked by the compiler but
will not be in the compiled program and is not accessible due to the Blank Identifier _:
The blank identifier may be used like any other identifier in a declaration, but it does not introduce a binding and thus is not declared.
This enables you do declare an arbitrary number of these constructs without interfering with the
rest of the program.
You can declare a zero value of a pointer of any type by writing:
(*T)(nil)
Check this example on play.
Next, assignability says that x is assignable to T if T is an interface and x implements T.
So to summarize:
T _ = (*x)(nil)
enforces that x implements T as everything else would be an error.
The for loop is used to continually dereference the type until it is no longer a pointer. This will handle case where the type acquired an extra indirection(s).
e.g. play.golang.org/p/vR2gKNJChE
As for the (*MyInterface)(nil), pointers to interfaces always1 an error Go code. I assume the author is just describing what he means by pointer to interface with a code snippet since they are so rare.
If you're still intrigued by the forbidden type Russ Cox has some info how exactly all this works under the hood: research.swtch.com/interfaces. You'll have a hard time finding info on the use of pointers to an interface because [1].
(1) OK not really always, but honestly don't do it unless you're a Go pro. In which case don't tell anyone about it.
That for loop is identical to while loop in other languages
The second thing is just a syntax for conversions:
(*Point)(p) // p is converted to *Point
Because how this library works you just have to pass the pointer to interface, for loop then dereferences it (if we pass something like (***MyInterface)(nil)) and then if statement checks if the ty[e pointed to is an interface.