Is there a way to use the reflection libraries in Go to go from the name of a type to its Type representation?
I've got a library where the user needs to provide Type representations for some code generation. I know it must be possible (in a sense) because they can just create a variable of that type and call the TypeOf function, but is there a way to circumvent this and just get representation from the name?
The question is not quite explicit, it can be interpreted in 2 ways, to one of which the answer is no, not possible; and the other to which the answer is yes, it's possible.
At runtime
If the type name is provided as a string value, then at runtime it's not possible as types that are not referred to explicitly may not get compiled into the final executable binary (and thus obviously become unreachable, "unknown" at runtime). For details see Splitting client/server code. For possible workarounds see Call all functions with special prefix or suffix in Golang.
At "coding" time
If we're talking about "coding" time (source code writing / generating), then it's possible without creating / allocating a variable of the given type and calling reflect.TypeOf() and passing the variable.
You may start from the pointer to the type, and use a typed nil pointer value without allocation, and you can navigate from its reflect.Type descriptor to the descriptor of the base type (or element type) of the pointer using Type.Elem().
This is how it looks like:
t := reflect.TypeOf((*YourType)(nil)).Elem()
The type descriptor t above will be identical to t2 below:
var x YourType
t2 := reflect.TypeOf(x)
fmt.Println(t, t2)
fmt.Println(t == t2)
Output of the above application (try it on the Go Playground):
main.YourType main.YourType
true
Related
This question already has answers here:
When to use pointers [duplicate]
(1 answer)
I don't understand when to use pointers on go [duplicate]
(2 answers)
Difference between returning a pointer and a value in initialization methods [duplicate]
(1 answer)
Pointers vs. values in parameters and return values
(5 answers)
Golang struct initialization
(2 answers)
Closed 15 days ago.
I am wondering what the differences are between those two ways of initializing a struct in Go. Does the second one give you a pointer storing the address of the struct? But they have the same syntax if we want to change the field variables, so how are they different?
I can de-reference the pointer, apparently. But do they have different functionalities? In what scenario would I prefer one over the other?
b := Student{Name:"Bob"}
pb := &Student{Name:"Bob", Age:8,}
They only differ in what the type of the local variable is. Whether to prefer one over the other depends entirely on how you would use that variable in the remainder of the function's body. If you are always passing it around (meaning use it on the right-hand side of an assignment, as an argument to a function call, or returning it) as a pointer, use the pb := &Student{...} form. If you intend to create (shallow) copies of it, the b := Student{...} form might be more convenient. This is however purely based on the fact that typing pb is easier than &b, and that typing b is easier than *pb. If you apply these substitutions everywhere (except for in the context of field accesses, where it is allowed but not mandatory, see below), they are completely interchangeable.
If you want to be able to set the variable holding the struct (pointer) to nil, use a pointer, obviously. But this can also be solved by introducing a new, pointer-typed variable and setting that to either nil or &b.
Some types in Go cannot be copied (this is typically accomplished by embedding a [0]sync.Mutex field in the struct, which takes up no space but triggers the "mutex copy" error of the compiler). An example would be protobuf structs. For these, it is almost always preferable to use the pointer form; however, this is not required, as the initialization via b := Student{...} is not a copy.
Go allows transparently dereferencing when accessing struct fields. b.Name, (&b).Name, pb.Name, (*pb).Name all work and are equivalent. Note however that transparent dereferencing only works over one level, (&pb).Name will result in a compiler error.
The composite literals differ in the initialization of the Age field. The first sets the field to the zero value. The second sets the field to the number 8. Otherwise, the initialization of the struct values is identical.
What happens next is where the key difference lies. The first example assigns the value to a variable named b. The second example takes assigns the value to an anonymous variable; takes the address of the variable; and assigns the result to a variable named pb.
The type of b is the struct type and the type of pb is a pointer to the struct type.
See I don't understand when to use pointers on go to see a discussion on why one might want to use a pointer.
I'm currently trying to learn GO and mainly knowing and working with Java, ASP.Net and some Python, there is no experience working with C-like pointers, which causes my current confusion.
A library I'm currently using to write my first GO project is called Commando.
There I have the struct CommandRegistry and the variable of interest is called Commands.
In the struct the variable is described as the following:
// registered command configurations
Commands map[string]*Command
On a first glimpse I would understand this as a Map object containing a list of Strings, however it also shows the pointer reference to the actual Command object.
All I can see is that it is a map I can loop over which returns the name of the command ( the string ),
however I'm wondering if the *Command in the type description means I can somehow dereference the pointer and retrieve the object itself to extract the additional information of it.
As I know the & operand is used to create a new pointer of another object. Pass-by-reference basically instead of pass-by-value.
And the * operand generally signals the object is a pointer or used to require a pointer in a new function.
Is there a way I can retrieve the Command object or why does the type contain the *Command in it's declaration?
Commands is a map (dictionary) which has strings as keys, and pointers to Commands as values. By passing it a key, you will get a pointer to the command it belongs to. You can then dereference the pointer to an actual Command object by using the * operator. Something like dereferencedCommand := *Commands["key"].
The * operator can be quite confusing, at least it was for me. When used as a type it denotes that we are receiving the memory address of some variable. But to dereference a memory address to a concrete type, you also use the * operator.
Is there a way to use the reflection libraries in Go to go from the name of a type to its Type representation?
I've got a library where the user needs to provide Type representations for some code generation. I know it must be possible (in a sense) because they can just create a variable of that type and call the TypeOf function, but is there a way to circumvent this and just get representation from the name?
The question is not quite explicit, it can be interpreted in 2 ways, to one of which the answer is no, not possible; and the other to which the answer is yes, it's possible.
At runtime
If the type name is provided as a string value, then at runtime it's not possible as types that are not referred to explicitly may not get compiled into the final executable binary (and thus obviously become unreachable, "unknown" at runtime). For details see Splitting client/server code. For possible workarounds see Call all functions with special prefix or suffix in Golang.
At "coding" time
If we're talking about "coding" time (source code writing / generating), then it's possible without creating / allocating a variable of the given type and calling reflect.TypeOf() and passing the variable.
You may start from the pointer to the type, and use a typed nil pointer value without allocation, and you can navigate from its reflect.Type descriptor to the descriptor of the base type (or element type) of the pointer using Type.Elem().
This is how it looks like:
t := reflect.TypeOf((*YourType)(nil)).Elem()
The type descriptor t above will be identical to t2 below:
var x YourType
t2 := reflect.TypeOf(x)
fmt.Println(t, t2)
fmt.Println(t == t2)
Output of the above application (try it on the Go Playground):
main.YourType main.YourType
true
I was under the impression that using the unsafe package allows you to read/write arbitrary data. I'm trying to change the value the interface{} points to without changing the pointer itself.
Assuming that interface{} is implemented as
type _interface struct {
type_info *typ
value unsafe.Pointer
}
setting fails with a SIGSEGV, although reading is successful.
func data(i interface{}) unsafe.Pointer {
return unsafe.Pointer((*((*[2]uintptr)(unsafe.Pointer(&i))))[1])
}
func main() {
var i interface{}
i = 2
fmt.Printf("%v, %v\n", (*int)(data(i)), *(*int)(data(i)))
*((*int)(data(i))) = 3
}
Am I doing something wrong, or is this not possible in golang?
Hm... Here's how I understand your second code example currently, in case I've made an error (if you notice anything amiss in what I'm describing, my answer is probably irredeemably wrong and you should ignore the rest of what I have to say).
Allocate memory for interface i in main.
Set the value of i to an integer type with the value 2.
Allocate memory for interface i in data.
Copy the value of main's i to data's i; that is, set the value of the new interface to an integer type with the value 2.
Cast the address of the new variable into a pointer to length-2 array of uintptr (with unsafe.Pointer serving as the intermediary that forces the compiler to accept this cast).
Cast the second element of the array (whose value is the address of the value-part of i in data) back into an unsafe.Pointer and return it.
I've made an attempt at doing the same thing in more steps, but unfortunately I encountered all the same problems: the program recognizes that I have a non-nil pointer and it's able to dereference the pointer for reading, but using the same pointer for writing produces a runtime error.
It's step 6 that go vet complains about, and I think it's because, according to the package docs,
A uintptr is an integer, not a reference. Converting a Pointer to a uintptr creates an integer value with no pointer
semantics. Even if a uintptr holds the address of some object, the garbage collector will not update that uintptr's value if the object moves, nor will that uintptr keep the object from being reclaimed.
More to the point, from what I can tell (though I'll admit I'm having trouble digging up explicit confirmation without scanning the compiler and runtime source), the runtime doesn't appear to track the value-part of an interface{} type as a discrete pointer with its own reference count; you can, of course, trample over both the interface{}'s words by writing another interface value into the whole thing, but that doesn't appear to be what you wanted to do at all (write to the memory address of a pointer that is inside an interface type, all without moving the pointer).
What's interesting is that we seem to be able to approximate this behavior by just defining our own structured type that isn't given special treatment by the compiler (interfaces are clearly somewhat special, with type-assertion syntax and all). That is, we can use unsafe.Pointer to maintain a reference that points to a particular point in memory, and no matter what we cast it to, the memory address never moves even if the value changes (and the value can be reinterpreted by casting it to something else). The part that surprises me a bit is that, at least in my own example, and at least within the Playground environment, the value that is pointed to does not appear to have a fixed size; we can establish an address to write to once, and repeated writes to that address succeed even with huge (or tiny) amounts of data.
Of course, with at least this implementation, we lose a bunch of the other nice-to-have things we associate with interface types, especially non-empty interface types (i.e. with methods). So, there's no way to use this to (for example) make a super-sneaky "generic" type. It seems that an interface is its own value, and part of that value's definition is an address in memory, but it's not entirely the same thing as a pointer.
I was just reading through Effective Go and in the Pointers vs. Values section, near the end it says:
The rule about pointers vs. values for receivers is that value methods can be invoked on pointers and values, but pointer methods can only be invoked on pointers. This is because pointer methods can modify the receiver; invoking them on a copy of the value would cause those modifications to be discarded.
To test it, I wrote this:
package main
import (
"fmt"
"reflect"
)
type age int
func (a age) String() string {
return fmt.Sprintf("%d yeasr(s) old", int(a))
}
func (a *age) Set(newAge int) {
if newAge >= 0 {
*a = age(newAge)
}
}
func main() {
var vAge age = 5
pAge := new(age)
fmt.Printf("TypeOf =>\n\tvAge: %v\n\tpAge: %v\n", reflect.TypeOf(vAge),
reflect.TypeOf(pAge))
fmt.Printf("vAge.String(): %v\n", vAge.String())
fmt.Printf("vAge.Set(10)\n")
vAge.Set(10)
fmt.Printf("vAge.String(): %v\n", vAge.String())
fmt.Printf("pAge.String(): %v\n", pAge.String())
fmt.Printf("pAge.Set(10)\n")
pAge.Set(10)
fmt.Printf("pAge.String(): %v\n", pAge.String())
}
And it compiles, even though the document says it shouldn't since the pointer method Set() should not be invocable through the value var vAge. Am I doing something wrong here?
That's valid because vAge is addressable. See the last paragraph in Calls under the language spec:
A method call x.m() is valid if the method set of (the type of) x
contains m and the argument list can be assigned to the parameter list
of m. If x is addressable and &x's method set contains m, x.m() is
shorthand for (&x).m().
vAge is not considered as only a "value variable", because it's a known location in memory that stores a value of type age. Looking at vAge only as its value, vAge.Set(10) is not valid as an expression on its own, but because vAge is addressable, the spec declares that it's okay to treat the expression as shorthand for "get the address of vAge, and call Set on that" at compile-time, when we will be able to verify that Set is part of the method set for either age or *age. You're basically allowing the compiler to do a textual expansion on the original expression if it determines that it's necessary and possible.
Meanwhile, the compiler will allow you to call age(23).String() but not age(23).Set(10). In this case, we're working with a non-addressable value of type age. Since it's not valid to say &age(23), it can't be valid to say (&age(23)).Set(10); the compiler won't do that expansion.
Looking at the Effective Go example, you're not directly calling b.Write() at the scope where we know b's full type. You're instead making a temporary copy of b and trying to pass it off as a value of type interface io.Writer(). The problem is that the implementation of Printf doesn't know anything about the object being passed in except that it has promised it knows how to receive Write(), so it doesn't know to take a byteSlice and turn it into a *ByteSlice before calling the function. The decision of whether to address b has to happen at compile time, and PrintF was compiled with the precondition that its first argument would know how to receive Write() without being referenced.
You may think that if the system knows how to take an age pointer and convert it to an age value, that it should be able to do the reverse; t doesn't really make sense to be able to, though. In the Effective Go example, if you were to pass b instead of &b, you'd modify a slice that would no longer exist after PrintF returns, which is hardly useful. In my age example above, it literally makes no sense to take the value 23 and overwrite it with the value 10. In the first case, it makes sense for the compiler to stop and ask the programmer what she really meant to do when handing b off. In the latter case, it of course makes sense for the compiler to refuse to modify a constant value.
Furthermore, I don't think the system is dynamically extending age's method set to *age; my wild guess is that pointer types are statically given a method for each of the base type's methods, which just dereferences the pointer and calls the base's method. It's safe to do this automatically, as nothing in a receive-by-value method can change the pointer anyway. In the other direction, it doesn't always make sense to extend a set of methods that are asking to modify data by wrapping them in a way that the data they modify disappears shortly thereafter. There are definitely cases where it makes sense to do this, but this needs to be decided explicitly by the programmer, and it makes sense for the compiler to stop and ask for such.
tl;dr I think that the paragraph in Effective Go could use a bit of rewording (although I'm probably too long-winded to take the job), but it's correct. A pointer of type *X effectively has access to all of X's methods, but 'X' does not have access to *X's. Therefore, when determining whether an object can fulfill a given interface, *X is allowed to fulfill any interface X can, but the converse is not true. Furthermore, even though a variable of type X in scope is known to be addressable at compile-time--so the compiler can convert it to a *X--it will refuse to do so for the purposes of interface fulfillment because doing so may not make sense.