Is it safe?
(*TeamData)(unsafe.Pointer(&team.Id))
Example code:
func testTrans() []*TeamData {
teams := createTeams()
teamDatas := make([]*TeamData, 0, len(teams))
for _, team := range teams {
// is this safe?
teamDatas = append(teamDatas, (*TeamData)(unsafe.Pointer(&team.Id)))
}
return teamDatas
}
// ??
teams := testTrans()
Will the members of the teams := testTrans() array be garbage collected?
There are many structs and many fields returned through grpc and their definitions are the same as the local definitions, so I want to use this more efficient way((*TeamData)(unsafe.Pointer(&team.Id))), but I don't know if there will be any risks.
Full Example:
https://go.dev/play/p/q3gwp2mERvj
The documentation for unsafe.Pointer describes supported uses. In particular:
(1) Conversion of a *T1 to Pointer to *T2.
Provided that T2 is no larger than T1 and that the two share an
equivalent memory layout, this conversion allows reinterpreting data
of one type as data of another type.
Go's garbage collector recognises interior pointers an will not collect the original allocation until there are no remaining references to that block.
Hence the larger allocation (GrpcRetTeam in your example) will be pinned while references to *TeamData exists.
Another critical consideration is the alignment of the struct fields. Eg:
type Parent struct {
A uint8
B uint8
// 6 bytes of padding to align C.
C uint64
}
type Bad struct {
B uint8
// 7 bytes of padding to align C.
C uint64
}
In this case it would be invalid to use unsafe to extract Bad from Parent since the memory layout is different.
In most cases it's typically better to avoid unsafe.Pointer tricks unless required to meet functionality or performance requirements. It's often possible to refactor code to minimise allocations instead.
If you must use unsafe to meet performance requirements --
I would recommend implementing a test using the reflect package to ensure the memory alignment/layout is valid for the child struct.
Related
I'm interfacing with C code in Go using cgo, and I need to call a C function with a pointer to the underlying value in an Interface{} object. The value will be any of the atomic primitive types (not including complex64/complex128), or string.
I was hoping I'd be able to do something like this to get the address of ptr as an unsafe.Pointer:
unsafe.Pointer(reflect.ValueOf(ptr).UnsafeAddr())
But this results in a panic due to the value being unaddressable.
A similar question to this is Take address of value inside an interface, but this question is different, as in this case it is known that the value will always be one of the types specified above (which will be at most 64 bits), and I only need to give this value to a C function. Note that there are multiple C functions, and the one that will be called varies based off of a different unrelated parameter.
I also tried to solve this using a type switch statement, however I found myself unable to get the address of the values even after the type assertion was done. I was able to assign the values to temporary copies, then get the address of those copies, but I'd rather avoid making these copies if possible.
interface{} has own struct:
type eface struct {
typ *rtype
val unsafe.Pointer
}
You have no access to rtype directly or by linking, on the other hand, even though you'll copy whole rtype, it may be changed (deprecated) at future.
But thing is that you can replace pointer types with unsafe.Pointer (it may be anything else with same size, but pointer is much idiomatic, because each type has own pointer):
type eface struct {
typ, val unsafe.Pointer
}
So, now we can get value contained in eface:
func some_func(arg interface{}) {
passed_value := (*eface)(unsafe.Pointer(&arg)).val
*(*byte)(passed_value) = 'b'
}
some_var := byte('a')
fmt.Println(string(some_var)) // 'a'
some_func(some_var)
fmt.Println(string(some_var)) // 'a', it didn't changed, just because it was copied
some_func(&some_var)
fmt.Println(string(some_var)) // 'b'
You also might see some more usages at my repo:
https://github.com/LaevusDexter/fast-cast
Sorry for my poor English.
I am going on a project which process some data, I am wondering that if it is better to use pointer in non-primitive typed fields of struct.
What I've found is that the reason of using pointer is that nil can be used as a zero-value, is this the only reason to use pointer?
For example, I am going to store time.Time in my struct and it cannot be nil, then is it better to use non-pointer field?
So is it okay to use
type A struct {
CreatedAt time.Time
}
rather than
type A struct {
CreatedAt *time.Time
}
when Now is not going to be nil?
Not sure I understand the question. In the case of "Now" I would make it a function of the struct i.e.:
type A struct{}
func (a A) Now() time.Time { return time.Now(); }
otherwise what does Now mean? Now is constantly changing.
There are great blogs on when to use pointers
The short would be it doesn't really depend on if the value can be nil, but more on memory and concurrency. Pointers will be passed as references, so less memory, and faster, but also means that changing in one go routine can be very dangerous because the value could be referenced in another go routine and cause race conditions and unexpected behaviors.
I'm not really a professional or know the ins and outs of Go, so take everything I say with some grain of salt.
But as I am understanding it, you should most likely use pointers.
This is because every time you use a non-pointer type, the whole struct will be part of your struct in memory. As a consequence, you can't share a single instance of a struct between multiple structs - every single one gets a copy of your original struct.
Heres a small example:
// This struct has 2x64 bits in size
type MyStruct struct {
A uint64
B uint64
}
// This struct has 32 + 2x64 bits in size
type MyOtherStruct struct {
C uint32
Parent MyStruct
}
// This struct has 32 + the length of an address bits size
type MyPointerStruct struct {
D uint32
Parent *MyStruct
}
But apart from memory concerns, there is also a performance hit if your inner struct is very big. Because every time you set the inner struct the whole memory has to be copied to your instance.
However you have to be careful if your are dealing with interfaces or structs. At runtime, an interface is represented as a type with two fields: A reference to the actual (runtime) type and one with a reference to the actual instance.
So I - with my unprofessional opinion - would recommend to not use pointers if you have interface types because otherwise the CPU has to deference twice (once to get the interface reference, and then again to get the instance of the interface).
Similar to what I've learned in C++, I believe it's the padding that causes a difference in the size of instances of both structs.
type Foo struct {
w byte //1 byte
x byte //1 byte
y uint64 //8 bytes
}
type Bar struct {
x byte //1 byte
y uint64 //8 bytes
w byte// 1 byte
}
func main() {
fmt.Println(runtime.GOARCH)
newFoo := new(Foo)
fmt.Println(unsafe.Sizeof(*newFoo))
newBar := new(Bar)
fmt.Println(unsafe.Sizeof(*newBar))
}
Output:
amd64
16
24
Is there a rule of thumb to follow when defining struct members? (like ascending/descending order of size of types)
Is there a compile time optimisation which we can pass, that can automatically take care of this?
Or shouldn't I be worried about this at all?
Currently there's no compile-time optimisation; the values are padded to 8 bytes on x64.
You can manually arrange structs to optimally utilise space; typically by going from larger types to smaller; 8 consecutive byte fields for example, will only use 8 bytes, but a single byte would be padded to an 8 byte alignment, consider this: https://play.golang.org/p/0qsgpuAHHp
package main
import (
"fmt"
"unsafe"
)
type Compact struct {
a, b uint64
c, d, e, f, g, h, i, j byte
}
// Larger memory footprint than "Compact" - but less fields!
type Inefficient struct {
a uint64
b byte
c uint64
d byte
}
func main() {
newCompact := new(Compact)
fmt.Println(unsafe.Sizeof(*newCompact))
newInefficient := new(Inefficient)
fmt.Println(unsafe.Sizeof(*newInefficient))
}
If you take this into consideration; you can optimise the memory footprint of your structs.
Or shouldn't I be worried about this at all?
Yes you should.
This is also called mechanical sympathy (see this Go Time podcast episode), so it also depends on the hardware architecture you are compiling for.
See as illustration:
"The day byte alignment came back to bite me" (January 2014)
"On the memory alignment of Go slice values" (July 2016)
The values in Go slices are 16-byte aligned. They are not 32 byte aligned.
Go pointers are byte-aligned.
It depends on type of application that you are developing and on usage of those structures. If application needs to meet some memory/performance criteria you definitely should care about memory alignment and paddings, but not only - there is nice article https://www.usenix.org/legacy/publications/library/proceedings/als00/2000papers/papers/full_papers/sears/sears_html/index.html that highlights theme of optimal CPU caches usage and correlation between struct layouts and performance. It highlights cache line alignment, false sharing, etc.
Also there is a nice golang tool https://github.com/1pkg/gopium that helps to automate those optimizations, check it out!
Some guideline
To minimize the number of padding bytes, we must lay out the fields from
the highest allocation to lowest allocation.
One exception is an empty structure
As we know the size of empty is zero
type empty struct {
a struct{}
}
Following the common rule above, we may arrange the fields of structure as below
type E struct {
a int64
b int64
c struct{}
}
However, the size of E is 24,
When arrange the fields of structure as
type D struct {
b struct{}
a int64
c int64
}
The size of D is 16, refer to https://go.dev/play/p/ID_hN1zwIwJ
IMO, it is better to use tools that help us to automate structure alignment optimizations
aligncheck — https://gitlab.com/opennota/check
maligned — https://github.com/mdempsky/maligned, the original maligned is deprecated. Use https://pkg.go.dev/golang.org/x/tools/go/analysis/passes/fieldalignment
golang clint
you just need to enable ‘maligned’ in the ‘golangci-lint’ settings.
Example, from the configuration file.golangci.example.yml
linters-settings:
maligned:
# print struct with more effective memory layout or not, false by default
suggest-new: true
What's the difference? Is map[T]bool optimized to map[T]struct{}? Which is the best practice in Go?
Perhaps the best reason to use map[T]struct{} is that you don't have to answer the question "what does it mean if the value is false"?
From "The Go Programming Language":
The struct type with no fields is called the empty struct, written
struct{}. It has size zero and carries no information but may be
useful nonetheless. Some Go programmers use it instead of bool as the
value type of a map that represents a set, to emphasize that only the
keys are significant, but the space saving is marginal and the syntax
more cumbersome, so we generally avoid it.
If you use bool testing for presence in the "set" is slightly nicer since you can just say:
if mySet["something"] {
/* .. */
}
Difference is in memory requirements. Under the bonnet empty struct is not a pointer but a special value to save memory.
An empty struct is a struct type like any other. All the properties you are used to with normal structs apply equally to the empty struct. You can declare an array of structs{}s, but they of course consume no storage.
var x [100]struct{}
fmt.Println(unsafe.Sizeof(x)) // prints 0
If empty structs hold no data, it is not possible to determine if two struct{} values are different.
Considering the above statements it means that we may use them as method receivers.
type S struct{}
func (s *S) addr() { fmt.Printf("%p\n", s) }
func main() {
var a, b S
a.addr() // 0x1beeb0
b.addr() // 0x1beeb0
}
It seems pointless to be used in primitive language constructs, as you can't specify any sort of values
func main() {
y := new([]float)
fmt.Printf("Len = %d", len(*y) ) // => Len = 0
}
For stucts it makes a bit more sense, but what's the difference between saying y := new(my_stuct) and the seemingly more concise y := &my_struct?
And since anything you create is based on those primitives, they will be initialized to the said zero values. So what's the point? When would you ever want to use new()?
Sorry for the very-beginner question, but the documentation isn't always that clear.
You can't use new for slices and maps, as in your code example, but instead you must use the make command: make([]float, 100)
Both new(MyStruct) and &MyStruct{} do to the same thing, because Go will allocate values on the heap if you get their address with &. Sometimes the code just expresses it intent better in one style or the other.
Go does not have built-in support for constructors, so usually you would wrap the call to new into a function, for example NewMyStruct() which does all the necessary initialization. It also makes it possible to initialize private fields or hide the struct behind an interface, to prevent users of the object from directly messing with its internals. Also evolving the structure of the struct is easier that way, when you don't need to change all of its users when adding/removing/renaming/reordering fields.
make does only work for maps, slices and channels and composite literals like type{} work only for structs, arrays, slices, and maps. For other types, you'll have to use new to get a pointer to a newly allocated instance (if you don't want to use a longer var v T; f(&v)).
I guess this is useful if you want to initialize a struct:
typedef foo struct {
bar *int
}
v := foo{bar: new(int)}