I create C structs in my Go code, like this:
var data C.MyStruct_t
Do I have to free them manually at some point, like I do when I use CString? With CString I often do something like:
ctitle := C.String(title)
defer C.free(unsafe.Pointer(&ctitle))
C.my_func(&ctitle)
No. You only call free on something that was allocated via the C *alloc functions. The C.CString and C.CBytes functions are documented as doing so internally, and requiring the use of C.free.
In this case even though data is of type C.MyStruct_t it is allocated in Go, and therefor will be handled by the Go garbage collector.
Related
Brief Context:
I am trying to create a cgo cache from uint64->[]byte for my company. Golang's map[uint64][]byte incur considerable latency due to the garbage collector as []byte is considered a pointer. As such, I would like to try to use CGO to avoid the issue. I am currently implementing a C++ unordered_map<unsigned long long, char*> to handle that. I managed to get the C++ wrappers to work but I am facing a major issue.
Currently, I am converting my Go byte array using
b := []byte{1,0,3,32,2,2,2,2}
str := C.String(b)
bb := []byte(C.GoString(str))
However, it turns out my my bb is []byte{1}. The 0 in the byte array is seen as the '/0' and thus shorten the string. Furthermore, it seems to have cause out of memory issue when I delete entries with
delete (map->find(key))->second. I suspect this is because that chars after the first '/0' does not get deallocated.
I am not sure how else to do this. Personally, I am new to CGO so I never used it prior to this project so any help would be appreciated.
There are two problems:
Use C.CBytes, not C.CString.
Please read this in its entirety before using cgo.
The C.C* functions which allocate (C.CString and C.CBytes do that) internally call malloc() from the linked in libc library to get the destination memory block, and you're supposed to eventually call C.Free() on them (which calls libc's free()), as documented.
AFAIK, C++ compiler is by no means oblidged to use libc's malloc() and free() to implement new and delete, so calling delete on the results of C.C* functions is a sure path to disaster.
The simplest way to solve this, IMO, is to export a "constructor" function from your C++ side: something like
extern "C" {
char* clone(char *src, size_t len);
}
…which would 1) allocate a memory block of length len using whatever method works best for C++; 2) copy len bytes from src to it; 3) return it.
You could then call it from the Go side—as C.clone(&b[0], len(b)) and call it a day: the C++ side is free to call delete on the result.
I saw the issue on Github which says sync.Pool should be used only with pointer types, for example:
var TPool = sync.Pool{
New: func() interface{} {
return new(T)
},
}
Does it make sense? What about return T{} and which is the better choice, why?
The whole point of sync.Pool is to avoid (expensive) allocations. Large-ish buffers, etc. You allocate a few buffers and they stay in memory, available for reuse. Hence the use of pointers.
But here you'll be copying the values on every step, defeating the purpose. (Assuming your T is a "normal" struct and not something like SliceHeader)
It is not necessary. In most cases it should be a pointer as you want to share an object, not to make copies.
In some use cases this can be a non pointer type, like an id of some external resource. I can imagine a pool of paths (mounted disk drives) represented with strings where some large file operations are being conducted.
I have the following code in Go which uses cgo:
func foo() {
bar := new(C.struct_bar)
...
}
Will bar be garbage collected after the function finishes its execution or I need to explicitly call free?
Yes, even though the struct_bar is a C type, the memory is allocated by Go, and will be collected by Go.
Any memory allocated in C however is not tracked by the Go GC. The C code should manage this on it's own, but in cases like C.CString where Go triggers the allocation, you must manually free the memory with C.free.
I have a package that interfaces with a C library. Now I need to store a pointer to a C struct in the Go struct
type A struct {
s *C.struct_b
}
Obviously this pointer needs to be freed before the struct is collected by the GC. How can I accomplish that?
The best thing to do is when possible copy the C struct into go controlled memory.
var ns C.struct_b
ns = *A.s
A.s = &ns
Obviously, that won't work in all cases. C.struct_b may be too complicated or shared with something still in C code. In this case, you need to create a .Free() or .Close() method (whichever makes more sense) and document that the user of your struct must call it. In Go, a Free method should always be safe to call. For example, after free is run, be sure to set A.s = nil so that if the user calls Free twice, the program does not crash.
There is also a way to create finalizers. See another answer I wrote here. However, they may not always run and if garbage is created fast enough, it is very possible that the creation of garbage will out pace collection. This should be considered as a supplement to having a Free/Close method and not a replacement.
I have a Go object whose address in memory I would like to keep constant. in C# one can pin an object's location in memory. Is there a way to do this in Go?
An object on which you keep a reference won't move. There is no handle or indirection, and the address you get is permanent.
From the documentation :
Note that, unlike in C, it's perfectly OK to return the address of a
local variable; the storage associated with the variable survives
after the function returns
When you set a variable, you can read this address using the & operator, and you can pass it.
tl;dr no - but it does not matter unless you're trying to do something unusual.
Worth noting that the accepted answer is partially incorrect.
There is no guarantee that objects are not moved - either on the stack or on the Go heap - but as long as you don't use unsafe this will not matter to you because the Go runtime will take care of transparently updating your pointers in case an object is moved.
If OTOH you use unsafe to obtain a uintptr, invoke raw syscalls, perform CGO calls, or otherwise expose the address (e.g. oldAddr := fmt.Sprintf("%p", &foo)), etc. you should be aware that addresses can change, and that nor compiler nor runtime will magically patch things for you.
While currently the standard Go compiler only moves objects on the stack (e.g. when a goroutine stack needs to be resized), there is nothing in the Go language specification that prevents a different implementation from moving objects on the Go heap.
While there is (yet) no explicit support for pinning objects in the stack or in the Go heap, there is a recommended workaround: allocate manually the memory outside of the Go heap (e.g. via mmap) and using finalizers to automatically free that allocation once all references to it are dropped. The benefit of this approach is that memory allocated manually outside of the Go heap will never be moved by the Go runtime, so its address will never change, but it will still be deallocated automatically when it's not needed anymore, so it can't leak.