Now I'm learning go memory management, when allocating a small object, the allocator will look in the corresponding mspan in mcache, and we also know goroutine has a stack memory, so does that mean the spans in mcache belongs to stack memory(not heap)? What happened in memory if we define a varible in a func like a := 1?
this might be helpful https://go101.org/article/memory-block.html.
Also you can see what exact allocation is happening in memory using
go build -gcflags=-m your_file.go
Only things which used dynamically those only assigned to the heap.
package main
import "fmt"
func f() *int {
var x int
return &x
}
func main() {
fmt.Println(f())
}
go build -gcflags=-m encyptionDecryption.go
./encyptionDecryption.go:5:6: can inline f
./encyptionDecryption.go:11:15: inlining call to f
./encyptionDecryption.go:7:9: &x escapes to heap
./encyptionDecryption.go:6:6: moved to heap: x
./encyptionDecryption.go:11:15: f() escapes to heap
./encyptionDecryption.go:11:15: &x escapes to heap
./encyptionDecryption.go:11:15: moved to heap: x
./encyptionDecryption.go:11:13: main ... argument does not escape
This is escape analysis. You can read more here
Related
i read the golang FAQ:https://go.dev/doc/faq#stack_or_heap，i want to know when golang allocate variable on stack or heap. so i write code like below :
package main
import (
"fmt"
)
type Object struct {
Field int
}
func main() {
A := Object{1}
B := Object{2}
fmt.Println(A,B)
//fmt.Printf("A:%p;B:%p\n",&A,&B)
//m := testStackOrHeap()
//C:=m[A]
//D:=m[B]
//fmt.Printf("C:%p;D:%p\n",&C,&D)
}
//go:noinline
func testStackOrHeap() map[Object]Object {
one:=1
two:=2
A := Object{one}
B := Object{two}
C:= Object{one}
D := Object{two}
fmt.Println(C,D)
fmt.Printf("A:%p;B:%p\n",&A,&B)
m := map[Object]Object{A: A, B: B}
return m
}
then see how the compiler allocate the memory .the cmd is go tool compile "-m" main.go
the output is below :
main.go:15:13: inlining call to fmt.Println
main.go:30:13: inlining call to fmt.Println
main.go:31:12: inlining call to fmt.Printf
main.go:15:13: A escapes to heap
main.go:15:13: B escapes to heap
main.go:15:13: []interface {} literal does not escape
main.go:26:2: moved to heap: A
main.go:27:2: moved to heap: B
main.go:30:13: C escapes to heap
main.go:30:13: D escapes to heap
main.go:30:13: []interface {} literal does not escape
main.go:31:12: []interface {} literal does not escape
main.go:32:24: map[Object]Object literal escapes to heap
<autogenerated>:1: .this does not escape
my question is:
why not golang allocate variable A B in testStackOrHeap() to the stack ,they can not escape to stackframe ,if it allocate to heap , the gcworker need to collect it,but if it allocate in stack, it will release when function return.
As #Volker pointed out in a comment, the heap/stack distinction is an implementation detail, and the rules for escape analysis are defined by the compiler, not by the language. Correctness is the most important trait of a compiler, so a compiler's rules will frequently favor simplicity and performance over absolute "optimalness".
In this case, it's quite likely that the compiler doesn't know what fmt.Printf() will do with the pointers it receives. Therefore, it has to assume that the pointers might be stored somewhere on the heap by that function and that the references to those two objects might thus survive the call to testStackOrHeap(). Therefore, it errs on the side of caution and promotes those two variables to the heap.
(Note that your conclusion that they do not escape was presumably based on an assumption that fmt.Printf() won't store the pointers. Did you actually read the source code of that function to learn that it doesn't? If not, you can't actually be sure that it doesn't - just like the compiler isn't sure. And even if the current version of that function doesn't, future versions might.)
All my questions are written in code ...
I have no idea about if golang has heap memory and stack memory.
If a func has its own stack memory, does another func can read it ??!
the code:
package main
import (
"fmt"
"math/rand"
"strconv"
"time"
)
type People struct {
name string
age int
}
func receiver(ch chan *People){
for{
// 2. Why the pointer a variable in receiver stack returned by <-ch can reach
// the stack belongs to sender?
fmt.Println((<-ch).name)
}
}
func sender(ch chan *People){
for{
time.Sleep(3*time.Second)
// 1. the People instance is created by sender and
// the pointer may points to the local stack in sender's call stack.
ch <- &People{name: strconv.Itoa(rand.Int())}
}
}
func main(){
ch := make(chan *People, 10)
go receiver(ch)
go sender(ch)
// join...
time.Sleep(time.Hour)
}
Go compiler does escape analysis to check whether an object should be on stack OR heap. The language specs doesn't make a distinction though, which means the compiler can take different course in future than exists today. By course I mean optimization and decision technique than one exists today not the genesis.
Check this stack or heap.
If you build the program with escape analysis flags:
go build -gcflags="-m"
You will notice that it shows:
(<-ch).name escapes to heap
...
"&People literal escapes to heap"
So go compiler by virtue of escape analysis knows before hand what things could be on stack and what should be on heap.
=> so the things that you have doubt on are escaped to be on heap.
In plain C, if I want a shallow heap copy of a struct, I would malloc() and memcpy() it.
In Go, I guess I have to do something like this:
original := Data{...}
copy := &Data{} // malloc
*copy = original // memcpy
But it doesn't look nice to me, nor idiomatic. What's the "right" way to do it?
The idiomatic way is to do a simple assignment and let the compiler allocate copy on the heap after performing escape analysis:
original := Data{...}
copy := original
return © // Or call some function with © as a parameter
Upon noticing that copy is used by reference and outlives the stack, Go will automatically allocate it on the heap rather than on the stack (the copy is still done properly of course)
We effectively no longer care about the heap, letting the compiler allocate it there as needed based on escape analysis. Our only concern is the copy itself.
You can see an example in action on godbolt:
Given the following simple code:
func main() {
type Data struct{
foo string
}
original := Data{"hi"}
copy := original
copyPtr := ©
fmt.Println(copyPtr)
}
Go will automatically allocate copy on the heap:
call runtime.newobject(SB)
We can also see this in action by passing extra flags at compile time showing escape and inlining decisions:
$ go build -gcflags '-m' .
...
./main.go:11:2: moved to heap: copy
...
Note: copy is a builtin function. It might be a good idea to avoid reusing the name (it works just fine, but it's not great practice).
A struct variable in Golang can be copied to another simply by an assignment statement:
https://play.golang.org/p/4Zcbxhy5UoB
package main
import (
"fmt"
)
type User struct {
name string
}
func main() {
u1 := User{name: "foo"}
u2 := u1
u2.name = "bar"
fmt.Println("u1: ", u1)
fmt.Println("u2: ", u2)
}
output:
u1: {foo}
u2: {bar}
I'm learning Go and have a C/C++ background. In the following example, is it safe to append the address of a into slice? When I run this example, the correct value (2) is printed, but wanted to be sure. If this is wrong, how should I do it?
func add(mapping map[string]*[]*int) {
sliceptr := &[]*int{}
mapping["foo"] = sliceptr
ele := mapping["foo"]
a := 2
// won't address of `a` go out of scope?
ele2 := append(*ele, &a)
mapping["foo"] = &ele2
}
func main() {
mapping := map[string]*[]*int{}
add(mapping)
fmt.Println(*(*mapping["foo"])[0])
}
It's safe to reference a after the function declaring it ends, because go does escape analysis. If the compiler can prove it can be accessed safely, it puts it on the stack, if not, it allocates it on the heap.
Build flags can give some insight into the escape analysis:
go build -gcflags "-m" main.go
...
./main.go:10:2: moved to heap: a
...
This might be helpful: Allocation efficiency.
Also, it's less common to see pointers to slices, since a slice is small: a pointer, length and capacity. See slice internals.
http://golang.org/ref/spec#Allocation
There is a way to allocate memory, but I don't see a way to deallocate memory (without the Go GC turned on).
If I wanted to write an OS using Go, I would need to either write a low level GC for Go or disable the Go GC. In the latter case, how could I free memory?
PS - this topic has been talked about extensively on the Go mailing list, but I wanted to pose this specific question to SO.
You can free arbitrary memory by making runtime·free accessible to your program
using cgo.
Build your own package called, for example, mem and create two files:
mem.go
package mem
import "unsafe"
import "reflect"
func FreePtr(p unsafe.Pointer)
func Free(v interface {}) {
FreePtr(unsafe.Pointer(reflect.ValueOf(v).Elem().Pointer()))
}
runtime.c
// +build gc
#include <runtime.h>
void ·Free(void* foo) {
runtime·free(foo);
}
Example usage (free a slice and print number of frees):
import "free/mem"
func main() {
var m1, m2 runtime.MemStats
runtime.ReadMemStats(&m1)
c := make([]int, 10000)
inspect.Free(&c)
runtime.ReadMemStats(&m2)
fmt.Printf("%d vs %d\n", m1.Frees, m2.Frees)
}
You should see that there was one more free than before.