How does Go allocate memory when using new(string) [closed] - go

Closed. This question is not reproducible or was caused by typos. It is not currently accepting answers.
This question was caused by a typo or a problem that can no longer be reproduced. While similar questions may be on-topic here, this one was resolved in a way less likely to help future readers.
Closed 1 year ago.
Improve this question
Consider the following code:
str := new(string)
p := str
fmt.Printf("Current value = %v %p\n", *str, str)
*str = "abc"
fmt.Printf("New value = %v %p\n", *str, str)
fmt.Printf("Current value = %v %p\n", *p, p)
In the second line, p := str both pointers point to the same location, the memory address where new(string) allocated the default value of a string, an empty string. I would have expected *str = "abc" to be a problematic assignment (in c for example, this could overwrite memory). But not only does this work and p take on the same value, it also seems to continue to work if I set *str to be some very large string.
I would expect that str is pointing to a location that has a fixed number of bytes allocated to it. For a default string value, that seems to be 16 bytes. How does golang allow the assignment of some arbitrarily sized string to the location of a pointer?

In the second line, p := str both pointers point to the same location, the memory address where new(string) allocated the default value of a string, an empty string.
Correct
I would have expected *str = "abc" to be a problematic assignment (in c for example, this could overwrite memory). But not only does this work and p take on the same value, it also seems to continue to work if I set *str to be some very large string.
Your expectation is just plain wrong. Go is not C and strings are represented differently in Go than in C.
I would expect that str is pointing to a location that has a fixed number of bytes allocated to it.
Thats true. Any type has some defined fixed size. This is a feature of Go.
For a default string value, that seems to be 16 bytes.
No. 1. This the size of the string variable is not dependent on the string content (also see above). 2. This is platform dependent. 3. Yes 16 bytes = 2 words on 64bit systems.
How does golang allow the assignment of some arbitrarily sized string to the location of a pointer?
A variable of type string doesn't hold the actual string content, it is just a 2 word descriptor, roughly (length, pointer-to-actual-content). *str = "abc" doesn't store "abc" in the string variable: It stores "abc" somewhere and make the pointer-to-actual-content point there. The type string in Go is not what a char* is in C.

Related

Go pointer subtraction [closed]

Closed. This question is not reproducible or was caused by typos. It is not currently accepting answers.
This question was caused by a typo or a problem that can no longer be reproduced. While similar questions may be on-topic here, this one was resolved in a way less likely to help future readers.
Closed 1 year ago.
Improve this question
When I run this program it looks like the value of the two pointers is 16 bytes (x’10”) bytes apart but how can that be if the first string is over 16 bytes long? Or am I looking at this incorrectly?
package main
import (
"fmt"
"math"
)
func main() {
fmt.Println("Hello, playground")
x := "string 4"
xx := "string2"
y := "string3"
xptr := &x
fmt.Println(xptr)
fmt.Println(*xptr)
xxptr := &xx
fmt.Println(xxptr)
fmt.Println(*xxptr)
fmt.Println("hey")
fmt.Println("hey")
fmt.Println("hey")
fmt.Println("hhey")
fmt.Println("hey")
yptr := &y
fmt.Println(yptr, *yptr)
xxx := math.Pow(2,3)
fmt.Printf("%.6f",xxx)
}
The program prints the addresses of the string variables, not the address of the array of bytes backing the strings. A string variable contains a pointer to the string data and the string length. The structure of a string variable is reflected in the reflect.StringHeader type:
type StringHeader struct {
Data uintptr
Len int
}
See Go Data Structures for a detailed description of the string memory layout in Go.
On a 64-bit architecture, the size of a string variable is 16 bytes.
Use the unsafe package to extract the pointer to the string data:
xh := (*reflect.StringHeader)(unsafe.Pointer(&x))
xxh := (*reflect.StringHeader)(unsafe.Pointer(&xx))
fmt.Printf("0x%x %d\n", xh.Data, xh.Len) // prints 0x4c0648 26
fmt.Printf("0x%x %d\n", xxh.Data, xxh.Len) // prints 0x4bcfd4 7

Why is the golang range operator implemented with declaration of a local variable? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
When looping using the golang range operator combined with the address-of & operator we can get a somewhat unexpected behaviour.
Take the example:
list := []int{1, 2}
pointerList := []*int{}
for _, value := range list {
pointerList = append(pointerList, &value)
}
fmt.Print(*pointerList[0], *pointerList[1])
// Prints: 2 2
This prints 2 2 because the variable value is only declared once and will change value for each iteration (while the address of value stays the same).
Q: Why does the go compiler not interpret the above code as if value was replaced by list[i], like so:
list := []int{1, 2}
pointerList := []*int{}
for i := range list {
pointerList = append(pointerList, &list[i])
}
fmt.Print(*pointerList[0], *pointerList[1])
// Prints: 1 2
If I'm not mistaken it is possible to do this interpretation for any use of range in a for loop. The behaviour seems more predictable and should be more performant if iterating over large objects (no extra copying to the local variable).
The example code:
https://play.golang.org/p/y89nMxVgBEs
https://play.golang.org/p/qHnJXMuHKdJ
Go does not have reference variables.
It is not possible to create a Go program where two variables share the same storage location in memory. It is possible to create two variables whose contents point to the same storage location, but that is not the same thing as two variables who share the same storage location.
Since list is an slice of int and not a slice of *int, there should not be any expectation that the address of value := list[i] would be the same as the address of list[i]. Since we already know that value is a copy, and that modifying it in any way won't affect list[i], it is far more performant to allocate the memory for an int once, and overwrite it each iteration of the loop, than to allocate a new int every iteration, and garbage collect the old one.

Fastest way to allocate a large string in Go?

I need to create a string in Go that is 1048577 characters (1MB + 1 byte). The content of the string is totally unimportant. Is there a way to allocate this directly without concatenating or using buffers?
Also, it's worth noting that the value of string will not change. It's for a unit test to verify that strings that are too long will return an error.
Use strings.Builder to allocate a string without using extra buffers.
var b strings.Builder
b.Grow(1048577)
for i := 0; i < 1048577; i++ {
b.WriteByte(0)
}
s := b.String()
The call to the Grow method allocates a slice with capacity 1048577. The WriteByte calls fill the slice to capacity. The String() method uses unsafe to convert that slice to a string.
The cost of the loop can be reduced by writing chunks of N bytes at a time and filling single bytes at the end.
If you are not opposed to using the unsafe package, then use this:
p := make([]byte, 1048577)
s := *(*string)(unsafe.Pointer(&p))
If you are asking about how to do this with the simplest code, then use the following:
s := string(make([]byte, 1048577)
This approach does not meet the requirements set forth in the question. It uses an extra buffer instead of allocating the string directly.
I ended up using this:
string(make([]byte, 1048577))
https://play.golang.org/p/afPukPc1Esr

What is the meaning of '*' and '&'?

I am doing the http://tour.golang.org/. Could anyone explain this function to me lines 1,3,5 and 7, especially what '*' and '&' do? By mentioning them in a function declaration, what are they supposed/expected to do? A toy example:
1: func intial1(var1 int, var2 int, func1.newfunc[]) *callproperfunction {
2:
3: addition:= make ([] add1, var1)
4: for i:=1;i<var2;i++ {
5: var2 [i] = *addtother (randomstring(lengthofcurrent))
6: }
7: return &callproperfunction {var1 int, var2 int, func1.newfunc[], jackpot}
8: }
It seems that they are pointers like what we have in C++. But I cannot connect those concepts to what we have here. In other words, what '*' an '&' do when I use them in function declaration in Go.
I know what reference and dereference mean. I cannot understand how we can use a pointer to a function in Go? For example lines 1 and 7, what do these two lines do? The function named intial1 is declared that returns a pointer? And in line 7, we call it with arguments using the return function.
This is possibly one of the most confusing things in Go. There are basically 3 cases you need to understand:
The & Operator
& goes in front of a variable when you want to get that variable's memory address.
The * Operator
* goes in front of a variable that holds a memory address and resolves it (it is therefore the counterpart to the & operator). It goes and gets the thing that the pointer was pointing at, e.g. *myString.
myString := "Hi"
fmt.Println(*&myString) // prints "Hi"
or more usefully, something like
myStructPointer = &myStruct
// ...
(*myStructPointer).someAttribute = "New Value"
* in front of a Type
When * is put in front of a type, e.g. *string, it becomes part of the type declaration, so you can say "this variable holds a pointer to a string". For example:
var str_pointer *string
So the confusing thing is that the * really gets used for 2 separate (albeit related) things. The star can be an operator or part of a type.
Your question doesn't match very well the example given but I'll try to be straightforward.
Let's suppose we have a variable named a which holds the integer 5 and another variable named p which is going to be a pointer.
This is where the * and & come into the game.
Printing variables with them can generate different output, so it all depends on the situation and how well you use.
The use of * and & can save you lines of code (that doesn't really matter in small projects) and make your code more beautiful/readable.
& returns the memory address of the following variable.
* returns the value of the following variable (which should hold the memory address of a variable, unless you want to get weird output and possibly problems because you're accessing your computer's RAM)
var a = 5
var p = &a // p holds variable a's memory address
fmt.Printf("Address of var a: %p\n", p)
fmt.Printf("Value of var a: %v\n", *p)
// Let's change a value (using the initial variable or the pointer)
*p = 3 // using pointer
a = 3 // using initial var
fmt.Printf("Address of var a: %p\n", p)
fmt.Printf("Value of var a: %v\n", *p)
All in all, when using * and & in remember that * is for setting the value of the variable you're pointing to and & is the address of the variable you're pointing to/want to point to.
Hope this answer helps.
Those are pointers like we have in C++.
The differences are:
Instead of -> to call a method on a pointer, you always use ., i.e. pointer.method().
There are no dangling pointers. It is perfectly valid to return a pointer to a local variable. Golang will ensure the lifetime of the object and garbage-collect it when it's no longer needed.
Pointers can be created with new() or by creating a object object{} and taking the address of it with &.
Golang does not allow pointer-arithmetic (arrays do not decay to pointers) and insecure casting. All downcasts will be checked using the runtime-type of the variable and either panic or return false as second return-value when the instance is of the wrong type, depending on whether you actually take the second return type or not.
This is by far the easiest way to understand all the three cases as explained in the #Everett answer
func zero(x int) {
x = 0
}
func main() {
x := 5
zero(x)
fmt.Println(x) // x is still 5
}
If you need a variable to be changed inside a function then pass the memory address as a parmeter and use the pointer of this memory address to change the variable permanently.
Observe the use of * in front of int in the example. Here it just represents the variable that is passed as a parameter is the address of type int.
func zero(xPtr *int) {
*xPtr = 0
}
func main() {
x := 5
zero(&x)
fmt.Println(x) // x is 0
}
Simple explanation.. its just like, you want to mutate the original value
func zero(num *int){ // add * to datatype
*num = 0 // can mutate the original number
}
i := 5
zero(&i) // passing variable with & will allows other function to mutate the current value of variable```
& Operator gets the memory address where as * Opeartor holds the memory address of particular variable.

Skipping ahead n codepoints while iterating through a unicode string in Go

In Go, iterating over a string using
for i := 0; i < len(myString); i++{
doSomething(myString[i])
}
only accesses individual bytes in the string, whereas iterating over a string via
for i, c := range myString{
doSomething(c)
}
iterates over individual Unicode codepoints (calledrunes in Go), which may span multiple bytes.
My question is: how does one go about jumping ahead while iterating over a string with range Mystring? continue can jump ahead by one unicode codepoint, but it's not possible to just do i += 3 for instance if you want to jump ahead three codepoints. So what would be the most idiomatic way to advance forward by n codepoints?
I asked this question on the golang nuts mailing list, and it was answered, courtesy of some of the helpful folks on the list. Someone messaged me however suggesting I create a self-answered question on Stack Overflow for this, to save the next person with the same issue some trouble. That's what this is.
I'd consider avoiding the conversion to []rune, and code this directly.
skip := 0
for _, c := range myString {
if skip > 0 {
skip--
continue
}
skip = doSomething(c)
}
It looks inefficient to skip runes one by one like this, but it's the same amount of work as the conversion to []rune would be. The advantage of this code is that it avoids allocating the rune slice, which will be approximately 4 times larger than the original string (depending on the number of larger code points you have). Of course converting to []rune is a bit simpler so you may prefer that.
It turns out this can be done quite easily simply by casting the string into a slice of runes.
runes := []rune(myString)
for i := 0; i < len(runes); i++{
jumpHowFarAhead := doSomething(runes[i])
i += jumpHowFarAhead
}

Resources