use unsafe pointer to get value from a []string - go

I am trying to learn how pointers work on go. Why is the following example not working?
package main
import (
"fmt"
"unsafe"
)
type SliceOfStrings []string
// function that creates an slice of []string
// returns interface{} cause I am interested on learning how pointers work
func Create() interface{} {
var mySlice1 SliceOfStrings = make([]string, 0)
mySlice1 = append(mySlice1, "str1")
mySlice1 = append(mySlice1, "str2")
// return a slice with as ["str1","str2"]
return mySlice1
}
func main() {
x := Create()
// 0xc000021940
fmt.Printf("address of x is %p \n", &x)
// get unsafe pointer to address of x
// unsafe pointer. Prints 0xc000021940
p1 := unsafe.Pointer(&x)
fmt.Println(p1)
// unsigned pointer. Prints 824633858368
p2 := uintptr(p1)
fmt.Println(p2)
// prints same value as p1 0xc000021940
p3 := unsafe.Pointer(p2)
fmt.Println(p3)
// Make p4 point to same address as 0xc000021940
p4 := (*SliceOfStrings)(p3)
//fmt.Println(p4)
// why this does not print "str1" ??
fmt.Println((*p4)[0])
// I get error: runtime error: invalid memory address or nil pointer dereference
}

Create() returns a value of type interface{}, so type of x is interface{}, so type of &x is *interface{}, and not *SliceOfStrings. So x points to an interface{} value and not to a SliceOfStrings value!
If you type assert SliceOfStrings from the return value of Create(), it works:
x := Create().(SliceOfStrings)
Also add runtime.KeepAlive(x) at the end of your main(), because if you don't refer to x anymore, it can be garbage collected at any time.
With this change it works and outputs str1. Try it on the Go Playground.
In general, stay away from package unsafe as much as possible. You can learn and use pointers without package unsafe. Only think of unsafe as a last-last resort!

I was able to understand why this happens:
package main
import (
"fmt"
"unsafe"
)
type SliceOfStrings []string
// when passing a slice to a method you are passing this data. Lets prove it
type SliceHeader struct {
Data uintptr
Len int
Cap int
}
func main() {
// On go everything is passed by coping values. When you pass a slice to a function you are passing this:
// reference: https://stackoverflow.com/a/39993797/637142
/*
type SliceHeader struct {
Data uintptr
Len int
Cap int
}
*/
// create a new slice
var mySlice1 SliceOfStrings = make([]string, 0)
// when appending we need to keep the new content that is why we reasig it
mySlice1 = append(mySlice1, "str1")
mySlice1 = append(mySlice1, "str2")
// in other words this will make no sense:
// _ = append(mySlice1, "str3") // doing this will lose the new header value
// lets see the content of header mySlice
var pointerToMySlice1 = unsafe.Pointer(&mySlice1)
var header *SliceHeader = ((*SliceHeader)(pointerToMySlice1))
fmt.Println(*header)
// {824634220576 2 2}
// lets copy that header to another slice
var copy SliceOfStrings = mySlice1
var pointerToCopy = unsafe.Pointer(&copy)
header = ((*SliceHeader)(pointerToCopy))
fmt.Println(*header)
// prints the same thing
// {824634220576 2 2}
// now lets do the same thing but with an interface
var copy2 interface{} = mySlice1
var pointerToCopy2 = unsafe.Pointer(&copy2)
header = ((*SliceHeader)(pointerToCopy2))
fmt.Println(*header)
// this prints!
// {4845280 824634375976 0}
// I dont understand why this happens. But this is the reason why the example does not work
// I was trying to access an array from memory address 4845280 that is wrong
// now lets do what izca told us
var copy3 interface{} = mySlice1
tmp := (copy3).(SliceOfStrings)
var pointerToCopy3 = unsafe.Pointer(&tmp)
header = ((*SliceHeader)(pointerToCopy3))
fmt.Println(*header)
// this prints!
// {824634220576 2 2}
// that is the correct value
// lets try to get the array from that memory address (824634220576)
pointerToActualArray := unsafe.Pointer(header.Data)
// lets cast that to (*[2]string)
var pointerFinal *[2]string = ((*[2]string)(pointerToActualArray))
// now print the first value
fmt.Println((*pointerFinal)[0])
// prints str1
}

Related

Copying struct value to a interface{ } in golang

I will like to understand why when I copy a struct value into an interface it behaves like it does. In this code can someone help me understand why when I copy the value from mySlice into foo3 it behaves different than the other copies?
package main
import (
"fmt"
"unsafe"
)
type SliceOfInt []int
// when passing a slice to a method you are passing this data. Lets prove it
type SliceHeader struct {
Data uintptr
Len int
Cap int
}
// Print the header of a slice. Its Data. Len and Cap
func GetHeaderOfSliceOfInt(s unsafe.Pointer) *SliceHeader {
// this header needs to point to &mySlice but compiler will not let us. we have to use unsafe pointers
var header *SliceHeader
pointerToMySlice := s
header = ((*SliceHeader)(pointerToMySlice))
return header
}
func main() {
// On go everything is passed by coping values. When you pass a slice to a function you are passing this:
// reference: https://stackoverflow.com/a/39993797/637142
/*
type SliceHeader struct {
Data uintptr
Len int
Cap int
}
*/
// create a new slice
var mySlice SliceOfInt = make([]int, 0)
mySlice = append(mySlice, 123456789) // append this number to mySlice
// now we have a slice with len:1 and capacity:1. Lets prove it
header := GetHeaderOfSliceOfInt(unsafe.Pointer(&mySlice))
fmt.Println(*header)
// this prints: {824635465728 1 1}
// this means that on memory address 824635465728 there is an array with cap:1 and len:1
// copy that header to someOtherSlice
someOtherSlice := mySlice
header = GetHeaderOfSliceOfInt(unsafe.Pointer(&someOtherSlice))
fmt.Println(*header)
// prints the same value {824635465728 1 1}
// anyways if I go to address 824635465728 on my computer I shoul dbe able to find integer 123456789
pointerToInteger := unsafe.Pointer((*header).Data)
var integerVal *int = ((*int)(pointerToInteger))
fmt.Println(*integerVal)
// if I copy like this, it will print the correct header {824635465728 1 1}
foo1 := mySlice
fmt.Println(*GetHeaderOfSliceOfInt(unsafe.Pointer(&foo1)))
// copy like this will also print the correct header {824635465728 1 1}
foo2 := foo1
fmt.Println(*GetHeaderOfSliceOfInt(unsafe.Pointer(&foo2)))
// If I copy like this it will print the incorrect header. Why?
var foo3 interface{} = mySlice
fmt.Println(*GetHeaderOfSliceOfInt(unsafe.Pointer(&foo3)))
// this last line prints {4746976 824635330392 0}
}
The output of the program is:
{824635465728 1 1}
{824635465728 1 1}
123456789
{824635465728 1 1}
{824635465728 1 1}
{4746976 824635330392 0}
Edit
I know that if I cast foo3 as: foo3.(SliceOfInt) it will work. But why is that?
An interface type, empty or not, is a type in its own right. It has its own memory representation and it is a legitimate member of Go's type system.
An interface value, and the value wrapped in that interface, are not one and the same.
The variables foo1 and foo2 have the same type and value as the mySlice variable. But the variable foo3 has a different type, therefore also a different value. And yes, the dynamic type and value are the same as mySlice but the static type and value are not.
An interface value is NOT represented in memory by a structure that's compatible with the three-field SliceHeader and therefore it is wrong, not only semantically, to try to take the slice header off of an interface value. Instead, an interface value is represented by a 2-field structure (that's why in your attempt the third field is 0). The first field points to the type information of the wrapped value, the second field points to the data of the wrapped value.
Something like this:
type iface struct {
typ uintptr
data uintptr
}
And you can test this by doing this:
x := (*iface)(unsafe.Pointer(&foo3))
s := (*SliceHeader)(unsafe.Pointer(x.data))
fmt.Printf("%+v\n", x)
fmt.Printf("%+v\n", s)
https://go.dev/play/p/2KUgCU8h7O7
Also, consider reading this: https://research.swtch.com/interfaces.

Converting []*string to []string in golang

i'm new to go.
i have a function that accepts []string as input, but i the input i have to pass is an []*string, how do i convert []*string to []string.
is there any way to convert it using any utilities, or do i have to iterate it using a for-loop and construct an array?
Playground link https://play.golang.org/p/_s2g7-IfGAy
package main
import (
"fmt"
)
func main() {
//Declaring an array of []*string and assigning value to it
var a [1]*string
var strPointer = new(string)
*strPointer = "1"
a[0] = strPointer
fmt.Println(*a[0])
// accept(a) this wont work
//Converting to array []string
var b []string
for i := range a {
b = append(b, *a[i])
}
accept(b)// this works
fmt.Println(b)
}
func accept(param []string) {
fmt.Println("Works!")
}
Your accept(param []string) expect a slice of string.
var a [1]*string This declares Go array with a length of 1. So it's not a slice.
You can declare an empty slice of string pointers using this. var a []*string
And you have to iterate through the array and make a slice with the value of pointer elements and call the accept with that slice.
Example function to convert []*string to []string
func stringer(str []*string) []string{
var strs []string
for _, s := range str {
if s == nil {
strs = append(strs, "")
continue
}
strs = append(strs, *s)
}
return strs
}
run here
how do i convert []*string to []string
You cannot. This kind of type conversion not possible in Go.
is there any way to convert it using any utilities [...]
Asking for 3rd party libraries/tools/packages is OT on SO.
[...] or do i have to iterate it using a for-loop and construct an array
This is the only clean, normal, "idiomatic" way of doing this.

golang copy pointer value from pointer type slice

type Order struct {
*Res
Status int
}
type Res struct {
ResID int64
OtaBookID string
StayDetail []*ResElement
TotalCharge float64
CustFName string
CustLName string
CreateTime time.Time
}
type ResElement struct {
Res *Res
OtaEleID string
OtaRoomID string
RoomID int
Arrival time.Time
Depart time.Time
Charge float64
CreateTime time.Time
}
I have a slice named orderList to record orders.
Now i have the first order which OtaBookID is "A123", and i want to copy this order and change its OtaBookID to "B123". The ResElement type have some other detail info to record this order and i hide them because it not affect my question. I can success change Res.OtaBookID but i have no idea how to change Res.StayDetail[0].Res.OtaBookID
func main() {
var orderList []*Order
res := new(Res)
res.OtaBookID = "A123"
resElt := new(ResElement)
resElt.Res = res
res.StayDetail = append(res.StayDetail, resElt)
order := new(Order)
order.Res = res
orderList = append(orderList, order)
originalOrder := new(Order)
originalOrder.Res = new(Res)
*originalOrder.Res = *order.Res
//originalOrder.Res.StayDetail[0].Res.OtaBookID = "B123" //this will make all become "B123"
originalOrder.Res.OtaBookID = "B123"
orderList = append(orderList, originalOrder)
fmt.Println(orderList[0].Res.OtaBookID) //A123
fmt.Println(orderList[1].Res.OtaBookID) //B123
fmt.Println(orderList[0].Res.StayDetail[0].Res.OtaBookID) //A123
fmt.Println(orderList[1].Res.StayDetail[0].Res.OtaBookID) //A123, i want this become B123
}
The result i want is orderList[0].Res.OtaBookID and orderList[0].Res.StayDetail[0].Res.OtaBookID is "A123" and other is "B123"
ps: Because i'm taking over other's works. So the three type already fixed and can not adjust. I want to know in this situation whether have any method to arrive my goal
From your example and clarification we can focus on main to achieve your goal:
type Order struct {
*Res
Status int
}
type Res struct {
ResID int64
OtaBookID string
StayDetail []*ResElement
TotalCharge float64
CustFName string
CustLName string
CreateTime time.Time
}
type ResElement struct {
Res *Res
OtaEleID string
OtaRoomID string
RoomID int
Arrival time.Time
Depart time.Time
Charge float64
CreateTime time.Time
}
func main() {
var orderList []*Order
res := new(Res)
res.OtaBookID = "A123"
resElt := new(ResElement)
resElt.Res = res
res.StayDetail = append(res.StayDetail, resElt)
order := new(Order)
order.Res = res
orderList = append(orderList, order)
originalOrder := new(Order)
originalOrder.Res = new(Res)
// The following statement will copy the values in the memory space of order to the memory
// space of originalOrder, but take note that a pointer contains a memory address. Thus,
// following this statement originalOrder.Res.StayDetail will point to the same address
// as order.Res.StayDetail as slices are pointers...
*originalOrder.Res = *order.Res
// We now have to manually copy the slice to ensure it is different from the one in order.
// We must create a new slice to avoid overwriting the source.
originalOrder.Res.StayDetail := make([]*ResElement, len(order.Res.StayDetail))
// Further, this slice contains pointers so we must copy these as well to ensure they do not
// point to the ResElements of order.
for i, v := range order.Res.StayDetail {
re := new(ResElement)
// copy the values of the memory in v to the values in the memory of re
*re = *v
// set re.Res to point to originalOrder.Res as it currently point to order.Res
re.Res = originalOrder.Res
// Now we must place re in the slice of originalOrder as it currently points to an
// empty array of the correct size. Thus, we will not use append (which will grow
// the size), we will just set the index to the correct value. Thus:
originalOrder.StayDetail[i] = re
}
// The pointers and structures have now been properly set up, thus the following are equivalent:
// originalOrder.OtaBookID = "B123" - Res via embedding
// originalOrder.Res.OtaBookID = "B123" - Res directly
// originalOrder.Res.StayDetail[0].Res.OtaBookID = "B123" - Res via ResElement pointer Res
// They all point to the exact same memory space
originalOrder.Res.OtaBookID = "B123"
orderList = append(orderList, originalOrder)
fmt.Println(orderList[0].Res.OtaBookID) //A123
fmt.Println(orderList[1].Res.OtaBookID) //B123
fmt.Println(orderList[0].Res.StayDetail[0].Res.OtaBookID) //A123
fmt.Println(orderList[1].Res.StayDetail[0].Res.OtaBookID) //B123
}
Always remember that a pointer is just a memory address. If the address value is 0xffab67e1 a copy of the pointer will also contain the same value of 0xffab67e1 and effectively point to the same thing. Thus you must deliberately assign new memory space at a new address and copy the internal values. This must be done for any structures that contain pointers. We call this a Deep Copy or a Clone.

Expanding a slice's size to prevent slice bounds out of range error

I have written the following:
func main() {
//inside main
fileInputBytes, err := ioutil.ReadFile("/tmp/test")
byteSize2 := len(fileInputBytes)
var inputFileByteSlice = fileInputBytes[0:]
var numberOfIndexes = math.Floor(float64(byteSize / indexingOffset))
for i := 1; i <= int(numberOfIndexes); i++ {
// adding i to the indexer insures that we use lookahed to ignore previously inserted indexing values
var v int = (i * indexingOffset) + i
Insert(&inputFileByteSlice, v+i, indexingByteValue)
fmt.Println(i)
}
}
//outside main
//variation of https://blog.golang.org/slices with pointers and such
func Insert(slice *[]byte, index int, value byte) {
// Grow the slice by one element.
(*slice) = (*slice)[0 : len(*slice)+1]
// Use copy to move the upper part of the slice out of the way and open a hole.
copy((*slice)[index+1:], (*slice)[index:])
// Store the new value.
(*slice)[index] = value
// Return the result.
}
The slice bounds out of range error is getting on my nerves. The length of the slice grows outside of the size and overflows, the reason I don't understand is that I thought the call to 'grow' the slice by one element(before copy) will dynamically allocate more space. Since that is not the case, can anyone offer me a better suggestion?
First of all, a slice is already a reference type. So you don't need to pass its pointer around if you are not going to change its capacity. So your main can be simplified as:
func main() {
fileInputBytes, err := ioutil.ReadFile("/tmp/test")
byteSize2 := len(fileInputBytes)
// No need to use pointer to slice. If you want a brand new slice
// that does not affect the original slice values, use copy()
inputFileByteArray := fileInputBytes
var numberOfIndexes = math.Floor(float64(byteSize / indexingOffset))
for i := 1; i <= int(numberOfIndexes); i++ {
var v int = (i * indexingOffset) + i
// Insert needs to return the newly updated slice reference
// which should be assigned in each iteration.
inputFileByteArray = Insert(inputFileByteArray, v+i, indexingByteValue)
fmt.Println(i)
}
}
Then, the Insert function can be simplified simply by using append along with copy and returning the newly created slice:
func Insert(slice []byte, index int, value byte) []byte {
if index >= len(slice) {
// add to the end of slice in case of index >= len(slice)
return append(slice, value)
}
tmp := make([]byte, len(slice[:index + 1]))
copy(tmp, slice[:index])
tmp[index] = value
return append(tmp, slice[index:]...)
}
This may not be the best implementation but it is simple enough. Example usage at: https://play.golang.org/p/Nuq4RX9XQD
Your function only works if the slice happens to have enough initial capacity. If you need more capacity, you can only "grow" the slice using the append function. You can still use the *[]byte pointer argument to modify the slice in place like so:
func Insert(slice *[]byte, index int, value byte) {
*slice = append(*slice, 0)
copy((*slice)[index+1:], (*slice)[index:])
(*slice)[index] = value
}
However, it's more customary to return a new slice value, and reassign it each time. This gives you a similar function signature to the builtin append.
func Insert(slice []byte, index int, value byte) []byte {
slice = append(slice, 0)
copy(slice[index+1:], slice[index:])
slice[index] = value
return slice
}

Pass By reference in golang [duplicate]

Say I want to pass a pointer to a function and change the value of a struct which that pointer points to by doing so. I would normally do this by dereferencing the pointer:
type Test struct { Value int}
func main() {
var i Test = Test {2}
var p *Test = &i
f(p)
println(i.Value) // 4
}
func f(p *Test) {
*p = Test{4}
}
My question is, why this code doesn't change the value
type Test struct { Value int}
func main() {
var i Test = Test {2}
var p *Test = &i
f(p)
println(i.Value) // 2
}
func f(p *Test) {
// ?
p = &Test{4}
}
while this one does:
type Test struct { Value int}
func main() {
var i Test = Test {2}
var p *Test = &i
f(p)
println(i.Value) // 4
}
func f(p *Test) {
p.Value = 4
}
Because this line:
p = &Test{4}
Just assigns a new pointer value to the p variable. Inside the f() function, p is just a local variable. By assigning any new value to p, you are just changing the value of the local variable and not the pointed value.
The p local variable in f() has nothing to do with the p local variable in main(). If you change p in f(), it will not change p in main() (and it won't change the pointed struct value either).
In your second example:
p.Value = 4
It is a shorthand for:
(*p).Value = 4
This changes the pointed value, hence you will observe the change when f() returns.
Note:
Just as a side note, if in your main() function you would pass the address of p (the local variable in main() which is a pointer) to function f(), you could modify the address stored in main's p:
func f(p **Test) {
*p = &Test{4}
}
And from main(), call it like:
var i Test = Test{2}
var p *Test = &i
f(&p)
println(i.Value) // 2 - Note that 'i' won't change!
println(p.Value) // 4 - Only the address in 'p' to the new struct value created in f()
But obviously passing a single pointer *Test and modifying the pointed value (p.Value = 4) is more efficient, much more convenient and much cleaner.

Resources