I have the following variables:
var foo int8
var bar [5]int8
I want to count number of bytes in both variables and output the sum - so right here it should be 1 + 5 = 6. Is this possible?
You can use reflect.Size, this returns a uintptr for some reason, but is actually just the exact same thing as unsafe.SizeOf without having to use unsafe.
var x [8]byte
t := reflect.TypeOf(x)
fmt.Println(t.Size())
The snags are going to be maps, slices, strings, and pointers, which you'll get the size of the header/metadata (or just the pointer size for pointers). If that's okay, great! If not, you can handle this in various ways, this is the best I have off the top of my head:
func UnderlyingSize(x interface{}) uintptr {
v := reflect.ValueOf(x)
t := v.Type()
var size uintptr;
switch t.Kind() {
// For the builtin collections, we just multiply the len by the
// element size, for maps also do the key
case reflect.Map:
l := uintptr(v.Len())
size = t.Key().Size()*l + t.Elem().Size()*l
case reflect.Slice:
t := t.Elem();
size = t.Size() * uintptr(v.Len())
case reflect.Ptr:
t := t.Elem();
size = t.Size()
// Strings are just byte arrays, so it's just the len
case reflect.String:
size = uintptr(v.Len())
// For an interface, we need to find the underlying type
case reflect.Interface:
v := v.Elem()
size = UnderlyingSize(v)
// For anything else, including arrays, Size returns the correct value
default:
size = t.Size();
}
return size
}
There is an argument to be made for using Cap rather than Len, but it's easy enough to change yourself. You can also add t.Size() to any of these values if you want the size of the header information AND the underlying size. Note a word of warning that the real map probably takes more memory than just the key+value+header size, since there's probably some extra information under the hood.
If you have a data structure that's a collection, you'll have to implement something like this yourself, but if it's a simple struct (i.e. only made of POD structs and builtin types), you can simply add up UnderlyingSize of all the members.
You can use unsafe.Sizeof (https://play.golang.org/p/FroasKud7I):
unsafe.Sizeof(foo) + unsafe.Sizeof(bar)
Related
In go, I'm trying to iterate through an array that was created in C. I have the length of the array (int arrc) and its pointer (mytype *arrv).
I have found a way, but it involves transferring back and forth between go and c and is super super hacky.
// void *nextelement(void *p, int i, int size) {
// return (void*)((uint64_t)p+i*size);
// }
#import "C"
...
for i := 0; i < protoc; i++ {
adr := (*C.mytype)(C.nextelement(unsafe.Pointer(myarr), C.int(i), C.sizeof_mytype))
All that code just to get myarr[i]... it doesn't feel right.
Note: this is simply copied from the CGo documentation.
Turning C arrays into Go slices
C arrays are typically either null-terminated or have a length kept
elsewhere.
Go provides the following function to make a new Go byte slice from a
C array:
func C.GoBytes(cArray unsafe.Pointer, length C.int) []byte
To create a Go slice backed by a C array (without copying the original
data), one needs to acquire this length at runtime and use a type
conversion to a pointer to a very big array and then slice it to the
length that you want (also remember to set the cap if you're using Go
1.2 or later), for example (see http://play.golang.org/p/XuC0xqtAIC for a runnable example):
var theCArray *C.YourType = C.getTheArray()
length := C.getTheArrayLength()
slice := (*[1 << 28]C.YourType)(unsafe.Pointer(theCArray))[:length:length]
It is important to keep in mind that the Go garbage collector will not
interact with this data, and that if it is freed from the C side of
things, the behavior of any Go code using the slice is
nondeterministic.
The reason for the magic array size constant is described in What does (*[1 << 30]C.YourType) do exactly in CGo? The actual value of the constant, whether it's 1 << 30 or 1 << 28 or whatever, is not crucial, but it must be at least as large as the largest length value will be.
I am working with a very large map of pointer to struct. It is growing over the lifetime of the program (I use it as a buffer) and I wrote a function that is supposed to reduce it size when it is called.
type S struct {
a uint32
b []uint32
}
s := make(map[uint32]*S)
for k, v := range s {
delete(s, k)
s[k] = &S{a: v.a}
}
I remove b from every element of the map, so I expected the size of the map in memory to shrink (b is a slice of length > 10). However the memory is not freed, why?
The size of the map value &S, a pointer, is the same irrespective of the capacity of slice b.
package main
import (
"fmt"
"unsafe"
)
type S struct {
a uint32
b []uint32
}
func main() {
size := unsafe.Sizeof(&S{})
fmt.Println(size)
size = unsafe.Sizeof(&S{b: make([]uint32, 10)})
fmt.Println(size)
s := make(map[uint32]*S)
for k, v := range s {
delete(s, k)
s[k] = &S{a: v.a}
}
}
Output:
8
8
A slice is represented internally by
type slice struct {
array unsafe.Pointer
len int
cap int
}
When you set &S{a: v.a} you set b to initial values: array to nil and len and cap to zero. The memory formerly occupied by the underlying array is returned to the garbage collector for reuse.
The map size is bounded to the maximum size it had at any point. Because you store pointers (map[uint32]*S) and not values the deleted objects will get garbage collected eventually but usually not immediately and that why you don’t see it in top/htop like monitors.
The runtime is clever enough and reserves memory for future use if the system is not under pressure or low on resources.
See https://stackoverflow.com/a/49963553/1199408 to understand more about memory.
In your example you don't need to call delete. You will achieve what you want just by clearing the slice in the struct.
type S struct {
a uint32
b []uint32
}
s := make(map[uint32]*S)
for k, v := range s {
v.b = []uint32{}
}
I have two variables with big numbers set as strings:
var numA = "340282366920938463463374607431768211456"
var numB = "17014118346046923173168730371588410572"
I want to be able to add and subtract these kinds of large string numbers in Go.
I know I need to use math/big but I still can not for the life of me figure out how, so any example help will be greatly appreciated!
You may use big.NewInt() to create a new big.Int value initialized with an int64 value. It returns you a pointer (*big.Int). Alternatively you could simply use the builtin new() function to allocate a big.Int value which will be 0 like this: new(big.Int), or since big.Int is a struct type, a simple composite literal would also do: &big.Int{}.
Once you have a value, you may use Int.SetString() to parse and set a number given as string. You can pass the base of the string number, and it also returns you a bool value indicating if parsing succeeded.
Then you may use Int.Add() and Int.Sub() to calculate the sum and difference of 2 big.Int numbers. Note that Add() and Sub() write the result into the receiver whose method you call, so if you need the numbers (operands) unchanged, use another big.Int value to calculate and store the result.
See this example:
numA := "340282366920938463463374607431768211456"
numB := "17014118346046923173168730371588410572"
ba, bb := big.NewInt(0), big.NewInt(0)
if _, ok := ba.SetString(numA, 10); !ok {
panic("invalid numA")
}
if _, ok := bb.SetString(numB, 10); !ok {
panic("invalid numB")
}
sum := big.NewInt(0).Add(ba, bb)
fmt.Println("a + b =", sum)
diff := big.NewInt(0).Sub(ba, bb)
fmt.Println("a - b =", diff)
Output (try it on the Go Playground):
a + b = 357296485266985386636543337803356622028
a - b = 323268248574891540290205877060179800884
I am trying to assign a value to the empty slice as follows.
func main() {
var s []int
fmt.Println(s, len(s), cap(s))
s[0] = 99
}
And it throws an exception,
panic: runtime error: index out of range
Note:
I know one way of doing this by initializing the value at declaration part as follows. But in the above example I am trying to assign a value after the declaration.
var s []int{99}
Is there a way to achieve this?
Empty slices cannot just be assigned to. Your print statement shows that the slice has length and capacity of 0. Indexing at [0] is definitely out of bounds.
You have (at least) three choices:
Append to the slice: s = append(s, 99)
or Initialize the slice to be non-empty: s := make([]int, 1)
or Initialize your slice with the element you want: s := []int{99}
You can find tutorials on slices in the Go tour, or a lot more details about slice usage and internals.
var s []int{99}
The above works but if you want to assign after declaration, then you would need to create a slice using make function with enough length
s := make([]int, 10)
s[0] = 10
fmt.Println(s)
This will initialize slice and set the length to 10 and its elements to zero values
Note: doing s[10] or any greater index will panic since the slice is initialised with length 10. If you want to dynamically increase the slice size, then use append
You can do that by using append function.
func main() {
var s []int
s = append(s,99)
fmt.Println(s) // [99]
}
https://play.golang.org/p/XATvSo2OB6f
// slice declaration; no memory allocation
var slice []int
//slice initialization with length (0) and capacity (10);
//memory allocated for 10 ints
slice = make([]int, 0, 10)
// push to the slice value - than increase length
slice = append(slice, 1)
//change the value. Index has to be lower then length of slice
slice[0] = 2
Take a loot at this output - https://play.golang.com/p/U426b1I5zRq
Of course, you can skip initialization with make, append will do it for you with default value of capacity (2). But for performance it is better to allocate memory only once (if you know how many elements are going to be added to the slice)
The code snippet below is the library implementation of the push methods for a priority queue. I am wondering why the line with the code a = a[0 : n+1] does not throw an out of bounds errors.
func (pq *PriorityQueue) Push(x interface{}) {
// Push and Pop use pointer receivers because they modify the slice's length,
// not just its contents.
// To simplify indexing expressions in these methods, we save a copy of the
// slice object. We could instead write (*pq)[i].
a := *pq
n := len(a)
a = a[0 : n+1]
item := x.(*Item)
item.index = n
a[n] = item
*pq = a
}
a slice is not an array; it is a view onto an existing array. The slice in question is backed by an array larger than itself. When you define a slice of an existing slice, you're actually slicing the underlying array, but the indexes referenced are relative to the source slice.
That's a mouthful. Let's prove this in the following way: we'll create a slice of zero length, but we'll force the underlying array to be larger. When creating a slice with make, the third parameter will set the size of the underlying array. The expression make([]int, 0, 2) will allocate an array of size 2, but it evaluates to a size-zero slice.
package main
import ("fmt")
func main() {
// create a zero-width slice over an initial array of size 2
a := make([]int, 0, 2)
fmt.Println(a)
// expand the slice. Since we're not beyond the size of the initial
// array, this isn't out of bounds.
a = a[0:len(a)+1]
a[0] = 1
fmt.Println(a)
fmt.Println(a[0:len(a)+1])
}
see here. You can use the cap keyword to reference the size of the array that backs a given slice.
The specific code that you asked about loops over cap(pq) in the calling context (container/heap/example_test.go line 90). If you modify the code at the call site and attempt to push another item into the queue, it will panic like you expect. I ... probably wouldn't suggest writing code like this. Although the code in the standard library executes, I would be very sour if I found that in my codebase. It's generally safer to use the append keyword.
Because it works in a specific example program. Here are the important parts from the original/full example source)
const nItem = 10
and
pq := make(PriorityQueue, 0, nItem)
and
for i := 0; i < cap(pq); i++ {
item := &Item{
value: values[i],
priority: priorities[i],
}
heap.Push(&pq, item)
}
Is it an example from container/heap? If yes, then it doesn't throws an exception because capacity is big enough (see how the Push method is used). If you change the example to Push more items then the capacity, then it'll throw.
It does in general; it doesn't in the container/heap example. Here's the general fix I already gave you some time ago.
func (pq *PriorityQueue) Push(x interface{}) {
a := *pq
n := len(a)
item := x.(*Item)
item.index = n
a = append(a, item)
*pq = a
}
Golang solution to Project Euler problem #81