If I have an existing []byte, what is the recommended way to append the bytes of one or more uint32 value(s) to it?
For example, what should I replace // ??? with:
s := []byte{0x00, 0x01, 0x02, 0x03}
u := uint32(0x07060504)
// ???
fmt.Println(s) // Should print [0 1 2 3 4 5 6 7]
Edit: One option would be s = append(s, byte(u)); s = append(s, byte(u >> 8)); s = append(s, byte(u >> 16)); s = append(s, byte(u >> 24)), but is there a more idiomatic way to do this? Perhaps using package binary and/or package bytes?
One option is to append the individual bytes as suggested in the question. The multiple append calls can be combined into a single call:
s = append(s, byte(u), byte(u>>8), byte(u>>16), byte(u>>24))
The binary package can also be used as the question suggests:
var b [4]byte
binary.LittleEndian.PutUint32(b[:], u)
s = append(s, b[:]...)
Run it on the Go playground.
The last snippet should allocate b on the stack. If it does not, then the extra heap allocation can be avoided with the following code:
s = append(s, " "...) // append four bytes (the values don't matter)
binary.LittleEndian.PutUint32(s[len(s)-4:], u) // overwrite those bytes with the uint32
encoding/binary has the functions you need:
import "encoding/binary"
b := make([]byte,4)
binary.LittleEndian.PutUint32(b, u)
s = append(s, b)
There's unsafe (actually safe, if you will only copy its bytes) way to get byte representation of any primitive:
const sizeUInt32 = int(unsafe.Sizeof(uint32(0)))
func GetBytesUInt32(i *uint32) []byte {
return (*[1 << 30]byte)(unsafe.Pointer(i))[:sizeUInt32:sizeUInt32]
}
https://play.golang.org/p/WPC5jeYLDth
Created slice will carry passed int's storage, so by making a manipulations with it keep in mind that uint32 value will be changed too.
Hey, what a hate without discussion? I realize that you guys don't like unsafe code, and I realize that such answer isn't recommended way for which topic starter looking... but I think that such place as stackoverflow should offer all of possibles ways to implement thing people googling for.
Related
I'm still new to programming. Forgive my lack of computer science knowledge. Not sure if this question is specific to Golang or computer science in general...
I always thought that functions do not alter variables/data held outside their own scope unless you use a return statement back into the other scope, or unless they are higher in the hierarchy of scopes. One may argue that functions f1 and f2 in this example are called from a lower scope. However, this still doesn't explain why I'm getting different results for variable num and nums.
package main
import "fmt"
func f1(a int) {
a = 50 // this will not work, as it shouldn't
}
func f2(a ...int) {
a[0] = 50 // this will work without return statement
a[1] = 50 // this will work without return statement
}
func main() {
num := 2
nums := []int{2, 2}
f1(num)
f2(nums...)
fmt.Printf("function f1 doesn't affect the variable num and stays: %v\n", num)
fmt.Printf("function f2 affects the variable nums and results in: %v", nums)
Questions:
Why doesn't f2 require a return statement to modify nums like num
would within f1?
Golang functions are said to pass values (rather than reference),
shouldn't that force the function to return copies?
Can this happen in other languages? (I think I may have
seen this in other languages).
This is the correct behaviour, since a ...int is equal to a slice e.g.: a []int
func f2(a []int) {
a[0] = 50
a[1] = 50
}
func main() {
b := []int{2, 2}
f2(b)
fmt.Println(b) // [50 50]
}
And a slice is a view to the original data, here 'b'.
"Why doesn't f2 require a return statement to modify nums like num would in f1?"
In f2 you are using the slice, which has a pointer to the original array, so f2 can change the outside array.
"Golang functions are said to pass values (not reference), shouldn't that force to return copies? (If the question is related...)"
In f2 the slice itself is passed by value, meaning pointer and length and capacity of the original array.
"Can this happen in other languages? (I think I may have seen this in other langues)"
Too broad to answer, there are many languages and in general if you have a pointer to the outside world array, yes.
Edit:
package main
import "fmt"
func sum(a ...int) int {
s := 0
for _, v := range a {
s += v
}
return s
}
func f2(a []int) {
c := make([]int, len(a))
copy(c, a)
c[0] = 50
fmt.Println(sum(c...)) // 52
}
func main() {
b := []int{2, 2}
fmt.Println(sum(1, 2, 3, 4)) // 10
fmt.Println(sum(b...)) // 4
f2(b)
fmt.Println(b) // [2 2]
}
Notes:
The sum() function above is a pure function, since it has no side effect.
The new f2 function above is a pure function, since it has no side effect: it makes a copy of a into c then calls the sum.
In go, function arguments are passed by value. That means, if you pass an int (like in f1), compiler will pass the value of f1, essentially copying it. If the function takes a *int and you pass &num, then the compiler passes the value of &num, which is a pointer to num. When the function changes *num, the value of the variable outside the function will change. If the function changes num, the pointer value of num will change, and it will point to a different variable.
As a contrast, Java passes all primitive values as value, and all objects by reference. That is, if you pass an int, there is no way for the function to modify the value of that int that is visible to the caller. If you want to pass an int the function can modify, you put that in a class and pass an instance of that class in Java.
A slice (as in f2) contains a pointer to the underlying array. When you call a function with a slice, the slice header (containing a pointer to the underlying array) is copied, so when the function changes the slice elements, the underlying array elements change.
The question of scope is somewhat different. Scope of a function is all the variables it can see. Those are the global variables (if from different packages, exported global variables), function arguments, and if the function is declared nested within another function, all the variables visible in that function at that point.
1 & 2) Both questions can be answered when looking at how slices work in Go. There's a blog article on it.
In general, all variables are passed by value in Go. You can use pointers (e.g. *int for f1) to pass by reference (or more correct, the address of the pointer).
However, slices are technically also passed by value.
When we look here, we can get an idea how they work:
type SliceHeader struct {
Data uintptr
Len int
Cap int
}
Len and Cap are integers, but Data is a pointer.
When this struct is copied (when passing by value), a copy of Len, Cap and Data will be made. Since Data is a pointer, any modifications made to the value it's pointing to will be visible after your function returns.
You can also read this
Is there a way to make this golang code shorter?
func MergeSlices(s1 []float32, s2 []int32) []int {
var slice []int
for i := range s1 {
slice = append(slice, int(s1[i]))
}
for i := range s2 {
slice = append(slice, int(s2[i]))
}
return slice
}
You can't eliminate the loops to convert each element to int individually, because you can't convert whole slices of different element types. For explanation, see this question: Type converting slices of interfaces in go
The most you can do is use named result type, and a for range with 2 iteration values, where you can omit the first (the index) by assigning it to the blank identifier, and the 2nd will be the value:
func MergeSlices(s1 []float32, s2 []int32) (s []int) {
for _, v := range s1 {
s = append(s, int(v))
}
for _, v := range s2 {
s = append(s, int(v))
}
return
}
But know that your code is fine as-is. My code is not something to always follow, it was to answer your question: how to make your code shorter. If you want to improve your code, you could start by looking at its performance, or even refactoring your code to not end up needing to merge slices of different types.
Your code should be correct, maintainable, readable, and reasonably efficient. Note that shortness of code is not one of the important goals. For good reason, Stack Exchange has another site for Code Golf questions: Programming Puzzles & Code Golf.
Your code could be improved; it's inefficient. For example, merging two len(256) slices,
BenchmarkMergeSlices 200000 8350 ns/op 8184 B/op 10 allocs/op
Here's a more efficient (and longer) version:
BenchmarkMergeSlices 300000 4420 ns/op 4096 B/op 1 allocs/op
.
func MergeSlices(s1 []float32, s2 []int32) []int {
slice := make([]int, 0, len(s1)+len(s2))
for i := range s1 {
slice = append(slice, int(s1[i]))
}
for i := range s2 {
slice = append(slice, int(s2[i]))
}
return slice
}
Use the Go Code Review Comments for Named Result Parameters. For example: "Don't name result parameters just to avoid declaring a var inside the function; that trades off a minor implementation brevity at the cost of unnecessary API verbosity. Clarity of docs is always more important than saving a line or two in your function."
var s1 []int
var s2 []int
newSlice = append(s1, s2...)
The code can't get any shorter, but that's a goal of dubious value to begin with; it's not overly verbose as-is. You can, however, likely improve performance by eliminating the intermediate allocations. Every time you call append, if the target slice doesn't have enough space, it expands it, guessing at the necessary size since you haven't told it how much space it will need.
The simplest would just be to presize your target slice (replace var slice []int with slice := make([]int, 0, len(s1) + len(s2)); that way the appends never have to expand it. Setting the second parameter to 0 is important, that sets the length to zero, and the capacity to the total size needed, so that your appends will work as expected.
Once you've presized it though, you can get rid of the appends entirely, and directly set each index:
func MergeSlices(s1 []float32, s2 []int32) []int {
slice := make([]int, len(s1) + len(s2))
for i,v := range s1 {
slice[i] = int(v)
}
for i,v := range s2 {
slice[i+len(s1)] = int(v)
}
return slice
}
Playground link
I have the following variables:
var foo int8
var bar [5]int8
I want to count number of bytes in both variables and output the sum - so right here it should be 1 + 5 = 6. Is this possible?
You can use reflect.Size, this returns a uintptr for some reason, but is actually just the exact same thing as unsafe.SizeOf without having to use unsafe.
var x [8]byte
t := reflect.TypeOf(x)
fmt.Println(t.Size())
The snags are going to be maps, slices, strings, and pointers, which you'll get the size of the header/metadata (or just the pointer size for pointers). If that's okay, great! If not, you can handle this in various ways, this is the best I have off the top of my head:
func UnderlyingSize(x interface{}) uintptr {
v := reflect.ValueOf(x)
t := v.Type()
var size uintptr;
switch t.Kind() {
// For the builtin collections, we just multiply the len by the
// element size, for maps also do the key
case reflect.Map:
l := uintptr(v.Len())
size = t.Key().Size()*l + t.Elem().Size()*l
case reflect.Slice:
t := t.Elem();
size = t.Size() * uintptr(v.Len())
case reflect.Ptr:
t := t.Elem();
size = t.Size()
// Strings are just byte arrays, so it's just the len
case reflect.String:
size = uintptr(v.Len())
// For an interface, we need to find the underlying type
case reflect.Interface:
v := v.Elem()
size = UnderlyingSize(v)
// For anything else, including arrays, Size returns the correct value
default:
size = t.Size();
}
return size
}
There is an argument to be made for using Cap rather than Len, but it's easy enough to change yourself. You can also add t.Size() to any of these values if you want the size of the header information AND the underlying size. Note a word of warning that the real map probably takes more memory than just the key+value+header size, since there's probably some extra information under the hood.
If you have a data structure that's a collection, you'll have to implement something like this yourself, but if it's a simple struct (i.e. only made of POD structs and builtin types), you can simply add up UnderlyingSize of all the members.
You can use unsafe.Sizeof (https://play.golang.org/p/FroasKud7I):
unsafe.Sizeof(foo) + unsafe.Sizeof(bar)
I have a []byte object and I want to get the size of it in bytes. Is there an equivalent to C's sizeof() in golang? If not, Can you suggest other ways to get the same?
To return the number of bytes in a byte slice use the len function:
bs := make([]byte, 1000)
sz := len(bs)
// sz == 1000
If you mean the number of bytes in the underlying array use cap instead:
bs := make([]byte, 1000, 2000)
sz := cap(bs)
// sz == 2000
A byte is guaranteed to be one byte: https://golang.org/ref/spec#Size_and_alignment_guarantees.
I think your best bet would be;
package main
import "fmt"
import "encoding/binary"
func main() {
thousandBytes := make([]byte, 1000)
tenBytes := make([]byte, 10)
fmt.Println(binary.Size(tenBytes))
fmt.Println(binary.Size(thousandBytes))
}
https://play.golang.org/p/HhJif66VwY
Though there are many options, like just importing unsafe and using sizeof;
import unsafe "unsafe"
size := unsafe.Sizeof(bytes)
Note that for some types, like slices, Sizeof is going to give you the size of the slice descriptor which is likely not what you want. Also, bear in mind the length and capacity of the slice are different and the value returned by binary.Size reflects the length.
The code snippet below is the library implementation of the push methods for a priority queue. I am wondering why the line with the code a = a[0 : n+1] does not throw an out of bounds errors.
func (pq *PriorityQueue) Push(x interface{}) {
// Push and Pop use pointer receivers because they modify the slice's length,
// not just its contents.
// To simplify indexing expressions in these methods, we save a copy of the
// slice object. We could instead write (*pq)[i].
a := *pq
n := len(a)
a = a[0 : n+1]
item := x.(*Item)
item.index = n
a[n] = item
*pq = a
}
a slice is not an array; it is a view onto an existing array. The slice in question is backed by an array larger than itself. When you define a slice of an existing slice, you're actually slicing the underlying array, but the indexes referenced are relative to the source slice.
That's a mouthful. Let's prove this in the following way: we'll create a slice of zero length, but we'll force the underlying array to be larger. When creating a slice with make, the third parameter will set the size of the underlying array. The expression make([]int, 0, 2) will allocate an array of size 2, but it evaluates to a size-zero slice.
package main
import ("fmt")
func main() {
// create a zero-width slice over an initial array of size 2
a := make([]int, 0, 2)
fmt.Println(a)
// expand the slice. Since we're not beyond the size of the initial
// array, this isn't out of bounds.
a = a[0:len(a)+1]
a[0] = 1
fmt.Println(a)
fmt.Println(a[0:len(a)+1])
}
see here. You can use the cap keyword to reference the size of the array that backs a given slice.
The specific code that you asked about loops over cap(pq) in the calling context (container/heap/example_test.go line 90). If you modify the code at the call site and attempt to push another item into the queue, it will panic like you expect. I ... probably wouldn't suggest writing code like this. Although the code in the standard library executes, I would be very sour if I found that in my codebase. It's generally safer to use the append keyword.
Because it works in a specific example program. Here are the important parts from the original/full example source)
const nItem = 10
and
pq := make(PriorityQueue, 0, nItem)
and
for i := 0; i < cap(pq); i++ {
item := &Item{
value: values[i],
priority: priorities[i],
}
heap.Push(&pq, item)
}
Is it an example from container/heap? If yes, then it doesn't throws an exception because capacity is big enough (see how the Push method is used). If you change the example to Push more items then the capacity, then it'll throw.
It does in general; it doesn't in the container/heap example. Here's the general fix I already gave you some time ago.
func (pq *PriorityQueue) Push(x interface{}) {
a := *pq
n := len(a)
item := x.(*Item)
item.index = n
a = append(a, item)
*pq = a
}
Golang solution to Project Euler problem #81