I have a []byte object and I want to get the size of it in bytes. Is there an equivalent to C's sizeof() in golang? If not, Can you suggest other ways to get the same?
To return the number of bytes in a byte slice use the len function:
bs := make([]byte, 1000)
sz := len(bs)
// sz == 1000
If you mean the number of bytes in the underlying array use cap instead:
bs := make([]byte, 1000, 2000)
sz := cap(bs)
// sz == 2000
A byte is guaranteed to be one byte: https://golang.org/ref/spec#Size_and_alignment_guarantees.
I think your best bet would be;
package main
import "fmt"
import "encoding/binary"
func main() {
thousandBytes := make([]byte, 1000)
tenBytes := make([]byte, 10)
fmt.Println(binary.Size(tenBytes))
fmt.Println(binary.Size(thousandBytes))
}
https://play.golang.org/p/HhJif66VwY
Though there are many options, like just importing unsafe and using sizeof;
import unsafe "unsafe"
size := unsafe.Sizeof(bytes)
Note that for some types, like slices, Sizeof is going to give you the size of the slice descriptor which is likely not what you want. Also, bear in mind the length and capacity of the slice are different and the value returned by binary.Size reflects the length.
Related
For below code:
package main
import (
"fmt"
"unicode/utf8"
)
func main() {
// Declare a string with both chinese and english characters
s := "世界 means world"
// UTFMax is 4 -- up to 4 bytes per encoded rune
var buf [utf8.UTFMax]byte
fmt.Println(len(buf))
fmt.Println(cap(buf))
// Iterate over the string
for i, r := range s {
// Capture the number of bytes for this rune
rl := utf8.RuneLen(r)
fmt.Println("Rune lenght is:", rl)
// Calculate the slice offset for the bytes associated
// with this rune
si := i + rl
fmt.Println("Index is:", i)
fmt.Println("Slice offset:", si)
//Copy of rune from the string to buffer
copy(buf[:], s[i:si])
// Display the details
fmt.Printf("%2d: %q; codepoint: %#6x; encoded bytes: %#v\n", i, r, r, buf[:rl])
}
}
Built-in functions len & cap give the value for array type [4]byte
var buf [utf8.UTFMax]byte
fmt.Println(len(buf))
fmt.Println(cap(buf))
....
copy(buf[:], s[i:si])
Q1) Does buf[:] create a new slice header that points to array storage buf?
Q2) Is len(buf) performing implicit conversion of buf from array type to slice type?
Q3) Does s[i:si] create a new slice header that points to string s?
buf[:] is a slice whose cap and len are the length of the array, and the backing store is the array buf.
len(buf) gets the length of the array. It does not convert buf to a slice to do that. Length of buf is fixed at compile time.
s[i:si] creates a new string, not a new slice.
If I have an existing []byte, what is the recommended way to append the bytes of one or more uint32 value(s) to it?
For example, what should I replace // ??? with:
s := []byte{0x00, 0x01, 0x02, 0x03}
u := uint32(0x07060504)
// ???
fmt.Println(s) // Should print [0 1 2 3 4 5 6 7]
Edit: One option would be s = append(s, byte(u)); s = append(s, byte(u >> 8)); s = append(s, byte(u >> 16)); s = append(s, byte(u >> 24)), but is there a more idiomatic way to do this? Perhaps using package binary and/or package bytes?
One option is to append the individual bytes as suggested in the question. The multiple append calls can be combined into a single call:
s = append(s, byte(u), byte(u>>8), byte(u>>16), byte(u>>24))
The binary package can also be used as the question suggests:
var b [4]byte
binary.LittleEndian.PutUint32(b[:], u)
s = append(s, b[:]...)
Run it on the Go playground.
The last snippet should allocate b on the stack. If it does not, then the extra heap allocation can be avoided with the following code:
s = append(s, " "...) // append four bytes (the values don't matter)
binary.LittleEndian.PutUint32(s[len(s)-4:], u) // overwrite those bytes with the uint32
encoding/binary has the functions you need:
import "encoding/binary"
b := make([]byte,4)
binary.LittleEndian.PutUint32(b, u)
s = append(s, b)
There's unsafe (actually safe, if you will only copy its bytes) way to get byte representation of any primitive:
const sizeUInt32 = int(unsafe.Sizeof(uint32(0)))
func GetBytesUInt32(i *uint32) []byte {
return (*[1 << 30]byte)(unsafe.Pointer(i))[:sizeUInt32:sizeUInt32]
}
https://play.golang.org/p/WPC5jeYLDth
Created slice will carry passed int's storage, so by making a manipulations with it keep in mind that uint32 value will be changed too.
Hey, what a hate without discussion? I realize that you guys don't like unsafe code, and I realize that such answer isn't recommended way for which topic starter looking... but I think that such place as stackoverflow should offer all of possibles ways to implement thing people googling for.
package main
import (
"encoding/binary"
"fmt"
"bytes"
)
func main(){
b := new(bytes.Buffer)
c := new(bytes.Buffer)
binary.Write(b, binary.LittleEndian, []byte{0, 1})
binary.Write(b, binary.BigEndian, []byte{0, 1})
binary.Write(c, binary.LittleEndian, uint16(256))
binary.Write(c, binary.BigEndian, uint16(256))
fmt.Println(b.Bytes()) // [0 1 0 1]
fmt.Println(c.Bytes()) // [0 1 1 0]
}
It is very interesting, why binary.Write() byte ordering is working for uint8, uint16, uint64..etc, but []byte?
If []byte need to be ordered by binary.LittleEndian and write to bytes.Buffer, it needs to be reversed first? Is there any effective ways to solve this problem?
Thanks.
Only integer types get swapped by byte ordering.
When it's a slice of bytes, the binary package wouldn't really know what to swap.
For example, how would it know what to do if you passed 1k of data?
Treat it as int16, int32 or int64?
Or would you expect it to just reverse the whole slice?
Because there is nothing to order by. A byte is 8 bits, so you can go (unsigned) from 0 to 255. ie. uint16 you have 2 bytes, so you can order them differently.
ByteOrder is not even defined for int8. You can check the source code to see that binary.Write simply not uses the passed order when types are uint8 or int8. (A byte is an alias to uint8)
If I have a slice of bytes in Go, similar to this:
numBytes := []byte { 0xFF, 0x10 }
How would I convert it to it's uint16 value (0xFF10, 65296)?
you may use binary.BigEndian.Uint16(numBytes)
like this working sample code (with commented output):
package main
import (
"encoding/binary"
"fmt"
)
func main() {
numBytes := []byte{0xFF, 0x10}
u := binary.BigEndian.Uint16(numBytes)
fmt.Printf("%#X %[1]v\n", u) // 0XFF10 65296
}
and see inside binary.BigEndian.Uint16(b []byte):
func (bigEndian) Uint16(b []byte) uint16 {
_ = b[1] // bounds check hint to compiler; see golang.org/issue/14808
return uint16(b[1]) | uint16(b[0])<<8
}
I hope this helps.
To combine two bytes into uint16
x := uint16(numBytes[i])<<8 | uint16(numBytes[i+1])
where i is the starting position of the uint16. So if your array is always only two items it would be x := uint16(numBytes[0])<<8 | uint16(numBytes[1])
Firstly you have a slice not an array - an array has a fixed size and would be declared like this [2]byte.
If you just have a 2 bytes slice, I wouldn't do anything fancy, I'd just do
numBytes := []byte{0xFF, 0x10}
n := int(numBytes[0])<<8 + int(numBytes[1])
fmt.Printf("n =0x%04X = %d\n", n, n)
Playground
EDIT: Just noticed you wanted uint16 - replace int with that in the above!
You can use the following unsafe conversion:
*(*uint16)(unsafe.Pointer(&numBytes[0])
I'm trying to encode a large number to a list of bytes(uint8 in Go).
The number of bytes is unknown, so I'd like to use vector.
But Go doesn't provide vector of byte, what can I do?
And is it possible to get a slice of such a byte vector?
I intends to implement data compression.
Instead of store small and large number with the same number of bytes,
I'm implements a variable bytes that uses less bytes with small number
and more bytes with large number.
My code can not compile, invalid type assertion:
1 package main
2
3 import (
4 //"fmt"
5 "container/vector"
6 )
7
8 func vbEncodeNumber(n uint) []byte{
9 bytes := new(vector.Vector)
10 for {
11 bytes.Push(n % 128)
12 if n < 128 {
13 break
14 }
15 n /= 128
16 }
17 bytes.Set(bytes.Len()-1, bytes.Last().(byte)+byte(128))
18 return bytes.Data().([]byte) // <-
19 }
20
21 func main() { vbEncodeNumber(10000) }
I wish to writes a lot of such code into binary file,
so I wish the func can return byte array.
I haven't find a code example on vector.
Since you're trying to represent large numbers, you might see if the big package serves your purposes.
The general Vector struct can be used to store bytes. It accepts an empty interface as its type, and any other type satisfies that interface. You can retrieve a slice of interfaces through the Data method, but there's no way to convert that to a slice of bytes without copying it. You can't use type assertion to turn a slice of interface{} into a slice of something else. You'd have to do something like the following at the end of your function: (I haven't tried compiling this code because I can't right now)
byteSlice = make([]byte, bytes.Len())
for i, _ := range byteSlice {
byteSlice[i] = bytes.At(i).(byte)
}
return byteSlice
Take a look at the bytes package and the Buffer type there. You can write your ints as bytes into the buffer and then you can use the Bytes() method to access byte slices of the buffer.
I've found the vectors to be a lot less useful since the generic append and copy were added to the language. Here's how I'd do it in one shot with less copying:
package main
import "fmt"
func vbEncodeNumber(n uint) []byte {
bytes := make([]byte, 0, 4)
for n > 0 {
bytes = append(bytes, byte(n%256))
n >>= 8
}
return bytes
}
func main() {
bytes := vbEncodeNumber(10000)
for i := len(bytes)-1; i >= 0 ; i-- {
fmt.Printf("%02x ", bytes[i])
}
fmt.Println("")
}