package main
import (
"encoding/binary"
"fmt"
"bytes"
)
func main(){
b := new(bytes.Buffer)
c := new(bytes.Buffer)
binary.Write(b, binary.LittleEndian, []byte{0, 1})
binary.Write(b, binary.BigEndian, []byte{0, 1})
binary.Write(c, binary.LittleEndian, uint16(256))
binary.Write(c, binary.BigEndian, uint16(256))
fmt.Println(b.Bytes()) // [0 1 0 1]
fmt.Println(c.Bytes()) // [0 1 1 0]
}
It is very interesting, why binary.Write() byte ordering is working for uint8, uint16, uint64..etc, but []byte?
If []byte need to be ordered by binary.LittleEndian and write to bytes.Buffer, it needs to be reversed first? Is there any effective ways to solve this problem?
Thanks.
Only integer types get swapped by byte ordering.
When it's a slice of bytes, the binary package wouldn't really know what to swap.
For example, how would it know what to do if you passed 1k of data?
Treat it as int16, int32 or int64?
Or would you expect it to just reverse the whole slice?
Because there is nothing to order by. A byte is 8 bits, so you can go (unsigned) from 0 to 255. ie. uint16 you have 2 bytes, so you can order them differently.
ByteOrder is not even defined for int8. You can check the source code to see that binary.Write simply not uses the passed order when types are uint8 or int8. (A byte is an alias to uint8)
Related
I am searching to convert an array of byte ([]byte) to array of float64 ([]float64) but I didn't have any clues to do it.
I read data from rtlsdr package (https://pkg.go.dev/github.com/jpoirier/gortlsdr) and would pass it to Pwelch algorithm (https://pkg.go.dev/github.com/mjibson/go-dsp/spectral)
Is it possible to convert large array of byte to float64 ?
(+ another question: did you think that possible to convert Pwelch function to work with uint8 ?)
Thank you so much,
pefr
byte/uint8 has 8 bits, float64 has 64. You can convert [8]byte to float64 like this:
package main
import (
"encoding/binary"
"fmt"
"math"
)
main()
{
in := 1.
// Convert float64 to [8]byte
var bytes [8]byte
binary.LittleEndian.PutUint64(bytes[:], math.Float64bits(in))
// Convert [8]byte to float64
out := math.Float64frombits(binary.LittleEndian.Uint64(bytes[:]))
fmt.Println(out)
}
You may want to chose binary.BigEndian instead.
Is that what you are looking for? Or do you want to convert an array of byte with length 8n to an array of float64 with length n? Feel free to comment :-)
If I have a slice of bytes in Go, similar to this:
numBytes := []byte { 0xFF, 0x10 }
How would I convert it to it's uint16 value (0xFF10, 65296)?
you may use binary.BigEndian.Uint16(numBytes)
like this working sample code (with commented output):
package main
import (
"encoding/binary"
"fmt"
)
func main() {
numBytes := []byte{0xFF, 0x10}
u := binary.BigEndian.Uint16(numBytes)
fmt.Printf("%#X %[1]v\n", u) // 0XFF10 65296
}
and see inside binary.BigEndian.Uint16(b []byte):
func (bigEndian) Uint16(b []byte) uint16 {
_ = b[1] // bounds check hint to compiler; see golang.org/issue/14808
return uint16(b[1]) | uint16(b[0])<<8
}
I hope this helps.
To combine two bytes into uint16
x := uint16(numBytes[i])<<8 | uint16(numBytes[i+1])
where i is the starting position of the uint16. So if your array is always only two items it would be x := uint16(numBytes[0])<<8 | uint16(numBytes[1])
Firstly you have a slice not an array - an array has a fixed size and would be declared like this [2]byte.
If you just have a 2 bytes slice, I wouldn't do anything fancy, I'd just do
numBytes := []byte{0xFF, 0x10}
n := int(numBytes[0])<<8 + int(numBytes[1])
fmt.Printf("n =0x%04X = %d\n", n, n)
Playground
EDIT: Just noticed you wanted uint16 - replace int with that in the above!
You can use the following unsafe conversion:
*(*uint16)(unsafe.Pointer(&numBytes[0])
I have a []byte object and I want to get the size of it in bytes. Is there an equivalent to C's sizeof() in golang? If not, Can you suggest other ways to get the same?
To return the number of bytes in a byte slice use the len function:
bs := make([]byte, 1000)
sz := len(bs)
// sz == 1000
If you mean the number of bytes in the underlying array use cap instead:
bs := make([]byte, 1000, 2000)
sz := cap(bs)
// sz == 2000
A byte is guaranteed to be one byte: https://golang.org/ref/spec#Size_and_alignment_guarantees.
I think your best bet would be;
package main
import "fmt"
import "encoding/binary"
func main() {
thousandBytes := make([]byte, 1000)
tenBytes := make([]byte, 10)
fmt.Println(binary.Size(tenBytes))
fmt.Println(binary.Size(thousandBytes))
}
https://play.golang.org/p/HhJif66VwY
Though there are many options, like just importing unsafe and using sizeof;
import unsafe "unsafe"
size := unsafe.Sizeof(bytes)
Note that for some types, like slices, Sizeof is going to give you the size of the slice descriptor which is likely not what you want. Also, bear in mind the length and capacity of the slice are different and the value returned by binary.Size reflects the length.
Please see the example: http://play.golang.org/p/6d4uX15EOQ
package main
import (
"fmt"
"reflect"
"unsafe"
)
func main() {
c := "foofoofoofoofoofofoofoofoofoofoofoofoofoofoofoofoofoofoofoofoo"
fmt.Printf("c: %T, %d\n", c, unsafe.Sizeof(c))
fmt.Printf("c: %T, %d\n", c, reflect.TypeOf(c).Size())
}
Output:
c: string, 8 //8 bytes?!
c: string, 8
It seems like so large string can not have so small size! What's going wrong?
Package unsafe
import "unsafe"
func Sizeof
func Sizeof(v ArbitraryType) uintptr
Sizeof returns the size in bytes occupied by the value v. The size is
that of the "top level" of the value only. For instance, if v is a
slice, it returns the size of the slice descriptor, not the size of
the memory referenced by the slice.
The Go Programming Language Specification
Length and capacity
len(s) string type string length in bytes
You are looking at the "top level", the string descriptor, a pointer to and the length of the underlying string value. Use the len function for the length, in bytes, of the underlying string value.
Conceptually and practically, the string descriptor is a struct containing a pointer and a length, whose lengths (32 or 64 bit) are implementation dependent. For example,
package main
import (
"fmt"
"unsafe"
)
type stringDescriptor struct {
str *byte
len int
}
func main() {
fmt.Println("string descriptor size in bytes:", unsafe.Sizeof(stringDescriptor{}))
}
Output (64 bit):
string descriptor size in bytes: 16
Output (32 bit):
string descriptor size in bytes: 8
A string is essentially a pointer the the data, and an int for the length; so on 32bit systems, it's 8 bytes, and 16 bytes on 64-bit systems.
Both unsafe.Sizeof and reflect.TypeOf(foo).Size() show the size of the string header (two words, IIRC). If you want to get the length of a string, use len(foo).
Playground: http://play.golang.org/p/hRw-EIVIQg.
I'm trying to encode a large number to a list of bytes(uint8 in Go).
The number of bytes is unknown, so I'd like to use vector.
But Go doesn't provide vector of byte, what can I do?
And is it possible to get a slice of such a byte vector?
I intends to implement data compression.
Instead of store small and large number with the same number of bytes,
I'm implements a variable bytes that uses less bytes with small number
and more bytes with large number.
My code can not compile, invalid type assertion:
1 package main
2
3 import (
4 //"fmt"
5 "container/vector"
6 )
7
8 func vbEncodeNumber(n uint) []byte{
9 bytes := new(vector.Vector)
10 for {
11 bytes.Push(n % 128)
12 if n < 128 {
13 break
14 }
15 n /= 128
16 }
17 bytes.Set(bytes.Len()-1, bytes.Last().(byte)+byte(128))
18 return bytes.Data().([]byte) // <-
19 }
20
21 func main() { vbEncodeNumber(10000) }
I wish to writes a lot of such code into binary file,
so I wish the func can return byte array.
I haven't find a code example on vector.
Since you're trying to represent large numbers, you might see if the big package serves your purposes.
The general Vector struct can be used to store bytes. It accepts an empty interface as its type, and any other type satisfies that interface. You can retrieve a slice of interfaces through the Data method, but there's no way to convert that to a slice of bytes without copying it. You can't use type assertion to turn a slice of interface{} into a slice of something else. You'd have to do something like the following at the end of your function: (I haven't tried compiling this code because I can't right now)
byteSlice = make([]byte, bytes.Len())
for i, _ := range byteSlice {
byteSlice[i] = bytes.At(i).(byte)
}
return byteSlice
Take a look at the bytes package and the Buffer type there. You can write your ints as bytes into the buffer and then you can use the Bytes() method to access byte slices of the buffer.
I've found the vectors to be a lot less useful since the generic append and copy were added to the language. Here's how I'd do it in one shot with less copying:
package main
import "fmt"
func vbEncodeNumber(n uint) []byte {
bytes := make([]byte, 0, 4)
for n > 0 {
bytes = append(bytes, byte(n%256))
n >>= 8
}
return bytes
}
func main() {
bytes := vbEncodeNumber(10000)
for i := len(bytes)-1; i >= 0 ; i-- {
fmt.Printf("%02x ", bytes[i])
}
fmt.Println("")
}