Does go use something like space padding for structs? [duplicate] - go

This question already has answers here:
Sizeof struct in Go
(6 answers)
Closed 4 months ago.
I was playing around in go, and was trying to calculate and get the size of struct objects. And found something interesting, if you take a look at the following structs:
type Something struct {
anInteger int16 // 2 bytes
anotherInt int16 // 2 bytes
yetAnother int16 // 2 bytes
someBool bool // 1 byte
} // I expected 7 bytes total
type SomethingBetter struct {
anInteger int16 // 2 bytes
anotherInt int16 // 2 bytes
yetAnother int16 // 2 bytes
someBool bool // 1 byte
anotherBool bool // 1 byte
} // I expected 8 bytes total
type Nested struct {
Something // 7 bytes expected at first
completingByte bool // 1 byte
} // 8 bytes expected at first sight
But the result I got using unsafe.Sizeof(...) was as following:
Something -> 8 bytes
SomethingBetter -> 8 bytes
Nested -> 12 bytes, still, after finding out that "Something" used 8 bytes, though this might use 9 bytes
I suspect that go does something kind of like padding, but I don't know how and why it does that, is there some formula? Or logics? If it uses space padding, is it done randomly? Or based on some rules?

Yes, we have padding! if your system architecture is 32-bit the word size is 4 bytes and if it is 64-bit, the word size is 8 bytes. Now, what is the word size? "Word size" refers to the number of bits processed by a computer's CPU in one go (these days, typically 32 bits or 64 bits). Data bus size, instruction size, address size are usually multiples of the word size.
For example, suppose this struct:
type data struct {
a bool // 1 byte
b int64 // 8 byte
}
This struct it's not 9 bytes because, when our word size is 8, for first cycle, cpu reads 1 byte of bool and padding 7 bytes for others.
Imagine:
p: padding
+-----------------------------------------+----------------+
| 1-byte bool | p | p | p | p | p | p | p | int-64 |
+-----------------------------------------+----------------+
first 8 bytes second 8 bytes
For better performance, sort your struct items from bigger to small.
This is not good performance:
type data struct {
a string // 16 bytes size 16
b int32 // 4 bytes size 20
// 4 bytes padding size 24
c string // 16 bytes size 40
d int32 // 4 bytes size 44
// 4 bytes padding size 48 - Aligned on 8 bytes
}
Now It's better:
type data struct {
a string // 16 bytes size 16
c string // 16 bytes size 32
d int32 // 4 bytes size 36
b int32 // 4 bytes size 40
// no padding size 40 - Aligned on 5 bytes
}
See here for more examples.

Related

Split uint32 into two uint16

How does one go about splitting a single uint32 var in Go into two uint16 vars, representing the 16 MSB and 16 LSB respectively?
Here is a representation of what I am trying to do:
var number uint32
var a uint16
var b uint16
number = 4206942069
Now how would one go about assigning the 16 MSB in number into a and the 16 LSB into b ?
Use the following code to assign the 16 most significant bits in number to a and the 16 least significant bits to b:
a, b := uint16(number>>16), uint16(number)
Run it on the playground.

How to encode struct into binary with bit packing in Golang

I am trying to encode large data structs into binary. I have specified number of bits for each struct element. So I need to encode struct into binary according to bit length. Standard Golang library Encoding/binary packs each item minimum as one byte. Therefore I need another solution. How can I encode struct elements as specified bit number in Go?
For example; Item1 = 00001101 Item2 = 00000110 Result will be as 01101110
type Elements struct{
Item1 uint8 // number of bits = 5
Item2 uint8 // number of bits = 3
Item3 uint8 // number of bits = 2
Item4 uint64 // number of bits = 60
Item5 uint16 // number of bits = 11
Item6 []byte // bit length = 8
Item7 Others
}
type Others struct{
Other1 uint8 // number of bits = 4
Other2 uint32 // number of bits = 21
Other3 uint16 // number of bits = 9
}

How to change the size of the ESP32's UART-TX-FIFO

Configuration of the ESP32's UART_MEM_CONF_REG register does not change the size of the uart TX FIFO as expected.
I'm trying to change the size of UART0's TX FIFO o 512 Bytes.
The FIFO's size (in byte) can be set in UART_MEM_CONF_REG configuring bits 7 to bit 10. (ESP32 TRM V4.0, page 364)
This register is 0x88 by default: 128 Byte TX FIFO and 128 byte RX FIFO. So bit 7 = 1 sets 128 Byte TX FIFO size.
Unfortunately there is no info how to set Bits 7, 8,9, and 10 to change the FIFO size. My first idea was to set bit 8 for 256 bytes size, bit 9 for 512 bytes and bit 10 to 1024 bytes. I intend to use UART0 only, so there's no problem with the other UART's FIFO size.
I tried the following lines:
// Create a byte pattern to send
char buffer[256];
for (int i = 0; i < 256; i++) buffer[i] = i;
// f.e.set bit 8 for (maybe??) 256 bytes TX FIFO size, other configurations has been tested as well
WRITE_PERI_REG(UART_MEM_CONF_REG(uart_num),0x108);
// Start uart driver, no event queue, no TX ringbuffer
uart_driver_install(uart_num, UART_BUF_SIZE, 0, 0, NULL, 0);
// send 256 bytes from a buffer
uart_tx_chars(uart_num, (const char*)buffer, 256);
// but only 128 bytes are sent
At least I expected some change of the TX-FIFO size. But that's not working. The transmission ends after 128 bytes are sent out - no matter how I set the bits 7 to 10 in the UART_MEM_CONF_REG.
What's wrong, what did I miss?

Index Out of Range when using binary.PutVarint(...)

http://play.golang.org/p/RqScJVvpS7
package main
import (
"fmt"
"math/rand"
"encoding/binary"
)
func main() {
buffer := []byte{0, 0, 0, 0, 0, 0, 0, 0}
num := rand.Int63()
count := binary.PutVarint(buffer, num)
fmt.Println(count)
}
I had this working awhile ago when num was just an incrementing uint64 and I was using binary.PutUvarint but now that it's a random int64 and binary.PutVarint I get an error:
panic: runtime error: index out of range
goroutine 1 [running]:
encoding/binary.PutUvarint(0x1042bf58, 0x8, 0x8, 0x6ccb, 0xff9faa4, 0x9acb0442, 0x7fcfd52, 0x4d658221)
/usr/local/go/src/encoding/binary/varint.go:44 +0xc0
encoding/binary.PutVarint(0x1042bf58, 0x8, 0x8, 0x6ccb, 0x7fcfd52, 0x4d658221, 0x14f9e0, 0x104000e0)
/usr/local/go/src/encoding/binary/varint.go:83 +0x60
main.main()
/tmp/sandbox010341234/main.go:12 +0x100
What am I missing? I would have thought this to be a trivial change...
EDIT: I just tried extending my buffer array. For some odd reason it works and I get a count of 10. How can that be? int64 is 64 bits = 8 bytes, right?
Quoting the doc of encoding/binary:
The varint functions encode and decode single integer values using a variable-length encoding; smaller values require fewer bytes. For a specification, see https://developers.google.com/protocol-buffers/docs/encoding.
So the binary.PutVarint() is not a fixed, but a variable-length encoding. When passing an int64, it will need more than 8 bytes for large numbers, and less than 8 bytes for small numbers. Since the number you're encoding is a random number, it will have random bits even in its highest byte.
See this simple example:
buffer := make([]byte, 100)
for num := int64(1); num < 1<<60; num <<= 4 {
count := binary.PutVarint(buffer, num)
fmt.Printf("Num=%d, bytes=%d\n", num, count)
}
Output:
Num=1, bytes=1
Num=16, bytes=1
Num=256, bytes=2
Num=4096, bytes=2
Num=65536, bytes=3
Num=1048576, bytes=4
Num=16777216, bytes=4
Num=268435456, bytes=5
Num=4294967296, bytes=5
Num=68719476736, bytes=6
Num=1099511627776, bytes=6
Num=17592186044416, bytes=7
Num=281474976710656, bytes=8
Num=4503599627370496, bytes=8
Num=72057594037927936, bytes=9
The essence of variable-length encoding is that small numbers use less bytes, but this can only be achieved if in turn big numbers may use more than 8 bytes (that would be size of int64).
Details of the specific encoding is on the linked page.
A very easy example would be: A byte is 8 bits. Use 7 bits of the output byte as the "useful" bits to encode the data/number. If the highest bit is 1, that means more bytes are required. If highest bit is 0, we're done. You can see that small numbers can be encoded using 1 output byte (e.g. n=10), while we're using 1 extra bit for every 7-bit useful data, so if the input number uses all the 64 bits, we will end up with more than 8 bytes: 10 groups are required to cover 64 bits, so we will need 10 bytes (9 groups is only 9*7=63 bits).

how to compress the three bytes data into two bytes in java

I have to compress the 3 bytes of data in to two bytes.
the 3 bytes data includes day is in one byte,hour is in another byte, finally minutes is in one more byte.so totally i have 3 bytes data.how could i flip this data into two bytes only.
Thanks,
Minutes are ranging from 0 to 59 so the number could be stored on 6
bits (6 bits => 0 to 63)
Hours are ranging from 0 to 23 the number
could be stored on 5 bits (5 bits => 0 to 31)
Days... Err... Ranging from 0 to 6 ? Let's asusme that. 2 bytes = 16 bits, minus the 11
other bits, so you have 5 bits left which is more than enough.
To pack your 3 bytes of data into two, you have to dispatch the bits :
I will set bits 0 to 5 for the minutes, bits 6 to 10 for the hours, and the bits left for the day number.
So the formula to pack the bits is :
packed=minutes+hours*64+days*2048
To get back your uncompressed data :
minutes=packed & 63
hours=(packed & 1984) / 64
days=(packed & 63488) / 2048
I assume you need day from 1-31, hour from 0-23 and minute from 0-59, you thus need 5 bits for the day, 5 bits for the hours and 6 bits for the minutes. This makes exactly 16 bits. You should put 5 bits (day) and the first 3 bits for hours into your first byte and the remaining 2 bits for hours and the 6 bits for minutes into the second byte:
int day = 23;
int hour = 15;
int minute = 34;
byte fromint_dh = (day << 3) | (hour >> 2);
byte fromint_hm = ((hour & 0x03) << 6) | (minute); // take the last two bits from hour and put them at the beginning
....
int d = fromint_dh >> 3;
int h = ((fromint_dh & 0x07) << 2) | ((fromint_hm & 0xc0) >> 6); // take the last 3 bits of the fst byte and the fst 2 bits of the snd byte
int m = fromint_hm & 0x3F // only take the last 6 bits
Hope this helps. It's easy to get the bits wrong ...

Resources