Configuration of the ESP32's UART_MEM_CONF_REG register does not change the size of the uart TX FIFO as expected.
I'm trying to change the size of UART0's TX FIFO o 512 Bytes.
The FIFO's size (in byte) can be set in UART_MEM_CONF_REG configuring bits 7 to bit 10. (ESP32 TRM V4.0, page 364)
This register is 0x88 by default: 128 Byte TX FIFO and 128 byte RX FIFO. So bit 7 = 1 sets 128 Byte TX FIFO size.
Unfortunately there is no info how to set Bits 7, 8,9, and 10 to change the FIFO size. My first idea was to set bit 8 for 256 bytes size, bit 9 for 512 bytes and bit 10 to 1024 bytes. I intend to use UART0 only, so there's no problem with the other UART's FIFO size.
I tried the following lines:
// Create a byte pattern to send
char buffer[256];
for (int i = 0; i < 256; i++) buffer[i] = i;
// f.e.set bit 8 for (maybe??) 256 bytes TX FIFO size, other configurations has been tested as well
WRITE_PERI_REG(UART_MEM_CONF_REG(uart_num),0x108);
// Start uart driver, no event queue, no TX ringbuffer
uart_driver_install(uart_num, UART_BUF_SIZE, 0, 0, NULL, 0);
// send 256 bytes from a buffer
uart_tx_chars(uart_num, (const char*)buffer, 256);
// but only 128 bytes are sent
At least I expected some change of the TX-FIFO size. But that's not working. The transmission ends after 128 bytes are sent out - no matter how I set the bits 7 to 10 in the UART_MEM_CONF_REG.
What's wrong, what did I miss?
Related
This question already has answers here:
Sizeof struct in Go
(6 answers)
Closed 4 months ago.
I was playing around in go, and was trying to calculate and get the size of struct objects. And found something interesting, if you take a look at the following structs:
type Something struct {
anInteger int16 // 2 bytes
anotherInt int16 // 2 bytes
yetAnother int16 // 2 bytes
someBool bool // 1 byte
} // I expected 7 bytes total
type SomethingBetter struct {
anInteger int16 // 2 bytes
anotherInt int16 // 2 bytes
yetAnother int16 // 2 bytes
someBool bool // 1 byte
anotherBool bool // 1 byte
} // I expected 8 bytes total
type Nested struct {
Something // 7 bytes expected at first
completingByte bool // 1 byte
} // 8 bytes expected at first sight
But the result I got using unsafe.Sizeof(...) was as following:
Something -> 8 bytes
SomethingBetter -> 8 bytes
Nested -> 12 bytes, still, after finding out that "Something" used 8 bytes, though this might use 9 bytes
I suspect that go does something kind of like padding, but I don't know how and why it does that, is there some formula? Or logics? If it uses space padding, is it done randomly? Or based on some rules?
Yes, we have padding! if your system architecture is 32-bit the word size is 4 bytes and if it is 64-bit, the word size is 8 bytes. Now, what is the word size? "Word size" refers to the number of bits processed by a computer's CPU in one go (these days, typically 32 bits or 64 bits). Data bus size, instruction size, address size are usually multiples of the word size.
For example, suppose this struct:
type data struct {
a bool // 1 byte
b int64 // 8 byte
}
This struct it's not 9 bytes because, when our word size is 8, for first cycle, cpu reads 1 byte of bool and padding 7 bytes for others.
Imagine:
p: padding
+-----------------------------------------+----------------+
| 1-byte bool | p | p | p | p | p | p | p | int-64 |
+-----------------------------------------+----------------+
first 8 bytes second 8 bytes
For better performance, sort your struct items from bigger to small.
This is not good performance:
type data struct {
a string // 16 bytes size 16
b int32 // 4 bytes size 20
// 4 bytes padding size 24
c string // 16 bytes size 40
d int32 // 4 bytes size 44
// 4 bytes padding size 48 - Aligned on 8 bytes
}
Now It's better:
type data struct {
a string // 16 bytes size 16
c string // 16 bytes size 32
d int32 // 4 bytes size 36
b int32 // 4 bytes size 40
// no padding size 40 - Aligned on 5 bytes
}
See here for more examples.
I wrote a script in C to allocate memory with malloc() on a infinite loop.
My aim was to realize a simple Denial of Service using WebAssembly by opening multiple tabs and make the browser crash.
I can allocate about 2 GB max for each tab to prevent the tab crash (memory limitation for x64 browsers).
#include <stdlib.h>
#define MAX_MEM 2147483630 //2 GB
int main() {
long int mem_used=209715000;
while(1){
if(mem_used<MAX_MEM){
int *ptr = malloc(sizeof(int));
mem_used+=4;
}
}
return 0;
}
I expected it to work, but instead the tab crashes.
From tests I've made, mem_used+=16 is the right choice to prevent the tab crash.
I don't know deeply the WebAssembly memory management, so my thought is that maybe it requires 4x the memory. Is that correct?
With emscripten, malloc adds some minimum chunk size and then aligns the address to at least 8 byte boundaries. So for small allocations (even zero bytes), malloc will appear to take significantly more space than needed. For big allocations, the overhead will be relatively small.
See comments in dlmalloc.c.
The following program demonstrates how much space malloc takes:
#include <iostream>
int main() {
char *previous, *current;
previous = (char*)malloc(0);
for(int i=0; i<32; ++i) {
current = (char*)malloc(i+1);
std::cout << "malloc(" << i << ") consumed " << (current-previous) << " bytes\n";
previous = current;
}
std::cout << "\n";
previous = (char*)malloc(1);
for(int i=0; i<12; ++i) {
current = (char*)malloc( 1<<(i+1) );
std::cout << "malloc(" << (1<<i) << ") consumed " << (current-previous) << " bytes\n";
previous = current;
}
return 0;
}
This yields the following output:
malloc(0) consumed 16 bytes
malloc(1) consumed 16 bytes
malloc(2) consumed 16 bytes
malloc(3) consumed 16 bytes
malloc(4) consumed 16 bytes
malloc(5) consumed 16 bytes
malloc(6) consumed 16 bytes
malloc(7) consumed 16 bytes
malloc(8) consumed 16 bytes
malloc(9) consumed 16 bytes
malloc(10) consumed 16 bytes
malloc(11) consumed 16 bytes
malloc(12) consumed 16 bytes
malloc(13) consumed 24 bytes
malloc(14) consumed 24 bytes
malloc(15) consumed 24 bytes
malloc(16) consumed 24 bytes
malloc(17) consumed 24 bytes
malloc(18) consumed 24 bytes
malloc(19) consumed 24 bytes
malloc(20) consumed 24 bytes
malloc(21) consumed 32 bytes
malloc(22) consumed 32 bytes
malloc(23) consumed 32 bytes
malloc(24) consumed 32 bytes
malloc(25) consumed 32 bytes
malloc(26) consumed 32 bytes
malloc(27) consumed 32 bytes
malloc(28) consumed 32 bytes
malloc(29) consumed 40 bytes
malloc(30) consumed 40 bytes
malloc(31) consumed 40 bytes
malloc(1) consumed 16 bytes
malloc(2) consumed 16 bytes
malloc(4) consumed 16 bytes
malloc(8) consumed 16 bytes
malloc(16) consumed 24 bytes
malloc(32) consumed 40 bytes
malloc(64) consumed 72 bytes
malloc(128) consumed 136 bytes
malloc(256) consumed 264 bytes
malloc(512) consumed 520 bytes
malloc(1024) consumed 1032 bytes
malloc(2048) consumed 2056 bytes
See full source code in this repo
Your problem is that malloc implementations typically:
a) Include overhead; and
b) Round up to some unit
malloc (sizeof(int)) is using more than sizeof(int) bytes behind the scenes.
In any system, malloc() always slightly uses more memory than you request. Emscripten uses dlmalloc, a popular malloc() implementation, as default. According to Wikipedia:
Memory on the heap is allocated as "chunks", an 8-byte aligned data structure which contains a header, and usable memory. Allocated memory contains an 8 or 16 byte overhead for the size of the chunk and usage flags. Unallocated chunks also store pointers to other free chunks in the usable space area, making the minimum chunk size 16 bytes (32-bit system) and 24 bytes (64-bit system).
This means that even a single byte allocated memory block malloc(1) uses at least 16 bytes to 24 bytes. This is because memory alignment problem and each allocated block needs additional bytes to store metadata of the block. You can easily google how malloc() works to understand why there is such overhead.
Therefore, to meet your purpose, the test should allocate much larger memory block at each iteration to minimize such overhead. I would personally recommend 4kb or 1MB instead of sizeof(int).
I'm trying to read byte per byte from the RX FIFO, that's 8 bytes deep.
The problem is that when the first byte is received, I have to wait for 8 other bytes to finally get the first byte on U1RXREG.
I'd like to know how to perform some dummy reads on the FIFO to access a particuar byte and "flush" it. Simply putting U1RXREG in an array doesn't do the trick.
Here's my initialization :
void UART_Initialize(void)
{
// SIDL: Stop in Idle Mode bit : 1 = Discontinue module operation when the device enters Idle mode
U1MODEbits.SIDL = 0;
// IREN: IrDA Encoder and Decoder Enable bit : 0 = IrDA is disabled
U1MODEbits.IREN = 0;
// RTSMD: Mode Selection for UxRTS Pin bit : 0 = UxRTS pin is in Flow Control mode
U1MODEbits.RTSMD = 0;
// UEN<1:0>: UARTx Enable bits : 10 = UxTX, UxRX, UxCTS and UxRTS pins are enabled and used
U1MODEbits.UEN1 = 1;
U1MODEbits.UEN0 = 0;
// WAKE: Enable Wake-up on Start bit Detect During Sleep Mode bit : 0 = Wake-up disabled
U1MODEbits.WAKE = 0;
// LPBACK: UARTx Loopback Mode Select bit : 0 = Loopback mode is disabled
U1MODEbits.LPBACK = 0;
// ABAUD: Auto-Baud Enable bit : 0 = Baud rate measurement disabled or completed
U1MODEbits.ABAUD = 0;
// RXINV: Receive Polarity Inversion bit : 0 = UxRX Idle state is ‘1’
U1MODEbits.RXINV = 0;
// BRGH: High Baud Rate Enable bit : 0 = Standard Speed mode – 16x baud clock enabled
U1MODEbits.BRGH = 0;
// PDSEL<1:0>: Parity and Data Selection bits : 01 = 8-bit data, even parity
U1MODEbits.PDSEL1 = 0;
U1MODEbits.PDSEL0 = 1;
// STSEL: Stop Selection bit : 0 = 1 Stop bit
U1MODEbits.STSEL = 0;
// ADM_EN: Automatic Address Detect Mode Enable bit : 0 = Automatic Address Detect mode is disabled
U1STAbits.ADM_EN = 0;
// UTXISEL<1:0>: TX Interrupt Mode Selection bits : 00 = Interrupt is generated and asserted while the transmit buffer contains at least one empty space
U1STAbits.UTXISEL1 = 0;
U1STAbits.UTXISEL0 = 0;
// UTXINV: Transmit Polarity Inversion bit : 0 = UxTX Idle state is ‘1’ (with IrDA disbled)
U1STAbits.UTXINV = 0;
// URXEN: Receiver Enable bit : 1 = UARTx receiver is enabled. UxRX pin is controlled by UARTx (if ON = 1)
U1STAbits.URXEN = 1;
// UTXBRK: Transmit Break bit : 0 = Break transmission is disabled or completed
U1STAbits.UTXBRK = 0;
// URXISEL<1:0>: Receive Interrupt Mode Selection bit : 00 = Interrupt flag bit is asserted while receive buffer is not empty (i.e., has at least 1 data character)
U1STAbits.URXISEL1 = 0;
U1STAbits.URXISEL0 = 0;
// ADDEN: Address Character Detect bit (bit 8 of received data = 1) : 0 = Address Detect mode is disabled
U1STAbits.ADDEN = 0;
// Baud Rate Calculation :
// FPB = 10MHz ; Desired Baud Rate = 9600 bauds
// => U1BRG = FPB/(16*BaudRate)-1 = 64 (error = 0,16%)
U1BRG = 64;
// Enable UART RX interrupts
//IEC1bits.U1RXIE = 1;
// Enable UART
// ON: UARTx Enable bit : 1 = UARTx is enabled. UARTx pins are controlled by UARTx as defined by the UEN<1:0> and UTXEN control bits.
U1MODEbits.ON = 1;
// UTXEN: Transmit Enable bit : 1 = UARTx transmitter is enabled. UxTX pin is controlled by UARTx (if ON = 1).
U1STAbits.UTXEN = 1;
}
For the moment I tried reading like this, in the while(1) loop, without success :
while (1)
{
uint8_t rxbyte[8];
bool b;
//if (U1STAbits.URXDA == 1)
while(!U1STAbits.URXDA);
rxbyte[0] = U1RXREG;
rxbyte[1] = U1RXREG;
rxbyte[2] = U1RXREG;
rxbyte[3] = U1RXREG;
rxbyte[4] = U1RXREG;
rxbyte[5] = U1RXREG;
rxbyte[6] = U1RXREG;
rxbyte[7] = U1RXREG;
sprintf(s, "I received : %u %u %u %u %u %u %u %u\n\r", rxbyte[0], rxbyte[1], rxbyte[2], rxbyte[3], rxbyte[4], rxbyte[5], rxbyte[6], rxbyte[7]);
myPrint(s);
IFS1bits.U1RXIF = 0;
}
The 8 bytes in rxbytes[] are always the same. I tried to clear the RX interrupt flag between the reads, reading URXDA also between the reads, add a delay, still no success, I still have to wait for 8 incoming bytes to access the first one.
Thanx in advance for your help !
Best regards.
Eric
I finally found the solution : I had a double configuration, one by MCC and one by myself, and it seems that having the interruptions activated caused this. Now I switched the interrupts off and I simply poll URXDA, and everything works fine.
Sorry for bothering you guys !
You can use your original method too. The problem with your While(1) loop is that you check if there is "at least 1 byte in the hardware Rx buffer" ( U1STAbits.URXDA tells you as soon as 1 byte is availlable ). When 1 byte is availlable, then you read 8 bytes inside the loop... so of course you will get 8 identical bytes when there is only 1 byte in the buffer.
http://play.golang.org/p/RqScJVvpS7
package main
import (
"fmt"
"math/rand"
"encoding/binary"
)
func main() {
buffer := []byte{0, 0, 0, 0, 0, 0, 0, 0}
num := rand.Int63()
count := binary.PutVarint(buffer, num)
fmt.Println(count)
}
I had this working awhile ago when num was just an incrementing uint64 and I was using binary.PutUvarint but now that it's a random int64 and binary.PutVarint I get an error:
panic: runtime error: index out of range
goroutine 1 [running]:
encoding/binary.PutUvarint(0x1042bf58, 0x8, 0x8, 0x6ccb, 0xff9faa4, 0x9acb0442, 0x7fcfd52, 0x4d658221)
/usr/local/go/src/encoding/binary/varint.go:44 +0xc0
encoding/binary.PutVarint(0x1042bf58, 0x8, 0x8, 0x6ccb, 0x7fcfd52, 0x4d658221, 0x14f9e0, 0x104000e0)
/usr/local/go/src/encoding/binary/varint.go:83 +0x60
main.main()
/tmp/sandbox010341234/main.go:12 +0x100
What am I missing? I would have thought this to be a trivial change...
EDIT: I just tried extending my buffer array. For some odd reason it works and I get a count of 10. How can that be? int64 is 64 bits = 8 bytes, right?
Quoting the doc of encoding/binary:
The varint functions encode and decode single integer values using a variable-length encoding; smaller values require fewer bytes. For a specification, see https://developers.google.com/protocol-buffers/docs/encoding.
So the binary.PutVarint() is not a fixed, but a variable-length encoding. When passing an int64, it will need more than 8 bytes for large numbers, and less than 8 bytes for small numbers. Since the number you're encoding is a random number, it will have random bits even in its highest byte.
See this simple example:
buffer := make([]byte, 100)
for num := int64(1); num < 1<<60; num <<= 4 {
count := binary.PutVarint(buffer, num)
fmt.Printf("Num=%d, bytes=%d\n", num, count)
}
Output:
Num=1, bytes=1
Num=16, bytes=1
Num=256, bytes=2
Num=4096, bytes=2
Num=65536, bytes=3
Num=1048576, bytes=4
Num=16777216, bytes=4
Num=268435456, bytes=5
Num=4294967296, bytes=5
Num=68719476736, bytes=6
Num=1099511627776, bytes=6
Num=17592186044416, bytes=7
Num=281474976710656, bytes=8
Num=4503599627370496, bytes=8
Num=72057594037927936, bytes=9
The essence of variable-length encoding is that small numbers use less bytes, but this can only be achieved if in turn big numbers may use more than 8 bytes (that would be size of int64).
Details of the specific encoding is on the linked page.
A very easy example would be: A byte is 8 bits. Use 7 bits of the output byte as the "useful" bits to encode the data/number. If the highest bit is 1, that means more bytes are required. If highest bit is 0, we're done. You can see that small numbers can be encoded using 1 output byte (e.g. n=10), while we're using 1 extra bit for every 7-bit useful data, so if the input number uses all the 64 bits, we will end up with more than 8 bytes: 10 groups are required to cover 64 bits, so we will need 10 bytes (9 groups is only 9*7=63 bits).
I am basically a beginner in Computer Science. Please forgive me if I ask elementary questions. I am trying to understand radix sort. I read that a 32 bit unsigned integer can be broken down into 4 8-bit chunks. After that, all it takes is "4 passes" to complete the radix sort. Can somebody please show me an example for how this breakdown (32 bit into 4 8-bit chunks) works? Maybe, a 32-bit integer like 2147507648.
Thanks!
You would divide the 32 bit integer up in 4 pieces of 8 bits. Extracting those pieces is a matter of using using some of the operators available in C.:
uint32_t x = 2147507648;
uint8_t chunk1 = x & 0x000000ff; //lower 8 bits
uint8_t chunk2 = (x & 0x0000ff00) >> 8;
uint8_t chunk3 = (x & 0x00ff0000) >> 16;
uint8_t chunk4 = (x & 0xff000000) >> 24; //highest 8 bits
2147507648 decimal is 0x80005DC0 hex. You an pretty much eyeball those 8 bits out of the hex representation, since each hex digit represents 4 bits, two and two of them represents 8 bits.
So that now means chunk 1 is 0xC0, chunk 2 is 0x5D, chunk3 is 0x00 and chunk 4 is 0x80
It's done as follows:
2147507648
=> 0x80005DC0 (hex value of 2147507648)
=> 0x80 0x00 0x5D 0xC0
=> 128 0 93 192
To do this, you'd need bitwise operations as nos suggested.