I am attempting to send camera frames to a web socket server from an ESP32 with a camera. I am using the ESP-IDF to implement the client web socket on the device. Specifically, I am using: https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/protocols/esp_websocket_client.html.
I have noticed an issue where the server does not receive full frames from the device. I did some digging, read through RFC6455 and became aware of the concept of frame fragmentation. My thinking now is that the camera frames (640 x 480 pixels) need to be fragmented before being sent. I read through the ESP-IDF docs and only saw one mention of fragmentation.
I feel comfortable enough to implement it myself, but it seems that there is no way to set the FIN bit, OPCODE, or any other parameters of the header described by RFC6455 using the ESP-IDF web socket client library.
Does any one have any idea how these parameters can be set, or if there is some way to enable native frame fragmentation on the device?
Looking at the source code, it appears that the message is already being fragmented, if the message exceeds the size of buffer_size.
The default size of buffer_size is 1024 bytes. You can change it in the config.
const esp_websocket_client_config_t websocket_cfg = {
...
.buffer_size = 512,
};
Related
I use Embarcadero RAD Studio & Indy and I have a problem.
I am using component IdUDPServer and try to send file via UDP messages.
The file is successfully transferred if it is not large.
However, the error message “Package Size Too Big” appears if the file is large.
My code:
FileAPP->ReadBuffer(Buffer, Size);
IdUDPServer1->SendBuffer("10.6.1.255", 34004, RawToBytes(&Buffer, Size));
I see two ways to solve this problem:
1) Compose small messages and send these messages in a loop.
This is an easy way to solve the problem, however I will need to fix the remote application.
This is currently difficult. I do not have much time.
2)I want to find a condition in the source code where this error message is generated.
Maybe I can fix the condition and it will work for my specific task.
I have added some pictures. This is all the information that I now have.
Screenshots from help file: Picture 1, Picture 2, Picture 3
Error message: Picture 4
Please help me if you know the solution.
"Package Size Too Big" means you tried to send a datagram that is too large for UDP to handle. UDP has an effective max limit of 65507 bytes of payload data per datagram. You can't send a large file in a single datagram, it just won't fit.
So yes, you will have to break up your file into smaller chucks that are sent in separate datagrams. And as such, because UDP is connection-less and does not guarantee delivery, you will also have to solve the issues of lost packets and out-of-order packets. You will have to put a rolling sequence number inside your packets so that the receiver can save packets in the proper order. And you will have to implement a re-transmission mechanism to re-send lost packets that the receiver did not receive.
I would suggest not using TIdUDPClient/TIdUDPServer to implement file transfers manually, if possible. Indy has TIdTrivialFTP/TIdTrivialFTPServer components that implement the standardized UDP-based TFTP protocol, which handles these kind of details for you. By default, TFTP has a limit of ~32MB per file (512 bytes per datagram * 65535 datagrams max), but you can increase that limit using the TIdTrivialFTP.RequestedBlockSize property to request permission from a TFTP server to send more bytes per datagram. RequestedBlockSize is set to 1500 bytes by default, which can accommodate files up to ~93.7MB in size (though in practice it should not be set higher than 1468 bytes so TFTP datagrams won't exceed the MTU size of Ethernet packets, thus limiting the max file size closer to ~91.7MB). The RequestedBlockSize can be set as high as 65464 bytes per datagram to handle files up to ~3.9GB (at the risk of fragmenting UDP packets at the IP layer).
It is also possible for TFTP to handle unlimited file sizes with smaller datagram sizes, though TidTrivialFTP/TIdTrivialFTPServer do not currently implement this, because this behavior is not currently standardized in the TFTP protocol, but is commonly implemented as an extension in 3rd party implementations.
Otherwise, you should consider using TCP instead of UDP for your file transfers. TCP doesn't suffer from these problems.
I'm trying to implement S-MAC protocol on waspmote xbee sensors and i know it has its own CSMA/CA. So first of all I need to understand the basic of xBee collision avoidance.
Two senders set up in api mode in libraries and both periodically sending single bytes to a common receiver. I reduce the delay and many changing in libabries to make collision and to see how algorithm works. But when i monitor data at the receiver all looks as expected at the receiver .. byte1, byte 2 .. byte1, byte2.
Do u have any idea how can i make collision?
Are you sniffing the 802.15.4 traffic? That's the only way you'd see a collision.
The XBee module buffers the data you want to send, using the host communication parameters (baud rate, API mode, etc.) and then sends it out over 802.15.4 at 250kbps. The module has all of the collision avoidance built in, and will retransmit as necessary to deliver your message. If it's unable to deliver after some number of transmission attempts, you'll get a Transmit Status frame indicating failure.
On the receiving end, it buffers the data and delivers it to the local host using local serial settings (baud rate and API mode).
If you're trying to implement S-MAC, you need a different radio processor where you have low-level control over the radio. The XBee module provides an application layer and handles the MAC layer itself.
I am using Arduino Leonardo to transmit an string to a wifi module. The format of command that wifi module can recognize is:
AT60,1,content to a server
I am using an virtual server(TCP/IP Builder) to test the content I can received.
Here is the content I want to send:
smart/device/deviceCmd?userId=1010002003&deviceId=A00019999990002&cmd=ON
Since I try to send it again and again, I use a loop to send it. In the virtual server side, the content I got is:
smart/device/deviceCmd?userId=1010002003&devceId=A00019999990002&cmd=ON
smart/device/deviceCmd?userId=1010002003&devceId=A00019999990002&cmd=ON
smart/device/deviceCmd?userId=1010002003&dviceId=A00019999990002&cmd=ON
smart/device/deviceCmd?userId=1010002003&eviceId=A00019999990002&cmd=ON
smart/device/deviceCmd?userId=1010002003&devieId=A00019999990002&cmd=ON
smart/device/deviceCmd?userId=1010002003deviceId=A00019999990002&cmd=ON
smart/device/deviceCmd?userId=1010002003&dviceId=A00019999990002&cmd=ON
smart/device/deviceCmd?userId=1010002003&dviceId=A00019999990002&cmd=ON
smart/device/deviceCmd?userId=1010002003&deiceId=A00019999990002&cmd=ON
smart/device/deviceCmd?userId=1010002003&dviceId=A00019999990002&cmd=ON
This is the QUESTION: There exist one terrible mistake in the content I received, which is the deviceId part never correct. It's so weird.
Here is part of related code:
//In Uart.cpp
//These three lines can sent a formatted string as "AT60,1,content"
Serial1.write("AT60,");
Serial1.write(channelID); //channel ID = 1 here
Serial1.write(reportIsFire, 76);
//In Uart.h
//Definition of the string I need to send, which has 76 characters.
char reportIsFire[76] = ",smart/device/deviceCmd?userId=1010002003&deviceId=A00019999990002&cmd=ON \n";
Here is few background of this application:
I am using Arduino 1.5.8 IDE with VisualStudio
Since the serial buffer of Arduino is only 64 Bytes, I have already
change the buffer size to 128 Bytes in "HardwareSerial.h" to send
out this large string.
The baud rate is 115200 and I am using Serial 1. I have used Serial 1
to transmit few other characters and it works fine.
I will appreciate that If you have any idea about this question.
I am betting that the serial baud rate of the Arduino is not 100% correct. Increasing the buffer size will not matter if the data is being lost due to a timing issue in the physical link.
I'd recommend double-checking the code that initializes the serial baud rate generator. It may be possible to get a closer rate to 115,200 by either adjusting the available settings, altering the main clock speed (if possible), implementing some form of flow control, or all of the above.
In extreme cases, you may consider using a special-frequency oscillator. Many Microchip PICs use an internal or external 4MHz or 8MHz crystal, but this can produce far too much timing error for lengthy serial transmissions at high speed. In that case, something special, like a 7.3728MHz crystal can be used, bringing the accuracy to exactly 100% (at least on some PIC devices.)
Lastly, another consideration is if any pre-emptive code is running on the device, such as interrupts or timers which could inadvertently interfere with the serial output.
I don't have an answer, but I suspect the most likely problem is that the Wifi card can't read characters at a sustained 115200 baud rate. If possible, set the Wifi baud rate and the Arduino Serial.begin() to a lower rate, such as 57600 or 19200.
If the Arduino baud rate was simply inaccurate, I'd expect to see the problem appearing at random locations in the string, rather than about 40 characters in.
I am using EVK1105 development board with AVR Studio 5 as development IDE for my AVR project.
I am using FreeRTOS in it. I have 3 USART ports on this board. One external module is connected to my AVR32 board via USART-RS232 mode. It sends me continuous serial data to my board on USART0 with 19230 baudrate, 7-databits, odd parity, stopbit-1 and normal-channel mode. I created a new task for this purpose. After each 9 data bytes it sends '\n' and '\r'. So in my task I keep on collecting the 9 databytes in a string buffer and then transmit it on USART1. I am using polling method to collect data from USAR0 which is receiving port. But I am facing problem in receiving data. I don't know if its timing issue or something or the scheduler switches the task while collects the data. But I don't get the required data.
Following are things I have already checked as troubleshooting
1. Connected my external module to my PC hyper-terminal which gives me perfect result.
2. Implemented the same thing of using receiving from USART0 and whatever received is transmitted to USART1 as without FreeRTOS. Its works fine.
Please suggest some idea what may be wrong. I am using a queue to communicate between Tx and Rx task to pass the string buffer from USART0 to USART1. Is it problem in handling queue? How can I troubleshoot the queue?
I am using a delay of 50ms in my infinite task loop in Rx Task. Can it create a problem? If I don't use any delay the OS crashes. Please suggest some good practices to create a new task in FreeRTOS so that I will not get any timing issue.
For such a use case, I would not use a polling method with 50ms delay to retrieve data from UART peripheral. You can easily lose received data depending on the system load and UART reception buffer size.
At least use an interrupt on UART data reception that copies every received byte into a local buffer that will be read by your TX thread.
You can have an even better solution using a DMA channel to receive your data frame and be notified when 9 bytes have been received. I don't know if your AVR device has a DMA peripheral or not.
Are you still working on this? The statement of your problem is vague, but there I have several suggestions/leading questions.
1) You may want some documents to see what the registers are
Get the giant datasheet pdfs at
http://www.atmel.com/dyn/products/product_docs.asp?category_id=163&family_id=607&subfamily_id=2138&part_id=4117
2) In this and an earlier post you state that you have, in some cases, been able to RX data. You will need to find the USART HW initialization code from those example projects and get them into the freeRTOS example project. In particular calls to
gpio_enable_module() with {AVR32_USART0_RXD_0_0_PIN, AVR32_USART0_RXD_0_0_FUNCTION}
To connect to USART to CPU
and i believe
InitRs232()
Just doing this requires poking around a lot of code - there's alot of dependencies.
2) What function are you calling to retrieve data from USART0? 19kbaud is approximately 2000bytes/sec or 1 byte/0.5ms, so 50ms polling is not nearly enough. I'd suggest that your RX task poll continuously (never sleep explicitly) but at a lower priority than the TX task.
3) Concentrate on debugging the RX task at the call to retrieve data. Use the debugger to look at the hardware registers for the usart. In particular
USART0 cr register AVR32_USART_CR_RXEN_MASK should be set to enable RX
USART0 csr register AVR32_USART_CSR_RXRDY_MASK will indicate if there is new data there
You can also check the overlow flag to see if you have missed some data.
When the read of USART0 rhr occurs it should be a byte that you sent.
If you are still working on this I can look into this a bit more.
I am working on a problem in SNMP extension agent in windows, which is passing traps to snmp.exe via SnmpExtensionTrap callback.
We added a couple of fields to the agent recently, and I am starting to see that some traps are getting lost. When I intercept the call in debugger and reduce the length of some strings, the same traps, that would have been lost, will go through.
I cannot seem to find any references to size limit or anything on the data passed via SnmpExtensionTrap. Does anyone know of one?
I would expect the trap size to be limited by the UDP packet size, since SNMP runs over the datagram-oriented UDP protocol.
The maximum size of a UDP packet is 64Kb but you'll have to take into account the SNMP overhead plus any limitations of the transport you're running over (e.g. ethernet).