I have been testing the SDK with code based on the Azure-ESP-Starter code.
When I try to send a larger message body the message fails. I am using an ESP32 Wroom and the SDK installed in VScode.
Message length of 5438 transmits ok, but 5458 fails.
I was looking for a reason and noticed that in the TCP settings, the default send buffer size = 5744
If I change this to 15744 I can send a 10,000 byte message.
The SDK seems to be failing when trying to fragment & send larger messages.
At first I thought it may have something to do with the TLS maximum outgoing fragment length setting (default 4096), but increasing this did not resolve the problem. The TCP buffer setting does allow larger messages.
I thought that Azure allows a 256k message. Has anyone else noticed this issue?
The following document section lists the limits associated with the different service tiers S1, S2, S3, and F1. Device-to-cloud messages can be at most 256 KB, and can be grouped in batches to optimize sends. Batches can be at most 256 KB.
I would suggest making use of a different Azure IoT Library or a different device which you can repro the same issue or maybe with another protocol?
Throttling details
IoT Hub measures message size in a protocol-agnostic way, considering only the actual payload. The size in bytes is calculated as the sum of the following values:
The body size in bytes.
The size in bytes of all the values of the message system properties.
The size in bytes of all user property
names and values.
device-to-cloud messages can be at most 256 KB
The total message size, including the enrichments, can't exceed 256 KB. If a message size exceeds 256 KB, the IoT Hub will drop the message. You can use IoT Hub metrics to identify and debug errors when messages are dropped. For example, you can monitor the telemetry messages incompatible (d2c.telemetry.egress.invalid) metric in the routing metrics. To learn more, see Monitor IoT Hub.
I am writing an interface to a clinical lab machine, which uses ASTM protocol for communication (http://makecircuits.com/blog/2010-06-25-astm-protocol.html).
To start with, I am trying to use golang's TCP Server to receive data. But not able to figure how to send ACK back to lab machine. I am a newbie in golang. Can any one suggest how I can proceed?
The protocol in the link you supplied is for RS232. When sending data over TCPIP it is a stream (the receiver has to know when the data ends). Normally when changing an RS232 protocol to TCPIP, a header is added to each message which is the length of the message (often two bytes), so if you want to send ASCII ACK you send three bytes a two byte length and one byte data. When writing this you must flush the buffer, as for such a small packet it will not be transmitted until you do.
EMI/UCP is a protocol to communicate to SMS gateways.
I am confused how should I do in respect to network connection.
Do you open a socket for each logic bundle of data (and close it of course), or do you re-use the same socket?
How do you handle out-of-sequence responses in both cases?
My use case is send a couple of SMS with status request (submit short message), each will generate a traffic of 4 messages (a 51 operation and its ack and a 53 operation from the gateway plus the 53 ack).
If I open two sockets, may I be confident each dialog is performed on the some socket or is it a false assumption?
If I use a single socket, how I distinguish the messages from the two conversations? From the OaDC (originator address) only?
Answering the last question: you have to match the time stamp in the Ucp51 response message (ACK). It is the field: SCTS.
The correlation id is the couple SCTS and ADC (address caller: it is the recipient cell phone).
Suppose that several machines are interacting together using python's zeroMQ client.
These messages are naturally formatted as strings.
Is there a limit to the length of a message (string)?
There is no limit to the size of messages being sent however small messages are handled differently than large messages (see here).
The max size of a small messages is defined in the source code at 30 bytes (see here, look for ZMQ_MAX_VSM_SIZE).
There is the socket option ZMQ_MAXMSGSIZE which causes a peer sending an oversized message to be disconnected, but the default is "no limit".
No limit
As for small size messages transmitted within zmq_msg_t structures, their limit is 29 bytes (for zmq version 3.2.2)
"max_vsm_size = 29," quoted from https://github.com/zeromq/libzmq/blob/master/src/msg.hpp
Some socket types support up to 2^64, but some less than 2^31.
You should build a protocol that keeps chunks below that size anyway, but this is the real answer.
https://github.com/zeromq/libzmq/issues/1332
Can you explain me what exactly are SO_SNDBUF and SO_RCVBUF options?
OK, for some reason the OS buffers the outgoing/incomming data but I'd like to clarify this subject.
What is their role (generally)?
Are they per-socket buffers?
Is there a connection between Transport layer's buffers (the TCP buffer, for example) and these buffers?
Do they have a different behaviour/role when using stream sockets (TCP) and when using connectionless sockets (UDP)?
A good article will be great too.
I googled it but didn't find any useful information.
The "SO_" prefix is for "socket option", so yes, these are per-socket settings for the per-socket buffers. There are usually system-wide defaults and maximum values.
SO_RCVBUF is simpler to understand: it is the size of the buffer the kernel allocates to hold the data arriving into the given socket during the time between it arrives over the network and when it is read by the program that owns this socket. With TCP, if data arrives and you aren't reading it, the buffer will fill up, and the sender will be told to slow down (using TCP window adjustment mechanism). For UDP, once the buffer is full, new packets will just be discarded.
SO_SNDBUF, I think, only matters for TCP (in UDP, whatever you send goes directly out to the network). For TCP, you could fill the buffer either if the remote side isn't reading (so that remote buffer becomes full, then TCP communicates this fact to your kernel, and your kernel stops sending data, instead accumulating it in the local buffer until it fills up). Or it could fill up if there is a network problem, and the kernel isn't getting acknowledgements for the data it sends. It will then slow down sending data on the network until, eventually, the outgoing buffer fills up. If so, future write() calls to this socket by the application will block (or return EAGAIN if you've set the O_NONBLOCK option).
This all is best described in the Unix Network Programming book.
What is their role (generally)?
Data, that you want to send over a socket, is copied to the send buffer of the socket, so your code doesn't have to wait (=block) until the data has really been sent out to the network. When the send call returns successfully, this only means that the data has been placed into the send buffer from where the protocol implementation will read it as soon as it is ready to send that data over the network.
Keep in mind that multiple sockets from multiple processes may all want to send data at the same time, yet at any time only one data packet can be send over a network line. While sending is in progress, all other senders have to wait and once the line is free, the implementation can only process one send request after another.
Data, that arrives from the network, is written to the receive buffer of the socket by the protocol implementation, where it will wait until your code is reading it from there. Otherwise all receiving would have to stop until your code has processed the incoming packet, yet your code may do other things while a packet arrives in the background and again, the interface is shared, so the system must avoid that other processes cannot receive their network data just because your process is refusing to process its own incoming data.
Are they per-socket buffers?
Yes. Every socket has its own set of buffers.
Is there a connection between Transport layer's buffers (the TCP buffer, for example) and these buffers?
I'm not sure what you mean by "TCP buffers" but if you are referring to the TCP receive and send windows, the answer is yes.
TCP will tell the other side regularly how much room is left in your receive buffer, so that the other side will never send more data than would fit into your receive buffer. If your receive buffer is full, the other side will stop sending completely until there's room again, which will be the case as soon as soon as you read some data from it.
So if you cannot read data as often as would be required to prevent your socket buffer from running full, increasing the receive buffer size can prevent that TCP connections will have to pause sending data.
On the other hand, if the send buffer is running full, the socket will not accept any more data from your code. Any attempt to send will either block or fail with an error (non-blocking socket) until there's room again.
And as TCP can only work with the data currently in the send buffer, the send buffer size also influences the sending behavior of TCP. The TCP sending strategy can depend on various factors. One of them is the amount of data that is known to be sent. If your send buffer is just 2 KB, then TCP will never see more than 2 KB for sending, even though your app may know, that much more data is going to follow. If your send buffer is 256 KB and you put 128 KB of data into it, TCP will know that it has to send 128 KB of data for this connection and this may (and most likely will) influence the sending strategy that TCP uses.
Do they have a different behaviour/role when using stream sockets (TCP) and when using connectionless sockets (UDP)?
Yes. That's because for TCP the data you send is just a stream of bytes. There is no relationship between the bytes and the packets being sent out. Sending 80 bytes could mean sending one packet with 80 bytes or sending 10 packets with each 8 bytes. TCP will decide that on its own. Same for incoming. If there are 200 bytes in your receive buffer, you cannot know how these got there, an amount of bytes you read from a TCP socket may have been transported using any number of packets. So despite transporting data in chunks over packet based networks, a TCP connection behaves like a serial line link.
UDP on the other hand sends datagrams. If you place 80 bytes into the send buffer of an UDP socket, then these 80 bytes are for sure sent out in a single UDP packet containing 80 bytes of payload data. Data is sent in exactly the same way you write it into the send buffer. If you write 80 bytes one after another, 80 packets are sent out, each containing one byte. If you tell a TCP socket to send 200 bytes but there are only 100 bytes room left in the send buffer, TCP will add 100 bytes to the buffer and let you know that 100 of your 200 bytes were added. UDP on the ohter hand will block or fail with an error, as either all 200 bytes fit or nothing fits; there is no partial fit with UDP.
Also when receiving, datagrams are stored in the UDP receive buffer, not bytes. If a TCP socket first receives 80 bytes data and then 200 bytes data, you can perform a read call that reads all 280 bytes at once. If an UDP socket first receives a datagram with 80 bytes and then a datagram with 200 bytes and you request to read 280 bytes from it, you get exactly 80 bytes, as all data returned by a read call must be from the same datagram. You cannot read across datagram borders. Also note that if you request to only read 20 bytes, you receive the first 20 bytes of the datagram and the other 60 bytes are discarded. Next time you read data, it will be from the next datagram whose size was 200 bytes.
So the difference in two sentences: TCP sockets store bytes in the socket buffers, UDP sockes store datagrams in the socket buffers. And datagrams must fit completely in the buffers, incoming datagrams that cannot fit completely into the socket buffer are silently discarded, even though the buffer had some room available.
In Windows, the send buffer does have an effect in UDP. If you blast packets out faster than the network can transmit them, eventually you will fill the socket output buffer and SendTo will fail with "would block". Increasing SO_SNDBUF will help with this. I had to increase both the send and receive buffers for a test I was doing to find the maximum packet rate I could send between a Windows box and a Linux box. I could have also handled the send size by detecting the "would block" error code, sleeping a bit, and retrying. But pumping up the send buffer size was simpler.
The default in Windows is 8K, which seems needlessly small in this era of PC's with GB's of RAM!
Searching Google for "SO_RECVBUF msdn" gave me...
http://msdn.microsoft.com/en-us/library/ms740476(VS.85).aspx
which answers your "are they per socket" with these lines from the options table:
SO_RCVBUF int Specifies the total per-socket buffer space reserved for receives.
SO_SNDBUF int Specifies the total per-socket buffer space reserved for sends.
With more detail later on:
SO_RCVBUF and SO_SNDBUF
When a Windows Sockets implementation supports the SO_RCVBUF and
SO_SNDBUF options, an application can request different buffer sizes
(larger or smaller). The call to setsockopt can succeed even when the
implementation did not provide the whole amount requested. An
application must call getsockopt with the same option to check the
buffer size actually provided.
Above answers didn't answer all questions, especially about the relationship between Socket buffer and TCP buffer.
I think they are different things in different layer. TCP buffer is the consumer of Socket buffer.
Socket buffers (input & output) is an IO buffer that is accessed by System calls from the application code in user space.
For example, with output buffer, the application code can
Send data immediately before the buffer is full and be blocked when buffer is full.
Set the buffer size.
Flush the data in buffer to the underlying storage (TCP send buffer).
Close the output buffer by close the stream.
TCP buffers (send & receive) are in kernel space that only OS can access.
For example, with TCP send buffer, the TCP protocol implementation can
Send packets and accept ACK.
Guarantee delivery and ordering of packets.
Control congestion by resizing the inflight packets window.
By the way, UDP protocol doesn't have buffer but UDP socket can still have IO buffer.
These are my understanding and I'm more than happy to get any feedback/modification/correction.