Serial communication with minimal delay - windows

I have a computer which is connected with external devices via serial communication (i.e. RS-232/RS-422 of physical or emulated serial ports). They communicate with each other by frequent data exchange (30Hz) but with only small data packet (less than 16 bytes for each packet).
The most critical requirement of the communication is low latency or delay between transmitting and receiving.
The data exchange pattern is handshake-like. One host device initiates communication and keeps sending notification on a client device. A client device needs to reply every notification from the host device as quick as possible (this is exactly where the low latency needs to be achieved). The data packets of notifications and replies are well defined; namely the data length is known.
And basically data loss is not allowed.
I have used following common Win API functions to do the I/O read/write in a synchronous manner:
CreateFile, ReadFile, WriteFile
A client device uses ReadFile to read data from a host device. Once the client reads the complete data packet whose length is known, it uses WriteFile to reply the host device with according data packet. The reads and writes are always sequential without concurrency.
Somehow the communication is not fast enough. Namely the time duration between data sending and receiving takes too long. I guess that it could be a problem with serial port buffering or interrupts.
Here I summarize some possible actions to improve the delay.
Please give me some suggestions and corrections :)
call CreateFile with FILE_FLAG_NO_BUFFERING flag? I am not sure if this flag is relevant in this context.
call FlushFileBuffers after each WriteFile? or any action which can notify/interrupt serial port to immediately transmit data?
set higher priority for thread and process which handling serial communication
set latency timer or transfer size for emulated devices (with their driver). But how about the physical serial port?
any equivalent stuff on Windows like setserial/low_latency under Linux?
disable FIFO?
thanks in advance!

I solved this in my case by setting the comm timeouts to {MAXDWORD,0,0,0,0}.
After years of struggling this, on this very day I finally was able to make my serial comms terminal thingy fast enough with Microsoft's CDC class USB UART driver (USBSER.SYS, which is now built in in Windows 10 making it actually usable).
Apparently the aforementioned set of values is a special value that sets minimal timeouts as well as minimal latency (at least with the Microsoft driver, or so it seems to me anyway) and also causes ReadFile to return immediately if no new characters are in the receive buffer.
Here's my code (Visual C++ 2008, project character set changed from "Unicode" to "Not set" to avoid LPCWSTR type cast problem of portname) to open the port:
static HANDLE port=0;
static COMMTIMEOUTS originalTimeouts;
static bool OpenComPort(char* p,int targetSpeed) { // e.g. OpenComPort ("COM7",115200);
char portname[16];
sprintf(portname,"\\\\.\\%s",p);
port=CreateFile(portname,GENERIC_READ|GENERIC_WRITE,0,0,OPEN_EXISTING,0,0);
if(!port) {
printf("COM port is not valid: %s\n",portname);
return false;
}
if(!GetCommTimeouts(port,&originalTimeouts)) {
printf("Cannot get comm timeouts\n");
return false;
}
COMMTIMEOUTS newTimeouts={MAXDWORD,0,0,0,0};
SetCommTimeouts(port,&newTimeouts);
if(!ComSetParams(port,targetSpeed)) {
SetCommTimeouts(port,&originalTimeouts);
CloseHandle(port);
printf("Failed to set COM parameters\n");
return false;
}
printf("Successfully set COM parameters\n");
return true;
}
static bool ComSetParams(HANDLE port,int baud) {
DCB dcb;
memset(&dcb,0,sizeof(dcb));
dcb.DCBlength=sizeof(dcb);
dcb.BaudRate=baud;
dcb.fBinary=1;
dcb.Parity=NOPARITY;
dcb.StopBits=ONESTOPBIT;
dcb.ByteSize=8;
return SetCommState(port,&dcb)!=0;
}
And here's a USB trace of it working. Please note the OUT transactions (output bytes) followed by IN transactions (input bytes) and then more OUT transactions (output bytes) all within 3 milliseconds:
And finally, since if you are reading this, you might be interested to see my function that sends and receives characters over the UART:
unsigned char outbuf[16384];
unsigned char inbuf[16384];
unsigned char *inLast = inbuf;
unsigned char *inP = inbuf;
unsigned long bytesWritten;
unsigned long bytesReceived;
// Read character from UART and while doing that, send keypresses to UART.
unsigned char vgetc() {
while (inP >= inLast) { //My input buffer is empty, try to read from UART
while (_kbhit()) { //If keyboard input available, send it to UART
outbuf[0] = _getch(); //Get keyboard character
WriteFile(port,outbuf,1,&bytesWritten,NULL); //send keychar to UART
}
ReadFile(port,inbuf,1024,&bytesReceived,NULL);
inP = inbuf;
inLast = &inbuf[bytesReceived];
}
return *inP++;
}
Large transfers are handled elsewhere in code.
On a final note, apparently this is the first fast UART code I've managed to write since abandoning DOS in 1998. O, doest the time fly when thou art having fun.
This is where I found the relevant information: http://www.egmont.com.pl/addi-data/instrukcje/standard_driver.pdf

I have experienced similar problem with serial port.
In my case I resolved the problem decreasing the latency of the serial port.
You can change the latency of every port (which by default is set to 16ms) using control panel.
You can find the method here:
http://www.chipkin.com/reducing-latency-on-com-ports/
Good Luck!!!

Related

RFduino - Faulty Transmissions

I have a project where I send data to an Android phone. I get data via the serial port to the rfduino and then send this data to the phone.
First I had 20 byte data chunks. I use "Serial Event" to set a flag and then read and send the data within the main loop. The serial baud rate is 9600 as recommended by the rfduino staff.
Now I have 40 bytes of data. The Rfduino reads 40 bytes of serial data and sends it in two parts (see code). The first byte of each part is an identifier used to distunguish between the different packets.
After some successful transmissions the data becomes corrupt. Usually, once the transmission starts being faulty, I have to reinitiate the connection to get correct data.
This is a basic idea of the code:
void loop() {
if (mData)
{
mData = false;
Serial.readBytes(data, 40);
while(!RFduinoBLE.send(&data[0], 20));
while(!RFduinoBLE.send(&data[20], 20));
}
}
void serialEvent(void) {
mData = true;
}
I checked the serial data with a logic analyzer and there is nothing wrong with it. I also used wireshark to check the ble transmissions. Both 20 Byte chunks are send within the same connection interval. The data sent by the RFduino becomes corrupt after some time.
When I omit the second packet and just send 20 Bytes, the problem does not occur.
I assume, that the radio interferes with the reading of the serial port. I tried to use while(RFduinoBLE.radioActive); before the Serial.read, but I think the data is buffered before that. So this did not change anything.
Additionally I tried to lower the sending power to minimum, without any improvements.
I also tried different connection intervals. I have new data every 128ms. This limits the max. connection interval. All changes did not help at all.
I've read that the BLE radio priority takes about 5-6ms and that there is a 6 byte buffer for data. Using 9600 baud this buffer overflows.
I haven't looked for sources to those statements yet, but lowering the baud rate to 4800 seems to improve the issue with the faulty data.
Still this is no satisfying data rate and the transmission faults still occur. Not that often anymore, but still.
I've run out of ideas how to fix this and would appreciate any thought!
I mean it must be possible to send 40 bytes every 128ms... that's not that much.
I already posted this question in the RFduino Forum - without answer until now - and I wonder if anyone experienced similar issues with the RFduino.
RFduino Forum

Serial I/O Overlapped/Non-Overlapped with Windows/Windows CE

I'm sorry this isn't much of a question, but more of to help people having problems with these particular things. The problem I'm working on requires the use of Serial I/O, but is primarily running under Windows CE 6.0. However, I was recently asked if the application could also be made to work under Windows too, so I set about solving this problem. I did spend quite a lot of time looking around to see if anyone had the answers I was looking for and it all came across as a lot of misinformation and things that were just basically wrong in some instances. So having solved this problem, I thought I'd share my findings with everyone so anyone encountering these difficulties would have answers.
Under Windows CE, OVERLAPPED I/O is NOT supported. This means that bi-directional communication through the serial port can be quite troublesome. The main problem being that when you are waiting on data from the serial port, you cannot send data because doing so will cause your main thread to block until the read operation completes or timeouts (depending on whether you've set timeouts up)
Like most people doing serial I/O, I had a reader serial thread set up for reading the serial port, which used WaitCommEvent() with an EV_RXCHAR mask to wait for serial data. Now this is where the difficulty arises with Windows and Windows CE.
If I have a simple reader thread like this, as an example:-
UINT SimpleReaderThread(LPVOID thParam)
{
DWORD eMask;
WaitCommEvent(thParam, &eMask, NULL);
MessageBox(NULL, TEXT("Thread Exited"), TEXT("Hello"), MB_OK);
}
Obviously in the above example, I'm not reading the data from the serial port or anything and I'm assuming that thParam contains the opened handle to the comm port etc. Now, the problem is under Windows when your thread executes and hits the WaitCommEvent(), your reader thread will go to sleep waiting for serial port data. Okay, that's fine and as it should be, but... how do you end this thread and get the MessageBox() to appear? Well, as it turns out, it's not actually that easy and is a fundamental difference between Windows CE and Windows in the way it does its Serial I/O.
Under Windows CE, you can do a couple of things to make the WaitCommEvent() fall through, such as SetCommMask(COMMPORT_HANDLE, 0) or even CloseHandle(COMMPORT_HANDLE). This will allow you to properly terminate your thread and therefore release the serial port for you to start sending data again. However neither of these things will work under Windows and both will cause the thread you call them from to sleep waiting on the completion of the WaitCommEvent(). So, how do you end the WaitCommEvent() under Windows? Well, ordinarily you'd use OVERLAPPED I/O and the thread blocking wouldn't be an issue, but since the solution has to be compatible with Windows CE as well, OVERLAPPED I/O isn't an option. There is one thing you can do under Windows to end the WaitCommEvent() and that is to call the CancelSynchronousIo() function and this will end your WaitCommEvent(), but be aware this can be device dependent. The main problem with CancelSynchronousIo() is that it isn't supported by Windows CE either, so you're out of luck using that for this problem!
So how do you do it? The fact is, to solve this problem, you simply can't use WaitCommEvent() as there is no way to terminate this function on Windows that is supported by Windows CE. That then leaves you with ReadFile() which again will block whilst it is reading NON OVERLAPPED I/O and this WILL work with Comm Timeouts.
Using ReadFile() and a COMMTIMEOUTS structure does mean that you will have to have a tight loop waiting for your serial data, but if you're not receiving large amount of serial data, it shouldn't be a problem. Also an event for ending your loop with a small timeout will also ensure that resources are passed back to the system and you're not hammering the processor at 100% load. Below is the solution I came up with and would appreciate some feedback, if you think it could be improved.
typedef struct
{
UINT8 sync;
UINT8 op
UINT8 dev;
UINT8 node;
UINT8 data;
UINT8 csum;
} COMMDAT;
COMSTAT cs = {0};
DWORD byte_count;
COMMDAT cd;
ZeroMemory(&cd, sizeof(COMMDAT));
bool recv = false;
do
{
ClearCommError(comm_handle, 0, &cs);
if (cs.cbInQue == sizeof(COMMDAT))
{
ReadFile(comm_handle, &cd, sizeof(COMMDAT), &byte_count, NULL);
recv = true;
}
} while ((WaitForSingleObject(event_handle, 2) != WAIT_OBJECT_0) && !recv);
ThreadExit(recv ? cd.data : 0xFF);
So to end the thread you just signal the event in the event_handle and that allow you to exit the thread properly and clean up resources and works correctly on Windows and Windows CE.
Hope that helps everyone who I've seen has had difficulty with this problem.
Since I think there was a misunderstanding in my comment above, here's more detail on two possible solutions that don't use a tight loop. Note that these use runtime determination and aretherefore fine under both OSes (though you have to compile for each target separately anyway) and since neither use an #ifdef it's less likely to end up breaking the compiler on one side or the other without you noticing immediately.
First, you could dynamically load CancelSynchonousIo and use it when present in the OS. Even optionally doing something instead of the Cancel for CE (like maybe closing the handle?);
typedef BOOL (WINAPI *CancelIo)(HANDLE hThread);
HANDLE hPort;
BOOL CancelStub(HANDLE h)
{
// stub for WinCE
CloseHandle(hPort);
}
void IoWithCancel()
{
CancelIo cancelFcn;
cancelFcn = (CancelIo)GetProcAddress(
GetModuleHandle(_T("kernel32.dll")),
_T("CancelSynchronousIo"));
// if for some reason you want something to happen in CE
if(cancelFcn == NULL)
{
cancelFcn = (CancelIo)CancelStub;
}
hPort = CreateFile( /* blah, blah */);
// do my I/O
if(cancelFcn != NULL)
{
cancelFcn(hPort);
}
}
The other option, which takes a bit more work as you're going to likely have different threading models (though if you're using C++, it would be an excellent case for separate classes based on platform anyway) would be to determine the platform and use overlapped on the desktop:
HANDLE hPort;
void IoWithOverlapped()
{
DWORD overlapped = 0;
OSVERSIONINFO version;
GetVersionEx(&version);
version.dwOSVersionInfoSize = sizeof(OSVERSIONINFO);
if((version.dwPlatformId == VER_PLATFORM_WIN32_WINDOWS)
|| (version.dwPlatformId == VER_PLATFORM_WIN32_NT))
{
overlapped = FILE_FLAG_OVERLAPPED;
}
else
{
// create a receive thread
}
hPort = CreateFile(
_T("COM1:"),
GENERIC_READ | GENERIC_WRITE,
FILE_SHARE_READ | FILE_SHARE_WRITE,
NULL,
OPEN_EXISTING,
overlapped,
NULL);
}

How to pass an integer from an Arduino to a C application on Mac OS X

What is the standard method to read values on an Arduino from a C application?
I have an accelerometer and a few poentiometers which I would like to bind to Cocoa controls, an NSSlider for example.
My arduino is currently connected to: /dev/cu.usbmodem26431 and I can read the values printed by the AnalogReadSerial code sample in the Serial Monitor.
How do I read from /dev/cu.usbmodem26431 ?
Cocoa Serial (Edit)
Three methods of interfacing with Objective C are presented here
http://arduino.cc/playground/Interfacing/Cocoa
Arduino Serial
If you know how to establish a Serial/USB connection then you can send the values as either string or binary to the Arduino.
In the setup() method on the Arduino you establish as Serial connection like this.
nb: using 115200 baud/speed for this example
Serial.begin(115200);
In the loop() method on the Arduino you can read data from the c application. here is a full Arduino example including how to send data back to the c application using Serial.print()
int incomingByte = 0; // for incoming serial data
void setup() {
Serial.begin(115200); // opens serial port, sets data rate to 115200 bps
}
void loop() {
// send data only when you receive data:
if (Serial.available() > 0) {
// read the incoming byte:
incomingByte = Serial.read();
// say what you got:
Serial.print("I received: ");
Serial.println(incomingByte, DEC);
}
}
Source http://arduino.cc/en/Serial/read
The example above is very crude, you can use Serial.available() to see how much data is waiting to be read.
For strings you might want to use a \n line terminator or other type of terminator as an indicator for the end of a packet (to know when it has been fully received).
It is a good idea to design your own header and check sum to ensure data integrity but I don't bother with check sums for simple unimportant projects.
As an example, gps systems often send sentences when in string/text mode. If you look up the Arduino library called tinyGPS, you will see one way to read an entire sentence into different variables within an Arduino program.
This is an NMEA GPS sentence, do not use the same header in your projects, instead, design your own. This is just an example of how you might transmit multiple values (int, string, float etc) to an Arduino
$GPBWC,081837,,,,,,T,,M,,N,*13
http://aprs.gids.nl/nmea/

Why does DeviceIoControl prepend 12 bytes of information to the user-provided input buffer?

I hope this does not turn out to be a totally braindead question.
I am editing a template WDF Windows USB device driver to send formatted data to one of the device's bulk out pipes; the data has to be set up in a certain way to tell the device to read an internal register.
The problem is that I cannot get the data to go across the bus in the exact format necessary. I wrote a small test app to enumerate the device and call DeviceIoControl with the input buffer set to a struct I set up according to spec.
I have a copy of a USB bus trace for a working case (performed by a driver whose source I have no access to), and I captured a bus trace for what happens when I call the custom IOCTL in my driver. What I see go across the bus is the data structure I set up prefixed with twelve bytes of data; the data structure is correct, but I want to know what the initial twelve bytes of data are, and stop the driver from sending them.
The driver, I believe, has been written properly; I put some debug traces in the driver and it looks like the buffer retrieved by WdfRequestRetrieveInputMemory already has the 12 bytes prepended, so this seems like this is happening pre-driver.
If it is useful information, the IOCTL is set up as METHOD_BUFFERED with FILE_ANY_ACCESS.
The relevant portion of the test code that sets this up is very simple:
const ULONG ulBufferSize = sizeof( CONTROL_READ_DATA );
unsigned char pBuffer[sizeof(CONTROL_READ_DATA)];
DWORD dwBytesReturned;
CONTROL_READ_DATA* readData = (CONTROL_READ_DATA*)pBuffer;
readData->field1 = data;
readData->field2 = moreData;
// ... all fields filled in...
// Send IOCTLs into camera
if( !::DeviceIoControl( hDevice,
IOCTL_CUSTOM_000,
&readData,
ulBufferSize,
&readData,
ulBufferSize,
&dwBytesReturned,
NULL ) )
{
dwError = ::GetLastError();
// Clean up here
return dwError;
}
The data I see go across the bus is: 80FD1200 CCCCCCCC CCCCCCCC + (My data).
Does anyone have any insights?
Wow, really ridiculous error. Notice I'm passing the address of readData to DeviceIoControl, which itself is already a pointer. I can't believe I wasted so much time on this.
Thanks all!
Alignment of the data is the culprit. Check out http://msdn.microsoft.com/en-us/library/2e70t5y1(v=vs.80).aspx to set it to one.

Win32 API: ReadFile not timing out

I'm writing some code to interface with a piece of hardware. The hardware connects to the PC via a USB with a USB-to-Serial converter inside the device (it shows up as a COM port device in Windows).
I'm having issues with the Win32 API ReadFile system call. I can't seem to get it to work as advertised. I've setup the COMMTIMEOUTS structure as so:
COMMTIMEOUTS ct;
ct.ReadIntervalTimeout = MAXDWORD;
ct.ReadTotalTimeoutconstant = 0;
ct.ReadTotalTimeoutMultiplier = 0;
ct.WriteTotalTimeoutConstant = 0;
ct.WriteTotalTimeoutMultiplier = 0;
if(SetCommTimeouts(device_id_, &ct) == 0)
{
return ERROR; // this is never hit.
}
Which according to the Win32 API documentation, says:
ReadIntervalTimeout
The maximum time
allowed to elapse between the arrival
of two bytes on the communications
line, in milliseconds. During a
ReadFile operation, the time period
begins when the first byte is
received. If the interval between the
arrival of any two bytes exceeds this
amount, the ReadFile operation is
completed and any buffered data is
returned. A value of zero indicates
that interval time-outs are not used.
A value of MAXDWORD, combined with
zero values for both the
ReadTotalTimeoutConstant and
ReadTotalTimeoutMultiplier members,
specifies that the read operation is
to return immediately with the bytes
that have already been received, even
if no bytes have been received.
The command I'm sending is supposed to return a single byte integer. Most of the time, the command is received by the device and it returns the appropriate value. Sometimes, however, it doesn't seem to return a value and ReadFile() blocks until more bytes are recieved (eg. by pressing buttons on the device). Once a button is hit, the initial integer response I was expecting is received along with the button press code. While this isn't the behavior I'm expecting from the device itself, I'm more concerned with ReadFile() blocking when it shouldn't be, according to the MSDN documentation. Is there a remedy for ReadFile() blocking here?
D'oh! Turns out ReadFile blocking was just a symptom, not the problem. The hardware device in question only has a 4MHz processor in it. Splitting up the 3 character command written to the device and sending them individually with a 1ms pause between characters fixes the issue.

Resources