WaitForSingleObject() gets signalled without occurrence of awaited event - winapi

I'm writing a Class for COM ports in C++ using win-api. Righ now I test the functionality on RS232 with connected Rx and Tx pins.
I've encountered somewhat weird problem. I use separate thread for reading from the COM port. Within the thread I use SetCommMask, WaitCommEvent and WaitForSingleObject to wait for char arrival into buffer. However the WaitForSingleObject tends to exit without actually receiving any chars.
I assumed this was caused by wrong use of mentioned functions, but then I discovered, that premature exit does not occur every time (first time always works as intended).
In second go the thread enters the waiting state and exits a while later proceeding to ReadFile, where it waits indefinitely, because the buffer is empty, no data is to be sent and Total timeout is not used.
I've already been advised to simply use ReadFile and process only the data I acquire, however I use another thread to check if the communication channel has been disconnected and right now I need to distinguish between waiting for data and reading data.
Calling ClearCommError to check input buffer with ReadFile is not an option, because in such a case InQue is always 0. Therefore I cannot tell whether ReadFile is actually reading or waiting.
//following code runs in separate thread
DWORD dwEventMask1, dwEventMask2, LastError, Status;
OVERLAPPED Overlapped; HANDLE Serial_Port_Handle;
std::string stringBuffer("");
const size_t ReadBufferLength = 256;
char tempBuffer[ReadBufferLength];
GetCommMask(Serial_Port_Handle, &dwEventMask1);
if (dwEventMask1) // Before starting the thread I check the state of Input Buffer with GetCommError().
{ // If Buffer is not empty, CommMask is set to 0 signaling there is no need for waiting.
Overlapped.hEvent = CreateEvent(NULL, TRUE, FALSE, NULL);
//wait for comm event
if (!WaitCommEvent(Serial_Port_Handle, &dwEventMask1, &Overlapped))
{
if ((LastError = GetLastError()) == ERROR_IO_PENDING)
{
Waiting = true; //signal bool for synchronization purposes
if ((Status = WaitForSingleObject(Overlapped.hEvent, INFINITE)) == WAIT_OBJECT_0)
{
GetCommMask(Serial_Port_Handle, &dwEventMask2);
Waiting = false;
CloseHandle(Overlapped.hEvent);
// I close handle and set all members of Overlapped struct to 0
}
if (dwEventMask2 !== dwEventMask1) // check if wait have not exited because of SetCommMast()
return;
}
}
}
do // this loop reads from input buffer until empty
{
Overlapped.hEvent = CreateEvent(NULL, TRUE, FALSE, NULL);//set up read overlapped operation
if (ReadFile(Serial_Port_Handle, tempBuffer, ReadBufferLength - 1, &NoBytesRead, &Overlapped)) //start read opperation
{ //Read operation done on 1 go
GetOverlappedResult(Serial_Port_Handle, &Overlapped, &NoBytesRead, FALSE); //get NoBytesRead
CloseHandle(Overlapped.hEvent)
stringBuffer.append(tempBuffer, (size_t) NoBytesRead); // save read data
}
else if ((LastError = GetLastError()) == ERROR_IO_PENDING) //operation not yet complete
{
GetOverlappedResult(Serial_Port_Handle, &Overlapped, &NoBytesRead, TRUE); // wait for completion
stringBuffer.append(tempBuffer, (size_t)NoBytesRead);
}
else
{
CloseHandle(Overlapped.hEvent)
return;
}
} while ((NoBytesRead == (ReadBufferLength - 1)) && NoBytesRead);
// Loop runs while tempBuffer's full capacity is used.
// I realize that since I don't use Total Timeout there is a possibility
// of the loop getting stuck. If you can suggest other solution than using
// Total Timeout or use GetCommError() to get input buffer state, please do so.
return;
This code is somewhat simplified (checking for return values, ets.).
1) Have any of you experienced such a behaviour?
2) I use OVERLAPPED operations in the code. After the operation exits I always use CloseHandle and reinitialize the OVERLAPPED structure before using it for another operation. Is this correct or is resetting the structure sufficient?

It is bad logic as a whole. For example, there are the following issues.
CreateEvent/CloseHandle should not executed at every ReadFile/WaitCommEvent.
The usage of GetCommMask/WaitCommEvent is also wrong.
The read data size specified in ReadFile is fixed regardless of the situation.
It also includes the comment on #RbMm.
You may want to redesign your program with reference to the following articles and source code:
Synchronization and Overlapped Input and Output
Serial Communications
bmo/mttty
In Addition:
I did not notice that a file handle (not an event handle) was specified for WaitForSingleObject, as pointed out by #Rita Han.
The biggest problem is that.
However, the situation that it is better to redesign has not changed.
There is no description of WaitCommEvent and Overlapped for it in the source of #Rita Han's answer. Also, the read data size is fixed in ReadFile.
On the other hand:
Although it does not occur in the source code of the question, it is possible for WaitCommEvent/WaitForSingleObject to generate an event that is not specified in SetCommMask.
While the WaitCommEvent is waiting for completion, change the event mask with SetCommMask.
Remarks - WaitCommEvent function
If a process attempts to change the device handle's event mask by using the SetCommMask function while an overlapped WaitCommEvent operation is in progress, WaitCommEvent returns immediately. The variable pointed to by the lpEvtMask parameter is set to zero.
While WaitCommEvent is waiting for completion, call WaitCommEvent multiple times using the same Overlapped structure.
Synchronization and Overlapped Input and Output
When performing multiple simultaneous overlapped operations on a single thread, the calling thread must specify an OVERLAPPED structure for each operation. Each OVERLAPPED structure must specify a handle to a different manual-reset event object.
A thread should not reuse an event with the assumption that the event will be signaled only by that thread's overlapped operation. An event is signaled on the same thread as the overlapped operation that is completing. Using the same event on multiple threads can lead to a race condition in which the event is signaled correctly for the thread whose operation completes first and prematurely for other threads using that event.
The document is described as above, but depending on the device driver/vendor, the WaitCommEvent that is called later ends with an error, and the WaitCommEvent waiting for completion is lpEvtMask return with zero (as in SetCommMask).
For multiple Overlapped structure variables:
A common programming know-how is that using a single variable for multiple purposes is prone to bugs.
If you are designing in asynchronous and/or multi-threading, it is better to prepare at least three Overlapped structure variables for ReadFile, WriteFile, WaitCommEvent.
About starting ReadFile regardless of the state of the input buffer:
This is about calling ReadFile with a fixed length of 256 bytes without acquiring the size of the received data in the input buffer of the device driver.
In fact, even if all the data arrives, if it is less than 256 bytes, it will always be delayed until a 256 byte receive timeout occurs.
For example, the loop reads one byte at a time until a timeout error occurs, which means the end of the received data (1-byte read timeout would have no impact).
Or, as answered in the previous article, use ClearCommError to obtain the size of the data stored in the device driver's input buffer, and call ReadFile specifying that size.
There will be no problem with the application-side buffer handling you are explaining.
About the behavior of WaiCommEvent when calling SetCommMask:
It may depend on the device driver you are using.

However the WaitForSingleObject tends to exit without actually
receiving any chars.
Wait for the event handle instead of serial device handle.
I got it working. The following is my code example you can have a try:
DWORD errCode = 0;
BOOL result = false;
HANDLE serialDeviceHdl = CreateFile(L"COM8", GENERIC_READ | GENERIC_WRITE, 0, NULL, OPEN_EXISTING, FILE_FLAG_OVERLAPPED, NULL);
if (!serialDeviceHdl)
{
errCode = GetLastError();
cout << "Open device failed. Error code: " << errCode << endl;
return 0;
}
OVERLAPPED overlappedForWrite = {};
overlappedForWrite.hEvent = CreateEvent(NULL, TRUE, FALSE, NULL);
DWORD writenSize = 0;
result = WriteFile(serialDeviceHdl, "hello", 5, &writenSize, &overlappedForWrite);
if (FALSE == result)
{
errCode = GetLastError();
if (ERROR_IO_PENDING == errCode)
{
cout << "Overlapped I/O operation is in progress." << endl;
}
else
{
cout << "Write to device failed. Error code: " << errCode << endl;
}
}
DWORD returnValue = WaitForSingleObject(overlappedForWrite.hEvent, INFINITE);
if (WAIT_OBJECT_0 == returnValue)
{
cout << "The state of the specified object is signaled." << endl;
}
else
{
cout << "Wait for single object failed. Error code: " << returnValue << endl;
}
CHAR readBuf[5];
DWORD readSize = 0;
OVERLAPPED overlappedForRead = {};
overlappedForRead.hEvent = CreateEvent(NULL, TRUE, FALSE, NULL);
result = ReadFile(serialDeviceHdl, readBuf, 5, &readSize, &overlappedForRead);
if (FALSE == result)
{
errCode = GetLastError();
if (ERROR_IO_PENDING == errCode)
{
cout << "Overlapped I/O operation is in progress." << endl;
}
else
{
cout << "Write to device failed. Error code: " << errCode << endl;
}
}
returnValue = WaitForSingleObject(overlappedForRead.hEvent, INFINITE);
if (WAIT_OBJECT_0 == returnValue)
{
cout << "The state of the specified object is signaled." << endl;
}
else
{
cout << "Wait for single object failed. Error code: " << returnValue << endl;
}

Related

Win32API ReadFile Call worked and now never returns after call

When the code executes the ReadFile the call never returns. Previously the way I saved the Handle and passed it from function to function was not properly done. I changed the way the handle was stored and it works with all of the other calls in the program except the read. I have looked and compared all of the variables used and they check out.
Here is the code. instance->Master has the handle from the create. I added a GetCommState call before the read and it executes correctly.
'''
ATCA_STATUS swi_silab_receive_byte(ATCASWIMaster_t* instance, uint8_t* data)
{
uint8_t retries = 3;
DWORD NoBytesRead = 0; // Bytes read by ReadFile()
uint8_t SerialBuffer; //Buffer to send and receive data
DWORD Byte_count = (DWORD)sizeof(SerialBuffer);
ATCA_STATUS status;
while ((retries > 0) && (NoBytesRead < 1))
{
//Read data and store in a buffer
status = GetCommState(instance->hMaster, &instance->dcbMaster);
if (status == 0)
printf(" init GetcommState failed\n");
status = ReadFile(instance->hMaster, &SerialBuffer, Byte_count, &NoBytesRead, NULL);
retries--;
}
if (status == FALSE)
{
printf_s("\nError! in ReadFile()\n\n");
return ATCA_TIMEOUT;
}
else
{
printf("Read Success Serial Buffer = %x\n", &SerialBuffer);
*data = SerialBuffer;
//printf("Read Success Data = %x\n", *data);
return ATCA_SUCCESS;
}
}
'''
I am happy to state there is no issue in the code. Thanks to Zhu Song who made a comment about reading the ReadFile remarks. If there is no data to read then readfile will just wait. A check with the logic analyzer showed the write executes but doesn't actually write, hence the read is not actually able to read.
Thanks to everyone who commented
According to ReadFile:
The ReadFile function returns when one of the following conditions occur:
The number of bytes requested is read.
A write operation completes on the write end of the pipe.
An asynchronous handle is being used and the read is occurring asynchronously.
An error occurs.
To cancel all pending asynchronous I/O operations, use either:
CancelIo—this function only cancels operations issued by the calling thread for the specified file handle.
CancelIoEx—this function cancels all operations issued by the threads for the specified file handle.

How to CancelSynchronousIo() on WaitForSingleObject() waiting on stdin?

On Windows 10, I'm waiting for input from the console using
WaitForSingleObject( GetStdHandle(STD_INPUT_HANDLE), ... )
and to cancel this waiting using CancelSynchronousIo().
But the cancellation does nothing (returns 0 and GetLastError() is ERROR_NOT_FOUND).
Any idea what I could be doing wrong?
Should I be able to cancel this waiting for new input on stdin?
(I actually want to do this with any HANDLE whose GetFileType() is FILE_TYPE_CHAR, not only stdin, but stdin is certainly the most important use case and the simplest to test with).
Related discussions I've found:
Synchronous ReadFile() on stdin cannot be unblocked by CancelSynchronousIo()
win32: how stop ReadFile (stdin|pipe)
But unfortunately they only discuss ReadFile(), not WaitForSingleObject(). I've also tried WaitForMultipleObjects() (with just a single object in the array), same problem.
(Background: I'm trying to improve input handling in the GHC Haskell compiler runtime.)
CancelSynchronousIo cancel I/O operations that are issued by the specified thread. more concrete it cancel IRP packets which associated with specified thread via call IoCancelIrp. if use undocumented NtCancelSynchronousIoFile (CancelSynchronousIo internally call it with IoRequestToCancel = 0) we can be more selective - cancel only i/o request which used specified IoRequestToCancel (system check that Irp->UserIosb == IoRequestToCancel and cancel only this requests)
but WaitForSingleObject this is not I/O request. this call not create any IRP which can be canceled. so - no way do this.
however if you use WaitForSingleObjectEx with bAlertable set to TRUE - you can break wait by queue apc to thread by using QueueUserAPC . also if use NtWaitForSingleObject instead WaitForSingleObjectEx we can also alert thread by using undocumented call NtAlertThread. in this case NtWaitForSingleObject will break with STATUS_ALERTED (note that WaitForSingleObjectEx which internally call NtWaitForSingleObject do special check for STATUS_ALERTED and in case this status - again run NtWaitForSingleObject - as result we can not break WaitForSingleObjectEx by call NtAlertThread, but NtWaitForSingleObject will be breaked.
so if you need break waiting for std input - create additional thread, which must call not CancelSynchronousIo (this senseless) but QueueUserAPC or NtAlertThread (only if you use NtWaitForSingleObject for wait). and input thread must wait in alertable state. so demo code can look like:
extern "C" NTSYSCALLAPI NTSTATUS NTAPI NtAlertThread(HANDLE ThreadHandle);
VOID NTAPI OnApc(ULONG_PTR Parameter)
{
DbgPrint("OnApc(%p)\n", Parameter);
}
DWORD CALLBACK BreakWaitThread(HANDLE hThread)
{
switch (LONG status = MessageBoxW(0, L"Use Apc(yes) or Alert(No) ?", L"BreakWaitThread",
MB_ICONQUESTION|MB_YESNOCANCEL|MB_DEFBUTTON3))
{
case IDYES:
if (!QueueUserAPC(OnApc, hThread, 0))
{
DbgPrint("QueueUserAPC=%u\n", GetLastError());
}
break;
case IDNO:
if (0 > (status = NtAlertThread(hThread)))
{
DbgPrint("AlertThread=%x\n", status);
}
break;
case IDCANCEL:
DbgPrint("canceled\n");
break;
default:
DbgPrint("MessageBox=%x\n", status);
}
CloseHandle(hThread);
return 0;
}
void ConsoleLoop(HANDLE hStdIn)
{
ULONG NumberOfEvents, NumberOfEventsRead, n;
INPUT_RECORD buf[8], *p;
for (;;)
{
switch (ZwWaitForSingleObject(hStdIn, TRUE, 0))
//switch (WaitForSingleObjectEx(hStdIn, INFINITE, TRUE))
{
case WAIT_OBJECT_0:
while (GetNumberOfConsoleInputEvents(hStdIn, &NumberOfEvents) && NumberOfEvents)
{
do
{
NumberOfEventsRead = min(RTL_NUMBER_OF(buf), NumberOfEvents);
if (ReadConsoleInput(hStdIn, buf, NumberOfEventsRead, &NumberOfEventsRead) && NumberOfEventsRead)
{
n = NumberOfEventsRead;
p = buf;
do
{
if (p->EventType == KEY_EVENT)
{
DbgPrint("%u(%u) %C %x %x %x\n",
p->Event.KeyEvent.bKeyDown,
p->Event.KeyEvent.wRepeatCount,
p->Event.KeyEvent.uChar.UnicodeChar,
p->Event.KeyEvent.wVirtualKeyCode,
p->Event.KeyEvent.wVirtualScanCode,
p->Event.KeyEvent.dwControlKeyState);
if (VK_OEM_PERIOD == p->Event.KeyEvent.wVirtualKeyCode)
{
return ;//if user type '.' return for demo
}
}
} while (p++, --n);
}
else
{
FlushConsoleInputBuffer(hStdIn);
break;
}
} while (NumberOfEvents -= NumberOfEventsRead);
}
continue;
case STATUS_USER_APC:
DbgPrint("\nUSER_APC\n");
return;
case STATUS_ALERTED:
DbgPrint("\nALERTED\n");
return;
case WAIT_FAILED :
DbgPrint("\nWAIT_FAILED=%u\n", GetLastError());
return;
default:
__debugbreak();
return;
}
}
}
void SimpleDemo()
{
if (HANDLE hCurrentThread = OpenThread(THREAD_ALERT|THREAD_SET_CONTEXT , FALSE, GetCurrentThreadId()))
{
ULONG dwThreadId;
HANDLE hThread = CreateThread(0, 0, BreakWaitThread, hCurrentThread, 0, &dwThreadId);
if (hThread)
{
ConsoleLoop(GetStdHandle(STD_INPUT_HANDLE));
PostThreadMessage(dwThreadId, WM_QUIT, 0, 0);
WaitForSingleObject(hThread, INFINITE);
CloseHandle(hThread);
}
else
{
CloseHandle(hCurrentThread);
}
}
}
Console I/O is difficult to use asynchronously, it is simply not designed for it. See IO Completion Ports (IOCP) and Asynchronous I/O through STDIN, STDOUT and STDERR for some possible workarounds.
If that is not an option for you, then you will have to either:
use WaitForSingleObject() in a loop with a short timeout. Create a flag variable that your loop can look at on each iteration to break the loop if the flag is set.
use WaitForMutipleObjects(), giving it 2 HANDLEs to wait on - one for the console (or whatever), and one for an event object from CreateEvent(). Then you can signal the event with SetEvent() when you want to break the wait. The return value of WaitForMutipleObjects() will tell you which HANDLE was signaled.

Using IRPs for I/O on device object returned by IoGetDeviceObjectPointer()

Can one use IoCallDriver() with an IRP created by IoBuildAsynchronousFsdRequest() on a device object returned by IoGetDeviceObjectPointer()? What I have currently fails with blue screen (BSOD) 0x7E (unhandled exception), which when caught shows an Access Violation (0xc0000005). Same code worked when the device was stacked (using the device object returned by IoAttachDeviceToDeviceStack()).
So what I have is about the following:
status = IoGetDeviceObjectPointer(&device_name, FILE_ALL_ACCESS, &FileObject, &windows_device);
if (!NT_SUCCESS(status)) {
return -1;
}
offset.QuadPart = 0;
newIrp = IoBuildAsynchronousFsdRequest(io, windows_device, buffer, 4096, &offset, &io_stat);
if (newIrp == NULL) {
return -1;
}
IoSetCompletionRoutine(newIrp, DrbdIoCompletion, bio, TRUE, TRUE, TRUE);
status = ObReferenceObjectByPointer(newIrp->Tail.Overlay.Thread, THREAD_ALL_ACCESS, NULL, KernelMode);
if (!NT_SUCCESS(status)) {
return -1;
}
status = IoCallDriver(bio->bi_bdev->windows_device, newIrp);
if (!NT_SUCCESS(status)) {
return -1;
}
return 0;
device_name is \Device\HarddiskVolume7 which exists according to WinObj.exe .
buffer has enough space and is read/writable. offset and io_stat are on stack (also tried with heap, didn't help). When catching the exception (SEH exception) it doesn't blue screen but shows an access violation as reason for the exception. io is IRP_MJ_READ.
Do I miss something obvious? Is it in general better to use IRPs than the ZwCreateFile / ZwReadFile / ZwWriteFile API (which would be an option, but isn't that slower?)? I also tried a ZwCreateFile to have an extra reference, but this also didn't help.
Thanks for any insights.
you make in this code how minimum 2 critical errors.
can I ask - from which file you try read (or write) data ? from
FileObject you say ? but how file system driver, which will handle
this request know this ? you not pass any file object to newIrp.
look for IoBuildAsynchronousFsdRequest - it have no file object
parameter (and impossible get file object from device object - only
visa versa - because on device can be multiple files open). so it
and can not be filled by this api in newIrp. you must setup it
yourself:
PIO_STACK_LOCATION irpSp = IoGetNextIrpStackLocation( newIrp );
irpSp->FileObject = FileObject;
I guess bug was exactly when file system try access FileObject
from irp which is 0 in your case. also read docs for
IRP_MJ_READ - IrpSp->FileObject -
Pointer to the file object that is associated with DeviceObject
you pass I guess local variables io_stat (and offset) to
IoBuildAsynchronousFsdRequest. as result io_stat must be valid
until newIrp is completed - I/O subsystem write final result to it
when operation completed. but you not wait in function until request
will be completed (in case STATUS_PENDING returned) but just exit
from function. as result later I/O subsystem, if operation completed
asynchronous, write data to arbitrary address &io_stat (it became
arbitrary just after you exit from function). so you need or check
for STATUS_PENDING returned and wait in this case (have actually
synchronous io request). but more logical use
IoBuildSynchronousFsdRequest in this case. or allocate io_stat
not from stack, but say in your object which correspond to file. in
this case you can not have more than single io request with this
object at time. or if you want exactly asynchronous I/O - you can do
next trick - newIrp->UserIosb = &newIrp->IoStatus. as result you
iosb always will be valid for newIrp. and actual operation status
you check/use in DrbdIoCompletion
also can you explain (not for me - for self) next code line ?:
status = ObReferenceObjectByPointer(newIrp->Tail.Overlay.Thread, THREAD_ALL_ACCESS, NULL, KernelMode);
who and where dereference thread and what sense in this ?
Can one use ...
we can use all, but with condition - we understand what we doing and deep understand system internally.
Is it in general better to use IRPs than the ZwCreateFile / ZwReadFile
/ ZwWriteFile API
for performance - yes, better. but this require more code and more complex code compare api calls. and require more knowledge. also if you know that previous mode is kernel mode - you can use NtCreateFile, NtWriteFile, NtReadFile - this of course will be bit slow (need every time reference file object by handle) but more faster compare Zw version
Just wanted to add that the ObReferenceObjectByPointer is needed
because the IRP references the current thread which may exit before
the request is completed. It is dereferenced in the Completion
Routine. Also as a hint the completion routine must return
STATUS_MORE_PROCESSING_REQUIRED if it frees the IRP (took me several
days to figure that out).
here you make again several mistakes. how i understand you in completion routine do next:
IoFreeIrp(Irp);
return StopCompletion;
but call simply call IoFreeIrp here is error - resource leak. i advice you check (DbgPrint) Irp->MdlAddress at this point. if you read data from file system object and request completed asynchronous - file system always allocate Mdl for access user buffer in arbitrary context. now question - who free this Mdl ? IoFreeIrp - simply free Irp memory - nothing more. you do this yourself ? doubt. but Irp is complex object, which internally hold many resources. as result need not only free it memory but call "destructor" for it. this "destructor" is IofCompleteRequest. when you return StopCompletion (=STATUS_MORE_PROCESSING_REQUIRED) you break this destructor at very begin. but you must latter again call IofCompleteRequest for continue Irp (and it resources) correct destroy.
about referencing Tail.Overlay.Thread - what you doing - have no sense:
It is dereferenced in the Completion Routine.
but IofCompleteRequest access Tail.Overlay.Thread after it
call your completion routine (and if you not return
StopCompletion). as result your reference/dereference thread lost
sense - because you deference it too early, before system
actually access it.
also if you return StopCompletion and not more call
IofCompleteRequest for this Irp - system not access
Tail.Overlay.Thread at all. and you not need reference it in this
case.
and exist else one reason, why reference thread is senseless. system
access Tail.Overlay.Thread only for insert Apc to him - for call
final part (IopCompleteRequest) of Irp destruction in original
thread context. really this need only for user mode Irp's requests,
where buffers and iosb located in user mode and valid only in
context of process (original thread ). but if thread is terminated -
call of KeInsertQueueApc fail - system not let insert apc to
died thread. as result IopCompleteRequest will be not called and
resources not freed.
so you or dereference Tail.Overlay.Thread too early or you not need do this at all. and reference for died thread anyway not help. in all case what you doing is error.
you can try do next here:
PETHREAD Thread = Irp->Tail.Overlay.Thread;
IofCompleteRequest(Irp, IO_NO_INCREMENT);// here Thread will be referenced
ObfDereferenceObject(Thread);
return StopCompletion;
A second call to IofCompleteRequest causes the I/O manager to resume calling the IRP's completion. here io manager and access Tail.Overlay.Thread insert Apc to him. and finally you call ObfDereferenceObject(Thread); already after system access it and return StopCompletion for break first call to IofCompleteRequest. look like correct but.. if thread already terminated, how i explain in 3 this will be error, because KeInsertQueueApc fail. for extended test - call IofCallDriver from separate thread and just exit from it. and in completion run next code:
PETHREAD Thread = Irp->Tail.Overlay.Thread;
if (PsIsThreadTerminating(Thread))
{
DbgPrint("ThreadTerminating\n");
if (PKAPC Apc = (PKAPC)ExAllocatePool(NonPagedPool, sizeof(KAPC)))
{
KeInitializeApc(Apc, Thread, 0, KernelRoutine, 0, 0, KernelMode, 0);
if (!KeInsertQueueApc(Apc, 0, 0, IO_NO_INCREMENT))
{
DbgPrint("!KeInsertQueueApc\n");
ExFreePool(Apc);
}
}
}
PMDL MdlAddress = Irp->MdlAddress;
IofCompleteRequest(Irp, IO_NO_INCREMENT);
ObfDereferenceObject(Thread);
if (MdlAddress == Irp->MdlAddress)
{
// IopCompleteRequest not called due KeInsertQueueApc fail
DbgPrint("!!!!!!!!!!!\n");
IoFreeMdl(MdlAddress);
IoFreeIrp(Irp);
}
return StopCompletion;
//---------------
VOID KernelRoutine (PKAPC Apc,PKNORMAL_ROUTINE *,PVOID *,PVOID *,PVOID *)
{
DbgPrint("KernelRoutine(%p)\n", Apc);
ExFreePool(Apc);
}
and you must got next debug output:
ThreadTerminating
!KeInsertQueueApc
!!!!!!!!!!!
and KernelRoutine will be not called (like and IopCompleteRequest) - no print from it.
so what is correct solution ? this of course not documented anywhere, but based on deep internal understand. you not need reference original thread. you need do next:
Irp->Tail.Overlay.Thread = KeGetCurrentThread();
return ContinueCompletion;
you can safe change Tail.Overlay.Thread - if you have no any pointers valid only in original process context. this is true for kernel mode requests - all your buffers in kernel mode and valid in any context. and of course you not need break Irp destruction but continue it. for correct free mdl and all irp resources. and finally system call IoFreeIrp for you.
and again for iosb pointer. how i say pass local variable address, if you exit from function before irp completed (and this iosb accessed) is error. if you break Irp destruction, iosb will be not accessed of course, but in this case much better pass 0 pointer as iosb. (if you latter something change and iosb pointer will be accessed - will be the worst error - arbitrary memory corrupted - with unpredictable effect. and research crash of this will be very-very hard). but if you completion routine - you not need separate iosb at all - you have irp in completion and can direct access it internal iosb - for what you need else one ? so the best solution will be do next:
Irp->UserIosb = &Irp->IoStatus;
full correct example how read file asynchronous:
NTSTATUS DemoCompletion (PDEVICE_OBJECT /*DeviceObject*/, PIRP Irp, BIO* bio)
{
DbgPrint("DemoCompletion(p=%x mdl=%p)\n", Irp->PendingReturned, Irp->MdlAddress);
bio->CheckResult(Irp->IoStatus.Status, Irp->IoStatus.Information);
bio->Release();
Irp->Tail.Overlay.Thread = KeGetCurrentThread();
return ContinueCompletion;
}
VOID DoTest (PVOID buf)
{
PFILE_OBJECT FileObject;
NTSTATUS status;
UNICODE_STRING ObjectName = RTL_CONSTANT_STRING(L"\\Device\\HarddiskVolume2");
OBJECT_ATTRIBUTES oa = { sizeof(oa), 0, &ObjectName, OBJ_CASE_INSENSITIVE };
if (0 <= (status = GetDeviceObjectPointer(&oa, &FileObject)))
{
status = STATUS_INSUFFICIENT_RESOURCES;
if (BIO* bio = new BIO(FileObject))
{
if (buf = bio->AllocBuffer(PAGE_SIZE))
{
LARGE_INTEGER ByteOffset = {};
PDEVICE_OBJECT DeviceObject = IoGetRelatedDeviceObject(FileObject);
if (PIRP Irp = IoBuildAsynchronousFsdRequest(IRP_MJ_READ, DeviceObject, buf, PAGE_SIZE, &ByteOffset, 0))
{
Irp->UserIosb = &Irp->IoStatus;
Irp->Tail.Overlay.Thread = 0;
PIO_STACK_LOCATION IrpSp = IoGetNextIrpStackLocation(Irp);
IrpSp->FileObject = FileObject;
bio->AddRef();
IrpSp->CompletionRoutine = (PIO_COMPLETION_ROUTINE)DemoCompletion;
IrpSp->Context = bio;
IrpSp->Control = SL_INVOKE_ON_CANCEL|SL_INVOKE_ON_ERROR|SL_INVOKE_ON_SUCCESS;
status = IofCallDriver(DeviceObject, Irp);
}
}
bio->Release();
}
ObfDereferenceObject(FileObject);
}
DbgPrint("DoTest=%x\n", status);
}
struct BIO
{
PVOID Buffer;
PFILE_OBJECT FileObject;
LONG dwRef;
void AddRef()
{
InterlockedIncrement(&dwRef);
}
void Release()
{
if (!InterlockedDecrement(&dwRef))
{
delete this;
}
}
void* operator new(size_t cb)
{
return ExAllocatePool(PagedPool, cb);
}
void operator delete(void* p)
{
ExFreePool(p);
}
BIO(PFILE_OBJECT FileObject) : FileObject(FileObject), Buffer(0), dwRef(1)
{
DbgPrint("%s<%p>(%p)\n", __FUNCTION__, this, FileObject);
ObfReferenceObject(FileObject);
}
~BIO()
{
if (Buffer)
{
ExFreePool(Buffer);
}
ObfDereferenceObject(FileObject);
DbgPrint("%s<%p>(%p)\n", __FUNCTION__, this, FileObject);
}
PVOID AllocBuffer(ULONG NumberOfBytes)
{
return Buffer = ExAllocatePool(PagedPool, NumberOfBytes);
}
void CheckResult(NTSTATUS status, ULONG_PTR Information)
{
DbgPrint("CheckResult:status = %x, info = %p\n", status, Information);
if (0 <= status)
{
if (ULONG_PTR cb = min(16, Information))
{
char buf[64], *sz = buf;
PBYTE pb = (PBYTE)Buffer;
do sz += sprintf(sz, "%02x ", *pb++); while (--cb); sz[-1]= '\n';
DbgPrint(buf);
}
}
}
};
NTSTATUS GetDeviceObjectPointer(POBJECT_ATTRIBUTES poa, PFILE_OBJECT *FileObject )
{
HANDLE hFile;
IO_STATUS_BLOCK iosb;
NTSTATUS status = IoCreateFile(&hFile, FILE_READ_DATA, poa, &iosb, 0, 0,
FILE_SHARE_VALID_FLAGS, FILE_OPEN, FILE_NO_INTERMEDIATE_BUFFERING, 0, 0, CreateFileTypeNone, 0, 0);
if (0 <= (status))
{
status = ObReferenceObjectByHandle(hFile, 0, *IoFileObjectType, KernelMode, (void**)FileObject, 0);
NtClose(hFile);
}
return status;
}
and output:
BIO::BIO<FFFFC000024D4870>(FFFFE00001BAAB70)
DoTest=103
DemoCompletion(p=1 mdl=FFFFE0000200EE70)
CheckResult:status = 0, info = 0000000000001000
eb 52 90 4e 54 46 53 20 20 20 20 00 02 08 00 00
BIO::~BIO<FFFFC000024D4870>(FFFFE00001BAAB70)
the eb 52 90 4e 54 46 53 read ok

Callback passed to boost::asio::async_read_some never invoked in usage where boost::asio::read_some returns data

I have been working on implementing a half duplex serial driver by learning from a basic serial terminal example using boost::asio::basic_serial_port:
http://lists.boost.org/boost-users/att-41140/minicom.cpp
I need to read asynchronously but still detect when the handler is finished in the main thread so I pass async_read_some a callback with several additional reference parameters in a lambda function using boost:bind. The handler never gets invoked but if I replace the async_read_some function with the read_some function it returns data without an issue.
I believe I'm satisfying all of the necessary requirements for this function to invoke the handler because they are the same for the asio::read some function which returns:
The buffer stays in scope
One or more bytes is received by the serial device
The io service is running
The port is open and running at the correct baud rate
Does anyone know if I'm missing another assumption unique to the asynchronous read or if I'm not setting up the io_service correctly?
Here is an example of how I'm using the code with async_read_some (http://www.boost.org/doc/libs/1_56_0/doc/html/boost_asio/reference/basic_serial_port/async_read_some.html):
void readCallback(const boost::system::error_code& error, size_t bytes_transfered, bool & finished_reading, boost::system::error_code& error_report, size_t & bytes_read)
{
std::cout << "READ CALLBACK\n";
std::cout.flush();
error_report = error;
bytes_read = bytes_transfered;
finished_reading = true;
return;
}
int main()
{
int baud_rate = 115200;
std::string port_name = "/dev/ttyUSB0";
boost::asio::io_service io_service_;
boost::asio::serial_port serial_port_(io_service_,port_name);
serial_port_.set_option(boost::asio::serial_port_base::baud_rate(baud_rate));
boost::thread service_thread_;
service_thread = boost::thread(boost::bind(&boost::asio::io_service::run, &io_service_));
std::cout << "Starting byte read\n";
boost::system::error_code ec;
bool finished_reading = false;
size_t bytes_read;
int max_response_size = 8;
uint8_t read_buffer[max_response_size];
serial_port_.async_read_some(boost::asio::buffer(read_buffer, max_response_size),
boost::bind(readCallback,
boost::asio::placeholders::error,
boost::asio::placeholders::bytes_transferred,
finished_reading, ec, bytes_read));
std::cout << "Waiting for read to finish\n";
while (!finished_reading)
{
boost::this_thread::sleep(boost::posix_time::milliseconds(1));
}
std::cout << "Finished byte read: " << bytes_read << "\n";
for (int i = 0; i < bytes_read; ++i)
{
printf("0x%x ",read_buffer[i]);
}
}
The result is that the callback does not print out anything and the while !finished loop never finishes.
Here is how I use the blocking read_some function (boost.org/doc/libs/1_56_0/doc/html/boost_asio/reference/basic_serial_port/read_some.html):
int main()
{
int baud_rate = 115200;
std::string port_name = "/dev/ttyUSB0";
boost::asio::io_service io_service_;
boost::asio::serial_port serial_port_(io_service_,port_name);
serial_port_.set_option(boost::asio::serial_port_base::baud_rate(baud_rate));
boost::thread service_thread_;
service_thread = boost::thread(boost::bind(&boost::asio::io_service::run, &io_service_));
std::cout << "Starting byte read\n";
boost::system::error_code ec;
int max_response_size = 8;
uint8_t read_buffer[max_response_size];
int bytes_read = serial_port_.read_some(boost::asio::buffer(read_buffer, max_response_size),ec);
std::cout << "Finished byte read: " << bytes_read << "\n";
for (int i = 0; i < bytes_read; ++i)
{
printf("0x%x ",read_buffer[i]);
}
}
This version prints from 1 up to 8 characters that I send, blocking until at least one is sent.
The code does not guarantee that the io_service is running. io_service::run() will return when either:
All work has finished and there are no more handlers to be dispatched
The io_service has been stopped.
In this case, it is possible for the service_thread_ to be created and invoke io_service::run() before the serial_port::async_read_some() operation is initiated, adding work to the io_service. Thus, the service_thread_ could immediately return from io_service::run(). To resolve this, either:
Invoke io_service::run() after the asynchronous operation has been initiated.
Create a io_service::work object before starting the service_thread_. A work object prevents the io_service from running out of work.
This answer may provide some more insight into the behavior of io_service::run().
A few other things to note and to expand upon Igor's answer:
If a thread is not progressing in a meaningful way while waiting for an asynchronous operation to complete (i.e. spinning in a loop sleeping), then it may be worth examining if mixing synchronous behavior with asynchronous operations is the correct solution.
boost::bind() copies its arguments by value. To pass an argument by reference, wrap it with boost::ref() or boost::cref():
boost::bind(..., boost::ref(finished_reading), boost::ref(ec),
boost::ref(bytes_read));
Synchronization needs to be added to guarantee memory visibility of finished_reading in the main thread. For asynchronous operations, Boost.Asio will guarantee the appropriate memory barriers to ensure correct memory visibility (see this answer for more details). In this case, a memory barrier is required within the main thread to guarantee the main thread observes changes to finished_reading by other threads. Consider using either a Boost.Thread synchronization mechanism like boost::mutex, or Boost.Atomic's atomic objects or thread and signal fences.
Note that boost::bind copies its arguments. If you want to pass an argument by reference, wrap it with boost::ref (or std::ref):
boost::bind(readCallback, boost::asio::placeholders::error, boost::asio::placeholders::bytes_transferred, boost::ref(finished_reading), ec, bytes_read));
(However, strictly speaking, there's a race condition on the bool variable you pass to another thread. A better solution would be to use std::atomic_bool.)

Events/Interrupts in Serial Communication

I want to read and write from serial using events/interrupts.
Currently, I have it in a while loop and it continuously reads and writes through the serial. I want it to only read when something comes from the serial port. How do I implement this in C++?
This is my current code:
while(true)
{
//read
if(!ReadFile(hSerial, szBuff, n, &dwBytesRead, NULL)){
//error occurred. Report to user.
}
//write
if(!WriteFile(hSerial, szBuff, n, &dwBytesRead, NULL)){
//error occurred. Report to user.
}
//print what you are reading
printf("%s\n", szBuff);
}
Use a select statement, which will check the read and write buffers without blocking and return their status, so you only need to read when you know the port has data, or write when you know there's room in the output buffer.
The third example at http://www.developerweb.net/forum/showthread.php?t=2933 and the associated comments may be helpful.
Edit: The man page for select has a simpler and more complete example near the end. You can find it at http://linux.die.net/man/2/select if man 2 select doesn't work on your system.
Note: Mastering select() will allow you to work with both serial ports and sockets; it's at the heart of many network clients and servers.
For a Windows environment the more native approach would be to use asynchronous I/O. In this mode you still use calls to ReadFile and WriteFile, but instead of blocking you pass in a callback function that will be invoked when the operation completes.
It is fairly tricky to get all the details right though.
Here is a copy of an article that was published in the c/C++ users journal a few years ago. It goes into detail on the Win32 API.
here a code that read serial incomming data using interruption on windows
you can see the time elapsed during the waiting interruption time
int pollComport(int comport_number, LPBYTE buffer, int size)
{
BYTE Byte;
DWORD dwBytesTransferred;
DWORD dwCommModemStatus;
int n;
double TimeA,TimeB;
// Specify a set of events to be monitored for the port.
SetCommMask (m_comPortHandle[comport_number], EV_RXCHAR );
while (m_comPortHandle[comport_number] != INVALID_HANDLE_VALUE)
{
// Wait for an event to occur for the port.
TimeA = clock();
WaitCommEvent (m_comPortHandle[comport_number], &dwCommModemStatus, 0);
TimeB = clock();
if(TimeB-TimeA>0)
cout <<" ok "<<TimeB-TimeA<<endl;
// Re-specify the set of events to be monitored for the port.
SetCommMask (m_comPortHandle[comport_number], EV_RXCHAR);
if (dwCommModemStatus & EV_RXCHAR)
{
// Loop for waiting for the data.
do
{
ReadFile(m_comPortHandle[comport_number], buffer, size, (LPDWORD)((void *)&n), NULL);
// Display the data read.
if (n>0)
cout << buffer <<endl;
} while (n > 0);
}
return(0);
}
}

Resources