I'm working on a Qt native messaging host for a chrome extension on windows.
The circumstances seem to be as follows:
There's no way to tell, before attempting to read, whether data is available or how much data is available to be read on stdin.
Making a read call, e.g. via a QFile that has opened the stdin handle, will block forever until enough data is written to stdin to satisfy it.
So it looks like this will involve a thread resigned to staying in these blocking read calls waiting for data to be written. The only option I can see for ending this operation is to call terminate() on the QThread, which is highly discouraged. Is there a better way of interrupting an endlessly blocking call? What are the consequences of terminating the thread - does the object that was running on it still exist, and would I be able to salvage it to perform cleanup?
Related
I'm building a Qt windows application that needs to react to data being written to its stdin.
I realise I could use blocking threads or polling timers here, but it seems like this should be doable using QWinEventNotifier.
However, if I initialise a notifier as notifier_ = new QWinEventNotifier(GetStdHandle(STD_INPUT_HANDLE));, this notifier seems to emit activated constantly, regardless of whether or not there is any data available to be read.
What am I missing? Do I need to manually reset the event somehow? Is there a different handle I should be using?
The consequence here is that when I then try to read from stdin when this signal is emitted, the calls block forever - the stdin stream is still open, but not enough data is available to be read, so the calls don't return until more is written. Does Qt provide any tools that would let me avoid getting stuck like this without resorting to messily terminating threads?
In KEXT, I am listening for file close via vnode or file scope listener. For certain (very few) files, I need to send file path to my system daemon which does some processing (this has to happen in daemon) and returns the result back to KEXT. The file close call needs to be blocked until I get response from daemon. Based on result I need to some operation in close call and return close call successfully. There is lot of discussion on KEXT communication related topic on the forum. But they are not conclusive and appears be very old (year 2002 around). This requirement can be handled by FtlSendMessage(...) Win32 API. I am looking for equivalent of that on Mac
Here is what I have looked at and want to summarize my understanding:
Mach message: Provides very good way of bidirectional communication using sender and reply ports with queueing mechansim. However, the mach message APIs (e.g. mach_msg, mach_port_allocate, bootstrap_look_up) don't appear to be KPIs. The mach API mach_msg_send_from_kernel can be used, but that alone will not help in bidirectional communication. Is my understanding right?
IOUserClient: This appears be more to do with communicating from User space to KEXT and then having some callbacks from KEXT. I did not find a way to initiate communication from KEXT to daemon and then wait for result from daemon. Am I missing something?
Sockets: This could be last option since I would have to implement entire bidirectional communication channel from KEXT to Daemon.
ioctl/sysctl: I don't know much about them. From what I have read, its not recommended option especially for bidirectional communication
RPC-Mig: Again I don't know much about them. Looks complicated from what I have seen. Not sure if this is recommended way.
KUNCUserNotification: This appears to be just providing notification to the user from KEXT. It does not meet my requirement.
Supported platform is (10.5 onwards). So looking at the requirement, can someone suggest and provide some pointers on this topic?
Thanks in advance.
The pattern I've used for that process is to have the user-space process initiate a socket connection to the KEXT; the KEXT creates a new thread to handle messages over that socket and sleeps the thread. When the KEXT detects an event it needs to respond to, it wakes the messaging thread and uses the existing socket to send data to the daemon. On receiving a response, control is passed back to the requesting thread to decide whether to veto the operation.
I don't know of any single resource that will describe that whole pattern completely, but the relevant KPIs are discussed in Mac OS X Internals (which seems old, but the KPIs haven't changed much since it was written) and OS X and iOS Kernel Programming (which I was a tech reviewer on).
For what it's worth, autofs uses what I assume you mean by "RPC-Mig", so it's not too complicated (MIG is used to describe the RPC calls, and the stub code it generates handles calling the appropriate Mach-message sending and receiving code; there are special options to generate kernel-mode stubs).
However, it doesn't need to do any lookups, as automountd (the user-mode daemon to which the autofs kext sends messages) has a "host special port" assigned to it. Doing the lookups to find an arbitrary service would be harder.
If you want to use the socket established with ctl_register() on the KExt side, then beware: The communication from kext to user space (via ctl_enqueuedata()) works OK. However opposite direction is buggy on 10.5.x and 10.6.x.
After about 70.000 or 80.000 send() calls with SOCK_DGRAM in the PF_SYSTEM domain complete net stack breaks with disastrous consequences for complete system (hard turning off is the only way out). This has been fixed in 10.7.0. I workaround by using setsockopt() in our project for the direction from user space to kext as we only send very small data (just to allow/disallow some operation).
I'm pulling my hair trying to figure out when a serial port finishes closing so I can reopen it. It turns out that CloseHandle() returns before the port is actually unlocked.
I am opening a serial port using CreateFile(FILE_FLAG_OVERLAPPED), associating it with a CompletionPort using CreateIoCompletionPort(), reading/writing to it using ReadFile(), WriteFile() and closing it using CloseHandle().
I've noticed that if I close and reopen a serial port quickly enough, I get an ERROR_ACCESS_DENIED back from CreateFile(). This is happening in spite of the fact that I'm waiting for CloseHandle() to return, then waiting for all outstanding read/write operations associated with that handle to come back from the completion port. Surely there is a better way :)
How do I close a serial port synchronously? Please no retry loops, sleep() or some other cheap hacks.
EDIT: Maybe this has something to do with my use of completion ports and FILE_FLAG_OVERLAPPED. I get a callback when the read/write operations complete. Is there some sort of callback for the port closing?
I believe the problem is with the driver that serves the COM port. Hence - there'll be no API to "actually close" the COM port.
BTW, after you close the file handle, there's no need to wait for all oustanding I/Os to complete with errors. At the time CloseHandle returns all outstanding I/Os are already completed/cancelled, you just receive the callbacks asynchronously (be it via completion port or APC queue, no matter).
Specifically FTDI drivers (those that emulate COM->USB) are known to be very glitchy.
I may only recommend to try flushing the data before closing the handle. You may wait for all I/Os to complete before closing the COM port (if this is applicable for your case). Alternatively you may call SetCommMask and WaitCommEvent to ensure there's no send data pending. Hopefully this may help.
EDIT:
Does CloseHandle immediately (before it returns) cancel all the pending I/Os on the file handle?
Strictly speaking - no.
There may be other references left to the file object. For example, a user-mode code may call DuplicateHandle, or a kernel-mode driver may call ObReferenceObjectByXXXX. In such a case the object the handle refers to is not necessarily released.
When the last handle is closed the driver's DispatchCleanup is called. It must cancel all the outstanding I/Os according to this.
However while one thread is within a CloseHandle - you may theoretically issue another I/O from another thread (if you're lucky - the handle will still be valid). While you're in the call to an I/O function (such as WriteFile or etc.) the OS temporarily increases the reference counter to the object. This in turn may start another I/O, which won't be cancelled directly by the CloseHandle invocation.
However in this case the handle will be closed by the O/S immediately after the new I/O is issued, because the reference count to the object reached 0 again.
So that in such a perverted scenario there may happen a situation where immediately after a call to CloseHandle you're unable to reopen the file again.
Ensure that your reader and writer fail (with invalid handle) after you have issued the CloseHandle() request (presumably on some other thread) before attempting to touch the device again. You can use this as a cue to clean up all your io handling before attempting to issue another CreateFile. I must say completion ports do seem needlessly baroque.
When you open a COM Port in a thread, the operating system opens another hidden thread to do the I/O. I am trying to resolve closing a port as well. Near as I can tell one method which may work is: 1) Suspend the thread which opened the COM Port, 2) PurgeComm() to stop all I/O and empty all read/write I/O buffers, and 3) then attempt to close the COM Port. There may be a waitforsingleobject involved to determine when the thread with the COM Port actually suspends. Unless the COM Port absolutely needs to be closed, I am considering leaving the COM Port open and letting the WinProc process the IDM_Exit command to close the COM Port, threads, and the application. Not what I wanted but an option I could live with. I am using a USB to Serial Port connection and not sure what problems this is causing me. I know if you are using an USB to Serial Port interface you need to set pin 20 high (DTR). Pin 20 helps power the interface, but is only providing around 30ma or so. If all this works I will make an update to this post and let you know how if I was able to close the COM Port. Windows has made a mess of the Serial Port driver and seem to be content to leave things in disarray!
I am trying to handle SSL error scenarios where, for example, SSL async_handshake() is taking too long.
After some time (say 20sec) i want to close this connection (lowest_layer().close()).
I pass shared_ptr with connection object as a parameter to async_handshake(), so object still exists, eventually handshake handler is invoked and object gets destroyed.
But, still I'm getting sporadic crashes! Looks like after close() SSL is still trying to read or operate on read buffer.
So, the basic question - is it safe to hard close() SSL connection?
Any ideas?
Typically the method I've used stop outstanding asynchronous operations on a socket is socket::cancel as described in the documentation. Their handlers will be invoked with asio::error::operation_aborted as the error parameter, which you'll need to handle somehow.
That said, I don't see a problem using close instead of cancel. Though it is difficult to offer much help or advice without some code to analyze.
Note that some Windows platforms have problems when canceling outstanding asynchronous operations. The documentation has suggestions for portable cancelation if your application needs to support Windows.
What is Windows' best I/O event notification facility?
By best I mean something that ...
doesn't have a limit on number of input file descriptors
works on all file descriptors (disk files, sockets, ...)
provides various notification modes (edge triggered, limit triggered)
In Windows, async operations are done by file operation, not by descriptor. There are several ways to wait on file operations to complete asynchronously.
For example, if you want to know when data is available on a network socket, issue an async read request on the socket and when it completes, the data was available and was retrieved.
In Win32, async operations use the OVERLAPPED structure to contain state about an outstanding IO operation.
Associate the files with an IO Completion Port and dispatch async IO requests. When an operation completes, it will put a completion message on the queue which your worker thread(s) can wait on and retrieve as they arrive. You can also put user defined messages into the queue. There is no limit to how many files or queued messages can be used with a completion port
Dispatch each IO operation with an event. The event associated with an operation will become signaled (satisfy a wait) when it completes. Use WaitForMultipleObjects to wait on all the events at once. This has the disadvantage of only being able to wait on MAXIMUM_WAIT_OBJECTS objects at once (64). You can also wait on other types of events at the same time (process/thread termination, mutexes, events, semaphores)
Use a thread pool. The thread pool can take an unlimited number of objects and file operations to wait on and execute a user defined function upon completion each.
Use ReadFileEx and WriteFileEx to queue Asynchronous Procedure Calls (APCs) to the calling thread and SleepEx (or WaitFor{Single|Multiple}ObjectsEx) with Alertable TRUE to receive a notification message for each operation when it completes. This method is similar to an IO completion port, but only works for one thread.
The Windows NT kernel makes no distinction between socket, disk file, pipe, etc. file operations internally: all of these options will work with all the file types.
libuv
libuv offers evented I/O for Unix and Windows and has support for socket, files and pipes. It is the platform layer of Node.js.
More details are at: http://nikhilm.github.io/uvbook/introduction.html
There isn't one yet, as far as I am aware. A friend and I are working on an open source Windows epoll implementation (link below) but we're running into issues figuring out how to make it act the same as the Linux implementation.
Current obstacles:
In Linux, file descriptors and socket descriptors are interchangeable, but in Windows they are not. Both must be compatible with an epoll implementation.
In Windows it's quite tricky to get kernel events... which is how epoll works in Linux. We're guessing that a program using our cross-platform epoll library will run noticeably slower in Windows than Linux.
I'll try to come back and update this post as we make progress with the project.
http://sourceforge.net/projects/cpoll
select() function is POSIX and usable on windows including "winsock.h" or "winsock2.h".