There are 2 processes running on Windows. They communicate with each other through named pipe. When one of them is ready to send a message, I want to notificate the other process asynchronously like signal on Linux so that the other process don't need to check for the pipe continously. Are there some similar methods like the signal mechanism on Windows or other way to solve my problem?
A direct signal mechanism which conceptually works the same way does not exist (one could probably simulate it with a thread injection hack, but don't even think about that). It is not much of a problem, since you can do otherwise.
Every waitable kernel object which can take a name such as an event or a semaphore can be accessed by different processes.
You can WaitForSingleObject on the synchronization primitive until the other process signals it. That would be a Unix-like readiness notification mechanism (not quite as elegant, but to the same effect).
However, that isn't even necessary. Named pipes (not true for anyonymous pipes!) can be used with overlapped I/O. Which means you can use ReadFileEx to initiate a read from the pipe, and it will linger there in the background until it can complete.
You can think of this kind of I/O as "fire and forget". Your process continues running while the read operation is blocked. When the read operation completes, it signals an event or posts a completion message to a completion port (which you can query) or posts an asynchronous procedure call ("APC", a more fancy name for "callback") to the thread that originally called it. That's as close to a "signal" as you can get under Windows.
Unluckily, APCs don't quite work as one would wish, since they only execute at well-defined points (when a thread is in an "alertable wait state", which you must do explicitly by setting the altertable flag in a wait function or calling NtTestAlert).
The likely reasoning why the Windows designers made it that way that this is "safer", but it is also more annoying from an usability point of view. Alas, that is how it works.
Note that the overlapped I/O model is the exact opposite of the readiness notification system under e.g. Linux. Rather than asking the OS whether a descriptor is ready to be read, you tell the OS to read it, and you can have yourself be notified (or verify) whether this has completed.
Related
In Pthreads if a conditional variable is signaled and there are no threads blocking on a pthread_cond_wait() then this signal is lost.
When you use SetEvent() to signal an Event in Windows this event remains in the signaled state until another thread waits for this event object.
Which of the above behaviours do CONDITIONAL_VARIABLEs signaled using WakeConditionVariable() follow?
That's an undocumented implementation detail, subject to change. So your code must behave correctly in either case. As Raymond already pointed out, if you use condition variables as intended, this happens automatically.
If for some reason you are not using the usual pattern, you should assume that the wake will be lost, but remember that threads waiting on a condition variable may be woken at any time for any reason or none:
Condition variables are subject to spurious wakeups (those not associated with an explicit wake)
This means that if you assume the wake is lost, but in fact it isn't, it just looks like a spurious wakeup as far as the code is concerned. Provided that the code deals with spurious wakeups correctly, it will also deal correctly with this scenario.
From the following sentence on MSDN (https://msdn.microsoft.com/en-us/library/windows/desktop/ms687080(v=vs.85).aspx)
Waking one thread is similar to setting an auto-reset event, while waking all
threads is similar to pulsing a manual reset event but more reliable (see
PulseEvent for details).
I deduce that the behavior is the same as in the pthread library. But as said in the comment above, this aspect actually does not matter much if you plan to use condition variables in a standard way.
I'm about to implement my first Windows service. The service will connect to a Bluetooth dongle, and liaison some commands and data to a single client process.
Each process (client, server), shall have at least two thread - one blocking on Read(), another crunching business logic and doing an occasionally Write().
Checking the alternatives, I've decided to go with Named Pipes for IPC, but I'm having trouble understanding some of the settings. Specifically:
I wish to allow simultaneous reads and writes. Do I need to create the pipe with FILE_FLAG_OVERLAPPED, even though I do not intend to do reads and writes on the same thread?
If the answer to the above is 'yes', do I still need to pass an OVERLAPPED structure to ReadFile(), WriteFile(), use GetOverlappedResult() etc? If so, what is the rational behind this?
What is so great about having a single thread do non-blocking reads and writes anyway? What are the use cases?
EDIT
I wish to clarify the question:
Assuming both threads (Read / Write) access the pipe simultaneously, will one block until the other is done due to some internal pipe mutex?
Will setting FILE_FLAG_OVERLAPPED change this behavior?
You don't need to use FILE_FLAG_OVERLAPPED if all you're doing is reading and writing using the same handle at the same time. Other threads reading or writing to the same end of the pipe won't cause it to block. You only need it if you want to perform asynchronous I/O, which apparently you don't.
If you do use the FILE_FLAG_OVERLAPPED you must pass a valid OVERLAPPED structure to ReadFile and WriteFile via the lpOverlapped argument. If you don't use this flag and the handle isn't seekable (eg. a named pipe) then you must pass NULL instead.
The big advantage of using single threaded asynchronous I/O over a multi-threaded implementation is that you don't have to worry about concurrency issues. You can't have race conditions and deadlocks if you only have one thread. (Actually in your case, since you would still have two threads, one in the server and one the client, you can still have deadlocks and maybe race conditions if you really try, but asynchronous I/O would still make it easier to avoid them.)
To enable bi-directional access to a pipe, you have to specify the PIPE_ACCESS_DUPLEX flag for the dwOpenMode parameter. Asynchronous operation (FILE_FLAG_OVERLAPPED) is not strictly required to enable bi-directional mode.
Using asynchronous I/O, however, is recommended for bi-directional pipes. It allows you to issue both read and write operations at the same time, on the same thread (see WaitForMultipleObjects). Either operation is signaled upon completion. This prevents a lengthy write operation from blocking a potential read, for example, and you'll be able to respond to either one in a timely manner. Maybe even more importantly, since you never know when data is available, you'll typically want to always issue a read operation, without having it block your thread.
This is outlined in the documentation for CreateNamedPipe:
If [overlapped] mode is enabled, functions performing read, write, and connect operations that may take a significant time to be completed can return immediately. This mode enables the thread that started the operation to perform other operations while the time-consuming operation executes in the background. For example, in overlapped mode, a thread can handle simultaneous input and output (I/O) operations on multiple instances of a pipe or perform simultaneous read and write operations on the same pipe handle. If overlapped mode is not enabled, functions performing read, write, and connect operations on the pipe handle do not return until the operation is finished. The ReadFileEx and WriteFileEx functions can only be used with a pipe handle in overlapped mode. The ReadFile, WriteFile, ConnectNamedPipe, and TransactNamedPipe functions can execute either synchronously or as overlapped operations.
Vsync blocks SwapBuffers(), which is what I want. My problem is that, since input messages go to the same thread that owns the window, any messages that come in while SwapBuffers() is blocked won't be processed immediately, but only after the vsync triggers the buffer swap and SwapBuffers() returns. So I have all my compute threads sitting idle instead of processing the scene for rendering in the next frame using the most recent input. I'm particularly concerned with having very low latency. I need some way to access all pending input messages to the window from other threads.
Windows API provides a way to wait for either Windows events or input messages using MsgWaitForMultipleObjects(), yet there's no similar way to wait for a buffer swap together with other things. That's very unfortunate.
I considered calling SwapBuffers() in another thread, but that requires glFinish() to be called in the window's thread before signalling another thread to SwapBuffers(), and glFinish() is still a blocking call so it's not a good solution.
I considered hooking, but that also looks like a dead end. Hooking with WH_GETMESSAGE will have the GetMsgProc() be called not asynchronously, but when the window's thread calls GetMessage()/PeekMessage(), so it's no help. Installing a global hook doesn't help me either due to the need of calling RegisterTouchWindow() with a specific window handle to process WM_TOUCH -- and my input is touch. And, while for mouse and keyboard, you can install low level hooks that capture messages as they're posted to the thread's queue, rather than when the thread calls GetMessage()/PeekMessage(), there appears to be no similar option for touch.
I also looked at wglDelayBeforeSwapNV(), but I don't see what's preventing the OS from preempting a thread sometimes after the call to that function but before SwapBuffers(), causing a miss of the next vsync signal.
So what's a good workaround? Can I make a second, invisible window, that will somehow be always the active one and so get all input messages, while the visible one is displaying the rendering? According to another discussion, message-only windows (CreateWindow with HWND_MESSAGE) are not compatible with WM_TOUCH. Is there perhaps some undocumented event that SwapBuffers() is internally waiting on that I could access and feed to MsgWaitForMultipleObjects()? My target is a fixed platform (Windows 8.1 64-bit) so I'm fine with using undocumented functionality, should it exist. I do want to avoid writing my own touchscreen driver, however.
Out of curiosity, why not implement your entire drawing logic in that other thread? It appears the problem you are running into is that the message pump is driven by the same thread that draws. Since Windows does not let you drive the message pump from a different thread than the one that created the window, the easiest solution would just be to push all the GL stuff into a different thread.
SwapBuffers (...) is also not necessarily going to block. As-per requirements of VSYNC an implementation need only block the next command that would modify the backbuffer while all backbuffers are pending a swap. Triple buffering changes things up a little bit by introducing a second backbuffer.
One possible implementation of triple buffering will discard the oldest backbuffer when it comes time to swap, thus SwapBuffers (...) would never cause blocking (this is effectively how modern versions of Windows work in windowed mode with the DWM enabled). Other implementations will eventually present both backbuffers, this reduces (but does not eliminate) blocking but also results in the display of late frames.
Unfortunately WGL does not let you request the number of backbuffers in a swap-chain (beyond 0 single-buffered or 1 double-buffered); the only way to get triple buffering on Windows is using driver settings. Lowest message latency will come from driving GL in a different thread, but triple buffering can help a little bit while requiring no effort on your part.
On Unix: I’ve been through FAM and Gamin, and both seem to provide a client/server file monitoring system. I would rather have a system where I tell the kernel to monitor some inodes and it pokes me back when events occur. Inotify looked promising at first on that side: inotify_init1 let me pass IN_NONBLOCK which in turn caused poll() to return directly. However I understood that I would have to call it regularly if I wanted to have news about the monitored files. Now I’m a bit short of ideas.
Is there something to monitor files asynchronously?
PS: I haven’t looked on Windows yet, but I would love to have some answers about it too.
As Celada says in the comments above, inotify and poll are the right way to do this.
Signals are not a mechanism for reasonable asynchronous programming -- and signal handlers are remarkably dangerous for the inexperienced and even for the experienced. One does not use them for such purposes voluntarily.
Instead, one should structure one's program around an event loop (see http://en.wikipedia.org/wiki/Event-driven_programming for an overall explanation) using poll, select, or some similar system call as the core of your program's event handling mechanism.
Alternatively, you can use threads, or threads plus an event loop.
However interesting are you answers, I am sorry but I can’t accept a mechanism based on blocking calls on poll or select, when the question states “asynchronously”, regardless of how deep it is hidden.
On the other hand, I found out that one could manage to run inotify asynchronously by passing to inotify_init1 the flag IN_NONBLOCK. Signals are not triggered as they would have with aio, and a read call that would block blocking would set errno to EWOULDBLOCK instead.
To clarify, I am wondering what are the cons and pros of writing a "multiple simultaneous clients to a single server" using TCP/IP sockets and signal handlers that are called in response to "can read / can write" signal conditions on client socket file descriptors? As far as I understand at least the Linux kernel uses signals to notify a process of conditions related to socket descriptors? Obviously one has to be careful in a signal handler, which, again as I understand, interrupts the process - reentrancy, atomicity, undefined state for variables, etc.
But one does not have to have signals do most work, in fact quite the opposite - add the socket to a set of sockets ready for reading, writing, much like select, poll and epoll_wait do, and let the default process code flow work with these sets? In effect, one emulates much the same pattern as with the functions mentioned, but purely principally, is it doable and how can it be worth it?
There is already a couple of such methods. One is using the SIGIO signal, check man 7 socket and look for the section named "Signals" for more information.
The other method is standardized by POSIX and called async I/O. The functions to use are all prefixed with aio_ (for example aio_read). See this link for an example on how to use this or check the manual page.