I'm using kqueue for socket synchronization in OS X. I can register an event of interest like the following:
struct kevent change;
EV_SET(&change, connected_socket, EVFILT_READ, EV_ADD, 0, NULL, NULL);
kevent(k_queue_, &change, 1, NULL, 0, NULL);
And the question is, is there a way to trigger this event by force so that the waiting kevent call would return?
Some possibilities aside from natural writing of data to the other side of the socket :)
shutdown(2) the read side of that socket - you'll get EV_EOF in flags (silly),
Use the timeout argument and call the same handling function,
Use the self-pipe trick when you need to break the wait.
My question though: why do you need this?
Edit:
If I understand your comments correctly you are looking for a way to get around edge-triggered behavior (EV_CLEAR) for write events. I believe that the proper way of doing this is to un-register your socket from EVFILT_WRITE when you don't have anything in the outgoing queue, then re-register it again when there's data to send. It's a bit more work, but that's how it works, and you don't need any additional system calls since kevent(2) accepts both changes and results. Take a look into libevent and see how it handles this sort of stuff. And you are using non-blocking sockets, right?
I would recommend a slightly different solution.
Add another registered event to the kqueue. Specifically a EVFILT_USER.
You can use this to trigger whatever behavior you want to wake the kevent() thread up for without the code looking weird or being hard to maintain.
The OSX sources have a real rough test for it in
http://www.opensource.apple.com/source/xnu/xnu-1699.24.23/tools/tests/xnu_quick_test/kqueue_tests.c
OSX 10.6 and FreeBSD 8.1 add support for EVFILT_USER, which we can use to wake up the event loop from another thread.
Note that if you use this to implement your own condition and timedwait, you still need locks in order to avoid race conditions, as explained in this excellent answer.
See my other answer for a full code example: https://stackoverflow.com/a/31174803/432
Related
On Unix: I’ve been through FAM and Gamin, and both seem to provide a client/server file monitoring system. I would rather have a system where I tell the kernel to monitor some inodes and it pokes me back when events occur. Inotify looked promising at first on that side: inotify_init1 let me pass IN_NONBLOCK which in turn caused poll() to return directly. However I understood that I would have to call it regularly if I wanted to have news about the monitored files. Now I’m a bit short of ideas.
Is there something to monitor files asynchronously?
PS: I haven’t looked on Windows yet, but I would love to have some answers about it too.
As Celada says in the comments above, inotify and poll are the right way to do this.
Signals are not a mechanism for reasonable asynchronous programming -- and signal handlers are remarkably dangerous for the inexperienced and even for the experienced. One does not use them for such purposes voluntarily.
Instead, one should structure one's program around an event loop (see http://en.wikipedia.org/wiki/Event-driven_programming for an overall explanation) using poll, select, or some similar system call as the core of your program's event handling mechanism.
Alternatively, you can use threads, or threads plus an event loop.
However interesting are you answers, I am sorry but I can’t accept a mechanism based on blocking calls on poll or select, when the question states “asynchronously”, regardless of how deep it is hidden.
On the other hand, I found out that one could manage to run inotify asynchronously by passing to inotify_init1 the flag IN_NONBLOCK. Signals are not triggered as they would have with aio, and a read call that would block blocking would set errno to EWOULDBLOCK instead.
The MSDN states in its description of ReadFile() function:
If hFile is opened with FILE_FLAG_OVERLAPPED, the lpOverlapped parameter must point to a valid and unique OVERLAPPED structure, otherwise the function can incorrectly report that the read operation is complete.
I have some applications that are violating the above recommendation and I would like to know the severity of the problem. I mean the program uses named pipe that has been created with FILE_FLAG_OVERLAPPED, but it reads from it using the following call:
ReadFile(handle, &buf, n, &n_read, NULL);
That means it passes NULL as the lpOverlapped parameter. That call should not work correctly in some circumstances according to documentation. I have spent a lot of time trying to reproduce the problem, but I was unable to! I always got all data in right place at right time. I was testing only Named Pipes though.
Would anybody know when can I expect that ReadFile() will incorrectly return and report successful completion even the data are not yet in the buffer? What would have to happen in order to reproduce the problem? Does it happen with files, pipes, sockets, consoles, or other devices? Do I have to use particular version of OS? Or particular version of reading (like register the handle to I/O completion port)? Or particular synchronization of reading and writing processes/threads?
Or when would that fail? It works for me :/
Please help!
With regards, Martin
Internally the system only supports asynchronous I/O. For synchronous I/O the system creates a temporary OVERLAPPED structure with hEvent = NULL;, issues an asynchronous I/O request passing in this temporary, and then waits for completion using GetOverlappedResult( bWait = TRUE ).
Recall that the hEvent of the temporary OVERLAPPED structure is NULL and pay attention to the Remarks section of GetOverlappedResult:
If the hEvent member of the OVERLAPPED structure is NULL, the system uses the state of the hFile handle to signal when the operation has been completed.
A file HANDLE is a waitable object that becomes unsignaled when an I/O operation begins, and signaled when an I/O operation ends.
Now consider a scenario where an asynchronous file HANDLE has a pending I/O request at the time you issue a synchronous I/O request. The system creates an OVERLAPPED structure and waits on the hFile HANDLE for completion. In the meantime the asynchronous I/O completes, thereby signaling the HANDLE causing the synchronous I/O to return prematurely without having actually completed.
Worse though is that by the time the asynchronous I/O that was initiated in response to the synchronous I/O request completes it will update the temporary OVERLAPPED structure that no longer exists. The result is memory corruption.
The full story can be found at The Old New Thing.
Seems like you are in a situation where you are deliberately calling an API in contravention of the documented best practices. In such situations all bets are off. It may work, it may not. If may work on this OS, but not on the next iteration of the OS, or the next service pack of the same OS. What happens when you port to Win64? Will it still work then?
Does calling GetLastError() (or looking at #ERR,hr in the debugger) give any value that is useful in addition to the error code?
I recommend that you call it with a valid OVERLAPPED structure, get it working and remove all doubt (and possibility of random failure). Why have possibly buggy code (and very hard to reproduce bugs) in your software when you can fix the problem easily by using a valid OVERLAPPED structure?
Why ask the question rather than fix the code to call the API as it was intended?
I suspect it always appears to work because, even though this is an asynchronous I/O, it completes very quickly. Depending on how you're testing for success, it's possible the function is incorrectly reporting that the operation completed, but it actually completes before you test the results.
The real test would be to do a read on the pipe before there's data to be read.
But really, you should just fix the code. If your architecture cannot handle asynchronous I/O, then remove the FILE_FLAG_OVERLAPPED from the creation of the named pipe.
When they say
Blockquote
If hFile is opened with FILE_FLAG_OVERLAPPED, the lpOverlapped parameter must point to a valid and unique OVERLAPPED structure, otherwise the function can incorrectly report that the read operation is complete.
they mean that there's nothing in the code preventing it working, but there's also a path through their code that can produce erroneous results. Just because you can't reproduce the problem with your particular hardware does not mean there is no problem.
If you really want to reproduce this problem, leave the code as is and go on with your life. Right about the time you've forgotten all about this problem, strange behavior will surface that will not have any obvious relations to calling ReadFile. You'll spend days pulling your hair out, and the problem will appear to come and go randomly. Eventually you'll find it and kick yourself for not following the instructions. Been there, done that, no fun!
The other way to recreate the problem is to schedule an important demo for your customer. It's sure to fail then!
If you don't want to splatter your code with OVERLAPPED structures and all of the related return value checks, Waits, Events, etc, you can write a wrapper function that takes a handle from which to read, and a timeout. Simply replace your calls to ReadFile with this handy-dandy wrapper.
I need to determine whether a handle that my code did not create, for which GetFileType()==FILE_TYPE_PIPE, is a socket or not. There does not seem to be an API for this.
I have tried the following. The general idea is to use a socket-specific function and treat failure as meaning non-socket.
getsockopt() -- This was my first attempt. Unfortunately it seems to hang when called by many threads on the same (non-socket) handle.
WSAEnumNetworkEvents() -- this is what Gnulib does but will have undesirable side effects if the handle is a socket.
getpeername() -- this is what cygwin does but this will fail for some sockets too. Guessing whether an error implies socket-ness does not seem reliable and future safe.
I do not mind if the solution only work on some versions of Windows, e.g. Vista, I can always fall back to some other method in the general case.
I'm thinking that perhaps you could attempt to call GetNamedPipeInfo() on your handle. If the call succeeds you know that the handle is a pipe handle, otherwise it must be a socket.
Have you tried WSADuplicateSocket. Then just check WSAPROTOCOL_INFO to see if it is in fact a named pipe...
You can use GetNamedPipeHandleState() as well, evaluating the result with GetLastError().
I am developing a multi-threaded application in Cocoa. The main thread takes values from the user, and when a button is clicked I invoke a secondary thread in which a long calculation takes place. Now from this thread I have to return the output of every step of the calculation to the main thread. I want to periodically send data from one thread to the other. I can't find any simple example that does this. Any ideas?
There are a number of ways to do this, in rough order of complexity (easiest first):
use NSObject's performSelectorOnMainThread:withObject:waitUntilDone: which is pretty self explanatory.
use performSelector:onThread:withObject:waitUntilDone:, which will let you go the other way
use an NSNotification (and NSDistributedNotificationCenter), though you can easily run into a race condition if you're not careful
Use NSPorts to send data back and forth
Check out the doc that Abizer mentioned for details on all of these.
performSelectorOnMainThread:withObject:waitUntilDone: is often the easiest way to update the UI with a background thread's progress. You could also create your own storage area that's safe to access between threads using NSLock or a similar mechanism, or even use distributed objects (which also works between processes or over a network).
Then there's NSOperationQueue and NSOperation which does help a lot to simplify multi-threaded programming, although a lot of programmers have been avoiding it since it can cause a crash in certain circumstances under Leopard.
Have a look at the Apple docs for this.
You may need to create an ADC member account, but this is free
Multi-threaded Cocoa Programs
I'm going back over some crufty code to tidy it up and I see I've been retaining NSTimers returned from scheduledTimerWithTimeInterval:target:selector:userInfo:repeats: and storing them in a field - then releasing and nulling the field when it fires. In some cases I use the timer to send an invalidate message to it later - but in many cases not.
In more recent code I usually just schedule the timer and forget about it. I understand that the constructor method is autoreleasing and the timer is retained by the run loop while it is active - so I don't see any issue with this.
So, just to round out my understanding - is there any other reason I should be holding on to my timers, or is what I am doing now the accepted idiom?
This is all in the context of iPhone code, but I don't believe this is iPhone specific.
I just let the run loop handle the retention, myself - it's the run loop that owns the timer and not me. If you see what I mean.
I'm answering my own question.
I was mostly asking the question because I'd been browsing through the class reference docs and this issue was not really made clear. I've since read the Timer Programming Topics: Using Timers article in the ADC and it covers it pretty well - especially the section on memory management at the end.
According to that what I am doing now (just scheduling, unless I need to be able to call invalidate, or isValid etc) is the right way.