Can some one tell me what could be the adverse effect of calling syscall(SYS_gettid) from Signal Handler?
I know it is not in the safe functions list to be called from signal handler but I want to know reason behind it?
I'm pretty sure this has to do with the Signal Handler methods being reentrant. Suppose a signal is sent, and your handler grabs the signal and starts processing. While processing, another signal may be sent by a concurrent program, and your handler again grabs that signal, and starts processing it.
Depending on how the scheduling works out, it's possible that the same chunk of code, the Signal Handler, executes during its own execution. The problem is that it uses the same pointers and variables, so it can corrupt itself, especially because gettid() returns the ID of the current thread. Which is the current thread in this case?
Related
I noticed the asyncio library has a loop.add_signal_handler(signum, callback, *args) method.
So far I have just been catching unix signals in the main file using the signals module in with my asynchronous code like this:
signal.signal(signal.SIGHUP, callback)
async def main():
...
Is that an oversight on my part?
The add_signal_handler documentation is sparse1, but looking at the source, it appears that the main added value compared to signal.signal is that add_signal_handler will ensure that the signal wakes up the event loop and allow the loop to invoke the signal handler along with other queued callbacks and runnable coroutines.
So far I have just been catching unix signals in the main file using the signals module [...] Is that an oversight on my part?
That depends on what the signal handler is doing. Printing a message or updating a global is fine, but if it is invoking anything in any way related to asyncio, it's most likely an oversight. A signal can be delivered at (almost) any time, including during execution of an asyncio callback, a coroutine, or even during asyncio's own bookkeeping.
For example, the implementation of asyncio.Queue freely assumes that the access to the queue is single-threaded and non-reentrant. A signal handler adding something to a queue using q.put_nowait() would be disastrous if it interrupted an on-going invocation of q.put_nowait() on the same queue. Similar to typical race conditions experienced in multi-threaded code, an interruption in the middle of assignment to _unfinished_tasks might well cause it to get incremented only once instead of twice (once for each put_nowait).
Asyncio code is designed for cooperative multi-tasking, where the points where a function may suspend defined are clearly denoted by the await and related keywords. The add_signal_handler function ensures that your signal handler gets invoked at such a point, and that you're free to implement it as you'd implement any other asyncio callback.
1 When this answer was originally written, the add_signal_handler documentation was briefer than today and didn't cover the difference to signal.signal at all. This question prompted it getting expanded in the meantime.
I am attempting to send a signal to a specific thread with pthread_kill. I use pthread_from_mach_thread_np() to get a handle and then use pthread_kill to send the signal.
This worked well in my other testing, but now I see that when attempting to signal a thread internally created by GCD, I get a return code of 45 from pthread_kill.
GCD API that spawned that thread:
dispatch_async(dispatch_get_global_queue(QOS_CLASS_USER_INITIATED, 0), ^{ ... });
Any reason this is happening?
—-
To add some further information, I am not attempting to kill threads. pthread_kill() is the standard POSIX API to send signals to threads. If a signal handler is installed, the thread’s context is switched with a trampoline to the handler.
While what I attempt to achieve using my signal handler can be achieved in better ways, this is not in question here. Even if for purely academic reasons, I would like to understand what is going on here internally.
The pthread_kill() API is specifically disallowed on workqueue threads (the worker threads underlying GCD) and returns ENOTSUP for such threads.
This is primarily intended to prevent execution of arbitrary signal handlers in the context of code that may not expect it (since these threads are a shared resource used by many independent subsystems in a process), as well as to abstract away that execution context so that the system has the freedom to change it in the future.
You can see the details of how this is achieved in the implementation.
That is a very bad idea. You don't own GCDs thread-pool, and you absolutely must not kill its threads out from under it.
The answer to your question is DO NOT DO THAT UNDER ANY CIRCUMSTANCES.
I remember there was a way to do this, something similar to unix signals, but not so widely used. But can't remember the term. No events/mutexes are used: the thread is just interrupted at random place, the function is called and when it returns, the thread continues.
Windows has Asynchronous Procedure Calls which can call a function in the context of a specific thread. APC's do not just interrupt a thread at a random place (that would be dangerous - the thread could be in the middle of writing to a file or obtaining a lock or in Kernel mode). Instead an APC will be dispatched when the calling thread enters an alterable wait by calling a specific function (See the APC documentation).
If the reason that you need to call code in a specific thread is because you are interacting with the user interface, it would be more direct to send or post a window message to the window handle that you want to update. Window messages are always processed in the thread that created the window.
you can search RtlRemoteCall, it's an undocumented routine though. there's APC in Windows semantically similar to Unix signal, however APC requires target thread is in an alertable state to get delivered, it's not guaranteed this condition is always met
I have iocp running and working (mostly) -- but should calling CloseHandle() on a handle cause it to complete?
e.g., I've called ReadFile() and it's now waiting for input to read. At another point I'm calling CloseHandle() from another thread in the pool used to service iocp completion packets. I expect for there to be a completion on the previous ReadFile() call with an ERROR_INVALID_HANDLE, but I'm not seeing that. Instead, it never seems to return. Could this be a sign that something else is holding a reference to the handle? If so, how would you suggest figuring that out/debugging it?
Any suggestions?
Thanks!
CancelIo() is supposed to cancel all pending I/O operations associated with the calling thread. In my experience, CancelIo() sometimes cancels future I/O operations as well. Given:
ReadFile(port, buffer, length, &bytesTransferred, overlapped);
If I invoke CancelIo(port) immediately before the read, GetQueuedCompletionStatus() will block forever, never receiving the read operation.
If I invoke CancelIo(port) immediately after the read, GetQueuedCompletionStatus() will return 0 with GetLastError()==ERROR_OPERATION_ABORTED
If I invoke CancelIo(port) and there are no pending or subsequent reads, GetQueuedCompletionStatus() will block forever.
The key point here is that there is no way to detect when CancelIo() has finished executing. How can I ensure that CancelIo() is done executing and it is safe to issue further read requests?
PS: Looking at http://osdir.com/ml/lib.boost.asio.user/2008-02/msg00074.html and http://www.boost.org/doc/libs/1_44_0/doc/html/boost_asio/using.html it sounds like CancelIo() is not really usable. Must customer requires Windows XP support. What are my options?
NOTE: I am reading from a serial port.
CancelIo() works fine. I misunderstood my code.
Upon further investigation it turns out that the code was invoking CancelIo() followed by ReadFile() with a timeout INFINITE. The completion port was never getting notified of the read because the remote end was never sending anything. In other words, CancelIo() did not cancel subsequent operations.
I found some eye-opening documentation here:
Be careful when coding for asynchronous I/O because the system reserves the right to make an operation synchronous if it needs to. Therefore, it is best if you write the program to correctly handle an I/O operation that may be completed either synchronously or asynchronously. The sample code demonstrates this consideration.
It turns out that device drivers may choose to treat an asynchronous operation in a synchronous manner if the data being read is already cached by the device driver. Upon further investigation, I discovered that when CancelIo() was being invoked before ReadFile() it would sometimes cause the latter to return synchronously. I have no idea why the completion port was never getting notified of ReadFile() after a CancelIo() but I can no longer reproduce this problem.
The completion port is signaled regardless of whether ReadFile() is synchronous or asynchronous.
Wait on (possibly with zero timeout) overlapped.Handle. It will be set whether the operation is completed or cancelled.
If you're already using overlapped operations, why do you need to cancel I/O at all? The entire concept of 'cancelling' an in-flight I/O operation is really race-prone, and totally subject to the underlying device stack you're trying to write to; really the only time you'd want to do this is to unblock another thread who is waiting on the completion of that I/O.
It is possible to write asynchronous I/O code without CancelIo function. The question depends on the scenario you are using CancelIO. Let's say that you need to implement file reading thread. Thread pseudo-code:
for(;;)
{
ReadFile(port, buffer, length, &bytesTransferred, overlapped);
WaitForMultipleObjects( overlapped event + stop event);
if ( stop event is signaled )
break;
if (overlapped event is signaled )
handle ReadFile results
}
Such thread reads file (socket, port etc.) using overlapped I/O. Most of the time it waits on WiatForMultipleObjects line. It wakes up when new data is available, or stop event is signaled. To stop this thread, set stop event from another thread. CancelIO is not used.