In the Linux kernel, I need to periodically check the state of the switch chip by calling the mdiobus_read() (drivers/net/phy/mdio_bus.c).
I tried to use Linux timer add_timer() but I found out that the callback is called in the interrupt context. Actually, there is a warning in the comment to the function mdiobus_read():
NOTE: MUST NOT be called from interrupt context,
because the bus read/write functions may wait for an interrupt
to conclude the operation.
So, how to periodically call the function mdiobus_read()?
Related
I’ve been trying to refresh my understanding of sleeping in the kernel with regards to wait queues. So started browsing the source code for bcmgenet.c (kernel version 4.4) which is the driver responsible for driving the 7xxx series of Broadcom SoC for their set top box solution.
As part of the probe callback, this driver initializes a work queue which is part of the driver’s private structure and adds itself to the Q. But I do not see any blocking of any kind anywhere. Then it goes on to initialize a work queue with a function to call when woken up.
Now coming to the ISR0 for the driver, within that is an explicit call to the scheduler as part of the ISR (bcmgenet_isr0) if certain conditions are met. Now AFAIK, this call is used to defer work to a later time, much like a tasklet does.
Post this we check some MDIO status flags and if the conditions are met, we wake up the process which was blocked in process context. But where exactly is the process blocked?
Also, most of the time, wait queues seem to be used in conjunction with work queues. Is that the typical way to use them?
As part of the probe callback, this driver initializes a work queue which is part of the driver’s private structure and adds itself to the Q. But I do not see any blocking of any kind anywhere.
I think you meant the wait queue head, not the work queue. I do not see any evidence of the probe adding itself to the queue; it is merely initializing the queue.
The queue is used by the calls to the wait_event_timeout() macro in the bcmgenet_mii_read() and bcmgenet_mii_write() functions in bcmmii.c. These calls will block until either the condition they are waiting for becomes true or the timeout period elapses. They are woken up by the wake_up(&priv->wq); call in the ISR0 interrupt handler.
Then it goes on to initialize a work queue with a function to call when woken up.
It is initializing a work item, not a work queue. The function will be called from a kernel thread as a result of the work item being added to the system work queue.
Now coming to the ISR0 for the driver, within that is an explicit call to the scheduler as part of the ISR (bcmgenet_isr0) if certain conditions are met. Now AFAIK, this call is used to defer work to a later time, much like a tasklet does.
You are referring to the schedule_work(&priv->bcmgenet_irq_work); call in the ISR0 interrupt handler. This is adding the previously mentioned work item to the system work queue. It is similar to as tasklet, but tasklets are run in a softirq context whereas work items are run in a process context.
Post this we check some MDIO status flags and if the conditions are met, we wake up the process which was blocked in process context. But where exactly is the process blocked?
As mentioned above, the process is blocked in the bcmgenet_mii_read() and bcmgenet_mii_write() functions, although they use a timeout to avoid blocking for long periods. (This timeout is especially important for those versions of GENET that do not support MDIO-related interrupts!)
Also, most of the time, wait queues seem to be used in conjunction with work queues. Is that the typical way to use them?
Not especially. This particular driver uses both a wait queue and a work item, but I wouldn't describe them as being used "in conjunction" since they are being used to handle different interrupt conditions.
I remember there was a way to do this, something similar to unix signals, but not so widely used. But can't remember the term. No events/mutexes are used: the thread is just interrupted at random place, the function is called and when it returns, the thread continues.
Windows has Asynchronous Procedure Calls which can call a function in the context of a specific thread. APC's do not just interrupt a thread at a random place (that would be dangerous - the thread could be in the middle of writing to a file or obtaining a lock or in Kernel mode). Instead an APC will be dispatched when the calling thread enters an alterable wait by calling a specific function (See the APC documentation).
If the reason that you need to call code in a specific thread is because you are interacting with the user interface, it would be more direct to send or post a window message to the window handle that you want to update. Window messages are always processed in the thread that created the window.
you can search RtlRemoteCall, it's an undocumented routine though. there's APC in Windows semantically similar to Unix signal, however APC requires target thread is in an alertable state to get delivered, it's not guaranteed this condition is always met
I want to use inverted model of ioctl. I mean I want to schedule some work item which is a user space thread when a particular activity is detected by the driver. For eg.
1. I register a callback for a particular interrupt in my kernel mode driver.
2. Whenever I get an interrupt, I want to schedule some user space thread which user had registered using ioctl.
Can I use either DPC, APC or IRP to do so. I do know that one should not/cant differ driver space work to user space. What I want is to do some independent activities in the user space when a particular hardware event happens.
Thanks
creating usermode threads from driver is really bad practice and you can`t simple transfer control from kernel mode to user mode. You must create worker threads in user app and wait in this threads for event. There are two main approaches for waiting.
1) You can wait on some event, witch you post to driver in ioctl. In some moment driver set event to alertable and thread go and process event. This is major and simple approach
2) You can post ioctl synchronously and in driver pend this irp -> thread blocks in DeviceIoControl call. When event occured driver complete these irp and thread wake up and go for processing.
Whenever I get an interrupt, I want to schedule some user space threads which user had registered using ioctl.
You must go to safe irql (< DISPATCH_IRQL) first : Interrupt -> DPC push into queue -> worker thread, because for example you can`t signal event on high irql.
read this
http://www.osronline.com/article.cfm?id=108
and Walter Oney book
You don't need to queue a work item or do anything too fancy with posting events down. The scheduler is callable at DISPATCH_LEVEL, so a DPC is sufficient for signalling anyone.
Just use a normal inverted call:
1) App sends down an IOCTL (if more than one thread must be signalled, it must use FILE_FLAG_OVERLAPPED and async I/O).
2) Driver puts the resulting IRP into a driver managed queue after setting cancel routines, etc. Marks the irp pending and returns STATUS_PENDING.
3) Interrupt arrives... Queue a DPC from your ISR (or if this is usb or some other stack, you may already be at DISPATCH_LEVEL).
4) Remove the request from the queue and call IoCompleteRequest.
Use KMDF for steps 2 and 4. There's lot of stuff you can screw up with queuing irps, so it's best to use well-tested code for that.
CancelIo() is supposed to cancel all pending I/O operations associated with the calling thread. In my experience, CancelIo() sometimes cancels future I/O operations as well. Given:
ReadFile(port, buffer, length, &bytesTransferred, overlapped);
If I invoke CancelIo(port) immediately before the read, GetQueuedCompletionStatus() will block forever, never receiving the read operation.
If I invoke CancelIo(port) immediately after the read, GetQueuedCompletionStatus() will return 0 with GetLastError()==ERROR_OPERATION_ABORTED
If I invoke CancelIo(port) and there are no pending or subsequent reads, GetQueuedCompletionStatus() will block forever.
The key point here is that there is no way to detect when CancelIo() has finished executing. How can I ensure that CancelIo() is done executing and it is safe to issue further read requests?
PS: Looking at http://osdir.com/ml/lib.boost.asio.user/2008-02/msg00074.html and http://www.boost.org/doc/libs/1_44_0/doc/html/boost_asio/using.html it sounds like CancelIo() is not really usable. Must customer requires Windows XP support. What are my options?
NOTE: I am reading from a serial port.
CancelIo() works fine. I misunderstood my code.
Upon further investigation it turns out that the code was invoking CancelIo() followed by ReadFile() with a timeout INFINITE. The completion port was never getting notified of the read because the remote end was never sending anything. In other words, CancelIo() did not cancel subsequent operations.
I found some eye-opening documentation here:
Be careful when coding for asynchronous I/O because the system reserves the right to make an operation synchronous if it needs to. Therefore, it is best if you write the program to correctly handle an I/O operation that may be completed either synchronously or asynchronously. The sample code demonstrates this consideration.
It turns out that device drivers may choose to treat an asynchronous operation in a synchronous manner if the data being read is already cached by the device driver. Upon further investigation, I discovered that when CancelIo() was being invoked before ReadFile() it would sometimes cause the latter to return synchronously. I have no idea why the completion port was never getting notified of ReadFile() after a CancelIo() but I can no longer reproduce this problem.
The completion port is signaled regardless of whether ReadFile() is synchronous or asynchronous.
Wait on (possibly with zero timeout) overlapped.Handle. It will be set whether the operation is completed or cancelled.
If you're already using overlapped operations, why do you need to cancel I/O at all? The entire concept of 'cancelling' an in-flight I/O operation is really race-prone, and totally subject to the underlying device stack you're trying to write to; really the only time you'd want to do this is to unblock another thread who is waiting on the completion of that I/O.
It is possible to write asynchronous I/O code without CancelIo function. The question depends on the scenario you are using CancelIO. Let's say that you need to implement file reading thread. Thread pseudo-code:
for(;;)
{
ReadFile(port, buffer, length, &bytesTransferred, overlapped);
WaitForMultipleObjects( overlapped event + stop event);
if ( stop event is signaled )
break;
if (overlapped event is signaled )
handle ReadFile results
}
Such thread reads file (socket, port etc.) using overlapped I/O. Most of the time it waits on WiatForMultipleObjects line. It wakes up when new data is available, or stop event is signaled. To stop this thread, set stop event from another thread. CancelIO is not used.
What happens when you call WaitForSingleObject() on a handle you've created with CreateFile() or _get_osfhandle()?
For reasons not worth explaining I would like to use WaitForSingleObject() to wait on a HANDLE that I've created with _get_osfhandle(fd), where fd comes from a regular call to _open(). Is this possible?
I have tried it in practice, and on some machines it works as expected (the HANDLE is always in the signaled state because you can read more data from it), and on some machines WaitForSingleObject() will block indefinitely if you let it.
The MSDN page for WaitForSingleObject() says that the only supported things that it handles are "change notifications, console input, events, memory resource notifications, mutex, processes, semaphores, threads, and waitable timers."
Additionally, would it be different if I used CreateFile() instead of _get_osfhandle() on a CRT file descriptor?
Don't do it. As you can see, it has undefined behavior.
Even when the behavior is defined, it's defined in such a way as to be relatively not useful unless you don't like writing additional code. It is signaled when any asynchronous I/O operation on that handle completes, which does not generalize to tracking which I/O operation finished.
Why are you trying to wait on a file handle? Clearly the intent matters when you are doing something that isn't even supported well enough to not block indefinitely.
I found the following links. The concensus seems to me, don't do it.
Asynch IO explorer
Waiting on a file handle
When an I/O operation is started on an
asynchronous handle, the handle goes
into a non-signaled state. Therefore,
when used in the context of a
WaitForSingleObject or
WaitForMultipleObjects operation, the
file handle will become signaled when
the I/O operation completes. However,
Microsoft actively discourages this
technique; it does not generalize if
there exists more than one pending I/O
operation; the handle would become
signaled if any I/O operation
completed. Therefore, although this
technique is feasible, it is not
considered best practice.
Egghead Cafe:
Use ReadDirectoryChangesW in
overlapped mode. WaitForSingleObject
can wait on the event in the
OVERLAPPED struct.
You can also use the API
WaitForSingleObject() to wait on a
file change if you use the following
change notification function:
FindFirstChangeNotification()
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/fileio/fs/findfirstchangenotification.asp
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dllproc/base/waitforsingleobject.asp
An interesting note on "evilness" of ReadDirectoryChangesW:
http://blogs.msdn.com/ericgu/archive/2005/10/07/478396.aspx