I want to use inverted model of ioctl. I mean I want to schedule some work item which is a user space thread when a particular activity is detected by the driver. For eg.
1. I register a callback for a particular interrupt in my kernel mode driver.
2. Whenever I get an interrupt, I want to schedule some user space thread which user had registered using ioctl.
Can I use either DPC, APC or IRP to do so. I do know that one should not/cant differ driver space work to user space. What I want is to do some independent activities in the user space when a particular hardware event happens.
Thanks
creating usermode threads from driver is really bad practice and you can`t simple transfer control from kernel mode to user mode. You must create worker threads in user app and wait in this threads for event. There are two main approaches for waiting.
1) You can wait on some event, witch you post to driver in ioctl. In some moment driver set event to alertable and thread go and process event. This is major and simple approach
2) You can post ioctl synchronously and in driver pend this irp -> thread blocks in DeviceIoControl call. When event occured driver complete these irp and thread wake up and go for processing.
Whenever I get an interrupt, I want to schedule some user space threads which user had registered using ioctl.
You must go to safe irql (< DISPATCH_IRQL) first : Interrupt -> DPC push into queue -> worker thread, because for example you can`t signal event on high irql.
read this
http://www.osronline.com/article.cfm?id=108
and Walter Oney book
You don't need to queue a work item or do anything too fancy with posting events down. The scheduler is callable at DISPATCH_LEVEL, so a DPC is sufficient for signalling anyone.
Just use a normal inverted call:
1) App sends down an IOCTL (if more than one thread must be signalled, it must use FILE_FLAG_OVERLAPPED and async I/O).
2) Driver puts the resulting IRP into a driver managed queue after setting cancel routines, etc. Marks the irp pending and returns STATUS_PENDING.
3) Interrupt arrives... Queue a DPC from your ISR (or if this is usb or some other stack, you may already be at DISPATCH_LEVEL).
4) Remove the request from the queue and call IoCompleteRequest.
Use KMDF for steps 2 and 4. There's lot of stuff you can screw up with queuing irps, so it's best to use well-tested code for that.
Related
I’ve been trying to refresh my understanding of sleeping in the kernel with regards to wait queues. So started browsing the source code for bcmgenet.c (kernel version 4.4) which is the driver responsible for driving the 7xxx series of Broadcom SoC for their set top box solution.
As part of the probe callback, this driver initializes a work queue which is part of the driver’s private structure and adds itself to the Q. But I do not see any blocking of any kind anywhere. Then it goes on to initialize a work queue with a function to call when woken up.
Now coming to the ISR0 for the driver, within that is an explicit call to the scheduler as part of the ISR (bcmgenet_isr0) if certain conditions are met. Now AFAIK, this call is used to defer work to a later time, much like a tasklet does.
Post this we check some MDIO status flags and if the conditions are met, we wake up the process which was blocked in process context. But where exactly is the process blocked?
Also, most of the time, wait queues seem to be used in conjunction with work queues. Is that the typical way to use them?
As part of the probe callback, this driver initializes a work queue which is part of the driver’s private structure and adds itself to the Q. But I do not see any blocking of any kind anywhere.
I think you meant the wait queue head, not the work queue. I do not see any evidence of the probe adding itself to the queue; it is merely initializing the queue.
The queue is used by the calls to the wait_event_timeout() macro in the bcmgenet_mii_read() and bcmgenet_mii_write() functions in bcmmii.c. These calls will block until either the condition they are waiting for becomes true or the timeout period elapses. They are woken up by the wake_up(&priv->wq); call in the ISR0 interrupt handler.
Then it goes on to initialize a work queue with a function to call when woken up.
It is initializing a work item, not a work queue. The function will be called from a kernel thread as a result of the work item being added to the system work queue.
Now coming to the ISR0 for the driver, within that is an explicit call to the scheduler as part of the ISR (bcmgenet_isr0) if certain conditions are met. Now AFAIK, this call is used to defer work to a later time, much like a tasklet does.
You are referring to the schedule_work(&priv->bcmgenet_irq_work); call in the ISR0 interrupt handler. This is adding the previously mentioned work item to the system work queue. It is similar to as tasklet, but tasklets are run in a softirq context whereas work items are run in a process context.
Post this we check some MDIO status flags and if the conditions are met, we wake up the process which was blocked in process context. But where exactly is the process blocked?
As mentioned above, the process is blocked in the bcmgenet_mii_read() and bcmgenet_mii_write() functions, although they use a timeout to avoid blocking for long periods. (This timeout is especially important for those versions of GENET that do not support MDIO-related interrupts!)
Also, most of the time, wait queues seem to be used in conjunction with work queues. Is that the typical way to use them?
Not especially. This particular driver uses both a wait queue and a work item, but I wouldn't describe them as being used "in conjunction" since they are being used to handle different interrupt conditions.
According to here, GetMessage() is a blocking call which won't return until there's a message can be retrieved from the message queue.
So, how is this blocking behavior implemented?
Does GetMessage() use some kind of spin lock so that the UI thread just busy waits for new messages showing up in the message queue? If so, I guess at least one of my CPU cores should have high usage when a UI application is running. But I didn't see that happen. So how does it work?
ADD 1
Thanks for the hint in the comments. Spin lock is meant to reduce the cost of thread context switch. It shouldn't be used here. I also thought about maybe some event paradigm is used here. But if it is event-driven, how does this event model work?
My guess is like this:
An event for input checking is raised periodically. Perhaps via some hardware timer interrupt. Then the timer interrupt handler will check for various input device buffers for input events. And then put that into certain application's message queue based on the current desktop context, such as which is the active window.
And I guess maybe some other things are also based on the timer interrupt, such as thread context switching.
ADD 2
Based on replies so far. There's some event object that the UI thread waits on. But since UI thread is waiting on something, it is not active and can do nothing by itself yet. And the event object is just some passive state information. So there has to be someone else to wake up the thread upon the event state change. I tink it should be the thread scheduler. And the thread scheduler may be pulsed by the timer interrupt.
The thread scheduler will check the event state periodically and wake up thread and put messages into its queue as necessary.
Am I right about the whole picture?
ADD 3
And there's a remaining question: who modify the state of an event object? Based on here, it seems events are just some data structures that can be modified by any active parties. I think thread scheduler just use the relations among threads and events to decide which thread to run or not.
And by the time a thread is scheduled to run, all it's requirements should already been fulfilled. Such as a message should have been put into its queue before the event it waits on is raised. This is reasonable because otherwise it may be too late. (thanks to RbMm's comments.)
ADD 4
In JDK, the LinkedBlockingDeque type also offers a similar blocking behavior with the take() method.
Retrieves and removes the head of the queue represented by this deque
(in other words, the first element of this deque), waiting if
necessary until an element becomes available.
And the .NET counterpart is the BlockingCollection< T > type. A thread to discuss it.
Here is a thread about how to implement a blocking queue in C#.
The exact implementation is internal and not documented.
Most of the window manager in Windows is implemented in kernel mode. GetMessage is no different. It waits on an event object in kernel mode. When a thread is waiting on an event (or any other kernel-based synchronization object), it does not use any CPU time, because it is not scheduled to run until the wait is satisfied.
When a message is posted to the waiting thread's message queue, or another thread sends a message to a window belonging to the waiting thread, or a timer in the waiting thread fires, or a window in the waiting thread is ready to be painted, the event is signaled, the thread wakes up, and GetMessage() acts accordingly, whether that is to return a posted message to the caller, or to dispatch a sent message directly to the target window procedure and then go back to waiting on the event.
You can see the implementation leaking into the public API if you compare the maximum number of objects you can wait on in WaitForMultipleObjects and MsgWaitForMultipleObjects:
The maximum number of object handles is MAXIMUM_WAIT_OBJECTS.
vs
The maximum number of object handles is MAXIMUM_WAIT_OBJECTS minus
one.
I remember there was a way to do this, something similar to unix signals, but not so widely used. But can't remember the term. No events/mutexes are used: the thread is just interrupted at random place, the function is called and when it returns, the thread continues.
Windows has Asynchronous Procedure Calls which can call a function in the context of a specific thread. APC's do not just interrupt a thread at a random place (that would be dangerous - the thread could be in the middle of writing to a file or obtaining a lock or in Kernel mode). Instead an APC will be dispatched when the calling thread enters an alterable wait by calling a specific function (See the APC documentation).
If the reason that you need to call code in a specific thread is because you are interacting with the user interface, it would be more direct to send or post a window message to the window handle that you want to update. Window messages are always processed in the thread that created the window.
you can search RtlRemoteCall, it's an undocumented routine though. there's APC in Windows semantically similar to Unix signal, however APC requires target thread is in an alertable state to get delivered, it's not guaranteed this condition is always met
I am a bit confused as to what actually happens when an IO completion port completes.
I presume that the Win API allows access to an IOCP queue that somehow is able to queue (or stack) a callback reference with a specific handle (let's say a socket).
When windows receives an interrupt from the NIC, then it at some point gets to the IOCP queue for the NIC and executes the callbacks on its own (IOCP) thread pool.
My question is, is this thread from the thread pool spawned upon the interrupt being received, or is it in fact spawned when the call to the Win API is made, effectively having the thread in a wait state until it is then woken by the IOCP queue?
EDIT:
I found this: http://msmvps.com/blogs/luisabreu/archive/2009/06/04/multithreading-i-o-and-the-thread-pool.aspx where is states: "Whenever that operation completes, it will queue a packet on that I/O completion port. The port will then proceed and use one of the thread pool’s thread to run the callback you’ve specified."
It's probably easier to think of an I/O completion port simply as a thread safe queue that the operating system places the results of overlapped operations into for you when they have completed.
You create the IOCP, you then create some threads and these threads call a function to remove items from this queue. Generally this is GetQueuedCompletionStatus(). This function essentially blocks your thread until there's something in the IOCP (queue) and then allows your thread to retrieve that something and run.
You associate file handles and sockets with the IOCP and this simply means that once associated their overlapped completions will be placed in the IOCP (queue) for you.
It's more complex than that, but that's the way you should be thinking.
Why do we need Mutex and Events in Windows? In the sense couldn't windows have just Mutex? What is that can be done with Events that cannot be done with Mutex?
Events allows threads to block until some event (hence the name) is broadcast. Blocking on an Event means "Wake me when something happened"; you expect to be put to sleep. Event's are a signalling mechanism and provides support for this not found on mutexes, such as automatically being able to clear the signal as soon as someone who waited on it was woken up. Also, the API allows for blocking until one of or all of several events are signalled.
The mutex (Mutual Exclusion), on the other hand, is a scoped coordination mechanism for shared resources. Think transaction. You're not expecting to wait but want to access some shared resource, and only in the event that others are already accessing it, you're blocking.
If you tried to simulate an Event using a mutex, you'd face the problem that as soon as you acquired the lock (when should mean "event signalled"), you're keeping everybody else out until you release that lock. That is not the semantics of signalling an event; it may remain posted, and the "gates" would be open for every thread testing for the event, without acquiring any locks.
Mutex dedicated for interprocess synchronization. This is kernel-mode object.
Events for multithreaded synchronization within one process. This is user-mode object.
Mutex object is very general and to heavy, on the other hand Event object is much more light. In most of situations you must to use user-mode synchronization, because it supplies less CPU cycles.