I want to know which threads processes device interrupts. What happens when there is a interrupt when a user mode thread is running? Also do other user threads get a chance to run when the system is processing an interrupt?
Kindly suggest me some reference material describing how interrupts are handled by windows.
Device interrupts themselves are (usually) processed by whatever thread had the CPU that took the interrupt, but in a ring 0 and at a different protection level. This limits some of the actions an interrupt handler can take, because most of the time the current thread will not be related to the thread that is waiting for the event to happen that the interrupt is indicating.
The kernel itself is closed source, and only documented through its internal API. That API is exposed to device driver authors, and described in the driver development kits.
Some resources to get you started:
Any edition of Microsoft Windows Internals by Solomon and Russinovich. The current seems to be the 4th edition, but even an old edition will help.
The Windows DDK, now renamed the WDK. Its documentation is available online too. Be sure to read the Kernel Mode Design Guide...
Sysinternals has tools and articles to probe at and explain the kernel's behavior. This used to be an independent site until Microsoft got tired of Mark Russinovich seeming to know more about how the kernel worked than they did. ;-)
Note that source code to many of the common device drivers are included in the DDK in the samples. Although the production versions are almost certainly different, reading the sample drivers can answer some questions even if you don't want to implement a driver yourself.
Like any other operating system, Windows processes interrupts in Kernel mode, with an elevated Interrupt Priority Level (I think they call them IRPL's, but I don't know what the "R" stands for). Any user thread or lower-level Kernel thread running on the same machine will be interrupted while the interrupt request is processed, and will be resumed when the ineterrupt processing is complete.
In order to learn more about device interrupts on Windows you need to study device driver development. This is a niche topic, I don't think you can find many useful resources in the Web and you may have to look for a book or a training course.
Anyway, Windows handle interrupts with Interrupt Request Levels (IRQLs) and Deferred procedure calls. An interrupt is handled in Kernel mode, which runs in higher priority than user mode. A proper interrupt handler needs to react very quickly. It only performs the absolutely necessary operations and registers a Deferred Procedure Call to run in the future. This will happen, when the system is in a Interrupt Request Level.
Related
I am programming a pci device with verilog and also writing its driver,
I have probably inserted some bug in the hardware design and when i load the driver with insmod the kernel just gets stuck and doesnt respond. Now Im trying to figure out what's the last driver code line that makes my computer stuck. I have inserted printk in all relevant functions like probe and init but non of them get printed.
What other code is running when i use insmod before it gets to my init function? (I guess the kernel gets stuck over there)
printks are often not useful debugging such a problem. They are buffered sufficiently that you won't see them in time if the system hangs shortly after printk is called.
It is far more productive to selectively comment out sections of your driver and by process of elimination determine which line is the (first) problem.
Begin by commenting out the entire module's init section leaving only return 0;. Build it and load it. Does it hang? Reboot system, reenable the next few lines (class_create()?) and repeat.
From what you are telling, it is looks like that Linux scheduler is deadlocking by your driver. That's mean that interrupts from the system timer doesn't arrive or have a chance to be handled by kernel. There are two possible reasons:
You hang somewhere in your driver interrupt handler (handler starts its work but never finish it).
Your device creates interrupts storm (Device generates interrupts too frequently as a result your system do the only job -- handling of your device interrupts).
You explicitly disable all interrupts in your driver but doesn't reenable them.
In all other cases system will either crash, either oops or panic with all appropriate outputs or tolerate potential misbehavior of your device.
I guess that printk won't work for such extreme scenario as hang in kernel mode. It is quite heavy weight and due to this unreliable diagnostic tool for scenarios like your.
This trick works only in simpler environments like bootloaders or more simple kernels where system runs in default low-end video mode and there is no need to sync access to the video memory. In such systems tracing via debugging output to the display via direct writing to the video memory can be great and in many times the only tool that can be used for debugging purposes. Linux is not the case.
What techniques can be recommended from the software debugging point of view:
Try to review you driver code devoting special attention to interrupt handler and places where you disable/enable interrupts for synchronization.
Commenting out of all driver logic with gradual uncommenting can help a lot with localization of the issue.
You can try to use remote kernel debugging of your driver. I advice to try to use virtual machine for that purposes, but I'm not aware about do they allow to pass the PCI device in the virtual machine.
You can try the trick with in-memory tracing. The idea is to preallocate the memory chunk with well known virtual and physical addresses and zeroes it. Then modify your driver to write the trace data in this chunk using its virtual address. (For example, assign an unique integer value to each event that you want to trace and write '1' into the appropriate index of bytes array in the preallocated memory cell). Then when your system will hang you can simply force full memory dump generation and then analyze the memory layout packed in the dump using physical address of the memory chunk with traces. I had used this technique with VmWare Workstation VM on Windows. When the system had hanged I just pause a VM instance and looked to the appropriate .vmem file that contains raw memory latout of the physical memory of the VM instance. Not sure that this trick will work easy or even will work at all on Linux, but I would try it.
Finally, you can try to trace the messages on the PCI bus, but I'm not an expert in this field and not sure do it can help in your case or not.
In general kernel debugging is a quite tricky task, where a lot of tricks in use and all they works only for a specific set of cases. :(
I would put a logic analyzer on the bus lines (on FPGA you could use chipscope or similar). You'll then be able to tell which access is in cause (and fix the hardware). It will be useful anyway in order to debug or analyze future issues.
Another way would be to use the kernel crash dump utility which saved me some headaches in the past. But depending your Linux distribution requires installing (available by default in RH). See http://people.redhat.com/anderson/crash_whitepaper/
There isn't really anything that is run before your init. Bus enumeration is done at boot, if that goes by without a hitch the earliest cause for freezing should be something in your driver init AFAIK.
You should be able to see printks as they are printed, they aren't buffered and should not get lost. That's applicable only in situations where you can directly see kernel output, such as on the text console or over a serial line. If there is some other application in the way, like displaying the kernel logs in a terminal in X11 or over ssh, it may not have a chance to read and display the logs before the computer freezes.
If for some other reasons the printks still do not work for you, you can instead have your init function return early. Just test and move the return to later in the init until you find the point where it crashes.
It's hard to say what is causing your freezes, but interrupts is one of those things I would look at first. Make sure the device really doesn't signal interrupts until the driver enables them (that includes clearing interrupt enables on system reset) and enable them in the driver only after all handlers are registered (also, clear interrupt status before enabling interrupts).
Second thing to look at would be bus master transfers, same thing applies: Make sure the device doesn't do anything until it's asked to and let the driver make sure that no busmaster transfers are active before enabling busmastering at the device level.
The fact that the kernel gets stuck as soon as you install your driver module makes me wonder if any other driver (built in to kernel?) is already driving the device. I made this mistake once which is why i am asking. I'd look for the string "kernel driver in use" in the output of 'lspci' before installing the module. In any case, your printk's should be visible in dmesg output.
in addition to Claudio's suggestion, couple more debug ideas:
1. try kgdb (https://www.kernel.org/doc/htmldocs/kgdb/EnableKGDB.html)
2. use JTAG interfaces to connect to debug tools (these i think vary between devices, vendors so you'll have to figure out which debug tools you need to the particular hardware)
From what I understand, a thread that executes in user mode can eventually enter code that switches to kernel mode (using sysenter). But how can a thread that's been emanating from user code execute kernel code?
Eg: I'm calling CreateFile(), it then delegates to NtCreateFile(), which in turn calls ZwCreateFile(), than ZiFastSystemCall()... than sysenter... profit, kernel access?
Edit
This question:
How does Windows protect transition into kernel mode has an answer that helped me understand, see quote:
"The user mode thread is causing an exception that's caught by the Ring 0 code. The user mode thread is halted and the CPU switches to a kernel/ring 0 thread, which can then inspect the context (e.g., call stack & registers) of the user mode thread to figure out what to do." Also see this blog post, very informative: http://duartes.org/gustavo/blog/post/cpu-rings-privilege-and-protection
The short answer is that it can't.
What happens is that when you create a user-mode thread, the kernel creates a matching kernel mode thread. When "your" thread needs to execute some code in kernel mode, it's actually executed in the matching kernel mode thread.
Disclaimer: The last time I really looked closely at this was probably with Win2K or maybe even NT4 -- but I doubt much has changed in this respect.
I'm writing to a user-space buffer from a kernel-level driver (from the IOControl functionality) and I need to make sure the user-land program/service won't overwrite the buffer or read it before the driver has finished writing to it.
Is there a way (and if so, what is the preferred way) to enter a kind of 'global critical section' within a kernel-mode driver on Windows allowing a driver to obtain exclusivity for processing system-wide for a short time so that the driver can have guaranteed exclusive access to a buffer in user-space?
Taking into account your reply in comments, one way to achieve that is to maintain kernel-mode threads affinitized to each system processor and raise their IRQL to DPC at the time when you write to the buffer. Thread scheduling is not allowed at DPC IRQL so the user-mode application won't be able to take control.
Note: this is the answer to the question, but basically I agree with the comments saying that you are not supposed to do that. You should probably redesign the driver so that it works under the assumption that user-mode buffer can change at any moment.
I have intensive processing that I need to perform in a device driver, at DISPATCH_LEVEL or lower IRQL.
How do I create a kernel-thread?
What IRQL does it run at? Can I control this?
How is it scheduled? Because I am thinking from a user-mode perspective here, what priority does it run at?
What kernel functions can I use to provide locking / synchronization?
you can create system thread with this As you can see one of its parameters is a start routine which can hold custom code - in it you can use KeRaiseIrql and KeLowerIrql. By default threads will run in PASSIVE_LEVEL. "Locks, Deadlocks, and Synchronization" is a very helpful paper regarding synchronization in kernel on windows and everyone who has to do some tinkering with the windows kernel should read or at least skim it
There's this WIN32 process, which someone says:
The servide takes a lock on the kernel
object and does not release. After a
while the machine becomes irresponsive
and has to restarted. Restanting only
the service won't fix the issue.
According to my knowledge applications were not supposed to be able to crash the entire system in windows. Does anyone know if such an indecent behavior (taking a lock on the kernel object and not releasing it) can indeed crash the OS? If so why don't we see this more in malicious software?
Some clarifications:
This is not a device driver.
Any amiguity left in the reponse is also ambiguous to me. Please elaborate on the cases you can think of.
This doesn't mean anything. There is no "kernel object" in NT, and any lock you could possibly take would be released if the service were restarted.
This depends on what type of application it is. Some applications install and use kernel drivers as part of their usage. A kernel driver has the most low level access possible in the system and is capable of crashing or hanging the system. If the process uses a kernel driver, and the description alludes to this, then yes it can crash / hang the system.
I believe Windows Vista started limiting the amount of damage a kernel driver can accidentally do (graphics drivers especially). But intentionally, you can still cause lots of problems.
Depending on which precise kernel object they mean, and which service, this may very well be true. See for instance Raymond Chen on Loader Lock, a kernel lock which applications can monopolize. Restarting the service will then become a problem because the very unload of that service will require the loader lock, too.