I am actually reading Windows Internals 5th edition and i am enjoying, although isn't a easy book to read and understand.
I am confused about IRQLs and IDT Table.
I read that windows implement custom priorization levels with IRQL and the Plug and Play Manager maps IRQ from devices to IRQL.
Alright, so, IRQLs are used for Software and Hardware interrupts, and for exceptions is used the Exception Dispatch handler.
When one device generates an interrupt, the interrupt controller pass this information to the CPU with the IRQ.
So Windows takes this IRQ and translates to IRQL to schedule when to execute the routine (routine that IDT[IRQ_VALUE] is pointing to?
Is that what is happening?
Yes, on a very high level.
Everything starts with a kernel trap. Kernel trap handler handles interrupts, exceptions, system service calls and virtual memory pager.
When an interrupt happens (line based - using dedicated pin or message based- writing to an address) windows uses IRQL to determine the priority of the interrupt and uses this to see if the interrupt can be served or not during that time. HAL does the job of translating the IRQ to IRQL.
It then uses IRQ to get an index of the IDT to find the appropriate ISR routing to invoke. Note there can be multiple ISR associated for a given IRQ. All of them execute in order.
Each processor has its own IDT so you could potentially have multiple ISR's running at the same time.
Exception dispatch, as I mentioned before, is also handled by the kernel trap but the procedure for it is different. It usually starts by checking for any exception handlers by stack unwinding, then checking for debugger port etc.
Related
When we use irq_set_chained_handler the irq line will not be disabled or not, when we are servicing the associated handler, as in case of request_irq.
It doesn't matter how the interrupt was setup. When any interrupt occurred, all interrupts (for this CPU) will be disabled during the interrupt handler. For example, on ARM architecture first place in C code where interrupt handling is found is asm_do_IRQ() function (defined in arch/arm/kernel/irq.c). It's being called from assembler code. For any interrupt (whether it was requested by request_irq() or by irq_set_chained_handler()) the same asm_do_IRQ() function is called, and interrupts are disabled automatically by ARM CPU. See this answer for details.
Historical notes
Also, it worth to be mentioned that some time ago Linux kernel was providing two types of interrupts: "fast" and "slow" ones. Fast interrupts (when using IRQF_DISABLED or SA_INTERRUPT flag) were running with disabled interrupts, and those handlers supposed to be very short and quick. Slow interrupts, on the other hand, were running with re-enabled interrupts, because handlers for slow interrupts may take much of time to be handled.
On modern versions of Linux kernel all interrupts are considered as "fast" and are running with interrupts disabled. Interrupts with huge handlers must be implemented as threaded (or enable interrupts manually in ISR using local_irq_enable_in_hardirq()).
That behavior was changed in Linux kernel v2.6.35 by this commit. You can find more details about this here.
Refer https://www.kernel.org/doc/Documentation/gpio/driver.txt
This means the GPIO irqchip is registered using
irq_set_chained_handler() or the corresponding
gpiochip_set_chained_irqchip() helper function, and the GPIO irqchip
handler will be called immediately from the parent irqchip, while
holding the IRQs disabled. The GPIO irqchip will then end up calling
something like this sequence in its interrupt handler:
I am confused between above mentioned concepts while reading Windows internals.
All three terms - trap handler , interrupt dispatch routine and interrupt service routine (ISR) - relate to Windows driver level programming (as opposed to user-mode Windows applications).
"Traps" are programmer-initiated interrupts (vs. automatically generated "exceptions").
An "Interrupt service routine" (ISR) is a procedure written to handle an "interrupt". Although there are different kinds of interrupts (hardware interrupts, programmatic traps, CPU exceptions, etc, etc), the format of an ISR is similar in all cases. A "trap handler" is an ISR.
Interrupts should always be serviced as quickly as possible.
Finally "Dispatch routines" are the main entry points for performing hardware I/O.
Is it possible to run multiple instances of a same interrupt simultaneously on a multi processor system in linux?
If not possible, why do we need to synchronize between interrupt handlers using spin locks?
Thanks
Venkatesh
On a SMP architecture Advanced Programmable Interrupt Controller(APIC) is used to route the interrupts from peripherals to the CPU's.
the APIC, based on
1. the routing table (where interrupt affinity is set to a particular processor),
2. priority of the interrupt,
3. the load on the CPU's
For example, consider a interrupt is received at IRQ line 32, this goes through APIC,the interrupt is routed to a particular CPU, for now consider CPU0, this interrupt line is masked until the ISR is handled, which means you will not get a interrupt of the same type if ISR execution is in progress
Once ISR is handled, only then the interrupt line is unmasked for future interrupts
Is it possible to run multiple instances of a same interrupt simultaneously on a multi processor system in linux?
The interrupt handlers are generally serialized. Meaning, that only one instance of the handler would be running(on either of the processors). While this is running, if same type of interrupt is again generated, it is processed only after the current one is done, thus serialized. While "this" handler is being executed by one of the core, other core might service handler of a different instance.
Why do we need to synchronize between interrupt handlers using spin locks?
The spinlocks are used even in such cases as the data has to be protected against some other threads(for example bottom halved, user read/write handler functions, etc).
The scenario could be something like this :
my_ISR()
{
lock(&l);
// data is accessed here
unlock(&l);
}
my_other_thread()
{
lock(&l);
// same data is accessed here
unlock(&l);
}
Our group is using a custom driver to interface four MAX3107 UARTs on a shared I2C bus. The interrupts of the four MAX3107's are connected (i.e. shared interrupt via logic or'ing)) to a GPIO pin on the ARM9 processor (LPC3180 module). When one or more of these devices interrupt, they pull the GPIO line, which is configured as a level-sensitive interrupt, low. My question concerns the need, or not, to disable the specific interrupt line in the handler code. (I should add that we are running Linux 2.6.10).
Based on my reading of several ARM-specific app notes on interrupts, it seems that when the ARM processor receives an interrupt, it automatically disables (masks?) the corresponding interrupt line (in our case this would seem to be the line corresponding to the GPIO pin we selected). If this is true, then it seems that we should not have to disable interrupts for this GPIO pin in our interrupt handler code as doing so would seem redundant (though it seems to work okay). Stated differently, it seems to me that if the ARM processor automatically disables the GPIO interrupt upon an interrupt occurring, then if anything, our interrupt handler code should only have to re-enable the interrupt once the device is serviced.
The interrupt handler code that we are using includes disable_irq_nosync(irqno); at the very beginning of the handler and a corresponding enable_irq() at the end of the handler. If the ARM processor has already disabled the interrupt line (in hardware), what is the effect of these calls (i.e. a call to disable_irq_nosync() followed by a call to enable(irq())?
From the Arm Information Center Documentation:
On entry to an exception (interrupt):
interrupt requests (IRQs) are disabled for all exceptions
fast interrupt requests (FIQs) are disabled for FIQ and Reset exceptions.
It then goes on to say:
Handling an FIQ causes IRQs and subsequent FIQs to be disabled,
preventing them from being handled until after the FIQ handler enables
them. This is usually done by restoring the CPSR from the SPSR at the
end of the handler.
So you do not have to worry about disabling them, but you do have to worry about re-enabling them.
You will need to include enable_irq() at the end of your routine, but you shouldn't need to disable anything at the beginning. I wouldn't think that calling disable_irq_nosync(irqno) in software after it has been called in hardware would effect anything. Since the hardware call is most definitely called before the software call has a chance to take over. But it's probably better to remove it from the code to follow convention and not confuse the next programmer who takes a look at it.
More info here:
Arm Information Center
Does the CPU disable all interrupts on local CPU before calling the interrupt handler?
Or does it only disable that particular interrupt line, which is being served?
x86 disables all local interrupts (except NMI of course) before jumping to the interrupt vector. Linux normally masks the specific interrupt and re-enables the rest of the interrupts (which aren't masked), unless a specific flags is passed to the interrupt handler registration.
Note that while this means your interrupt handler will not race with itself on the same CPU, it can and will race with itself running on other CPUs in an SMP / SMT system.
Normally (at least in x86), an interrupt disables interrupts.
When an interrupt is received, the hardware does these things:
1. Save all registers in a predetermined place.
2. Set the instruction pointer (AKA program counter) to the interrupt handler's address.
3. Set the register that controls interrupts to a value that disables all (or most) interrupts. This prevents another interrupt from interrupting this one.
An exception is NMI (non maskable interrupt) which can't be disabled.
Yes, that's fine.
I'd like to also add what I think might be relevant.
In many real-world drivers/kernel code, "bottom-half" (bh) handlers are used pretty often- tasklets, softirqs. These bh's run in interrupt context and can run in parallel with their top-half (th) handlers on SMP (esp softirq's).
Of course, recently there's a move (mainly code migrated from the PREEMPT_RT project) towards mainline, that essentially gets rid of the 'bh' mechanism- all interrupt handlers will run with all interrupts disabled. Not only that, handlers are (can be) converted to kernel threads- these are the so-called "threaded" interrupt handlers.
As of today, the choice is still left to the developer- you can use the 'traditional' th/bh style or the threaded style.
Ref and Details:
http://lwn.net/Articles/380931/
http://lwn.net/Articles/302043/
Quoting Intels own, surprisingly well-written "Intel® 64 and IA-32 Architectures Software Developer’s Manual", Volume 1, pages 6-10:
If an interrupt or exception handler is called
through an interrupt gate, the processor clears the interrupt enable (IF) flag in the EFLAGS register to prevent
subsequent interrupts from interfering with the execution of the handler. When a handler is called through a trap
gate, the state of the IF flag is not changed.
So just to be clear - yes, effectively the CPU "disables" all interrupts before calling the interrupt handler. Properly described, the processor simply triggers a flag which makes it ignore all interrupt requests. Except probably non-maskable interrupts and/or its own software exceptions (please someone correct me on this, not verified).
We want ISR to be atomic and no one should be able to preempt the ISR.
Therefore, An ISR disables the local interrupts ( i.e. the interrupt on the current processor) and once the ISR calls ret_from_intr() function ( i.e. we have finished the ISR) , interrupts are again enabled on the current processor.
If an interrupt occurs, it will now be served by the other processor ( in SMP system) and ISR related to that interrupt will start running.
In SMP system , We also need to include the proper synchronization mechanism ( spin lock) in an ISR.