I read this article http://www.linuxjournal.com/article/5833 to learn about spinlock. I try this to use it in my kernel driver.
Here is what my driver code needs to do:
In f1(), it will get the spin lock, and caller can call f2() will wait for the lock since the spin lock is not being unlock. The spin lock will be unlock in my interrupt handler (triggered by the HW).
void f1() {
spin_lock(&mylock);
// write hardware
REG_ADDR += FLAG_A;
}
void f2() {
spin_lock(&mylock);
//...
}
The hardware will send the application an interrupt and my interrupt handler will call spin_unlock(&mylock);
My question is if I call
f1()
f2() // i want this to block until the interrupt return saying setting REG_ADDR is done.
when I run this, I get an exception in kernel saying a deadlock " INFO: possible recursive locking detected"
How can I re-write my code so that kernel does not think I have a deadlock?
I want my driver code to wait until HW sends me an interrupt saying setting REG_ADDR is done.
Thank you.
First, since you'll be expecting to block while waiting for the interrupt, you shouldn't be using spinlocks to lock the hardware as you'll probably be holding the lock for a long time. Using a spinlock in this case will waste a lot of CPU cycles if that function is called frequently.
I would first use a mutex to lock access to the hardware register in question so other kernel threads can't simultaneously modify the register. A mutex is allowed to sleep so if it can't acquire the lock, the thread is able to go to sleep until it can.
Then, I'd use a wait queue to block the thread until the interrupt arrives and signals that the bit has finished setting.
Also, as an aside, I noticed you're trying to access your peripheral by using the following expression REG_ADDR += FLAG_A;. In the kernel, that's not the correct way to do it. It may seem to work but will break on some architectures. You should be using the read{b,w,l} and write{b,w,l} macros like
unsigned long reg;
reg = readl(REG_ADDR);
reg |= FLAG_A;
writel(reg, REG_ADDR);
where REG_ADDR is an address you obtained from ioremap.
I will agree with Michael that Spinlock, Semaphores, Mutex ( Or any other Locking Mechanisms) must be used when any of the resources(Memory/variable/piece of code) has the probability of getting shared among the kernel/user threads.
Instead of using any of the Locking primitives available I would suggest using other sleeping functionalities available in kernel like wait_event_interruptibleand wake_up. They are simple and easy to exploit them into your code. You can find its details and exploitation on net.
Related
What would happen if I use semaphore and mutex locks in interrupt context?
Normally semaphore is used in synchronization mechanism. What would happen if I use this one in an interrupt context?
I am working on a project on gpio pins and when interrupt happens, I have to send one signal in ISR. I am using spinlocks.
What would happen if I use semaphore and mutext in ISR?
Waiting in mutexes and semaphores are implemented using switching current task state to TASK_INTERRUPTIBLE/TASK_UNINTERRUPTIBLE and similar with futher call to schedule().
Calling schedule() with current task state differed from TASK_RUNNING leads to switching to another process. And if current refers to interrupt context, you will never return back to it, because scheduling can switch only to the process.
So, when you lock contended(that is, currently locked) semaphore/mutex in interrupt context, you just lost current execution "thread".
If you lock semaphore/mutex which is uncontended(currently not locked), execution will be correct except warning in system log about improper semaphore/mutex usage.
In Linux Kernel Development book (Robert Love), It is mentioned that :
we must disable local interrupts before obtaining spinlock in
interrupt handler. Otherwise it is possible for an interrupt handler
to interrupt kernel code while the lock is held and attempt to
re-acquire the lock. Which finally can lead to double-acquire
deadlock.
Now my doubt is:
In general, doesn't do_IRQ() disables local interrupt ?
And if lock is acquire, it means thatpreempt_count variable is not zero, which makes that no other handler should get chance, as kernel is not preempt_safe. So how other interrupt handler can work in this situation ?
First, the do_IRQ() function dosn't disable the local interrupt, but some function written in assembly language does, which is the interrupt entrance. And later, before executing the interrupt function registering by request_irq(), in function handle_IRQ_event() a flag which also pass by request_irq() is compare with IRQF_DISABLED to determine whether we should enable the local interrupt when executing the interrupt function. So the answer to your question one is depending on the flags that you pass to the request_irq() function.
Second, preempt_count just means for kernel preemption in process context, but not for interrupt. To avoid interrupt handlers be executed in UP, the only way is involving the irqs_disable(). When the preempt_count is zero, it's said that the kernel can safely does the process switch, otherwise not.
First of all sorry for a little bit ambiguity in Question... What I want to understand is the below scenario
Suppose porcess is running, it holds one lock, Now after acquiring the lock HW interrupt is generated, So How kernel will handle this situation, will it wait for lock ? if yes, what if the interrupt handler need to access that lock or the shared data protected by that lock in process ?
The Linux kernel has a few functions for acquiring spinlocks, to deal with issues like the one you're raising here. In particular, there is spin_lock_irq(), which disables interrupts (on the CPU the process is running on) and acquires the spinlock. This can be used when the code knows interrupts are enabled before the spinlock is acquired; in case the function might be called in different contexts, there is also spin_lock_irqsave(), which stashes away the current state of interrupts before disabling them, so that they can be reenabled by spin_unlock_irqrestore().
In any case, if a lock is used in both process and interrupt context (which is a good and very common design if there is data that needs to be shared between the contexts), then process context must disable interrupts (locally on the CPU it's running on) when acquiring the spinlock to avoid deadlocks. In fact, lockdep ("CONFIG_PROVE_LOCKING") will verify this and warn if a spinlock is used in a way that is susceptible to the "interrupt while process context holds a lock" deadlock.
Let me explain some basic properties of interrupt handler or bottom half.
A handler can’t transfer data to or from user space, because it doesn’t execute in the context of a process.
Handlers also cannot do anything that would sleep, such as calling wait_event, allocating memory with anything other than GFP_ATOMIC, or locking a semaphore
handlers cannot call schedule.
What i am trying to say is that Interrupt handler runs in atomic context. They can not sleep as they cannot be rescheduled. interrupts do not have a backing process context
The above is by design. You can do whatever you want in code, just be prepared for the consequences
Let us assume that you acquire a lock in interrupt handler(bad design).
When an interrupt occur the process saves its register on stack and start ISR. now after acquiring a lock you would be in a deadlock as their is no way ISR know what the process was doing.
The process will not be able to resume execution until it is done it with ISR
In a preemptive kernel the ISR and the process can be preempt but for a non-preemptive kernel you are dead.
Imagine that a device function holds a spinlock to control access to the device. While the lock is held, the device issues an interrupt, which causes an interrupt handler to run. The interrupt handler, before accessing the device, must also obtain the lock.
Suppose that the interrupt handler executes in the same processor as the code that took out the lock originally.
Knowing that to hold spinlock disables preemption on the relevant processor, is it possible that the code that holds the spinlock be executed on another processor (because of preemption on this processor)? (We suppose that this is a SMP machine)
Is it possible that the code that holds the spinlock be executed on another processor (because of preemption on this processor)?
No, the code just keeps waiting for the interrupt handler to return.
Just use spin_lock_irq*(), or spin_lock_bh() if you also want to protect against softirqs/tasklets.
I've recently read section 5.5.2 (Spinlocks and Atomic Context) of LDDv3 book:
Avoiding sleep while holding a lock can be more difficult; many kernel functions can sleep, and this behavior is not always well documented. Copying data to or from user space is an obvious example: the required user-space page may need to be swapped in from the disk before the copy can proceed, and that operation clearly requires a sleep. Just about any operation that must allocate memory can sleep; kmalloc can decide to give up the processor, and wait for more memory to become available unless it is explicitly told not to. Sleeps can happen in surprising places; writing code that will execute under a spinlock requires paying attention to every function that you call.
It's clear to me that spinlocks must always be held for the minimum time possible and I think that it's relatively easy to write correct spinlock-using code from scratch.
Suppose, however, that we have a big project where spinlocks are widely used.
How can we make sure that functions called from critical sections protected by spinlocks will never sleep?
Thanks in advance!
What about enabling "Sleep-inside-spinlock checking" for your kernel ? It is usually found under Kernel Debugging when you run make config. You might also try to duplicate its behavior in your code.
One thing I noticed on a lot of projects is people seem to misuse spinlocks, they get used instead of the other locking primitives that should have be used.
A linux spinlock only exists in multiprocessor builds (in single process builds the spinlock preprocessor defines are empty) spinlocks are for short duration locks on a multi processor platform.
If code fails to aquire a spinlock it just spins the processor until the lock is free. So either another process running on a different processor must free the lock or possibly it could be freed by an interrupt handler but the wait event mechanism is much better way of waiting on an interrupt.
The irqsave spinlock primitive is a tidy way of disabling/ enabling interrupts so a driver can lock out an interrupt handler but this should only be held for long enough for the process to update some variables shared with an interrupt handler, if you disable interupts you are not going to be scheduled.
If you need to lock out an interrupt handler use a spinlock with irqsave.
For general kernel locking you should be using mutex/semaphore api which will sleep on the lock if they need to.
To lock against code running in other processes use muxtex/semaphore
To lock against code running in an interrupt context use irq save/restore or spinlock_irq save/restore
To lock against code running on other processors then use spinlocks and avoid holding the lock for long.
I hope this helps