I'm using some global variables in iokit based driver i.e. outside the main class instance. However, this cause some unexpected panics upon driver startup due to using uninitialize global variables or attempt to double free
a global variable upon teardown.
What is the life cycles of global variables in iokit driver lifecycle ?
if i set a global variable upon declaration,
For example, if i've got global variable from type lck_grp_t * my_lock_grp...
Can i assume my global variable is already allocated and ready to be set when my iokit driver reaches the ::start method ?
(my_lock_grp = lck_grp_alloc_init("my-locks", my_lock_grp_attr);)
Can i assume my global variable is still valid when I attempt to release it on my iokit ::free method? (lck_grp_free(my_lock_grp))
And the general question is what is the life-cycle of global variables in iokit based driver compared to the driver instance itself.
The lifetime will definitely be the same as the lifetime of the kext. IOKit init/start/stop/free functions on your classes will be happening between the kext start and stop functions (you may not have explicit kext start & stop functions), and global constructors are run before the kext start function, and likewise global destructors are run after the kext stop function. The memory allocation/deallocation for global/static variables is done by the dynamic kernel linker at the same time as the kext's code itself is loaded and unloaded.
There are 3 things I can think of:
The IOService start() and free() functions are not matched - free() is called even if start() was never called. So for example if you have a probe() function, and this is called and returns nullptr, then start() is never called, but free() definitely will be, and it tries to free a lock group that was never allocated. Similarly if an init() function returned false - start() will never run, but free() will. The equivalent of free() is the init() family of member functions, so only unconditionally destroy (no nullptr check) in free() what is unconditionally created in all possible init… functions.
start() can be called any number of times on different instances, so if you always run my_lock_grp = lck_grp_alloc_init() in start() and 2 instances are created, my_lock_grp only remembers the last one, so if then both instances of your class are freed, you end up trying to free one lock group twice and the other not at all. This is bad news obviously. For initialising/destroying truly global state, I recommend using kext start and stop functions or global constructors/destructors.
Otherwise I suspect you might be running into a situation where some other part of the running kernel still has a dangling reference past the point where your kext has already been unloaded, for example if you created a new kernel thread and this thread is still running, or if you've not deregistered all callbacks you registered, or if a callback has been deregistered but is not guaranteed to have completed all invocations. (kauth listeners are notorious for this latter situation)
If none of those sound like they might be the problem, I suggest posting the affected code and the panic log, maybe we can make more sense of the problem if we have some hard data.
Related
I wrote an interrupt handler in linux.
Part of the handler logic I need to access physical address.
I used iormap function but then I fell into KDB during handler time.
I started to debug it and i saw the below code which finally called by ioremap
What should I do? Is there any other way instead of map the region before?
If i will need to map it before it means that i will probably need to map and cache a lot of unused area.
BTW what are the limits for ioremap?
Setting up a new memory mapping is an expensive operation, which typically requires calls to potentially blocking functions (e.g. grabbing locks). So your strategy has two problems:
Calling a blocking function is not possible in your context (there is no kernel thread associated with your interrupt handler, so there is no way for the kernel to resume it if it had to be put to sleep).
Setting up/tearing down a mapping per IRQ would be a bad idea performance-wise (even if we ignore the fact that it can't be done).
Typically, you would setup any mappings you need in a driver's probe() function (or in the module's init() if it's more of a singleton thing). This mapping is then kept in some private device data structure, which is passed as the last argument to some variant of request_irq(), so that the kernel then passes it back as the second argument to the IRQ handler.
Not sure what you mean by "need to map and cache a lot of unused area".
Depending on your particular system, you may end up consuming an entry in your CPU's MMU, or you may just re-use a broader mapping that was setup by whoever wrote the BSP. That's just the cost of doing business on a virtual memory system.
Caching is typically not enabled on I/O memory because of the many side-effects of both reads and writes. For the odd cases when you need it, you have to use ioremap_cached().
FreeRTOS and ESP-IDF provide xSemaphoreCreateMutex() which allocates and initializes a mutex. Their docs say:
If a mutex is created using xSemaphoreCreateMutex() then the required
memory is automatically dynamically allocated inside the
xSemaphoreCreateMutex() function. (see
http://www.freertos.org/a00111.html).
However, I can't find any info on whether it is necessary to free the memory created by the mutex. This would be important if using C++ with a mutex member variable, like:
class MyClass
{
MyClass::MyClass()
{
mAccessMutex = xSemaphoreCreateMutex();
}
MyClass::~MyClass()
{
// What to do here??
}
SemaphoreHandle_t mAccessMutex;
}
REFERENCE
https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/system/freertos.html?highlight=xsemaphore#semaphore-api
According to FreeRTOS API reference, the proper way to destroy/delete a mutex is vSemaphoreDelete()
Deletes a semaphore, including mutex type semaphores and recursive
semaphores.
Do not delete a semaphore that has tasks blocked on it.
If you're using heap_1, deleting is not possible. Also, make sure that you fully understand the perils of dynamic memory allocation in embedded systems before using it. If MyClass is going to be created and destroyed in a regular/periodic basis, this may cause problems.
So yes, it's necessary to call vSemaphoreDelete(mAccessMutex) in ~MyClass(). But it's probably best to make sure that MyClass instances never get destroyed. In my projects, I generally use one-time dynamic allocation during initialization and forget to implement a proper destructor (which is a bad habit that I need to fix).
I'm making an emulation driver that requires me to call schedule() in ATOMIC contexts in order to make the emulation part work. For now I have this hack that allows me to call schedule() inside ATOMIC (e.g. spinlock) context:
int p_count = current_thread_info()->preempt_count;
current_thread_info()->preempt_count = 0;
schedule();
current_thread_info()->preempt_count = p_count;
But that doesn't work inside IRQs, the system just stops afer calling schedule().
Is there any way to hack the kernel in a way to allow me to do it? I'm using Linux kernel 4.2.1 with User Mode Linux
In kernel code you can be either in interrupt context or in process context.
When you are in interrupt context, you cannot call any blocking function (e.g., schedule()) or access the current pointer. That's related to how the kernel is designed and there is no way for having such functionalities in interrupt context. See also this answer.
Depending on what is your purpose, you can find some strategy that allows you to reach your goal. To me, it sounds strange that you have to call schedule() explicitly instead of relying on the natural kernel flow.
One possible approach follows (but, again, it depends on your specific goal). Form the IRQ you can schedule the work on a work queue through schedule_work(). The work queue, in fact, by design, executes kernel code in process context. From there, you are allowed to call blocking functions and access the current process data.
I guess this is a really nasty issue - seems like one of the property destructors of my class creates deadlock. Property destructors are called automatically after class destructor. And I'd like to call them manually and make a log entry after every single one succeeds.
The problem only occurs on devices, where debugger can't be used, so I am using log instead.
Client::~Client() {
// Stops io service and disconnects sockets
exit();
LOG("io_service stopped"<<endl);
// Destroy IO service
io_.~io_service();
LOG("io_service destroyed"<<endl);
}
But the code above actually causes exception, because the ~io_service() gets called twice.
So is there a way to do this properly? If not, what's an alternative to debugging destructors?
You can't alter the compiler behaviour like that. the compiler will augment the destructor to destruct nested objects.
What you can do is to declare io as a pointer and allocate it dynamically with new. then call delete io and monitor what happens there.
Other solution is just to put a breakpoint on the io destructor and follow what happens there upon destruction. this is probably the best idea.
I have a kernel module that uses hrtimers to notify userspace when the timer has fired. I understand I can just use userspace timers, but it is emulating a driver that will actually talk to hardware in the future. Every once in a while I get a BUG: Scheduling while atomic. After doing some research I am assuming that the hrtimer.function that I register as a callback, is being called from an interrupt routine by the kernel internals (making my callback function in an "Atomic Context"). Then when I call sysfs_notify() within the callback, I get the kernel bug, because sysfs_notify() acquires a mutex.
1) Is this a correct assumption?
If this is correct, I have seen that there is a function called sys_notify_dirent() that I can use to notify userspace from an atomic context. But according to this source:
http://linux.derkeiler.com/Mailing-Lists/Kernel/2009-10/msg07510.html
It can only be called from a "process" context, and not an interrupt context (due to the spinlock).
2) Could someone explain the difference between process, interrupt, and atomic context?
3) If this cannot be used in an interrupt context, what is an alternative to notifying userspace in this context?
Correct, sysfs_notify() cannot be called from atomic context. And yes, sysfs_notify_dirent() appears to be safe to call from atomic context. The source you cite is a bug report that notices in an old kernel version that statement wasn't actually true, along with a patch to fix it. It now appears to be safe to call.
Follow the source code in gpiolib_sysfs.c, and you'll notice that sysfs_notify_dirent() eventually calls schedule_work(), which defers the actual call to sysfs_notify(), which is exactly what the comments to your question are advising you to do. It's just wrapped inside the convenience function.