I have come across a scenario where two drivers (driver1 and driver2) are registering with kernel using module_init(). driver1 is configuring the chip and driver2 is accessing that chip, so driver1's module_init() has to be called before driver2's module_init() call. How we can create this order, as both driver1 and driver2 are registering at the same init call level (i.e using module_init()).
Can driver2 somehow determine that the chip has been properly configured, using either device status (preferred) or the value of a shared variable (messy)?
If so, then driver2's probe routine should return with -EPROBE_DEFER when the chip has not yet been configured (by driver1).
See https://lwn.net/Articles/485194/
drivercore: Add driver probe deferral mechanism
Allow drivers to report at probe time that they cannot get all the resources
required by the device, and should be retried at a later time.
This should completely solve the problem of getting devices
initialized in the right order. Right now this is mostly handled by
mucking about with initcall ordering which is a complete hack, and
doesn't even remotely handle the case where device drivers are in
modules. This approach completely sidesteps the issues by allowing
driver registration to occur in any order, and any driver can request
to be retried after a few more other drivers get probed.
Related
I'm working on an embedded Linux device (based on an NXP i.MX8 mini SoC) and it needs to support microphone audio input using the NXP "micfil" driver (sound/soc/fsl/fsl_micfil.c).
As a part of initializing the microphone, we have added code (to the driver's fsl_micfil_probe function) to set a GPIO line necessary to enable the microphone (by calling devm_gpiod_get_optional)
Our current Linux BSP (based on Yocto's "sumo" release) works great with this patch.
When upgrading to a newer BSP (based on Yocto's "hardknott" release), we find that the GPIO setup call fails. It appears to be happening because the driver's probe function is called before the GPIO subsystem is available. I think we can hack around this by making the GPIO call from some other driver callback (e.g. after an application opens the device), but I would prefer to simply defer this driver's probe operation until after GPIO is available.
Is there a way to do this? To tell Linux that this driver should not be probed until after GPIO starts up? Or maybe have the probe function explicitly call the GPIO startup function?
Or is there a better solution that I haven't thought of yet?
The device driver's probe function can return -EPROBE_DEFER (after any necessary clean-up) to indicate that a resource it requires is not available yet. The driver core will then add the device to a "deferred probe" list and try to call the probe function after a successful probe of some other device.
See the documentation for further information, and heed the warning about not returning -EPROBE_DEFER if any child devices have been created by the probe function.
devm_gpiod_get_optional and similar devm_gpiod_get_ and gpiod_get_ functions that return a struct gpio_desc * value will return ERR_PTR(-EPROBE_DEFER) if the requested GPIO is not yet available. (This is different to the GPIO not being found, which will result in the _optional variants returning NULL and the non-_optional variants returning ERR_PTR(-ENOENT).) The caller can propogate the -EPROBE_DEFER error to its own caller, but again the warning about not returning -EPROBE_DEFER if any child devices have been created should be heeded.
I am writing a driverkit extension whose goal is to block some categories of USB devices, such as flash drives. The driver should block (match to) any device of the relevant device classes, except those, which are whitelisted (based on their vendor and product ID). The whitelist can be set dynamically by user of the application.
The question is, how to pass these data to the driver as reading from a file or something like Windows registry is not available in the DriverKit. The tricky part is that the driver requires the whitelist data before the device is matched.
From what I understood, rejection of device is possible by returning an error from Start() method and returning from it prematurely. I got an idea to send the data while the driver is running this function, however this is not possible as the communication via IOUserClass is not available until the Start method returns.
Is this somehow doable?
As far as I'm aware, communicating with user space apps from the initial Start() method is not possible from DriverKit extensions. As you say, IOUserClients are the mechanism to use for user space communication, and those aren't available until the service is started and registered. You can have your driver match IOResources/IOUserResources so it is always loaded, but each matched service starts up an independed process of your dext, and I'm not aware of a way to directly communicate between these instances.
If I understand you correctly, you're trying to block other drivers from acquiring the device. I don't think the solution you have in mind will help you with this. If you return success from Start(), your dext will drive the device. If you return failure, no driver is loaded for the device, because matching has already concluded. So other drivers would never get a chance anyway, regardless of whether the device is on your allow-list or deny-list.
It's new in DriverKit 21 (i.e. macOS Monterey), and I've not had a chance to try it yet, but there is an API for reading files, OSMappedFile. I would imagine that the DriverKit sandbox will have something to say about which files a dext can open, but this seems like an avenue worth exploring whether you can open configuration files this way.
Note that none of this will help you during early boot, as your dext will never be considered for matching at that time. And you may not be able to get required entitlements from Apple to build a dext which matches USB device classes rather than specific product/vendor ID patterns. (Apologies for repeating myself, but other users may come across this answer and not be aware of this issue.)
I have tried extending IOUserNetworkEthernet and calling RegisterEthernetInterface(). This works perfectly for one ethernet interface, though the driver crashes when RegisterEthernetInterface is called a second time (doesn't return an error code). I have tried registering with separate queues.
Another approach was extending IOUserClient instead, and calling IOService::Create to create child IOUserNetworkEthernet instances. Everything about this approach works (the children appear within ioreg). However, once I call RegisterEthernetInterface on just one of the children, macOS crashes.
How would I go about creating a dext with multiple ethernet interfaces? Have I been approaching it the right way?
Appreciate any help.
I haven't yet implemented an ethernet dext myself, but based on my experience with using DriverKit for other types of drivers and knowing its design goals, I have an idea what the solution might be.
First off, let's clarify: you're implementing either a virtual ethernet device, or you're building a driver for hardware that unites multiple independent ports in one physical device. In the former case (I'm guessing this is what you're doing based on your IOUserClient comment), your driver will be matching IOUserResources and you create the ethernet driver instance when you receive the corresponding message from your user space component.
Now, DriverKit design: DriverKit is built around a goal of keeping the driver instance for each device in its own separate process. Sort of circling back to the microkernel idea. Specifically, this means that multiple devices that use the same driver will each create an independent instance of that driver in its own process. This very likely means that NetworkingDriverKit was never designed to support more than one instance of IOUserNetworkEthernet, because they are seen as separate devices. Hence the crashes.
OK, what do we do about it? Use DriverKit the way it was intended. Put each virtual ethernet adapter in its own driver instance. This gets a bit tricky. I think this should work:
Your dext has a "control" instance. This is the thing that matches IOUserResources. It's a simple IOService class which listens for the signal (presumably IOUserClient) to create or destroy a virtual ethernet device. When creating a virtual ethernet device, you need that to run in its own driver instance though. (For an actual device with multiple ports, this would match the USB/PCI device nub, manage bus communication with the device, and enumerate the ports.)
Instead of creating an instance of a IOUserNetworkEthernet subclass, create an instance of a "nub" class and attach it to your central control class instance. Call RegisterService() in its startup code, so it's considered for IOKit matching.
Set up a second IOKit matching dictionary in your Info.plist, which matches your new "nub" object. Here, use your IOUserNetworkEthernet-derived class to "drive" the nub. Once the virtual ethernet device is ready to use, call RegisterEthernetInterface().
2 extra things to note:
The virtual device will run in a separate process from the control object, so they can only communicate via DriverKit inter-process calls, i.e. user clients. Hopefully, they don't really need to communicate much though, and the control client can pass all the information required via properties on the nub. If you're implementing support for multi-port hardware, you probably won't be able to avoid this part though.
Your user space component (app, daemon) will need to open a new communication channel with the virtual device, so you'll probably need to implement user client support there too.
I'm not sure how you'd go about shutting down an individual virtual ethernet device, you could try calling Terminate() on either it or the nub it's matched on and see what happens.
I have question about device removal.
When we want to notify PnP manager that device has disappeared we call IoInvalidateDeviceRelations with BusRelations. After that OS will send IRP_MN_QUERY_DEVICE_RELATIONS request with BusRelations. In this request handler we will exclude device from array and will do another necessary job to "disconnect" it from bus, also we will set RemovePending flag in its device extension.
I don't understand how to deal with incoming IO requests to the device after it becomes "remove pending" and before OS sends IRP_MN_REMOVE_DEVICE request. Should we check RemovePending flag and return STATUS_DEVICE_DOES_NOT_EXIST or should we proceed as usual?
Now imagine that IRP_MN_REMOVE_DEVICE request has finally arrived. MSDN says that we must call IoReleaseRemoveLockAndWait that releases the current acquisition of remove lock, prevents subsequent acquisitions and waits while existing acquisitions are released. So it forces us to always acquire remove lock inside PnP request handler with IoAcquireRemoveLock; and release it with IoReleaseRemoveLockAndWait for IRP_MN_REMOVE_DEVICE or with IoReleaseRemoveLock for another minor codes.
I don't understand why we need to acquire remove lock inside PnP request handler? In my understanding we need to acquire remove lock only for pending irp and release it when such irp is completed. So Windows folks could provide us with IoWaitForExistingRemoveLocks routine instead of IoReleaseRemoveLockAndWait.
Sorry if it's kinda messy, I just can't figure it out. Thanks.
After that OS will send IRP_MN_QUERY_DEVICE_RELATIONS request with
BusRelations. In this request handler we will exclude device from
array and will do another necessary job to "disconnect" it from bus,
also we will set RemovePending flag in its device extension.
here need only exclude device from array and set RemovePending flag in its device extension. but do another necessary job to "disconnect" it from bus - need do only when you process IRP_MN_REMOVE_DEVICE (after device was not included in the bus driver's most recent response to an IRP_MN_QUERY_DEVICE_RELATIONS request for BusRelations - or in another words - when RemovePending flag in its device extension)
how to deal with incoming IO requests to the device after it becomes
"remove pending" and before OS sends IRP_MN_REMOVE_DEVICE request.
Should we check RemovePending flag and return
STATUS_DEVICE_DOES_NOT_EXIST or should we proceed as usual?
i think possible both behavior - you can process it as usual and can return STATUS_DEVICE_DOES_NOT_EXIST too. and assume next situation - you get some IO request in concurrent with removal device process. when you check RemovePending flag - it yet not set. and you begin process request "as usual". but just after you check RemovePending flag inside IO request you can set it while handling IRP_MN_QUERY_DEVICE_RELATIONS request with BusRelations. and this situation direct related to sense using Remove Locks or Run-Down Protection.
we really can use Run-Down Protection instead Remove Locks almost in same way and exactly in same places and for same reason as Remove Locks. and i think Run-Down Protection api (more new compare remove locks) - better design and better for use (however different is minimal)
I don't understand why we need to acquire remove lock inside PnP
request handler?
first of all note that about remove lock said only in Removing a Device in a Function Driver. you how i understand have not function, but bus driver- so Removing a Device in a Bus Driver more suitable for you. and in documentation for Removing a Device in a Function Driver exist serious error - advice first call IoReleaseRemoveLockAndWait at point 4 - before point 8 - Pass the IRP_MN_REMOVE_DEVICE request down to the next driver. but correct must be
driver should call IoReleaseRemoveLockAndWait after it passes
the IRP_MN_REMOVE_DEVICE request to the next-lower driver, and
before it releases memory, calls IoDetachDevice, or calls
IoDeleteDevice.
this is correct and stated in Using Remove Locks and IoReleaseRemoveLockAndWait. funny that in old msdn versions was
call IoReleaseRemoveLockAndWait before it passes..
but now this is fixed. why after ? because next-lower driver can pending some IRPs (on which we call IoAcquireRemoveLock or ExAcquireRundownProtection) and complete it only when got IRP_MN_REMOVE_DEVICE and our driver call IoReleaseRemoveLock or ExReleaseRundownProtection only when this IRP will be completed. as result if call IoReleaseRemoveLockAndWait or ExWaitForRundownProtectionRelease before passes the remove IRP to the next-lower driver - we can wait here forever - next-lower driver can not complete some IRP (until not got remove request) and we not release remove lock or rundown protection.
so for what we need remove locks or rundown protection ? because we can got IRP_MN_REMOVE_DEVICE in concurrent with another IO requests. and this IO requests can use some resources on device. from another sized when we process IRP_MN_REMOVE_DEVICE we destroy this resources. what be if we will use some resource in IO request after he will be destroyed in IRP_MN_REMOVE_DEVICE ? think not need answer. for prevent this and exist removal locks or rundown protection. before use any resource (which will be destroyed in remove) need call IoAcquireRemoveLock or ExAcquireRundownProtection and use it only if ok status returned. after we finish use resource call IoReleaseRemoveLock or ExReleaseRundownProtection. and in IRP_MN_REMOVE_DEVICE we call IoReleaseRemoveLockAndWait or ExWaitForRundownProtectionRelease. after this call is returned - we can be sure that nobody use our resources and never will be used more (calls to IoAcquireRemoveLock or ExAcquireRundownProtection return error status (false)). at this point we can safe begin destroy resources: releases memory (etc), calls IoDetachDevice, IoDeleteDevice.
the pointer to the next-lower device - this is also resource, which we use in process IO request and destroy in IRP_MN_REMOVE_DEVICE (by call IoDetachDevice). really are correct call IofCallDriver(_nextDeviceObject, Irp); (while process some IO request) after call IoDetachDevice(_nextDeviceObject); (inside IRP_MN_REMOVE_DEVICE) ? because this removal locks (or i use yourself rundown-protection here ) always used in functions and filter drivers. for bus driver, where we usually have not pointer to the next-lower device (PDO not attached to another device, when FDO attached to PDO and filter always attached to something) - may be not need locks (or rundown protection at all). this is depend - are exist another resources - used and destroyed (on removal).
and for bus devices - usual situation when we got IRP_MN_REMOVE_DEVICE 2 time - first before device marked as RemovePending - so we go to point 4. and after device marked as RemovePending (so device was not included in the bus driver's most recent response to an IRP_MN_QUERY_DEVICE_RELATIONS request for BusRelations) we finally destroy resources and call IoDeleteDevice
From a general standpoint, I am trying to figure out how to access a platform device from userspace. To be more specific, I have a EMIF controller on and SoC of which I have added to my device tree and I believe it is correctly bound to a pre-written EMIF platform device driver. Now I am trying to figure out how I can access this EMIF device from a userspace application. I have come accross a couple different topics that seem to have some connection to this issue but I cannot quite find out how they relate.
1) As I read it seems like most I/O is done through the use of device nodes which are created by mknod(), do I need to create a device node in order to access this device?
2) I have read a couple threads that talk about writting a Kernel module (Character?, Block?) that can interface with both userspace and the platform device driver, and use it as an intermediary.
3) I have read about the possibility of using mmap() to map the memory of my platform device into my virtual memory space. Is this possible?
4) It seems that when the EMIF driver is instantiated, it calls the probe() fucntion. What functions would a userpace application call in the driver?
It's not completely clear what you're needing to do (and I should caveat that I have no experience with EMIF or with "platform devices" specifically), but here's some overview to help you get started:
Yes, the usual way of providing access to a device is via a device node. Usually this access is provided by a character device driver unless there's some more specific way of providing it. Most of the time if an application is talking "directly" to your driver, it's a character device. Most other types of devices are used in interfacing with other kernel subsystems: for example, a block device is typically used to provide access from a file system driver (say) to an underlying disk drive; a network driver provides access to the network from the in-kernel TCP/IP stack, etc.
There are several char device methods or entry points that can be supported by your driver, but the most common are "read" (i.e. if a user-space program opens your device and does a read(2) from it), "write" (analogous for write(2)) and "ioctl" (often used for configuration/administrative tasks that don't fall naturally into either a read or write). Note that mknod(2) only creates the user-space side of the device. There needs to be a corresponding device driver in the kernel (the "major device number" given in the mknod call links the user-space node with the driver).
For actually creating the device node in the file system, this can be automated (i.e. the node will automatically show up in /dev) if you call the right kernel functions while setting up your device. There's a special daemon that gets notifications from the kernel and responds by executing the mknod(2) system call.
A kernel module is merely a dynamically loadable way of creating a driver or other kernel extension. It can create a character, block or network device (et al.), but then so can a statically linked module. There are some differences in capability mostly because not all kernel functions you might want to use are "exported" to (i.e. visible to) dynamically loaded modules.
It's possible to support mapping of the device memory into user virtual memory space. This would be implemented by yet another driver entry point (mmap). See struct file_operations for all the entry points a char driver can support.
This is pretty much up to you: it depends on what the application needs to be able to do. There are many drivers in the kernel that provide no direct function to user-space, only to other kernel code. As to "probe", there are many probe functions defined in various interfaces. In most cases, these are called by the kernel (or perhaps by a 'higher level "class" driver') to allow the specific driver to discover, identify and "claim" individual devices. They (probe functions) don't usually have anything directly to do with providing access from user-space but I might well be missing something in a particular interface.
You need to create a device node in order to access the device.
The probe function is called when the driver finds a matching device.
For information on platform device API, the following articles could be useful.
The platform device API
Platform devices and device trees