This is all in Linux 4.14.73. I'd upgrade if I could but I cant.
I'm trying to trigger an LED flash in a standard LED class instance from another kernel space driver. I know all about the "bad form" of not accessing files from Kernel Space so I figure there must be some way already defined way for accessing Sysfs attributes from Kernel Space.
The LED is defined here:
/sys/class/leds/fpga_led0
Its trigger is set to [oneshot] so it has a device attribute called "shot" exposed. To get a single LED flash all I need to do from the command line is this:
echo 1 > /sys/class/leds/fpga_led0/shot
I can easily write a User Space program to open the "shot" attribute and write a "1" string to it. There are various published methods of forcing file operations into a kernel driver. Most of them are fairly limited. I've yet to see one that exposes file seek operations which are key to repeatedly writing to an attribute without wasting time opening and closing the file. To be clear, this is not setting values at boot time. In this case I have one driver that needs to send a value to another driver's Sysfs entry at a specific moment in its own operation. Is there a standard, accepted way of sending a value from one running kernel driver to the Sysfs attribute of another kernel driver?
Related
You have done it a thousand times: you unplug some USB equipment and any device associated with that USB equipment is removed by the driver. Any program that uses some previously opened file handle will get an error. Somehow most Linux drivers take care of that.
I am currently struggling to implement the same in a simple driver. My driver creates a character device. When the device is opened, I set the private_data member of struct file to the address of some management data that exists once per character device. That management data also includes a mutex that I use to synchronize operations like read, write, and ioctl.
The problem now arises when the USB equipment is unplugged. I must not free the memory where that management data lives. First, any currently running read, write, or ioctl should finish. Any such running call will likely also hold a lock on the mutex and will attempt to unlock it. So the memory where the mutex lives will be accessed.
Any read, write, or ioctl call subsequent to unplugging the equipment should fail, so every such call must read some variable telling whether the USB equipment is still plugged in or not. Again, that variable must live somewhere and the memory where it lives must stay allocated as long as there are open file handles.
Long story short, it seems to me that I must do some sort reference counting: The management data must stay allocated until all file handles have been closed. I could implement that myself, but I have the feeling that I would reinvent the wheel. Such a thing must already exist, I'm not the first to have this problem.
Does Linux internally keep track of the number of open file handles? Can I define a callback that is called when all file handles have been closed? Is that even a viable thing? What is the proper way to remove a character device from the system?
Global variables shall not be avoided, since any number of USB devices can be attached.
As suggested by 0andriy, I have used reference counting to track the number of open file handles. Per character device, I have added a new struct containing a mutex, a kref, the cdev struct, and a pointer to further data. I use kref_get to increase the reference counter when a new file handle is opened. Accordingly, I use kref_put when file handles are released or the USB equipment is unplugged.
Since all I/O operations (read, write, ioctl, etc.) use the mutex, I can safely change the pointer to NULL when the USB equipment is unplugged. The I/O operations then start reporting errors.
The kref reference counter only returns to zero when the USB equipment is unplugged and all file handles are closed. So then I can also free the memory of the new struct.
This seems to work. It was a surprise that the cdev struct is referenced by file handles. It seems to be needed as long as there are open file handles.
But I'm still surprised that I had to go down this route. It seems redundant to implement that in every driver.
UPDATE: As it turns out, doing reference counting yourself is dangerous. In particular counting the number of open file handles is wrong. There is a whole reference-count-based garbage collection system in Linux and you need to use it.
For example, Linux calls cdev_get when a process opens a character device and cdev_put when the process exists and the file handle is still open. Unfortunately, the call to cdev_put would occur after the file handle's release function. As we would free the memory after the last file handle has been released, we would end up freeing the underlying memory of the cdev and the mutex before cdev_put is called.
The solution is to assign a parent to the cdev via cdev_set_parent. The parent kobject will have its reference counter increased whenever cdev_get is called and decreased after cdev_put is called. The kobject then can have its own release function which frees any memory required by the cdev. Basically I replaced the kref in my struct with a kobject.
Lesson learned: don't do reference counting on your own. Use the existing hierarchy of kobjects, which is the kernel's way of doing garbage collection.
UPDATE2: It turns out, we reinvented the wheel again. The device struct includes a release hook, which is called when the internal kobject reached a reference count of zero. This is the mechanism typically used in the kernel to release resources associated with an device - including the device struct itself.
I was looking for a release hook in the cdev struct. There was none. But it turns out I should have looked one step up in the hierarchy for such a hook.
Long story short: Use cdev_device_add in combination with the device release hook. cdev_device_add will internally call cdev_set_parent, so the device becomes the parent of the cdev. Those are the mechanism other kernel drivers (e.g. evdev) use to release their resources.
I have managed to create a virtual IOPCIDevice which attaches to IOResources and basically does nothing. I'm able to get existing drivers to register and match to it.
However when it comes to IO handling, I have some trouble. IO access by functions (e.g. configRead, ioRead, configWrite, ioWrite) that are described in IOPCIDevice class can be handled by my own code. But drivers that use memory mapping and IODMACommand are the problem.
There seems to be two things that I need to manage: IODeviceMemory(described in the IOPCIDevice) and DMA transfer.
How could I create a IODeviceMemory that ultimately points to memory/RAM, so that when driver tries to communicate to PCI device, it ultimately does nothing or just moves the data to RAM, so my userspace client can handle this data and act as an emulated PCI device?
And then could DMA commands be directed also to my userspace client without interfering to existing drivers' source code that use IODMACommand.
Thanks!
Trapping memory accesses
So in theory, to achieve what you want, you would need to allocate a memory region, set its protection bits to read-only (or possibly neither read nor write if a read in the device you're simulating has side effects), and then trap any writes into your own handler function where you'd then simulate device register writes.
As far as I'm aware, you can do this sort of thing in macOS userspace, using Mach exception handling. You'd need to set things up that page protection fault exceptions from the process you're controlling get sent to a Mach port you control. In that port's message handler, you'd:
check where the access was going to
if it's the device memory, you'd suspend all the threads of the process
switch the thread where the write is coming from to single-step, temporarily allow writes to the memory region
resume the writer thread
trap the single-step message. Your "device memory" now contains the written value.
Perform your "device's" side effects.
Turn off single-step in the writer thread.
Resume all threads.
As I said, I believe this can be done in user space processes. It's not easy, and you can cobble together the Mach calls you need to use from various obscure examples across the web. I got something similar working once, but can't seem to find that code anymore, sorry.
… in the kernel
Now, the other problem is you're trying to do this in the kernel. I'm not aware of any public KPIs that let you do anything like what I've described above. You could start looking for hacks in the following places:
You can quite easily make IOMemoryDescriptors backed by system memory. Don't worry about the IODeviceMemory terminology: these are just IOMemoryDescriptor objects; the IODeviceMemory class is a lie. Trapping accesses is another matter entirely. In principle, you can find out what virtual memory mappings of a particular MD exist using the "reference" flag to the createMappingInTask() function, and then call the redirect() method on the returned IOMemoryMap with a NULL backing memory argument. Unfortunately, this will merely suspend any thread attempting to access the mapping. You don't get a callback when this happens.
You could dig into the guts of the Mach VM memory subsystem, which mostly lives in the osfmk/vm/ directory of the xnu source. Perhaps there's a way to set custom fault handlers for a VM region there. You're probably going to have to get dirty with private kernel APIs though.
Why?
Finally, why are you trying to do this? Take a step back: What is it you're ultimately trying to do with this? It doesn't seem like simulating a PCI device in this way is an end to itself, so is this really the only way to do what greater goal you're ultimately trying to achieve? See: XY problem
I am working on a boot loader for TMS320DM6437. The idea is to create 2 independent firmware that one will update another. In firmware1 I will download firmware2 file and write it to NOR flash in a specified address. Both firmware are stored in NOR flash in ais format. Now I have two applications in flash. One is my custom boot loader and the second one is my main project. I want to know how I can jump from the first program to the second program located at a specified address. I also expect information about documents which may help me to create custom bootloader
Any recommendations?
You can jump to the entry point. I'm using this approach on TMS320 2802x and 2803x, but it should be the same.
The symbol of the entry point is c_int00.
To get to know the address of c_int00 in the second application, you have to fix the Run-Time Support (RTS) library at a specific address, by modifying the linker command file.
Otherwise you can leave the RTS unconstrained, and create a C variable (at a fixed address) that is initialized with the value of cint_00. Using this method your memory map is more flexible and you can add, togheter with the C variable, a comprehensive data structure with other information for your bootloader, e.g. CRC, version number, etc.
Be carefull with the (re)initialization of the peripherals in the second application, since you are not starting from a hardware reset, and you may need to explicity reset some more registers, or clear interrupt requests.
I would like to copy data to user space from kernel module which receives data from serial port and transfers it to DMA, which in turn forwards the data to tty layer and finally to user space.
the current flow is
serial driver FIFO--> DMA-->TTY layer -->User space (the data to tty layer is emptied from DMA upon expiration of timer)
What I want to achieve is
serial driver FIFO-->DMA-->user space. (I am OK with using timer to send the data to user space, if there is a better way let me know)
Also the kernel module handling the serialFIFO->DMA is not a character device.
I would like to bypass tty layer completely. what is the best way to achieve so?
Any pointers/code snippet would be appreciated.
In >=3.10.5 the "serial FIFO" that you refer to is called a uart_port. These are defined in drivers/tty/serial.
I assume that what you want to do is to copy the driver for your UART to a new file, then instead of using uart_insert_char to insert characters from the UART RX FIFO, you want to insert the characters into a buffer that you can access from user space.
The way to do this is to create a second driver, a misc class device driver that has file operations, including mmap, and that allocates kernel memory that the driver's mmap file operation function associates with the userspace mapped memory. There is a good example of code for this written by Maxime Ripard. This example was written for a FIQ handled device, but you can use just the probe routine's dma_zalloc_coherent call and the mmap routine, with it's call to remap_pfn_range, to do the trick, that is, to associate a user space mmap on the misc device file with the alloc'ed memory.
You need to connect the memory that you allocated in your misc driver to the buffer that you write to in your UART driver using either a global void pointer, or else by using an exported symbol, if your misc driver is a module. Initialize the pointer to a known invalid value in the UART driver and test it to make sure the misc driver has assigned it before you try to insert characters to the address to which it points.
Note that you can't add an mmap function to the UART driver directly because the UART driver class does not support an mmap file operation. It only supports the operations defined in the include/linux/serial_core.h struct uart_ops.
Admittedly this is a cumbersome solution - two device drivers, but the alternative is to write a new device class, a UART device that has an mmap operation, and that would be a lot of work compared with the above solution although it would be elegant. No one has done this to date because as Jonathan Corbet say's "...not every device lends itself to the mmap abstraction; it makes no sense, for instance, for serial ports and other stream-oriented devices", though this is exactly what you are asking for.
I implemented this solution for a polling mode UART driver based on the mxs-auart.c code and Maxime's example. It was non-trivial effort but mostly because I am using a FIQ handler for the polling timer. You should allow two to three weeks to get the whole thing up and running.
The DMA aspect of your question depends on whether the UART supports DMA transfer mode. If so, then you should be able to set it using the serial flags. The i.MX28's PrimeCell auarts support DMA transfer but for my application there was no advantage over simply reading bytes directly from the UART RX FIFO.
I am looking into the fpga driver code which will write some value to FPGA device at low level. At top level in user space value is being written to /dev/fpga, now I guess this is the logic how driver gets its value from user-space and exposed file in user space is "/dev/fpga".
But now how actually this value from fpga is reached to device , there must some callback maintained.
But I really could not figure out how it actually happens,Is there any standard way?
Anybody can help me find out this userspace to kernel space link.
It's probably a character device. You can create one in your kernel module, and your callback functions will be called in the kernel when it is opened, something is written to it etc. See:
http://linux.die.net/lkmpg/x569.html
for an explanation how it works and sample code.