I'm currently writing a driver for a PCIe device that should send data to a Linux system using DMA. As far as I can understand my PCIe device needs a DMA controller (DMA master) and my Linux system too (DMA slave). Currently the PCIe device has no DMA controller and should not get one. That confuses me.
A. Is the following possible?
PCIe device sends interrupt
Wait for interrupt in the Linux driver
Start DMA transfer from memory mapped PCIe registers to Linux system DMA.
Read the data from memory in userspace
I have everything setup for this, the only thing I miss is how to transfer the data from the PCIe registers to the memory.
B. Which system call (or series of) do I need to call to do a DMA transfer?
C. I probably need to setup the DMA on the Linux system but what I find points to code that assumes there is a slave, e.g. struct dma_slave_config.
The use case is collecting data from the PCIe device and make it available in memory to userspace.
Any help is much appreciated. Thanks in advance!
DMA, by definition, is completely independent of the CPU and any software (i.e. OS kernel) running on it. DMA is a way for devices to perform memory reads and writes against host memory without the involvement of the host CPU.
The way DMA usually works is something like this: software will allocate a DMA accessible region in memory and share the physical address with the device, say, by performing memory writes against the address space associated with one of the device's BARs. Then, the device will perform a DMA read or write against that block of memory. When that operation is complete, the device will issue an interrupt to the device driver so it can handle the data and/or free the memory.
If your device does not have the capability of issuing a DMA read or write against host memory, then you'll have to interact with it with using the CPU only. Discrete DMA controllers have not been a thing for a very long time.
Related
There are some questions that make me confused:
Msi interrupt is a memory write request. Can msi ensure that all DMA data have been written into ram? or only ensure that the data has been transferred completely on pci bridge?
If msi interrupt only ensures the data transfer completely on pci bridge. How to guarantee all DMA data write into ram when getting msi interrupt?
Does msi memory write request really write into ram?
Thanks in advance.
Ensuring that DMA data have been written to the bus prior to the MSI write is the responsibility of the device. The device should not issue the MSI write until everything that the driver/OS needs to see with respect to the device request has been done, whether that entails memory reads, memory writes or whatever else. But, assuming things have been done in the appropriate order (on the bus) by the device (DMA write(s), then MSI write), it is then up to the host bridge to ensure that the data is written to RAM in the correct order. But typically the MSI write itself has nothing to do with any guarantees. The host bridge simply ensures that its memory transactions are executed in the order given (and the memory subsystem ensures coherence among all the CPUs and peripherals so that the data appear to have been written to memory in the correct order even if there are caches and such).
As for your question 3, the MSI write goes to wherever the device is told to send it when you setup MSI in the device registers. Typically, that "MSI memory write" is directed to an address associated with the system interrupt controller and not to actual RAM, but it's the OS/driver responsibility to configure the correct address.
for my application I would to send some data allocated in RAM to PWM fifo through DMA in Kernel Space.
I would to use DMA to generate an Interrupt when the data vector is completed, so to load next one vector and trigger other behavior...
I read "Linux Device Drivers" 3rd edition from O'Reilly but I'm a bit confused about using DMA Engine.
I would ask which step I have to follow to start a DMA transaction Memory-to-Device (PWM) with Interrupt callback?
EDIT 1:
I need to learn how to use Linux DMA API for my case (memory -> pwm fifo), in kernel space.
I have sumbit a patch to improve ethernet performance by using dma engine. In this patch, driver is able to move packets from rx fifo to RAM (from device to mem. ) So you can get some infomation about using dma engine in linux kernel from this patch. sun4i-emac.c: add dma support
steps:
request an dma channel (api: dma_request_chan)
setup dma channel (api: dmaengine_slave_config)
map data buf to dma region (api: dma_map_single)
prepare for transfer (api: dmaengine_prep_slave_single)
submit dma transfer request (api: dmaengine_submit)
launch! (api: dma_async_issue_pending)
I am writing some proof of concept code for KVM for communication between Windows 10 and the Host Linux system.
What I have is a virtual RAM device that is actually connected to a shared memory segment on the Host. The PCIe BAR 2 is a direct mapping to this RAM.
My intent is to provide a high bandwidth low latency means of transferring data that doesn't involve other common means used (sockets, etc). ZeroCopy would be ideal.
So far I have pretty much everything working, I have written a driver that calls MmAllocateMdlForIoSpace and then maps the memory using MmMapLockedPagesSpecifyCache to user mode via a DeviceIOControl. This works perfectly, the user mode application is able to address the shared memory and write to it.
What I am missing is the ability to use CreateFileMapping in user mode to obtain a HANDLE to a mapping of this memory. I am fairly new to windows driver programming and as such I am uncertain as to if this is even possible. Any pointers as to the best way to achieve this would be very helpful.
We are writing a DMA-based driver for a custom made PCI-Express device using WDF for Windows 7.
As you may know, PCI-Express bus transactions are not allowed to cross a 4k memory boundary. The custom device does not check this, and therefore we need to ensure that the driver only requests DMA transfers which are aligned to 4k memory boundaries.
The profile for the device is WdfDmaProfilePacket64.
We tried using WdfDeviceSetAlignmentRequirement(DevExt->Device, 4095), but this does not result in the DMA start address to be properly aligned.
How can we configure the WDF framework so that it only requests properly aligned addresses?
you can handle this in user space application, somehow that you initiate/allocate an aligned memory in user space and then send it to kernel program. it is not easy for a driver to align a memory which already allocated and initiated. even in user-space application we have to allocating extra space and then using the aligned part(I know, it's not pretty, that's why i recommend to solve this problem in device side)
for example if you use C++ for your user-space application you can do something like this
I am writing a kernel module in a guest operating system that will be run on a virtual machine using KVM. Here I want to allcoate a memory page at a particular physical address. kmalloc() gives me memory but at a physical address chosen by the OS.
Background : I am writing a device emulation technique in qemu that wouldn't exit when the guest communicates with the device (It exits, for example, in I/O mapped as well as port mapped devices). The basic idea is as follows : The guest device driver will write to a specific (guest) physical memory address. A thread in the qemu process will be polling it continuously to check for new data (through some status bits etc.). And will take action accordingly without causing an exit. Since there is no (existing) way by which guest can tell the host what address is being used by the device driver, I want a pre-specified memory page to be allocated for it.
You cannot allocate memory at a specific address, however, you can reserve certain physical addresses on boot time using reserve_bootmem(). Calling reserve_bootmem() early on boot (of course, it requires a modified kernel) will ensure that the reserved memory will not be passed on to the buddy system (i.e. alloc_pages() and higher level friends - kmalloc()), and you will be able to use that memory for any purpose.
It sounds like you should be attacking this from the other side, by having a physical memory range reserved in the memory map that the QEMU BIOS passes to the guest kernel at boot.