if want to allocate non-cacheable physical memory (DRAM) for usage in the driver,
(ie. don't want the data being cached into the CPU's data cache when
the data are accessed) how could I do this?
there are functions like kmalloc(), get_free_pages, vmalloc, etc,
but seems like that I can't specify if the data can be cached or not using these functions?
any suggestion on how to do it?
thanks!
In short there is no easy way to do this, it is very platform dependent.
If you want a go at it read drivers/char/mem.c and Chapter 15 of the Linux Device Drivers 3rd Edition book.
Related
According to the following documentation
https://www.kernel.org/doc/html/latest/x86/pat.html,
Drivers wanting to export some pages to userspace do it by using mmap interface and a combination of:
pgprot_noncached()
io_remap_pfn_range() or remap_pfn_range() or vmf_insert_pfn()
Note that this set of APIs only works with IO (non RAM) regions. If driver wants to export a RAM region, it has to do set_memory_uc() or set_memory_wc() as step 0 above and also track the usage of those pages and use set_memory_wb() before the page is freed to free pool.
Why is the extra step set_memory_uc() or set_memory_wc() needed for RAM regions?
This is needed since set_memory_uc() and set_memory_wc() are specifically written to work with memory regions; the other API functions you're being told to use here are for I/O regions.
Since you want to work with page(s) in a RAM region using the API functions listed, your driver needs to mark them as uncached or write-combined first so that they can essentially be treated like I/O pages, use the APIs, and then be sure to follow up with explicit writeback(s) of the memory page(s) in order to sync their contents before your driver considers itself "finished" with them.
I am porting Windows 7 Network driver code to WEC7. I got stuck with the API MmGetPhysicalAddress. I didn't find equivalent API to this in WEC7. Can anyone help to proceed further..
Thanks.
MmGetPhysicalAddress is not available in Windows CE, but you probably don't need it anyway.
Somewhere in the InitializeHandlerEx callback, the driver should be calling NdisMAllocateSharedMemory to allocate RX/TX buffers.
NdisMAllocateSharedMemory returns both the virtual and physical address of the allocated buffer, so you can keep the physical address around, and then there won't be any need to request it from the OS.
Normally the physical address would be kept in a driver-specific, per-buffer structure along with the virtual buffer address.
You can find a sample implementation of this in C:\WINCE700\public\COMMON\oak\drivers\netcard\e100bex\60. In mp_init.c, notice how NICAllocAdapterMemory calls NdisMAllocateSharedMemory and stores the physical address of each buffer in pMpTxbuf->BufferPa.
You may have a look at LockPages:
https://msdn.microsoft.com/en-us/library/ee482989.aspx
But if the buffer was not allocated using NDIS functions it may not be fully contiguous in physical memory, so you may need to check that.
I am just writing my very first linux device driver, and I have ran into a problem. I want to prevent one memory region from being cached, so I have been trying to use flush_cache_range() and flush_tlb_range() to flush the cache for this memory region. Everything compiles well, but when I try to load the kernel module I get the following errors:
Unknown symbol flush_cache_range (err 0)
Unknown symbol flush_tlb_range (err 0)
I find this very strange. Shouldn't they be defined in kernel?
I know that alternatively I could also use dma_alloc_coherent() to allocate a non-cached memory region. But I don't have a device structure and passing NULL for this parameter didn't cause any errors, but I also couldn't see any of the data that was supposed to be there.
Some information about my system: I'm trying to get this running on a ARM microcontroller with an integrated FPGA (the Xilinx Zynq). The FPGA copies some data to a memory location specified by the CPU. Now I want to access this memory without getting old data from the caches.
Any help is very appreciated.
You cannot use functions such as flush_cache_range() because they are not intended to be used by modules.
To allocate memory that can be accessed by a DMA device, you must use dma_alloc_coherent().
This requires a valid device structure so that it can do proper mapping between memory addresses and bus addresses.
If your device is not on a bus that is handled by an existing framework (such as PCI), you have to create a platform device.
A few notes:
1- flush_cache_range doesn't "prevent one memory region from being cached" .. It just simply flush (clean + invalidate) the caches. Any future writes/reads to this memory region through the same virtual range will go through the cache again.
2- If the FPGA is writing to memory and then the CPU are going to read from this memory, probably flushing the cache isn't the correct thing to do any way. Usually what you need to do is to invalidate the memory region and then tell the FPGA to write.
3- Please take a look at "${kernel-src}/Documentation/DMA-API.txt" in the kernel sources. It has plenty of information about how you can safely ( cache maintenance + phys_to_dma translation ) use a specific region of memory for DMA.
I am writting a Kernel Module that is going to trigger and external PCIe device to read a block of data from my internel memory. To do this I need to send the PCIe device a pointer to the physical memory address of the data that I would like to send. Ultimately this data is going to be written from Userspace to the kernel with the write() function (userspace) and copy_from_user() (kernel space). As I understand it, the address that my kernel module will see is still a virtual memory address. I need a way to get the physical address of it so that the PCIe device can find it.
1) Can I just use mmap() from userspace and place my data in a known location in DDR memory, instead of using copy_from_user()? I do not want to accidently overwrite another processes data in memory though.
2) My kernel module reserves PCIe data space at initialization using ioremap_nocache(), can I do the same from my kernel module or is it a bad idea to treat this memory as io memory? If I can, what would happen if the memory that I try to reserve is already in use? I do not want to hard code a static memory location and then find out that it is in use.
Thanks in advance for you help.
You don't choose a memory location and put your data there. Instead, you ask the kernel to tell you the location of your data in physical memory, and tell the board to read that location. Each page of memory (4KB) will be at a different physical location, so if you are sending more data than that, your device likely supports "scatter gather" DMA, so it can read a sequence of pages at different locations in memory.
The API is this: dma_map_page() to return a value of type dma_addr_t, which you can give to the board. Then dma_unmap_page() when the transfer is finished. If you're doing scatter-gather, you'll put that value instead in the list of descriptors that you feed to the board. Again if scatter-gather is supported, dma_map_sg() and friends will help with this mapping of a large buffer into a set of pages. It's still your responsibility to set up the page descriptors in the format expected by your device.
This is all very well written up in Linux Device Drivers (Chapter 15), which is required reading. http://lwn.net/images/pdf/LDD3/ch15.pdf. Some of the APIs have changed from when the book was written, but the concepts remain the same.
Finally, mmap(): Sure, you can allocate a kernel buffer, mmap() it out to user space and fill it there, then dma_map that buffer for transmission to the device. This is in fact probably the cleanest way to avoid copy_from_user().
I'm new to kernel programming, and currently working with the "Madwifi" driver for tplink wireless NIC (Atheros chipset). My kernel version is 2.6.32-37 (ubuntu).
I'm working with a driver which has been modified by other people (whom i don't know and can't contact to get more information) and i'm afraid they dynamically allocated memory (using kmalloc) but didn't free it appropriately.
My questions are:
If they actually did not free allocated memory, does it make the operating system to consider the module as "busy", making it unable to unload the module?
And how or where should i free allocated memory in modules (considering the fact that the module is supposed to run "forever")?
thanks!
omer.
No. Having allocated memory with kmalloc (or any of its colleagues) does not increment the use-count of a module. The ownership of allocated memory can change at any time just by passing the address (which is kept in a pointer-variable) to another module/block which store is somewhere. The kernel is not tracking this.
How memory is shared, given and taken depends a lot on the choice of coding-architecture and coding-model. This is written in C; there are no rules for that.
One way to handle the use-count of a module is to use kref. See for more information see Documentation/kref.txt in the kernel-source-tree or here.