I'm studying the MM in Linux and I got very confused when I could't find where the raw data is stored. I thought it was stored in some field of a page struct but I couldn't find there.
Where is the actual data represented by a page stored? And how to get a pointer to it?
page struct is just a helper which stores the metadata. it doesn't actually store any data, but the directions to locate the data in memory. That is, the address space mapping to the physical addresses etc. The actual data is still stored in the physical memory.
Where is the actual data represented by a page stored?
The actual data is in a physical page address by at least one virtual address AND/OR it is on disk in an inode and has never been mapped. For the inode case, accessing the virtual address will trigger a page fault and that handler will read the memory into a physical page and the faulted code will resume.
And how to get a pointer to it?
I believe that the struct page is contained in another array, like mem_map. For instance the function mem_map_next, is use to iterate through an array of struct page. Perhaps the structure you are interested in is struct vm_area_struct? This is a virtual address tracking structure. There maybe multiple virtual addresses mapping to the same physical page.
You need to know the context of a composing struct to know the address a struct page represents. Then it is simply a base address plus the index multiplied by the page size.
You could use page_address() to get virtual address of a page.
But the return address might be NULL due to the fact that not all pages have mapped virtual addresses.
void *page_address(const struct page *page);
You could use kmap to map a highmen page to a virtual address.
Also, remember to use kunmap to unmap this page when you don't need to access it.
struct page *page = alloc_pages(GFP_KERNEL | __GFP_HIGHMEM, 0);
if (page) {
void *addr = kmap(page);
if (addr) {
memset(addr, 0, PAGE_SIZE);
kunmap(addr);
}
}
Related
I am trying to get an idea on how does memory mapping take place using the system call mmap.
So far I know mmap takes arguments from the user and returns a logical address of where the file is stored. When the user tries to access it takes this address to the map table converts it to a a physical address and carries the operation as requested.
However I found articles as code example and Theoretical explanation
What it mentions is the memory mapping is carried out as:
A. Using system call mmap ()
B. file operations using (struct file *filp, struct vm_area_struct *vma)
What I am trying to figure out is:
How the arguments passed in the mmap system call are used in the struct vm_area_struct *vma) More generally how are these 2 related.
for instance: the struct vm_area_struct has arguments such as starting address, ending address permissions,etc. How are the values sent by the user used to fill values of these variables.
I am trying to write a driver so, Does the kernal fill the values for variables in the structure for us and I simply use it to call and pass values to remap_pfn_range
And a more fundamental question, why is a different file systems operation needed. The fact that mmap returns the virtual address means that it has already achieved a mapping doesnt it ?
Finally I am not that clear about how the entire process would work in user as well as kernal space. Any documentation explaining the process in details would be helpful.
Can we access memory through a struct page structure?
Note: The page belongs to high memory and has not been mapped to kernel logical address space.
Yes we can access the page belonging to highmem through struct page's virtual field. But in your case you can't access as you mentioned that highmem page is not mapped into kernel virtual memory.
To access it you need to create mapping either permanent or temporary mappping.
To create permanent mapping map page through kmap.
void *kmap(struct page *page)
This function works on either high or low memory. If the page structure belongs to a page in low memory, the page’s virtual address is simply returned. If the page resides in high memory, a permanent mapping is created and the address is returned.The function may sleep, so kmap() works only in process context. Because the number of permanent mappings are limited (if not, we would not be in this mess and could just permanently map all memory), high memory should be unmapped when no longer needed.This is done via the following function, which unmaps the given page:
void kunmap(struct page *page)
The temporary mapping can be created via:
void *kmap_atomic(struct page *page, enum km_type type)
This is an atomic function so you can't sleep and can be called in interrupt context. It is called temporary because next call to kmap_atomic will overwrite the previous mapping.
in case there is no value for virtual field then you can not access that specific physical frame. the simple reason is struct page denotes the mappings between physical and virtual addresses so a system with large memory can not map all memory in kernel space. so high memory is mapped dynamically. but to access that memory it should be mapped i.e. void *virtual should not be NULL.
This is an assignment problem which asks for partial implementation of process checkpointing:
The test program allocates an array, does a system call and passes the start and end address of array to the call. In the system call function I have to save the contents in the give range to a file.
From my understanding, I could simply use copy_from_usr function to save the contents from the give range. However since the assignment is based on topic "Process address space", I probably need to walk through page tables. Say I manage to get the struct pages that correspond to given range. How do I get the data corresponding to the pages?
Can I just use page_to_virt function and access data directly?
Since the array is contiguous in virtual space, I guess I will just need to translate the starting address to page and then back to virtual address and then just copy the range size of data to file. Is that right?
I think copy_from_user() is ok, nothing else needed. When executing the system call, although it trap to kernel space, the context is still the process context which doing the system call. The kernel still use the process's page table. So just to use copy_from_user(), and nothing else needed.
Okey, if you want to do this experiment, I think you can use the void __user *vaddr to traverse the mm->pgd(page table), using pgd_offset/pud_offset/pmd_offset/pte_offset to get the page physical address(page size alignment). Then in kernel space, using ioremap() to create a kernel space mapping, then using the kernel virtual address(page size) + offset(inside the page), you get the start virtual address of the array. Now in kernel, you can using the virtual address to access the array.
For some reason, I need to translate the virtual address of the code section to physical address. I did following experiment:
I get the virtual address from the start_code and end_code in mm_struct of process A, which are the initial address and final address of the executable code.
I get the CR3 of process A.
I translate the virtual address to physical address page by page. For example, there are 10 pages for code section in process A. I will translate 10 virtual address of each beginning of the page.
I found out some pages will get Page Table Entry(PTE) == 0.Some pages could successfully translate to a physical address.
I tried Firefox and Minicom as my Process, and both of them will get into situation.
I guess my question is: could anyone explain to me why PTE == 0? Does it mean these pages have been swap out to disk? If this is the case, how can I find these pages?
Thanks for any input!!
It looks as if you are trying to perform page table introspection without using the kernel APIs for it. Note that the address space is arranged in a red-black tree of vm_area_struct structs and you should probably use the APIs that traverse them. The mappings might change at any time so using the proper locking for these data structures is necessary.
For example, see the get_user_pages() function. It can be used to swap-in and temporarily pin the pages into memory. Using this function for page table introspection is usually asked for because once have the physical address in hand then the kernel can swap out the page at any time.
I'd like pass a pointer from a user space memory into a function in my kernel module. I don't want to use copy_from_user. I've read that I should use get_user_pages function.
For example one page.
struct page **pages;
pages = kmalloc(1 * sizeof(*pages), GFP_KERNEL);
down_read(¤t->mm->mmap_sem);
get_user_pages(current,current->mm,uaddr, 1, 1, 0,pages,NULL);
up_read(¤t->mm->mmap_sem);
uaddr is an address in User Space.
After doing this, am I allowed to cast and pass uaddr into my kernel module function? Or maybe I have to use these struct pages in some way?
Why do I have to use down/up read?
After everything do I have to use SetPageDirty() and page_cache_release() functions ?
No, you cannot directly access the userspace pages via uaddr. The struct pages are filled out to allow the kernel to access the physical pages that correspond to the userspace pages. Note also that they are most unlikely to be contiguous, so one must be careful to use the correct page index into the array from the start of the uaddr.
You are changing the page mapping structures for this process, so need to protect them while the page mapping in the kernel is set up.
When you are done with the mappings that were setup by get_user_pages(), you must 'release' them by the referenced functions.
This is not what get_user_pages is for (and no - you can't then just cast and pass uaddr into your kernel module function).
If you don't want to call copy_from_user in the calling function, then just pass a void __user * to your module function and have it do the copy_from_user.
You can only use the user pages for page type activity, for example setting up Scatter/Gather DMA into userspace memory. You cannot use it to directly access user space from kernel mode code. Hence the copy_to/from functions that are there for that reason. Unless your moving large amounts of data why not use these functions?
Once you get the valid user space address, use get_user_pages to get the struct page pointer. once struct page pointer received , to access it in the kernel mode you have to map it to kernel virtual address by using kmap. hope that helps