I'm having trouble reading the Raspberry Pi 4 system timer.
My understanding is that the LO 32 bits should be at address 0x7e003004.
My reads always return -1.
Here's how I am trying:
int fd;
unsigned char* start;
uint32_t* t4lo;
fd = open("/dev/mem", O_RDONLY);
if (fd == -1)
{
perror("open /dev/mem");
exit(1);
}
start = (unsigned char*)mmap(0, getpagesize(), PROT_READ, MAP_SHARED,
fd, 0x7e003000);
t4lo = (unsigned int *)(start + 0x04);
...
uint32_t Rpi::readTimer(void)
{
return *t4lo;
}
I should be checking the value of start, but gdb tells me it's reasonable so I don't think that's the problem.
(gdb) p t4lo
$4 = (uint32_t *) 0xb6f3a004
and gdb won't let me access *t4lo. Any ideas?
Edit: clock_gettime() is fulfilling my needs, but I'm still curious.
A closer look at https://www.raspberrypi.org/documentation/hardware/raspberrypi/bcm2711/rpi_DATA_2711_1p0.pdf
figure 1 on page 5 shows that addresses vary depending upon who's looking at things. If you start with 0x7c00_0000 on the left side and follow it over to the right, it's apparent that it shows up at 0xfc00_0000 to the processor. So changing the timer base address to 0xfe00_3000 fixed the problem.
The secret is hidden in section 1.2.4:
So a peripheral described in this document as being at legacy address 0x7Enn_nnnn
is available in the 35-bit address space at 0x4_7Enn_nnnn, and visible to the ARM
at 0x0_FEnn_nnnn if Low Peripheral mode is enabled.
The address of the BCM2711 ARM Peripherals is the bus address which is not the same as the physical address in most systems. The bus address is easily used by DMA(Direct Memory Access) controller. mmap creates a new mapping from physical address to virtual address not bus address. So you can't use mmap funtion with parameter 0x7e003000. The rich answer is right.
So changing the timer base address to 0xfe00_3000 fixed the problem.
In addtion, your program run in the User space, only virtual address can you directly use.
Related
Kernel document https://www.kernel.org/doc/gorman/html/understand/understand010.html says, that for vmalloc-ing
It searches through a linear linked list of vm_structs and returns a new struct describing the allocated region.
Does that mean vm_struct list is already created while booting up, just like kmem_cache_create and vmalloc() just adjusts the page entries? If that is the case, say if I have a 16GB RAM in x86_64 machine, the whole ZONE_NORMAL i.e
16GB - ZONE_DMA - ZONE_DMA32 - slab-memory(cache/kmalloc)
is used to create vm_struct list?
That document is fairly old. It's talking about Linux 2.5-2.6. Things have changed quite a bit with those functions from what I can tell. I'll start by talking about code from kernel 2.6.12 since that matches Gorman's explanation and is the oldest non-rc tag in the Linux kernel Github repo.
The vm_struct list that the document is referring to is called vmlist. It is created here as a struct pointer:
struct vm_struct *vmlist;
Trying to figure out if it is initialized with any structs during bootup took some deduction. The easiest way to figure it out was by looking at the function get_vmalloc_info() (edited for brevity):
if (!vmlist) {
vmi->largest_chunk = VMALLOC_TOTAL;
}
else {
vmi->largest_chunk = 0;
prev_end = VMALLOC_START;
for (vma = vmlist; vma; vma = vma->next) {
unsigned long addr = (unsigned long) vma->addr;
if (addr >= VMALLOC_END)
break;
vmi->used += vma->size;
free_area_size = addr - prev_end;
if (vmi->largest_chunk < free_area_size)
vmi->largest_chunk = free_area_size;
prev_end = vma->size + addr;
}
if (VMALLOC_END - prev_end > vmi->largest_chunk)
vmi->largest_chunk = VMALLOC_END - prev_end;
}
The logic says that if the vmlist pointer is equal to NULL (!NULL), then there are no vm_structs on the list and the largest_chunk of free memory in this VMALLOC area is the entire space, hence VMALLOC_TOTAL. However, if there is something on the vmlist, then figure out the largest chunk based on the difference between the address of the current vm_struct and the end of the previous vm_struct (i.e. free_area_size = addr - prev_end).
What this tells us is that when we vmalloc, we look through the vmlist to find the absence of a vm_struct in a virtual memory area big enough to accomodate our request. Only then can it create this new vm_struct, which will now be part of the vmlist.
vmalloc will eventually call __get_vm_area(), which is where the action happens:
for (p = &vmlist; (tmp = *p) != NULL ;p = &tmp->next) {
if ((unsigned long)tmp->addr < addr) {
if((unsigned long)tmp->addr + tmp->size >= addr)
addr = ALIGN(tmp->size +
(unsigned long)tmp->addr, align);
continue;
}
if ((size + addr) < addr)
goto out;
if (size + addr <= (unsigned long)tmp->addr)
goto found;
addr = ALIGN(tmp->size + (unsigned long)tmp->addr, align);
if (addr > end - size)
goto out;
}
found:
area->next = *p;
*p = area;
By this point in the function we have already created a new vm_struct named area. This for loop just needs to find where to put the struct in the list. If the vmlist is empty, we skip the loop and immediately execute the "found" lines, making *p (the vmlist) point to our struct. Otherwise, we need to find the struct that will go after ours.
So in summary, this means that even though the vmlist pointer might be created at boot time, the list isn't necessarily populated at boot time. That is, unless there are vmalloc calls during boot or functions that explicitly add vm_structs to the list during boot as in future kernel versions (see below for kernel 6.0.9).
One further clarification for you. You asked if ZONE_NORMAL is used for the vmlist, but those are two separate memory address spaces. ZONE_NORMAL is describing physical memory whereas vm is virtual memory. There are lots of resources for explaining the difference between the two (e.g. this Stack Overflow question). The specific virtual memory address range for vmlist goes from VMALLOC_START to VMALLOC_END. In x86, those were defined as:
#define VMALLOC_START 0xffffc20000000000UL
#define VMALLOC_END 0xffffe1ffffffffffUL
For kernel version 6.0.9:
The creation of the vm_struct list is here:
static struct vm_struct *vmlist __initdata;
At this point, there is nothing on the list. But in this kernel version there are a few boot functions that may add structs to the list:
void __init vm_area_add_early(struct vm_struct *vm)
void __init vm_area_register_early(struct vm_struct *vm, size_t align)
As for vmalloc in this version, the vmlist is now only a list used during initialization. get_vm_area() now calls get_vm_area_node(), which is a NUMA ready function. From there, the logic goes deeper and is much more complicated than the linear search described above.
I'm seeing a weird case in a simple linux driver test(arm64).
The user program calls ioctl of a device driver and passes array 'arg' of uint64_t as argument. By the way, arg[2] contains a pointer to a variable in the app. Below is the code snippet.
case SetRunParameters:
copy_from_user(args, (void __user *)arg, 8*3);
offs = args[2] % PAGE_SIZE;
down_read(¤t->mm->mmap_sem);
res = get_user_pages( (unsigned long)args[2], 1, 1, &pages, NULL);
if (res) {
kv_page_addr = kmap(pages);
kv_addr = ((unsigned long long int)(kv_page_addr)+offs);
args[2] = page_to_phys(pages) + offset; // args[2] changed to physical
}
else {
printk("get_user_pages failed!\n");
}
up_read(¤t->mm->mmap_sem);
*(vaddr + REG_IOCTL_ARG/4) = virt_to_phys(args); // from axpu_regs.h
printk("ldd:writing %x at %px\n",cmdx,vaddr + REG_IOCTL_CMD/4); // <== line 248. not ok w/o this printk line why?..
*(vaddr + REG_IOCTL_CMD/4) = cmdx; // this command is different from ioctl cmd!
put_page(pages); //page_cache_release(page);
break;
case ...
I have marked line 248 in above code. If I comment out the printk there, a trap occurs and the virtual machine collapses(I'm doing this on a qemu virtual machine). The cmdx is a integer value set according to the ioctl command from the app, and vaddr is the virtual address of the device (obtained from ioremap). If I keep the printk, it works as I expect. What case can make this happen? (cache or tlb?)
Accessing memory-mapped registers by simple C constructs such as *(vaddr + REG_IOCTL_ARG/4) is a bad idea. You might get away with it on some platforms if the access is volatile-qualified, but it won't work reliably or at all on some platforms. The proper way to access memory-mapped registers is via the functions declared by #include <asm/io.h> or #include <linux/io.h>. These will take care of any arch-specific requirements to ensure that writes are properly ordered as far as the CPU is concerned1.
The functions for memory-mapped register access are described in the Linux kernel documentation under Bus-Independent Device Accesses.
This code:
*(vaddr + REG_IOCTL_ARG/4) = virt_to_phys(args);
*(vaddr + REG_IOCTL_CMD/4) = cmdx;
can be rewritten as:
writel(virt_to_phys(args), vaddr + REG_IOCTL_ARG/4);
writel(cmdx, vaddr + REG_IOCTL_CMD/4);
1 Write-ordering for specific bus types such as PCI may need extra code to read a register inbetween writes to different registers if the ordering of the register writes is important. That is because writes are "posted" asynchronously to the PCI bus, and the PCI device may process writes to different registers out of order. An intermediate register read will not be handled by the device until all preceding writes have been handled, so it can be used to enforce ordering of posted writes.
in a linux device driver, in the init function for the device, I tried reading an address (which is SMMUv3 device for arm64) like below.
uint8_t *addr1;
addr1 = ioremap(0x09050000, 0x20000);
printk("SMMU_AIDR : 0x%X\n", *(addr1 + 0x1c));
but I get Internal error: synchronous external abort: 96000010 [#1] SMP error.
Is it not permitted to map an address to virtual address using ioremap and just reading that address?
I gave a fixed value 0x78789a9a to SMMU IDR[2] register. (at offset 0x8, 32 bit register. This is possible because it's qemu.)
SMMU starts at 0x09050000 and it has address space 0x20000.
__iomem uint32_t *addr1 = NULL;
static int __init my_driver_init(void)
{
...
addr1 = ioremap(0x09050000, 0x20000); // smmuv3
printk("SMMU_IDR[2] : 0x%X\n", readl(addr1 +0x08/4));
..}
This is the output when the driver is initialized.(The value is read ok)
[ 453.207261] SMMU_IDR[2] : 0x78789A9A
The first problem was that the access width was wrong for that address. Before, it was defined as uint8_t *addr1; and I used printk("SMMU_AIDR : 0x%X\n", *(addr1 + 0x1c)) so it was reading byte when it was not allowed by the SMMU model.
Second problem (I think this didn't cause the trap because arm64 provides memory mapped io) was that I used memory access(pointer dereferencing) for memory mapped IO registers. As people commented, I should have used readl function. (Mainly because to make the code portable. readl works also for iomap platforms like x86_64. using the mmio adderss as pointer will not work on such platforms. I later found that readl function takes care of the memory barrier problem too).
ADD : I fixed volatile to __iomem for variable addr1.(thanks #0andriy)
According to U-Boot's README.enetaddr "Usage" section:
If the hardware design mandates that the MAC address is stored in some special place (like EEPROM etc...), then the board specific init code (such as the board-specific misc_init_r() function) is responsible for locating the MAC address(es) and initializing the respective environment variable(s) from it. Note that this shall be done if, and only if, the environment does not already contain these environment variables, i.e. existing variable definitions must not be overwritten.
During runtime, the ethernet layer will use the environment variables to sync the MAC addresses to the ethernet structures. All ethernet driver code should then only use the enetaddr member of the eth_device structure. This is done on every network command, so the ethernet copies will stay in sync.
Let us suppose a device has an EEPROM containing a universally administered MAC address and that the device's misc_init_r() function reads that MAC address and writes it to an environment variable (e.g. "ethaddr") if and only if the environment variable doesn't exist. Let us also suppose that the environment is later saved by the U-Boot saveenv command, and moreover, that the whole system is running off a removable storage device such as an SD card, where the U-Boot environment is also saved.
EDIT
In my case, I am in the above situation for a new custom board which has a "MAC address EEPROM" from Micronix containing a unique, universally administered MAC address, and the board's only available storage location for a saved U-Boot environment is the SD card from which it boots. I have the following code for the board-specific misc_init_r() function:
#include <common.h>
#ifndef CONFIG_SPL_BUILD
static int my_board_read_mac(uchar *mac)
{
int ret;
/* EEPROM is at bus 0. */
ret = i2c_set_bus_num(0);
if (ret) {
printf("Cannot select EEPROM I2C bus - err %d\n", ret);
return ret;
}
/* EEPROM is at address 0x50. MAC address is at offset 0xfa. */
ret = eeprom_read(0x50, 0xfa, mac, 6);
if (ret) {
printf("Cannot read I2C EEPROM - err %d\n", ret);
return ret;
}
return 0;
}
static int my_board_do_mac(void)
{
int ret;
uchar mac[6];
ret = my_board_read_mac(mac);
if (ret) {
printf("Failed to read MAC address - err %d\n", ret);
return ret;
}
if (!is_valid_ethaddr(mac)) {
printf("Read invalid MAC address %pM\n", mac);
return -EINVAL;
}
if (!getenv("ethaddr")) {
return eth_setenv_enetaddr("ethaddr", mac);
}
return 0;
}
int misc_init_r(void)
{
int ret;
ret = my_board_do_mac();
if (ret) {
printf("Failed to set MAC address - err %d\n", ret);
}
return 0;
}
#endif
My question is: If the SD card is removed from the original device and placed into a similar device (with a different universally administered MAC address in its EEPROM), will the new device use the original device's MAC address rather than it's own, unique MAC address? If so, what is the best way to prevent that from happening?
EDIT2
What I'm looking for is some way to set the MAC address automatically, but not to "pollute" any copy of the environment saved on the SD card with this board specific MAC address. I.e. I want the MAC address to remain tied to the board, rather than to the SD card.
The answer is that this becomes a policy question. The non-default environment (so, what gets saved to someplace when you use saveenv) is intended to be instance-specific. This is why generally speaking board code will see if ethaddr is set, and if so, not try and derive one.
When you move from board to board you can use env default -f -a to restore the running environment to the built-in defaults.
If you want to share the same SD card between multiple boards then you need to come up with a policy to manage this situation.
I am working on kernel extension and want to find out how to find process name by pid in kernel extension
This code works great in user space
static char procdata[4096];
int mib[3] = { CTL_KERN, KERN_PROCARGS, pid };
procdata[0] = '\0'; // clear
size_t size = sizeof(procdata);
if (sysctl(mib, 3, procdata, &size, NULL, 0)) {
return ERROR(ERROR_INTERNAL);
}
procdata[sizeof(procdata)-2] = ':';
procdata[sizeof(procdata)-1] = '\0';
ret = procdata;
return SUCCESS;
but for the kernel space, there are errors such as "Use of undeclared identifier 'CTL_KERN'" (even if I add #include )
What is the correct way to do it in kernel extension?
The Kernel.framework header <sys/proc.h> is what you're looking for.
In particular, you can use proc_name() to get a process's name given its PID:
/* this routine copies the process's name of the executable to the passed in buffer. It
* is always null terminated. The size of the buffer is to be passed in as well. This
* routine is to be used typically for debugging
*/
void proc_name(int pid, char * buf, int size);
Note however, that the name will be truncated to MAXCOMLEN - 16 bytes.
You might also be able to use the sysctl via sysctlbyname() from the kernel. In my experience, that function doesn't work well though, as the sysctl buffer memory handling isn't expecting buffers in kernel address space, so most types of sysctl will cause a kernel panic if called from a non-kernel thread. It also doesn't seem to work for all sysctls.