Linux kernel module - accessing memory mapping - linux-kernel

I'm running into some odd issue on kernel module load that I'm suspecting having to do with linking and loading. How to I programmatically figure out the address of each section after they are loaded in memory (from inside the module itself). Like where is .bss / .data / .text and so on.
From reading this article
https://lwn.net/Articles/90913/
It is sorta in the directly that I'm looking for.

You can see the sections begin addresses like this from userspace (need root permissions):
sudo cat /sys/module/<modulename>/sections/.text
I have browsed how syfs retrieves this addresses, and i found the following:
There is a section attributes in struct module
309 /* Section attributes */
310 struct module_sect_attrs *sect_attrs;
This attrs is a bunch of attr structs
1296 struct module_sect_attrs {
1297 struct attribute_group grp;
1298 unsigned int nsections;
1299 struct module_sect_attr attrs[0];
1300 };
where sect attr is the thing you are looking for
1290 struct module_sect_attr {
1291 struct module_attribute mattr;
1292 char *name;
1293 unsigned long address;

From the module's code THIS_MODULE macro is actually a pointer to the struct module object. Its module_init and module_core fields point to memory regions, where all module sections are loaded.
As I understand, sections division is inaccessible from the module code(struct load_info is dropped after module is loaded into memory). But having module's file you can easily deduce section's addresses after load:
module_init:
- init sections with code (.init.text)
- init sections with readonly data
- init sections with writable data
module_core:
- sections with code (.text)
- sections with readonly data
- sections with writable data
If several sections suit to one category, they are placed in the same order, as in the module's file.
Within module's code you can also print address of any its symbol, and after calculate start of the section, contained this symbol.

While this question is five years old, I thought I would contribute my two-cents. I was able to access the kernel's sections in a sort of hack-y way inspired by Alex Hoppus' answer. I don't advocate doing things this way, unless you are writing the kernel module to debug things or understand the kernel etc.
Anyway, I copy the following two structs into my module to help resolve incomplete types.
struct module_sect_attr {
struct module_attribute mattr;
char *name;
unsigned long address;
};
struct module_sect_attrs {
struct attribute_group grp;
unsigned int nsections;
struct module_sect_attr attrs[0];
};
Then, in my module initialization function, I do the following to get the section addresses.
unsigned long text = 0;
unsigned int nsections = 0;
unsigned int i;
struct module_sect_attr* sect_attr;
nsections = THIS_MODULE->sect_attrs->nsections;
sect_attr = THIS_MODULE->sect_attrs->attrs;
for (i = 0; i < nsections; i++) {
if (strcmp((sect_attr + i)->name, ".text") == 0)
text = (sect_attr + i)->address;
}
Finally, it should be noted that if you are looking for the address of .rodata, .bss, or .data you will need to define constant global variables, uninitialized global variables, or regular global variables, respectively, if you don't want those sections to be omitted.

Related

Structure defined in a dynamically loaded library

I am dynamically loading the cudart (Cuda Run Time Library) to access just the cudaGetDeviceProperties function. This one requires two arguments:
A cudaDeviceProp structure which is defined in a header of the run time library;
An integer which represents the device ID.
I am not including the cuda_runtime.h header in order to not get extra constants, macros, enum, class... that I do not want to use.
However, I need the cudaDeviceProp structure. Is there a way to get it without redefining it? I wrote the following code:
struct cudaDeviceProp;
class CudaRTGPUInfoDL
{
typedef int(*CudaDriverVersion)(int*);
typedef int(*CudaRunTimeVersion)(int*);
typedef int(*CudaDeviceProperties)(cudaDeviceProp*,int);
public:
struct Properties
{
char name[256]; /**< ASCII string identifying device */
size_t totalGlobalMem; /**< Global memory available on device in bytes */
size_t sharedMemPerBlock; /**< Shared memory available per block in bytes */
int regsPerBlock; /**< 32-bit registers available per block */
int warpSize; /**< Warp size in threads */
size_t memPitch; /**< Maximum pitch in bytes allowed by memory copies */
/*... Tons of members follow..*/
};
public:
CudaRTGPUInfoDL();
~CudaRTGPUInfoDL();
int getCudaDriverVersion();
int getCudaRunTimeVersion();
const Properties& getCudaDeviceProperties();
private:
QLibrary library;
private:
CudaDriverVersion cuDriverVer;
CudaRunTimeVersion cuRTVer;
CudaDeviceProperties cuDeviceProp;
Properties properties;
};
As everybody can see, I simply "copy-pasted" the declaration of the structure.
In order to get the GPU properties, I simply use this method:
const CudaRTGPUInfoDL::Properties& CudaRTGPUInfoDL::getCudaDeviceProperties()
{
// Unsafe but needed.
cuDeviceProp(reinterpret_cast<cudaDeviceProp*>(&properties), 0);
return properties;
}
Thanks for your answers.
If you need the structure to be complete, you should define it (probably by including the appropriate header).
If you're just going to be passing around references or pointers, such as in the method you show, then it doesn't need to be complete and can just be forward declared:
class cudaDeviceProp;

How to attach file operations to sysfs attribute in platform driver?

I wrote a platform driver for a peripheral we developed and would like to expose some configuration options to the sysfs. I have managed to create the appropriate files using attribute structs (see below) and sysfs_create_file in the probe function, but I can't figure out how to attach the show/store functions to the structs in a platform driver.
Most resources I found online used a device_attribute struct or something similar to create their files, is that also appropriate here? Is there another way to do this for a platform driver?
My attribute struct looks like this:
struct attribute subkey_attr = {
.name = "subkeys",
.mode = S_IWUGO | S_IRUGO,
};
And I register the file using this call:
riddler_kobject = &pdev->dev.kobj;
ret_val = sysfs_create_file(riddler_kobject, &subkey_attr);
It boils down to next:
reuse existing kobject from struct device (from your struct platform_device) for sysfs_create_group() (instead of creating your own kobject)
use DEVICE_ATTR() to declare struct device_attribute instead of regular __ATTR(), which creates struct kobj_attribute.
Here is how I created sysfs attributes for my platform driver.
Create structure you'll be using as private data in show() / store() operations for your sysfs attribute (file). For example:
struct mydrv {
struct device *dev;
long myparam;
};
Allocate this structure in your driver's probe():
static int mydrv_probe(struct platform_device *pdev)
{
struct mydrv *mydrv;
mydrv = devm_kzalloc(&pdev->dev, sizeof(*mydrv), GFP_KERNEL);
mydrv->dev = &pdev->dev;
platform_set_drvdata(pdev, mydrv);
...
}
Create show() / store() functions:
static ssize_t mydrv_myparam_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
struct mydrv *mydrv = dev_get_drvdata(dev);
int len;
len = sprintf(buf, "%d\n", mydrv->myparam);
if (len <= 0)
dev_err(dev, "mydrv: Invalid sprintf len: %d\n", len);
return len;
}
static ssize_t mydrv_myparam_store(struct device *dev,
struct device_attribute *attr, const char *buf, size_t count)
{
struct mydrv *mydrv = dev_get_drvdata(dev);
kstrtol(buf, 10, &mydrv->myparam);
return count;
}
Create device attribute for those functions (right after those functions):
static DEVICE_ATTR(myparam, S_IRUGO | S_IWUSR, mydrv_myparam_show,
mydrv_myparam_store);
Declare attributes table (listing in fact sysfs files for you driver):
static struct attribute *mydrv_attrs[] = {
&dev_attr_myparam.attr,
NULL
};
Declare attribute group (specifying in fact sysfs directory for your driver):
static struct attribute_group mydrv_group = {
.name = "mydrv",
.attrs = mydrv_attrs,
};
static struct attribute_group *mydrv_groups[] = {
&mydrv_group,
NULL
}
which can be actually replaced with one line:
ATTRIBUTE_GROUPS(mydrv);
Create sysfs directory and files in your driver's probe() function:
static int mydrv_probe(struct platform_device *pdev)
{
int ret;
...
ret = sysfs_create_group(&pdev->dev.kobj, &mydrv_group);
if (ret) {
dev_err(&pdev->dev, "sysfs creation failed\n");
return ret;
}
...
}
Remove your sysfs files in your driver's remove() function:
static int mydrv_remove(struct platform_device *pdev)
{
sysfs_remove_group(&pdev->dev.kobj, &mydrv_group);
...
}
Race condition note
As #FranzForstmayr correctly pointed out, there may be race condition when adding sysfs files with sysfs_create_group() in mydrv_probe(). That's because user-space can be already notified that those files exist before mydrv_probe() called (where those files are actually being created by sysfs_create_group() function). This issue covered in details in "How to Create a sysfs File Correctly" article by Greg Kroah-Hartman.
So in our case of platform_device, instead of calling sysfs_create_group() (and its counterpart sysfs_remove_group()), you can use default attribute group. To do so, you need to assign corresponding .groups field of your struct device to your attribute groups variable:
static int mydrv_probe(struct platform_device *pdev)
{
...
pdev->dev.groups = mydrv_groups;
...
}
DISCLAIMER: I didn't test this code, though it should work, because of this code.
See [1,2,3] links for more insights on mentioned race condition.
For more examples, run next command in kernel source directory:
$ git grep -l --all-match -e platform_device -e attribute -e '\.groups =' -- drivers/
Also you can search by "default attribute" in commit messages:
$ git log --no-merges --oneline --grep="default attribute" -- drivers/
Some commits I found this way: [4,5,6,7].
References
[1] My attributes are way too racy, what should I do?
[2] PATCH: sysfs: add devm_sysfs_create_group() and friends
[3] [GIT PATCH] Driver core patches for 3.11-rc2
[4] commit 1
[5] commit 2
[6] commit 3
[7] commit 4
Not enough reputation to post a comment, but I just want to comment on the default attribute group note from the accepted answer.
My understanding is that this should not be added in the probe function, as given in the example, but instead should be set in the device struct, (or device_driver, class, or bus depending on your driver) usually defined at the end of your file.
For example:
static struct device iio_evgen_dev = {
.bus = &iio_bus_type,
.groups = iio_evgen_groups,
.release = &iio_evgen_release,
};
from this example
Strangely, according to this it doesn't work correctly when using DEVICE_INT_ATTR to create the attribute, so not sure what that's all about.
Also, I'm not 100% sure, but I think that this is invoked when the driver is loaded, not when the device is probed.
This is an addition to Sam Protsenko's and Anthony's answers
If you create device attributes via the DEVICE_ATTR macros then you have to register the attribute groups (mydrv_groups) in the .dev_groups instead of the .groups field.
static struct device iio_evgen_dev = {
.bus = &iio_bus_type,
.dev_groups = iio_evgen_groups, // .dev_groups for DEVICE_ATTR
.groups = another_attr_group, // .groups for DRIVER_ATTR
.release = &iio_evgen_release,
};
Then the attributes are automatically registered correctly without setting up anything in the probe/remove functions, as described in Greg Kroah-Hartman's article.
Assume that the module has been loaded into the kernel and the driver is registered in
/sys/bus/platform/drivers/mydrv
every device instances will be a subdirectory of the driver's folder like
/sys/bus/platform/drivers/mydrv/mydrv1
/sys/bus/platform/drivers/mydrv/mydrv2
Registering attributes in the .groups field creates the attributes in the driver folder.
Registering attributes in the .dev_groups field creates the attributes in the device's instance folder.
mydrv
├── driver_attr1
├── driver_attr2
└── mydrv1
├── device_attr1
└── device_attr2
The show/store functions of the attributes in the .groups field do not have access to the driver data set via platform_set_drvdata(pdev, mydrv).
At least not by accessing it via dev_get_drvdata(dev).
Accessing the driver data via dev_get_drvdata(dev) returns NULL and dereferencing it will result in a kernel oops.
In turn, he show/store functions of the attributes in the .dev_groups field have access to the driver data via
struct mydrv *mydrv = dev_get_drvdata(dev);

Dynamically creating variable in Kernel Module

I am planning to use kthread_run API in my kernel module.
As kthread_run, returns a struct task_struct *, which i want to store in a variable global across my module.
However, i want one thread each for a cpu and i obtain no of cpus using num_online_cpus.
However, when i write following code:
//outside init_module function
int cpus = num_online_cpus();
static struct task_struct *my_tasks[cpus];
static int __init init_module(){
for(int j = 0; j < cpus; ++j){
my_tasks[j] = kthread_run(...);
}
}
However, i get following error on this:
error: variably modified ‘tasks’ at file scope
How can i achieve this ???
If your variables are actually one per cpu, you might want to use the per_cpu macros. The gist is, you declare such a variable with:
DEFINE_PER_CPU(struct task_struct, my_tasks);
and then access the variable using
get_cpu_var(my_tasks).foo = bar;
You can get more info at See http://www.makelinux.net/ldd3/chp-8-sect-5 (or percpu.h) for more details.

What is the use of 'i2c_get_clientdata" and "i2c_set_clientdata"

I have been studying I2C driver (client) code for a while.
I have seen this function "i2c_get_clientdata" and "i2c_set_clientdata" every where.
I have seen the this question here .
Use of pointer to structure instead of creating static local copy
Some times i think like it is like "container_of" macro to get a pointer to the structure.
But still i didn't understood properly why to use it and when to use it.
Below i am posting a sample code where I see its usage.
If any one could help me understand why it is used there and when we shall use it when we write our own drivers.
struct max6875_data {
struct i2c_client *fake_client;
struct mutex update_lock;
u32 valid;
u8 data[USER_EEPROM_SIZE];
unsigned long last_updated[USER_EEPROM_SLICES];
};
static ssize_t max6875_read(struct file *filp, struct kobject *kobj,
struct bin_attribute *bin_attr,
char *buf, loff_t off, size_t count)
{
struct i2c_client *client = kobj_to_i2c_client(kobj);
struct max6875_data *data = i2c_get_clientdata(client);
int slice, max_slice;
if (off > USER_EEPROM_SIZE)
return 0;
if (off + count > USER_EEPROM_SIZE)
count = USER_EEPROM_SIZE - off;
/* refresh slices which contain requested bytes */
max_slice = (off + count - 1) >> SLICE_BITS;
for (slice = (off >> SLICE_BITS); slice <= max_slice; slice++)
max6875_update_slice(client, slice);
memcpy(buf, &data->data[off], count);
return count;
}
Those functions are used to get/set the void *driver_data pointer that is part of the struct device, itself part of struct i2c_client.
This is a void pointer that is for the driver to use. One would use this pointer mainly to pass driver related data around.
That is what is happening in your example. The max6875_read is a callback getting a structu kobject. That kobject is an i2c_client which is enough to communicate with the underlying device using the driver_data pointer here allows to get back the driver related data (instead of using global variables for example).

sysfs: free to use struct device platform_data field?

Summary: is the platform_data field of struct device free to use in a device driver module?
I am creating a very simple sysfs entry for my character device driver module to allow me to control an internal variable (because I know using ioctl() and the proc filesystem are deprecated.) I call class_create() to make a class in /sys/class/ and then device_create() to make a new device entry. Then I call device_create_file() to set up my load and store routines for the driver. I want to lock my driver in these routines. I have a mutex in my driver's main structure. Can I use the platform_data field to store a pointer to this structure like I would the private_data field of struct file in the module's open() routine or is this reserved? It's set to NULL after device_create so it would appear OK but I don't know for sure.
What I'd like to do is:
struct mymodule mymod; // main module structure, has a mutex called lockmx
static ssize_t mydev_store_val(struct device *dev,
struct device_attribute *attr,
const char *buf,size_t count)
{
struct mymodule *mymodp=(struct mymodule*)dev->platform_data;
if(mutex_lock_interruptible(&mymodp->lockmx))
return 0;
// get data from buf
mutex_unlock(&mymodp->lockmx);
return count;
}
DEVICE_ATTR(mydeva,S_IWUSR|S_IRUGO,NULL,mydev_store_val);
static int __init modinit(void)
{
...
dev_t dev; // alloc'ed already
myclass=class_create(THIS_MODULE,"myclass");
mydev=device_create(myclass,NULL,dev,NULL,"mydev");
mydev->platform_data=&mymod;
device_create_file(mydev,&dev_attr_mydeva);
...
}
So this will create the entry /sys/class/myclass/mydev/mydeva which can be written to. If the platform_data field is available then I can avoid using globals. But if it moves under me my kernel is going to oops at best and probably panic.
Such a pointer can be stored in the drvdata field (which has been cleverly hidden so that you will not see it if you look at the definition of struct device).
Initialize it through the fourth parameter of device_create, and read it with dev_get_drvdata:
mydev = device_create(myclass, NULL, dev, &mymod, "mydev");
...
struct mymodule *mymodp = dev_get_drvdata(dev);

Resources