I/O to device from kernel module fails with EFAULT - linux-kernel

I have created block device in kernel module. When some I/O happens I read/write all data from/to another existing device (let's say /dev/sdb).
It opens OK, but read/write operations return 14 error(EFAULT,Bad Address). After some research I found that I need map address to user space(probably buffer or filp variables), but copy_to_user function does not help. Also I looked to mmap() and remap_pfn_range() functions, but I can not get how to use them in my code, especially where to get correct vm_area_struct structure. All examples that I found, used char devices and file_operations structure, not block device.
Any hints? Thanks for help.
Here is my code for reading:
mm_segment_t old_fs;
old_fs = get_fs();
set_fs(KERNEL_DS);
filp = filp_open("/dev/sdb", O_RDONLY | O_DIRECT | O_SYNC, 00644);
if(IS_ERR(filp))
{
set_fs(old_fs);
int err = PTR_ERR(filp);
printk(KERN_ALERT"Can not open file - %d", err);
return;
}
else
{
bytesRead = vfs_read(filp, buffer, nbytes, &offset); //It gives 14 error
filp_close(filp, NULL);
}
set_fs(old_fs);

I found a better way for I/O to block device from kernel module. I have used bio structure for that. Hope this information save somebody from headache.
1) So, if you want to redirect I/O from your block device to existing block device, you have to use own make_request function. For that you should use blk_alloc_queue function to create queue for your block device like this:
device->queue = blk_alloc_queue(GFP_KERNEL);
blk_queue_make_request(device->queue, own_make_request);
Than into own_make_request function change bi_bdev member into bio structure to device in which you redirecting I/O and call generic_make_request function:
bio->bi_bdev = device_in_which_redirect;
generic_make_request(bio);
More information here at 16 chapter. If link is broken by some cause, here is name of the book - "Linux Device Drivers, Third Edition"
2) If you want read or write your own data to existing block device from kernel module you should use submit_bio function.
Code for writing into specific sector(you need to implement writeComplete function also):
void writePage(struct block_device *device,
sector_t sector, int size, struct page *page)
{
struct bio *bio = bio_alloc(GFP_NOIO, 1);
bio->bi_bdev = vnode->blkDevice;
bio->bi_sector = sector;
bio_add_page(bio, page, size, 0);
bio->bi_end_io = writeComplete;
submit_bio(WRITE_FLUSH_FUA, bio);
}
Code for reading from specific sector(you need to implement readComplete function also):
int readPage(struct block_device *device, sector_t sector, int size,
struct page *page)
{
int ret;
struct completion event;
struct bio *bio = bio_alloc(GFP_NOIO, 1);
bio->bi_bdev = device;
bio->bi_sector = sector;
bio_add_page(bio, page, size, 0);
init_completion(&event);
bio->bi_private = &event;
bio->bi_end_io = readComplete;
submit_bio(READ | REQ_SYNC, bio);
wait_for_completion(&event);
ret = test_bit(BIO_UPTODATE, &bio->bi_flags);
bio_put(bio);
return ret;
}
page can be allocated with alloc_page(GFP_KERNEL). Also for changing data in page use page_address(page). It returns void* so you can interpret that pointer as whatever you want.

Related

MMAP buffer kernel writes are not seen by user space

i have a kernel driver which shares a buffer with the user space layer.
Everything seemed to work fine in my VM prototype (Ubuntu, Kernel 5.4) but when i moved my code to the target (same kernel but this is an embedded distro) I can clearly see that Kernel writes to the buffer (using memcpy, or memset) are not reflected in the User space side of the buffer.
Note that, i use direct buffer accesses on both sides. There is no concurrency issue, as the Kernel writes to, then the user space reads from.
I ended up believing this is a cache issue ... as the same code works perfectly in my VM.
The buffer size is 4 * PAGE_SIZE.
It is allocated as follows:
int _size = (SFP_BUFFER_SIZE + (PAGE_SIZE-1)) & ~(PAGE_SIZE-1);
input_buffer = (char*) kzalloc (_size, GFP_KERNEL); // aligned on page boundary
if (!input_buffer) {
dev_dbg(&dev, "open/ENOMEM (input_buffer)\n");
status = -ENOMEM;
goto err_all
When mmap'ing, i used the following code pattern:
vma->vm_ops = &fpgadrv_vm_ops;
vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
pfn = virt_to_phys((void*)(input_buffer)) >> PAGE_SHIFT;
if (remap_pfn_range (vma, vma->vm_start, pfn, size, vma->vm_page_prot))
{
printk(KERN_DEBUG "remap page range failed\n");
return -EAGAIN;
}
User space code, and kernel code user memcpy to update the buffer. Note also that I cannot use write/read entry points, as they are already used for very specific operations.
The user code is calling mmap as follows:
buf = mmap(NULL, BUF_SIZE, PROT_READ | PROT_WRITE, MAP_SHARED, device_fd, 0);
if (buf == MAP_FAILED)
{
perror("USERDRV:cannot mmap");
return -1; // for testing, ignore the return code and continue
}
and upon IOCTL call, the kernel would fill up the mmap buffer as follows:
case IOCTL_RESET:
printk(KERN_DEBUG "FPGADRV: IOCTL RESET");
// reset the buffer (zero + put back the signature)
memset(input_buffer, 0xA5, SFP_BUFFER_SIZE);
memcpy((void*)(input_buffer), (void*)signature, 10);
break;
Is there something more i should do to make sure the pages are not cached (assuming this is the cause of my pb) ?
Thanks,
Jacques

Busy inodes/dentries after umount in self-written virtual fs

I wrote a simple FS that should only statically contain one file named hello. This file should contain the string Hello, world!. I did this for educational purposes. While the fs is mounted it actually behaves like expected. I can read the file just fine.
However after unmounting I always get
VFS: Busy inodes after unmount of dummyfs. Self-destruct in 5 seconds. Have a nice day...
If I called ls on the rootdir while the fs was mounted I get
BUG: Dentry (ptrval){i=2,n=hello} still in use (-1) [unmount of dummyfs dummyfs]
on top of that.
What does this mean in detail and how can I fix it?
The mount and kill_sb routines call mount_nodev and allocate space for a struct holding the 2 inodes this FS uses.
static struct dentry *dummyfs_mount(struct file_system_type* fs_type,
int flags, const char* dev_name, void* data)
{
struct dentry *ret;
ret = mount_nodev(fs_type, flags, data, dummyfs_fill_super);
if (IS_ERR(ret)) {
printk(KERN_ERR "dummyfs_mount failed");
}
return ret;
}
static void dummyfs_kill_sb(struct super_block *sb) {
kfree(sb->s_fs_info);
kill_litter_super(sb);
}
The fill superblock method creates the 2 inodes and saves them in the struct allocated by mount:
static int dummyfs_fill_super(struct super_block *sb, void *data, int flags)
{
struct dummyfs_info *fsi;
sb->s_magic = DUMMYFS_MAGIC;
sb->s_op = &dummyfs_sops;
fsi = kzalloc(sizeof(struct dummyfs_info), GFP_KERNEL);
sb->s_fs_info = fsi;
fsi->root = new_inode(sb);
fsi->root->i_ino = 1;
fsi->root->i_sb = sb;
fsi->root->i_op = &dummyfs_iops;
fsi->root->i_fop = &dummyfs_dops;
fsi->root->i_atime = fsi->root->i_mtime = fsi->root->i_ctime = current_time(fsi->root);
inode_init_owner(fsi->root, NULL, S_IFDIR);
fsi->file = new_inode(sb);
fsi->file->i_ino = 2;
fsi->file->i_sb = sb;
fsi->file->i_op = &dummyfs_iops;
fsi->file->i_fop = &dummyfs_fops;
fsi->file->i_atime = fsi->file->i_mtime = fsi->file->i_ctime = current_time(fsi->file);
inode_init_owner(fsi->file, fsi->root, S_IFREG);
sb->s_root = d_make_root(fsi->root);
return 0;
}
The lookup method just adds the fsi->file_inode to the dentry if the parent is the root dir:
if (parent_inode->i_ino == fsi->root->i_ino) {
d_add(child_dentry, fsi->file);
}
And the iterate method just emits the dot files and the hello file when called:
if (ctx->pos == 0) {
dir_emit_dots(file, ctx);
ret = 0;
}
if (ctx->pos == 2) {
dir_emit(ctx, "hello", 5, file->f_inode->i_ino, DT_UNKNOWN);
++ctx->pos;
ret = 0;
}
The read method just writes a static string using copy_to_user. The offsets are calculated correctly and on EOF the method just returns 0. However since the problems occur even when the read method was not called I think it is out-of-scope for this already too long question.
For actually running this I use user-mode linux from the git master (4.15+x commit d48fcbd864a008802a90c58a9ceddd9436d11a49). The userland is compiled from scratch and the init process is a derivative of Rich Felker's minimal init to which i added mount calls for /proc, /sys and / (remount).
My command line is ./linux ubda=../uml/image root=/dev/ubda
Any pointers to more thorough documentation are also appreciated.
Using gdb watching the dentry->d_lockref.count I realized that the kill_litter_super call in umount was actually responsible for the dentry issues. Replacing it with kill_anon_super solved that problem.
The busy inode problem vanished too mostly except when i unmounted after immediately after mounting. Allocating the second inode lazily solved that problem too.

How to read/write to an USB storage device with a linux driver?

During the attempt to write my own simple usb driver for an usb-flash-drive, I got stuck reading the data that I wrote to the device.
So, my first question is:
How is the transfer and the storage on a device going on? (in detail)
I know I have to perform the following steps:
Create an urb (USB request block)
Allocate a DMA buffer
Transfer the data from the user-space into the DMA buffer
Send the data through a pipe to the device
I couldn't find any documentation on how a device handles this data.
Is this even possible to write such a driver, or would it be necessary to disassemble the usb device, to send special commands?
The code I have written looks something like the following and is from the ldd3 and "http://lxr.free-electrons.com/source/drivers/usb/usb-skeleton.c". It only shows a shortened version of the important functions.
After loading the driver into the kernel, I can write to the device without any error, but if I read, an EPIPE error occurs. Ldd3 mentions that the usb_clear_halt() could solve this problem, but it doesn't.
// This function is called when the device is plugged in
static int my_driver_probe(struct usb_interface* interface, const struct usb_device_id* id)
{
struct usb_skel* dev = NULL;
struct usb_device* udev = interface_to_usbdev(interface);
struct usb_host_interface* iface_desc;
struct usb_endpoint_descriptor* endpoint;
int retval = -ENODEV;
int i = 0;
size_t buffer_size;
dev = kzalloc(sizeof(struct usb_skel), GFP_KERNEL);
// Check vendor and product id
// …
dev->udev = udev;
dev->interface = interface;
// Set up the endpoint information
iface_desc = interface->cur_altsetting;
for(i=0; i < iface_desc->desc.bNumEndpoints; ++i) {
endpoint = &iface_desc->endpoint[i].desc;
if(!dev->bulk_in_endpointAddr && usb_endpoint_is_bulk_in(endpoint)) {
buffer_size = endpoint->wMaxPacketSize;
dev->bulk_in_size = buffer_size;
dev->bulk_in_endpointAddr = endpoint->bEndpointAddress;
dev->bulk_in_buffer = kmalloc(buffer_size, GFP_KERNEL);
if(!dev->bulk_in_buffer) {
printk("Could not allocate bulk_in_buffer\n");
goto error;
}
dev->bulk_in_urb = usb_alloc_urb(0, GFP_KERNEL);
}
if(!dev->bulk_out_endpointAddr && usb_endpoint_is_bulk_out(endpoint))
dev->bulk_out_endpointAddr = endpoint->bEndpointAddress;
}
// Check that the endpoints are set
// …
// Save our data pointer in this interface device
usb_set_intfdata(interface, dev);
// Register the device
retval = usb_register_dev(interface, &class_descr);
return retval;
}
// Is called when another program writes into /dev/my_usb_driver
static ssize_t my_driver_write( struct file* file, const char __user* user_buffer, size_t count, loff_t* offs)
{
struct usb_skel* dev = file->private_data;
struct urb* urb = NULL;
char* buf = NULL;
int retval = 0;
size_t writesize = min(count, (size_t)MAX_TRANSFER);
// Create a urb, and a buffer for it, and copy the data to the urb
urb = usb_alloc_urb(0, GFP_KERNEL);
// Creates a DMA buffer
buf = usb_alloc_coherent(dev->udev, writesize, GFP_KERNEL, &urb->transfer_dma);
// The data that is passed to the driver should be copied into the DMA buffer
copy_from_user(buf, user_buffer, writesize;
// Initialize the urb proberly
usb_fill_bulk_urb(urb, dev->udev,
usb_sndbulkpipe(dev->udev, dev->bulk_out_endpointAddr),
buf, writesize, (void*)my_write_bulk_callback, dev);
// Send the data out the bulk port
urb->transfer_flags |= URB_NO_TRANSFER_DMA_MAP;
usb_submit_urb(urb, GFP_KERNEL);
return writesize;
}
// Is called when another program reads from /dev/my_usb_driver
static ssize_t my_driver_read( struct file *file, char* buffer, size_t count, loff_t* offs)
{
struct usb_skel* dev = file->private_data;
int retval = 0;
// Check that we have data to read
// …
usb_fill_bulk_urb(dev->bulk_in_urb, dev->udev,
usb_rcvbulkpipe(dev->udev, dev->bulk_in_endpointAddr),
dev->bulk_in_buffer,
min(dev->bulk_in_size, count), read_bulk_callback, dev);
retval = usb_submit_urb(dev->bulk_in_urb, GFP_KERNEL);
// If the read was succesful, copy the data to user space
copy_to_user(buffer, dev->bulk_in_buffer, count);
return retval;
}
USB is just a transport layer. Storage devices generally implement SCSI protocol. Create a SCSI command for reading or writing from the data that user space has sent. Then create URB for the SCSI command and send it to the USB device.
SCSI is a huge protocol, for learning USB device driver development it is better to start with simple devices like USB to serial devices.

corrupted pointer in 'net_device'

the device driver I'm working on is implementing a virtual device. The logic
is as follows:
static struct net_device_ops virt_net_ops = {
.ndo_init = virt_net_init,
.ndo_open = virt_net_open,
.ndo_stop = virt_net_stop,
.ndo_do_ioctl = virt_net_ioctl,
.ndo_get_stats = virt_net_get_stats,
.ndo_start_xmit = virt_net_start_xmit,
};
...
struct net_device *dev;
struct my_dev *virt;
dev = alloc_netdev(..);
/* check for NULL */
virt = netdev_priv(dev);
dev->netdev_ops = &virt_net_ops;
SET_ETHTOOL_OPS(dev, &virt_ethtool_ops);
dev_net_set(dev, net);
virt->magic = MY_VIRT_DEV_MAGIC;
ret = register_netdev(dev);
if (ret) {
printk("register_netdev failed\n");
free_netdev(dev);
return ret;
}
...
What happens is that somewhere somehow the pointer net_device_ops in
'net_dev' gets corrupted, i.e.
1) create the device the first time (allocated net_dev, init the fields
including net_device_ops,which is
initialized with a static structure containing function pointers), register
the device with the kernel invoking register_netdev() - OK
2) attempt to create the device with the same name again, repeat the above
steps, call register_netdev() which will return negative and we
free_netdev(dev) and return error to the caller.
And between these two events the pointer to net_device_ops has changed,
although nowhere in the code it is done explicitly except the initialization
phase.
The kernel version is 2.6.31.8, platform MIPS. Communication channel between the user space and the kernel is implemented via netlink sockets.
Could anybody suggest what possibly can go wrong?
Appreciate any advices, thanks.
Mark
"The bug is somewhere else. "
The second device should not interact with the existing one. If you register_netdev with an existing name, nevertheless the ndo_init virtual function is called first before the condition is detected and -EEXIST is returned. Maybe your init function does something nasty involving some global variables. (For example, does the code assume there is one device, and stash a global pointer to it during initialization?)

Drive Letter to Device Instance ID

How do I get from a drive letter to a device instance ID?
My process starts with a device arrival message. I have been successful in getting the drive letter from the arrival message and in opening the dvd tray.
I have searched the various Setup API items; but I haven't found anything that gets me from a drive letter to a device instance ID.
A solution in C# or VB.NET would be ideal, but I'm willing to figure it out from any other language as long as I can see the API calls.
Thanks in advance...
You cannot do it directly.
The link is to use STORAGE_DEVICE_NUMBER. You can use DeviceIoControl with IOCTL_STORAGE_GET_DEVICE_NUMBER on your device name to populate this structure. Put this value to one side.
You then need to get device infomation on your system using SetupDiGetClassDevs setting the GUIDS as approriate, indicicating the drives your are insterested in. Then enumerate through the devices using SetupDiEnumDeviceInfo. Then enumerate the interfaces using SetupDiEnumDeviceInterfaces and finally get the information using SetupDiGetDeviceInterfaceDetail. In this structure returned you can get a DevicePath you can use to get the STORAGE_DEVICE_NUMBER as above. Match this with the STORAGE_DEVICE_NUMBER from your drive letter, and you have now linked a driver letter to your structure. Phew! Inside this structure is a DevInst.
i know it's late for you now but not for everybody ^^
I had the same need and this is main line of how I did it:
-You need a window to receive device arrival and removal (as you said)
-Then you create a DeviceNotificationFilter initiated to dbcc_devicetype = DBT_DEVTYP_DEVICEINTERFACE
-Then in the message loop of your window you look for VM_DEVICECHANGE
-When u receive it if wParam == DBT_DEVICEARRIVAL, use the lParam to check if it is a DBT_DEVTYPE_VOLUME (i was getting the letter and the type of the drive here) or a DBT_DEVTYPE_DEVICEINTERFACE ( there you can use your wellcasted lParam to get the InstanceId from the input structure).
When you connect a drive your receive DEVINTERFACE first then the other.
I only give the main line beacause i did this long time ago and i don't have the code here, and also I had found a lot of code pieces on the net (long time ago so there should be more now ^^^) maybe msdn give a full code example to do that now.
If you read this and need more informations, i'll reply or make a full documented answer if many need it.
Hope it will help some of you.
I know it's years later but I had to do this and searching brought me here and #DanDan 's answer worked. In order to save future people a lot of work, I thought I'd give back a little and present the technique a bit more explicitly. You'll still have to write a bit of code, but the part I found difficult is below as code:
As DanDan mentioned, the idea is to use CreateFile and DeviceIoControl to get the Windows STORAGE_DEVICE_NUMBER for the disk associated with a file path, and then use the Setup API to enumerate disk devices until we find one whose device instance equals the SDN.
First, here's a summary of how you get the STORAGE_DEVICE_NUMBER from the path (e.g. c:\\users\\bob);
Strip the path to the root (e.g down to C:) and prepend it with \\\\.\\ so you have \\\\.\\C:
Open that path up using CreateFileW with to get metadata
Use DeviceIoControl with IOCTL_VOLUME_GET_VOLUME_DISK_EXTENTS to get the extents
Get the DiskNumber member from the first extent returned.
Close the file
Open up \\\\.\\PhysicalDrive<n> where <n> is the that DiskNumber from the first extent
Use DeviceIoControl with code IOCTL_STORAGE_GET_DEVICE_NUMBER to get make it fill out a STORAGE_DEVICE_NUMBER struct as output
Use SetupDiGetClassDevs with arguments &GUID_DEVCLASS_DISKDRIVE and DICGF_PRESENT to get all disks on the system
In a loop, use SetupDiEnumDeviceInfo to get a SP_DEVINFO_DATA repeatedly (on the device list returned by step #8 above) and a call the function below to determine which one, if any, matches the STORAGE_DEVICE_NUMBER for the give path.
(This is edited to remove custom utility classes of mine right on the SO web page so I might have introduced errors/typos)
bool DoesDeviceInstanceEqualStorageDeviceNumber(
const std::string& devInstance,
STORAGE_DEVICE_NUMBER sdn)
{
// Open up this device instance, specifying that we want the *interfaces*.
// The interfaces are key key because examining them will let us get a
// string we can use the Win32 CreateFile function.
const auto hDevInfo = SetupDiGetClassDevsA(
nullptr,
devInstance.c_str(),
nullptr,
DIGCF_DEVICEINTERFACE | DIGCF_ALLCLASSES);
if (hDevInfo == INVALID_HANDLE_VALUE)
throws std::runtime_error("Unable to get disk devices");
DWORD dwSize = 0;
SP_DEVINFO_DATA did;
WCHAR buffer[4096];
did.cbSize = sizeof (did);
bool foundValidMatch = false;
int deviceNumber = 0;
// Iterate through all such devices, looking for one that has a storage device number that matches the given one.
while ( !foundValidMatch && SetupDiEnumDeviceInfo(hDevInfo, deviceNumber, &did))
{
deviceNumber++;
DEVPROPTYPE devPropType;
// We'll only bother comparing this one if it is fixed. Determine that.
const auto getPropResult = SetupDiGetDevicePropertyW (
hDevInfo,
&did,
&DEVPKEY_Device_RemovalPolicy, // Ask for the "removal policy"
&devPropType,
(BYTE*)buffer,
sizeof(buffer),
&dwSize,
0);
if (!getPropResult)
{
std::cerr << "Unable to to get removal policy for disk device: " << ::GetLastError() << std::endl;
continue;
}
/* This bit *would* skip removable disks, you wanted...
else if (buffer[0] != 1)
{
std::cerr << "Skipping removable disk device " << devInstance << std::endl;
continue;
}
*/
// OK this is a fixed disk so it might be the one we'll compare against
// 1. Get the very first disk interface from this particular disk device
// 2. Open a file on it
// 3. Query the resulting file for its device number.
// 4. Compare the device number to the one we determined above
// 5. If it matches ours, then we succeed. If not, continue
SP_DEVICE_INTERFACE_DATA devIntData;
devIntData.cbSize = sizeof(SP_DEVICE_INTERFACE_DATA);
// Get the disk interfaces
const auto result = SetupDiEnumDeviceInterfaces(
hDevInfo,
&did, //&did,
&GUID_DEVINTERFACE_DISK, // Get Disk Device Interface (from winioctl.h)
0, // We only need the very FIRST one. I think...
&devIntData);
if (!result)
continue;
DWORD dwRequiredSize = 0;
// Want to get the detail but don't yet know how much space we'll need
// Do a dummy call to find out
SetupDiGetDeviceInterfaceDetail(
hDevInfo,
&devIntData,
nullptr,
0,
&dwRequiredSize,
nullptr);
if (ERROR_INSUFFICIENT_BUFFER != ::GetLastError())
{
std::cerr << "Unable to get device interface Detail: " << ::GetLastError() << std::endl;;
}
else
{
// Get the detail data so we can get the device path and open a file.
std::vector<TCHAR> buf(dwRequiredSize);
auto pDidd = reinterpret_cast<PSP_DEVICE_INTERFACE_DETAIL_DATA>(buf.data());
// WARNING: HARD CODED HACK
// ------------------------
// https://stackoverflow.com/questions/10405193/vb-net-hid-setupdigetdeviceinterfacedetail-getlasterror-shows-1784-error-inv
//
// Don't ask. Just do what they tell you.
// -----------------------------------------------------------------
#ifdef BUILD_64
pDidd->cbSize = 8;
#else
pDidd->cbSize = 6;
#endif
// -----------------------------------------------------------------
if (!SetupDiGetDeviceInterfaceDetail(
hDevInfo,
&devIntData,
pDidd,
dwRequiredSize,
&dwRequiredSize,
nullptr))
{
std::cerr << "Cannot get interface detail: " << ::GetLastError());
}
else
{
// FINALLY: We now have a DevicePath that we can use to open up
// in a Win32 CreateFile() call. That will let us get the
// STORAGE_DEVICE_NUMBER and compare it to the one we were given.
const auto hFile = ::CreateFileW(pDidd->DevicePath, 0, FILE_SHARE_READ, nullptr, OPEN_EXISTING, 0, NULL);
if (INVALID_HANDLE_VALUE != hFile)
{
std::cerr << "Unable to open logical volume: " + devicePath << std::endl;
continue;
}
STORAGE_DEVICE_NUMBER sdnTest;
ZeroMemory(&sdnTest, sizeof(STORAGE_DEVICE_NUMBER));
if (0 == DeviceIoControl(
hDevInfo
IOCTL_STORAGE_GET_DEVICE_NUMBER,
nullptr, // output only so not needed
0, // output only so not needed
&sdnTest,
sizeof(STORAGE_DEVICE_NUMBER),
nullptr,
nullptr))
{
std::cerr << "Unable to determine storage device number: " << ::GetLastError() << std::endl;);
}
else
{
// All this for a one-line test...
foundValidMatch = sdnTest.DeviceNumber == sdn.DeviceNumber;
}
}
}
}
SetupDiDestroyDeviceInfoList(hDevInfo);
return foundValidMatch;
}
I hope this saves someone a headache

Resources