I'm curious to know how the per-process file/socket descriptor table is implemented in Linux. Specifically, which data structure(s) and algorithms are used to implement it and keep it efficient.
Thanks in advance!
Opened files by a process are managed by struct files_struct, which is in the process's struct task_struct
struct task_struct {
...
/* open file information */
struct files_struct *files;
The per-process file descriptor table (fdt) is in struct files_struct
struct files_struct {
...
struct fdtable __rcu *fdt;
When a process is trying to open a file, it issues a open syscall. Which will call sys_open. This is basically the code flow:
sys_open(filename, …)
// 1) copy filename from user space
getname(filename)
strncpy_from_user()
// 2) get first unused file descriptor (will be returned to process)
int fd = get_unused_fd()
struct files_struct *files = current->files
// 3) get file from filesystem
struct file *f = file_open(filename)
open_namei
// lookup operation for filesystem
dentry = cached_lookup or real_lookup
// initializes file struct
dentry_open
// 4) install file returned by filesystem into file descriptor table for the process
fd_install
current->files->fd[fd] = file
The process gets in return the index to the file descriptor table for the file opened.
Related
Today, I decided to practice by creating a simple program that creates a new file. But I got a problem that the file is not being created. How to solve this?
Firstly, I'm importing the required crate, then creating the err() function for printing errors after creating the file, if they will.
Secondly, I'm creating a function to_u16(), because LPCWSTR type needs *const u16.
Thirdly, I'm calling the CreateFileW() function for creating file.rs, and passing all necessary arguments (according to this and this).
createfile.rs
use winapi::shared::minwindef::DWORD;
use winapi::um::winnt;
use std::ffi::CString;
use winapi::um::errhandlingapi;
use winapi::ctypes::*;
use winapi::um::minwinbase::*;
use winapi::um::fileapi;
use std::io;
fn err(){//print the last error
unsafe{
let xui =errhandlingapi::GetLastError();
print!("{:?}",xui) ;// 2
}
}
fn to_u16(s: &str) ->*const u16 {
let c_str = CString::new(s).unwrap();
c_str.as_ptr() as *const u16
}
fn main() {
unsafe{
let name:winnt::LPCWSTR =to_u16("file.rs") ; //lpFileName
let acces:DWORD = winnt::GENERIC_WRITE; //dwDesiredAccess
let share = winnt::FILE_SHARE_DELETE|winnt::FILE_SHARE_READ|winnt::FILE_SHARE_WRITE;//dwShareMode
let security = 0 as *mut SECURITY_ATTRIBUTES ;//lpSecurityAttributes
let disposition = fileapi::OPEN_EXISTING; //dwCreationDisposition
let atr = winnt::FILE_ATTRIBUTE_NORMAL;//dwFlagsAndAttributes
let handle = 0 as *mut c_void;//hTemplateFile
fileapi::CreateFileW(name,acces,share,security,disposition,atr,handle);
err();
}
}
cargo.toml
[package]
name = "createfile"
version = "0.1.0"
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
kernel32-sys = "0.2.2"
winapi = "0.3.9"
[target.'cfg(windows)'.dependencies]
winapi = { version = "0.3.9", features = ["winuser", "fileapi", "errhandlingapi"] }
Error 2 is ERROR_FILE_NOT_FOUND.
You say you want to create a new file, but you are setting the disposition argument to OPEN_EXISTING, so if file.rs does not already exist then CreateFileW() will fail with this error.
Try using CREATE_ALWAYS, CREATE_NEW, or OPEN_ALWAYS, per the documentation:
[in] dwCreationDisposition
An action to take on a file or device that exists or does not exist.
For devices other than files, this parameter is usually set to OPEN_EXISTING.
For more information, see the Remarks section.
This parameter must be one of the following values, which cannot be combined:
Value
Meaning
CREATE_ALWAYS2
Creates a new file, always.If the specified file exists and is writable, the function overwrites the file, the function succeeds, and last-error code is set to ERROR_ALREADY_EXISTS (183).If the specified file does not exist and is a valid path, a new file is created, the function succeeds, and the last-error code is set to zero.For more information, see the Remarks section of this topic.
CREATE_NEW1
Creates a new file, only if it does not already exist.If the specified file exists, the function fails and the last-error code is set to ERROR_FILE_EXISTS (80).If the specified file does not exist and is a valid path to a writable location, a new file is created.
OPEN_ALWAYS4
Opens a file, always.If the specified file exists, the function succeeds and the last-error code is set to ERROR_ALREADY_EXISTS (183).If the specified file does not exist and is a valid path to a writable location, the function creates a file and the last-error code is set to zero.
OPEN_EXISTING3
Opens a file or device, only if it exists.If the specified file or device does not exist, the function fails and the last-error code is set to ERROR_FILE_NOT_FOUND (2).For more information about devices, see the Remarks section.
TRUNCATE_EXISTING5
Opens a file and truncates it so that its size is zero bytes, only if it exists.If the specified file does not exist, the function fails and the last-error code is set to ERROR_FILE_NOT_FOUND (2).The calling process must open the file with the GENERIC_WRITE bit set as part of the dwDesiredAccess parameter.
Under Linux, my program would do something like this:
Process 1 opens a file (e.g. by mapping it into memory). Let's call this file#1
Process 2 unlinks the file, and creates a new file with the same name. Let's call this file#2.
Process 1 continues to work with file#1. When it is closed, it is deleted (since it has no links). Process 1 continues to work with the content in file#1, and does not see content from file#2.
When both processes have exited, file#2 remains on disk.
I want to achieve the same semantics in Windows. After reading this question, I think FILE_SHARE_DELETE does basically this. Is opening the file with FILE_SHARE_DELETE enough, or do I need to consider something more?
The above execution flow is just an example, I know there are other ways of solving that exact problem in Windows, but I want to understand how to make such code portable between Windows and Linux.
Edit: to clarify: The use cases would be to reuse a filename for different unrelated files, but let existing processes keep their data (think transactional update of a config file for example), and to make a file anonymous (unnamed), but continue to use it like an anonymous memory map. Again I know both are possible on Windows through other means, but I am trying to find a way that is portable across platforms.
You can achieve this by using a combination of CreateFile, CreateFileMapping and MapViewOfFile calls. MapViewOfFile will give you a memory-mapped buffer of the file backed by the file on disk.
Following code when executed from different processes, will write the process id of last closing process in the file at c:\temp\temp.txt
int main()
{
TCHAR szMsg[256];
HANDLE hMapFile;
LPCTSTR pBuf;
HANDLE hFile = CreateFileW(
L"C:\\Temp\\temp.txt",
GENERIC_WRITE|GENERIC_READ,
FILE_SHARE_DELETE|FILE_SHARE_READ|FILE_SHARE_WRITE,
NULL,
CREATE_ALWAYS,
FILE_ATTRIBUTE_NORMAL,NULL);
hMapFile = CreateFileMapping(
hFile, // Handle of file opened with rw access
NULL, // default security
PAGE_READWRITE, // read/write access
0, // maximum object size (high-order DWORD)
BUF_SIZE, // maximum object size (low-order DWORD)
szName); // name of mapping object
if (hMapFile == NULL)
{
printf( "Could not create file mapping object (%d).\n", GetLastError());
return 1;
}
pBuf = (LPTSTR) MapViewOfFile(hMapFile, // handle to map object
FILE_MAP_ALL_ACCESS, // read/write permission
0,
0,
BUF_SIZE);
if (pBuf == NULL)
{
printf("Could not map view of file (%d).\n", GetLastError());
CloseHandle(hMapFile);
return 1;
}
wsprintfW(szMsg, L"This is process id %d", GetCurrentProcessId());
CopyMemory((PVOID)pBuf, szMsg, (wcslen(szMsg) * sizeof(TCHAR)));
MessageBoxW(NULL, szMsg, L"Check", MB_OK);
UnmapViewOfFile(pBuf);
CloseHandle(hMapFile);
CloseHandle(hFile);
return 0;
}
Make sure you open the file with GENERIC_READ|GENERIC_WRITE access and allow FILE_SHARE_READ|FILE_SHARE_WRITE|FILE_SHARE_DELETE access to subsequent opens.
Also note the use of CREATE_ALWAYS in CreateFile which will delete the old file and open a new one every-time CreateFile is called. This is the 'unlink' effect you talk about.
Code inspired from Creating Named Shared Memory at msdn.
I have created block device in kernel module. When some I/O happens I read/write all data from/to another existing device (let's say /dev/sdb).
It opens OK, but read/write operations return 14 error(EFAULT,Bad Address). After some research I found that I need map address to user space(probably buffer or filp variables), but copy_to_user function does not help. Also I looked to mmap() and remap_pfn_range() functions, but I can not get how to use them in my code, especially where to get correct vm_area_struct structure. All examples that I found, used char devices and file_operations structure, not block device.
Any hints? Thanks for help.
Here is my code for reading:
mm_segment_t old_fs;
old_fs = get_fs();
set_fs(KERNEL_DS);
filp = filp_open("/dev/sdb", O_RDONLY | O_DIRECT | O_SYNC, 00644);
if(IS_ERR(filp))
{
set_fs(old_fs);
int err = PTR_ERR(filp);
printk(KERN_ALERT"Can not open file - %d", err);
return;
}
else
{
bytesRead = vfs_read(filp, buffer, nbytes, &offset); //It gives 14 error
filp_close(filp, NULL);
}
set_fs(old_fs);
I found a better way for I/O to block device from kernel module. I have used bio structure for that. Hope this information save somebody from headache.
1) So, if you want to redirect I/O from your block device to existing block device, you have to use own make_request function. For that you should use blk_alloc_queue function to create queue for your block device like this:
device->queue = blk_alloc_queue(GFP_KERNEL);
blk_queue_make_request(device->queue, own_make_request);
Than into own_make_request function change bi_bdev member into bio structure to device in which you redirecting I/O and call generic_make_request function:
bio->bi_bdev = device_in_which_redirect;
generic_make_request(bio);
More information here at 16 chapter. If link is broken by some cause, here is name of the book - "Linux Device Drivers, Third Edition"
2) If you want read or write your own data to existing block device from kernel module you should use submit_bio function.
Code for writing into specific sector(you need to implement writeComplete function also):
void writePage(struct block_device *device,
sector_t sector, int size, struct page *page)
{
struct bio *bio = bio_alloc(GFP_NOIO, 1);
bio->bi_bdev = vnode->blkDevice;
bio->bi_sector = sector;
bio_add_page(bio, page, size, 0);
bio->bi_end_io = writeComplete;
submit_bio(WRITE_FLUSH_FUA, bio);
}
Code for reading from specific sector(you need to implement readComplete function also):
int readPage(struct block_device *device, sector_t sector, int size,
struct page *page)
{
int ret;
struct completion event;
struct bio *bio = bio_alloc(GFP_NOIO, 1);
bio->bi_bdev = device;
bio->bi_sector = sector;
bio_add_page(bio, page, size, 0);
init_completion(&event);
bio->bi_private = &event;
bio->bi_end_io = readComplete;
submit_bio(READ | REQ_SYNC, bio);
wait_for_completion(&event);
ret = test_bit(BIO_UPTODATE, &bio->bi_flags);
bio_put(bio);
return ret;
}
page can be allocated with alloc_page(GFP_KERNEL). Also for changing data in page use page_address(page). It returns void* so you can interpret that pointer as whatever you want.
I am trying to delete all the temporary files created by my application during uninstall. I use the following code:
bool DeleteFileNow( QString filenameStr )
{
wchar_t* filename;
filenameStr.toWCharArray(filename);
QFileInfo info(filenameStr);
// don't do anything if the file doesn't exist!
if (!info.exists())
return false;
// determine the path in which to store the temp filename
wchar_t* path;
info.absolutePath().toWCharArray(path);
TRACE( "Generating temporary name" );
// generate a guaranteed to be unique temporary filename to house the pending delete
wchar_t tempname[MAX_PATH];
if (!GetTempFileNameW(path, L".xX", 0, tempname))
return false;
TRACE( "Moving real file name to dummy" );
// move the real file to the dummy filename
if (!MoveFileExW(filename, tempname, MOVEFILE_REPLACE_EXISTING))
{
// clean up the temp file
DeleteFileW(tempname);
return false;
}
TRACE( "Queueing the OS" );
// queue the deletion (the OS will delete it when all handles (ours or other processes) close)
return DeleteFileW(tempname) != FALSE;
}
My application is crashing. I think its due to some missing windows dll for the operations performed. Is there any other way to perform the same operation using Qt alone?
Roku have already told your problem in manipulating with QString and wchar_t*.
See the documentation: QString Class Reference, method toWCharArray:
int QString::toWCharArray ( wchar_t * array ) const
Fills the array with the data contained in this QString object. The array is encoded in utf16 on platforms where wchar_t is 2 bytes wide (e.g. windows) and in ucs4 on platforms where wchar_t is 4 bytes wide (most Unix systems).
array has to be allocated by the caller and contain enough space to hold the complete string (allocating the array with the same length as the string is always sufficient).
returns the actual length of the string in array.
If you are simply looking for a way to remove a file using Qt, use QFile::remove:
QFile file(fileNameStr);
file.remove(); // Returns a bool; true if successful
If you want Qt to manage the entire life cycle of a temporary file for you, take a look at QTemporaryFile:
QTemporaryFile tempFile(fileName);
if (tempFile.open())
{
// Do stuff with file here
}
// When tempFile falls out of scope, it is automatically deleted.
I'm porting old linux kernel code for newer version 2.6.32.
There is a part that copies a file descriptor. The idea was to allocate a new file descriptor and a new struct file and use them with another f_op and , leaving all other fields of struct file equivalent to original's.
How do I do this in a modern kernel?
I've written an approximate implementation but I don't know whether i should call file_get, path_get or do others use counter incrementation.
struct file * copy_file(const struct file * iOrig, int * oNewFd) {
if (!orig)
return 0;
*oNewFd = get_unused_fd();
if (*oNewFd < 0)
return 0;
struct file * rv = alloc_file(orig->f_path.mnt, orig->f_path.dentry, orig->f_path.mode, orig->f_op);
if (!rv)
goto free_fd;
fd_install(fd, rv);
return rv;
free_fd:
put_unused_fd(*oNewFd)
return 0;
}
P.S. In fact having all fileds of original file copied is not neccessary. I just need to allow a new set of file operations in user-space. So creating a new descriptor owned by current with a given f_op will do.
path_get sounds fine. Check out an example here http://tomoyo.sourceforge.jp/cgi-bin/lxr/source/fs/pipe.c#L1046 and you'll be able to find more refs there if you need them.