I need to send a string from kernel to a user space function without asking for it in particular from the user space, sort of triggering a function or application in the user space via some event in kernel.
So far I have tried an Ioctl that starts on an init in user space and then sleeps and kept reading about netlink but couldn't find a good working example for it.
Any suggestions or examples will be much obliged.
Here's how my process works, I would be interested in any suggestions for improvements as well:
Start the kernel module
Start user space application, which sends a custom command to the kernel module to register the user space PID for kernel module signals. In my case this was via a write to /dev/mymodule. The kernel module registers the PID:
...
printk("registering a new process id to receive signals: %d\n", current->pid);
signal_pid = current->pid;
...
The user space application also registers a handler with the kernel for certain types of signals.
void local_sig_handler(int signum) {
printf("received a signal from my module\n");
fflush(stdout); }
...
signal(SIGIO, local_sig_handler);
Kernel module generates a signal
...
struct siginfo info;
struct task_struct *t;
info.si_signo=SIGIO;
info.si_int=1;
info.si_code = SI_QUEUE;
printk("<1>IRQ received: %d\n", irq);
printk("<1>searching for task id: %d\n", signal_pid);
t= pid_task(find_vpid(signal_pid),PIDTYPE_PID);//user_pid has been fetched successfully
if(t == NULL){
printk("<1>no such pid, cannot send signal\n");
} else {
printk("<1>found the task, sending signal\n");
send_sig_info(SIGIO, &info, t);
}
Kernel relays the signal to the application's handler
You've got a few options:
Signals. User process defines a signal handler, and kernel signals the user process upon receipt of an event. This works well, but requires that the handling code run in an async signal handler (which makes it trickier to write correct code). The downside is that the amount of data you can transmit using a signal handler is somewhat limited. Make sure to use a signal that can be queued (e.g. a realtime signal) so you don't lose messages when the process is in the middle of handling a signal.
Blocking system call or file access. User process executes a system call (or reads/writes a file) which puts it to sleep. The kernel driver for the call maintains a queue of events, and dequeues events when they arrive and when a blocked waiter exists (this avoids losing events when the user process is unblocked).
Create a system call which configures a sigevent. On the kernel side, create a sigqueue to fire the relevant events.
An example I used in the past was to send signal to user space from hardware interrupt in kernel space.
KERNEL SPACE
You have to prepare siginfo and task_struct before sending a signal:
struct siginfo info;
struct task_struct *t;
info.si_signo = SIG_TEST;
info.si_code = SI_QUEUE;
info.si_int = 1234; // Any value you want to send
rcu_read_lock();
And also find the task with user space application PID. You have to send it from user-space to kernel-space through write or ioctl operations.
t = pid_task(find_pid_ns(pid, &init_pid_ns), PIDTYPE_PID);
Then you can send the signal.
rcu_read_unlock();
send_sig_info(SIG_TEST, &info, t);
I omitted here, but you must check the result of every operation.
The previous code prepare the signal structure and send it. Bear in mind that you need the application's PID. In my case the application from user space send its PID through ioctl driver procedure.
USER SPACE
You have to define and implement the callback function:
void signalFunction(int n, siginfo_t *info, void *unused) {
.....
.....
}
In main procedure:
struct sigaction sig;
sig.sa_sigaction = signalFunction; // Callback function
sig.sa_flags = SA_SIGINFO;
sigaction(SIG_TEST, &sig, NULL);
I hope it helps.
Related
AVFormatContext's interrupt_callback field is a
Custom interrupt callbacks for the I/O layer.
It's type is AVIOInterruptCB, and it explains in comment section:
Callback for checking whether to abort blocking functions.
AVERROR_EXIT is returned in this case by the interrupted function. During blocking operations, callback is called with opaque as parameter. If the callback returns 1, the blocking operation will be aborted.
No members can be added to this struct without a major bump, if new elements have been added after this struct in AVFormatContext or AVIOContext.
I have 2 questions:
what does the last section means? Especially "without a major bump"?
If I use this along with an RTSP source, when I close the input by avformat_close_input, the "TEARDOWN" message is being sent out, however it won't reach the RTSP server.
For 2: here is a quick pseudo-code for demo:
int pkts = 0;
bool early_exit = false;
int InterruptCallback(void* ctx) {
return early_exit ? 1 : 0;
}
void main() {
ctx = avformat_alloc_context
ctx->interrupt_callback.callback = InterruptCallback;
avformat_open_input
avformat_find_stream_info
pkts=0;
while(!early_exit) {
av_read_frame
if (pkts++ > 100) early_exit=true;
}
avformat_close_input
}
In case I don't use the interrupt callback at all, TEARDOWN is being sent out, and it also reaches the RTSP server so it can actually tear down the connection. Otherwise, it won't tear down it, and I have to wait until TCP socket times out.
What is the proper way of using this interrupt callback?
It means that they are not going to change anything for this structure (AVIOInterruptCB). However, if thats the case it would be in a major bump (major change from 4.4 eg to 5.0)
You need to pass a meaningful parameter to void* ctx. Anything that you like so you can check it within the static function. For example a bool that you will set as cancel so you will interrupt the av_read_frame (which will return an AVERROR_EXIT). Usually you pass a class of your decoder context or something similar which also holds all the info that you required to check whether to return 1 to interrupt or 0 to continue the requests properly. A real example would be that you open a wrong rtsp and then you want to open another one (the right one) so you need to cancel your previous requests.
I am trying to understand how wait_event is implemented in linux kernel. There is a code example in ldd3 where the internal implementation is explained using prepare_to_wait (http://www.makelinux.net/ldd3/chp-6-sect-2).
static int scull_getwritespace(struct scull_pipe *dev, struct file *filp)
{
while (spacefree(dev) == 0) {
DEFINE_WAIT(wait);
up(&dev->sem);
if (filp->f_flags & O_NONBLOCK)
return -EAGAIN;
PDEBUG("\"%s\" writing: going to sleep\n",current->comm);
prepare_to_wait(&dev->outq, &wait, TASK_INTERRUPTIBLE);
if (spacefree(dev) == 0) // Why is this check necessary ??
schedule( );
finish_wait(&dev->outq, &wait);
if (signal_pending(current))
return -ERESTARTSYS; /* signal: tell the fs layer to handle it */
if (down_interruptible(&dev->sem))
return -ERESTARTSYS;
}
return 0;
}
In the book, it is explained as below.
Then comes the obligatory check on the buffer; we must handle the case
in which space becomes available in the buffer after we have entered
the while loop (and dropped the semaphore) but before we put ourselves
onto the wait queue. Without that check, if the reader processes were
able to completely empty the buffer in that time, we could miss the
only wakeup we would ever get and sleep forever. Having satisfied
ourselves that we must sleep, we can call schedule.
I am not able to understand this piece of explanation. How we would go to a indefinite sleep if the if (spacefree(dev) == 0) is not done before calling schedule() ?
if this obligatory check is not present, wakeup() still resets the process state to TASK_RUNNING and schedule returns as explained in the next paragraph.
It is worth looking again at this case: what happens if the wakeup
happens between the test in the if statement and the call to schedule?
In that case, all is well. The wakeup resets the process state to
TASK_RUNNING and schedule returns—although not necessarily right away.
As long as the test happens after the process has put itself on the
wait queue and changed its state, things will work.
The important thing is that the (last) check is done after prepare_to_wait() was called.
prepare_to_wait() puts a pointer to the current process into the wait queue. If the wakeup happens before the prepare_to_wait() call, the wakeup would not be able to affect the current process.
I suppose I actually have two separate questions, but I think that they are related enough to include them both. The context is a Linux USB device driver (not userspace).
After transmitting a request URB, how do I receive the response once my complete callback is called?
How can I use interrupt URBs for single request/response pairs, and not as actual continuous interrupt polling (as they are intended)?
So for some background, I'm working on a driver for the Microchip MCP2210 a USB-to-SPI Protocol Converter with GPIO (USB 2.0, datasheet here). This device advertises as generic HID and exposes two interrupt endpoints (an in and an out) as well as it's control endpoint.
I am starting from a working, (but alpha-quality) demo driver written by somebody else and kindly shared with the community. However, this is a HID driver and the mechanism it uses to communicate with the device is very expensive! (sending a 64 byte message requires allocating a 6k HID report struct, and allocation is sometimes performed in the context of an interrupt, requiring GFP_ATOMIC!). We'll be accessing this from an embedded low-memory device.
I'm new to USB drivers and still pretty green with Linux device drivers in general. However, I'm trying to convert this to a plain-jane USB driver (not HID) so I can use the less expensive interrupt URBs for my communications. Here is my code for transmitting my request. For the sake of (attempted) brevity, I'm not including the definition of my structs, etc, but please let me know if you need more of my code. dev->cur_cmd is where I'm keeping the current command I'm processing.
/* use a local for brevity */
cmd = dev->cur_cmd;
if (cmd->state == MCP2210_CMD_STATE_NEW) {
usb_fill_int_urb(dev->int_out_urb,
dev->udev,
usb_sndintpipe(dev->udev, dev->int_out_ep->desc.bEndpointAddress),
&dev->out_buffer,
sizeof(dev->out_buffer), /* always 64 bytes */
cmd->type->complete,
cmd,
dev->int_out_ep->desc.bInterval);
ret = usb_submit_urb(dev->int_out_urb, GFP_KERNEL);
if (ret) {
/* snipped: handle error */
}
cmd->state = MCP2210_CMD_STATE_XMITED;
}
And here is my complete fn:
/* note that by "ctrl" I mean a control command, not the control endpoint */
static void ctrl_complete(struct urb *)
{
struct mcp2210_device *dev = urb->context;
struct mcp2210_command *cmd = dev->cur_cmd;
int ret;
if (unlikely(!cmd || !cmd->dev)) {
printk(KERN_ERR "mcp2210: ctrl_complete called w/o valid cmd "
"or dev\n");
return;
}
switch (cmd->state) {
/* Time to rx the response */
case MCP2210_CMD_STATE_XMITED:
/* FIXME: I think that I need to check the response URB's
* status to find out if it was even transmitted or not */
usb_fill_int_urb(dev->int_in_urb,
dev->udev,
usb_sndintpipe(dev->udev, dev->int_in_ep->desc
.bEndpointAddress),
&dev->in_buffer,
sizeof(dev->in_buffer),
cmd->type->complete,
dev,
dev->int_in_ep->desc.bInterval);
ret = usb_submit_urb(dev->int_in_urb, GFP_KERNEL);
if (ret) {
dev_err(&dev->udev->dev,
"while attempting to rx response, "
"usb_submit_urb returned %d\n", ret);
free_cur_cmd(dev);
return;
}
cmd->state = MCP2210_CMD_STATE_RXED;
return;
/* got response, now process it */
case MCP2210_CMD_STATE_RXED:
process_response(cmd);
default:
dev_err(&dev->udev->dev, "ctrl_complete called with unexpected state: %d", cmd->state);
free_cur_cmd(dev);
};
}
So am I at least close here? Secondly, both dev->int_out_ep->desc.bInterval and dev->int_in_ep->desc.bInterval are equal to 1, will this keep sending my request every 125 microseconds? And if so, how do I say "ok, ty, now stop this interrupt". The MCP2210 offers only one configuration, one interface and that has just the two interrupt endpoints. (I know everything has the control interface, not sure where that fits into the picture though.)
Rather than spam this question with the lsusb -v, I'm going to pastebin it.
Typically, request/response communication works as follows:
Submit the response URB;
submit the request URB;
in the request completion handler, if the request was not actually sent, cancel the response URB and abort;
in the response completion handler, handle the response data.
All that asynchronous completion handler stuff is a big hassle if you have a single URB that is completed almost immediately; therefore, there is the helper function usb_interrupt_msg() which works synchronously.
URBs to be used for polling must be resubmitted (typically from the completion handler).
If you do not resubmit the URB, no polling happens.
I would like to discuss potential kind of asynchronous (Overlapped) I/O implementations in Windows, because there are many ways to implement this.
Overlapped I/O in Windows provides the ability to process data asynchronously, ie the execution of the operations are nonblocking.
Edit: The purpose of this question is the discussion about improvement of my own implementation on the one hand, and the discussion of alternate implementation on the other hand. What asynchronous I/O implementation would make most sense on parallel heavy I/O, what make most sense in small mostly single threaded application.
I will cite MSDN:
When a function is executed synchronously, it does not return until the operation has been completed. This means that the execution of the calling thread can be blocked for an indefinite period while it waits for a time-consuming operation to finish. Functions called for overlapped operation can return immediately, even though the operation has not been completed. This enables a time-consuming I/O operation to be executed in the background while the calling thread is free to perform other tasks. For example, a single thread can perform simultaneous I/O operations on different handles, or even simultaneous read and write operations on the same handle.
I assume that the reader is familiar with the basic concept of overlapped I/O.
Another solution for asynchronous I/O are completions ports, but this shall not be the subject of this discussion. More information on other I/O concepts can be found on MSDN "About File Management > Input and Output (I/O) > I/O Concepts"
I would like to present my (C/C++) implementation here and share it for discussion.
This is my extended OVERLAPPED struct called IoOperation:
struct IoOperation : OVERLAPPED {
HANDLE Handle;
unsigned int Operation;
char* Buffer;
unsigned int BufferSize;
}
This struct is created each time an asynchronous operation like ReadFile or WriteFile is called. The Handle field shall be initialized with the corresponding device/file handle. Operation is a user defined field that tells what operation was called. The field Buffer is a pointer to a previously allocated chunk of memory with the given size BufferSize. Of course, this struct can be expanded at will. It could contain the operation result, acutaully transfered size etc.
The first thing we need is an (auto reset) event handle to be signaled each time an overlapped I/O is completed.
HANDLE hEvent = CreateEvent(0, FALSE, FALSE, 0);
First I decided to use only one event for all asynchronous operations. Then I decided to register this event with a thread pool thread with RegisterWaitForSingleObject.
HANDLE hWait = 0;
....
RegisterWaitForSingleObject(
&hWait,
hEvent,
WaitOrTimerCallback,
this,
INFINITE,
WT_EXECUTEINPERSISTENTTHREAD | WT_EXECUTELONGFUNCTION
);
So each time this event is signaled, my callback WaitOrTimerCallback is called.
An asynchronous operation is initialized like this:
IoOperation* Io = new IoOperation(hFile, hEvent, IoOperation::Write, Data, DataSize);
if (IoQueue->Enqueue(Io)) {
WriteFile(hFile, Io->Buffer, Io->BufferSize, 0, Io);
}
Each operation is queued and is removed after successful GetOverlappedResult call in my WaitOrTimerCallback callback. Instead calling new all the time here, we could use a memory pool to avoid memory fragmentation and to make allocation faster.
VOID CALLBACK WaitOrTimerCallback(PVOID Parameter, BOOLEAN TimerOrWaitFired) {
list<IoOperation*>::iterator it = IoQueue.begin();
while (it != IoQueue.end()) {
bool IsComplete = true;
DWORD Transfered = 0;
IoOperation* Io = *it;
if (GetOverlappedResult(Io->Handle, Io, &Transfered, FALSE)) {
if (Io->Operation == IoOperation::Read) {
// Handle Read, virtual OnRead(), SetEvent, etc.
} else if (Io->Operation == IoOperation::Write) {
// Handle Read, virtual OnWrite(), SetEvent, etc.
} else {
// ...
}
} else {
if (GetLastError() == ERROR_IO_INCOMPLETE) {
IsComplete = false;
} else {
// Handle Error
}
}
if (IsComplete) {
delete Io;
it = IoQueue.erase(it);
} else {
it++;
}
}
}
Of course, to be multi threading safe, we need a lock protection (critical section) when accessing the I/O queue for example.
There are advantages but also disadvantage of this kind of implementation.
Advantages:
Execution in persistent thread pool thread, no manual thread creation is required
Only one event is required
Each operation is queued in an I/O queue (CancelIoEx can be called later)
Disadvantages:
I/O queue requires extra memory/cpu time
GetOverlappedResult is called for all queued I/O's even incompleted ones
I know that I can use condition variable to synchronize work between the threads, but is there any class like this (condition variable) to synchronize work between the processes, thanks in advance
Use a pair of named Semaphore objects, one to signal and one as a lock. Named sync objects on Windows are automatically inter-process, which takes care of that part of the job for you.
A class like this would do the trick.
class InterprocessCondVar {
private:
HANDLE mSem; // Used to signal waiters
HANDLE mLock; // Semaphore used as inter-process lock
int mWaiters; // # current waiters
protected:
public:
InterprocessCondVar(std::string name)
: mWaiters(0), mLock(NULL), mSem(NULL)
{
// NOTE: You'll need a real "security attributes" pointer
// for child processes to see the semaphore!
// "CreateSemaphore" will do nothing but give you the handle if
// the semaphore already exists.
mSem = CreateSemaphore( NULL, 0, std::numeric_limits<LONG>::max(), name.c_str());
std::string lockName = name + "_Lock";
mLock = CreateSemaphore( NULL, 0, 1, lockName.c_str());
if(!mSem || !mLock) {
throw std::runtime_exception("Semaphore create failed");
}
}
virtual ~InterprocessCondVar() {
CloseHandle( mSem);
CloseHandle( mLock);
}
bool Signal();
bool Broadcast();
bool Wait(unsigned int waitTimeMs = INFINITE);
}
A genuine condition variable offers 3 calls:
1) "Signal()": Wake up ONE waiting thread
bool InterprocessCondVar::Signal() {
WaitForSingleObject( mLock, INFINITE); // Lock
mWaiters--; // Lower wait count
bool result = ReleaseSemaphore( mSem, 1, NULL); // Signal 1 waiter
ReleaseSemaphore( mLock, 1, NULL); // Unlock
return result;
}
2) "Broadcast()": Wake up ALL threads
bool InterprocessCondVar::Broadcast() {
WaitForSingleObject( mLock, INFINITE); // Lock
bool result = ReleaseSemaphore( mSem, nWaiters, NULL); // Signal all
mWaiters = 0; // All waiters clear;
ReleaseSemaphore( mLock, 1, NULL); // Unlock
return result;
}
3) "Wait()": Wait for the signal
bool InterprocessCondVar::Wait(unsigned int waitTimeMs) {
WaitForSingleObject( mLock, INFINITE); // Lock
mWaiters++; // Add to wait count
ReleaseSemaphore( mLock, 1, NULL); // Unlock
// This must be outside the lock
return (WaitForSingleObject( mSem, waitTimeMs) == WAIT_OBJECT_0);
}
This should ensure that Broadcast() ONLY wakes up threads & processes that are already waiting, not all future ones too. This is also a VERY heavyweight object. For CondVars that don't need to exist across processes I would create a different class w/ the same API, and use unnamed objects.
You could use named semaphore or named mutex. You could also share memory between processes by shared memory.
For a project I'm working on I needed a condition variable and mutex implementation which can handle dead processes and won't cause other processes to end up in a deadlock in such a case. I implemented the mutex with the native named mutexes provided by the WIN32 api because they can indicate whether a dead process owns the lock by returning WAIT_ABANDONED. The next issue was that I also needed a condition variable I could use across processes together with these mutexes. I started of with the suggestion from user3726672 but soon discovered that there are several issues in which the state of the counter variable and the state of the semaphore ends up being invalid.
After doing some research, I found a paper by Microsoft Research which explains exactly this scenario: Implementing Condition Variables with Semaphores . It uses a separate semaphore for every single thread to solve the mentioned issues.
My final implementation uses a portion of shared memory in which I store a ringbuffer of thread-ids (the id's of the waiting threads). The processes then create their own handle for every named semaphore/thread-id which they have not encountered yet and cache it. The signal/broadcast/wait functions are then quite straight forward and follow the idea of the proposed solution in the paper. Just remember to remove your thread-id from the ringbuffer if your wait operation fails or results in a timeout.
For the Win32 implementation I recommend reading the following documents:
Semaphore Objects and Using Mutex Objects as those describe the functions you'll need for the implementation.
Alternatives: boost::interprocess has some robust mutex emulation support but it is based on spin locks and caused a very high cpu load on our embedded system which was the final reason why we were looking into our own implementation.
#user3726672: Could you update your post to point to this post or to the referenced paper?
Best Regards,
Michael
Update:
I also had a look at an implementation for linux/posix. Turns out pthread already provides everything you'll need. Just put pthread_cond_t and pthread_mutex_t in some shared memory to share it with the other process and initialize both with PTHREAD_PROCESS_SHARED. Also set PTHREAD_MUTEX_ROBUST on the mutex.
Yes. You can use a (named) Mutex for that. Use CreateMutex to create one. You then wait for it (with functions like WaitForSingleObject), and release it when you're done with ReleaseMutex.
For reference, Boost.Interprocess (documentation for version 1.59) has condition variables and much more. Please note, however, that as of this writing, that "Win32 synchronization is too basic".