Linux provides many functions to make a file descriptor close-on-exec upon creation.
int efd = eventfd(0, O_CLOEXEC);
int sfd = socket(AF_UNIX, SOCK_STREAM | SOCK_CLOEXEC, 0);
...
My question is that: is this mechanism thread-safe? What if one thread forks at the same time another thread calls these functions to create fds? Will I run into file-descriptor leak problem?
This is the whole point of the CLOEXEC flag: make it impossible to get this kind of race. The flag gets passed all the way down to the kernel, so when the fd is created, it already has the CLOEXEC flag set on it. Here is an example. Let's say we have two threads. Thread 1 opens an fd and then sets the CLOEXEC flag on it with a separate fcntl system call. Thread 2 forks between the calls to open and fcntl. We have an fd leak.
If Thread 1 instead passed CLOEXEC into the open (or socket) call, race is resolved. If Thread 2 forks before the open then there is no fd, and so nothing to leak. If it forks after, the fd will get closed because it is already marked as CLOEXEC.
Related
My situation is this: In MacOS/X, I've called AuthorizationExecuteWithPrivileges to spawn a privileged child process, and the only way I have to communicate with the child process is by calling fread() and/or fwrite() on the FILE * file-handle returned to me by the final argument to that call.
What I want to do is indicate to the child process that it should go away, which I can do by calling fclose() on the file-handle -- the child process sees that its STDIN_FILENO has closed and responds by exiting.
However, I also want to be able to read any text that the child process printed to its stdout stream before exiting, but calling fclose() on the file-handle precludes doing that.
So my question is, is there any way to "half-close" a FILE *, such that is becomes closed-for-writing but still-open-for-reading? I'm imagining something analogous to the shutdown(SHUT_WR) that can be used on a socket-descriptor.
How to execute xterm from XWindow program, insert it into my window, but continue execution both while xterm is active and after it was closed?
In my XWindows (XLib over XCB) application I want to execute xterm -Into <handle>. So that my window contains xterm window in it. Unfortunately something wrong is happening.
pseudo code:
if (fork() == 0) {
pipe = popen('xterm -Into ' + handle);
while (feof(pipe)) gets(pipe);
exit(0);
}
I tired system() and execvp() as well. Every thing is fine until I exit from bash that runs in xterm, then my program exits. I guess that connection to X server is lost because it is shared between parent and child.
UPDATE: here is what is shown on terminal after program exits (or rather crashes).
XIO: fatal IO error 11 (Resource temporarily unavailable) on X server ":0.0"
after 59 requests (59 known processed) with 1 events remaining.
[xcb] Unknown sequence number while processing queue
[xcb] Most likely this is a multi-threaded client and XInitThreads has not been called
[xcb] Aborting, sorry about that.
y: ../../src/xcb_io.c:274: poll_for_event: Assertion `!xcb_xlib_threads_sequence_lost' failed.
Aborted
One possibility is that you are terminating due to the SIGCHLD signal not
being ignored and causing your program to abort.
signal(SIGCHLD, SIG_IGN);
Another is, as you suspect something actively closing the X session. Just
closing the socket itself should not matter but are you using a library that
registers an atexit call it could cause an issue.
Since from your snippet,
it looks like you don't actually care about the stdout of the xterm, a
better way to do it would be to actuall close fd's 0,1,2. Also since it looks
like you don't need to do anything in the child process after xterm
terminates you can use 'exec' rather than 'popen' to fully replace the
child process with that of the xterm including any cleanup handlers that
were left around. Though, I am not sure how pruned your snippet is from what you want to do as obviously the call to 'gets' is not what you want.
to make sure the X connection is closed, you can set its close on exec flag
with the following. (this will work on POSIX systems where the x connection
number is the fd of the server socket)
fcntl(XConnectionNumber(display), F_SETFD, fcntl(XConnectionNumber(display), F_GETFD) | FD_CLOEXEC);
Also note that 'popen' itself forks in the background in addition to your fork, I think you probably want to do an execvp there then use waitpid(... , WNOHANG) to check for the childs termination in your main X11 loop if you care to know when it exited.
I am running into an issue spawning a large number of processes (200) under MacOS X Mountain Lion (though I'm sure this issue is not version specific). What I am doing is launching processes (in my test it is /bin/cat) which have their STDIN, STDOUT, and STDERR connected to pipes -- the other end of which is the spawning (parent) process.
The parent process writes data into the STDIN of the processes, which is piped to the [/bin/cat] child processes, which in turn spit the data back out of STDOUT and is read by the parent process. /bin/cat is just used for testing.
I am actually using kqueue to be notified when there is space available in the STDIN pipe. When kqueue notifies you with a EVFILT_WRITE event that space is available, the event also tells you exactly how many bytes can be written without blocking.
This all works well, and I can easily spawn 100 child (/bin/cat) processes, and everything flows through the pipes without blocking (all day long). However, when I crank up the number of processes to 200 everything grinds to a halt when the single kqueue service thread blocks on a write() call to one of the STDIN pipes. The event says that there is 16384 bytes available (basically an empty pipe) but when the program tries to write exactly 16384 bytes into the pipe, the write() blocks anyway.
Initially I thought I was running into a max. open files issue, but I've jacked up the ulimit for open files to 8192, so that is not the issue. What I have discovered from some googling is that on OS X, STDIN/STDOUT/STDERR are not in fact "files" (or "pipes") but are actually devices. When the process is hung, running lsof on the command-line also hangs with a warning about not being able to stat() the file system:
lsof: WARNING: can't stat() hfs file system /
Output information may be incomplete.
assuming "dev=1000008" from mount table
As soon as I kill the process, lsof completes. This reinforces the notion that STDIN/OUT/ERR are in fact devices and I'm running into some kind of limit.
Does anyone have an idea of what limit I am running into, for example is there a limit on the number of "device" that can be created? Can this be increased somehow?
Just to answer my own question here. This appears to be related to how MacOS X will dynamically expand a pipe from 16K to 32K to 64K (and so on). My basic workaround was to prevent the pipe from expanding. It appears that whenever you fill the pipe completely the OS will expand it. So, when the kqueue triggers that I can write into the pipe, and indicates that I have 16384 bytes available to write, I simply write 16384 - 1 bytes. Basically, whatever it tells me I have available, I write at most (available - 1) bytes. This prevents the pipe from expanding, and is preventing my code from encountering the condition where a write() to the pipe would block (even though the pipe is non-blocking).
In my main process, i create a ffmpeg child process using CreateProcess(...).
I need to track the status of converting progress to update a progress bar. To do it, I read text from ffmpeg output and extract progress status from it.
I make a sample programm like this:
HANDLE rPipe, wPipe;
CreatePipe(&rPipe,&wPipe,&secattr,0);
STARTUPINFO sInfo;
ZeroMemory(&sInfo,sizeof(sInfo));
PROCESS_INFORMATION pInfo;
ZeroMemory(&pInfo,sizeof(pInfo));
sInfo.cb=sizeof(sInfo);
sInfo.dwFlags=STARTF_USESTDHANDLES;
sInfo.hStdInput=NULL;
sInfo.hStdOutput=wPipe;
sInfo.hStdError=wPipe;
// pStr contain ffmpeg command
CreateProcess(0,(LPTSTR)pStr,0,0,TRUE,NORMAL_PRIORITY_CLASS|CREATE_NO_WINDOW,0,0,&sInfo,&pInfo);
CloseHandle(wPipe);
BOOL ok;
do
{
memset(buf,0,bufsize);
ok=::ReadFile(rPipe,buf,100,&reDword,0);
result += buf;
}while(ok);
But I couldnt get "result" interactively updated. My app is held during conversion, and "result" string update only after ffmpeg's process finish.
How can I have my main process and ffmpeg's run simultaneously, and interactively read from/write to ffmpeg process's output/input?
Thanks for your time!
LRs
If the ffmpeg just uses stdout without explicitly flushing the output then it may not get sent to the calling process until it ends
Child processes that use such C run-time functions as printf() and
fprintf() can behave poorly when redirected. The C run-time functions
maintain separate IO buffers. When redirected, these buffers might not
be flushed immediately after each IO call. As a result, the output to
the redirection pipe of a printf() call or the input from a getch()
call is not flushed immediately and delays, sometimes-infinite delays
occur. This problem is avoided if the child process flushes the IO
buffers after each call to a C run-time IO function. Only the child
process can flush its C run-time IO buffers. A process can flush its C
run-time IO buffers by calling the fflush() function.
http://support.microsoft.com/kb/190351
In order of tracking the progress of your child process while it is running (and after its completion), you need to check the status of this child process.
After the process was launched, check the status periodically using the following code.
pi is the PROCESS_INFORMATION:
PROCESS_INFORMATION pi;
and the code:
DWORD exitCode = 0;
success = [GetExitCodeProcess][2](pi.hProcess, &exitCode);
exitCode will hold the value STILL_ACTIVE if the process is still running.
If the function succeeds, the return value of success is nonzero.
I'm currently writing a simple "multicaster" module.
Only one process can open a proc filesystem file for writing, and the rest can open it for reading.
To do so i use the inode_operation .permission callback, I check the operation and when i detect someone open a file for writing I set a flag ON.
i need a way to detect if a process that opened a file for writing has decided to close the file so i can set the flag OFF, so someone else can open for writing.
Currently in case someone is open for writing i save the current->pid of that process and when the .close callback is called I check if that process is the one I saved earlier.
Is there a better way to do that? Without saving the pid, perhaps checking the files that the current process has opened and it's permission...
Thanks!
No, it's not safe. Consider a few scenarios:
Process A opens the file for writing, and then fork()s, creating process B. Now both A and B have the file open for writing. When Process A closes it, you set the flag to 0 but process B still has it open for writing.
Process A has multiple threads. Thread X opens the file for writing, but Thread Y closes it. Now the flag is stuck at 1. (Remember that ->pid in kernel space is actually the userspace thread ID).
Rather than doing things at the inode level, you should be doing things in the .open and .release methods of your file_operations struct.
Your inode's private data should contain a struct file *current_writer;, initialised to NULL. In the file_operations.open method, if it's being opened for write then check the current_writer; if it's NULL, set it to the struct file * being opened, otherwise fail the open with EPERM. In the file_operations.release method, check if the struct file * being released is equal to the inode's current_writer - if so, set current_writer back to NULL.
PS: Bandan is also correct that you need locking, but the using the inode's existing i_mutex should suffice to protect the current_writer.
I hope I understood your question correctly: When someone wants to write to your proc file, you set a variable called flag to 1 and also save the current->pid in a global variable. Then, when any close() entry point is called, you check current->pid of the close() instance and compare that with your saved value. If that matches, you turn flag to off. Right ?
Consider this situation : Process A wants to write to your proc resource, and so you check the permission callback. You see that flag is 0, so you can set it to 1 for process A. But at that moment, the scheduler finds out process A has used up its time share and chooses a different process to run(flag is still o!). After sometime, process B comes up wanting to write to your proc resource also, checks that the flag is 0, sets it to 1, and then goes about writing to the file. Unfortunately at this moment, process A gets scheduled to run again and since, it thinks that flag is 0 (remember, before the scheduler pre-empted it, flag was 0) and so sets it to 1 and goes about writing to the file. End result : data in your proc resource goes corrupt.
You should use a good locking mechanism provided by the kernel for this type of operation and based on your requirement, I think RCU is the best : Have a look at RCU locking mechanism