I am building an application that uses either replxx or libedit to read a line from a terminal, along with tab completion. In order for replxx and libedit to work, each library puts stdio into raw unbuffered mode so that key bindings work.
During tab completion, a tool is executed in a fashion similar to bash completion. The parent application communicates with the tool over a pipe, after a fork and an execve, and this all works fine.
There is a problem once the tool has finished execution - in the parent application, stdin has lost its raw unbuffered setting and has reverted back to line buffering. This breaks replxx and libedit, which remain frozen until enter is pressed, and then catch up with all the line buffered keypresses.
What does code need to do (or avoid doing) to fork, then execve, in such a way that stdin in the parent remains untouched?
Most specifically, does closing stdin in the child process have any kind of effect on the stdin in the parent process with respect to buffering?
The code uses Apache Portable Runtime on Linux/MacOS, and looks like this:
apr_procattr_create(&procattr, first->pool);
apr_file_pipe_create_ex(&ioread, &iowrite, APR_FULL_BLOCK,
first->pool);
apr_procattr_child_in_set(procattr, NULL, NULL);
apr_procattr_child_out_set(procattr, NULL, NULL);
apr_procattr_child_err_set(procattr, NULL, NULL);
apr_procattr_dir_set(procattr, command->r.sysconf);
apr_procattr_cmdtype_set(procattr, APR_PROGRAM);
proc = apr_pcalloc(first->pool, sizeof(apr_proc_t));
apr_proc_create(proc, command->r.libexec, (const char* const*) argv->elts,
device_environment_make(d), procattr, first->pool);
apr_file_close(proc->in);
apr_file_close(proc->err);
[do stuff that reads proc->out]
apr_file_close(proc->out);
apr_proc_wait(proc, NULL, NULL, APR_WAIT);
Current replxx resets raw mode terminal settings after each callback returns.
Related
How to execute xterm from XWindow program, insert it into my window, but continue execution both while xterm is active and after it was closed?
In my XWindows (XLib over XCB) application I want to execute xterm -Into <handle>. So that my window contains xterm window in it. Unfortunately something wrong is happening.
pseudo code:
if (fork() == 0) {
pipe = popen('xterm -Into ' + handle);
while (feof(pipe)) gets(pipe);
exit(0);
}
I tired system() and execvp() as well. Every thing is fine until I exit from bash that runs in xterm, then my program exits. I guess that connection to X server is lost because it is shared between parent and child.
UPDATE: here is what is shown on terminal after program exits (or rather crashes).
XIO: fatal IO error 11 (Resource temporarily unavailable) on X server ":0.0"
after 59 requests (59 known processed) with 1 events remaining.
[xcb] Unknown sequence number while processing queue
[xcb] Most likely this is a multi-threaded client and XInitThreads has not been called
[xcb] Aborting, sorry about that.
y: ../../src/xcb_io.c:274: poll_for_event: Assertion `!xcb_xlib_threads_sequence_lost' failed.
Aborted
One possibility is that you are terminating due to the SIGCHLD signal not
being ignored and causing your program to abort.
signal(SIGCHLD, SIG_IGN);
Another is, as you suspect something actively closing the X session. Just
closing the socket itself should not matter but are you using a library that
registers an atexit call it could cause an issue.
Since from your snippet,
it looks like you don't actually care about the stdout of the xterm, a
better way to do it would be to actuall close fd's 0,1,2. Also since it looks
like you don't need to do anything in the child process after xterm
terminates you can use 'exec' rather than 'popen' to fully replace the
child process with that of the xterm including any cleanup handlers that
were left around. Though, I am not sure how pruned your snippet is from what you want to do as obviously the call to 'gets' is not what you want.
to make sure the X connection is closed, you can set its close on exec flag
with the following. (this will work on POSIX systems where the x connection
number is the fd of the server socket)
fcntl(XConnectionNumber(display), F_SETFD, fcntl(XConnectionNumber(display), F_GETFD) | FD_CLOEXEC);
Also note that 'popen' itself forks in the background in addition to your fork, I think you probably want to do an execvp there then use waitpid(... , WNOHANG) to check for the childs termination in your main X11 loop if you care to know when it exited.
I'm building a Win32 GUI app. Inside that app, I'm using a DLL that was intended to be used in a command line app.
Suppose Foo.exe is my GUI app, and bar() is a function in the DLL that prints "hello" to stdout. Foo.exe calls bar().
If I run Foo.exe from the command line with a redirect (>) (i.e. Foo.exe > out.txt) it writes "hello" to out.txt and exits normally (as expected).
However, if I run Foo.exe without a redirect (either from cmd.exe or by double-clicking in Windows Explorer), it crashes when bar() is called.
If I run Foo.exe inside the debugger with the redirect in the command line (set through VS's properties for the project) and call "GetStdHandle(STD_OUTPUT_HANDLE)", I get a reasonable address for a handle. If I call it without the redirect in the command line, I get 0.
Do I need something to "initialize" standard out? Is there a way that I can set up this redirect in the application startup? (Redirecting to a file would be ideal. But just throwing out the data printed by the DLL would be okay, too.)
Finally, I suspect that the DLL is writing to stdout through the CRT POSIX-like API, because it is a cross-platform DLL. I don't know if this matters.
I've tried creating a file with CreateFile and calling SetStdHandle, but that doesn't seem to work. I may be creating the file incorrectly, however. See code below.
HANDLE hStdOut = GetStdHandle(STD_OUTPUT_HANDLE);
// hStdOut is zero
HANDLE hFile;
hFile = CreateFile(TEXT("something.txt"), // name of the write
GENERIC_WRITE, // open for writing
0, // do not share
NULL, // default security
CREATE_NEW, // create new file only
FILE_ATTRIBUTE_NORMAL, // normal file
NULL); // no attr. template
BOOL r = SetStdHandle(STD_OUTPUT_HANDLE, hFile) ;
hStdOut = GetStdHandle(STD_OUTPUT_HANDLE);
// hStdOut is now equal to hFile, and r is 1
bar();
// crashes if there isn't a redirect in the program arguments
UPDATE: I just found this article: http://support.microsoft.com/kb/105305. It states "Note that this code does not correct problems with handles 0, 1, and 2. In fact, due to other complications, it is not possible to correct this, and therefore it is necessary to use stream I/O instead of low-level I/O."
My DLL definitely uses file handles 0,1 and 2. So, there may be no good solution to this problem.
I'm working on a solution that checks for this case, and re-launches the exe appropriately using CreateProcess. I'll post here when I'm done.
The solution that I've found is the following:
obtain a valid File HANDLE in some way to direct the standard output.
Lets call the file handle "fh".
(please note that on Windows a File HANDLE is not the same thing of a file descriptor)
associate a file descriptor to the file handle with _open_osfhandle
(see http://msdn.microsoft.com/en-us/library/kdfaxaay.aspx for details)
Lets call "fd" the new file descriptor, an int value.
call dup2 to associate the STDOUT_FILENO to the given file descriptor:
dup2(fd, STDOUT_FILENO)
create a file strem associated to the stdout file descriptor
FILE* f = _fdopen(STDOUT_FILENO, "w");
memset stdout to the content of f:
*stdout = *f
call SetStdHandle on the given file handle:
SetStdHandle(STD_OUTPUT_HANDLE, ofh);
Please note that I've not tested exactly this sequence but something slightly different.
I don't know if some steps are redundant.
In any case the following article explain very well the concept of file handle, descriptor et fiel stream:
http://dslweb.nwnexus.com/~ast/dload/guicon.htm
You must build foo.exe as a console application with the /SUBSYSTEM switch. Windows will allocate a console (stdout) for your application automatically, which can be :
The current console
A redirection to a file
A pipe to another program's STDIN
If you build foo.exe as a GUI application, the console is not allocated by default, wich explains the crash
If you must use the GUI subsystem, it can still be done with AllocConsole. This old WDJ article has sample code to help you.
Can you tell me which library do you use? This problem have good solution. Write small stub launcher EXE (in GUI mode but with NO windows!) that have your icon and that all shortcuts launch. Make this stub EXE "CreateProcess" the real EXE with redirected output to "NUL" or "CON", or, CreateProcess() it suspended, take its STDOUT, doing nothing with it. This way, your original EXE should work without visible console, but will actually have where to write - in the handles 0,1 and 2 that are taken by the parent invisible stub EXE. Note that killing the parent EXE may make the child lost its handles - and - crash.
You may end up with two processes in Task Manager. So you can try making these 2 processes a job like Google Chrome does.
On your question Do I need something to "initialize" standard out? - only your parent / launcher can pre-initialize your STDOUT for handles 0,1 and 2 "properly".
In my main process, i create a ffmpeg child process using CreateProcess(...).
I need to track the status of converting progress to update a progress bar. To do it, I read text from ffmpeg output and extract progress status from it.
I make a sample programm like this:
HANDLE rPipe, wPipe;
CreatePipe(&rPipe,&wPipe,&secattr,0);
STARTUPINFO sInfo;
ZeroMemory(&sInfo,sizeof(sInfo));
PROCESS_INFORMATION pInfo;
ZeroMemory(&pInfo,sizeof(pInfo));
sInfo.cb=sizeof(sInfo);
sInfo.dwFlags=STARTF_USESTDHANDLES;
sInfo.hStdInput=NULL;
sInfo.hStdOutput=wPipe;
sInfo.hStdError=wPipe;
// pStr contain ffmpeg command
CreateProcess(0,(LPTSTR)pStr,0,0,TRUE,NORMAL_PRIORITY_CLASS|CREATE_NO_WINDOW,0,0,&sInfo,&pInfo);
CloseHandle(wPipe);
BOOL ok;
do
{
memset(buf,0,bufsize);
ok=::ReadFile(rPipe,buf,100,&reDword,0);
result += buf;
}while(ok);
But I couldnt get "result" interactively updated. My app is held during conversion, and "result" string update only after ffmpeg's process finish.
How can I have my main process and ffmpeg's run simultaneously, and interactively read from/write to ffmpeg process's output/input?
Thanks for your time!
LRs
If the ffmpeg just uses stdout without explicitly flushing the output then it may not get sent to the calling process until it ends
Child processes that use such C run-time functions as printf() and
fprintf() can behave poorly when redirected. The C run-time functions
maintain separate IO buffers. When redirected, these buffers might not
be flushed immediately after each IO call. As a result, the output to
the redirection pipe of a printf() call or the input from a getch()
call is not flushed immediately and delays, sometimes-infinite delays
occur. This problem is avoided if the child process flushes the IO
buffers after each call to a C run-time IO function. Only the child
process can flush its C run-time IO buffers. A process can flush its C
run-time IO buffers by calling the fflush() function.
http://support.microsoft.com/kb/190351
In order of tracking the progress of your child process while it is running (and after its completion), you need to check the status of this child process.
After the process was launched, check the status periodically using the following code.
pi is the PROCESS_INFORMATION:
PROCESS_INFORMATION pi;
and the code:
DWORD exitCode = 0;
success = [GetExitCodeProcess][2](pi.hProcess, &exitCode);
exitCode will hold the value STILL_ACTIVE if the process is still running.
If the function succeeds, the return value of success is nonzero.
Is it possible to somehow change standart I/O functions handle on Windows? Language preffered is C++. If I understand it right, by selecting console project, compiler just pre-allocate console for you, and operates all standart I/O functions to work with its handle. So, what I want to do is to let one Console app actually write into another app Console buffer. I though that I could get first´s Console handle, than pass it to second app by a file (I don´t know much about interprocess comunication, and this seems easy) and than somehow use for example prinf with the first app handle. Can this be done? I know how to get console handle, but I have no idea how to redirect printf to that handle. Its just study-purpose project to more understand of OS work behind this. I am interested in how printf knows what Console it is assiciated with.
If I understand you correctly, it sounds like you want the Windows API function AttachConsole(pid), which attaches the current process to the console owned by the process whose PID is pid.
If I understand you correct you can find the source code of application which you want to write in http://msdn.microsoft.com/en-us/library/ms682499%28VS.85%29.aspx. This example show how to write in stdin of another application and read it's stdout.
For general understanding. Compiler don't "pre-allocate console for you". Compiler use standard C/C++ libraries which write in the output. So if you use for example printf() the following code will be executed at the end will look like:
void Output (PCWSTR pszwText, UINT uTextLenght) // uTextLenght is Lenght in charakters
{
DWORD n;
UINT uCodePage = GetOEMCP(); // CP_OEMCP, CP_THREAD_ACP, CP_ACP
PSTR pszText = _alloca (uTextLenght);
// in the console are typically not used UNICODE, so
if (WideCharToMultiByte (uCodePage, 0, pszwText, uTextLenght,
pszText, uTextLenght, NULL, NULL) != (int)uTextLenght)
return;
WriteFile (GetStdHandle (STD_OUTPUT_HANDLE), pszText, uTextLenght, &n, NULL);
//_tprintf (TEXT("%.*ls"), uTextLenght, pszText);
//_puttchar();
//fwrite (pszText, sizeof(TCHAR), uTextLenght, stdout);
//_write (
}
So if one changes the value of STD_OUTPUT_HANDLE all output will be go to a file/pipe and so on. If instead of WriteFile the program use WriteConsole function such redirection will not works, but standard C/C++ library don't do this.
If you want redirect of stdout not from the child process but from the current process you can call SetStdHandle() directly (see http://msdn.microsoft.com/en-us/library/ms686244%28VS.85%29.aspx).
The "allocating of console" do a loader of operation system. It looks the word of binary EXE file (in the Subsystem part of IMAGE_OPTIONAL_HEADER see http://msdn.microsoft.com/en-us/library/ms680339%28VS.85%29.aspx) and if the EXE has 3 on this place (IMAGE_SUBSYSTEM_WINDOWS_CUI), than it use console of the parent process or create a new one. One can change a little this behavior in parameters of CreateProcess call (but only if you start child process in your code). This Subsystem flag of the EXE you define with respect of linker switch /subsystem (see http://msdn.microsoft.com/en-us/library/fcc1zstk%28VS.80%29.aspx).
If you want to redirect printf to a handle (FILE*), just do
fprintf(handle, "...");
For example replicating printf with fprintf
fprintf(stdout, "...");
Or error reporting
fprintf(stderr, "FATAL: %s fails", "smurf");
This is also how you write to files. fprintf(file, "Blah.");
What's the best way for a running C or C++ program that's been launched from the command line to put itself into the background, equivalent to if the user had launched from the unix shell with '&' at the end of the command? (But the user didn't.) It's a GUI app and doesn't need any shell I/O, so there's no reason to tie up the shell after launch. But I want a shell command launch to be auto-backgrounded without the '&' (or on Windows).
Ideally, I want a solution that would work on any of Linux, OS X, and Windows. (Or separate solutions that I can select with #ifdef.) It's ok to assume that this should be done right at the beginning of execution, as opposed to somewhere in the middle.
One solution is to have the main program be a script that launches the real binary, carefully putting it into the background. But it seems unsatisfying to need these coupled shell/binary pairs.
Another solution is to immediately launch another executed version (with 'system' or CreateProcess), with the same command line arguments, but putting the child in the background and then having the parent exit. But this seems clunky compared to the process putting itself into background.
Edited after a few answers: Yes, a fork() (or system(), or CreateProcess on Windows) is one way to sort of do this, that I hinted at in my original question. But all of these solutions make a SECOND process that is backgrounded, and then terminate the original process. I was wondering if there was a way to put the EXISTING process into the background. One difference is that if the app was launched from a script that recorded its process id (perhaps for later killing or other purpose), the newly forked or created process will have a different id and so will not be controllable by any launching script, if you see what I'm getting at.
Edit #2:
fork() isn't a good solution for OS X, where the man page for 'fork' says that it's unsafe if certain frameworks or libraries are being used. I tried it, and my app complains loudly at runtime: "The process has forked and you cannot use this CoreFoundation functionality safely. You MUST exec()."
I was intrigued by daemon(), but when I tried it on OS X, it gave the same error message, so I assume that it's just a fancy wrapper for fork() and has the same restrictions.
Excuse the OS X centrism, it just happens to be the system in front of me at the moment. But I am indeed looking for a solution to all three platforms.
My advice: don't do this, at least not under Linux/UNIX.
GUI programs under Linux/UNIX traditionally do not auto-background themselves. While this may occasionally be annoying to newbies, it has a number of advantages:
Makes it easy to capture standard error in case of core dumps / other problems that need debugging.
Makes it easy for a shell script to run the program and wait until it's completed.
Makes it easy for a shell script to run the program in the background and get its process id:
gui-program &
pid=$!
# do something with $pid later, such as check if the program is still running
If your program forks itself, this behavior will break.
"Scriptability" is useful in so many unexpected circumstances, even with GUI programs, that I would hesitate to explicitly break these behaviors.
Windows is another story. AFAIK, Windows programs automatically run in the background--even when invoked from a command shell--unless they explicitly request access to the command window.
On Linux, daemon() is what you're looking for, if I understand you correctly.
The way it's typically done on Unix-like OSes is to fork() at the beginning and exit from the parent. This won't work on Windows, but is much more elegant than launching another process where forking exists.
Three things need doing,
fork
setsid
redirect STDIN, STDOUT and STDERR to /dev/null
This applies to POSIX systems (all the ones you mention claim to be POSIX (but Windows stops at the claiming bit))
On UNIX, you need to fork twice in a row and let the parent die.
A process cannot put itself into the background, because it isn't the one in charge of background vs. foreground. That would be the shell, which is waiting for process exit. If you launch a process with an ampersand "&" at the end, then the shell does not wait for process exit.
But the only way the process can escape the shell is to fork off another child and then let its original self exit back to the waiting shell.
From the shell, you can background a process with Control-Z, then type "bg".
Backgrounding a process is a shell function, not an OS function.
If you want an app to start in the background, the typical trick is to write a shell script to launch it that launches it in the background.
#! /bin/sh
/path/to/myGuiApplication &
To followup on your edited question:
I was wondering if there was a way to put the EXISTING process into the background.
In a Unix-like OS, there really is not a way to do this that I know of. The shell is blocked because it is executing one of the variants of a wait() call, waiting for the child process to exit. There is not a way for the child process to remain running but somehow cause the shell's wait() to return with a "please stop watching me" status. The reason you have the child fork and exit the original is so the shell will return from wait().
Here is some pseudocode for Linux/UNIX:
initialization_code()
if(failure) exit(1)
if( fork() > 0 ) exit(0)
setsid()
setup_signal_handlers()
for(fd=0; fd<NOFILE; fd++) close(fd)
open("/dev/null", O_RDONLY)
open("/dev/null", O_WRONLY)
open("/dev/null", o_WRONLY)
chdir("/")
And congratulations, your program continues as an independent "daemonized" process without a controlling TTY and without any standard input or output.
Now, in Windows you simply build your program as a Win32 application with WinMain() instead of main(), and it runs without a console automatically. If you want to run as a service, you'll have to look that up because I've never written one and I don't really know how they work.
You edited your question, but you may still be missing the point that your question is a syntax error of sorts -- if the process wasn't put in the background to begin with and you want the PID to stay the same, you can't ignore the fact that the program which started the process is waiting on that PID and that is pretty much the definition of being in the foreground.
I think you need to think about why you want to both put something in the background and keep the PID the same. I suggest you probably don't need both of those constraints.
As others mentioned, fork() is how to do it on *nix. You can get fork() on Windows by using MingW or Cygwin libraries. But those will require you to switch to using GCC as your compiler.
In pure Windows world, you'd use CreateProcess (or one of its derivatives CreateProcessAsUser, CreateProcessWithLogonW).
The simplest form of backgrounding is:
if (fork() != 0) exit(0);
In Unix, if you want to background an disassociate from the tty completely, you would do:
Close all descriptors which may access a tty (usually 0, 1, and 2).
if (fork() != 0) exit(0);
setpgroup(0,getpid()); /* Might be necessary to prevent a SIGHUP on shell exit. */
signal(SIGHUP,SIG_IGN); /* just in case, same as using nohup to launch program. */
fd=open("/dev/tty",O_RDWR);
ioctl(fd,TIOCNOTTY,0); /* Disassociates from the terminal */
close(fd);
if (fork() != 0) exit(0); /* just for good measure */
That should fully daemonize your program.
The most common way of doing this under Linux is via forking. The same should work on Mac, as for Windows I'm not 100% sure but I believe they have something similar.
Basically what happens is the process splits itself into two processes, and then the original one exits (returning control to the shell or whatever), and the second process continues to run in the background.
I'm not sure about Windows, but on UNIX-like systems, you can fork() then setsid() the forked process to move it into a new process group that is not connected to a terminal.
Under Windows, the closing thing you're going to get to fork() is loading your program as a Windows service, I think.
Here is a link to an intro article on Windows services...
CodeProject: Simple Windows Service Sample
So, as you say, just fork()ing will not do the trick. What you must do is fork() and then re-exec(), as this code sample does:
#include stdio.h>
#include <unistd.h>
#include <string.h>
#include <CoreFoundation/CoreFoundation.h>
int main(int argc, char **argv)
{
int i, j;
for (i=1; i<argc; i++)
if (strcmp(argv[i], "--daemon") == 0)
{
for (j = i+1; j<argc; j++)
argv[j-1] = argv[j];
argv[argc - 1] = NULL;
if (fork()) return 0;
execv(argv[0], argv);
return 0;
}
sleep(1);
CFRunLoopRun();
CFStringRef hello = CFSTR("Hello, world!");
printf("str: %s\n", CFStringGetCStringPtr(hello, CFStringGetFastestEncoding(hello)));
return 0;
}
The loop is to check for a --daemon argument, and if it is present, remove it before re-execing so an infinite loop is avoided.
I don't think this will work if the binary is put into the path because argv[0] is not necessarily a full path, so it will need to be modified.
/**Deamonize*/
pid_t pid;
pid = fork(); /**father makes a little deamon(son)*/
if(pid>0)
exit(0); /**father dies*/
while(1){
printf("Hello I'm your little deamon %d\n",pid); /**The child deamon goes on*/
sleep(1)
}
/** try 'nohup' in linux(usage: nohup <command> &) */
In Unix, I have learned to do that using fork().
If you want to put a running process into the background, fork it twice.
I was trying the solution.
Only one fork is needed from the parent process.
The most important point is that, after fork, the parent process must die by calling _exit(0); and NOT by calling exit(0);
When _exit(0); is used, the command prompt immediately returns on the shell.
This is the trick.
If you need a script to have the PID of the program, you can still get it after a fork.
When you fork, save the PID of the child in the parent process. When you exit the parent process, either output the PID to STD{OUT,ERR} or simply have a return pid; statement at the end of main(). A calling script can then get the pid of the program, although it requires a certain knowledge of how the program works.