I have a Go program that consumes "live" input from a shell pipe, eg:
tail -f some/file | my-program
my-program is an interactive program built with rivo/tview. I want to be able to close my program with Ctrl-C and have it also terminate the tail -f that supplies input to it.
Currently I have to hit Ctrl-C twice to get back to my shell prompt. Any way I can get back to my prompt by hitting Ctrl-C once?
Adjusted my program per #torek's explanation of progress groups and observation that I can get the progress group ID using unix.Getpgid(pid):
import (
"os"
"golang.org/x/sys/unix"
)
func main() {
// do stuff with piped input
pid := os.Getpid()
pgid, err := unix.Getpgid(pid)
if err != nil {
log.Fatalf("could not get process group id for pid: %v\n", pid)
}
processGroup, err := os.FindProcess(pgid)
if err != nil {
log.Fatalf("could not find process for pid: %v\n", pgid)
}
processGroup.Signal(os.Interrupt)
}
This delivers my desired behavior from my original question.
I opted to not use syscall because of the warning I found:
Deprecated: this package is locked down. Callers should use the corresponding package in the golang.org/x/sys repository instead. That is also where updates required by new systems or versions should be applied. See https://golang.org/s/go1.4-syscall for more information.
I plan to update my program to detect whether or not it was given a pipe using the strategy outlined in this article, so when a pipe is detected, I'll do the above process group signaling on interrupt.
Any issues with that?
We'll assume a Unix-like system, using a shell that understands and engages in job control (and they all do now). When you run a command, the shell creates something called a process group or "pgroup" to hold each of the processes that make up the command. If the command is a pipeline (as this one is), each process in the pipeline gets the same pgroup-ID (see setpgid).
If the command is run in the forgeground (without &), the controlling terminal has this particular pgid assigned to it. Pressing one of the signal-generating keys, such as CTRL-C or CTRL-\, sends the corresponding signal (SIGINT and SIGQUIT in these cases) to the pgroup, using an internal killpg or equivalent. This sends the signal to every member of the pgroup.
(Backgrounding a process is simply *cough* a matter of taking back the pgid on the controlling tty, then restarting the processes in the pipeline. To make that happen is not so simple, though, as indicated by the "restarting" here.)
The likely source of the problem here is that an interactive program will place the controlling terminal into cbreak or raw mode and disable some or all signalling from keyboard keys, so that, for instance, CTRL-C no longer causes the kernel's tty module to send a signal at all. Instead, if you see a key that should cause suspension (CTRL-Z) or termination, the program has to do its own suspending or terminating. Programmers sometimes assume that this consists of simply suspending or terminating—but since the entire pipeline never got the signal in question, that's not the case, unless the entire shell pipeline consisted solely of the interactive program.
The fix is to have the program send the signal to its own pgroup, after doing any necessary cleanup (temporarily or permanently) of the controlling terminal.
Related
It's actually a Scala REPL launched as a child process from a bash script.
I want to change what ctrl-c does, but otherwise not interfere with the scala REPL.
function ctrl_c_handler() {
# do some stuff, but don't exit
}
trap ctrl_c_handler SIGINT
scala
Interestingly ignoring the signal works fine trap '' SIGINT. ctrl-c can't kill my REPL, but if I actually attach a function I haven't been able to prevent the process from exiting.
What are the options? The scala REPL needs to be in the foreground. It would be ok for the ctrl_c_handler to launch a background process.
Bash will see neither terminal input nor signals generated by the terminal (such as SIGINT) once the subprocess is started, because the subprocess becomes the only member of the terminal's foreground process group. Bash only sees the terminal again once the subprocess terminates.
Ignoring SIGINT in bash works because subprocesses inherit the dispositions of signals which have not been assigned signal handlers.
You could reassign SIGINT to a different keyboard symbol (or to no keyboard symbol) using stty; the subprocess will inherit that setting, too. That will cause ctrl-C to become an ordinary character so it won't interrupt the subprocess. Unfortunately, it won't interrupt bash either, so that's not really a solution.
You can create a subprocess over which you exert more control by creating a pseudoterminal (pty) and attaching the subprocess to the slave end of the pseudo-tty. Then you can pass input you read from the terminal to the subprocess by writing the data to the master end of the pty, and pass output from the subprocess back to the terminal by reading from the master end and writing to the terminal. In this configuration, you can trap SIGINT in the master process and not pass it through to the subprocess. However, you would almost certainly need to write this in some language (not necessarily C) which gives you access to system facilities; it might be possible in bash, but it would be quite a hack. Read man pty for a lot more information on where to start.
I have some scripts that I'm developing on a vm but sometimes needs to run on a production server to be properly tested.
I need the output from the scripts for debugging so I've tinkered together the following solution:
function test_remote() {
scp $1 Prod:/home/xxx/tmp/
n=${1:t:r}
f=${1:t}
cmd="ssh Prod \"/usr/bin/php /home/xxx/tmp/$f\" > /home/xxx/tests/$n-remote-test.html"
eval ${cmd}
ssh Prod "rm /home/xxx/tmp/$f"
echo "done"
}
which I have placed in my .zshrc file
I would like to run it in the background using
test_remote path_to_file/php_file.php &
but as I do I always get the following result
[1] + 12996 suspended (tty input) test_remote path_to_file/php_file.php
if I resume it with bg it just repeats the same message
When a background process attempts to read from standard input, it is sent a signal which suspends it. This is so the user can bring the process to the foreground again and provide the necessary input.
If no input needs to be provided, you can redirect the standard input from /dev/null, either when calling test_remote or in cmd.
SSH is reading from its standard input, which is the terminal. It does this even if the program on the remote side doesn't try to read from its standard input, because it has no way to know that the remote side won't try (and also because the user could press a key such as Ctrl+C which the server side would translate to sending a signal).
There can only sensibly be a single process reading from the terminal: if there were multiple processes, which one would receive each key press? (When that does happen, the effect is that each byte goes more or less randomly to a different process.) The terminal management framework in the kernel ensures that (under normal circumstances) only the foreground process receives terminal input. If a background process tries to read from the terminal, it receives a SIGTTIN signal and the default action for that signal is to suspend the process. “12996 suspended (tty input)” is the shell letting you know that process 12996 was suspended by a SIGTTIN.
A generic way to avoid that, with any background command that might try to read from its standard input, is to redirect its standard input from somewhere else, for example /dev/null.
mycommand </dev/null &
With the SSH client, you can use the -n option to achieve the same effect. You can also use the -f option to tell ssh to go to the background after reading a password; this is useful if you have to use a password, but you should use keys instead if at all possible. These options have the downside that the background process is not a shell job so you can't wait for its termination and get.
Why on earth are you using eval? Just write
ssh Prod "/usr/bin/php /home/xxx/tmp/$f" > /home/xxx/tests/$n-remote-test.html
I am experiencing a hard to debug problem with makePSOCKcluster from the parallel package on R x64 on Windows. It does not happen on R i386 on Windows, nor on any OSX or Linux. Unfortunately it does not happen consistently either, only occasionally and quite randomly.
What happens is that the makePSOCKcluster function times out and freezes the R session, but only if earlier in the session some (arbitrary) system() calls were performed. The video and script below illustrate the problem more clearly.
Some stuff I tried without success:
Disable antivirus/firewalls.
Waiting a couple of seconds between calling system and makePSOCKcluser.
Using different system calls.
How would I further narrow this down? Here the video and the script used in the video is:
cmd_exists <- function(command){
iswin <- identical(.Platform$OS.type, "windows");
if(iswin){
test <- suppressWarnings(try(system(command, intern=TRUE, ignore.stdout=TRUE, ignore.stderr=TRUE, show.output.on.console=FALSE), silent=TRUE));
} else {
test <- suppressWarnings(try(system(command, intern=TRUE, ignore.stdout=TRUE, ignore.stderr=TRUE), silent=TRUE));
}
!is(test, "try-error")
}
options(hasgit = cmd_exists("git --version"));
options(haspandoc = cmd_exists("pandoc --version"));
options(hastex = cmd_exists("texi2dvi --version"));
cluster <- parallel::makePSOCKcluster(1);
makePSOCKCluster, or more generally makeCluster, can hang for any number of reasons when creating the so-called worker processes, which involves starting new R sessions using the Rscript command that will execute the .slaveRSOCK function, which will create a socket connection back to the master and then execute the slaveLoop function where it will eventually execute the tasks sent to it by the master. Wehen something goes wrong when starting any of the worker processes, the master will hang while executing socketConnection, waiting for the worker to connect to it even though that worker may have died or never even been created successfully.
Using the outfile argument is great because it often reveals the error that causes the worker process to die and thus the master to hang. But if that reveals nothing, then go to manual mode. In manual mode, the master prints the command to start each worker instead of executing the command itself. It's more work, but it gives you complete control, and you can even debug into the workers if you need to.
Here's an example:
> library(parallel)
> cl <- makePSOCKcluster(1, manual=TRUE, outfile='log.txt')
Manually start worker on localhost with
'/usr/lib/R/bin/Rscript' -e 'parallel:::.slaveRSOCK()' MASTER=localhost
PORT=10187 OUT=log.txt TIMEOUT=2592000 METHODS=TRUE XDR=TRUE
Next open a new terminal window (command prompt, or whatever), and paste in that Rscript command. As soon as you've executed it, makePSOCKcluster should return since we only requested one worker. Of course, if something goes wrong, it won't return, but if you're lucky, you'll get an error message in your terminal window and you'll have an important clue that will hopefully lead to a solution to your problem. If you're not so lucky, the Rscript command will also hang, and you'll have to dive in even deeper.
To debug the worker, you don't execute the displayed Rscript command because you need an interactive session. Instead, you start an R session with a command such as:
$ R --vanilla --args MASTER=localhost PORT=10187 OUT=log.txt TIMEOUT=2592000 METHODS=TRUE XDR=TRUE
In that R session, you can put a breakpoint on the .slaveRSOCK function and then execute it:
> debug(parallel:::.slaveRSOCK)
> parallel:::.slaveRSOCK()
Now you can start stepping through the code, possibly setting breakpoints on the slaveLoop and makeSOCKmaster functions.
Is there a way to get a process's child process status based on its PID in Ruby?
For example, in Python you can do psutil.Process(pid).status
I don't know of a portable ruby method to get process state of a running process. You can do Process.wait and check $?.exitstatus, but that doesn't look like what you want. For a posix solution, you could use
`ps -o state -p #{pid}`.chomp
to get the letter code ps produces for process state
PROCESS STATE CODES
Here are the different values that the s, stat and state output specifiers
(header "STAT" or "S") will display to describe the state of a process.
D Uninterruptible sleep (usually IO)
R Running or runnable (on run queue)
S Interruptible sleep (waiting for an event to complete)
T Stopped, either by a job control signal or because it is being traced.
W paging (not valid since the 2.6.xx kernel)
X dead (should never be seen)
Z Defunct ("zombie") process, terminated but not reaped by its parent.
I was looking for the same thing. It's a shame ProcessStatus doesn't seem to be able to get initialized from a live pid. This is vital stuff if you want to do anything like a safe timed kill of a child process.
In any case,
it's the second line in /proc/$pid/status if you're on Linux.:
status_line = File.open("/proc/#{pid}/status") {|f| f.gets; f.gets }
Most likely much much faster than anything involving an external program.
On OS X, I setup a string:
outputstring="ps -O=S -p #{mypid}"
then execute it in a %x call:
termoutput=%x[#{outputstring}]
I can display that if needed, or just keep the output clean and act on the State I found with the call.
My scripts cdist-deploy-to and cdist-mass-deploy (from cdist configuration management) run interactively (i.e. are called by a user).
These scripts call a lot of scripts, which again call some scripts:
cdist-mass-deploy ...
cdist-deploy-to ...
cdist-explorer-run-global ...
cdist-dir ....
What I want is to exit / kill all scripts, as soon as cdist-mass-deploy is either stopped by control C (SIGINT) or killed with SIGTERM.
cdist-deploy-to can also be called interactively and should exhibit the same behaviour.
Using ps -ef... and co variants to find out all processes with the ppid looks like it could be quite unportable. Using $! does not work as in the deeper levels the children are no background processes.
I tried using the following code:
__cdist_kill_on_interrupt()
{
__cdist_tmp_removal
kill 0
exit 1
}
trap __cdist_kill_on_interrupt INT TERM
But this leads to ugly Terminated messages as well as to a segfault in the shells (dash, bash, zsh) and seems not to stop everything instantly anyway:
# cdist-mass-deploy -p ikq04.ethz.ch ikq05.ethz.ch
core: Waiting for cdist-deploy-to jobs to finish
^CTerminated
Terminated
Terminated
Terminated
Segmentation fault
So the question is, how to cleanly exit including all (sub-)children in a portable manner (bourne shell, no csh support needed)?
You don't need to handle ^C, that will result in a signal being sent to the whole process group, which will kill all the processes that are not in the background. So you don't need to catch INT.
The only reason you get a Terminated when you kill them is that kill sends TERM by default, but that's reasonable if you are handling a TERM in the first place. You could use kill -INT 0 if you want to avoid the messages.
(responding with extra info)
If the child processes are run in the background, you can get their process ids just after you start them, using the $! special shell variable. Gather these together in a variable and just kill them all when you need to terminate.