For example, a container runs the main thread and a goroutine. The main thread encounters an issue and terminates. Note that for Golang, termination of the main thread does not result in auto-termination of the goroutine.
As the main thread has been terminated, will the container be killed and re-created? Or will the container continue running due to the goroutine is still running?
If the container will be killed and re-created after the main thread has been terminated, will this result in the goroutine getting terminated as well? Or will the goroutine continue running indefinitely and there is no easy way to terminate it now?
If the main functions exists, the program is stopped. Nothing will run any more. It will release any used resource, like file descriptors and database connections.
In the below program, we will never see done being printed.
func main() {
go func() {
time.Sleep(time.Minute)
fmt.Println("done")
}()
time.Sleep(time.Second * 3)
}
https://play.golang.com/p/kPKZDdMcduS
If the program with that main function was the foreground process of the container, then the container shuts down as its standard behaviour with containers.
If you run the below example, you can observe how the container shuts down as soon as the sleep finishes.
$ docker run --name sample busybox sleep 3 && docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
fd4319261a0d busybox "sleep 3" 4 seconds ago Exited (0) Less than a second ago sample
If the container, the program is running in, is shut down, it's more or less as if you would pull the plug of your computer. Nothing will run on your computer any more. It's impossible.
I would encourage you to create some test scenarios yourself and validate this.
Related
This question already has answers here:
At which point a goroutine can yield?
(2 answers)
Closed 9 months ago.
I am trying to understand how "go" can preempt a loop or blocking call. I understand that with SIGURG signal, the blocking surbroutine stack is poisoned and a JMP is made to preempt function. However when the subroutine is given a chance again to execute, how can it exit the loop as is mentioned in the article https://developpaper.com/goroutine-preemptive-scheduling-with-new-features-of-go-1-14/
package main
import (
"fmt"
"runtime"
"time"
)
func main() {
runtime.GOMAXPROCS(1)
fmt.Println("The program starts ...")
go func() {
for {
}
}()
time.Sleep(time.Second)
fmt.Println("I got scheduled!")
}
With go 1.13, the program hangs forever
$ docker run -it --rm app13:latest
The program starts ...
With go 1.14, the loop exits?
$ docker run -it --rm app14:latest
The program starts ...
I got scheduled!
Is there any good documentation, paper i can read to understand this behavior. To me this looks like a bug, a loop is a loop, how can it exit irrespective it is prempt or not.
The Go Language Specification says:
Program execution begins by initializing the main package and then invoking the function main. When that function invocation returns, the program exits. It does not wait for other (non-main) goroutines to complete.
Preemption in Go 1.14 and later ensures that the goroutine executing the main function runs and that that main function returns. The program exits when the main function returns. Any running goroutines cease to exist when the program exits.
In the absence of preemption, the goroutine executing the main function may not run. The program hangs if the main function does not return.
I'm using a Dockerfile that ends with a CMD ["/start.sh"]:
#!/bin/bash
service ssh start
/usr/bin/node /myApp/app.js
if for some reason i need to kill the node process, the ssh server is being closed as well (forces me to reboot the container to reconnect).
Any simple way to avoid this behavior?
Thank You.
The container exits as soon as main process of the container exits. In your case, the main process inside the container is start.sh shell script. The start.sh shell script is starting the ssh service and then running the nodejs process as child process. Once the nodejs process dies, the shell script exits as well and so the container exits. So what you can do is to put the nodejs process in background.
#!/bin/bash
service ssh start
/usr/bin/node /myApp/app.js &
# Need the following infinite loop as the shell script should not exit
while do:
sleep 2
done
I DO NOT recommend this approach though. You should have only a single process per container. Read the following answers to understand why -
Running multiple applications in one docker container
If you still want to run multiple processes inside container, there are better ways to do it like using supervisord - https://docs.docker.com/config/containers/multi-service_container/
I'm trying to use Elasticsearch in Docker for local dev. While I can find containers that work, when docker stop is sent, the containers hang for the default 10s, then docker forcibly kills the container. My assumption here is that ES is either not on PID 1 or other services prevent it from shutting down immediately.
I'm curious if anyone can expand on this, or explain why this is happening more accurately. I'm running numerous tests and 10s+ to shutdown is just annoying when other containers shutdown after 1-2s.
If you don't want to wait the 10 seconds, you can run a docker kill instead of a docker stop. You can also adjust the timeout on docker stop with the -t option, e.g. docker stop -t 2 $container_id to only wait 2 seconds instead of the default 10.
As for why it's ignoring the sigkill, that may depend on what image you are running (there's more than one for elasticsearch). However, if pid 1 is a shell like /bin/sh or /bin/bash, it will not pass signals through. If pid 1 is the elasticsearch process, it may ignore the signal, or 10 seconds may not be long enough for it to fully cleanup and shutdown.
I have been running a server with:
go run server.go &
When I am done with the process, this requires me to kill the process. The way I have been doing this is with kill PID of the go run process.
However, I've noticed my server appears connected still. Investigating further it seems that there are other processes go is starting which appear to "keep alive" my server:
$ps aux | grep go
username 70481 0.0 0.0 573416816 5228 ?? S 3:15PM 0:00.63 /var/folders/wf/89r2567s5hv48lj1g9l65mbw0000gp/T/go-build062422854/command-line-arguments/_obj/exe/server
username 70472 0.0 0.0 573407408 7720 ?? S 3:15PM 0:00.80 go run server.go
When I kill this associated process, too, I see that my connection is released as expected.
Is there a better way to "really" kill a golang process than kill PID? Or is there a reason why there are these phantom processes?
I have resorted to killing both but this seems... strange.
$ go version
go version go1.5.4 darwin/amd64
The one you're calling the "phantom process" is your server, which is why killing it kills your server. The other is the "go run" utility itself, which is compiling your code and then running the resulting executable.
I am experiencing a hard to debug problem with makePSOCKcluster from the parallel package on R x64 on Windows. It does not happen on R i386 on Windows, nor on any OSX or Linux. Unfortunately it does not happen consistently either, only occasionally and quite randomly.
What happens is that the makePSOCKcluster function times out and freezes the R session, but only if earlier in the session some (arbitrary) system() calls were performed. The video and script below illustrate the problem more clearly.
Some stuff I tried without success:
Disable antivirus/firewalls.
Waiting a couple of seconds between calling system and makePSOCKcluser.
Using different system calls.
How would I further narrow this down? Here the video and the script used in the video is:
cmd_exists <- function(command){
iswin <- identical(.Platform$OS.type, "windows");
if(iswin){
test <- suppressWarnings(try(system(command, intern=TRUE, ignore.stdout=TRUE, ignore.stderr=TRUE, show.output.on.console=FALSE), silent=TRUE));
} else {
test <- suppressWarnings(try(system(command, intern=TRUE, ignore.stdout=TRUE, ignore.stderr=TRUE), silent=TRUE));
}
!is(test, "try-error")
}
options(hasgit = cmd_exists("git --version"));
options(haspandoc = cmd_exists("pandoc --version"));
options(hastex = cmd_exists("texi2dvi --version"));
cluster <- parallel::makePSOCKcluster(1);
makePSOCKCluster, or more generally makeCluster, can hang for any number of reasons when creating the so-called worker processes, which involves starting new R sessions using the Rscript command that will execute the .slaveRSOCK function, which will create a socket connection back to the master and then execute the slaveLoop function where it will eventually execute the tasks sent to it by the master. Wehen something goes wrong when starting any of the worker processes, the master will hang while executing socketConnection, waiting for the worker to connect to it even though that worker may have died or never even been created successfully.
Using the outfile argument is great because it often reveals the error that causes the worker process to die and thus the master to hang. But if that reveals nothing, then go to manual mode. In manual mode, the master prints the command to start each worker instead of executing the command itself. It's more work, but it gives you complete control, and you can even debug into the workers if you need to.
Here's an example:
> library(parallel)
> cl <- makePSOCKcluster(1, manual=TRUE, outfile='log.txt')
Manually start worker on localhost with
'/usr/lib/R/bin/Rscript' -e 'parallel:::.slaveRSOCK()' MASTER=localhost
PORT=10187 OUT=log.txt TIMEOUT=2592000 METHODS=TRUE XDR=TRUE
Next open a new terminal window (command prompt, or whatever), and paste in that Rscript command. As soon as you've executed it, makePSOCKcluster should return since we only requested one worker. Of course, if something goes wrong, it won't return, but if you're lucky, you'll get an error message in your terminal window and you'll have an important clue that will hopefully lead to a solution to your problem. If you're not so lucky, the Rscript command will also hang, and you'll have to dive in even deeper.
To debug the worker, you don't execute the displayed Rscript command because you need an interactive session. Instead, you start an R session with a command such as:
$ R --vanilla --args MASTER=localhost PORT=10187 OUT=log.txt TIMEOUT=2592000 METHODS=TRUE XDR=TRUE
In that R session, you can put a breakpoint on the .slaveRSOCK function and then execute it:
> debug(parallel:::.slaveRSOCK)
> parallel:::.slaveRSOCK()
Now you can start stepping through the code, possibly setting breakpoints on the slaveLoop and makeSOCKmaster functions.