I commonly find myself making docker-compose environments for development purposes.
However, an annoying thing to have to do when I start up my services with docker-compose up -d is to have to run docker-compose logs -f every time all services come down.
I'd like to be able to run a script that attempts to run docker-compose logs -f and then once it succeeds, acts as it normally would. Then once the services are down, continue to retry to run until they are up again.
Does this make sense? I've tried using watch but that is not behaving how I'd like and trying a loop with a sleep command in it doesn't produce any useful results either.
If you want to run a command in a loop, just:
while sleep 1; do docker-compose logs -f; done
It will run docker-compose logs -f every second.
watch clears the screen each period, so you will not see all the messages then.
Related
In https://squidfunk.github.io/mkdocs-material/creating-your-site/#previewing-as-you-write, there's a command that will launch my document site.
docker run --rm -it -p 8000:8000 -v ${PWD}:/docs squidfunk/mkdocs-material
I want that after it is launch, I will automatically open the browser and see it.
I write a script as below
docker run --rm -it -p 8000:8000 -v ${PWD}:/docs squidfunk/mkdocs-material
open http://localhost:8000
But it turns out the open command cannot be triggered, as the previous docker run is still holding the process still.
If I use & as below, then the open will get called too fast before the page is ready
docker run --rm -it -p 8000:8000 -v ${PWD}:/docs squidfunk/mkdocs-material &
open http://localhost:8000
How can I get open called at the right time?
(FYI, I'm using GNU bash, version 3.2.57(1)-release)
How can I get open called at the right time?
Opening the browser at exactly the right time would require your server mkdocs to give some signal that it is ready. Since you probably don't want to modify the code of the server, you just have to wait for the right time and then open the page.
Either measure the startup time once by hand and then use a fixed wait time or check the page repeatedly until it loads.
In both cases, the docker command and the process of opening the page must run in parallel. bash can run run things in parallel using background jobs (... &). Since docker -it must run in the foreground, we run open as a the background job. This might seem a little strange, since we seemingly open the website before starting the server, but keep in mind that both commands run in parallel.
Either
# replace 2 with your measured time
sleep 2 && open http://localhost:8000 &
docker run --rm -it -p 8000:8000 -v "${PWD}:/docs" squidfunk/mkdocs-material
or
while ! curl http://localhost:8000 -s -f -o /dev/null; do
sleep 0.2
done && open http://localhost:8000 &
docker run --rm -it -p 8000:8000 -v "${PWD}:/docs" squidfunk/mkdocs-material
It sounds (to me) like:
docker run is a blocking process (it does not exit and/or return control to the console) so ...
the open is never run (unless the docker run command is aborted in which case the open will fail), and ...
pushing docker run into the background means the open is run before the URL is fully functional
If this is the case I'm wondering if you could do something like:
docker run ... & # put in background, return control to console
sleep 3 # sleep 3 seconds
open ...
NOTE: manually picking the number of seconds to sleep (3 in this case) isn't ideal but a decent number that guarantees URL availability and doesn't leave you hanging should be doable with some testing
Another 'basic' option might be a looping construct combined with a sleep, eg:
docker run ... &
while true # loop indefinitely
do
sleep 1 # sleep 1 sec
open ... 2>/dev/null # try the open
[[ $? == 0 ]] && break # if it doesn't fail then break out of loop, ie,
# if it does fail then repeat loop
done
In my Travis CI, part of my verification is to start a docker container and verify that it doesn't fail within 10 seconds.
I have a yarn script docker:run:local that calls docker run -it <mytag> node app.js.
If I call the yarn script with timeout from a bash shell, it works fine:
$ timeout 10 yarn docker:run:local; test $? -eq 124 && echo "Container ran for 10 seconds without error"
This calls docker run, lets it run for 10 seconds, then kills it (if not already returned). If the exit code is 124, the timeout did expire, which means the container was still running. Exactly what I need to verify that my docker container is reasonably sane.
However, as soon as I run this same command from within a script, either in a test.sh file called from the shell, or if putting it in another yarn script and calling yarn test:docker, the behaviour is completely different. I get:
ERRO[0000] error waiting for container: context canceled
Then the command hangs forever, there's no 10 second timeout, I have to ctrl-Z it and then kill -9 the process. If I run top I now have a docker process using all my CPU forever. If using timeout with any other command like sleep 20 && echo "Finished sleeping", this does not happen, so I suspect it may have something to do with how docker works in interactive mode or something, but that's only my guess.
What's causing timeout docker:run to fail from a script but work fine from a shell and how do I make this work?
Looks like running docker in interactive mode is causing the issue.
Run docker in detached more by removing the -it and allowing it to run in default detached mode or specify -d instead of -it and so:
docker run -d <mytag> node
or
docker run <mytag> node
I would like to run a CloudFoundry app-nozzle command for 10 seconds to gather some metrics about an application. Even though I stop the command, there is still new output in the output file afterwards. I have no idea what is happening.
My command (that would be run inside a script):
timeout 10s cf app-nozzle my_app --filter ContainerMetric > CF_nozzle.txt
It looks that it stopped and exited in Git Bash, I can run other scripts, even after minutes there are new lines in the file. I closed the whole window, and it is still ongoing.
Update: I tried it in CLI only and after the timeout it still emmits data even to command line.
It seems that this might be a bug in Windows Git bash. The same command works well in Ubuntu terminal.
Testing out AppFog and I've run into their tailing problem. Specifically, they don't offer tailing of your server. A big sticking point for me as I prefer to work on a remote dev server and without access to the logs it makes it very difficult to debug. Same with staging.
They do offer the following to pull the logs down:
af logs my-app-name --all
Which will dump whatever's in the log to your terminal. Not exactly elegant, but at least the info is there.
But it's not continuous. And having to type af logs my-app-name --all a million times will make me go out of my mind, especially while I'm trying to hunt down a bug.
So I thought I'd write a shell script that will fire the af logs command against my app server for me, and I came up with this:
#!/bin/bash
while true; do
af logs $1
sleep 3
done
Where $1 is the name of my app. So I'd use it like so:
af-tail my-app-name
And then every three seconds I'd get a log dump from my app server. However, I keep getting all the logs and really I'd like it to concatenate any missing entries to the existing 'stream' in my terminal, but I'm not sure how I'd go about doing that. Any help?
Maybe this could help. I use it to monitor remote logs in my local machine.
https://gist.github.com/iolloyd/da60ef316643d7894bdf
I'm creating a pallet crate for elasticsearch. I was stuck on the service not starting however after looking at the logs it seems that it's not really anything to do with pallet. I am using the elasticsearch apt package for 1.0 which includes an init script. If I run sudo service elasticsearch start then ES starts with no problems. If pallet does this for me then it records standard out as having started it successfully
start elasticsearch
* Starting Elasticsearch Server
...done.
However it is not started.
sudo service elasticsearch status
* elasticsearch is not running
I messed around with the init script and I found if I added sleep 1 after starting the daemon then it works correctly with pallet.
start-stop-daemon --start -b --user "$ES_USER" -c "$ES_USER" --pidfile "$PID_FILE" --exec $DAEMON -- $DAEMON_OPTS
#this sleep will allow it to work
#sleep 1
log_end_msg $?
I don't understand what is going on?
I've seen issues like this before, too. It generally comes down to expecting something to have finished before the script finishes, which may not always happen with services since they fork off background tasks that may still get killed when the ssh connection is terminated.
For these kinds of things you should use Pallet's built in code for running things under supervision. This also has the advantage of making it very easy to switch from plain init.d to runit or daemontools later, which is especially useful for Elasticsearch because it's a JVM process and nearly any JVM will eventually crash if you let it run long enough.