Keep colored logs when piping output with docker-compose v2.5.0 - bash

In docker-compose 1.25.3 I used to pipe the output of logs. E.g.,
docker-compose logs -ft | cat
The output is colored as expected.
In docker-compose 2.5.0, this does not happen anymore. Native output docker-compose logs -ft is still colored. However, when I run:
docker-compose logs -ft | cat
The piped output is not colored anymore. Why does this happen and how can I fix it?

Related

Save output of bash command from Dockerfile after Docker container was launched

I have a Dockerfile with ubuntu image as a base.
FROM ubuntu
ARG var_name
ENV env_var_name=$var_name
ENTRYPOINT ["/bin/bash", "-c", "echo $env_var_name"]
I expect from this
executing of a simple bash script, which will take an environment variable from user keyboard input and output this value after running docker container. It goes right.
(part where i have a problem) saving values of environment variables to file + after every running of docker run --rm -e env_var_name=%valueOfVar% IMAGE-NAME i can see a list of entered from keyboard values.
My idea about part 2 were like
docker run --rm -e env_var_name=%valueOfVar% IMAGE-NAME > /directory/tosave/values.txt. That works, but only one last value saves, not a list of values.
How can i change Dockerfile to save values to a file, which Docker will see and from which Docker after running will read and ouyput values? May be i shouldn`t use ENTRYPOINT?
Appreciate for any possible help. I`ve stuck.
Emphasizing that output and save of environment variables expected.
Like #lojza hinted at, > overwrites files whereas >> appends to them which is why your command is clobbering the file instead of adding to it. So you could fix it with this:
docker run --rm -e env_var_name=%valueOfVar% IMAGE-NAME >> /directory/tosave/values.txt
Or using tee(1):
docker run --rm -e env_var_name=%valueOfVar% IMAGE-NAME | tee -a /directory/tosave/values.txt
To clarify though, the docker container is not writing to values.txt, your shell is what is redirecting the output of the docker run command to the file. If you want the file to be written to by docker you should mount a file or directory into it the container using -v and redirect the output of the echo there. Here's an example:
FROM ubuntu
ARG var_name
ENV env_var_name=$var_name
ENTRYPOINT ["/bin/bash", "-c", "echo $env_var_name | tee -a /data/values.txt"]
And then run it like so:
$ docker run --rm -e env_var_name=test1 -v "$(pwd):/data:rw" IMAGE-NAME
test1
$ docker run --rm -e env_var_name=test2 -v "$(pwd):/data:rw" IMAGE-NAME
test2
$ ls -l values.txt
-rw-r--r-- 1 root root 12 May 3 15:11 values.txt
$ cat values.txt
test1
test2
One more thing worth mentioning. echo $env_var_name is printing the value of the environment variable whose name is literally env_var_name. For example if you run the container with -e env_var_name=PATH it would print the literal string PATH and not the value of your $PATH environment variable. This does seem to be the desired outcome, but I thought it was worth explicitly spelling this out.

Ignore command output when grep'ing

I am trying to count occurrences of "POST" string inside docker logs
I am doing it like that:
docker logs 2c02 | grep "POST" -c
But I am getting not only the count of "POST" but also the full output of docker logs. Can I somehow ignore docker logs output?
docker prints to stderr. In bash you can do:
docker logs 2c02 |& grep "POST" -c

View kubectl logs page by page on bash terminal

Team,
On bash I run below to view logs but when they are huge the terminal all is all occupied dumping them out and i can't stop it for minutes.
Is there a way i can execute kubectl on bash in a a way that it shows logs only till shell screen size and then i use return key or spacebar to see more ? just like journalctl? but i want to achieve this when using kubectl.
kubectl logs test-pod -n namespace-test
Above displans 10K lines at a time which i don't want. Neither I want to exec on to the pod and see log file physically. any hints? or is there something like which displays last 100lines? or first with xargs?
There are a few ways to obtain that.
1) less
kubectl logs test-pod -n namespace-test | less
This command will allow you to read logs page by page from top to the bottom. You can use arrows to go up or down.
2) --tail=
kubectl logs test-pod -n namespace-test --tail=500
Will display last 500 rows of logs
3) pipeline with grep
kubectl logs test-pod -n namespace-test | grep <some_phrase>
It will allow you to find logs with given phrases. Please note that you can join commands via pipeline i.e
kubectl logs test-pod -n namespace-test --tail=500 | grep <some_phrase>
4) --since=
kubectl logs test-pod -n namespace-test --since=60
It display logs from last 60 seconds
5) --since-time=''
Similar to previous one. It will display logs from provided time. Format in this command is
'YYYY-MM-DDTHH:MM:ssZ'
kubectl logs test-pod -n namespace-test --since-time='2019-04-23T12:00:00Z'
You could hear about more option like but it is old format. You can scroll only from top to the bottom, you cannot go up.
kubectl logs test-pod -n namespace-test | more
You can do:
kubectl logs --tail=100 test-pod -n namespace-test

Corrupted log is generated when using watch , tail and ccze command through putty

I use putty to connnect to a cubietruck board which has armbian debian jessie software on it. I want to see coloured live log of an app. I followed the following example using watch , tail and ccze together.
When I use the command :
tail -f app.log | ccze
It worked great. Also when I use the command :
watch `tail -f app.log`
It also worked great. However when I gave :
watch --color 'tail -f app.log | ccze'
or
watch -c 'tail -f app.log | ccze'
I get a lot of
(B
charachter and in the text in most of the cases no new lines are recognized and looks as seamless text. I assume that the color related ASCII characters are not decoded correctly.
I also changed the putty keyboard from ESC to VT400 and Linux but the same problem occured.
Does anyone has an idea what am I doing wrong?
watch -c -n5 'tail app.log | ccze -A'
Leaving out the -f parameter for tail, to stop tail watching for changes in the log file (because watch should do that)
Adding the -A parameter to ccze to enable raw ANSI colors

Docker logs, stderr

Is it possible to separate docker logs between stderr \ stdout? Via fluentd\logstash etc.
The ultimate goal - sending logs to elasticsearch and filter it by stderr\stdout
If you want to separate docker logs into stdout processing and stderr processing in fluentd side, you can use rewite-tag-filter plugin with source value.
http://docs.fluentd.org/articles/out_rewrite_tag_filter
Maybe this is a duplicate of
https://github.com/docker/docker/issues/7440
Here is an example:
$ docker run -d --name foo busybox ls abcd
$ docker logs foo > stdout.log 2>stderr.log
$ cat stdout.log
$ cat stderr.log
ls: abcd: No such file or directory
See latest from issue #Opal posted.
# stdout
docker logs container_name 2>/dev/null
# stderr
docker logs container_name >/dev/null

Resources