How to compute in one Bash command line the total memory used by Docker containers running on the local Docker engine?
I use the following command to compute the total memory used in MB.
docker stats --no-stream --format 'table {{.MemUsage}}' | sed 's/[A-Za-z]*//g' | awk '{sum += $1} END {print sum "MB"}'
or if any are larger than 1GiB
docker stats --no-stream --format 'table {{.MemUsage}}' | sed 's/\.\([0-9]*\)GiB/\1MiB/g' | sed 's/[A-Za-z]*//g' | awk '{sum += $1} END {print sum "MB"}'
tl;dr
docker stats --no-stream --format '{{.MemUsage}}' | awk '{print $1}' | sed 's/GiB/ * 1024/;s/MiB//;s/KiB/ \/ 1024/' | bc -l | awk '{s+=$1} END {print s}'
Breaking this down:
docker stats --no-stream --format '{{.MemUsage}}' - Get only the memory usage
awk '{print $1}' - Strip the total memory from each line
sed 's/GiB/ * 1024/;s/MiB//;s/KiB/ \/ 1024/' - Normalize values into MiB
bc -l - Run calculations
awk '{s+=$1} END {print s}' - Sum all lines
To get total memory regardless of container size --KiB, MiB, or GiB
docker stats --no-stream --format 'table {{.MemUsage}}' | sed -n '1!p' | cut -d '/' -f1 | sed 's/GiB/ * 1024 MiB/;s/MiB/ * 1024 KiB/;s/KiB/ * 1024/; s/$/ +\\/; $a0' | bc | numfmt --to=iec-i --suffix=B "$#"
Related
##test1 on rhel8 or centos8
$for i in 1 2;do cat /proc/net/dev | grep ens192 | awk '{print "RX:"$2"\n""TX:"$10}';done | awk '{print $0}' | tee 111
##result:
RX:2541598118
TX:1829843233
RX:2541598118
TX:1829843233
I have a requirement to retrieve 239, 631 etc from the below output and store it in a variable in linux -- this is output of impala results..
+-----------------+
| organization_id |
+-----------------+
| 239 |
| 631 |
| 632 |
| 633 |
+-----------------+
below is the query I am running.
x=$(impala-shell -q "${ORG_ID}" -ki "${impalaserver}");
How to do it?
Could you please try following. This will be deleting duplicates for all Input_file(even a single id comes in 1 organization_id it will NOT br printed in other stanza then too)
your_command | awk -v s1="'" 'BEGIN{OFS=","} /---/{flag=""} /organization_id/{flag=1;getline;next} flag && !a[$2]++{val=val?val OFS s1 $2 s1:s1 $2 s1} END{print val}'
In case you need to print ids(which are coming in 1 stanza and could come in other stanza of organization_id then try following):
your_command | awk -v s="'" 'BEGIN{OFS=","} /---/{print val;val=flag="";delete a} /organization_id/{flag=1;getline;next} flag && !a[$2]++{val=val?val OFS s1 $2 s1:s1 $2 s1} END{if(val){print val}}'
What about this :
x=$(impala-shell -B -q "${ORG_ID}" -ki "${impalaserver}")
I just added the -B option which removes the pretty-printing and the header.
If you want comma-separated values you can pipe the result to tr :
echo $x | tr ' ' ','
At the moment
I want to trim .fmbi1a5nn9sp5o4qy3eyazeq5.eddvrl9sa8t448pb38vibj8ef: and .ilwio0k43fgqt4jqzyfadx19v: so the output take less space :)
First step:
docker ps --format "{{.Names}}: {{.Status}}" | sort -k1 | column -t
mon_node-exporter.fmbi1a5nn9sp5o4qy3eyazeq5.eddvrl9sa8t448pb38vibj8ef: Up 7 days
mon_prometheus.1.ilwio0k43fgqt4jqzyfadx19v: Up 7 days
I know
I can do something like:
docker ps --format "{{.Names}}: {{.Status}}" | sort -k1 | rev | cut -d"." -f2- | rev
mon_node-exporter.fmbi1a5nn9sp5o4qy3eyazeq5
mon_prometheus.1
The issue
is that I'm losing the other columns :-/
Idea
It would sound logical to do something like this (with awk) but it does not work. Any ideas?
docker ps --format "{{.Names}} : {{.Status}}" | sort -k1 | awk '{(print $1 | rev | cut -d"." -f2- | rev),$2,$3,$4,$5,$6}' | column -t
Thank you in advance!
P
to cut the last dot extension
$ docker ... | sort | awk '{sub(/\.[^.]*$/,"",$1)}1' file | column -t
mon_node-exporter.fmbi1a5nn9sp5o4qy3eyazeq5 Up 7 days
mon_prometheus.1 Up 7 days
or, delete anything longer than 20 chars after a dot.
$ ... | sed -e 's/\(\.[a-z0-9:]\{20,\}\)* / /' | column -t
mon_node-exporter Up 7 days
mon_prometheus.1 Up 7 days
Works! This trick will make my life so much easier.
(I removed file)
docker ps --format "{{.Names}}: {{.Status}}" | sort -k1 | awk '{sub(/\.[^.]*$/,"",$1)}1' | column -t;
mon_grafana.1 Up 24 hours
mon_node-exporter.fmbi1a5nn9sp5o4qy3eyazeq5 Up 23 hours
Question #2:
Now how would you proceed to cut the characters after the first dot?
Cheers!
I have the following command that I use to rewrite some maxscale output to be able to use it in other software:
maxadmin list servers | sed -r 's/[^a-z 0-9]//gi;/^\s*$/d;1,3d;' | awk '$1=$1' | cut -d ' ' -f 1,5 | sed -e 's/ /":"/g' | sed -e 's/\(.*\)/"\1"/' | tr '\n' ',' | sed 's/.$/}\n/' | sed 's/^/{/'
I am thinking this is way to complex for what I want to do, but I am not able to see a simpler version of this myself. What I want is to rewrite this (output of maxadmin list servers):
Servers.
-------------------+-----------------+-------+-------------+--------------------
Server | Address | Port | Connections | Status
-------------------+-----------------+-------+-------------+--------------------
svr_node1 | 192.168.178.1 | 3306 | 0 | Master, Synced, Running
svr_node2 | 192.168.178.1 | 3306 | 0 | Slave, Synced, Running
svr_node3 | 192.168.178.1 | 3306 | 0 | Slave, Synced, Running
-------------------+-----------------+-------+-------------+--------------------
Into this:
{"svrnode1":"Master","svrnode2":"Slave","svrnode3":"Slave"}
My command does a good job but as I said, there should be a simpler way with less sed commands being run hopefully.
You can use awk, like this:
json.awk
BEGIN {
printf "{"
}
# Everything after line for and before the last ------ line
# plus the last empty line (if any).
NR>4&&!/^([-]|$)/{
sub(/,/,"",$9) # Remove trailing comma
printf "%s\"%s\":\"%s\"",s,$1,$9
s="," # Set comma separator after first iteration
}
END {
print "}"
}
Run it like this:
maxadmin list servers | awk -f json.awk
Output:
{"svr_node1":"Master","svr_node2":"Slave","svr_node3":"Slave"}
In comments there came up the question how to achieve that without an extra json.awk file:
maxadmin list servers | awk 'BEGIN{printf"{"}NR>4&&!/^([-]|$)/{sub(/,/,"",$9);printf"%s\"%s\":\"%s\"",s,$1,$9;s=","}END{print"}"}'
Ugly, but works. ;)
If you want to put this into a shell script, consider a multiline version like this:
maxadmin list servers | awk '
BEGIN{printf"{"}
NR>4&&!/^([-]|$)/{
sub(/,/,"",$9)
printf"%s\"%s\":\"%s\"",s,$1,$9
s=","
}
END{print"}"}'
This code show user's process load (%cpu)
ps aux | awk 'NR!=1{print $1}' | sort | uniq | awk '{print "ps aux | grep "$1}' | awk '{printf $0; printf " | awk"; printf "{sum +="; print "$3}" }' | sed "s/{/ '{/g" | sed "s/3}/3} END {print \$1,sum}'/g" > 0.out
chmod 755 0.out
bash 0.out
This Code show same user in some OS(UBUNTU) example:
root 11.5
root 0
root 0
root 1.8
root 1.3
root 0
root 1.1
but show different user(uniq) on some OS example2:
root 11.5
daemon 0
syslog 0 ....
How can i write for example2 only.i want diff3rent user's %cpu.
You can replace all that with:
ps ahx -o "%U %C" | awk '
{cpu[$1] += $2}
END {for (user in cpu) {print user, cpu[user]}}
'