Is there is the best way to read results of some bash commands, which are displayed as a table, if I need some particular row and column there?
For example, when I run this line:
gcloud app instances describe $instance_name --service=postprocessing --version=$instance_version
I get this:
startTime: '2020-08-03T16:29:29.142Z'
vmDebugEnabled: true
vmIp: xx.xx.xxx.xxx
vmStatus: RUNNING
I need only IP from this table, how do I get it?
I found only one way, to save the whole output as an array and then get -3rd element of it. I was wondering if there is a better way to it?
Also for other types of output with more "columns" which have a header, like this one:
mst#cloudshell:~ (me)$ free -m
total used free shared buff/cache available
Mem: 1995 469 279 0 1247 1379
Swap: 767 0 767
How do I get free/Swap, for example?
I suggest using awk to get the value you want in the output.
To set $VMIP variable with the vmIP value (2nd field):
VMIP=$(gcloud app ... | awk '/vmIp:/ {print $2}' )
To set $SWAP_FREE variable with the Swap free value (4th field):
SWAP_FREE=$(free -m | awk '/Swap:/ {print $4}')
Related
Concept is a debian running computer with an SH script file in retrieving data.
I have searched on how to convert data from csv back to columns and rows. This was fairly easy.
for instance, i have this:
And using this command: column -t -s, > this will gives me this:
Date SkyT_Min SkyT_Min_Time SkyT_Max SkyT_Max_Time SkyT_Mean AmbT_Min AmbT_Min_Time AmbT_Max AmbT_Max_Time
2019-09-19 -22.8 07:29:48.00 -1.6 12:27:57.00 -16.77 5.9 05:15:11.00 23.4 14:28:49.00
2019-09-25 -15.8 11:49:40.00 9.1 20:17:11.00 1.07 14.7 02:47:10.00 21.7 11:46:38.00
2019-09-26 -9.6 02:59:29.00 10.8 11:09:18.00 5.66 16.4 20:58:37.00 23.4 14:08:58.00
So overall that is already great!
But how do i get it to get the values in each column to start on the far right, instead on the left?
So i would get something like this:
Thank you very much for your answer.
UPDATE:
I have found the answer to my question:
--R
But it gives me errors: "invalied option -- 'R'
So if anyone has any idea. :)
You can work with awk:
-F tells awk the column separator
"%20s " tells awk to reserve 20 characters for each column and to concat a whitespace to each entry
awk -F "," '{for(i=1;i<=NF;i++){printf "%20s ", $i}; printf "\n"}' inputfile.csv
Output:
Date SkyT_Min SkyT_Min_Time SkyT_Max SkyT_Max_Time SkyT_Mean AmbT_Min AmbT_Min_Time AmbT_Max AmbT_Max_Time
2019-09-19 -22.8 07:29:48.00 -1.6 12:27:57.00 -16.77 5.9 05:15:11.00 23.4 14:28:49.00
2019-09-25 -15.8 11:49:40.00 9.1 20:17:11.00 1.07 14.7 02:47:10.00 21.7 11:46:38.00
2019-09-26 -9.6 02:59:29.00 10.8 11:09:18.00 5.66 16.4 20:58:37.00 23.4 14:08:58.00
P.S.: Please avoid screenshots in the future because nobody likes them.
Below is the HP ssacli command to see configured hardware RAID details:
ssacli ctrl slot=0 show config
and its output is as below:
HPE Smart Array P408i-a SR Gen10 in Slot 0 (Embedded)
Internal Drive Cage at Port 1I, Box 1, OK
Internal Drive Cage at Port 2I, Box 0, OK
Port Name: 1I (Mixed)
Port Name: 2I (Mixed)
Array A (Solid State SAS, Unused Space: 0 MB)
logicaldrive 1 (447.10 GB, RAID 1, OK)
physicaldrive 1I:1:1 (port 1I:box 1:bay 1, SAS SSD, 480 GB, OK)
physicaldrive 1I:1:2 (port 1I:box 1:bay 2, SAS SSD, 480 GB, OK)
SEP (Vendor ID HPE, Model Smart Adapter) 379 (Port: Unknown)
I have to figure out the Array name in order to delete it by searching for the matching disk info which I get as input from the user. For example if the disk input is 1I:1:1, then I have to search this string in the output of the above command. Since this disk is available and matching, I have to extract the Array name (here it is A) and once I get this Array parameter then I can go ahead delete this existing RAID configuration.
ssacli ctrl slot=0 show config | grep -B 4 '1I:1:1' | grep Array | awk '{print $2}'
The problem with the above command is,
value 4 in the grep -B cannot be always constant as the matching disk may come first, second or third or so on under an Array in the output.
there may be multiple RAID array configurations available in the output, so there may be Array A, B, C etc., I have to find and retrieve the nearest Array string that matches my input disk
I think your requirement could be simply solved with a single usage of awk. You store the disk name as a variable to be passed and first store the array names as you go through the list of lines. Once you do a match of the actual disk name, you print the array you just stored. So pipe your command output to
| awk -v disk="1I:1:1" '/^[[:space:]]*Array/{ array=$2; } $0 ~ disk { print array; exit }'
This answer assumes that the array names do not contain spaces or else it would be broken and would print the first part of the array name only.
You can process the file from the end:
tac infile \
| awk -v input='1I:1:1' '$0 ~ input {flag=1} flag && /Array/ {print $2; exit}'
This sets a flag when encountering a line matching the user input; after that, if a line matches Array and the flag is set, the second field is printed.
I just want to share a small script that I made to enhance the docker stats command.
I am not sure about the exactitude of this method.
Can I assume that the total amount of memory consumed by the complete Docker deployment is the sum of each container consumed memory ?
Please share your modifications and or corrections. This command is documented here: https://docs.docker.com/engine/reference/commandline/stats/
When running a docker stats The output looks like this:
$ docker stats --all --format "table {{.MemPerc}}\t{{.CPUPerc}}\t{{.MemUsage}}\t{{.Name}}"
MEM % CPU % MEM USAGE / LIMIT NAME
0.50% 1.00% 77.85MiB / 15.57GiB ecstatic_noether
1.50% 3.50% 233.55MiB / 15.57GiB stoic_goodall
0.25% 0.50% 38.92MiB / 15.57GiB drunk_visvesvaraya
My script will add the following line at the end:
2.25% 5.00% 350.32MiB / 15.57GiB TOTAL
docker_stats.sh
#!/bin/bash
# This script is used to complete the output of the docker stats command.
# The docker stats command does not compute the total amount of resources (RAM or CPU)
# Get the total amount of RAM, assumes there are at least 1024*1024 KiB, therefore > 1 GiB
HOST_MEM_TOTAL=$(grep MemTotal /proc/meminfo | awk '{print $2/1024/1024}')
# Get the output of the docker stat command. Will be displayed at the end
# Without modifying the special variable IFS the ouput of the docker stats command won't have
# the new lines thus resulting in a failure when using awk to process each line
IFS=;
DOCKER_STATS_CMD=`docker stats --no-stream --format "table {{.MemPerc}}\t{{.CPUPerc}}\t{{.MemUsage}}\t{{.Name}}"`
SUM_RAM=`echo $DOCKER_STATS_CMD | tail -n +2 | sed "s/%//g" | awk '{s+=$1} END {print s}'`
SUM_CPU=`echo $DOCKER_STATS_CMD | tail -n +2 | sed "s/%//g" | awk '{s+=$2} END {print s}'`
SUM_RAM_QUANTITY=`LC_NUMERIC=C printf %.2f $(echo "$SUM_RAM*$HOST_MEM_TOTAL*0.01" | bc)`
# Output the result
echo $DOCKER_STATS_CMD
echo -e "${SUM_RAM}%\t\t\t${SUM_CPU}%\t\t${SUM_RAM_QUANTITY}GiB / ${HOST_MEM_TOTAL}GiB\tTOTAL"
From the documentation that you have linked above,
The docker stats command returns a live data stream for running containers.
To limit data to one or more specific containers, specify a list of container names or ids separated by a space.
You can specify a stopped container but stopped containers do not return any data.
and then furthermore,
Note: On Linux, the Docker CLI reports memory usage by subtracting page cache usage from the total memory usage.
The API does not perform such a calculation but rather provides the total memory usage and the amount from the page cache so that clients can use the data as needed.
According to your question, it looks like you can assume so, but also do not forget it also factors in containers that exist but are not running.
Your docker_stats.sh does the job for me, thanks!
I had to add unset LC_ALL somewhere before LC_NUMERIC is used though as the former overrides the latter and otherwise I get this error:
"Zeile 19: printf: 1.7989: Ungültige Zahl." This is probably due to my using a German locale.
There is also a discussion to add this feature to the "docker stats" command itself.
Thanks for sharing the script! I've updated it in a way it depends on DOCKER_MEM_TOTAL instead of HOST_MEM_TOTAL as docker has it's own memory limit which could differ from host total memory count.
I am new to shell scripting, and want to implement a script on my server which will automatically send e-mail alerts if:
Disk usage exceeds 90%
Disk usage exceeds 95% (In addition to the previous e-mail)
My filesystem is abc:/xyz/abc and my mount is /pqr. How can I set this up via scripts?
You can use the df command to check the file system usage. As a starting point, you can use the below command:
df -h | awk -v val=90 '$1=="/pqr"{x=int($5)>val?1:0;print x}'
The above command will print 1 if more than threshold, else print 0. The threshold is set in val.
Note: Please ensure the 5th column of your df output is the use percentage, else use appropriate column.
I'm really new to UNIX/Shell Scripting I'm trying to extract disk usage from numerous servers. So what I'm trying to do is making a shell script that runs
df -g > diskusage.txt to obtain following table and extract ** data from below
Filesystem Size Used Avail Use% Mounted on
/dev/ibm_lv 84.00 56.81 33% 637452 5% /usr/IBM
/dev/apps_lv 10.00 9.95 **1%** 5 1% /usr/apps
/dev/upi_lv 110.00 85.85 **22%** 90654 1% /usr/app/usr
user08:/backup 2000.00 1611.22 20% 177387 1% /backup
Depending on the server, there are more file systems but i only want /usr/apps/usr,/usr/apps disk usage regardless of the number of filesystem. (/usr/apps/usr,/usr/apps will always located at last three row)
I'm pretty sure there are simpler ways than reading last 3 lines -> disregard last line -> search for % on each line.
If there is better way to extract these data, please let me know.
df -g | awk '/\/usr\/app/ {print $4}'
That gets you the available percentages, but it doesn't tell you which one goes with which. You can always include the mountpoint in the output, but then you still have to do some parsing to get the numbers out, something like this:
while read avail mount; do
echo "$mount has $avail available"
done < <(df -g | awk '/\/usr\/app/ {print $4, $NF}')