Bash- Converting a variable to human readable format (KB, MB, GB) - bash

In my bash script, I run through a list of directories and read in the size of each directory to a variable using the du command. I also keep a running total of the total size of the directories. The only problem is that after I get the total size, it's in an unreadable format (ex. 64827120). How can I convert the variable containing this number into GBs, MBs, etc?

You want to use du -h which gives you a 'human readable' output ie KB, MB, GB, etc.

You can use numfmt to convert raw (decimal) numbers
to human-readable form. 
Use --to=iec to output binary-prefix numbers
(i.e., K=1024, M=220, etc.)
$ printf '%s %s\n' 1000000 foo 1048576 bar | numfmt --to=iec
977K foo
1.0M bar
and use --to=si to output metric-prefix numbers
(i.e., K=1000, M=106, etc.)
$ printf '%s %s\n' 1000000 foo 1048576 bar | numfmt --to=si
1.0M foo
1.1M bar
If you specifically want to get “MB”, “GB”, etc., use --suffix:
$ printf '%s %s\n' 1000000 foo 1048576 bar | numfmt --to=si --suffix=B
1.0MB foo
1.1MB bar
If your numbers are in a column other than the first
(as in Mik R’s answer), use --field:
$ printf '/home/%s %s\n' foo 1000000 bar 1048576 | numfmt --to=si --field=2
/home/foo 1.0M
/home/bar 1.1M
Or you can convert numbers on the command line (instead of using a pipe):
$ numfmt --to=si 1000000 1048576
1.0M
1.1M

Try using du -sh for getting summarise size in human readable, also you can find the command related help in manual.
Try below command, it will give you the size in Human readable format
du | tail -1 | awk {'print $1'} | awk '{ total = $1 / 1024 ; print total "MB" }'
du | tail -1 | awk {'print $1'} | awk '{ total = $1 / 1024/1024 ; print total "GB" }'

This is a combination of #Mahattam response and some others I combined which tallys the total amount in the standard format and then formats the output in human readable.
for path in $(awk -F: '{if ($3 >= 1000) print $6}' < /etc/passwd); do disk_usage=0; disk_usage=$(du -s ${path} | grep -oE '[[:digit:]]+'); echo "$path: $(echo $disk_usage | tail -1 | awk {'print $1'} | awk '{ total = $1 / 1024/1024 ; printf("%.2fGB\n", total) }')"; myAssociativeArray[${path}]=${disk_usage}; done ; total=$(IFS=+; echo "$((${myAssociativeArray[*]}))"); echo "Total disk usage: $(echo $total | tail -1 | awk {'print $1'} | awk '{ total = $1 / 1024/1024 ; printf("%.2fGB\n", total) }')"; unset total; unset disk_usage ;
How it works.
This could be anything you want to iterate through path list but in this example its just using the /etc/pass to loop over users paths source is here
for path in $(awk -F: '{if ($3 >= 1000) print $6}' < /etc/passwd)
It then calculates the usage per folder and extracts only digits from the output in the loop
disk_usage=0; disk_usage=$(du -s ${path} | grep -oE '[[:digit:]]+')
It outputs the nice formatting rounded to 2 decimal points
echo "$path: $(echo $disk_usage | tail -1 | awk {'print $1'} | awk '{ total = $1 / 1024/1024 ; printf("%.2fGB\n", total) }')";
Adds this to the bash associative array
myAssociativeArray[${path}]=${disk_usage}
then it sums the total value in the original amount from the array
total=$(IFS=+; echo "$((${myAssociativeArray[*]}))")
then we use the same fancy output formatting to show this nicely
echo "Total disk usage: $(echo $total | tail -1 | awk {'print $1'} | awk '{ total = $1 / 1024/1024 ; printf("%.2fGB\n", total) }')";
I used a variation of this for calculating cPanel Resellers accounts disk usage in the below monster oneliner.
Reseller="CPUsernameInputField"; declare -A myAssociativeArray ; echo "==========================================" | tee -a ${Reseller}_disk_breakdown.txt ; echo "Reseller ${Reseller}'s Disk usage by account"| tee -a ${Reseller}_disk_breakdown.txt; for acct in $(sudo grep ${Reseller} /etc/trueuserowners | cut -d: -f1); do disk_usage=0; disk_usage=$(du -s /home/${acct} | grep -oE '[[:digit:]]+'); echo "$acct: $(echo $disk_usage | tail -1 | awk {'print $1'} | awk '{ total = $1 / 1024/1024 ; printf("%.2fGB\n", total) }')" | tee -a ${Reseller}_disk_breakdown.txt ; myAssociativeArray[${acct}]=${disk_usage}; done ; total=$(IFS=+; echo "$((${myAssociativeArray[*]}))"); echo "Total disk usage: $(echo $total | tail -1 | awk {'print $1'} | awk '{ total = $1 / 1024/1024 ; printf("%.2fGB\n", total) }')" | tee -a ${Reseller}_disk_breakdown.txt; unset total; unset disk_usage;echo "==========================================" | tee -a ${Reseller}_disk_breakdown.txt ; echo "Sorted by top users" | tee -a ${Reseller}_disk_breakdown.txt; for key in "${!myAssociativeArray[#]}"; do printf '%s:%s\n' "$key" "${myAssociativeArray[$key]}"; done | sort -t : -k 2rn | tee -a ${Reseller}_disk_breakdown.txt;echo "==========================================" | tee -a ${Reseller}_disk_breakdown.txt ;for key in "${!myAssociativeArray[#]}"; do USER_HOME=$(eval echo ~${key}); echo "Disk breakdown for $key" | tee -a ${Reseller}_disk_breakdown.txt ; sudo du -h ${USER_HOME} --exclude=/app --exclude=/home/virtfs| grep ^[0-9.]*[G,M] | sort -rh|head -n20 | tee -a ${Reseller}_disk_breakdown.txt;echo "=======================================" | tee -a ${Reseller}_disk_breakdown.txt; done

Related

Print output of two commands in one line

I'm got this working:
while sleep 5s
do
lscpu | grep 'CPU MHz:' | cut -d ':' -f 2 | awk '{$1=$1};1' && grep 'cpu ' /proc/stat | awk '{usage=($2+$4)*100/($2+$4+$5)} END {print usage "%"}'
done
And it gives me the following output:
1601.058
3.4811%
1452.514
3.48059%
1993.800
3.48006%
2085.585
3.47955%
2757.776
3.47902%
1370.237
3.47851%
1497.903
3.47798%
But I'd really like to get the two values onto a single line. Every time I try to do this I run into a double / single quote variable issue. Granted I pulled some of this awk stuff from online so I'm not really up to speed on that. I just want to print per line, CPU clock and load ever 5 seconds.
Can you help me find a better way to do that?
You may use process substitution to run lscpu and cat /proc/stat and feed to single command. No need to use pipes.
while sleep 5; do
awk '/CPU MHz:/{printf "%s ", $NF} /cpu /{print ($2+$4)*100/($2+$4+$5)"%"}' <(lscpu) /proc/stat
done
If there is only one input command:
date| awk '{print $1}'
Wed
OR
awk '{print $NF}' <(date)
2019
If more then one command: Example , get the year of of the two date command in same line. (not very useful example, only for sake of demo)
awk '{printf "%s ", $1=NF}END{print ""}' <(date) <(date)
2019 2019
pipe the output of the 2 commands into paste
while sleep 5; do
lscpu | awk -F':[[:blank:]]+' '$1 == "CPU MHz" {print $2}'
awk '$1 == "cpu" {printf "%.4f%%\n", ($2+$4)*100/($2+$4+$5)}' /proc/stat
done | paste - -
The 2 columns will be separated by a tab.
Writing this for readability rather than efficiency, you might consider something like:
while sleep 5; do
cpu_pct=$(lscpu | awk -F': +' '/CPU MHz:/ { print $2 }')
usage=$(awk '/cpu / {usage=($2+$4)*100/($2+$4+$5)} END {print usage "%"}' /proc/stat)
printf '%s\n' "$cpu_pct $usage"
done
Command substitutions implicitly trim trailing newlines, so if lscpu | awk has output that ends in a newline, var=$(lscpu | awk) removes it; thereafter, you can use "$var" without that newline showing up.
All you need to do is change the newline on the first line to a different separator. Something like:
lscpu | ... | tr \\n : && grep ...
You can also use echo -n $(command_with_stdout). The -n switch specifies that the new line (\n) will be omitted.
while sleep 5s; do
echo -n $( lscpu | grep 'CPU MHz:' | cut -d ':' -f 2 | awk '{$1=$1};1' )
echo -n ' **** '
echo $( grep 'cpu ' /proc/stat | awk '{usage=($2+$4)*100/($2+$4+$5)} END {print usage "%"}' )
done
Or the same representation in one line:
while sleep 5s; do echo -n $( lscpu | grep 'CPU MHz:' | cut -d ':' -f 2 | awk '{$1=$1};1' ); echo -n ' **** '; echo $( grep 'cpu ' /proc/stat | awk '{usage=($2+$4)*100/($2+$4+$5)} END {print usage "%"}' ); done
EDIT: (remove -n switch from echo according to Charles Duffy's comment)
while sleep 5s; do echo "$( lscpu | grep 'CPU MHz:' | cut -d ':' -f 2 | awk '{$1=$1};1' ) **** $( grep 'cpu ' /proc/stat | awk '{usage=($2+$4)*100/($2+$4+$5)} END {print usage "%"}' )"; done

How to get the second word of a string in shell?

I want to get the size of the directory's content. If I use the command line I can get like this:
ls -l | head -n 1 | awk '{ print $2 }'
So the output is the size of the directory's content:
16816
But I want to do this in a bash script:
x="ls -l DBw | head -n 1 | awk '{ print $2 }'"
while sleep 1; do
y=$(eval "$x");
echo $y
done
But the output of this script is the full line:
Total 16816
Why is this happening and how can I get just the second word?
x="ls -l DBw | head -n 1 | awk '{ print $2 }'"
It's happening because $2 is inside double quotes and so it's interpreted immediately. You could escape it with \$2, but better yet:
Don't store code in variables. Use a function.
x() {
ls -l DBw | head -n 1 | awk '{ print $2 }'
}
Then you can call x many times.
while sleep 1; do
x
done
That said, it's better not to parse the output of ls in the first place. Use du to compute the size of a directory:
$ du -sh /dir
1.3M /dir
$ du -sh /dir | cut -f1
1.3M

Bash math - Dividing a bunch of rows using for statement

Example file:
25 Firstname1 Lastname1 domain1.com planname #1.00 USD Monthly Active 04/24/2016 Edit
1068 Firstname2 Lastname2 domain2.com planname #7.95 USD Annually Active 05/09/2016 Edit
3888 Firstname3 Lastname3 domain3.com planname #19.95 USD Biennially Active 05/04/2016 Edit
I am extracting just the price and billing cycle and am converting the billing cycles into numerical value this way I can divide the price by the billing cycle to get a cost per month.
When using the for statement, its adding line breaks which is breaking the math.
Code:
for i in `cat asd | cut -d "#" -f 2 | awk '{print $1, $3}' | sed 's/Monthly/\/ 1/g' | sed 's/Annually/\/ 12/g' | sed 's/Biennially/\/ 24/g' |grep -Ev 0.00` ; do echo $i | bc -l' ; done
I would prefer to be able to get 1 answer meaning all the rows get divided up then added together to get one final answer.
All those calls to cat, cut, awk, sed, grep and bc - what a waste.
This is a mis-named post, because you are not using Bash to do any calculations. The reason is that bash, unlike korn shell (ksh), does not support floating point. So you fall back to utilities like bc. Hold on though, awk supports floating point as well.
awk is a programming language in its own right. This just uses one instance of awk. I have embedded it inside a bash script because you are probably doing other stuff, but with a little adjustment it could be stand-alone with #!/bin/awk at the top:
infile='asd'
# -f - means "read the program from stdin"
# << '_END_' is a here document. Redirect stdin from here to the label _END_
awk -f - "$infile" << '_END_'
BEGIN {
# an associative array for the billing cycles
cycles["Monthly"] = 1
cycles["Annually"] = 12
cycles["Biennially"] = 24
}
{
sub(/#/,"",$6) # Remove the # from the amount
total += $6/cycles[$8] # divide amount by the billing cycle, add to total
}
END { print total }
_END_
Don't you think this is simpler to understand and maintain? It's also more efficient. This awk script is probably a good exercise for an awk 101 training course.
You could do something like this: (If you are totally set on a single line)
cat asd | cut -d "#" -f 2 | awk '{print $1, $3}' | sed 's/Monthly/\/ 1/g' | sed 's/Annually/\/ 12/g' | sed 's/Biennially/\/ 24/g' | grep -Ev 0.00 | while IFS= read -r line; do echo "$line" | bc -l; done | tr '\n' '+' | sed 's/+$/\n/' | bc -l
But this would be way more clear:
tmp=$(mktemp)
cat asd | cut -d "#" -f 2 | awk '{print $1, $3}' | sed 's/Monthly/\/ 1/g' | sed 's/Annually/\/ 12/g' | sed 's/Biennially/\/ 24/g' | grep -Ev 0.00 > $tmp
tmp2=$(mktemp)
cat $tmp | while IFS= read -r line; do
echo "$line" | bc -l >> $tmp2
done
# Actual output
cat $tmp2 | tr '\n' '+' | sed 's/+$/\n/' | bc -l
rm $tmp $tmp2

BASH, read first column in file in loop

Here is what I am trying to do. I would like to catch the top 10 cpu consuming PID's and find the program name. Then display the program name and % CPU in file.
CPU_per=$(sar 1 1 | tail -1 | awk '{print 100 - $5}')
echo $CPU_per
if [ $CPU_per -gt 80 ]
(prstat -u user -n 900 0 1 | grep Type | head -n 10 | awk '{print $1 " " $9}') >> /tmp/PID
for i in $(cat /tmp/PID)
do
(awk '{print $1 } | ps -p $PID -o args | tail -1 | cut -d \ -f 2)
I would like output to look like
Process %CPU
Program1 5%
Program2 9%
Program3 12%
Like this ?
echo -e "COMMAND\t\t%CPU"; ps -eo "%c %C%%" --sort pcpu | tail -n10

I don't want array to do word splitting

Below is my code. But bash is doing word splitting therefore I have failure. How to make my script so that there is no word splitting.
namaSensor=$(sensors | egrep "°C" | awk '{print $0}' | awk -F ':' '{print $1}')
for sensor in $namaSensor
do
if [ $(sensors | grep -c "$sensor") -ne 0 ]
then
currentTemperature=$(sensors | egrep "$sensor" | awk '{print $0}' | awk -F '+' '{print $2}' | cut -c 1-4 | awk -F '.' '{print $1}')
maxTemperature=$(sensors | egrep "$sensor" | awk '{print $0}' | awk -F '+' '{print $3}' | cut -c 1-4 | awk -F '.' '{print $1}')
if [ $currentTemperature -lt $maxTemperature ]
then
printf "current temperature is %d°C and the maximum allowed temperature is %d°C\n" "$currentTemperature" "$maxTemperature"
printf "temperature is within the maximum allowed temperature\n"
echo "$sensor"
else
printf "current temperature is %d°C and the maximum allowed temperature is %d°C\n" "$currentTemperature" "$maxTemperature"
printf "temperature is more than the maximum allowed temperature\n"
#exit 255
fi
fi
done.
This is the output of sensors for my unit.
acpitz-virtual-0
Adapter: Virtual device
temp1: +40.0°C (crit = +111.0°C)
temp2: +40.0°C (crit = +111.0°C)
coretemp-isa-0000
Adapter: ISA adapter
Physical id 0: +34.0°C (high = +87.0°C, crit = +105.0°C)
Core 0: +31.0°C (high = +87.0°C, crit = +105.0°C)
Core 1: +22.0°C (high = +87.0°C, crit = +105.0°C)
Please help
As I understand you have your array splitted as following:
Physical
id
0
and so on. To change this behaviour you need to change your Internal Field Separator (IFS) to a newline. So your code should look like this:
IFS=$'\n'
nameSensor=$(sensors | egrep "°C" | awk '{print $0}' | awk -F ':' '{print $1}')
for sensor in $nameSensor
do
if [ $(sensors | grep -c "$sensor") -ne 0 ]; then
currentTemperature=$(sensors | egrep "$sensor" | awk '{print $0}' | awk -F '+' '{print $2}' | cut -c 1-4 | awk -F '.' '{print $1}')
maxTemperature=$(sensors | egrep "$sensor" | awk '{print $0}' | awk -F '+' '{print $3}' | cut -c 1-4 | awk -F '.' '{print $1}')
if [ $currentTemperature -lt $maxTemperature ]; then
printf "current temperature is %d°C and the maximum allowed temperature is %d°C\n" "$currentTemperature" "$maxTemperature"
printf "temperature is within the maximum allowed temperature\n"
echo "$sensor"
else
printf "current temperature is %d°C and the maximum allowed temperature is %d°C\n" "$currentTemperature" "$maxTemperature"
printf "temperature is more than the maximum allowed temperature\n"
#exit 255
fi
fi
done

Resources