How to convert file size to human readable and print with other columns? - bash

Question

I want to convert the 5th column in this command output to human readable format.
For ex if this is my input :
-rw-rw-r-- 1 bhagyaraj bhagyaraj 280000 Jun 17 18:34 demo1
-rw-rw-r-- 1 bhagyaraj bhagyaraj 2800000 Jun 17 18:34 demo2
-rw-rw-r-- 1 bhagyaraj bhagyaraj 28000000 Jun 17 18:35 demo3
To something like this :
-rw-rw-r-- 280K demo1
-rw-rw-r-- 2.8M demo2
-rw-rw-r-- 28M demo3
I tried this command, but this will return only the file size column.
ls -l | tail -n +2 |awk '{print $5 | "numfmt --to=si"}'
ls is just for example my use case is very huge and repeated execution must be avoided
Any help would be appreciated :)

Answer 1

Just use -h --si
-h, --human-readable with -l and -s, print sizes like 1K 234M 2G etc.
--si likewise, but use powers of 1000 not 1024
So the command would be
ls -lh --si | tail -n +2
If you don't use ls and the command you intend to run doesn't have an option similar to -h --si in ls then numfmt already has the --field option to specify which column you want to format. For example
$ df | LC_ALL=en_US.UTF-8 numfmt --header --field 2-4 --to=si
Filesystem 1K-blocks Used Available Use% Mounted on
udev 66M 0 66M 0% /dev
tmpfs 14M 7.2K 14M 1% /run
/dev/mapper/vg0-lv--0 4.1G 3.7G 416M 90% /
tmpfs 5.2K 4 5.2K 1% /run/lock
/dev/nvme2n1p1 524K 5.4K 518K 2% /boot/efi
Unfortunately although numfmt does try to preserve the columnation, it fails if there are some large variation in the line length after inserting group separators like you can see above. So sometimes you might still need to reformat the table with column
df | LC_ALL=en_US.UTF-8 numfmt --header --field 2-4 --to=si | column -t -R 2,3,4,5
The -R 2,3,4,5 option is for right alignment, but some column versions like the default one in Ubuntu don't support it so you need to remove that
Alternatively you can also use awk to format only the columns you want, for example column 5 in case of ls
$ ls -l demo* | awk -v K=1e3 -v M=1e6 -v G=1e9 'func format(v) {
if (v > G) return v/G "G"; else if (v > M) return v/M "M";
else if (v > K) return v/K "K"; else return v
} { $5 = format($5); print $0 }' | column -t
-rw-rw-r-- 1 ph ph 280K Jun 18 09:23 demo1
-rw-rw-r-- 1 ph ph 2.8M Jun 18 09:24 demo2
-rw-rw-r-- 1 ph ph 28M Jun 18 09:23 demo3
-rw-rw-r-- 1 ph ph 2.8G Jun 18 09:30 demo4
And column 2, 3, 4 in case of df
# M=1000 and G=1000000 because df output is 1K-block, not bytes
$ df | awk -v M=1000 -v G=1000000 'func format(v) {
if (v > G) return v/G "G"; else if (v > M) return v/M "M"; else return v
}
{
# Format only columns 2, 3 and 4, ignore header
if (NR > 1) { $2 = format($2); $3 = format($3); $4 = format($4) }
print $0
}' OFS="\t" | column -t
Filesystem 1K-blocks Used Available Use% Mounted on
udev 65.8273G 0 65.8273G 0% /dev
tmpfs 13.1772G 7M 13.1702G 1% /run
/dev/mapper/vg0-lv--0 4073.78G 3619.05G 415.651G 90% /
tmpfs 65.8861G 0 65.8861G 0% /dev/shm
tmpfs 5.12M 4 5.116M 1% /run/lock
tmpfs 65.8861G 0 65.8861G 0% /sys/fs/cgroup
/dev/nvme2n1p2 999.32M 363.412M 567.096M 40% /boot

Answer 2

UPDATE 1 :
if you need just a barebones module for byte size formatting (it's setup for base-2 now but modifying it for --si should be trivial):
{m,g}awk '
BEGIN { OFS="="
_____=log(__=(_+=_+=_^=_)^(____=++_))
} gsub("[.]0+ bytes",
" -bytes-",
$(!__*($++NF = sprintf("%#10.*f %s" ,____,
(_ = $!!__) / __^(___=int(log(_)/_____)),
!___ ? "bytes" : substr("KMGTPEZY",___,!!__)"iB"))))^!__'
=
734 734 -bytes-
180043 175.82324 KiB
232819 227.36230 KiB
421548373 402.01986 MiB
838593829 799.74540 MiB
3739382399 3.48257 GiB
116601682159 108.59378 GiB
147480014471 137.35147 GiB
11010032230111 10.01357 TiB
19830700070261 18.03592 TiB
111120670776601 101.06366 TiB
15023323323323321 13.34339 PiB
85255542224555233 75.72213 PiB
444444666677766611 394.74616 PiB
106941916666944416909 92.75733 EiB
111919999919911191991 97.07513 EiB
767777766776776776777767 650.33306 ZiB
5558888858993555888686669 4.59821 YiB
========================
this is probably waaaay overkill, but I wrote it a while back, which can calculate the human-readable value, as well as comma formatted of raw byte value, supporting everything from kilo-bit to yotta-byte
with options for :
base 2 or base 10 (enter 10 or "M/m" for metric)
bytes (B) or bits (b)
The only thing that needs to be hard coded in are the letters themselves, since they grow linearly upon either
every 3rd power of 10 (1,000), or
every 5th power of 4 (1,024)
.
{m,g}awk '
BEGIN {
1 FS = OFS = "="
}
2302812 $!NF = substr(bytesformat($2, 10, "B"), 1, 15)\
substr(bytesformat($2, 2, "B"), 1, 15)\
bytesformat($2, 2, "b")'
# Functions, listed alphabetically
6908436 function bytesformat(_,_______,________,__, ___, ____, _____, ______)
{
6908436 _____=__=(____^=___*=((____=___+=___^= "")/___)+___+___)
6908436 ___/=___
6908436 sub("^0+","",_)
6908436 ____=_____-= substr(_____,index(_____,index(_____,!__))) * (_______~"^(10|[Mm])$")
6908436 _______=length((____)____)^(________~"^b(it)?$")
6908436 if ((____*__) < (_______*_)) { # 6906267
24438981 do {
24438981 ____*=_____
24438981 ++___
} while ((____*__) < (_______*_))
}
6908436 __=_
6908436 sub("(...)+$", ",&", __)
6908436 gsub("[^#-.][^#-.][^#-.]", "&,", __)
6908436 gsub("[,]*$|^[,]+", "", __)
6908436 sub("^[.]", "0&", __)
6908436 return \
sprintf("%10.4f %s%s | %s byte%.*s",
_=="" ? +_:_/(_____^___)*_______,
substr("KMGTPEZY", ___, _^(_<_)),
--_______?"b":"B",__==""?+__:__,(_^(_<_))<_,"s")
}
|
In this sample, it's showing metric bytes, binary bytes, binary bits, and raw input byte value :
180.0430 KB | 175.8232 KB | 1.3736 Mb | 180,043 bytes
232.8190 KB | 227.3623 KB | 1.7763 Mb | 232,819 bytes
421.5484 MB | 402.0199 MB | 3.1408 Gb | 421,548,373 bytes
838.5938 MB | 799.7454 MB | 6.2480 Gb | 838,593,829 bytes
3.7394 GB | 3.4826 GB | 27.8606 Gb | 3,739,382,399 bytes
116.6017 GB | 108.5938 GB | 868.7502 Gb | 116,601,682,159 bytes
147.4800 GB | 137.3515 GB | 1.0731 Tb | 147,480,014,471 bytes
11.0100 TB | 10.0136 TB | 80.1085 Tb | 11,010,032,230,111 bytes
19.8307 TB | 18.0359 TB | 144.2873 Tb | 19,830,700,070,261 bytes
111.1207 TB | 101.0637 TB | 808.5093 Tb | 111,120,670,776,601 bytes
15.0233 PB | 13.3434 PB | 106.7471 Pb | 15,023,323,323,323,321 bytes
85.2555 PB | 75.7221 PB | 605.7771 Pb | 85,255,542,224,555,233 bytes
444.4447 PB | 394.7462 PB | 3.0840 Eb | 444,444,666,677,766,611 bytes
106.9419 EB | 92.7573 EB | 742.0586 Eb | 106,941,916,666,944,416,909 bytes
111.9200 EB | 97.0751 EB | 776.6010 Eb | 111,919,999,919,911,191,991 bytes
767.7778 ZB | 650.3331 ZB | 5.0807 Yb | 767,777,766,776,776,776,777,767 bytes
5.5589 YB | 4.5982 YB | 36.7856 Yb | 5,558,888,858,993,555,888,686,669 bytes