Multiple arithmetic operation in awk - shell
I have this awk function to do speed conversion.
function hum(x) {
split( "B/s KB/s MB/s GB/s TB/s", v); s=1
while( x>1000 ){ x/=1000; s++ }
printf( "%0.2f %s" , x , v[s] )
}
hum($1)
It works great if used standalone.
$ awk -f /tmp/test.awk <<< 1000000
1000.00 KB/s
$ awk -f /tmp/test.awk <<< 100000000
100.00 MB/s
Now, I want to convert bytes number to bit, to do that I need to multiple the number/input first by 8. I tried to modify the function a bit.
function hum(x) {
split( "B/s KB/s MB/s GB/s TB/s", v); s=1
while( x>1000 ){ res = x * 8 / 1000; s++ }
printf( "%0.2f %s\n" , res , v[s] )
}
hum($1)
But it hangs when I tried to execute it, I had to Ctrl-C to cancel the operation. Any idea what's wrong ?
Now, I want to convert bytes number to bit, to do that I need to multiple the number/input first by 8.
Then multiply the input value by 8 just once:
function hum(x) {
split("b/s Kb/s Mb/s Gb/s Tb/s", v); s=1
x *= 8
while(x > 1000) {x /= 1000; s++}
printf("%0.2f %s\n", x, v[s])
}
hum($1)
Instead of a loop, let's use log():
$ cat program.awk
function hum2(x, v,p) {
split( "b/s kb/s Mb/s Gb/s Tb/s", v) # remember to B -> b
x*=8 # to bits conversion
p=int(log(x)/log(1000)) # figure v slot
return sprintf("%0.2f %s" , x/1000^p , v[p+1] ) # p+1 due to 1 basedness
}
{
print $1,hum2($1) # input bytes, output bits
}
Take it for a spin:
$ for (( i=1 ; i<=10**14 ; i=i*10 )) ; do echo $i ; done | awk -f program.awk
Output:
1 8.00 b/s
10 80.00 b/s
100 800.00 b/s
1000 8.00 kb/s
10000 80.00 kb/s
100000 800.00 kb/s
1000000 8.00 Mb/s
10000000 80.00 Mb/s
100000000 800.00 Mb/s
1000000000 8.00 Gb/s
10000000000 80.00 Gb/s
100000000000 800.00 Gb/s
1000000000000 8.00 Tb/s
10000000000000 80.00 Tb/s
100000000000000 800.00 Tb/s
I wrote this one a while back that might not be exactly what you need but should be easily adaptable -
a function that takes in # of bytes as mandatory 1st parameter,
with optional 2nd param for calculating powers of 1000 instead of 1024
enter M m or 10 for 1000,
anything else, including blank, defaults to 1024
and auto formats it in the most logical range for human readability,
spanning all the way from kilo(kibi)-bit/s,
to YOTTA(yobi)-bit/s
*** extra credit for anyone who can figure out what's common among that list of numbers in my sample below.
——————————————————————— ———————————————————————
mawk/gawk '
function bytesformat(_, _______, __, ___, ____, _____, ______)
{
_____=__=(____^=___*=((____=___+=___^="")/___)+___+___)
___/=___
sub("^0+","",_)
____=_____-=substr(_____,index(_____,index(_____,!__)))*\
(_______ ~ "^(10|[Mm])$")
_______=length((____)____)
if ((____*__)<(_______*_)) {
do {
____ *= _____
++___
} while ((____*__)<(_______*_))
}
__=_
sub("(...)+$", ",&", __)
gsub("[^#-.][^#-.][^#-.]", "&,", __)
gsub("[,]*$|^[,]+", "", __)
sub("^[.]", "0&", __)
return \
sprintf(" %10.4f %sb/s (%42s byte%s) ",
_==""?+_:_/(_____^___)*_______,
substr("KMGTPEZY",___,___~___),
__==""?+__:__, (_~_)<_?"s":" ")
} { printf("%35s bytes ::: %s\n",
$1,
bytesformat($1, 10)) }'
6841 bytes ::: 54.7280 Kb/s ( 6,841 bytes)
15053 bytes ::: 120.4240 Kb/s ( 15,053 bytes)
23677 bytes ::: 189.4160 Kb/s ( 23,677 bytes)
32839 bytes ::: 262.7120 Kb/s ( 32,839 bytes)
42293 bytes ::: 338.3440 Kb/s ( 42,293 bytes)
52183 bytes ::: 417.4640 Kb/s ( 52,183 bytes)
62233 bytes ::: 497.8640 Kb/s ( 62,233 bytes)
72733 bytes ::: 581.8640 Kb/s ( 72,733 bytes)
83269 bytes ::: 666.1520 Kb/s ( 83,269 bytes)
138641 bytes ::: 1.1091 Mb/s ( 138,641 bytes)
149767 bytes ::: 1.1981 Mb/s ( 149,767 bytes)
162011 bytes ::: 1.2961 Mb/s ( 162,011 bytes)
174221 bytes ::: 1.3938 Mb/s ( 174,221 bytes)
186343 bytes ::: 1.4907 Mb/s ( 186,343 bytes)
199181 bytes ::: 1.5934 Mb/s ( 199,181 bytes)
211559 bytes ::: 1.6925 Mb/s ( 211,559 bytes)
224449 bytes ::: 1.7956 Mb/s ( 224,449 bytes)
237733 bytes ::: 1.9019 Mb/s ( 237,733 bytes)
128260807 bytes ::: 1.0261 Gb/s ( 128,260,807 bytes)
128565049 bytes ::: 1.0285 Gb/s ( 128,565,049 bytes)
128932561 bytes ::: 1.0315 Gb/s ( 128,932,561 bytes)
129304523 bytes ::: 1.0344 Gb/s ( 129,304,523 bytes)
129765859 bytes ::: 1.0381 Gb/s ( 129,765,859 bytes)
130111459 bytes ::: 1.0409 Gb/s ( 130,111,459 bytes)
130533133 bytes ::: 1.0443 Gb/s ( 130,533,133 bytes)
131012801 bytes ::: 1.0481 Gb/s ( 131,012,801 bytes)
131305043 bytes ::: 1.0504 Gb/s ( 131,305,043 bytes)
128004093619 bytes ::: 1.0240 Tb/s ( 128,004,093,619 bytes)
128026268633 bytes ::: 1.0242 Tb/s ( 128,026,268,633 bytes)
128056111093 bytes ::: 1.0244 Tb/s ( 128,056,111,093 bytes)
128071706179 bytes ::: 1.0246 Tb/s ( 128,071,706,179 bytes)
128082430067 bytes ::: 1.0247 Tb/s ( 128,082,430,067 bytes)
128102475287 bytes ::: 1.0248 Tb/s ( 128,102,475,287 bytes)
128115312811 bytes ::: 1.0249 Tb/s ( 128,115,312,811 bytes)
128157555781 bytes ::: 1.0253 Tb/s ( 128,157,555,781 bytes)
128175556181 bytes ::: 1.0254 Tb/s ( 128,175,556,181 bytes)
128004004377827 bytes ::: 1.0240 Pb/s ( 128,004,004,377,827 bytes)
128040044659991 bytes ::: 1.0243 Pb/s ( 128,040,044,659,991 bytes)
128074066014953 bytes ::: 1.0246 Pb/s ( 128,074,066,014,953 bytes)
128127783733093 bytes ::: 1.0250 Pb/s ( 128,127,783,733,093 bytes)
128177777757611 bytes ::: 1.0254 Pb/s ( 128,177,777,757,611 bytes)
128200131001829 bytes ::: 1.0256 Pb/s ( 128,200,131,001,829 bytes)
128221782218423 bytes ::: 1.0258 Pb/s ( 128,221,782,218,423 bytes)
128237784424429 bytes ::: 1.0259 Pb/s ( 128,237,784,424,429 bytes)
128262808216561 bytes ::: 1.0261 Pb/s ( 128,262,808,216,561 bytes)
128055360778053559 bytes ::: 1.0244 Eb/s ( 128,055,360,778,053,559 bytes)
128082834342828077 bytes ::: 1.0247 Eb/s ( 128,082,834,342,828,077 bytes)
128112814740831073 bytes ::: 1.0249 Eb/s ( 128,112,814,740,831,073 bytes)
128172605482718161 bytes ::: 1.0254 Eb/s ( 128,172,605,482,718,161 bytes)
128203333333333399 bytes ::: 1.0256 Eb/s ( 128,203,333,333,333,399 bytes)
128240343634404269 bytes ::: 1.0259 Eb/s ( 128,240,343,634,404,269 bytes)
128272818280928081 bytes ::: 1.0262 Eb/s ( 128,272,818,280,928,081 bytes)
128282816070718271 bytes ::: 1.0263 Eb/s ( 128,282,816,070,718,271 bytes)
128289494449498271 bytes ::: 1.0263 Eb/s ( 128,289,494,449,498,271 bytes)
128030578058078030329 bytes ::: 1.0242 Zb/s ( 128,030,578,058,078,030,329 bytes)
128172161171772727271 bytes ::: 1.0254 Zb/s ( 128,172,161,171,772,727,271 bytes)
128234814212823481421 bytes ::: 1.0259 Zb/s ( 128,234,814,212,823,481,421 bytes)
128282727262616060507 bytes ::: 1.0263 Zb/s ( 128,282,727,262,616,060,507 bytes)
128286164949865319531 bytes ::: 1.0263 Zb/s ( 128,286,164,949,865,319,531 bytes)
128372737272827373721 bytes ::: 1.0270 Zb/s ( 128,372,737,272,827,373,721 bytes)
128393838393839382839 bytes ::: 1.0272 Zb/s ( 128,393,838,393,839,382,839 bytes)
128505500051850550037 bytes ::: 1.0280 Zb/s ( 128,505,500,051,850,550,037 bytes)
128669659758768758857 bytes ::: 1.0294 Zb/s ( 128,669,659,758,768,758,857 bytes)
130000000000093999992023 bytes ::: 1.0400 Yb/s ( 130,000,000,000,093,999,992,023 bytes)
131111111311113111311131 bytes ::: 1.0489 Yb/s ( 131,111,111,311,113,111,311,131 bytes)
131111153353153531553111 bytes ::: 1.0489 Yb/s ( 131,111,153,353,153,531,553,111 bytes)
131111531315333335313531 bytes ::: 1.0489 Yb/s ( 131,111,531,315,333,335,313,531 bytes)
131113133333311333331111 bytes ::: 1.0489 Yb/s ( 131,113,133,333,311,333,331,111 bytes)
131113551355135511111111 bytes ::: 1.0489 Yb/s ( 131,113,551,355,135,511,111,111 bytes)
131131113131113131131111 bytes ::: 1.0490 Yb/s ( 131,131,113,131,113,131,131,111 bytes)
131131331133111331313111 bytes ::: 1.0491 Yb/s ( 131,131,331,133,111,331,313,111 bytes)
131133133131333313331311 bytes ::: 1.0491 Yb/s ( 131,133,133,131,333,313,331,311 bytes)
Any idea what's wrong ?
You have created infinte loop which cause it to hang, 1st code has
while( x>1000 ){ x/=1000; s++ }
here each turn decrease x 1000 times so it will finally be equal or lower 1000. 2nd code has
while( x>1000 ){ res = x * 8 / 1000; s++ }
here each turn x is used to compute res but value of x itself never changes, thus you will get either 0 executions of while body or +infinity executions of while body.
Note that But it hangs when I tried to execute it might be observed also in languages other than GNU AWK which do support while construct when you do not change value of variable on which truthiness of condition depends.
Related
bash process substitution with tee dropping data
I expect to see 1MiB in the wc's character count (which works at the bottom when 'tee' is writing to a file rather than a process substitution). Is there something I'm missing about using process substitution with tee? [192.168.20.40 (02b4b472) ~ 18:52:39]# dd if=/dev/zero bs=1M count=1 | tee >(dd bs=1M count=1 | wc | sed 's/^/ /' > /dev/stderr) | wc 0+1 records in 0+1 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.00022373 s, 36.6 MB/s 0 0 8192 0 0 81920 [192.168.20.40 (02b4b472) ~ 18:52:56]# dd if=/dev/zero bs=1M count=1 | tee /tmp/foo | wc 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0416881 s, 25.2 MB/s 0 0 1048576 [192.168.20.40 (02b4b472) ~ 18:53:41]# Using tee -p handles the stdout, but not the process substitution... which still comes up short. [192.168.20.40 (02b4b472) ~ 19:17:05]# dd if=/dev/zero bs=1M count=1 | tee -p warn >(dd bs=1M count=1 | wc | sed 's/^/ /' > /dev/stderr) | wc 0+1 records in 0+1 records out 8192 bytes (8.2 kB, 8.0 KiB) copied, 0.00084348 s, 9.7 MB/s 0 0 8192 1+0 records in 1+0 records out 1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0417175 s, 25.1 MB/s 0 0 1048576 [192.168.20.40 (02b4b472) ~ 19:18:25]#
Bash sed awk, format CPU/Mem info from /proc/cpuinfo and /proc/meminfo [closed]
Closed. This question needs details or clarity. It is not currently accepting answers. Want to improve this question? Add details and clarify the problem by editing this post. Closed 2 years ago. Improve this question The problem that I'm trying to solve is to produce portable output that I can display on all of the servers in our environment to show basic info at login using generic information on all CentOS / Red Hat systems. I would like to pluck info from /proc/cpuinfo and /proc/meminfo (or free -m -h); "why not just 'yum install some-great-tool'?" is not ideal as all of this information is freely available to us right in /proc. I know that this sort of thing can often be a very simple trick for sed/awk experts (I don't know how to approach this with my limited sed/awk knowledge). I would like to extract something like the following on a single line: <model name>, <cpu MHz> MHz, <cpu cores> cores, <detect "vmx" (Intel-VT) or "svm" (AMD-V support)> e.g. with the below output, this would look like (with "1300.000" rounded to "1300") "AMD Athlon(tm) II Neo N36L Dual-Core Processor, 1300 MHz, 2 cores, VMX-Virtualization" (or "SVM-Virtualization" or "No Virtualization") I would like to also combine this info with that of /proc/meminfo or free -mh, so: "AMD Athlon(tm) II Neo N36L Dual-Core Processor, 1300 MHz, 2 cores, 4.7 GB Memory (1.8 GB Free), SVM-Virtualization" I have spent some time searching for methods, but without luck, and maybe this is an interesting generic problem as involves taking the format of tables that a lot of info is held in and extracting as required so has some generic application. $ free -m -h total used free shared buff/cache available Mem: 4.5Gi 1.2Gi 1.8Gi 77Mi 1.6Gi 3.0Gi Swap: 4.8Gi 0B 4.8Gi $ cat /proc/cpuinfo processor : 0 vendor_id : AuthenticAMD cpu family : 16 model : 6 model name : AMD Athlon(tm) II Neo N36L Dual-Core Processor stepping : 3 microcode : 0x10000c8 cpu MHz : 1300.000 cache size : 1024 KB physical id : 0 siblings : 2 core id : 0 cpu cores : 2 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 5 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm 3dnowext 3dnow constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a 3dnowprefetch osvw ibs skinit wdt nodeid_msr hw_pstate vmmcall npt lbrv svm_lock nrip_save bugs : tlb_mmatch apic_c1e fxsave_leak sysret_ss_attrs null_seg amd_e400 spectre_v1 spectre_v2 bogomips : 2595.59 TLB size : 1024 4K pages clflush size : 64 cache_alignment : 64 address sizes : 48 bits physical, 48 bits virtual power management: ts ttp tm stc 100mhzsteps hwpstate $ cat /proc/meminfo MemTotal: 4771304 kB MemFree: 1862372 kB MemAvailable: 3195768 kB Buffers: 2628 kB Cached: 1542788 kB SwapCached: 0 kB Active: 1534572 kB Inactive: 909316 kB Active(anon): 917792 kB Inactive(anon): 62468 kB Active(file): 616780 kB Inactive(file): 846848 kB Unevictable: 8384 kB Mlocked: 0 kB SwapTotal: 5070844 kB SwapFree: 5070844 kB Dirty: 20 kB Writeback: 0 kB AnonPages: 881304 kB Mapped: 395420 kB Shmem: 79776 kB KReclaimable: 152892 kB Slab: 295508 kB SReclaimable: 152892 kB SUnreclaim: 142616 kB KernelStack: 9328 kB PageTables: 45156 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 7456496 kB Committed_AS: 5260708 kB VmallocTotal: 34359738367 kB VmallocUsed: 0 kB VmallocChunk: 0 kB Percpu: 2864 kB HardwareCorrupted: 0 kB AnonHugePages: 417792 kB ShmemHugePages: 0 kB ShmemPmdMapped: 0 kB HugePages_Total: 0 HugePages_Free: 0 HugePages_Rsvd: 0 HugePages_Surp: 0 Hugepagesize: 2048 kB Hugetlb: 0 kB DirectMap4k: 314944 kB DirectMap2M: 4796416 kB DirectMap1G: 0 kB
Using /proc/cpuinfo and free -mh along with awk, search for the strings required, using : as the field delimited, set variables accordingly, splitting the output of free -mh further into an array called arr based on " " as the delimiter. At the end, print the data in the required format using the variables created. When searching for lines beginning with flag, we search for strings svn or vmx using awk's match function. A match will signified by the RSTART variable not being 0 and so we check this to find the type of virtualisatiion being utilised. As we have set virt to No Virtualisation at the beginning, no matches will print No Virtualisation. awk -F: '/^model name/ { mod=$2 } /^cpu MHz/ { mhz=$2 } /^cpu core/ { core=$2 } /^flags/ { virt="No Virtualisation"; match($0,"svm"); if (RSTART!=0) { virt="SVM-Virtualisation" }; match($0,"vmx"); if (RSTART!=0) { virt="VMX-Virtualisation" } } /^Mem:/ { split($2,arr," "); tot=arr[1]; free=arr[2] } END { printf "%s %dMHz %s core(s) %s %sB Memory (%sB Free)\n",mod,mhz,core,virt,tot,free }' /proc/cpuinfo <(free -mh) One liner: awk -F: '/^model name/ { mod=$2 } /^cpu MHz/ { mhz=$2 } /^cpu core/ {core=$2} /^flags/ { virt="No Virtualisation";match($0,"svm");if (RSTART!=0) { virt="SVM-Virtualisation" };match($0,"vmx");if (RSTART!=0) { virt="VMX-Virtualisation" } } /^Mem:/ {split($2,arr," ");tot=arr[1];free=arr[2]} END { printf "%s %dMHz %s core(s) %s %sB Memory (%sB Free)\n",mod,mhz,core,virt,tot,free }' /proc/cpuinfo <(free -mh)
Memory is not even getting freed (back to OS)
I'm running a server written in Go and the RSS is very high and the memory is not even getting freed (back to OS). I used pprof to check but it seems that there is no memory leak. I also tried: GODEBUG=madvdontneed=1 ./memorytest Please tell me how to use madvdontneed. OS: CentOS 7 (Linux) Arch: amd64 Go version: 1.14.2 Code: package main import ( "fmt" "os" "runtime" "time" ) var garr []string var chn chan int func main() { chn = make(chan int, 1) go Alloc() <-chn } func Alloc() { for { arr := make([]string, 100000000) //copy(garr,arr) garr = arr print_heap_info() time.Sleep(5 * time.Second) } } func print_heap_info() { var m runtime.MemStats runtime.ReadMemStats(&m) fmt.Printf("env:%v, heapsys:%d,heapalloc:%d,heapidel:%d,heapreleased:%d,heapinuse:%d\n", os.Getenv("GODEBUG"), m.HeapSys, m.HeapAlloc, m.HeapIdle, m.HeapReleased, m.HeapInuse) } Output: env:madvdontneed=1, heapsys:1677262848,heapalloc:1600074304,heapidel:76955648,heapreleased:76914688,heapinuse:1600307200 env:madvdontneed=1, heapsys:3287875584,heapalloc:3200081392,heapidel:87556096,heapreleased:87506944,heapinuse:3200319488 env:madvdontneed=1, heapsys:4898619392,heapalloc:4800086512,heapidel:98295808,heapreleased:98115584,heapinuse:4800323584 env:madvdontneed=1, heapsys:6509101056,heapalloc:6400090640,heapidel:108773376,heapreleased:108724224,heapinuse:6400327680 env:madvdontneed=1, heapsys:6509232128,heapalloc:4800086176,heapidel:1708908544,heapreleased:108724224,heapinuse:4800323584 env:madvdontneed=1, heapsys:6509002752,heapalloc:6400090304,heapidel:108675072,heapreleased:108560384,heapinuse:6400327680 env:madvdontneed=1, heapsys:6509199360,heapalloc:4800086712,heapidel:1708875776,heapreleased:108560384,heapinuse:4800323584 env:madvdontneed=1, heapsys:6509068288,heapalloc:6400090744,heapidel:108740608,heapreleased:108560384,heapinuse:6400327680 env:madvdontneed=1, heapsys:6509199360,heapalloc:4800086712,heapidel:1708875776,heapreleased:108560384,heapinuse:4800323584 env:madvdontneed=1, heapsys:6509068288,heapalloc:6400090840,heapidel:108740608,heapreleased:108462080,heapinuse:6400327680
Optimize your memory allocations to lower the peak. See also: debug.FreeOSMemory() FreeOSMemory forces a garbage collection followed by an attempt to return as much memory to the operating system as possible. (Even if this is not called, the runtime gradually returns memory to the operating system in a background task.) Your formatted output is: sys: 1599 MB alloc: 1525 MB idel: 73 MB released: 73 MB inuse: 1526 MB sys: 3135 MB alloc: 3051 MB idel: 83 MB released: 83 MB inuse: 3052 MB sys: 4671 MB alloc: 4577 MB idel: 93 MB released: 93 MB inuse: 4577 MB sys: 6207 MB alloc: 6103 MB idel: 103 MB released: 103 MB inuse: 6103 MB sys: 6207 MB alloc: 4577 MB idel: 1629 MB released: 103 MB inuse: 4577 MB sys: 6207 MB alloc: 6103 MB idel: 103 MB released: 103 MB inuse: 6103 MB sys: 6207 MB alloc: 4577 MB idel: 1629 MB released: 103 MB inuse: 4577 MB sys: 6207 MB alloc: 6103 MB idel: 103 MB released: 103 MB inuse: 6103 MB sys: 6207 MB alloc: 4577 MB idel: 1629 MB released: 103 MB inuse: 4577 MB sys: 6207 MB alloc: 6103 MB idel: 103 MB released: 103 MB inuse: 6103 MB Which is (for the last line): sys: bytes of heap memory obtained from the OS: 6207 MB alloc: bytes of allocated heap objects: 6103 MB idel: bytes in idle (unused) spans: 103 MB released: bytes of physical memory returned to the OS: 103 MB inuse: bytes in in-use spans: 6103 MB It is not a memory leak. Output for a system with total 8GB RAM (system memory monitor): command: GODEBUG=madvdontneed=1 go run . Output: env: madvdontneed=1, sys: 1087 MB, alloc: 1024 MB, idel: 63 MB, released: 63 MB, inuse: 1024 MB env: madvdontneed=1, sys: 2111 MB, alloc: 2048 MB, idel: 63 MB, released: 63 MB, inuse: 2048 MB env: madvdontneed=1, sys: 3135 MB, alloc: 3072 MB, idel: 63 MB, released: 63 MB, inuse: 3072 MB env: madvdontneed=1, sys: 4159 MB, alloc: 4096 MB, idel: 63 MB, released: 63 MB, inuse: 4096 MB env: madvdontneed=1, sys: 4159 MB, alloc: 3072 MB, idel: 1087 MB, released: 63 MB, inuse: 3072 MB env: madvdontneed=1, sys: 4159 MB, alloc: 4096 MB, idel: 63 MB, released: 63 MB, inuse: 4096 MB env: madvdontneed=1, sys: 4159 MB, alloc: 3072 MB, idel: 1087 MB, released: 63 MB, inuse: 3072 MB env: madvdontneed=1, sys: 4159 MB, alloc: 4096 MB, idel: 63 MB, released: 63 MB, inuse: 4096 MB env: madvdontneed=1, sys: 4159 MB, alloc: 3072 MB, idel: 1087 MB, released: 63 MB, inuse: 3072 MB env: madvdontneed=1, sys: 4159 MB, alloc: 4096 MB, idel: 63 MB, released: 63 MB, inuse: 4096 MB Code: package main import ( "fmt" "os" "runtime" "time" ) func main() { for i := 0; i < 10; i++ { a = make([]byte, 1024*meg) var m runtime.MemStats runtime.ReadMemStats(&m) fmt.Printf("env: %v, sys: %4d MB, alloc: %4d MB, idel: %4d MB, released: %4d MB, inuse: %4d MB\n", os.Getenv("GODEBUG"), m.HeapSys/meg, m.HeapAlloc/meg, m.HeapIdle/meg, m.HeapReleased/meg, m.HeapInuse/meg) time.Sleep(1 * time.Second) } } var a []byte const meg = 1024 * 1024 htop: Output for vmstat 1 -S MB command, running two terminals: procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu----- r b swpd free buff cache si so bi bo in cs us sy id wa st 0 0 1161 6369 25 595 0 0 702 452 276 567 6 3 91 1 0 3 0 1160 6300 28 632 0 0 43432 0 6288 14013 13 5 82 1 0 1 0 1160 6243 28 634 0 0 528 0 6342 13008 11 3 85 0 0 1 0 1160 6177 28 634 0 0 56 56 979 2090 4 1 95 0 0 2 0 1160 6110 28 634 0 0 0 0 1061 2664 5 2 94 0 0 0 0 1160 6044 28 634 0 0 12 36 994 2233 4 1 94 0 0 0 0 1160 3991 28 634 0 0 32 188 1074 1787 5 6 89 0 0 2 0 1160 2120 28 634 0 0 0 136 1016 1634 4 5 90 0 0 1 0 1160 973 28 633 0 0 0 4 1077 1660 4 5 92 0 0 2 0 1160 143 25 390 0 0 1780 356 1849 2341 4 9 87 0 0 0 9 1323 102 0 108 0 165 51964 169652 20277 21017 1 20 53 25 0 1 5 1510 99 0 129 2 189 54376 193996 57829 52152 1 16 56 27 0 1 5 1794 99 0 129 1 286 10068 293856 81160 59511 0 16 77 6 0 4 5 2047 101 0 98 5 257 21236 263292 69923 65485 0 23 54 23 0 1 2 1479 2867 0 217 43 23 233508 24380 24023 48536 4 27 47 22 0 2 0 1452 2824 0 232 27 0 43960 0 8168 21085 4 5 90 1 0 0 1 1443 2814 0 233 9 0 10956 0 3341 8468 5 2 93 0 0 3 0 1425 2796 0 231 18 0 19688 0 5780 15490 4 3 92 1 0 1 1 1420 2672 10 337 3 0 121628 1920 3292 7934 5 7 81 7 0 0 0 1394 2646 10 338 25 0 27360 0 7975 21555 3 5 92 0 0 0 1 1359 6856 10 339 1 0 2416 0 1035 2108 3 2 95 0 0 0 0 1353 6847 10 348 4 0 13660 0 1696 3471 4 1 95 0 0 Output for first GODEBUG=madvdontneed=1 go run . command (auto killed): env: madvdontneed=1, sys: 1087 MB, alloc: 1024 MB, idel: 63 MB, released: 63 MB, inuse: 1024 MB env: madvdontneed=1, sys: 2111 MB, alloc: 2048 MB, idel: 63 MB, released: 63 MB, inuse: 2048 MB env: madvdontneed=1, sys: 3135 MB, alloc: 3072 MB, idel: 63 MB, released: 63 MB, inuse: 3072 MB env: madvdontneed=1, sys: 4159 MB, alloc: 4096 MB, idel: 63 MB, released: 63 MB, inuse: 4096 MB env: madvdontneed=1, sys: 4159 MB, alloc: 3072 MB, idel: 1087 MB, released: 63 MB, inuse: 3072 MB env: madvdontneed=1, sys: 4159 MB, alloc: 4096 MB, idel: 63 MB, released: 63 MB, inuse: 4096 MB env: madvdontneed=1, sys: 4159 MB, alloc: 3072 MB, idel: 1087 MB, released: 63 MB, inuse: 3072 MB signal: killed Output for second GODEBUG=madvdontneed=1 go run . command: env: madvdontneed=1, sys: 1087 MB, alloc: 1024 MB, idel: 63 MB, released: 63 MB, inuse: 1024 MB env: madvdontneed=1, sys: 2111 MB, alloc: 2048 MB, idel: 63 MB, released: 63 MB, inuse: 2048 MB env: madvdontneed=1, sys: 3135 MB, alloc: 3072 MB, idel: 63 MB, released: 63 MB, inuse: 3072 MB env: madvdontneed=1, sys: 4159 MB, alloc: 4096 MB, idel: 63 MB, released: 63 MB, inuse: 4096 MB env: madvdontneed=1, sys: 4159 MB, alloc: 3072 MB, idel: 1087 MB, released: 63 MB, inuse: 3072 MB env: madvdontneed=1, sys: 4159 MB, alloc: 4096 MB, idel: 63 MB, released: 63 MB, inuse: 4096 MB env: madvdontneed=1, sys: 4159 MB, alloc: 3072 MB, idel: 1087 MB, released: 63 MB, inuse: 3072 MB env: madvdontneed=1, sys: 4159 MB, alloc: 4096 MB, idel: 63 MB, released: 62 MB, inuse: 4096 MB env: madvdontneed=1, sys: 4159 MB, alloc: 3072 MB, idel: 1087 MB, released: 62 MB, inuse: 3072 MB env: madvdontneed=1, sys: 4159 MB, alloc: 4096 MB, idel: 63 MB, released: 62 MB, inuse: 4096 MB Code: package main import ( "fmt" "os" "runtime" "time" ) func main() { for i := 0; i < 10; i++ { a = make([]byte, 1024*meg) var m runtime.MemStats runtime.ReadMemStats(&m) fmt.Printf("env: %v, sys: %4d MB, alloc: %4d MB, idel: %4d MB, released: %4d MB, inuse: %4d MB\n\n", os.Getenv("GODEBUG"), m.HeapSys/meg, m.HeapAlloc/meg, m.HeapIdle/meg, m.HeapReleased/meg, m.HeapInuse/meg) time.Sleep(1 * time.Second) } } var a []byte const meg = 1024 * 1024 Command: GODEBUG=gctrace=1 go run . Output: gc 1 #0.008s 2%: 0.071+0.67+0.034 ms clock, 0.57+0.88/0.76/0.041+0.27 ms cpu, 4->4->0 MB, 5 MB goal, 8 P gc 2 #0.014s 3%: 0.069+1.3+0.027 ms clock, 0.55+0.48/0.80/0.73+0.21 ms cpu, 4->4->0 MB, 5 MB goal, 8 P gc 3 #0.029s 2%: 0.056+0.62+0.040 ms clock, 0.45+0.39/0.75/0.67+0.32 ms cpu, 4->4->0 MB, 5 MB goal, 8 P gc 4 #0.045s 3%: 0.31+0.97+0.12 ms clock, 2.5+0.91/1.2/0.92+1.0 ms cpu, 4->4->0 MB, 5 MB goal, 8 P gc 5 #0.060s 2%: 0.056+0.60+0.019 ms clock, 0.45+0.44/0.70/1.1+0.15 ms cpu, 4->4->0 MB, 5 MB goal, 8 P gc 6 #0.071s 2%: 0.025+1.0+0.018 ms clock, 0.20+0.47/1.1/3.5+0.15 ms cpu, 4->4->0 MB, 5 MB goal, 8 P gc 7 #0.084s 2%: 0.12+0.88+0.042 ms clock, 0.97+0.77/1.2/0.88+0.33 ms cpu, 4->4->1 MB, 5 MB goal, 8 P gc 8 #0.093s 2%: 0.039+0.83+0.028 ms clock, 0.31+0.38/0.66/1.2+0.22 ms cpu, 4->4->0 MB, 5 MB goal, 8 P gc 1 #0.007s 3%: 0.013+1.7+0.005 ms clock, 0.11+0.86/2.0/1.2+0.042 ms cpu, 4->5->4 MB, 5 MB goal, 8 P gc 1 #0.002s 5%: 0.024+2.0+0.042 ms clock, 0.19+0.25/1.5/1.3+0.34 ms cpu, 4->6->5 MB, 5 MB goal, 8 P gc 2 #0.017s 3%: 0.014+4.6+0.044 ms clock, 0.11+0.13/3.3/1.9+0.35 ms cpu, 9->10->7 MB, 10 MB goal, 8 P gc 3 #0.058s 2%: 0.030+6.7+0.036 ms clock, 0.24+0.12/5.3/1.8+0.29 ms cpu, 13->15->10 MB, 15 MB goal, 8 P gc 4 #0.100s 2%: 0.034+5.6+0.015 ms clock, 0.27+0/7.2/0.65+0.12 ms cpu, 18->18->12 MB, 21 MB goal, 8 P gc env: gctrace=1, sys: 1087 MB, alloc: 1024 MB, idel: 63 MB, released: 63 MB, inuse: 1024 MB 1 #0.024s 0%: 0.028+0.41+0.016 ms clock, 0.22+0.11/0.14/0.093+0.13 ms cpu, 1024->1024->1024 MB, 1025 MB goal, 8 P env: gctrace=1, sys: 2111 MB, alloc: 2048 MB, idel: 63 MB, released: 63 MB, inuse: 2048 MB gc 2 #1.049s 0%: 0.021+0.44+0.005 ms clock, 0.16+0.12/0.15/0.12+0.045 ms cpu, 2048->2048->2048 MB, 2049 MB goal, 8 P env: gctrace=1, sys: 3135 MB, alloc: 3072 MB, idel: 63 MB, released: 63 MB, inuse: 3072 MB env: gctrace=1, sys: 4159 MB, alloc: 4096 MB, idel: 63 MB, released: 63 MB, inuse: 4096 MB gc 3 #3.096s 0%: 0.023+0.56+0.017 ms clock, 0.18+0.13/0.20/0.17+0.13 ms cpu, 4096->4096->2048 MB, 4097 MB goal, 8 P env: gctrace=1, sys: 4159 MB, alloc: 3072 MB, idel: 1087 MB, released: 63 MB, inuse: 3072 MB env: gctrace=1, sys: 4159 MB, alloc: 4096 MB, idel: 63 MB, released: 63 MB, inuse: 4096 MB gc 4 #5.619s 0%: 0.023+0.31+0.018 ms clock, 0.18+0.18/0.18/0.22+0.15 ms cpu, 4096->4096->2048 MB, 4097 MB goal, 8 P env: gctrace=1, sys: 4159 MB, alloc: 3072 MB, idel: 1087 MB, released: 63 MB, inuse: 3072 MB env: gctrace=1, sys: 4159 MB, alloc: 4096 MB, idel: 63 MB, released: 63 MB, inuse: 4096 MB gc 5 #8.141s 0%: 0.025+0.26+0.004 ms clock, 0.20+0.16/0.15/0.15+0.033 ms cpu, 4096->4096->2048 MB, 4097 MB goal, 8 P env: gctrace=1, sys: 4159 MB, alloc: 3072 MB, idel: 1087 MB, released: 63 MB, inuse: 3072 MB gc 6 #10.413s 0%: 0.028+0.51+0.013 ms clock, 0.22+0.23/0.28/0.29+0.11 ms cpu, 4096->4096->2048 MB, 4097 MB goal, 8 P env: gctrace=1, sys: 4159 MB, alloc: 4096 MB, idel: 63 MB, released: 63 MB, inuse: 4096 MB The format of this line is subject to change. Currently, it is: gc # ##s #%: #+#+# ms clock, #+#/#/#+# ms cpu, #->#-># MB, # MB goal, # P where the fields are as follows: gc # the GC number, incremented at each GC ##s time in seconds since program start #% percentage of time spent in GC since program start #+...+# wall-clock/CPU times for the phases of the GC #->#-># MB heap size at GC start, at GC end, and live heap # MB goal goal heap size # P number of processors used Note: go version go1.15.5 linux/amd64 See also: Go 1.13 RSS keeps on increasing, suspected scavenging issue https://github.com/golang/go/issues/36398 https://github.com/golang/go/issues/39295 https://go.googlesource.com/proposal/+/master/design/14951-soft-heap-limit.md https://docs.google.com/document/d/1wmjrocXIWTr1JxU-3EQBI6BK6KgtiFArkG47XK73xIQ/edit# https://blog.cloudflare.com/go-dont-collect-my-garbage/
Is it "growpart" or "resize2fs" for these new c5 instanced?
$ sudo lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT loop0 7:0 0 4G 0 loop /var/tmp nvme0n1 259:0 0 500G 0 disk ├─nvme0n1p1 259:1 0 1M 0 part └─nvme0n1p2 259:2 0 300G 0 part / $ sudo fdisk -l /dev/nvme0n1p2 Disk /dev/nvme0n1p2: 322.1 GB, 322120433152 bytes, 629141471 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Thanks!
Oracle 11gr2 failed check of kernel parameters on hp-ux
I'm installing oracle 11gR2 on 64 bit itanium HP-UX (v 11.31) system ( for HP Operation Manager 9 ). According with the installation requiremens, I've changed the kernel parameters but when I start the installation process it don't recognize them. Below the parameters that I've set : Parameter ( Manual) (on server) ------------------------------------------------------------- fs_async 0 0 ksi_alloc_max (nproc*8) 10240*8 = 81920 executable_stack 0 0 max_thread_proc 1024 3003 maxdsiz 0x40000000 (1073741824) 2063835136 maxdsiz_64bit 0x80000000 (2147483648) 2147483648 maxfiles 256 (a) 4096 maxssiz 0x8000000 (134217728) 134217728 maxssiz_64bit 0x40000000 (1073741824) 1073741824 maxuprc ((nproc*9)/10) 9216 msgmni (nproc) 10240 msgtql (nproc) 32000 ncsize 35840 95120 nflocks (nproc) 10240 ninode (8*nproc+2048) 83968 nkthread (((nproc*7)/4)+16) 17936 nproc 4096 10240 semmni (nproc) 10240 semmns (semmni*2) 20480 semmnu (nproc-4) 10236 semvmx 32767 65535 shmmax size of memory or 0x40000000 (higher one) 1073741824 shmmni 4096 4096 shmseg 512 1024 vps_ceiling 64 64 if this can help: [root#HUG30055 /] # swapinfo Kb Kb Kb PCT START/ Kb TYPE AVAIL USED FREE USED LIMIT RESERVE PRI NAME dev 4194304 0 4194304 0% 0 - 1 /dev/vg00/lvol2 dev 8388608 0 8388608 0% 0 - 1 /dev/vg00/lvol10 reserve - 742156 -742156 memory 7972944 3011808 4961136 38% [root#HUG30055 /] # bdf /tmp Filesystem kbytes used avail %used Mounted on /dev/vg00/lvol6 4194304 1773864 2401576 42% /tmp