I know that using Activity Monitor we can see CPU utilisation.But i want to get this information of my remote system through script. top command is not helpful for me. Please comment me any other way to get it.
What is the objection to top in logging mode?
top -l 1 | grep -E "^CPU|^Phys"
CPU usage: 3.27% user, 14.75% sys, 81.96% idle
PhysMem: 5807M used (1458M wired), 10G unused.
Or use sysctl
sysctl vm.loadavg
vm.loadavg: { 1.31 1.85 2.00 }
Or use vm_stat
vm_stat
Mach Virtual Memory Statistics: (page size of 4096 bytes)
Pages free: 3569.
Pages active: 832177.
Pages inactive: 283212.
Pages speculative: 2699727.
Pages throttled: 0.
Pages wired down: 372883.
Related
I want to find the CPU process usage for all Oracle processes on an AIX box.
On Solaris I can do the following:
prstat -n 400 -c -s cpu -p 9013 1 1
PID USERNAME SIZE RSS STATE PRI NICE TIME CPU PROCESS/NLWP
9013 oracle 3463M 2928M sleep 53 0 0:00:35 0.9% oracle/2
Total: 1 processes, 2 lwps, load averages: 2.25, 2.32, 2.40
This basically reports the CPU usage for a given process ID (in this case 9013). Given a list of all Oracle PID’s I can use this command to get the CPU usage for each one, sum them up and hey presto I have my Oracle database CPU usage.
How can I get the same with AIX?
Thanks
You can try nmon or topas, which will show the current %CPU. You might also want to look into using WLM to create a class for all the Oracle processes, then use wlmstat to see the CPU usage for that class. That would save you the trouble of adding them up manually.
I have a hadoop cluster we assuming is performing pretty "bad". The nodes are pretty beefy.. 24 cores, 60+G RAM ..etc. And we are wondering if there are some basic linux/hadoop default configuration that prevent hadoop from fully utilizing our hardware.
There is a post here that described a few possibilities that I think might be true.
I tried logging in the namenode as root, hdfs and also myself and trying to see the output of lsof and also the setting of ulimit. Here are the output, can anyone help me understand why the setting doesn't match with the open files number.
For example, when I logged in as root. The lsof looks like this:
[root#box ~]# lsof | awk '{print $3}' | sort | uniq -c | sort -nr
7256 cloudera-scm
3910 root
2173 oracle
1886 hbase
1575 hue
1180 hive
801 mapred
470 oozie
427 yarn
418 hdfs
244 oragrid
241 zookeeper
94 postfix
87 httpfs
...
But when I check out the ulimit output, it looks like this:
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 806018
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 1024
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
I am assuming, there should be no more than 1024 files opened by one user, however, when you look at the output of lsof, there are 7000+ files opened by one user, can anyone help explain what is going on here?
Correct me if I had made any mistake understanding the relation between ulimit and lsof.
Many thanks!
You need to check limits for the process. It may be different from your shell session:
Ex:
[root#ADWEB_HAPROXY3 ~]# cat /proc/$(pidof haproxy)/limits | grep open
Max open files 65536 65536 files
[root#ADWEB_HAPROXY3 ~]# ulimit -n
4096
In my case haproxy has a directive on its config file to change maximum open files, there should be something for hadoop as well
I had a very similar issue, which caused one of the claster's YARN TimeLine server to stop due to reaching magical 1024 files limit and crashing with "too many open files" errors.
After some investigation it came out that it had some serious issues with dealing with too many files in TimeLine's LevelDB. For some reason YARN ignored yarn.timeline-service.entity-group-fs-store.retain-seconds setting (by default it's set to 7 days, 604800ms). We had LevelDB files dating back for over a month.
What seriously helped was applying a fix described in here: https://community.hortonworks.com/articles/48735/application-timeline-server-manage-the-size-of-the.html
Basically, there are a couple of options I tried:
Shrink TTL (time to live) settings First enable TTL:
<property>
<description>Enable age off of timeline store data.</description>
<name>yarn.timeline-service.ttl-enable</name>
<value>true</value>
</property>
Then set yarn.timeline-service.ttl-ms (set it to some low settings for a period of time):
\
<property>
<description>Time to live for timeline store data in milliseconds.</description>
<name>yarn.timeline-service.ttl-ms</name>
<value>604800000</value>
</property>
Second option, as described, is to stop TimeLine server, delete the whole LevelDB and restart the server. This will start the ATS database from scratch. Works fine if you failed with any other options.
To do it, find the database location from yarn.timeline-service.leveldb-timeline-store.path, back it up and remove all subfolders from it. This operation will require root access to the server where TimeLine is located.
Hope it helps.
I would like to obtain the virtual private memory consumed by a process under OSX from the command line. This is the value that Activity Monitor reports in the "Virtual Mem" column. ps -o vsz reports the total address space available to the process and is therefore not useful.
You can obtain the virtual private memory use of a single process by running
top -l 1 -s 0 -i 1 -stats vprvt -pid PID
where PID is the process ID of the process you are interested in. This results in about a dozen lines of output ending with
VPRVT
55M+
So by parsing the last line of output, one can at least obtain the memory footprint in MB. I tested this on OSX 10.6.8.
update
I realized (after I got downvoted) that #user1389686 gave an answer in the comment section of the OP that was better than my paltry first attempt. What follows is based on user1389686's own answer. I cannot take credit for it -- I've just cleaned it up a bit.
original, edited with -stats vprvt
As Mahmoud Al-Qudsi mentioned, top does what you want. If PID 8631 is the process you want to examine:
$ top -l 1 -s 0 -stats vprvt -pid 8631
Processes: 84 total, 2 running, 82 sleeping, 378 threads
2012/07/14 02:42:05
Load Avg: 0.34, 0.15, 0.04
CPU usage: 15.38% user, 30.76% sys, 53.84% idle
SharedLibs: 4668K resident, 4220K data, 0B linkedit.
MemRegions: 15160 total, 961M resident, 25M private, 520M shared.
PhysMem: 917M wired, 1207M active, 276M inactive, 2400M used, 5790M free.
VM: 171G vsize, 1039M framework vsize, 1523860(0) pageins, 811163(0) pageouts.
Networks: packets: 431147/140M in, 261381/59M out.
Disks: 487900/8547M read, 2784975/40G written.
VPRVT
8631
Here's how I get at this value using a bit of Ruby code:
# Return the virtual memory size of the current process
def virtual_private_memory
s = `top -l 1 -s 0 -stats vprvt -pid #{Process.pid}`.split($/).last
return nil unless s =~ /\A(\d*)([KMG])/
$1.to_i * case $2
when "K"
1000
when "M"
1000000
when "G"
1000000000
else
raise ArgumentError.new("unrecognized multiplier in #{f}")
end
end
Updated answer, thats work under Yosemite, from user1389686:
top -l 1 -s 0 -stats mem -pid PID
Why result of php memory_get_peak_usage differs so much from memory size that is shown as allocated to process when using 'top' or 'ps' commands in Linux?
I've set 2 Mb of memory_limit in php.ini
My single-string php-script with
echo memory_get_peak_usage(true);
says that it is using 786432 bytes (768 Kb)
If I try to ask system about current php process
echo shell_exec('ps -p '.getmypid().' -Fl');
it gives me
F S UID PID PPID C PRI NI ADDR SZ WCHAN RSS PSR STIME TTY TIME CMD
5 S www-data 14599 14593 0 80 0 - 51322 pipe_w 6976 2 18:53 ? 00:00:00 php-fpm: pool www
RSS param is 6976, so memory usage is 6976 * 4096 = 28573696 = ~28 Mb
Where that 28 Mb come from and is there any way to decrease memory size that is being used by php-fpm process?
The memory size is mostly used by the PHP process itself. memory_get_peak_usage() returns the memory used by your specific script. Ways to reduce the memory overhead is to remove the number of extensions, statically compile PHP, etc.. But don't forget that php-fpm (should) fork and that a lot of the memory usage that's not different between PHP process is in fact shared (until it changes).
PHP itself may only be set to a 2meg limit, but it's running WITHIN a Apache child process, and that process will have a much higher memory footprint.
If you were running the script from the command line, you'd get memory usage of PHP by itself, as it's not wrapped within Apache and is running on its own.
The peak memory usage is for the current script only.
Is there a shell command to know about how much memory is being used at a particular moment and details of how much each process is using, how much virtual memory is left etc?
For "each process", how about top:
PhysMem: 238M wired, 865M active, 549M inactive, 1652M used, 395M free.
VM: 162G vsize, 1039M framework vsize, 124775(0) pageins, 9149(0) pageouts.
PID COMMAND %CPU TIME #TH #WQ #POR #MREG RPRVT RSHRD RSIZE VPRVT VSIZE PGRP PPID STATE UID
7233 top 5.7 00:00.53 1/1 0 24 33 1328K 264K 1904K 17M 2378M 7233 3766 running 0
e.g.:
rprvt Resident private address space size.
rshrd Resident shared address space size.
rsize Resident memory size.
vsize Total memory size.
vprvt Private address space size.
Let's also hear it for the old classic, vmstat.
$ vmstat
procs -----------memory---------- ---swap-- -----io---- -system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
0 0 30160 15884 418680 281936 0 0 406 22 6 3 1 1 93 5
Depends on your operating system. In Linux, free answers two out of your three questions.
~> free
total used free shared buffers cached
Mem: 904580 895128 9452 0 63700 777728
-/+ buffers/cache: 53700 850880
Swap: 506036 0 506036
"Swap" refers to virtual memory.
If you're on Linux, give ps_mem.py a try.
If you are on an up-to-date Linux, cat /proc/$pid/smaps is the business.
If you are on OSX, check https://superuser.com/questions/97235/how-much-swap-is-a-given-mac-application-using.