I need command to get process memory usage in MiB or KiB by PID. I prefer the number I see in gnome-system-monitor in "Memory" column. I tried a lot of different commands but all of them give me some other numbers.
For example, I've tried ps with different keys. The most interesting was
ps aux. The RSS column is something close but number here greater than in gnome-system-monitor. A lot of answers on question similar to mine contain top command but it gives memory usage in percents.
cat /proc/YourPid/status
ps aux YourPid
top -p YourPid
pmap -x YourPid
All of them give process memory usage; top gives both percents and absolute value (RES - Resident size)
Look at Understanding Process memory and How to measure actual memory usage of an application or process? for more details about memory usage and monitor.
Related
I am running few experiments to see changes in system behavior under different memory and cpu loads. I was wondering is there a bash script which constantly uses high memory but low CPU?
For the purpose of simulating CPU/memory/IO load, most *NIX systems (Linux included) provide handy tool called stress.
The tool varies from OS to OS. On Linux, to take up 512MB of RAM with low CPU load:
stress --vm 1 --vm-bytes 512M --vm-hang 100
(The invocation means: start one memory thread (--vm 1), allocate/free 512MB of memory in every thread, sleep before freeing memory 100 seconds.)
This is silly, and can't be reasonably expected to provide data which will be useful in any real-world scenario. However, to generate at least the amount of memory consumption associated with a given power-of-two bytes:
build_string() {
local pow=$1
local dest=$2
s=' '
for (( i=0; i<pow; i++ )); do
s+="$s"
done
printf -v "$dest" %s "$s"
}
build_string 10 kilobyte # build a string of length 1024
echo "Kilobyte string consumes ${#kilobyte} bytes"
build_string 20 megabyte # build a string of length 1048576
echo "Megabyte string consumes ${#megabyte} bytes"
Note that transiently, during construction, at least 2x the requested space will be required (for the local); a version that didn't have this behavior would either be using namevars (depending on bash 4.3) or eval (depending on the author's willingness to do evil).
I am profiling binary data which has
increasing Unix block size (one got from stat > Blocks) when the number of events are increased as in the following figure
but the byte distance between events stay constant
I have noticed some changes in other fields of the file which may explain the increasing Unix block size
The unix block size is a dynamic measure.
I am interested in why it is increasing with bigger memory units in some systems.
I have had an idea that it should be constant.
I used different environments to provide the stat output:
Debian Linux 8.1 with its default stat
OSX 10.8.5 with Xcode 6 and its default stat
Greybeard's comment may have the answer to the blocks behaviour:
The stat (1) command used to be a thin CLI to the stat (2) system
call, which used to transfer relevant parts of a file's inode. Pretty
early on, the meaning of the st_blksize member of the C struct
returned by stat (2) was changed to "preferred" blocksize for
efficient file system I/O, which carries well to file systems with
mixed block sizes or non-block oriented allocation.
How can you measure the block size in case (1) and (2) separately?
Why can the Unix block size increase with bigger memory size?
"Stat blocks" is not a block size. It is number of blocks the file consists of. It is obvious that number of blocks is proportional to size. Size of block is constant for most file systems (if not all).
I'd like to be able to test some guesses about memory complexity of various command line utilities.
Taking as a simple example
grep pattern file
I'd like to see how memory usage varies with the size of pattern and the size of file.
For time complexity, I'd make a guess, then run
time grep pattern file
on various sized inputs to see if my guess seems to be borne out in reality, but I don't know how to do this for memory.
One possibility would be a wrapper script that initiates the job and samples memory usage periodically, but this seems inelegant and unlikely to give the real high watermark.
I've seen time -v suggested, but don't have that flag available on my machine (running bash on OSX) and don't know where to find a version that supports it.
I've also seen that on Linux this information is available through the proc filesystem, but again, it's not available to me in my context.
I'm wondering if dtrace might be an appropriate tool, but again am concerned that a simple sample-based figure might not be the true high watermark?
Does anyone know of a tool or approach that would be appropriate on OSX?
Edit
I removed two mentions of disk usage, which were just asides and perhaps distracted from the main thrust of the question.
Your question is interesting because, without the application source code, you need to make a few assumptions about what constitutes memory use. Even if you were to use procfs, the results will be misleading: both the resident set size and the total virtual address space will be over-estimates since they will include extraneous data such as the program text.
Particularly for small commands, it would be easier to track individual allocations, although even there you need to be sure to include all the possible sources. In addition to malloc() etc., a process can extend its heap with brk() or obtain anonymous memory using mmap().
Here's a DTrace script that traces malloc(); you can extend it to include other allocating functions. Note that it isn't suitable for multi-threaded programs as it uses some non-atomic variables.
bash-3.2# cat hwm.d
/* find the maximum outstanding allocation provided by malloc() */
size_t total, high;
pid$target::malloc:entry
{
self->size = arg0;
}
pid$target::malloc:return
/arg1/
{
total += self->size;
allocation[arg1] = self->size;
high = (total > high) ? total : high;
}
pid$target::free:entry
/allocation[arg0]/
{
total -= allocation[arg0];
allocation[arg0] = 0;
}
END
{
printf("High water mark was %d bytes.\n", high);
}
bash-3.2# dtrace -x evaltime=exec -qs hwm.d -c 'grep maximum hwm.d'
/* find the maximum outstanding allocation provided by malloc() */
High water mark was 62485 bytes.
bash-3.2#
A much more comprehensive discussion of memory allocators is contained in this article by Brendan Gregg. It provides a much better answer than my own to your question. In particular, it includes a link to a script called memleak.d; modify this to include time stamps for the allocations & deallocations, so that you can sort its output by time. Then, perhaps using the accompanying script as an example, use perl to track the current outstanding total allocation and high water mark. Such a DTrace/perl combination would be suitable for tracing multi-threaded processes.
You can use /usr/bin/time -l (which is not the time builtin in macos) and read the "maximum resident set size", which is not precisely high water mark but might give you some idea.
$ /usr/bin/time -l ls
...
0.00 real 0.00 user 0.00 sys
925696 maximum resident set size
0 average shared memory size
0 average unshared data size
0 average unshared stack size
239 page reclaims
0 page faults
0 swaps
0 block input operations
0 block output operations
0 messages sent
0 messages received
0 signals received
3 voluntary context switches
1 involuntary context switches
The meaning of this field is explained here.
Tried getrusage(). Inaccurate results. Tried Instruments. Pain in the arse.
Best solution by far: valgrind + massif.
command-line based: easy to run, script and automate; no apps to open, menus to click, blah blah; can run in background etc
provides a visual graph-- in your terminal-- of memory usage over time
valgrind --tool=massif /path/to/my_program arg1 ...
ms_print `ls -r massif.out.* | head -1` | grep Detailed -B50
To view more details, run ms_print `ls -r massif.out.* | head -1`
I want check how much memory used in real time, for instance, each time I set or insert some data, I want to know how much memory increased and how much used totally.
I try to use INFO command, and check the used_memory or used_memory_* property would work or not, but sorry I found it only show the memory allotted by the system, cuz each time I check it after I insert new data, they still stay the same
Is there any way I can check the real time memory used in Redis?
The used_memory field is what you are looking for. It is not the memory allocated by the system as you said, this is the memory given by the process memory allocator to Redis.
Example:
> info memory
...
used_memory:541368
...
> set y "titi"
OK
> info memory
...
used_memory:541448 # i.e. +80 bytes
...
> del y
(integer) 1
> info memory
...
used_memory:541368
...
Please note that Redis does a number of memory related optimizations. For instance, it is able to factorize values containing small integers. Or, if you append data to an existing string, the corresponding buffer will not grow at each append operation. So depending on these optimizations, the memory usage increase/decrease for a given set of operation is not always consistent.
I want to ensure that a long-running number crunching algorithm doesn't use too much memory. The algorithm is written in C++ and runs on OS X. A drastically simplified version is:
int main() {
while (someCondition) {
// notice nothing is allocated on the heap
vector<int> v(10, 0);
}
}
I've profiled the code using Instruments (allocations and leaks). I don't see any leaks. And while the "live bytes" count looks fine (hovers around 20 MB) the "overall bytes" count keeps growing. What concerned me is when the "overall count" reached about 80 GB I received an OS X warning about lack of hard disk space (I have a 120 GB solid state disk). I don't know much about OS/process interaction so I thought I'd ask:
Is memory used by a long running process on a UNIX-based OS available to other processes before the first process is killed or no longer running?
Edit: Looks like I'm misinterpreting the "overall bytes" number in Instruments:Instruments ObjectAlloc: Explanation of Live Bytes & Overall Bytes. When I check out the process in Activity Monitor the "real memory" is essentially constant.
The reason you get a disk space warning is probably related to virtual memory allocation. Every time your process (or the OS) requests memory it is usually first "allocated" in backing-store - swap.
Total virtual memory is size of available swap plus RAM. I do not have access to OSX, and I know it plays by its own rules, but there must be a command that shows swap usage
swap -l (Solaris)
swap -s (Solaris)
free (linux)
The only command I came up with is vm_stat, plus top - it appears top is probably the closest to what I am talking about.