Is there a Ruby way to view the memory usage of the running process? I have been using exec (or backticks) to call out to 'ps' but I'd rather do something that isn't system-specific. I'm starting to build docker containers and the handling of 'ps' seems to be inconsistent.
I'd much rather have a Ruby-specific method/class/etc. to get the current run-time memory of the Ruby process.
You can try with combination of rbtrace and ObjectSpace.
rbtrace -p PID -e 'Thread.new{require "objspace"; ObjectSpace.trace_object_allocations_start; GC.start(); ObjectSpace.dump_all(output: File.open("heap.json", "w"))}.join'
More info in the slides by Sam Saffron.
Related
I'm trying to dump trade-data off binance for multiple symbol-pairs, e.g. doge/btc, ada/btc, etc.
I can background, thus:
wscat -c wss://stream.binance.com:9443/ws/dogebtc#trade > doge.txt &
wscat -c wss://stream.binance.com:9443/ws/adabtc#trade > ada.txt &
But how to terminate them all?
Is there some smart way, like terminating the parent process?
I think the right answer depends a lot on the way your current system is implemented / used.
At the most basic scripting level, you could simply run kill against all wscat processes; but that may be too generic depending on the details.
Slightly better, in a BASH script, directly after creating these processes you'd have access to their PID as $!. You could stash those PIDs in a variable or file and later use them to kill each individual process.
If you're aiming for something slicker than that, you'd likely want to look into things like:
the SIGCHLD signal, becoming a subreaper (prctl PR_SET_CHILD_SUBREAPER), running as PID 1 in a PID-namespace (unshare --pid ...), things like that.
I am trying to daemonize a Ruby script, running on 2.1.1.
My daemon part of the code is like this:
case ARGV[0]
when "-start"
puts "TweetSearcher started."
Process.daemon
when "-stop"
Process.kill(9,Process.pid)
else
puts "Lacks arguments. Use -start/-stop"
abort
end
However, it looks like that the Process.kill(9,Process.pid) is not killing what I wanted to. I want to kill a previous "ruby tweetsearcher.rb -start", already running in background.
How do I proceed?
Typically, the PID is stored in a file that is then read to stop it.
Calling Process.kill(9,Process.pid) kills the "stopper" process itself, rather than the one it's trying to stop.
Here's a guide to writing daemons in Ruby: http://codeincomplete.com/posts/2014/9/15/ruby_daemons/
As you can see, it's not a trivial process.
Here is another blog that suggests that you should not try to daemonize at all, but instead rely on a process monitoring system to take care of those concerns: https://www.mikeperham.com/2014/09/22/dont-daemonize-your-daemons/
We're running sidekiq workers that utilize neography to do batch operations.
Our batch array holds up to 400 operations before flushing (we've tried lower numbers as well).
We hit the R14 memory error on heroku and things grind to almost a halt, so we suspect a memory leak of some sort ( I have already check for bloat ). However, we've been unable to figure out where it is or how to prevent it.
We've tried to use all debugging memory gem as ruby-prof, [...] without any results or clues, read object counts via ObjectSpace and even trying to debug line by line and launch the process without a background job but just via the rails c and the following command to monitor the memory usage top -pid `ps auw | grep -i 'rails c' | head -n 1 | awk '{print $2}'` -stats RSIZE.
I tried to update our ruby version to the latest (2.1.0) but no changes.
Any idea are welcome to help us to make our workers happier !
Neo4j internally uses a lot of caching which might consume a serious amount of memory. You can try to switch off Neo4j's object cache by setting cache_type=none, see http://docs.neo4j.org/chunked/stable/configuration-caches.html.
I’d like to view information for processes running in OS X. Running ps in the terminal just lists the open Terminal windows. How can I see all processes that are running?
Say I’m running a web browser, terminal and text editor. I’d like to see information for the text editor and web browser.
Running ps -e does the trick. Found the answer here.
You can just use top
It will display everything running on your OSX
Using top and ps is okay, but I find that using htop is far better & clearer than the standard tools Mac OS X uses. My fave use is to hit the T key while it is running to view processes in tree view (see screenshot). Shows you what processes are co-dependent on other processes.
You can install it from Homebrew using:
brew install htop
And if you have Xcode and related tools such as git installed on your system and you want to install the latest development code from the official source repository—just follow these steps.
First clone the source code from the htop GitHub repository:
git clone git#github.com:hishamhm/htop.git
Now go into the repository directory:
cd htop
Run autogen.sh:
./autogen.sh
Run this configure command:
./configure
Once the configure process completes, run make:
make
Finally install it by running sudo make install:
sudo make install
Try ps -ef. man ps will give you all the options.
-A Display information about other users' processes, including those without controlling terminals.
-e Identical to -A.
-f Display the uid, pid, parent pid, recent CPU usage, process start time, controlling tty, elapsed CPU usage, and the associated command. If the -u option is also used, display
the user name rather then the numeric uid. When -o or -O is used to add to the display following -f, the command field is not truncated as severely as it is in other formats.
Try the top command. It's an interactive command that will display the running processes.
You may also use the Apple's "Activity Monitor" application (located in /Applications/Utilities/).
It provides an actually quite nice GUI. You can see all the running processes, filter them by users, get extended informations about them (CPU, memory, network, etc), monitor them, etc...
Probably your best choice, unless you want to stick with the terminal (in such a case, read the top or ps manual, as those commands have a bunch of options).
To sort by cpu usage: top -o cpu
if you are using ps, you can check the manual
man ps
there is a list of keywords allowing you to build what you need. for example to show, userid / processid / percent cpu / percent memory / work queue / command :
ps -e -o "uid pid pcpu pmem wq comm"
-e is similar to -A (all inclusive; your processes and others), and -o is to force a format.
if you are looking for a specific uid, you can chain it using awk or grep such as :
ps -e -o "uid pid pcpu pmem wq comm" | grep 501
this should (almost) show only for userid 501. try it.
The slightly GUI way
if you are a cli (ui) fan. I recommend trying https://github.com/clementtsang/bottom which shows not only processes, but also temperature, disk usage and network. Screenshot is running from kitty (terminal) as an example, I use it on OSX default terminal and the color shows up a bit different, but still amazing.
The tree way
As described here : https://en.wikipedia.org/wiki/Pstree will give a better connection on the hierarchy of the processes
brew install pstree # if you need to install it
pstree
pstree -u <user> # show only processes by your user
pstree -s <string> # show only processes with string
pstree -help # show help
I'm writing a ruby bootstrapping script for a school project, and part of this bootstrapping process is to start a couple of background processes (which are written and function properly). What I'd like to do is something along the lines of:
`/path/to/daemon1 &`
`/path/to/daemon2 &`
`/path/to/daemon3 &`
However, that blocks on the first call to execute daemon1. I've seen references to a Process.spawn method, but that seems to be a 1.9+ feature, and I'm limited to Ruby 1.8.
I've also tried to execute these daemons from different threads, but I'd like my bootstrap script to be able to exit.
So how can I start these background processes so that my bootstrap script doesn't block and can exit (but still have the daemons running in the background)?
As long as you are working on a POSIX OS you can use fork and exec.
fork = Create a subprocess
exec = Replace current process with another process
You then need to inform that your main-process is not interested in the created subprocesses via Process.detach.
job1 = fork do
exec "/path/to/daemon01"
end
Process.detach(job1)
...
better way to pseudo-deamonize:
`((/path/to/deamon1 &)&)`
will drop the process into it's own shell.
best way to actually daemonize:
`service daemon1 start`
and make sure the server/user has permission to start the actual daemon. check out 'deamonize' tool for linux to set up your deamon.