I am trying to create a function that will execute a process and will kill it after 2 seconds:
def execute(command, input_file, output_file)
pid = Kernel.spawn(command,{
STDIN => input_file,
STDOUT => output_file,
:rlimit_cpu => [2,2], #2 secs,
:rlimit_nproc => 0,
:rlimit_as => 16 * 1024 * 1024
}
);
Process.wait(pid)
puts "exit status = " + $?.exitstatus.to_s
return File.read(output_file)
end
I tested this function with short-running processes and long-running processes (with a "sleep" call). The command always completes. I need that the spawn call kills the "command" after 2 seconds using rlimit_cpu. How I can do that?
EDIT: seems like rlimit_cpu does not works as I thought. According to this question:
The CPU limit is a limit on CPU seconds rather than elapsed time
Also:
When you do the fib call, you hammer the CPU so that elapsed and CPU time are close (most of the process time is spent using the CPU). That's not the case when printing since most time there is spent in I/O.
I will use another approach, since I need to kill the process regardless if the process is CPU-bound or IO-bound
To kill the process after two seconds, run a timer for two seconds, and then try to call Process.kill on the child.
Related
I have this worker process in Heroku, which does some cleaning. It runs every two hours and listens to Heroku terminate signals. It works fine, but I'm seeing 100% dyno load all the time.
My question is: How to run this kind of worker process in Heroku without 100% dyno load? The loop causes the dyno load, but what to use instead of the infinite loop?
# Scheduler here
cleanup = Rufus::Scheduler.new
cleanup.cron '* */2 * * *' do
do_some_cleaning
end
# Signal trapping
Signal.trap("TERM") {
terminate = true
shut_down
exit 0
}
# Infinite loop
while terminate == false
end
It's because you're doing an infinite loop with no sleep cycles. This means you're basically telling the CPU that every single cycle you should be immediately executing a loop condition.
This will quickly use up your CPU.
Instead, try throwing a sleep statement into your infinite loop -- this will pause execution and bring your usage down to 0% =)
while terminate == false
sleep 1
end
I should have thought about it sooner. You can actually simply join rufus-scheduler's loop:
cleanup_scheduler = Rufus::Scheduler.new
cleanup_scheduler.cron '* */2 * * *' do
do_some_cleaning
end
Signal.trap('TERM') do
shut_down
exit 0
end
cleanup_scheduler.join
That joins rufus-scheduler scheduling thread and is pseudo equivalent to:
while !terminated
sleep 0.3
trigger_schedules_if_any
end
Running a small jetty application on a raspberry pi I noticed that after the first access, the application keeps burning around 3% CPU. A quick inspection showed that the same is true, with less %, on my laptop. Checking with strace I find a never ending sequence of
...
12:58:01.999717 clock_gettime(CLOCK_MONOTONIC, {2923, 200177551}) = 0
12:58:01.999864 futex(0x693a0f44, FUTEX_WAIT_BITSET_PRIVATE, 1, {2923, 250177551}, ffffffff) = -1 ETIMEDOUT (Connection timed out)
12:58:02.050090 futex(0x693a0f28, FUTEX_WAKE_PRIVATE, 1) = 0
12:58:02.050236 gettimeofday({1436093882, 50296}, NULL) = 0
12:58:02.050403 gettimeofday({1436093882, 50444}, NULL) = 0
12:58:02.050767 clock_gettime(CLOCK_MONOTONIC, {2923, 251228114}) = 0
...
(This is Java 7 on ubuntu 14.04 with Jetty 9.3.* using an h2 db, just in case this rings any bells for someone.)
I learned that it suffices to capture strace -f -tt -p <pid> -o out.txt, grep for clock_gettime, extract the pid, sort and uniq -c to find the thread calling clock_gettime most often. Plotting the delta times nicely shows a line at 50 milliseconds. Further the PID can be found in a thread dump taken with jvisualvm as the nid in hex and turns out to be 'VM Periodic Task Thread'. But why so often? This does not seem to be a standard behaviour of the JVM.
I have a smoke test that I run against my servers before making them live. At the moment it runs in serial form and takes around 60s per server. I can run these in parallel and I've done it with Thread.new which is great as it runs them a lot faster but I lose track of whether the test actually passed or not.
I'm trying to improve this by using Process.spawn to manage my processes.
pids = []
uris.each do |uri|
command = get_http_tests_command("Smoke")
update_http_tests_config( uri )
pid = Process.spawn( system( command ) )
pids.push pid
Process.detach pid
end
# make sure all pids return a passing status code
# results = Process.waitall
I'd like to kick off all my tests but then afterwards make sure that all the tests return a passing status code.
I tried using Process.waitall but I believe that to be incorrect and used for forks, not spawns.
After all the process have completed I'd like to return the status of true if they all pas or false if any one of them fails.
Documentation here
Try:
statuses = pids.map { |pid| Process.wait(pid, 0); $? }
This waits for each of the process ids to finish, and checks for the result status set in $? for each process
I have follwing ruby scripts
rubyScript.rb:
require "rScript"
t1 = Thread.new{LongRunningOperation(); puts "doneLong"}
sleep 1
shortOperation()
puts "doneShort"
t1.join
rScript.rb:
def LongRunningOperation()
puts "In LongRunningOperation method"
for i in 0..100000
end
return 0
end
def shortOperation()
puts "IN shortOperation method"
return 0
end
THE OUTPUT of above script i.e.(ruby rubyScript.rb)
1) With use of sleep function
In veryLongRunningOperation method
doneLong
IN shortOperation method
doneShort
2) Without use of sleep function i.e. removing sleep function.(ruby rubyScript.rb)
In veryLongRunningOperation method
IN shortOperation method
doneShort
doneLong
why there is difference in output. What sleep does in ablve case. Thannks in advance.
The sleep lets the main thread sleep for 1 second.
Your long running function runs longer than your short running function but it is still faster than one second.
If you remove the sleep, then your long running function starts in a new thread and the main thread continues without any wait. It then starts the short running function, which finishes nearly immediatly, while the long running function is still running.
In the case of the none removed sleep it goes as following:
Your long running function starts in a new Thread and the main thread continues. Now the main thread encounters the sleep command and waits for 1 second. In this time the long running function in the other thread is still running and finishes. The main thread continues after its sleep time and starts the short running function.
sleep 1 makes the current thread sleep (i.e. do nothing) for one second. So veryLongRunningOperation (which despite being a very long running operation still takes less than a second) has enough time to finish before shortOperation even starts.
sleep 1
Makes the main thread to wait for 1 second, that allows t1 to finish before shortOperation is executed.
If there is more than one way, please list them. I only know of one, but I'm wondering if there is a cleaner, in-Ruby way.
The difference between the Process.getpgid and Process::kill approaches seems to be what happens when the pid exists but is owned by another user. Process.getpgid will return an answer, Process::kill will throw an exception (Errno::EPERM).
Based on that, I recommend Process.getpgid, if just for the reason that it saves you from having to catch two different exceptions.
Here's the code I use:
begin
Process.getpgid( pid )
true
rescue Errno::ESRCH
false
end
If it's a process you expect to "own" (e.g. you're using this to validate a pid for a process you control), you can just send sig 0 to it.
>> Process.kill 0, 370
=> 1
>> Process.kill 0, 2
Errno::ESRCH: No such process
from (irb):5:in `kill'
from (irb):5
>>
#John T, #Dustin: Actually, guys, I perused the Process rdocs, and it looks like
Process.getpgid( pid )
is a less violent means of applying the same technique.
For child processes, other solutions like sending a signal won't behave as expected: they will indicate that the process is still running when it actually exited.
You can use Process.waitpid if you want to check on a process that you spawned yourself. The call won't block if you're using the Process::WNOHANG flag and nil is going to be returned as long as the child process didn't exit.
Example:
pid = Process.spawn('sleep 5')
Process.waitpid(pid, Process::WNOHANG) # => nil
sleep 5
Process.waitpid(pid, Process::WNOHANG) # => pid
If the pid doesn't belong to a child process, an exception will be thrown (Errno::ECHILD: No child processes).
The same applies to Process.waitpid2.
This is how I've been doing it:
def alive?(pid)
!!Process.kill(0, pid) rescue false
end
You can try using
Process::kill 0, pid
where pid is the pid number, if the pid is running it should return 1.
Under Linux you can obtain a lot of attributes of running programm using proc filesystem:
File.read("/proc/#{pid}/cmdline")
File.read("/proc/#{pid}/comm")
A *nix-only approach would be to shell-out to ps and check if a \n (new line) delimiter exists in the returned string.
Example IRB Output
1.9.3p448 :067 > `ps -p 56718`
" PID TTY TIME CMD\n56718 ttys007 0:03.38 zeus slave: default_bundle \n"
Packaged as a Method
def process?(pid)
!!`ps -p #{pid.to_i}`["\n"]
end
I've dealt with this problem before and yesterday I compiled it into the "process_exists" gem.
It sends the null signal (0) to the process with the given pid to check if it exists. It works even if the current user does not have permissions to send the signal to the receiving process.
Usage:
require 'process_exists'
pid = 12
pid_exists = Process.exists?(pid)