I have a small HTTP server script I've written using eventmachine which needs to call external scripts/commands and does so via backticks (``). When serving up requests which don't run backticked code, everything is fine, however, as soon as my EM code executes any backticked external script, it stops serving requests and stops executing in general.
I noticed eventmachine seems to be sensitive to sub-processes and/or threads, and appears to have the popen method for this purpose, but EM's source warns that this method doesn't work under Windows. Many of the machines running this script are running Windows, so I can't use popen.
Am I out of luck here? Is there a safe way to run an external command from an eventmachine script under Windows? Is there any way I could fire off some commands to be run externally without blocking EM's execution?
edit: the culprit that seems to be screwing up EM the most is my usage of the Windows start command, as in: start java myclass. The reason I'm using start is because I want those external scripts to start running and keep running after the EM request is served
The ruby documentation states that the backtick operator "Returns the standard output of running cmd in a subshell"
So if your command i.e. start java myclass is continuing to run then ruby is waiting for it to finish to pass back it's output to your program.
Try win32-open3 (and if it needs to be cross-platform not windows-only, also have a look at POpen4)
EventMachine has a thread pool. You can EM.defer your backticks like this
EM.defer { `start java myclass` }
By default the thread pool has 20 threads, and you can change its size by assiging EM.threadpool_size a value.
Important to note, that EM.defer can be passed operation, which is executed in deferred thread, callback, which is executed in reactor thread, and error callback which is run in reactor thread when the operation raises the exception.
If you use Java, you may consider using jruby, which has real threads support, and you could probably reuse your Java code from within jruby.
Related
I'm using Chromium Embedded Framework 3 (via CEFGlue) to host a browser in a third-party process via a plugin. CEF spins up various external processes (e.g. the renderer process) and manages the lifetime of these.
When the third-party process exits cleanly, CefRuntime.Shutdown is called and all the processes exit cleanly. When the third-party process exits badly (for example it crashes) I'm left with CEF executables still running and this (sometimes) causes problems with the host application meaning it doesn't start again.
I'd like a way to ensure that whatever manner the host application exits CefRuntime.Shutdown is called and the user doesn't end up with spurious processes running.
I've been pointed in the direction of job objects (see here) but this seems like it might be difficult to ship in a real solution as on some versions of Windows it requires administrative rights.
I could also set CEF to run in single process mode, but the documentation specifies that this is really for "debugging" only, so I'm assuming shipping this in production code is bad for some reason (see here).
What other options do I have?
Following on from the comments, I've tried passing the PID of the host process through to the client (I can do this by overriding OnBeforeChildProcessLaunch). I've then created a simple watchdog with the following code:
ThreadPool.QueueUserWorkItem(_ => {
var process = Process.GetProcessById(pid);
while (!process.WaitForExit(5000)) {
Console.WriteLine("Waiting for external process to die...");
}
Process.GetCurrentProcess().Kill();
});
I can verify in the debugger that this code executes and that the PID I'm passing into it is correct. However, if I terminate the host process I find that the thread simply dies in a way that I can't control and that the lines following the while loop are never executed (even if I replace it with a Console.WriteLine I never see any more messages printed from this thread.
For posterity, the solution suggested by #IInspectable worked, but in order to make it work I had to switch the implementation of of the external process to use the non-multi threaded message loop.
settings.MultiThreadedMessageLoop = false;
CefRuntime.Initialize(mainArgs, settings, cefWebApp, IntPtr.Zero);
Application.Idle += (sender,e) => {
if (parentProcess.HasExited) Process.GetCurrentProcess().Kill();
CefRuntime.DoMessageLoopWork();
}
Application.Run();
I have a Windows application of which I need multiple instances running, with different command line parameters. The application is quite unstable and tends to crash every 48 hours or so.
Since manual checking for failure and restarting in case of one isn't what I love to do I want to write a "manager program" for this. It would launch the program (all its instances) and then watch them. In case a process crashes it would be restarted.
In Linux I could achieve this with fork()s and pids, but this obviously is not available in Windows. So, should I try to implement a CreateProcess version or is there a better way?
When you call CreateProcess, you are returned a handle to the new process in the hProcess member of the process information struct that you pass to CreateProcess. You can use this handle to detect when the process terminates.
For instance, you can create another thread and call WaitForSingleObject(hProcess) and block until the process terminates. Then you can decide whether or not to restart it.
Or your could call GetExitCodeProcess(hProcess, &exitcode) and test exitcode. If it has the value STILL_ACTIVE then your process has not terminated. This approach based on GetExitCodeProcess necessitates polling.
If it can be run as a daemon, the simplest way to ensure it keep running is Non-Sucking Service Manager.
It will allow to run as win32 service applications not designed as services. It will monitor and restart if necessary. And the source code is included, if any customization is needed.
All you need to do is define each of your instances as a service, with the required parameters, at it will do the rest.
If you have some kind of security police limitation and can't use third party tools, then coding will be necessary. The answer from David Heffernan gives you the appropiate direction.
Or it can be done in batch, vbs or js without need of anything out of the system. WMI Win32_Process class should allow you to handle it.
Everybody always explicitly mentions that next_tick will be executed in the Main Thread.
But what about Timers and callbacks/errbacks? Are they guaranteed to run in the Main Thread as well?
Regardless of Ruby version, EM runs everything except the EM.defer code block inside the reactor thread (EM.defer result and on error blocks are in the reactor too), so yes,
timers, callbacks and errbacks are guaranteed to run in the reactor (main) thread
Assuming you're using MRI Ruby, then there is only 1 'real/native' thread, which is the main/reactor thread, and all code is executed by this reactor thread. All timers and callbacks will be picked up on the reactor thread.
Eventmachine is not designed to use multiple threads as part of its standard operation, however, Eventmachine does have a 'EM::defer' method, which will execute the block on a different 'ruby' thread, and I believe that if you are running a multithread capable ruby such as jruby or rubinious, then this will be run on a 'native' OS thread. "Defer" should be used if you are going to be executing any 'blocking' code.
I'm designing a web service using Sinatra and I need to perform certain operations when the service is started and some other operations when the server is stopped.
How can I register those operations to be fully integrated with sinatra?
Thanks.
The answer depends on how you need to perform your operations. Does they need to be ran for each ruby process or do they need to be ran just once for the service. I suppose it's once for all the service and in the case of the latest :
You might be tempted to run some code before your Sinatra app is starting but this is not really the behavior you might expect. I'll explain why just after. The workaround would be adding code before your sinatra class like
require "sinatra"
puts "Starting"
get "/" do
...
end
You could add some code to your config.ru too btw, would have the same effect but I don't which one is uglier.
Why is this wrong ? Because when you host your web service, many web server instances will be fired and each one will execute the puts method or your "starting" code. This is correct when you want to initialize things that are local to your app instance, like a database connection but not to initialize things which are shared by all of them.
And about the code firing at its end, well you can't (or maybe you could with some really ugly workaround, but you'll end with the same issue you get with the start).
So the best way to handle on and off operations would be to wrap it within your tasks firing your service.
Run some rake task or ruby script that do your initalization stuff
Start your web server
And to stop it
Run a rake task or ruby script that stops the server
Run your rake task or ruby script that does the cleaning operations.
You can wrap those into a single rake task, by starting your app server directly from ruby, like I did there https://github.com/TactilizeTeam/photograph/blob/master/bin/photograph.
This way you can easily add some code to get ran before starting the service, still keeping it into a single task. With some plumbing, I guess you can fire multiple thin instances and then allow you to start your cluster of thin (or whatever you use) instances and have still one task to rely on.
I'd say that adding a handler to the SIGINT signal could allow you to run some code before exiting. See http://www.ruby-doc.org/core-1.9.3/Signal.html for how to do that. You might want to check if Thin isn't already registering a trap for that signal, I'm not sure if this is handled in the library or in the script used to launch thin ( the "thin" executable that gets in your $PATH).
Another way to handle the exit, would be to have a watchdog process, that check if your cluster is running and could ensure the stop code is being ran if no more instances are running.
I've used PPerl for deamon like processes.
This program turns ordinary perl
scripts into long running daemons,
making subsequent executions extremely
fast. It forks several processes for
each script, allowing many proceses to
call the script at once.
Does anyone know of something like this for ruby? Right now I am planing on using a wrapper around curl to call a REST WebService written in Sinatra running on JRuby. I'm hoping there is a simpler option.
Have you looked at using nailgun? It sets up a background JVM process that your scripts execute in. That way you can use jruby w/o incurring the JVM startup time you would normally get with each script run.
You mean like daemons?
Simple example of in-process daemonization
require 'rubygems'
require 'daemons'
Daemons.daemonize
loop do
`touch /tmp/me`
sleep 1
end
Also, instead of using curl, have you looked at rest-client?