Basically, all of my logic is in a bunch of event handlers that are fired by threads. After I establish the event handlers in the main thread:
puts 'Now connecting...'
socket = SocketIO::Client::Simple.connect 'http://localhost:3000'
socket.on :connect do
puts 'Connected'
end
I don't really have anything else to do in the main thread... but when I exit it, the whole process exits! I guess I could just do a while 1 {sleep 3} or something but that seems like a hack.
From what I can tell, daemon threads also don't work on Windows, so what am I supposed to do here?
If you're creating threads then it's your obligation to wait for them to finish before terminating. Normally this is done with join on the thread or threads in question.
Do you have a way of getting the thread out of that SocketIO instance? If so, join it.
Related
I am trying to fix a python3 application where multiple proceess and threads are created controlled by various queues and pipes. I am trying to make a form of controlled exit when someone tries to break the program with ctrl-c. However no mather what I do it always hangs just at the end.
I've tried to used Keyboard-interrupt exception and signal catch
The below code is part of the multi process code.
from multiprocessing import Process, Pipe, JoinableQueue as Queue, Event
class TaskExecutor(Process):
def __init__(....)
{inits}
def signal_handler(self, sig, frame):
print('TaskExecutor closing')
self._in_p.close()
sys.exit(1)
def run
signal.signal(signal.SIGINT, self.signal_handler)
signal.signal(signal.SIGTERM, self.signal_handler)
while True:
# Get the Task Groupe name from the Task queue.
try:
ExecCmd = self._in_p.recv() # type: TaskExecCmd
except Exceptions as e:
self._in_p.close()
return
if ExecCmd.Kill:
self._log.info('{:30} : Kill Command received'.format(self.name))
self._in_p.close()
return
else
{other code executing here}
I'm getting the above print that its closing.
but im still getting a lot of different exceptions which i try to catch but it will not.
I'm am looking for some documentation on how to and in which order to shut down multiprocess and its main process.
I know it's very general question however its a very large application so if there are any question or thing i could test i could narrow it down.
Regards
So after investigating this issue further I found that in situation where I had a pipe thread, Queue thread and 4 multiprocesses running. # of these processes could end up hanging when terminating the application with ctrl-c. The Pipe and Queue process where already shut down.
In the multiprocessing documentation there are a warning.
Warning If this method is used when the associated process is using a
pipe or queue then the pipe or queue is liable to become corrupted and
may become unusable by other process. Similarly, if the process has
acquired a lock or semaphore etc. then terminating it is liable to
cause other processes to deadlock.
And I think this is what's happening.
I also found that even though I have a shutdown mechanism in my multi-process class the threads still running would of cause be considered alive (reading is_alive()) even though I know that the run() method have return IE som internal was hanging.
Now of the solution. My multiprocesses was for a design view not a Deamon because I wanted to control the shot down of them. However I changed them to Deamon so they would always be killed regardless. I first added that anyone kill signal would raise and ProgramKilled exception throughout my entire program.
def signal_handler(signum, frame):
raise ProgramKilled('Task Executor killed')
I then changed my shut down mechanism in my multi process class to
while True:
# Get the Task Groupe name from the Task queue.
try:
# Reading from pipe
ExecCmd = self._in_p.recv() # type: TaskExecCmd
# If fatal error just close it all
except BrokenPipe:
break
# This can occure close the pipe and break the loop
except EOFError:
self._in_p.close()
break
# Exception for when a kill signal is detected
# Set the multiprocess as killed (just waiting for the kill command from main)
except ProgramKilled:
self._log.info('{:30} : Died'.format(self.name))
self._KilledStatus = True
continue
# kill command from main recieved
# Shut down all we can. Ignore exceptions
if ExecCmd.Kill:
self._log.info('{:30} : Kill Command received'.format(self.name))
try:
self._in_p.close()
self._out_p.join()
except Exception:
pass
self._log.info('{:30} : Kill Command executed'.format(self.name))
break
else if (not self._KilledStatus):
{Execute code}
# When out of the loop set killed event
KilledEvent.set()
And in my main thread I have added the following clean up process.
#loop though all my resources
for ThreadInterfaces in ResourceThreadDict.values():
# test each process in each resource
for ThreadIf in ThreadInterfaces:
# Wait for its event to be set
ThreadIf['KillEvent'].wait()
# When event have been recevied see if its hanging
# We know at this point every thing have been closed and all data have been purged correctly so if its still alive terminate it.
if ThreadIf['Thread'].is_alive():
try:
psutil.Process(ThreadIf['Thread'].pid).terminate()
except (psutil.NoSuchProcess, AttributeError):
pass
Af a lot of testing I know its really hard to control a termination of and app with multiple processes because you simply do not know in which order all of your processes receive this signal.
I've tried to in someway to save most of my data when its killed. Some would argue what I need that data for when manually terminating the app. But in this case this app runs a lot of external scripts and other application and any of those can lock the application and then you need to manually kill it but still retain the information for what have already been executed.
So this is my solution to my current problem with my current knowledge.
Any input or more in depth knowledge on what happening is welcome.
Please note that this app runs both on linux and windows.
Regards
Looking for a way to wait for the completion of all child processes, I found this code:
while true
p "waiting for child processes"
begin
exited_pid = Process.waitpid(-1,Process::WNOHANG)
if exited_pid and exited_pid > 0 then
p "Process exited : #{exited_pid} with status #{$?.exitstatus }"
end
sleep 5
rescue SystemCallError
puts "All children collected!"
break
end
end
This looks like it works in a similar way to Unix-systems process management, as I read on tutorialspoint HERE.
So in summary, it looks like this code:
Calls Process.waitpid, for any child process that exists. If no child process has exited, continue anyway.
If a child process has exited, then notify the user. Otherwise sleep, and check again.
When all child processes have exited an error is thrown, which is caught and the user is notified that processes are complete.
But looking at a similar question on waiting for child processes in C (Make parent wait for all child processes), which has as an answer:
POSIX defines a function: wait(NULL)；. It's shorthand for waitpid(-1,
NULL, 0);, which will block until all children processes exit.
I tested that Process.wait() in Ruby achieves pretty much the same thing as the more verbose code above.
What is the benefit of the more verbose code above? Or, which is considered a better approach to waiting for child processes? It seems in the verbose code that I would be able to wait for specific processes and listen for specific exit codes. But if I don't need to do this is there any benefit?
Also, regarding the more verbose code:
Why does the call to Process.waitpid() throw an error if there are no more child processes?
If more than 1 child process exists within the 5 second sleep period, it seems like there is a queue of completed processes and that Process.waitpid just returns the top member of the queue. What is actually happening here?
I have two threads in a ruby process. What I want to do is have one sleep and the other send a signal to wake up.
I know how to do it with Mutex and ConditionalVariables, but I don't have a critic section to run so it's not the right solution.
I know how to do it with thread stop and thread run, where on thread stops itself and the other calls run on it, but It's now what I'm really looking for.
Is there other way? What I'm trying to accomplish is have on thread wait for content in the database and the other notify when there is content.
Maybe something like this will work?
require "thread"
q = Queue.new
Thread.new do
sleep 10
q.push "*"
end
p q.pop
If I fire of 1 or 1000 of these in a controller action:
Thread.new {
# do some stuff
}
Will they indeed run asynchronously with the http request?
If an exception is raised, where does it trickle up to?
Anything else I should know about?
Threads and exceptions are not necessarily friends since exceptions can't bust out of the current thread and alert the parent thread. You also need to turn on thread exception notification or you'll never hear about them at all:
Thread.new do
Thread.abort_on_exception = true
end
You'll also need to call Thread.join on each new thread or the main one will hurry along without them.
This way your code will at least halt on an exception instead of simply terminating the thread that generated one and continuing on as if nothing had happened.
Make sure that the things you're calling inside your thread are thread safe or you may get unexpected results.
I write a simple bot using "SimpleMUCClient". But got error: app.rb:73:in stop': deadlock detected (fatal)
from app.rb:73:in'. How to fix it?
Most likely the code you're running is executed in another thread. That particular thread is then joined (meaning Ruby waits for it to finish upon exiting the script) using Thread.join(). Calling Thread.stop() while also calling .join() is most likely the cause of the deadlock. Having said that you should following the guides of StackOverflow regarding how to ask questions properly, since you haven't done so I've down voted your question.
Joining a thread while still calling Thread.stop can be done as following:
th = Thread.new do
Thread.stop
end
if th.status === 'sleep'
th.run
else
th.join
end
It's not the cleanest way but it works. Also, if you want to actually terminate a thread you'll have to call Thread.exit instead.