I have seen many answers on StackOverflow on how to create a Deadlock, but are written code.
I'm looking to understand on a more simple way. I wanted to know which would be the steps you would take to recreate this Deadlock. I'm not looking for codes, but more steps.
Traffic Deadlock
Thank you for your time.
Sorry for the link image, I need 10 reputation to post images.
We've both got a steak to eat, but we're low on silverware.
I've got a fork to eat it with, but you've got the only knife, so I have to wait for you to finish with it before I can cut my meat.
You've got a knife to cut it with, but I've got the only fork, so you have to wait until I'm done before you can grab the fork to hold it down.
Looks like we're gonna go hungry. That's a deadlock.
The standard solution is just to make sure that if anyone ever needs a knife and a fork at the same time, then they have to pick up the fork first.
Simplest way possible - Process A has to wait for process B to finish first, in order for A to finish it's job (let's say A needs the output of B for some calculation of its own) and the same thing happens in B: it has to wait for the output of process A in order to finish itself.
I hope this is clear enough for you. Since you specifically asked for no code, I won't write any code since it's probably too easy to find deadlock code. But basically, it's similar as in the picture of cars blocking each other.
The "lock" in deadlock came from the ability to lock files or database tables to prevent multiple processes from updating the same file/table at the same time.
For example, process 1 locks table A and process 2 locks table B. If process 1 wants to also update table B but finds it locked, then it has to wait for process 2 to release it. If process 2 has to update table A, it would also be locked out.
This situation is a deadlock where both processes are waiting on each other to release their locks.
Related
Two methods Thread#run and Thread#wakeup are different somehow, but it is not clear to me. Can someone provide a pair of code with minimal difference (i.e., difference being only the use of run in one, wakeup in another) that show different results, and possibly explanation for it?
Edit As Cary points out, it is indeed a duplicate of this question, and there is a good answer there, but now I am not sure what it means to have a thread awaken but not running. How is that different from a thread being in sleep situation?
To answer your second question. You cannot schedule a sleeping thread, however you when a thread is awake it can be scheduled to run, even if it isn't being ran currently.
To add to the first part from my understanding of wakeup vs run is that run calls wakeup inside of it, then calls run on it.
Hope that helps.
When I'm debugging, I'm usually looking at about 5000 processes, each of which could be one of about 100 gen_servers, fsms, etc. If I want to know WHAT an erlang process is, I can do:
process_info(pid(0,1,0), initial_call).
And get a result like:
{initial_call,{proc_lib,init_p,5}}
...which is all but useless.
More recently, I hit upon the idea (brace yourselves) of registering each process with a name that told me WHO that process represented. For example, player_1150 is the player process that represents player 1150. Yes, I end up making a couple million atoms over the course of a week-long run. (And I would love to hear comments on the drawbacks of boosting the limit to 10,000,000 atoms when my system runs with about 8GB of real memory unused, if there are any.) Doing this meant that I could, at the console of a live system, query all processes for how long their message queue was, find the top offenders, then check to see if those processes were registered and print out the atom they were registered with.
I've hit a snag with this: I'm moving processes from one node to another. Now a player process can have 3 different names; player_1158, player_1158_deprecating, player_1158_replacement. And I have to make absolutely sure I register and unregister these names with precision timing to make sure that a process is always named and that the appropriate names always exist, AND that I don't try to register a name that some dying process already holds. There is some slop room, since this is only used for console debugging of a live system Nonetheless, the moment I started feeling like this mechanism was affecting how I develop the system (the one that moves processes around) I felt like it was time to do something else.
There are two ideas on the table for me right now. An ets tables that associates process ids with their description:
ets:insert(self(), {player, 1158}).
I don't really like that one because I have to manually keep the tables clean. When a player exits (or crashes) someone is responsible for making sure that his data are removed from the ets table.
The second alternative was to use the process dictionary, storing similar information. When my exploration of a live system led me to wonder who a process is, I could just look at his process dictionary using process_info.
I realize that none of these solutions is functionally clean, but given that the system itself is never, EVER the consumer of these data, I'm not too worried about it. I need certain debugging tools to work quickly and easily, so the behavior described is not open for debate. Are there any convincing arguments to go one way or another (other than the academic "don't use the _, it's evil" canned garbage?) I'd be happy to hear other suggestions and their justifications.
You should try out gproc, it's a very convenient application for keeping process metadata.
A process can be registered with several names and you can associate arbitrary properties to a process (where the key and value can be any erlang term). Also gproc monitors the registered processes and unregisters them automatically if they crash.
If you're debugging gen_servers and gen_fsms while they're still running, I would implement the handle_info functions for these behaviors. When you send each process a {get_info, ReplyPid} tuple, the process in question can send back a term describing its own state, what it is, etc. That way you don't have to keep track of this information outside of the process itself.
Isac mentions there is already a built in way to do this
How would you test Ruby code that has some concurrency features? For instance, let's assume I have a synchronization mechanism that is expected to prevent deadlocks. Is there a viable way to test what it really does? Could controlled execution in fibers be the way forward?
I had the exact same problem and have implemented a simple gem for synchronizing subprocesses using breakpoints: http://github.com/remen/fork_break
I've also documented an advanced usage scenario for rails3 at http://www.hairoftheyak.com/testing-concurrency-in-rails/
I needed to make sure a gem (redis-native_hash) I authored could handle concurrent writes to the same Redis hash, detect the race condition, and elegantly recover. I found that to test this I didn't need to use threads at all.
it "should respect changes made since last read from redis" do
concurrent_edit = Redis::NativeHash.find :test => #hash.key
concurrent_edit["foo"] = "race value"
concurrent_edit.save
#hash["yin"] = "yang"
#hash["foo"] = "bad value"
#hash.save
hash = Redis::NativeHash.find :test => #hash.key
hash["foo"].should == "race value"
hash["yin"].should == "yang"
end
In this test case I just instantiated another object which represents the concurrent edit of the Redis hash, had it make a change, then make sure saving the already-existing object pointing to the same hash respected those changes.
Not all problems involving concurrency can be tested without actually USING concurrency, but in this case it was possible. You may want to try looking for something similar to test your concurrency solutions. If its possible its definitely the easier route to go.
It's definitely a difficult problem. I started writing my test using threads, and realized that they way the code I was testing was implemented, I needed the Process IDs (PID) to actually be different. Threads run using the same PID as the process that kicked off the Thread. Lesson learned.
It was at that point I started exploring forks, and came across this Stack Overflow thread, and played with fork_break. Pretty cool, and easy to set up. Though I didn't need the breakpoints for what I was doing, I just wanted processes to run through concurrently, using breakpoints could be very useful in the future. The problem I ran into was that I kept getting an EOFError and I didn't know why. So I started implementing forking myself, instead of going through fork_break, and found out it was that an exception was happening in the code under test. Sad that the stack trace was hidden from me by the EOFError, though I understand that the child process ended abruptly and that's kinda how it goes.
The next problem I came across was with the DatabaseCleaner. No matter which strategy it used (truncation, or transaction), the child process's data was truncated/rolled back when the child process finished, so the data that was inserted by child processes was gone and the parent process couldn't select and verify that it was correct.
After banging my head on that and trying many other unsuccessful things, I came across this post http://makandracards.com/makandra/556-test-concurrent-ruby-code which was almost exactly what I was already doing, with one little addition. Calling "Process.exit!" at the end of the fork. My best guess (based on my fairly limited understanding of forking) is that this causes the process to end abruptly enough that it completely bypasses any type of database cleanup when the child process ends. So my parent process, the actual test, can continue and verify the data it needs to verify. Then during the normal after hooks of the test (in this case cucumber, but could easily be rspec too), the database cleaner kicks in and cleans up data as it normally would for a test.
So, just thought I'd share some of my own lessons learned in this discusson of how to test concurrent features.
I'm having trouble which boils down to wishing CreateProcess were StartProcess. The trouble is that there are circumstances under which CreateProcess returns true when it created the process but the system could not start the process. For example, CreateProcess will succeed even if one of the launchee's imports cannot be resolved.
There are probably a dozen suggestions one could make depending on what exactly I hope to accomplish by having launched this process. However, I'm afraid none of those suggestions is likely to be useful because I'm not hoping to acccomplish anything in particular by having launched this process.
One example suggestion might be to call WaitForSingleObject against the process handle and then GetExitCodeProcess. But I can't wait for the process to exit because it might stick around forever.
Another example suggestion might be to call WaitForInputIdle, which would work well if I hoped to communicate with the launchee by means of a window I could reasonably expect the launchee to create. But I don't hope that and I can't reasonably expect that. For all I know, the launchee is a console process and/or will never have a message queue. As well, I can't afford to wait around (with heuristic intent) to find out.
In fact, I can't assume anything about the launchee.
To get a better idea of how I'm thinking here, let's look at the flip side of the issue. If the process doesn't start, I want an error code that tells me how I might advise the user. If the imports all resolved and the main thread realizes it's about to jump into the CRT startup code (or equivalent), and the error code I get back is ERROR_SUCCESS, great! But I'm actually disinterested in the launchee and merely wish to provide a good user experience in the launcher.
Oh, and one more thing: I want this to be simple. I don't want to write a debugger. :-)
Ideas?
One example suggestion might be to call WaitForSingleObject against the process handle and then GetExitCodeProcess. But I can't wait for the process to exit because it might stick around forever.
Why don't you wait for the process handle for some reasonable time. If the timer expires before the handle is signaled, you can presume the process is up and running. If the handle is signaled first, and the exit code is good, then you can presume it ran and completed successfully.
In case you haven't seen it, the CreateProcess vs started problem was mentioned in Raymond Chen's blog.
Honestly, if you're not willing to accept heuristics (like, "it hasn't ended with a failure code after three seconds, therefore we assume all is well") then you're going to have to write a 'debugger', by which I mean inspect the internals of the launched process.
This question has gone so long without an answer that I suspect it's safe to conclude that the answer is: "You can't."
Sometimes, we experiment "access denied" errors due to the antivirus which handles the file at the same time our program wants to write/rename/copy it.
This happens rarely but makes me upset because I don't find the good way to deal with: technically our response is to change our source code to implement kind of retry mechanism... but we are not satisfied.. . that smells a little bit... we can't afford telling our customers "please turn off your antivirus, let our software work properly"...
So if your have already experimented such issues, please let me know how you dealt with.
Thanks!
There is really very little scope for saying "turn avs off". That just won't fly in a lot of offices so we've done exactly what you've said: build a retry-queue.
Files that are locked are added to the queue. When the original operation ends, we pause for 1 second and sequentially pop through the queue. Files that fail the second time are added to a second queue and after the first completes, we wait 3 seconds and pop through the second.
Files that fail the second queue (the third attempt) are reported.