I would like to know how I could run the following loop in a way where it doesn't freeze the GUI, as the loop can take minutes to complete. Thank you.
For i = 0 To imageCount
'code
Next
The short answer is you run the loop on another thread. The long answer is a whole book and a couple of semesters at university, because it entails resource access conflicts and various ways of addressing them such as locking and queueing.
Since you appear to be using VB.NET I suggest you use the latest version of the .NET framework and take advantage of Async and Await, which you can learn about from MSDN.
These keywords implement a very sophisticated canned solution that will allow you to achieve your goals in blissful ignorance of the nightmare behind them :)
Why experienced parallel coders would bother with async/await
Standout features of async/await are
automatic temporary marshalling back to the UI thread as required
scope of exception handlers (try/catch/finally) can span both setup and callback code
you write what is conceptually linear code with blocking calls on the UI thread, but because you declare calls that block using "await", the compiler rewrites your code as a state machine makes the preceding points true
Linear code with blocking calls is easy to write and easy to read. So it's much better from a maintenance perspective. But it provides an atrocious UX. Async/await means you can have it both ways.
All this is built on TPL; in a quite real sense it's nothing more than a compiler supported design pattern for TPL, which is why methods tagged as async are required to return a Task<>. There's so much to love about this, and no technical downside that I've seen.
My only concern is that it's all too good, so a whole generation will have no idea how tall the giants on whose shoulders they perch, just as most modern programmers have only dim awareness of the mechanics of stack frames in call stacks (the magic behind local variables).
You can run the loop on a separate thread. Read about using BackgroundWorker here http://msdn.microsoft.com/en-us/library/system.componentmodel.backgroundworker.aspx
Related
It's possible to write parallel code in Elm? Elm is pure functional, so no locking is needed. Of course, I can use Javascript FFI, spawn workers here and do it on my own. But, I want more user friendly "way" of doing this.
Short answer
No, not currently. But the next release (0.15) release will have new ways to handle effects inside Elm so you will need to use ports + JavaScript code less. So there may well be a way to spawn workers inside Elm in the next version.
More background
If you're feeling adventurous, try reading the published paper on Elm (or the longer original thesis), which shows that the original flavour of FRP that Elm uses is well suited for fine-grained concurrency. There is also an async construct which can potentially make part of the program run separately in a more coarse-grained manner. That might be support with OS-level threads (like JS Webworkers) and parallelism.
There have been earlier experiments with Webworkers. There is certainly an interest in concurrency within the community, but JavaScript doesn't offer (m)any great options for concurrency.
For reading tips on the paper, here's post of mine from the elm-discuss mailing list:
If you want to know more about signals and opt-in async, I suggest you try Evan's PLDI paper on Elm. Read from the introduction (1) up to building GUIs (4). You can skip the type system (3.2) and functional evaluation (3.3.1), that may save you some time. Most in and after building GUIs (4) is probably stuff you know already. Figure 8 is probably the best overview of what the async keyword does (note that the async keyword is not implemented in the current Elm compiler).
A simple search for DoEvents brings up lots of results that lead, basically, to:
DoEvents is evil. Don't use it. Use threading instead.
The reasons generally cited are:
Re-entrancy issues
Poor performance
Usability issues (e.g. drag/drop over a disabled window)
But some notable Win32 functions such as TrackPopupMenu and DoDragDrop perform their own message processing to keep the UI responsive, just like DoEvents does.
And yet, none of these seem to come across these issues (performance, re-entrancy, etc.).
How do they do it? How do they avoid the problems cited with DoEvents? (Or do they?)
DoEvents() is dangerous. But I bet you do lots of dangerous things every day. Just yesterday I set off a few explosive devices (future readers: note the original post date relative to a certain American holiday). With care, we can sometimes account for the dangers. Of course, that means knowing and understanding what the dangers are:
Re-entry issues. There are actually two dangers here:
Part of the problem here has to do with the call stack. If you call .DoEvents() in a loop that itself handles messages that use DoEvents(), and so on, you're getting a pretty deep call stack. It's easy to over-use DoEvents() and accidentally fill up your call stack, resulting in a StackOverflow exception. If you're only using .DoEvents() in one or two places, you're probably okay. If it's the first tool you reach for whenever you have a long-running process, you can easily find yourself in trouble here. Even one use in the wrong place can make it possible for a user to force a stackoverflow exception (sometimes just by holding down the enter key), and that can be a security issue.
It is sometimes possible to find your same method on the call stack twice. If you didn't build the method with this in mind (hint: you probably didn't) then bad things can happen. If everything passed in to the method is a value type, and there is no dependance on things outside of the method, you might be fine. But otherwise, you need to think carefully about what happens if your entire method were to run again before control is returned to you at the point where .DoEvents() is called. What parameters or resources outside of your method might be modified that you did not expect? Does your method change any objects, where both instances on the stack might be acting on the same object?
Performance Issues. DoEvents() can give the illusion of multi-threading, but it's not real mutlithreading. This has at least three real dangers:
When you call DoEvents(), you are giving control on your existing thread back to the message pump. The message pump might in turn give control to something else, and that something else might take a while. The result is that your original operation could take much longer to finish than if it were in a thread by itself that never yields control, definitely longer than it needs.
Duplication of work. Since it's possible to find yourself running the same method twice, and we already know this method is expensive/long-running (or you wouldn't need DoEvents() in the first place), even if you accounted for all the external dependencies mentioned above so there are no adverse side effects, you may still end up duplicating a lot of work.
The other issue is the extreme version of the first: a potential to deadlock. If something else in your program depends on your process finishing, and will block until it does, and that thing is called by the message pump from DoEvents(), your app will get stuck and become unresponsive. This may sound far-fetched, but in practice it's surprisingly easy to do accidentally, and the crashes are very hard to find and debug later. This is at the root of some of the hung app situations you may have experienced on your own computer.
Usability Issues. These are side-effects that result from not properly accounting for the other dangers. There's nothing new here, as long as you looked in other places appropriately.
If you can be sure you accounted for all these things, then go ahead. But really, if DoEvents() is the first place you look to solve UI responsiveness/updating issues, you're probably not accounting for all of those issues correctly. If it's not the first place you look, there are enough other options that I would question how you made it to considering DoEvents() at all. Today, DoEvents() exists mainly for compatibility with older code that came into being before other credible options where available, and as a crutch for newer programmers who haven't yet gained enough experience for exposure to the other options.
The reality is that most of the time, at least in the .Net world, a BackgroundWorker component is nearly as easy, at least once you've done it once or twice, and it will do the job in a safe way. More recently, the async/await pattern or the use of a Task can be much more effective and safe, without needing to delve into full-blown multi-threaded code on your own.
Back in 16-bit Windows days, when every task shared a single thread, the only way to keep a program responsive within a tight loop was DoEvents. It is this non-modal usage that is discouraged in favor of threads. Here's a typical example:
' Process image
For y = 1 To height
For x = 1 to width
ProcessPixel x, y
End For
DoEvents ' <-- DON'T DO THIS -- just put the whole loop in another thread
End For
For modal things (like tracking a popup), it is likely to still be OK.
I may be wrong, but it seems to me that DoDragDrop and TrackPopupMenu are rather special cases, in that they take over the UI, so don't have the reentrancy problem (which I think is the main reason people describe DoEvents as "Evil").
Personally I don't think it's helpful to dismiss a feature as "Evil" - rather explain the pitfalls so that people can decide for themselves. In the case of DoEvents there are rare cases where it's still reasonable to use it, for example while a modal progress dialog is displayed, where the user can't interact with the rest of the UI so there is no re-entrancy issue.
Of course, if by "Evil" you mean "something you shouldn't use without fully understanding the pitfalls", then I agree that DoEvents is evil.
Is there a use case for recursive locks? Is there a scenario that absolutely requires recursive locking.
Seems to be complicated and dangerous to use. I can see that we may avoid deadlocks (provided the lock stack doesn't overflow) but don't we want to catch such problems.
Maybe I'm missing something here. Any pointers are appreciated.
Thanks in advance.
I like the title of this Blog entry:
Recursive locks will kill you
I also like this quote:
http://www.thinkingparallel.com/2006/09/27/recursive-locks-a-blessing-or-a-curse/
Don’t use recursive mutexes. It’s akin to sex with used condoms.
Finally, here's an extremely interesting article about how recursive locks got into Posix pthreads in the first place:
http://groups.google.com/group/comp.programming.threads/msg/d835f2f6ef8aed99?hl=en&pli=1
Recursive mutexes are a hack. There's nothing wrong with using them,
but they're a crutch. Got a broken leg or library? Fine, use the
crutch. But at least be aware that you're using a crutch, and why; and
once in a while check out the leg (or library) to be sure you still
need the crutch. And if it's not healing up, go see a doctor, because
that's just not OK. When you have no choice, there's no shame in using
a crutch... but you can't run very well on a crutch, and you'll also
be slowing down anyone who depends on you.
Recursive mutexes can be a great tool for prototyping thread support
in an existing library, exactly because it lets you defer the hard
part: the call path and data dependency analysis of the library. But
for that same reason, always remember that you're not DONE until
they're all gone, so you can produce a library you're proud of, that
won't unnecessarily contrain the concurrency of the entire
application.
What do you hate most about the modern game loop? Can the game loop be improved or is there just a better alternative, such as an event-driven architecture?
It seems like this really ought to be a CW...
I'm taking a grad-level game engine programming course right now and they're sticking with the game loop approach. Granted, that doesn't mean it's the only/best solution but it's certainly logical. Using a loop allows you to ensure that all game systems get their turn to run without requesting their own timed interrupts or something else. Control can be centralized: in my current project, I have a GameManager class that, each frame, loops through the Update(float deltaTime) function for every registered object in turn. I don't have to debug an event system or set up timed signals, I just use a loop to call a series of functions. No muss, no fuss.
To answer your question of what do I hate most, the loop approach does logically lend itself to liberal use of inheritance and polymorphism which can bloat the size/complexity of your objects. If you're not careful, this can be a mild-to-horrible pitfall. If you are careful, it may not be a problem at all.
No matter there is any event in the game or not, game is supposed to draw itself and update it at a fixed rate, so I don't think dropping the game loop is possible. Still I would be wondered if anyone can think of any other alternative of game loop.
Usually the event driven architectures work best for games (only do something if the user wants something done). BUT, you'll still always need to have a thread constantly redrawing the world.
To be fully event based you could spawn an extra thread that does nothing but put a CPUTick event into your event queue every x milliseconds.
In JavaScript this is usually the more natural route, because you can so easily create an extra 'thread' that sends you the events with setInterval().
Or, if you already have a loop in the framework containing your game - like JS has in the browser, or python has with twisted - you can tell that looper to call you back at fixed intervals. e.g.:
function gameLoop() {
// update, draw...
}
window.setInterval(gameLoop, 1000/fps);
I need to do some network bound calls (e.g., fetch a website) and I don't want it to block the UI. Should I be using NSThread's or python's threading module if I am working in pyobjc? I can't find any information on how to choose one over the other. Note, I don't really care about Python's GIL since my tasks are not CPU bound at all.
It will make no difference, you will gain the same behavior with slightly different interfaces. Use whichever fits best into your system.
Learn to love the run loop. Use Cocoa's URL-loading system (or, if you need plain sockets, NSFileHandle) and let it call you when the response (or failure) comes back. Then you don't have to deal with threads at all (the URL-loading system will use a thread for you).
Pretty much the only time to create your own threads in Cocoa is when you have a large task (>0.1 sec) that you can't break up.
(Someone might say NSOperation, but NSOperationQueue is broken and RAOperationQueue doesn't support concurrent operations. Fine if you already have a bunch of NSOperationQueue code or really want to prepare for working NSOperationQueue, but if you need concurrency now, run loop or threads.)
I'm more fond of the native python threading solution since I could join and reference threads around. AFAIK, NSThreads don't support thread joining and cancelling, and you could get a variety of things done with python threads.
Also, it's a bummer that NSThreads can't have multiple arguments, and though there are workarounds for this (like using NSDictionarys and NSArrays), it's still not as elegant and as simple as invoking a thread with arguments laid out in order / corresponding parameters.
But yeah, if the situation demands you to use NSThreads, there shouldn't be any problem at all. Otherwise, it's cool to stick with native python threads.
I have a different suggestion, mainly because python threading is just plain awful because of the GIL (Global Interpreter Lock), especially when you have more than one cpu core. There is a video presentation that goes into this in excruciating detail, but I cannot find the video right now - it was done by a Google employee.
Anyway, you may want to think about using the subprocess module instead of threading (have a helper program that you can execute, or use another binary on the system. Or use NSThread, it should give you more performance than what you can get with CPython threads.