Is anyone aware of a way to receive NSURLDownload's delegate methods on a separate thread, i.e. not the main one? I am using an NSOperationQueue to manage them but at the moment I need to use the performSelectorOnMainThread method to get it too work. The problem with this is that it drives the kernel task crazy reaching about 30% of CPU cycles. Curiously this has only happened since upgrading to SL, when NSOperationQueue changed behaviour (not that I am dissing it, GCD rocks!)
Thanks
Colin
My first question is, what are you using NSURLDownload to do? Are you just downloading a bunch of files to the disk, or do you really want the data in memory?
If you're downloading a bunch of files to the disk and you don't want to do any special processing, I'd first try just firing off all the NSURLDownloads on the main thread, without bothering with an NSOperationQueue... I mean, how many operations are we talking about? Can they all run concurrently? The callbacks on the main thread shouldn't be too much of a problem, unless you are doing something heavyweight when you get notified you got some data, in which case it seems like...
Otherwise, I'd switch to using NSURLConnection. It's specifically documented to call you back on the thread you set it up on, and is more flexible. Of course, it's not as high-level, so if you really want files saved to disk, you're going to have to write the I/O yourself. Shouldn't be a huge hardship - it's like four extra lines of code.
-W
NSOperationQueue changed behaviour because it was buggy. It's seems really solid now but yeah, it has a different personality.
Reference (http://www.mikeash.com/?page=pyblog/dont-use-nsoperationqueue.html)
Can you give more info on your problem? Do you only need to notify when the download is finished? Are you doing many downloads at once?
Related
I was attracted to Fyne (and hence Go) by a promise of thread safety. But now that I'm getting better at reading Go I'm seeing things that make be believe that the API as a whole is not thread safe and perhaps was never intended to be. So I'm trying to determine what "thread safe" means in Fyne.
I'm looking specifically at
func (l *Label) SetText(text string) {
l.Text = text
l.textProvider.SetText(text) // calls refresh
}
and noting that l.Text is also a string. Assignments in Go are not thread safe, so it seems obvious to me that if two threads fight over the text of a label and both call label.SetText at the same time, I can expect memory corruption.
"But you wouldn't do that", one might say. No, but I am worried about the case of someone editing the content of an Entry while an app thread decides it needs to replace all the Entry's text - this is entirely possible in my app because it supports simultaneous editing by multiple users over a network, so updates to all sorts of widgets come in asynchronously. (Note I don't care what happens if two people edit the same Entry at the same time; someone's changes will be lost and I don't care who's. But it must not result in memory corruption.) Note that one approach I could take would be to have the background thread create an entirely new Entry widget, which would then replace the one in the current Box. But is that thread safe?
It's not that I don't know how to serialize things with channels. But I was hoping that Fyne would eliminate the need for it (a blog post claims it does); and even using channels I can't convince myself that a user meddling with a widget in various ways while some background thread is altering it, hiding it, etc, isn't going to result in crashes. Maybe all that is serialized under the covers and is perfectly safe, but I don't want to find out the hard way that it isn't, because I'll have no way to fix it.
Fyne is clearly pretty new and seems to have tons of promise, but documentation seems light on details. Is more information available somewhere? Have people tried this successfully?
You have found some race conditions here. There are plans to improve, but the 1.2 release was required to get a new "BaseWidget" first - and that was only released a few weeks ago.
Setting fields directly is primarily for setup purposes and so not expected to be used in the way you illustrate. That said, we do want to support it. The base widget will soon introduce something akin to SetFieldsAndRefresh(func()) which will ensure the safety of the code passed and refresh the widget afterward.
There is indeed a race currently within Refresh(). The use of channels internally were designed to remove this - but there are some corners such as multiple goroutines calling it. This is the area that our new BaseWidget code can help with - as they can internally lock automatically. Using this approach will be thread safe with no changes to the developer in a future release.
The API so far has made it possible for developers to not worry about threading and work from any goroutines - we do need to work internally to make it safer - you are quite right. https://github.com/fyne-io/fyne/issues/506
What are some of the obscure pitfalls of using Core Data and threads? I've read much of the documentation, and so far I've come across the following either in the docs or through painful experience:
Use a new NSManagedObjectContext for each thread, but a single NSPersistentStoreCoordinator is enough for the whole app.
Before sending an NSManagedObject's objectID back to the main thread (or any other thread), be sure the context has been saved (or at a minimum, it wasn't a newly-inserted-but-not-yet-saved object) - otherwise the objectID will actually be a temporary ID and not a persistent one.
Use mergeChangesFromContextDidSaveNotification: to detect when a save happens in another thread and use that to merge those changes with the current thread's context.
Bonus question/observation: I was led to believe by the wording of some of the docs that mergeChangesFromContextDidSaveNotification: is something only needed by the main thread to merge changes into the "main" context from worker threads - but I don't think that's the case.
I set up my importer to create batches of data which are imported using a subclass of an NSOperation that owns it's own context. The operations are loaded into an NSOperationQueue that's set to allow the default number of concurrent operations, so it's possible for several import batches to be running at the same time. I would occasionally get very strange validation errors and exceptions (like trying to add nil to a relationship) and other failures that I had never seen when I did all the same stuff on the main thread. It occurred to me (and perhaps this should have been obvious) that maybe the context merging needed to be done for all contexts in every thread - not just the "main" one! I don't know why I didn't think of that before, but I think this helped. (It hasn't been tested well enough yet for me to feel sure, though.) In any case, is it true that you need to observe that notification for ALL import threads that may be working with the same datasets and adding/updating the same entities? If so, this is yet another pitfall bullet point, IMO, although I have yet to be certain that it'll work.
Given how many of these I've run into with Core Data in general (and not all of them just about multi-threading), I have to wonder how many more are lurking. Since multi-threading so often ends up with bugs that are difficult if not impossible to reproduce due to the timing issues, I figured I'd ask if anyone had other important things that I may be missing that I need to concern myself with.
There is an entire rather large bit of documentation devoted to the subject of Core Data and Threading.
It isn't clear from your set of issues what isn't covered by that documentation.
Does anyone ever see a lot of errors like this:
Exception `Net::HTTPBadResponse' at /usr/lib/ruby/1.8/net/http.rb:2022
- wrong status line: _SOME HTML CODE HERE_
When using threads and mechanize? I'm relatively certain that this is
some bad behavior between threads and the net/http library, but does
anyone have any advice as far as the upper limit of threads you want
to run at once when using mechanize/nethttp? And how can I capture this kind of exception because rescue Net::HTTPBadResponse doesn't work?
This could be something non-thread-safe in Mechanize, but I can think of other bugs that might cause the same problem. I'd start by disabling persistent connections, if you're using them. The next thing to do is to look at your code, and make sure that you're being careful with the objects you handle. If your application has multiple threads mucking about with common objects, that can break a library that would be otherwise thread-safe.
If there is a threading problem somewhere, the upper limit of threads you can use safely is 1. Any more, and you're just making a trade-off about how often you want the problem to occur, rather than whether it occurs or not.
Based on my grueling experience this evening trying to get two Mechanize-based tasks run in tandem in Event Machine and this somewhat ancient exchange, no, it seems it is not thread-safe.
According to this email by Aaron Patterson himself, if you don't share an agent between threads, you should be OK.
IMHO, this means Mechanize is not thread-safe.
I need to do some network bound calls (e.g., fetch a website) and I don't want it to block the UI. Should I be using NSThread's or python's threading module if I am working in pyobjc? I can't find any information on how to choose one over the other. Note, I don't really care about Python's GIL since my tasks are not CPU bound at all.
It will make no difference, you will gain the same behavior with slightly different interfaces. Use whichever fits best into your system.
Learn to love the run loop. Use Cocoa's URL-loading system (or, if you need plain sockets, NSFileHandle) and let it call you when the response (or failure) comes back. Then you don't have to deal with threads at all (the URL-loading system will use a thread for you).
Pretty much the only time to create your own threads in Cocoa is when you have a large task (>0.1 sec) that you can't break up.
(Someone might say NSOperation, but NSOperationQueue is broken and RAOperationQueue doesn't support concurrent operations. Fine if you already have a bunch of NSOperationQueue code or really want to prepare for working NSOperationQueue, but if you need concurrency now, run loop or threads.)
I'm more fond of the native python threading solution since I could join and reference threads around. AFAIK, NSThreads don't support thread joining and cancelling, and you could get a variety of things done with python threads.
Also, it's a bummer that NSThreads can't have multiple arguments, and though there are workarounds for this (like using NSDictionarys and NSArrays), it's still not as elegant and as simple as invoking a thread with arguments laid out in order / corresponding parameters.
But yeah, if the situation demands you to use NSThreads, there shouldn't be any problem at all. Otherwise, it's cool to stick with native python threads.
I have a different suggestion, mainly because python threading is just plain awful because of the GIL (Global Interpreter Lock), especially when you have more than one cpu core. There is a video presentation that goes into this in excruciating detail, but I cannot find the video right now - it was done by a Google employee.
Anyway, you may want to think about using the subprocess module instead of threading (have a helper program that you can execute, or use another binary on the system. Or use NSThread, it should give you more performance than what you can get with CPython threads.
I am developing a multi-threaded application in Cocoa. The main thread takes values from the user, and when a button is clicked I invoke a secondary thread in which a long calculation takes place. Now from this thread I have to return the output of every step of the calculation to the main thread. I want to periodically send data from one thread to the other. I can't find any simple example that does this. Any ideas?
There are a number of ways to do this, in rough order of complexity (easiest first):
use NSObject's performSelectorOnMainThread:withObject:waitUntilDone: which is pretty self explanatory.
use performSelector:onThread:withObject:waitUntilDone:, which will let you go the other way
use an NSNotification (and NSDistributedNotificationCenter), though you can easily run into a race condition if you're not careful
Use NSPorts to send data back and forth
Check out the doc that Abizer mentioned for details on all of these.
performSelectorOnMainThread:withObject:waitUntilDone: is often the easiest way to update the UI with a background thread's progress. You could also create your own storage area that's safe to access between threads using NSLock or a similar mechanism, or even use distributed objects (which also works between processes or over a network).
Then there's NSOperationQueue and NSOperation which does help a lot to simplify multi-threaded programming, although a lot of programmers have been avoiding it since it can cause a crash in certain circumstances under Leopard.
Have a look at the Apple docs for this.
You may need to create an ADC member account, but this is free
Multi-threaded Cocoa Programs