Right now I'm working on a program using IOBluetooth and I need to have synchronous reads, i.e. I call a method, it writes a given number of bytes to the port, then reads a given number and returns them. I currently have a complex system of NSThreads, NSLocks, and NSConditions that, while it sort of works, is very slow. Also, after certain calls, I need to make sure there's no extra data, so I'd normally flush the buffer, but with IOBluetooth's asynchronous callback that's not possible - any thoughts on how to make sure that no matter what, all data received after a specific point is data that's received after that point?
I really haven't dealt at all with synchronization and multithreading of this type, since all the work I've done so far is using synchronous calls, so I'd appreciate any thoughts on the matter.
Here's the callback for incoming data (the "incomingData" object is NSMutableData):
- (void)rfcommChannelData:(IOBluetoothRFCOMMChannel*)rfcommChannel data:(void *)dataPointer length:(size_t)dataLength {
[dataLock lock];
NSData *data = [NSData dataWithBytes:dataPointer length:dataLength];
[incomingData appendData:data];
if (dataWaitCondition && [incomingData length] >= bytesToWaitFor) {
[dataWaitCondition signal];
}
[dataLock unlock];
[delegate bluetoothDataReceived];
}
And here's the method that waits until the given number of bytes has been received before returning the data object (this gets called from an alternate thread).
- (NSData *)waitForBytes:(int)numberOfBytes {
bytesToWaitFor = numberOfBytes;
[dataLock lock];
dataWaitCondition = [[NSCondition alloc] init];
[dataWaitCondition lock];
[dataLock unlock];
while ([incomingData length] < numberOfBytes) {
[dataWaitCondition wait];
}
[dataLock lock];
NSData *data = [incomingData copy];
[dataWaitCondition unlock];
dataWaitCondition = NULL;
[dataLock unlock];
return data;
}
Doing any kind of IO/communication in a synchronous way will get you into trouble.
You can avoid multi threading and locks by using a simple state machine for your application logic. Whenever data is received, the state machine is triggered and can process the data. If all data is there, you can do the next step in your app. You can use synchronous calls for sending if you like as it will just drop the data with the Bluetooth system.
Related
I am seeing an issue with RestKit where an outstanding remote request will cause other requests to queue up and block until the previous one completes. I have read that RestKit defaults AFNetworking's setMaxConcurrentOperationCount value to 5, but I am encountering blocking with just a single outstanding request.
Does the current version of RestKit still default this to 5, and if so, what could be causing my lack of concurrent requests?
RestKit 0.20.3
Request
- (RKManagedObjectRequestOperation*) getUser {
RKObjectManager *objectManager = [RKObjectManager sharedManager];
NSDictionary *items = [objectManager.HTTPClient defaultHeaders];
NSString *auth = [items objectForKey:#"Authorization"];
if (auth == nil) {
NSLog(#"You must login first\n");
return nil;
}
auth = [auth stringByReplacingOccurrencesOfString:#"Bearer " withString:#""];
NSMutableString *path = [NSMutableString stringWithFormat:#"/users?accessToken=%#", auth];
return [[RKObjectManager sharedManager] appropriateObjectRequestOperationWithObject:nil method:RKRequestMethodGET path:path parameters:nil];
}
- (void) getUser:(void ( ^ ) ( RKObjectRequestOperation *operation , RKMappingResult *mappingResult ))success failure:(void ( ^ ) ( RKObjectRequestOperation *operation , NSError *error ))failure {
RKManagedObjectRequestOperation *requestOperation = [self getUser];
[requestOperation setCompletionBlockWithSuccess:success failure:failure];
[requestOperation start];
}
The former is a helper used in testing, and the latter is actually called by Production code. I don't see any place in my code where the Queue size is set explicitly.
RKObjectRequestOperation contains an internal queue (responseMappingQueue) which is shared amongst all instances. This queue is explicitly set with a concurrency count of 1, but it's just used for mapping, not for the URL connections.
Looking at what you're doing, calling start on the operation directly, you most likely want to think about adding the operation to a queue so that you can manage them better. Because you aren't using the object manager to run the operations the concurrent count of 5 that you refer to doesn't apply. Look at using enqueueObjectRequestOperation: to run the operations.
If I have two different threads via GCD accessing an NSMutableArray and one is merely creating a new array based off the mutable array while the other thread is deleting records from the array, should I expect this to be a problem? That is, shouldn't the copy, which I presume is merely "reading" the array, just get whatever happens to be in the array at that moment? I am not enumerating the array in either thread, but it is still crashing. As soon as I remove the read routine, it works fine.
Here is the "read" :
dispatch_async(saveQueue, ^{
NSDictionary*tempstocks=[NSDictionary dictionaryWithDictionary:self.data];
It crashes on this thread with: *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '*** -[__NSPlaceholderDictionary initWithObjects:forKeys:count:]: attempt to insert nil object from objects[9]'
Here is what is happening on another thread:
[self.data removeObjectForKey:item];
I know you cannot mutate while enumerating, but I'd think it would be okay to read while mutating, you might not know which version of the mutated object you get, but I wouldn't think this is a problem, but clearly it is. Perhaps the dictionaryWithDictionary method is performing an operation that first sees X objects but by the time the routine is done it contains X-Y objects, thus it is not "capturing" the entire self.data dictionary in one snap when it runs dictionaryWithDictionary and is instead enumerating over self.data which would essentially be the same problem as mutation while enumeration?
I guess that you might create three different queues using GCD: one for save, second one for something else and last one to operate with NSMutableArray.
dispatch_async(saveQueue, ^{
dispatch_barrier_async(_queue, ^{
NSDictionary*tempstocks=[NSDictionary dictionaryWithDictionary:self.data];
});
});
dispatch_async(anotherQueue, ^{
dispatch_barrier_async(_queue, ^{
[self.data removeObjectForKey:item];
});
});
It's like #synchronize but using GCD.
More info: GCD Reference/dispatch_barrier_async and http://www.mikeash.com/pyblog/friday-qa-2011-10-14-whats-new-in-gcd.html
EDIT
I have made a couple of performance test in order to understand which of the way is faster:
- (void)usingSynchronized
{
dispatch_queue_t writeQyeue = dispatch_queue_create("com.tikhop.writeQyeue", DISPATCH_QUEUE_CONCURRENT);
dispatch_sync(writeQyeue, ^{
for(size_t i=0; i<10000; i++)
#synchronized (arr) {
[arr replaceObjectAtIndex:0 withObject:[NSNumber numberWithInt:1]];
[arr replaceObjectAtIndex:0 withObject:[NSNumber numberWithInt:2]];
[arr replaceObjectAtIndex:0 withObject:[NSNumber numberWithInt:3]];
[arr replaceObjectAtIndex:0 withObject:[NSNumber numberWithInt:4]];
}
});
}
- (void)usingGCD
{
dispatch_queue_t writeQyeue = dispatch_queue_create("com.tikhop.writeQyeue", DISPATCH_QUEUE_CONCURRENT);
dispatch_sync(writeQyeue, ^{
for(size_t i=0; i<10000; i++)
dispatch_barrier_async(_queue, ^{
[arr replaceObjectAtIndex:0 withObject:[NSNumber numberWithInt:5]];
[arr replaceObjectAtIndex:0 withObject:[NSNumber numberWithInt:6]];
[arr replaceObjectAtIndex:0 withObject:[NSNumber numberWithInt:7]];
[arr replaceObjectAtIndex:0 withObject:[NSNumber numberWithInt:8]];
});
});
}
arr = [NSMutableArray arrayWithCapacity:1];
[arr addObject:#(0)];
[self usingSynchronized];
[self usingGCD];
I got the following result:
You cannot assume that any operation on NSDictionary is thread-safe. And almost all f them are not. You really need to set up a mutex, #synchronize access to your array or use a gcd serial queue for access.
dictionaryWithDictionary: is internally enumerating the argument, so you are basically mutating while enumerating.
Also, in general, you should never write to an object if another thread is going to access it in any way unless you use some sort of synchronization primitive.
Your reasoning that it "reads" whatever it's there at the moment is not valid in general. Here is a little more info on the problems inherent in multithreading Usage of registers by the compiler in multithreaded program
I need to see if an object that I have obtained from a CSV file with a unique identifier exists in my Core Data Database, and this is the code I deemed suitable for this task:
NSFetchRequest *fetchRequest = [[NSFetchRequest alloc] init];
NSEntityDescription *entity;
entity =
[NSEntityDescription entityForName:#"ICD9"
inManagedObjectContext:passedContext];
[fetchRequest setEntity:entity];
NSPredicate *pred = [NSPredicate predicateWithFormat:#"uniqueID like %#", uniqueIdentifier];
[fetchRequest setPredicate:pred];
NSError *err;
NSArray* icd9s = [passedContext executeFetchRequest:fetchRequest error:&err];
[fetchRequest release];
if ([icd9s count] > 0) {
for (int i = 0; i < [icd9s count]; i++) {
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc]init];
NSString *name = [[icd9s objectAtIndex:i] valueForKey:#"uniqueID"];
if ([name caseInsensitiveCompare:uniqueIdentifier] == NSOrderedSame && name != nil)
{
[pool release];
return [icd9s objectAtIndex:i];
}
[pool release];
}
}
return nil;
After more thorough testing it appears that this code is responsible for a huge amount of leaking in the app I'm writing (it crashes on a 3GS before making it 20 percent through the 1459 items). I feel like this isn't the most efficient way to do this, any suggestions for a more memory efficient way? Thanks in advance!
Don't use the like operator in your request predicate. Use =. That should be much faster.
You can specify the case insensitivity of the search via the predicate, using the [c] modifier.
It's not necessary to create and destroy an NSAutoreleasePool on each iteration of your loop. In fact, it's probably not needed at all.
You don't need to do any of the checking inside the for() loop. You're duplicating the work of your predicate.
So I would change your code to be:
NSFetchRequest *fetchRequest = [[NSFetchRequest alloc] init];
[fetchRequest setEntity:...];
[fetchRequest setPredicate:[NSPredicate predicateWithFormat:#"uniqueID =[c] %#", uniqueIdentifier]];
NSError *err = nil;
NSArray *icd9s = [passedContext executeFetchRequest:fetchRequest error:&err];
[fetchRequest release];
if (error == nil && [icd9s count] > 0) {
return [icd9s objectAtIndex:0]; //we know the uniqueID matches, because of the predicate
}
return nil;
Use the Leaks template in Instruments to hunt down the leak(s). Your current code may be just fine once you fix them. The leak(s) may even be somewhere other than code.
Other problems:
Using fast enumeration will make the loop over the array (1) faster and (2) much easier to read.
Don't send release to an autorelease pool. If you ever port the code to garbage-collected Cocoa, the pool will not do anything. Instead, send it drain; in retain-release Cocoa and in Cocoa Touch, this works the same as release, and in garbage-collected Cocoa, it pokes the garbage collector, which is the closest equivalent in GC-land to draining the pool.
Don't repeat yourself. You currently have two [pool release]; lines for one pool, which gets every experienced Cocoa and Cocoa Touch programmer really worried. Store the result of your tests upon the name in a Boolean variable, then drain the pool before the condition, then conditionally return the object.
Be careful with variable types. -[NSArray count] returns and -[NSArray objectAtIndex:] takes an NSUInteger, not an int. Try to keep all your types matching at all times. (Switching to fast enumeration will, of course, solve this instance of this problem in a different way.)
Don't hide releases. I almost accused you of leaking the fetch request, then noticed that you'd buried it in the middle of the code. Make your releases prominent so that you're less likely to accidentally add redundant (i.e., crash-inducing) ones.
Can anyone explain with really clear use cases what the purpose of dispatch_sync in GCD is for? I can't understand where and why I would have to use this.
Thanks!
You use it when you want to execute a block and wait for the results.
One example of this is the pattern where you're using a dispatch queue instead of locks for synchronization. For example, assume you have a shared NSMutableArray a, with access mediated by dispatch queue q. A background thread might be appending to the array (async), while your foreground thread is pulling the first item off (synchronously):
NSMutableArray *a = [[NSMutableArray alloc] init];
// All access to `a` is via this dispatch queue!
dispatch_queue_t q = dispatch_queue_create("com.foo.samplequeue", NULL);
dispatch_async(q, ^{ [a addObject:something]; }); // append to array, non-blocking
__block Something *first = nil; // "__block" to make results from block available
dispatch_sync(q, ^{ // note that these 3 statements...
if ([a count] > 0) { // ...are all executed together...
first = [a objectAtIndex:0]; // ...as part of a single block...
[a removeObjectAtIndex:0]; // ...to ensure consistent results
}
});
First understand its brother dispatch_async
//Do something
dispatch_async(queue, ^{
//Do something else
});
//Do More Stuff
You use dispatch_async to create a new thread. When you do that, the current thread will not stop. That means //Do More Stuff may be executed before //Do something else finish
What happens if you want the current thread to stop?
You do not use dispatch at all. Just write the code normally
//Do something
//Do something else
//Do More Stuff
Now, say you want to do something on a DIFFERENT thread and yet wait as if and ensure that stuffs are done consecutively.
There are many reason to do this. UI update, for example, is done on main thread.
That's where you use dispatch_sync
//Do something
dispatch_sync(queue, ^{
//Do something else
});
//Do More Stuff
Here you got //Do something //Do something else and //Do More stuff done consecutively even though //Do something else is done on a different thread.
Usually, when people use different thread, the whole purpose is so that something can get executed without waiting. Say you want to download large amount of data but you want to keep the UI smooth.
Hence, dispatch_sync is rarely used. But it's there. I personally never used that. Why not ask for some sample code or project that does use dispatch_sync.
dispatch_sync is semantically equivalent to a traditional mutex lock.
dispatch_sync(queue, ^{
//access shared resource
});
works the same as
pthread_mutex_lock(&lock);
//access shared resource
pthread_mutex_unlock(&lock);
David Gelhar left unsaid that his example will work only because he quietly created serial queue (passed NULL in dispatch_queue_create what is equal to DISPATCH_QUEUE_SERIAL).
If you wish create concurrent queue (to gain all of multithread power), his code will lead to crash because of NSArray mutation (addObject:) during mutation (removeObjectAtIndex:) or even bad access (NSArray range beyond bounds). In that case we should use barrier to ensure exclusive access to the NSArray while the both blocks run. Not only does it exclude all other writes to the NSArray while it runs, but it also excludes all other reads, making the modification safe.
Example for concurrent queue should look like this:
NSMutableArray *a = [[NSMutableArray alloc] init];
// All access to `a` is via this concurrent dispatch queue!
dispatch_queue_t q = dispatch_queue_create("com.foo.samplequeue", DISPATCH_QUEUE_CONCURRENT);
// append to array concurrently but safely and don't wait for block completion
dispatch_barrier_async(q, ^{ [a addObject:something]; });
__block Something *first = nil;
// pop 'Something first' from array concurrently and safely but wait for block completion...
dispatch_barrier_sync(q, ^{
if ([a count] > 0) {
first = [a objectAtIndex:0];
[a removeObjectAtIndex:0];
}
});
// ... then here you get your 'first = [a objectAtIndex:0];' due to synchronised dispatch.
// If you use async instead of sync here, then first will be nil.
If you want some samples of practical use look at this question of mine:
How do I resolve this deadlock that happen ocassionally?
I solve it by ensuring that my main managedObjectContext is created on the main thread. The process is very fast and I do not mind waiting. Not waiting means I will have to deal with a lot of concurency issue.
I need dispatch_sync because some code need to be done on main thread, which is the different thread than the one where to code is being executed.
So basically if you want the code to
1. Proceed like usual. You don't want to worry about race conditions. You want to ensure that the code is completed before moving on.
2. Done on a different thread
use dispatch_sync.
If 1 is violated, use dispatch_async. If 2 is violated just write the code like usual.
So far, I only do this once, namely when something need to be done on main thread.
So here's the code:
+(NSManagedObjectContext *)managedObjectContext {
NSThread *thread = [NSThread currentThread];
//BadgerNewAppDelegate *delegate = [BNUtilitiesQuick appDelegate];
//NSManagedObjectContext *moc = delegate.managedObjectContext;
if ([thread isMainThread]) {
//NSManagedObjectContext *moc = [self managedObjectContextMainThread];
return [self managedObjectContextMainThread];
}
else{
dispatch_sync(dispatch_get_main_queue(),^{
[self managedObjectContextMainThread];//Access it once to make sure it's there
});
}
// a key to cache the context for the given thread
NSMutableDictionary *managedObjectContexts =[self thread].managedObjectContexts;
#synchronized(self)
{
if ([managedObjectContexts objectForKey:[self threadKey]] == nil ) {
NSManagedObjectContext *threadContext = [[NSManagedObjectContext alloc] initWithConcurrencyType:NSPrivateQueueConcurrencyType];
threadContext.parentContext = [self managedObjectContextMainThread];
//threadContext.persistentStoreCoordinator= [self persistentStoreCoordinator]; //moc.persistentStoreCoordinator;// [moc persistentStoreCoordinator];
threadContext.mergePolicy = NSMergeByPropertyObjectTrumpMergePolicy;
[managedObjectContexts setObject:threadContext forKey:[self threadKey]];
}
}
return [managedObjectContexts objectForKey:[self threadKey]];
}
dispatch_sync is mainly used inside dispatch_async block to perform some operations on main thread(like update ui).
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
//Update UI in main thread
dispatch_sync(dispatch_get_main_queue(), ^{
self.view.backgroundColor = color;
});
});
Here's a half-way realistic example. You have 2000 zip files that you want to analyze in parallel. But the zip library isn't thread-safe. Therefore, all work that touches the zip library goes into the unzipQueue queue. (The example is in Ruby, but all calls map directly to the C library. "apply", for example, maps to dispatch_apply(3))
#!/usr/bin/env macruby -w
require 'rubygems'
require 'zip/zipfilesystem'
#unzipQueue = Dispatch::Queue.new('ch.unibe.niko.unzipQueue')
def extractFile(n)
#unzipQueue.sync do
Zip::ZipFile.open("Quelltext.zip") { |zipfile|
sourceCode = zipfile.file.read("graph.php")
}
end
end
Dispatch::Queue.concurrent.apply(2000) do |i|
puts i if i % 200 == 0
extractFile(i)
end
I've used dispatch sync when inside an async dispatch to signal UI changes back to the main thread.
My async block holds back only a little and I know the main thread is aware of the UI changes and will action them. Generally used this in a processing block of code that takes some CPU time but I still want to action UI changes from within that block. Actioning the UI changes in the async block is useless as UI, I believe, runs on the main thread. Also actioning them as secondary async blocks, or a self delegate, results in the UI only seeing them a few seconds later and it looks tardy.
Example block:
dispatch_queue_t myQueue = dispatch_queue_create("my.dispatch.q", 0);
dispatch_async(myQueue,
^{
// Do some nasty CPU intensive processing, load file whatever
if (somecondition in the nasty CPU processing stuff)
{
// Do stuff
dispatch_sync(dispatch_get_main_queue(),^{/* Do Stuff that affects UI Here */});
}
});
So I have this piece of code :
if ([receivedPage hasPrefix:[NSString stringWithUTF8String:"\xC3\xAF\xC2\xBB\xC2\xBF"]]) // UTF-8 BOM 'EF BB BF' as UTF-16 chars
{
//DebugLog(#"converting calls list to UTF8");
receivedPage = [[[NSString alloc] initWithData:[receivedPage dataUsingEncoding:NSISOLatin1StringEncoding] encoding:NSUTF8StringEncoding] autorelease];
}
However sometimes when the if is true the receivedPage becomes null. why would this happen?
The received page is the returned value of this function:
NSURLResponse * response;
NSData * result = [NSURLConnection sendSynchronousRequest:request returningResponse:&response error:error];
if ([result length] > 0)
return [[[NSString alloc] initWithBytes: (const void*)[result bytes] length:[result length] encoding: encoding] autorelease];
else
{
if (error && *error)
DebugLog(#"URL request got error: %#",*error);
return nil;
}
The encoding here is NSISOLatin1StringEncoding (don't know why ,I'm debugging someone else's code).
Any idea why this would happen?
It looks like you're trying to treat strings (objects containing characters) as data (objects containing bytes). Keep the data you received from the connection, and check for the UTF-8 BOM (the proper three-byte version) in it, then use either NSUTF8StringEncoding or NSISOLatin1StringEncoding based on whether you find it.
Or, just use UTF-8 conditionally, if you can fix the server to do that as well.
Also, you should probably switch this code to use the NSURLConnection asynchronously. If the user's internet connection is slow, you're hanging your app here. Doing it asynchronously lets you keep the UI running, display progress if appropriate, and enable the user to cancel.