GCDAsyncSocket didReadData only gets called once - gcdasyncsocket

I am trying to set up a Java server talking to a iPhone client using GCDAsyncSocket. For some reason my client code on the iPhone is not reading back all of the data.
I see didReadData gets called the first time, but never again. Ideally, I need to mimic the functionality of the HTTP protocol where it sends a header and then the payload. The size of the payload would be in the header. But that wasn't working, so I simplified my code even further in hopes of finding the issue. Below is the code, and below that the output.
client:
- (BOOL) sendString:(NSString *) string
{
[asyncSocket writeData:[string dataUsingEncoding:NSUTF8StringEncoding]
withTimeout:-1 tag:TAG_PAYLOAD];
[asyncSocket readDataToLength:1 withTimeout:(-1) tag:TAG_HEADER];
}
- (void) socket:(GCDAsyncSocket *) sock didReadData:(NSData *)data
withTag:(long)tag
{
NSString *str = [[NSString alloc] initWithData:data
encoding:NSUTF8StringEncoding];
NSLog(#"Read data %# with tag: %ld", str, tag);
if(tag == TAG_HEADER)
{
//TODO - parse the header, get the fields
[sock readDataToLength:3 withTimeout:-1 tag:TAG_PAYLOAD];
//[sock readDataToData:[GCDAsyncSocket CRLFData] withTimeout:-1
tag:TAG_PAYLOAD];
}
else
{
NSLog(#"Payload... %#", str);
NSLog(#"Tag: %ld", tag);
}
}
Java server:
BufferedReader in = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
PrintWriter out = new PrintWriter(new OutputStreamWriter(clientSocket.getOutputStream()));
String clientCommand = in.readLine();
System.out.println("Client Says :" + clientCommand);
out.println("Andrew\r\nAUSTIN\r\n");
out.flush();
However, on the client, the only thing I get back is A. The exact output is:
Read data A with tag: 10
My question is:
How come the didReadData method is never called again? There should be more of "Andrew\r\nAustin" recv'd back on the client. But it just hangs. The readDataToData and readDataToLength both seem to never get the full string.
I noticed the CRLF defined in GCDAsyncSocket.h is not \r\n but instead the hex values. Does this matter? Thats why I tried the readDataToLength method but that still failed. But I would like to know if this matters cross-platform or not.
Thanks.

OK - so I figured it out after pulling out what little hair I have left.
What is happening is that I have client code above in a separate class outside of the view. Practically all of the examples I came across had the GCDAsyncSocket stuff handled inside the view. It works great in there! I really didn't want to do this because on each view I need to send/read data and didn't want to duplicate my work. By placing an NSLog() line in the dealloc method of this helper class, called SocketComm, I was able to see it was getting deallocated before it was firing. So I needed to change the way I was calling my helper class. I declare SocketComm* sockComm a strong property in the viewController.h file and allocated it in the viewDidLoad() method. This means that it stays in scope the whole time. Of course, this means I need to deallocate it manually and do some other housekeeping things.
I still am not sure if this is the best way to handle this situation either, as far as memory management goes. Because now I will have to alloc this on every viewDidLoad method. It seems like it should be simpler than this, but here we are. And I still don't know why it never read the data the first time (my only guess is that the GCDAsyncSocket library or the iphone software detected a dead thread when the parent that spawned it got deallocated and decided to terminate it - but this is only a guess as I have just started objective-c).
This would also explain why sometimes it would work and sometimes it wouldn't. It seemed like it was in a race condition. Not sure if the above code I originally posted resulted in a race condition exactly, but some things I would try would work, and then the next time fail. It never read more than the first time though, and only about half the time would it even read that. Sometimes it wouldn't even send the data out over the socket!
In summation (and for whoever else comes looking for an answer):
Always check your memory management. I had to place an NSLog in dealloc() of the SocketComm helper class to fully see what was happening, and as soon as I did that I knew what the culprit was.
If you get weird results where sometimes it works and sometimes it doesn't, check your memory management. For me, sometimes it would do the first read and sometimes it wouldn't. This lead me to believe the thread was getting terminated.
If I find a better way to do this I will come back and update this answer.
Memory management. Let me repeat: memory management.

Related

Why do I sometimes get mangled replies with concurrent NSURLSession requests

I am working on an OS X (Yosemite) app which downloads two types of csv data (call them type A and type B) from the internet asynchronously using the NSURLSession API .There are multiple requests for each type of csv. Each request is it's own dedicated session wrapped in a custom class. There is one base request class with a subclass for each type. (In hindsight maybe not an ideal design but irrelevant for my issue I think).
The app is constructed such that each type of csv data is downloaded in a sequential queue. Only one request of each type can be active at a time but both types can occur simultaneously and both use the main thread for delegate callbacks. All of this works fine usually.
The issue I am seeing is that sometimes with heavy traffic I get "cross hearing", i.e. I sometimes get a response back to a type B request that is reported as completed successfully but it contains a number of type B cvs lines and then some type A lines tagged on after - so I sometimes (rarely) get type A data in my type B requests. (or the other way around).
Basically it look like the "switching" logic in Apples API gets confused about which incoming packet belongs to what request/session. The two different request types goes to different URLs but they are related and it may be that they both in the end resolve to the same IP, I am not sure about that. I wonder if there may be something related to the packet headers if they come from the same server that makes it difficult to determine what request they belong to (I'm not good enough at the internet protocols to know if this is a sensible guess). If that is the case then the solution must be to ensure all requests are in one queue so that they cannot be active simultaneously, but I do not want to do that large architecture change before I am confident there is no other workaround.
I looked for similar questions and found this old question (Why is my data getting corrupted when I send requests asynchronously in objective c for iOS?) which appears to describe the exact same issue but unfortunately it has no answer. Other than that I found nothing similar so I guess I am doing something stupid here but it would be good to know why this issue occurs before I start changing the architecture to fix it.
Has anyone seen this before and know what the cause and workaround is?
I did not include any code as I felt there was no point given it appears to be an architecture issue and if I added code it would need to be a lot. However I will be happy to add whatever you suggest if that helps understand the question.
Edit:
The relevant (I hope) code added below. Note objects are one shot only. The parameters for the request are injected by the init method and the NSURLSession is used for a single task only. Hence the session is invalidated after launch and the NSMutableData array released after parsing of the data.
-(BOOL)executeRequest {
NSURLSessionConfiguration *theConfig = [NSURLSessionConfiguration ephemeralSessionConfiguration];
NSURLSession *theSession = [NSURLSession sessionWithConfiguration:theConfig delegate:self delegateQueue:[NSOperationQueue mainQueue]];
NSURLRequest *theRequest = [NSURLRequest requestWithURL:self.queryURL cachePolicy: NSURLRequestReloadIgnoringLocalCacheData timeoutInterval:BSTTIMEOUT];
NSURLSessionDataTask *theTask = [theSession dataTaskWithRequest:theRequest];
if(!theTask) {
return NO;
}
[theTask resume];
[theSession finishTasksAndInvalidate];
self.internetData = [NSMutableData dataWithCapacity:0];
return YES;
}
-(void)URLSession:(NSURLSession *)session dataTask:(NSURLSessionDataTask *)dataTask didReceiveData:(NSData *)data {
[self.internetData appendData:data];
return;
}
-(void)URLSession:(NSURLSession *)session task:(NSURLSessionTask *)task didCompleteWithError:(NSError *)error {
if((error)||(![self parseData]))
{
self.internetData = nil;
if(!error) {
NSDictionary *errorDictionary = #{ NSLocalizedDescriptionKey : #"Parsing of internet data failed", NSLocalizedFailureReasonErrorKey : #"Bad data was found in received buffer"};
error = [NSError errorWithDomain:NSCocoaErrorDomain code:EIO userInfo:errorDictionary];
}
NSDictionary* ui = [NSDictionary dictionaryWithObject:error forKey:#"Error"];
[[NSNotificationCenter defaultCenter] postNotificationName:[self failNotification] object:self userInfo:ui];
return;
}
[[NSNotificationCenter defaultCenter] postNotificationName:[self successNotification] object:self];
return;
}
First of all: You should not create a new session for each request. This are no sessions anymore. From the docs:
With the NSURLSession API, your app creates one or more sessions, each of which coordinates a group of related data transfer tasks. For example, if you are writing a web browser, your app might create one session per tab or window, or one session for interactive use and another session for background downloads. Within each session, your app adds a series of tasks, each of which represents a request for a specific URL (following HTTP redirects if necessary).
Second: Where do you store the session et al., so it is not deallocated?
Your main problem: Obviously you start new requests while requests are potentially running. But you have only one NSMutableData instance that receives the data in -URLSession:task:didReceiveData:: Many requests, one storage … Of course that mixes up.
I finally managed to track down my (stupid) error. For future reference the issue was caused by a failure to realise that the data coming back was not zero terminated.
Most of the data requested in my case is XML and the NSXMLParserclass wants a NSDatawithout extra trailing zeros so that works well.
But the requests which occasionally failed uses a CSV format where the data passes over a NSStringwhich is created by [NSString stringWithUTF8String] which expects a zero terminated c style string as input. This was the main culprit. Often it worked as it should. Sometimes it failed outright and sometimes it just did a buffer overrun and got some of the previous request data that was in the same memory area. These were the cases I noticed when posting the question.
Thus the solution is to switch to the use of [[NSString alloc] initWithData: encoding:NSUTF8StringEncoding] which works with non null-terminated NSDatabuffers.

What is considered overloading the main thread?

I am displaying information from a data model on a user interface. My current approach to doing so is by means of delegation as follows:
#protocol DataModelDelegate <NSObject>
- (void)updateUIFromDataModel;
#end
I am implementing the delegate method in my controller class as follows, using GCD to push the UI updating to the main thread:
- (void)updateUIFromDataModel {
dispatch_async(dispatch_get_main_queue(), ^{
// Code to update various UI controllers
// ...
// ...
});
}
What I am concerned about is that in some situations, this method can be called very frequently (~1000 times per second, each updating multiple UI objects), which to me feels very much like I am 'spamming' the main thread with commands.
Is this too much to be sending to the main thread? If so does anyone have any ideas on what would be the best way of approaching this?
I have looked into dispatch_apply, but that appears to be more useful when coalescing data, which is not what I am after - I really just want to skip updates if they are too frequent so only a sane amount of updates are sent to the main thread!
I was considering taking a different approach and implementing a timer instead to constantly poll the data, say every 10 ms, however since the data updating tends to be sporadic I feel that it would be wasteful to do so.
Combining both approaches, another option I have considered would be to wait for an update message and respond by setting the timer to poll the data at a set interval, and then disabling the timer if the data appears to have stopped changing. But would this be over-complicating the issue, and would the sane approach be to simply have a constant timer running?
edit: Added an answer below showing the adaptations using a dispatch source
One option is to use a Dispatch Source with type DISPATCH_SOURCE_TYPE_DATA_OR which lets you post events repeatedly and have libdispatch combine them together for you. When you have something to post, you use dispatch_source_merge_data to let it know there's something new to do. Multiple calls to dispatch_source_merge_data will be coalesced together if the target queue (in your case, the main queue) is busy.
I have been experimenting with dispatch sources and got it working as expected now - Here is how I have adapted my class implementation in case it is of use to anyone who comes across this question:
#implementation AppController {
#private
dispatch_source_t _gcdUpdateUI;
}
- (void)awakeFromNib {
// Added the following code to set up the dispatch source event handler:
_gcdUpdateUI = dispatch_source_create(DISPATCH_SOURCE_TYPE_DATA_ADD, 0, 0,
dispatch_get_main_queue());
dispatch_source_set_event_handler(_gcdUpdateUI, ^{
// For each UI element I want to update, pull data from model object:
// For testing purposes - print out a notification:
printf("Data Received. Messages Passed: %ld\n",
dispatch_source_get_data(_gcdUpdateUI));
});
dispatch_resume(_gcdUpdateUI);
}
And now in the delegate method I have removed the call to dispatch_async, and replaced it with the following:
- (void)updateUIFromDataModel {
dispatch_source_merge_data(_gcdUpdateUI, 1);
}
This is working absolutely fine for me. Now Even during the most intense data updating the UI stays perfectly responsive.
Although the printf() output was a very crude way of checking if the coalescing is working, a quick scrolling back up the console output showed me that the majority of the messages print outs had a value 1 (easily 98% of them), however there were the intermittent jumps to around 10-20, reaching a peak value of just over 100 coalesced messages around a time when the model was sending the most update messages.
Thanks again for the help!
If the app beach-balls under heavy load, then you've blocked the main thread for too long and you need to implement a coalescing strategy for UI updates. If the app remains responsive to clicks, and doesn't beach-ball, then you're fine.

Writing to channel in a loop

I have to send a lot of data to I client connected to my server in small blocks.
So, I have something like:
for(;;) {
messageEvent.getChannel().write("Hello World");
}
The problem is that, for some reason, client is receiving dirty data, like Netty buffer is not clear at each iteration, so we got something like "Hello WorldHello".
If I make a little change in my code putting a thread sleep everything works fine:
for(;;) {
messageEvent.getChannel().write("Hello World");
Thread.sleep(1000);
}
As MRAB said, if the server is sending multiple messages on a channel without indicating the end of each message, then client can not always read the messages correctly. By adding sleep time after writing a message, will not solve the root cause of the problem either.
To fix this problem, have to mark the end of each message in a way that other party can identify, if client and server both are using Netty, you can add LengthFieldPrepender and LengthFieldBasedFrameDecoder before your json handlers.
String encodedMsg = new Gson().toJson(
sendToClient,newTypeToken<ArrayList<CoordinateVO>>() {}.getType());
By default, Gson uses html escaping for content, sometime this will lead to wired encoding, you can disable this if required by using a Gson factory
final static GsonBuilder gsonBuilder = new GsonBuilder().disableHtmlEscaping();
....
String encodedMsg = gsonBuilder.create().toJson(object);
In neither case are you sending anything to indicate where one item ends and the next begins, or how long each item is.
In the second case the sleep is getting the channel time out and flush, so the client sees a 'break', which it interprets as the end of the item.
The client should never see this "dirty data". If thats really the case then its a bug. But to be hornest I can't think of anything that could lead to this in netty. As every Channel.write(..) event will be added to a queue which then get written to the client when possible. So every data that is passed in the write(..) method will just get written. There is no "concat" of the data.
Do you maybe have some custom Encoder in the pipeline that buffers the data before sending it to the client ?
It would also help if you could show the complete code that gives this behavoir so we see what handlers are in the pipeline etc.

Uploading a file using post() method of QNetworkAccessManager

I'm having some trouble with a Qt application; specifically with the QNetworkAccessManager class. I'm attempting to perform a simple HTTP upload of a binary file using the post() method of the QNetworkAccessManager. The documentation states that I can give a pointer to a QIODevice to post(), and that the class will transmit the data found in the QIODevice. This suggests to me that I ought to be able to give post() a pointer to a QFile. For example:
QFile compressedFile("temp");
compressedFile.open(QIODevice::ReadOnly);
netManager.post(QNetworkRequest(QUrl("http://mywebsite.com/upload") ), &compressedFile);
What seems to happen on the Windows system where I'm developing this is that my Qt application pushes the data from the QFile, but then doesn't complete the request; it seems to be sitting there waiting for more data to show up from the file. The post request isn't "closed" until I manually kill the application, at which point the whole file shows up at my server end.
From some debugging and research, I think this is happening because the read() operation of QFile doesn't return -1 when you reach the end of the file. I think that QNetworkAccessManager is trying to read from the QIODevice until it gets a -1 from read(), at which point it assumes there is no more data and closes the request. If it keeps getting a return code of zero from read(), QNetworkAccessManager assumes that there might be more data coming, and so it keeps waiting for that hypothetical data.
I've confirmed with some test code that the read() operation of QFile just returns zero after you've read to the end of the file. This seems to be incompatible with the way that the post() method of QNetworkAccessManager expects a QIODevice to behave. My questions are:
Is this some sort of limitation with the way that QFile works under Windows?
Is there some other way I should be using either QFile or QNetworkAccessManager to push a file via post()?
Is this not going to work at all, and will I have to find some other way to upload my file?
Any suggestions or hints would be appreciated.
Update: It turns out that I had two different problems: one on the client side and one on the server side. On the client side, I had to ensure that my QFile object stayed around for the duration of the network transaction. The post() method of QNetworkAccessManager returns immediately but isn't actually finished immediately. You need to attach a slot to the finished() signal of QNetworkAccessManager to determine when the POST is actually finished. In my case it was easy enough to keep the QFile around more or less permanently, but I also attached a slot to the finished() signal in order to check for error responses from the server.
I attached the signal to the slot like this:
connect(&netManager, SIGNAL(finished(QNetworkReply*) ), this, SLOT(postFinished(QNetworkReply*) ) );
When it was time to send my file, I wrote the post code like this (note that compressedFile is a member of my class and so does not go out of scope after this code):
compressedFile.open(QIODevice::ReadOnly);
netManager.post(QNetworkRequest(QUrl(httpDestination.getCString() ) ), &compressedFile);
The finished(QNetworkReply*) signal from QNetworkAccessManager triggers my postFinished(QNetworkReply*) method. When this happens, it's safe for me to close compressedFile and to delete the data file represented by compressedFile. For debugging purposes I also added a few printf() statements to confirm that the transaction is complete:
void CL_QtLogCompressor::postFinished(QNetworkReply* reply)
{
QByteArray response = reply->readAll();
printf("response: %s\n", response.data() );
printf("reply error %d\n", reply->error() );
reply->deleteLater();
compressedFile.close();
compressedFile.remove();
}
Since compressedFile isn't closed immediately and doesn't go out of scope, the QNetworkAccessManager is able to take as much time as it likes to transmit my file. Eventually the transaction is complete and my postFinished() method gets called.
My other problem (which also contributed to the behavior I was seeing where the transaction never completed) was that the Python code for my web server wasn't fielding the POST correctly, but that's outside the scope of my original Qt question.
You're creating compressedFile on the stack, and passing a pointer to it to your QNetworkRequest (and ultimately your QNetworkAccessManager). As soon as you leave the method you're in, compressedFile is going out of scope. I'm surprised it's not crashing on you, though the behavior is undefined.
You need to create the QFile on the heap:
QFile *compressedFile = new QFile("temp");
You will of course need to keep track of it and then delete it once the post has completed, or set it as the child of the QNetworkReply so that it it gets destroyed when the reply gets destroyed later:
QFile *compressedFile = new QFile("temp");
compressedFile->open(QIODevice::ReadOnly);
QNetworkReply *reply = netManager.post(QNetworkRequest(QUrl("http://mywebsite.com/upload") ), compressedFile);
compressedFile->setParent(reply);
You can also schedule automatic deletion of a heap-allocated file using signals/slots
QFile* compressedFile = new QFile(...);
QNetworkReply* reply = Manager.post(...);
// This is where the tricks is
connect(reply, SIGNAL(finished()), reply, SLOT(deleteLater());
connect(reply, SIGNAL(destroyed()), compressedFile, SLOT(deleteLater());
IMHO, it is much more localized and encapsulated than having to keep around your file in the outer class.
Note that you must remove the first connect() if you have your postFinished(QNetworkReply*) slot, in which you must then not forget to call reply->deleteLater() inside it for the above to work.

Suggestions needed for architecting my code

Background
I'm writing an part of my app that has no UI. It sits in the background watching what you do and timing your work.
There should be no overlapping times, and there should be no breaks in the time data. If there are either of these things, the app has a bug somewhere and I need to be notified.
What I Want
A class called JGDataIntegrityController that does the following:
Check the data store for duplicate times. Scan since the last Duplicate Report Date stored in NSUserDefaults.
If duplicate times are found, build a report.
Send the report.
If the sending isn't successful, then exit. Otherwise continue.
Remove the duplicates
Update the last Duplicate Report Date in NSUserDefaults
Repeat the above for data breaks.
What I've Got
I've made a base class that does all the hard work of sending the report.
Class Diagram http://synapticmishap.co.uk/ReportClasses.jpg
JGReportSender has the following code:
-(void)postReport:(NSString *)report {
NSMutableDictionary *form = // Dictionary Holding Report;
NSURLRequest *request = [NSURLRequest requestWithURL:#"http://postURL" postForm:form];
[NSURLConnection connectionWithRequest:request delegate:self];
}
Where I'm Getting Stuck
What should I do when the report has been sent?
The delegate methods:
-(void)connectionDidFinishLoading:(NSURLConnection *)connection
-(void)connection:(NSURLConnection *)connection didFailWithError:(NSError*)error
are called when the report has been sent. But how should I communicate with JGDataIntegrityController?
My Crap Idea
My idea is to have a reportStatus NSNumber property in JGReportSender. Then when the delegate methods get called, this is updated.
reportStatus = 1 means "report sent OK".
reportStatus = 2 means "problem sending report".
Then I could add an observer for reportStatus for JGDataDuplicateReportSender and JGDataBreakReportSender. This would then handle the report sending error or continue on.
Any Good Ideas?
I get the feeling this is a really messy way of doing this. I also feel like I'm overlooking something really obvious.
Any ideas how to do this in a neat way?
Update
I totally forgot to mention - this will be a 100% opt in feature. It'll be disabled by default. It'll also have 3 levels of privacy - from "a data break occurred" through to "a data break occurred after this application was active with this document path". And the reports will also be anonymous.
I'm conscious of all the privacy concerns - this is so I can make the software better, not so I can spy on people!
Give the report sender a delegate property and protocol, with at least two methods: reportSenderDidSucceed: and reportSender:failedWithError:. The report sender will send the latter message from its connection:didFailWithError: method, passing along the error object it got.
I do hope you'll make this feature optional. Expect lots of angry/curious email from users (not to mention public warnings of “don't use this app because it phones home” on web pages) if you don't.
Just a quick note to say if anyone wants a good tutorial on implementing your own delegates as Peter is suggesting I do, I found this one:
http://cocoadevcentral.com/articles/000075.php
Check it out. It's excellent!

Resources