WCF, ThreadPool.QueueUserWorkItem and Windows Azure - performance

Question

I have a weird problem that I hope you can help me out with.
On our development server we are running Windows 2008R2 with IIS 7.5 on a virtual x64 instance with 8GB RAM.
Here I call a WCF method that uses ThreadPool.QueueUserWorkItem to process a large amount of hierarchical data. This works fine, and work rather fast (a 270 MB XML is read an processed producing 190.035 records within 379 seconds). The client is done calling the method about 250 seconds in.
Now the same "workflow" on Windows Azure is a whole other case. Although similar (Large instance in Round Robin configuration), Windows Azure stop within seconds the client disconnect. This means that only 160.055 records is written and far slower - 917 seconds. The problem here is, that I miss around 30.000 records which should now be queued on the two Azure instances, but it seems like - on client disconnect - to abandon the remaining work.
The client uses HttpWebRequest for communication and both solutions run .NET 4.0.
What is that I am missing out on here?
Thanks in advance for any help regarding this issue.

Answer 1

My bad guys - I simply could not in my wildest imagination foresee that Windows Azure would be so slow .. so by increasing the HttpWebRequest from 2 minutes to 30 minutes I was able to achieve the same data volume as in our development environment.
So - I will not delete the question - but let this stand as a reference for you soon to come Azure guys.
I am positive that Azure (and other cloud providers) is the future, but from Denmark to "North Europe" the latency is high - and SQL Azure has yet to prove it can perform when talking OLTP and normalized databases.
DEVELOPMENT (VIRTUAL ENVIRONMENT)
190.335 records from a 299 MB file took 379 seconds on a single instance
WINDOWS AZURE (NORTH EUROPE)
190.335 records from a 299 MB file took 1.400 seconds on two LARGE instances
The good news is, that WCF and ThreadPool work flawlessly and no special considerations (except a high timeout) is necessary.
Just to clarify, the 299MB file is split up in multiple REST calls to the server, in a format similiar to this one:
<?xml version="1.0" encoding="UTF-8"?>
<HttpPost absolutePath="A/B/C/D/E/OO">
<Parameters xmlns="http://somenamespace">
<A>Package</A>
<B>100</B>
<C>Generic</C>
<D>ReceiverParty</D>
<E>
<F xmlns="http://somenamespace">
<G xmlns="http://somenamespace/Product">Long Text</G>
<H xmlns="http://somenamespace/Product">1</H>
<I xmlns="http://somenamespace/Product">PK</I>
<J xmlns="http://somenamespace/Product">5995</J>
<K xmlns="http://somenamespace/Product">
<L xmlns="http://somenamespace/P/Q">Discount</L>
<M xmlns="http://somenamespace/P/Q">1000</M>
<N xmlns="http://somenamespace/P/Q">6995</N>
</K>
</F>
</E>
<OO>
<O>
<A>Product</A>
<B>100</B>
<C>Generic</C>
<D>ReceiverParty</D>
<E>
<F xmlns="http://somenamespace">
<G xmlns="http://somenamespace/Product">Long Text</G>
<H xmlns="http://somenamespace/Product">1</H>
<I xmlns="http://somenamespace/Product">PK</I>
<J xmlns="http://somenamespace/Product">5995</J>
<K xmlns="http://somenamespace/Product">
<L xmlns="http://somenamespace/P/Q">Discount</L>
<M xmlns="http://somenamespace/P/Q">1000</M>
<N xmlns="http://somenamespace/P/Q">6995</N>
</K>
</F>
</E>
</O>
</OO>
</Parameters>
</HttpPost>