MS Sync Toolkit how to handle huge initial download - performance

We're using Microsoft Sync Framework Toolkit to allow some iOS devices to synchronize their SQLite local storage with a central SQL Server database over an OData service (just like the sample does).
The problem is that the central database is quite big (more than 500 MB) and that the customer want to initially replicate all his content on their devices.
So we configured the service to sync all tables but it crashes the first time a device tries to sync (OutOfMemoryException).
How should we design/configure the service to manage such a situation? Setting filters doesn't seem to fit our needs since we need all the data.
I should add that we tried to configure SetDownloadBatchSize with no luck. Maybe there is something to configure on the client as well?

Related

Azure Storage Blob Policy to enforce ClientSideEncryptionVersion.V2_0

i updated my application because of this security vulnerability.
Is it possible to set a policy in Azure Storage Blob that only blobs without encryption or with ClientSideEncryptionVersion.V2_0 can be uploaded?
Upload attempts with ClientSideEncryptionVersion.V1_0 should be blocked.
We don't have this feature today. You may leave your feedback here All the feedback you share in these forums will be monitored and reviewed by the Microsoft engineering teams responsible for building Azure.
Client side encryption can be done only using SDK because we use CEK to encrypt the data before uploading, so you have to write your own custom logic for this scenario, Also make sure that all your application are upgraded to latest SDK.
Alternatively, would it be possible to use the Azure Metric Explorer to evaluate the upload attempts with the ClientSideEncryptionVersion?
No. Encryption metadata is stored with the blob, but we don’t have granular tracking for what metadata is set.
Or can you alternatively, for example, give a tag or an application version when uploading that is saved together with the blob?
This is completely dependent on the application and the features you use in storage/the SDK. There isn’t anything automatically enabled by the SDK.
Please let me know if you have any further queries.

Xamarin Mobile offline DB sync with SQL Server on-premise

Is there a way to sync a SQLite mobile offline DB with an on premise SQL Server? As far as I understand, Azure offline sync syncs with Table storage and not SQL server. Also, hybrid connection and hybrid connection manager doesn't take care of synchronization/replication as far as I know.Could anyone of you confirm whether there is a way to achieve this?
Came across something called Zumero, but not sure how much it is practifcally possible.

How to move Parse.com database to my own server and still use Parse.com SDK?

I'm using Parse.com SDK services for my Android app.
I've seen that Parse had released their Android SDK as an open source project on Github on this address.
My app is almost finished, and when I'm uploading it to the Play Store, I don't want to be controlled by Parse.com (I mean that I don't want to be blocked someday, or I don't know that), so I want to move my whole database to my own server that hosted on a secure company.
I've checked the open source project on Github and realized that all I need to use it on my own server is to generate an Application ID and a client key.
So I want to ask if someone knows how to generate an Application ID and a client key of Parse to use it on my own server, or that you maybe knows another way of moving it to my server? And one more question: Today I'm using also Facebook SDK with my app. If I will move my database to my own server, will I still be able to use Facebook SDK on my app?
Thanks!
I have write an article about how to migrate parse to a custom server.
https://medium.com/#jcminarro/run-parse-server-on-your-own-server-using-digitalocean-b2a7d66e1205
There's a massive difference between Parse open-sourcing their SDKs compared to revealing their entire backend architecture and its configuration.
The open-sourced SDKs are essentially wrappers for Parse's REST API along with some convenience functions and logic for natively interpreting the JSON data Parse is transmitting.
At a high level, Parse uses MongoDB for its core database and is entirely hosted using AWS (Amazon Web Services). The entire architecture is highly complex and is not something you could just drag and drop onto your own software stack or hardware backend.
To help give you a better idea of how Parse achieves all of their services, here's an interesting presentation their Dev Ops team gave at an AWS convention. Suffice it to say, hosting the backend services for over 180,000 apps requires a complex infrastructure and that is the "secret sauce" so to speak for Parse and is why Facebook purchased them for over $85 million two years ago.

Client Side Transactions in Azure App Service Offline Sync

Is it possible to leverage transaction on the client, when using azure app service offline sync?
It seems to me that you can pretty much only insert into the SQLite store a record at a time, but ideally i'd like to be able to commit multiple records in a transaction.
I'm using Xamarin with Android right now,but will be also supporting iOS.
Thanks
Matt
Unfortunately, on Windows, Xamarin, and Android you can only commit one record at a time on the local MobileServicesSQLiteStore. This is because changes need to be made one at a time using methods like SyncTable.UpdateAsync, which is how you mark a change as being tracked to send to the server.
On native iOS, the Mobile Services offline sync feature uses Core Data, which does support a form of transaction (https://developer.apple.com/library/ios/documentation/Cocoa/Conceptual/CoreData/Articles/cdMemory.html). If you use these APIs, you still need to make calls to the Mobile Services SDK methods to track each individual change in the internal tracking tables.
Why do you need transactions on the client side? There might be a different way to achieve your goal.

real time number crunching and storage on cloud

I have some hardware devices that send some data that need to be stored on the cloud server and also I need to do some real time processing on them.
The data they send need to be preserved for months in some custom binary files. These files related to each device can grow in size up to 10GB over time.
There will client programs (mobile / web) that will be looking at the processed data at real time.
My prefered choice of language is C/C++/C#, since there is time sensitive number crunching involved.
Goal is write scalable application that can have thousands of such devices monitored on the cloud.
Do I have to upfront write the code for running on the cloud ( undestand Azure / amazon EC2) ? Can I write multi threaded desktop application and later migrate to cloud ?
I have used Message passing interface (MPI) in the past for clusters. Can I still use MPI ?
If I use microsoft azure API can I still host my software on Amazon cloud ?
For mobile devices to talk to the server, I understand that I need to have a webservice running. how can I convert a desktop program writeen in C++ / C# to act as a web service talking to client?
Are there any 3rd part frame works or tools taht can help me with my work ?
With most cloud compute services you can deploy an off-the-shelf server and install your own software on it. So, yes, you can write and test you application locally then migrate to the cloud once you get all the bugs worked out. Here are the available EC2 server configurations.
I have not tried MPI but you should be able to run just about anything you want on the servers in the cloud. However, Amazon does offer the Simple Queue Service which provides message passing in the cloud. Your software does not need to run in the cloud to use this service.
I have not used Azure. I doubt there are any restrictions regarding which external servers you use for storage and/or compute. However, keeping your cloud storage and compute resources within a single provider will reduce costs, improve performance and provide you with a unified management interface and billing system.
Web servers are fairly simple things. See this post. That took me about 10 seconds to find.
There is plenty of third party software out there. Figure out what you need in more detail and ask more specific questions

Resources