I need to design an MVC 2 or 3 app to work on both internal Win server and on Azure. The only problem I see is with the data storage. The app listens to some very chatty data feeds, like 10GB a day, and will have to write that data. So I'm looking for a way to have both Azure table and some other DB with minimal differences in code.
Any suggestions for what DB to use?
TIA
What about using flat file system locally and Blob in the cloud?
This approach works quite well for storing a lot of data (without bothering with the scalability). For indexes you can use file-based SQlite (works perfectly in Azure), and for actual serialization - Open Source ProtoBuf.NET (Fasted and most compact serialization for .NET).
NB: If your system uses patterns from the CQRS Architecture approach, then it's much easier to pick persistence type that's make the most sense in your case.
Related
I´m developing a Jokenpo Game using React with Spring Rest, but I can´t have a database to store all the information needed(create and delete moves, create and delete players).
I don´t know the best practice of development, or if there is some design pattern on how to store that kind of information. I know there is the folder src/main/resources where maybe I can store a text file there and thought about on the startup of the api it loads that file with the begin of the game, maybe, and after changing it during the game.
Trying to be more clear: I just would like to know the simplest way of storing information without being a database inside of a Spring Rest application. I really appreciate any helps. Thanks.
Take a look at SQLite. It's a very light database library that you can include as a dependency of your Spring application, It doesn't require a separate database server to run, and the entire database is stored in a single file, that you can choose where to store in the connection string.
It offers the flexibility of a standard database, so you can use Spring Data / JPA to access the data. It has some limitations compared with robust databases like MySQL, specially related with concurrent writes that you should investigate and be aware of. Usually it works very well for small applications or embedded applications.
I learned to love how LINQ enables set operations on collections. I'm not saying that I plan to shun traditional RDMBS, because I do need it for reporting. There are NoSQL alternatives out there, but they seem to all need to fire up a separate service.
What I looking for is something local where a DLL can create a database and perform CRUD on it. As mentioned, I'm not going to report out of this, just internal data store. The main application that will be using it is in C#.
I'm hoping that someone can give me a lead. If not, if there is anyone willing, we can start a open-source project for it. I'm not interested in commercial products.
Thanks,
You can run RavenDB in embedded mode inside your .NET application - no need for external services or anything.
And RavenDB supports Linq....
Hi i am developing a metro style application where i will be connecting to web services and get the data from the web service and i will be binding it to the UI.
But my requirement is in my application i need to create tables and also provide relation between the tables and dump the data in to that local database and use that data in calling the other methods present in my service application (calling one more method in services by passing something as an input to that).
Can you please explain me the following :-
What is Database which is supported for metro style apps?
How can i create a database and create tables and dump the the data which i got as the response from my service application?
I am new to Metro style application development please help me out .
Thanks in Advance.
First of all WinRT has very poor db support. Most of this kind of things are done by web services, OData etc
BUT I`m almost 100% sure that you can use SQLite. On codeplex there are connectors from Win8 Metro app to SQLite DB so check this topic. I also saw somewhere on MS page that SQLite is support in some way. Check it
What Fixus said is correct. Personally , since my app doesnt have a large amount of data to store locally (it goes against the Metro guidelines to store large amounts of data) I serialize the objects instead to local storage. When needed, and if internet is available, the services will be called and the local data updated.
If you choose to use SQLlite make sure you use the real deal and not a third party db, as the DB library must be approved by Microsoft if you want to get the app accepted to the windows store. I'm not even sure that SQLite is yet approved, but by the looks of it they will be.
Tim Heuer always writes great articles on the subject, this one might help you
Let me know if you need help with serializing in WinRT, if you need it.
Best of luck!
We recommend using SQLite database with LinqConnect - Devart's LINQ to SQL compatible solution which supports SQLite engine (provided by http://code.google.com/p/csharp-sqlite/). You can employ LINQ and ADO.NET interfaces with our product. Starting from the 4.0 version, LinqConnect supports Windows Metro applications: http://blogs.devart.com/dotconnect/linqconnect-for-metro-quick-start-guide.html.
If you're building some application that has to keep working without any network connection, and needs to synchronize at some point in time, it is necessary to keep a local database.
You can read the following article, which has some basic guidelines and samples.
http://blogs.msdn.com/b/win8devsupport/archive/2013/01/10/using-database-in-windows-store-apps-i.aspx
All,
I'm beginning to explore what frameworks (open source) and tools for building web applications. What should I select and learn for the following layers,
Layer 1
Client side JavaScript / AJAX library or framework that will invoke REST style services provided by layer-
2
Layer 2
Provides a framework to rapidly create REST style services out of existing applications and out of a NoSQL document oriented database provided by layer-3. I need this layer in cases where I need to expose REST style services out of my traditional apps and RDBMS.
Layer 3
Which NoSQL to use - CouchDB or MongoDB that would work well with layer-2?
Will I need a MVC framework like RoR or a web/component framework like Wicket? Am I missing anything?
I also need recommendations for which tooling/IDE (and associated plugins) for the development environment. Thanks in advance for your answers/thoughts.
We've had pretty decent luck using a Java stack:
For the presentation, we use jQuery and jQueryUI, with Freemarker for XHTML/CSS templating, including to invoke REST web services through various UIs.
Restlet (www.restlet.org) is a wonderfully rich framework for crafting REST web services in Java. We decided to use it on a major product after it was strongly recommended to us by the engineering director of a top 10 e-commerce site in the US. And everything he said about it was true.
Unless you know you're going to face a really large amount of write volume, you're probably best off using one of the tried and true SQL databases supporting ACID transactional guarantees. We used Oracle, then switched to PostgreSQL, using the MyBatis (formerly iBatis) SQL Mapper to shield our code from the details of the database. With the advent of 64-bit addresses and scads of inexpensive DRAM, plus SSDs, these old workhorses do scale quite high.
If you are anticipating very large amounts of writes, by all means consider a so-called "NoSQL" database. I heard very good things about Vertica from the top network ops folks at a major technology company last week. MongoDB and CouchDB both look interesting. Or you may be able to leverage persistent distributed cache technology like Redis or EhCache to offload a traditional database.
The task you're trying to accomplish determines the technology you use.
If you're interested in the .NET platform, consider:
RavenDB for the data layer.
WCF Data Services for your REST services layer. Here's a guide to WCF Data Services with some great video tutorials.
If you need a web front-end, consider ASP.NET MVC or plain HTML as you see fit. jQuery and AJAX modules are well suited in both.
Arguably, ASP.NET MVC IS a REST endpoint of sorts, so you could skip the WCF services.
All the above can be leveraged with free tools, including:
Visual Studio Express
Raven DB's HTML interface at http://localhost:8080/
I've found no clear answer so far, but maybe I've searched the wrong way.
My Question is, can Core Data to be used as a Persitence Storage for a Server Project? Where are Core Data's Limits, how much Data can be handled with Core Data and SQLite? SQLite should handle a lot of Data very well according to their website. I know of a properitary Java Persitence Manager with an Oracle DB as Storage that handles Millions of Entries and 3000 Clients without Problems. For my own Project I wonder if I can use Core Data on the Server Side for User Mangament and intern microblogging, texting with up to 5000 clients. Will it handle such big amounts of Data or do I have to manage something like that myself? Does anyone happend to have experience with huge amounts if Data and Core Data?
Thank you
twickl
I wouldn't advise using Core Data for a server side project. Core Data was designed to handle the data of individual, object-oriented applications therefore it lacks many of the common features of dedicated server software such as easily handling multiple simultaneous accesses.
Really, the only circumstance where I would advise using it is when the server side logic is very complex and the number of users small. For example, if you wanted to write an in house web app and have almost all the logic on the server, then Core Data might serve well.
Apple used to have WebObjects which was a package to manage servers using an object-oriented DB much like Core Data. (Core Data was inspired by a component of WebObjects called Enterprise Objects.) However, IIRC Apple no longer supports WebObjects for external use.
Your better off using one of the many dedicated server packages out there than trying to roll your own.
I have no experience using Core Data in the manner you describe, but my understanding of the architecture leads me to believe that it could be used, depending on how you plan to query and manipulate the data.
Core Data is very good at maintaining an object graph and using faults to bring parts into memory as needed. In that manner, it could be good on a server for reducing memory requirements even with a large data set.
Core Data is not very good at manipulating collections of objects without loading them into memory, making a change, and writing them back out to disk. Brent Simmons wrote a blog post about this, where he decide to stop using Core Data for some of his RSS reader's model objects because an operation like "mark all as read" didn't scale. While you would like to be able to say something like UPDATE articles SET status = 'read', Core Data must load each article, set its status property, then write it back to disk.
This isn't because Apple engineers are stupid, but because the query layer can't make assumptions about the storage layer (you could be using XML instead of SQLite) and it also must take into account cascading changes and the fact that some article objects may already be loaded into memory and will need to be updated there.
Note that you can also write your own storage providers for Core Data, see Aaron Hillegass's BNRPersistence project. So if Core Data was "mostly good" you might be able to improve on it for your application.
So, a possible answer to your question is that Core Data may be appropriate to your application, as long as you do not need to rely on batch updates to large number of objects. In general, no algorithm or data structure is appropriate for every scenario. Engineering is about wisely choosing between trade-offs. You won't find anything that works well for many clients in every case. It always matters what you are doing.