subsonic with support for caching - caching

Having a project with following requirements in mind.
data reading intensive application.
100 max concurrent users a times. Application have very high traffic
Though data is huge it is getting modified only once a day
Decided to use subsonic cause of ease of development and potential to work in high traffic environment.
Though few things are not yet found/solved to work with SubSonic 3
Which type of layer to use Active Records, Repository, Linq To SQL
working with paging / sorting stored procedures (cause they will give better performance over inbuilt paging mechanism, when displaying 10000+ rows with paging and sorting. right?? )
Caching, with project requirement it is quite clear, heavy use of caching is required. But could not find suitable solution, which will work with subsonic.
do I have to create separate layer for it and if yes, a short example would be helpful.

I wrote a CacheUtil class for subsonic 2.x ActiveRecord. It's based on some code someone posted on the old subsonic forums. (This is from a forum that was deleted before the last forum was removed. This is why software forums should be permanent.) Here is an example of a cache Find method. You could adapt it to ss3. There are also inserts, fetchall, delete, clear, etc. Rob Connery said at the time that caching was problematic, and it was left out of ss2 on purpose. By using HttpRuntime.Cache I share the cache between a web application and service simultaneously. I believe I can do this since it's a small application, always on a single server.
public static RecordBase<T> Find<T, ListType>(object primaryKeyValue)
where T: RecordBase<T>, new()
where ListType: AbstractList<T, ListType>, new()
{
string key = typeof(T).ToString();
if(HttpRuntime.Cache[key] == null)
FetchAll<T, ListType>();
if(HttpRuntime.Cache[key] != null)
{
ListType collection = (ListType)HttpRuntime.Cache[key];
foreach(T item in collection)
{
if(item.GetPrimaryKeyValue().Equals(primaryKeyValue))
return item;
}
}
return null;
}

I wrote a post about how I used caching with SubSonic 2.x. It isn't 100% compatible with 3.x but the concepts are the same.

I answered this similarly over here Thread-safe cache libraries for .NET. Basically you need a CollectionCacheManager - I then add a layer on top for each type and funnel all requests through this individual cache controllers, which in turn are using the 1 collectioncachecontroller. At the outer layer I mix pure subsonic, linq, whatever fits the bill at the time. That's the beauty of SubSonic is that it should not get in your way. As far as stored proc performance I would point to Jeff Atwoods articles over at CodingHorror and reevaulaute your savings. Hardware is dirt cheap, as is memory, databases are not. Personally I keep the database super simple and lightweight, and prefer to let my webserver cache everything in memory. The database server gets to do very little work which is the way I like it. Adding a few extra load balanced web servers isn't nearly as big of a deal as increasing database throughput, clustering, or sharding a a DB. SQL & Stored Procs can also be ridiculously difficult to write, and maintain. Take that budget that you would have spent on your time doing that, and instead beef up your hardware... Remember hardware is dirt cheap, good developers are not. Good luck!

Related

RavenDB - slow write/save performance?

I started porting a simple ASP.NET MVC web app from SQL to RavenDB. I noticed that the pages were faster on SQL than on RavenDB.
Drilling down with Miniprofiler it seems the culprit is the time it takes to do: session.SaveChanges (150-220ms). The code for saving in RavenDB looks like:
var btime = new TimeData() { Time1 = DateTime.Now, TheDay = new DateTime(2012, 4, 3), UserId = 76 };
session.Store(btime);
session.SaveChanges();
Authentication Mode: When RavenDB is running as a service, I assume it using "Windows Authentication". When deployed as an IIS application I just used the defaults - which was "Windows Authentication".
Background: The database machine is separate from my development machine which acts as the web server. The databases are running on the same database machine. The test data is quite small - say 100 rows. The queries are simple returning an object with 12 properties 48 bytes in size. Using fiddler to run a WCAT test against RavenDB generated higher utilization on the database machine (vs SQL) and far fewer pages. I tried running Raven as a service and as an IIS application, but didn't see a noticible difference.
Edit
I wanted to ensure it wasn't a problem with a) one of my machines or b) the solution I created. So, decided to try testing it on Appharbor using another solution created by Michael Friis: RavenDN sample app and simply add Miniprofiler to that solution. Michael is one of the awesome guys at Apharbor and you can download the code here if you want to look at it.
Results from Appharbor
You can try it here (for now):
Read: (7-12ms with a few outliers at 100+ms).
Write/Save: (197-312ms) * WOW that's a long time to save *. To test the save, just create a new "thingy" and save it. You might want to do it at least twice since the first one usually takes longer as the application warms up.
Unless we're both doing something wrong, RavenDB is very slow to save - around 10-20x slower to save than read. Given that it re-indexes asynchronously, this seems very slow.
Are there ways to speed it up or is this to be expected?
First - Ayende is "the man" behind RavenDB (he wrote it). I have no idea why he's not addressing the question, although even in the Google groups, he seems to chime in once to ask some pointed questions, but rarely comes back to provide a complete answer. Maybe he's working hard to to get RavenHQ off the ground?!?
Second - We experienced a similar problem. Below's a link to a discussion on Google Groups that may be the cause:
RavenDB Authentication and 401 Response.
A reasonable question might be: "If these recommendations fix the problem, why doesn't RavenDB work that way out of the box?" or at least provide documentation about how to get decent write performance.
We played for a while with the suggestions that were made in the thread above and the response-time did improve. In the end though, we switched back to MySQL because it's well-tested, we ran into this problem early (luckily) which caused concern that we might hit more problems and, finally, because we did not have the time to:
fully test whether it fixed the performance problems we saw on the RavenDB Server
investigate and test the implications of using UnsafeAuthenticatedConnectionSharing & pre-authentication.
To summarize Ayende's response you're actually testing the summation of network latency and authentication chatter. As Joe pointed out there's ways you can optimize the authentication to be less chatty. This does however arguably reduce security, clearly Microsoft built security to be secure first and performance secondary. You as the user of RavenDB can choose if the default security model is too robust as it arguably is for protected server-to-server communication.
RavenDB is clearly defined to be READ orientated. 10-20x slower for writes than reads is entirely acceptable because writes are full ACID and transactional.
If write speed is your limiting factor with RavenDB you've likely not modeled transaction boundaries properly. That you are saving documents that are too similar to RDBMS table rows and not actually well modeled documents.
Edit: Reading your question again and looking into the background section, you explicitly define your test conditions to be an optimal scenario for SQL Server while being one of the least efficient methods for RavenDB. For data that size, that's almost certainly 1 document if it would be real world usage.

data access mvc3

question:
i'm building (or trying to build) a front end for an enterprise level web-app. the existing structure is driven by stored procedures (sql 2008 db). A stored proc is implemented as a class that ultimately handles the execution and the results are returned as an object.
i'm new to this game and would welcome an explanation about how my time would best be served...i read a repository pattern is a best practice, but all of the examples i read implement Entity Framework or Linq, etc..do i need an ORM? why or why not? i'd like to be able to have a maximum performance environment so that users can play with those result sets. thanks in advance
Well, I would suggest deciding on your use cases.
Some of the things that nHibernate / ORM's generally are not good for are:
Batch jobs.
Reporting
So if your work primarily involves either of those then you're best off not wasting your time, that being said there's nothing wrong with having multiple strategies... Building out a domain model is great for simplifying complex business rules, performance is generally very good too... Reporting and batch jobs can be built out separately, there's no reason why the different strategies can't co-exist... I woul however do my best to keep them decoupled...
So if you've got a big hairy business logic layer and it's littered with datasets / data access code and business logic IN your stored procedures then you will likely find it worth your while to invest in an ORM, but consider it a re-factoring step... IE you're improving existing code and making it testable before extending it...
In any case there's no one 'best' answer, the smartest thing I've done at previous companies has been to build new functionality (Test driven of course) in whichever data access pattern that seems to make sense to the functionality... Keep interfaces clean and decoupled... After doing that for awhile it usually becomes obvious which strategy / pattern is best suited for the application overall...
Good luck
Your question is somewhat unclear. Stored Procedures are SQL queries stored on the database which are used to interact with the data. However, it sounds like you already have an existing data access layer (DAL) which uses stored procedures and returns objects to you to play with. If this is the case, I would not throw away the entire data access layer and replace it with EF or any other ORM. Unless the existing DAL isn't working for you for either design or performance reasons, there's no reason to reinvent the wheel.

Front and back end techniques to increase performance

What are some of the common and notable performance issues/bottlenecks that are typically encountered in a web application in both, the front-end layer, and the back-end layer?
An example of what I mean in a database is not having something you are querying on be an index. That would slow down the query. On the front-end it might be something funky going on with JavaScript that makes your application seem slow.
What are the general rules of thumb that help navigate such issues? And what are some good to-do's?
Thanks,
Alex
On front-end:
-push all of your assets - css files, images, static content - to a CDN. Edgecast is pretty good and reasonably priced.
-don't use load entire javascript frameworks when you only need a few features from it. only load what's needed.
On back-end
-memcache the results from all database calls by using a hash of the sql query as the key name, and the result set as the value
-make sure you are not making your database tables really 'wide' - tons of columns and column types like 'text' and 'blob'
For the front-end, there are well-known guidelines/rules you can follow, and there are some great tools like YSlow that can help you pinpoint the bottlenecks.
For the back-end, as you've noted, efficient use of indexes is a must. Other optimizations usually involve caching, and basic stuff like avoiding doing stuff within loops that can be done once. I'm sure people here will have suggestions, but remember "premature optimization is the root of all evil!" :-)
Millhouse is on to it. I can also add:
Batch expensive operations up. For example: don't make lots of individual calls to a database if you can do it all in one hit.
Avoid server hops where you can.
Process in parallel if you can (not so common for your 'average' web app but quite possible in larger Enterprise scale apps).
Pre-process: crunching data, pre-puiblishing content etc, the more you can do before it's needed the better.
Use a CQRS-based architecture. CQRS stands for Command/Query Responsability Segregation; it basically means that you have different code (services) for reading from the DB and writing to the DB. A good practice for scalability is to have separate DB's for reading and writing (it actually does make sense, if you read more about CQRS), and you can scale out the reading database by having copies run on multiple servers.
CQRS is not only interesting from a scalability point of view, but also from a code maintenance and clarity point of view. It does take some effort to learn about CQRS and understand it, though.
Check out these links:
http://www.slideshare.net/skillsmatter/ddd-exchange-2010-udi-dahan-on-architectural-innovation-cqrs
http://www.slideshare.net/pjvdsande/rethink-your-architecture-with-cqrs
convert dynamic contents to static contents. regenerate those static contents if their dependent objects changed. I saw one article said that more than 80 percent contents are static on Amazon website.

Why do people use linq to sql?

Given the premise:
There are competent sql programmers
(correlary - writing sql queries are not an issue)
There are competent application developers
(correlary - there is simple/strong/flexible architecture for handling connections and simple queries from code)
Why do people use linq to sql?
There is overhead added to each transaction
There is strong likelihood of performance loss for moderate-complex calculations (DBs are made for processing sets and calculations and had teams of engineers working out optimization - why mess with this?)
There is loss of flexibility (if you want to add another ui (non .NET app) or access method, you either have to put the queries back in the db or make a separate data access layer)
There is loss of security by not having a centralized control of write/update/read on db (for example, a record has changed - if you allow applications to use linq to sql to update, then you cannot prove which application changed it or what instance of an application changed it)
I keep seeing questions about linq to sql and am wondering if I'm missing something.
I keep seeing questions about linq to sql and am wondering if I'm missing something.
It's not that you're missing something. It's that you have something most shops don't have:
There are competent sql programmers
Additionally, in your shop those competent sql programmers prefer to write sql.
Here's a point by point response:
There is overhead added to each transaction
Generally true. This can be avoided by translating the queries before they are needed to run using CompiledQuery for many (but not all!) scenarios.
There is strong likelihood of performance loss for moderate-complex calculations (DBs are made for processing sets and calculations and had teams of engineers working out optimization - why mess with this?)
Either you're writing linq, which is translated to sql, and then a plan is generated from the optimizer - or your writing sql from which a plan is generated by the optimizer. In both cases you are telling the machine what you want and it is supposed to figure out how to do it. Are you suggesting that subverting the optimizer by using query hints is a good practice? Many competent sql programmers will disagree with that suggestion.
There is loss of flexibility (if you want to add another ui (non .NET app) or access method, you either have to put the queries back in the db or make a separate data access layer)
A lot of people using linq are already SOA. The linq lives in a service. The non .NET app calls the service. Bada-bing bada-boom.
There is loss of security by not having a centralized control of write/update/read on db (for example, a record has changed - if you allow applications to use linq to sql to update, then you cannot prove which application changed it or what instance of an application changed it)
This is simply not true. You prove which application is connected and issuing sql commands the same way you prove which application is connected and calling a sproc.
Let me list you a few points:
There are small software companies or mid-sized companies who develop their software in-house who might rather focus on getting many application developers than getting a freelancer DB developer or even permanently hire one.
In most cases the overhead is a non-issue either due to the amount of data to be processed or due to the low traffic. Besides, when used properly, LINQ to SQL can perform as fast as most SQL queries + the associated .net code.
Many companies just stick with the Microsoft stack and they can only enjoy the integration. Some other company develops using SOA there's just no problem. The others aren't forced to choose LINQ-to-SQL and if they make that choice is their problem how to integrate it. Nobody ever said LINQ-to-SQL is a silver bullet :)
I believe security is gained with LINQ-to-SQL because I've bumped across lots of SQL queries taking in unescaped data with string concatenation etc and explaining the whole parametrized query idea has never been easy. Besides since all queries are eventually translated into SQL, unless the tracking issue you describe would happen via a stored procedure, there're again no problems at all.
I also believe your question can be posed more generally to address all ORMs and not just LINQ-to-SQL, and still most of what I said would hold true.
The problem is that it is very rare for somewhere to have a competent SQL developer who likes writing SQL and wouldn't rather be doing something else. I would consider myself competent in SQL, I used to do all my data access layers with stored procs or parametrized queries. Trouble is that it takes ages and is dull. I'd rather be writing great applications than messing around with data access layers that essentially have a select, insert, update and delete SQL statement(or proc) repeated dozens of times for each data object.
Linq-to-SQL takes away some of the repetitive nature. It has a tool to auto generate you business objects from your database schema, and it gives you a nice integrated query language that is compile time type verified and is in your code (Stored procs are a pain to source control neatly)
I can write a DAL in Linq-to-sql several times faster than I can using plain SQL, stored procs or parametrized queries.
If you want to maintain the use of stored procs both linq-to-sql and the EF both support the use of stored procs for all their data access, you just have to set up the appropriate mappings. So, you can still use your stored procs to log details and implement security if you want. We tend to opt for using windows auth, and use that to restrict access to each table for the various users, then we have a bunch of triggers on the tables that track details for audit purposes.
Two things I will quickly note is that firstly, the entity framework seems to be getting more support from MS at the moment, and I suspect that will be considered the kind of default standard for the future in preference to linq-to-sql. Secondly, in .Net 3.5 the EF and linq-to-sql do not have very good support for n-tier disconnected apps. In both of them you kind of have to muck around with either serializing data contexts across your disconnected tiers, or manually detach and re-attach your data objects. This is much improved in the .net 4.0 though. Just something to consider depending on which version you have available to you.
Existing question/answers in the same vein/spirit:
Doesn't Linq to SQL miss the point? Aren't ORM-mappers (SubSonic, etc.) sub-optimal solutions?
LINQ-to-SQL vs stored procedures?
What's wrong with Linq to SQL?
Why do I need Stored Procedures when I have LINQ to SQL
If using LINQ to SQL is there any good reason to learn SQL queries/syntax anymore?
https://stackoverflow.com/questions/216569/are-the-days-of-the-stored-procedure-numbered
I personally believe there's no right or wrong answer. It depends on what you're developing and how you're developing it. If you need razor-sharp performance, have an overly-complex data model, etc... skip the abstraction. If you feel the abstraction speeds up your development time, like the idea of capturing all application logic in a single codebase, etc... use it.
For me, it takes a lot less time to write linq to sql code than it does to write a bunch of stored procedures. That's especially true when the design isn't finished, in that case I don't yet know how much of the work I want to do on C# objects, and how much I want to do in SQL.
So, I can skip building datasets, I don't have to click click click to add queries, basically, linq to sql means I can change my code in less time.
Also, as a big fan of Haskell, I can write lots of functional-style code with linq to sql and it just works.
I'm not saying this is an ideal solution or even a great example (it was the result of a high level constraint on the architecture, not something we necessarily would have chosen from scratch), but...
I worked on an app where the code was completely isolated from the database except through a set of exposed stored procs. The code could not "know" anything about the database schema except was was returned from the stored procs.
While this isn't that unusual and it isn't too hard to write a DAL using ADO or whatever, I decided to try out Linq to Sql, even though it wouldn't be using it for its real intended purpose and wouldn't use most of the features. Turns out it was a great decision.
I created the Linq to Sql class, dragged the stored procs from server explorer onto the right side of the designer, then... Wait, there is no then. I was pretty much done.
Linq created strongly typed methods for each stored proc. For the procs that returned rows of data, Linq automatically created a class for the items in each row and returned a List<generatedClass> for them. I wrapped the calls themselves in a lightweight public DAL class that did some verification and some automatic parameter setting and I was done. I wrote a business object class and mapped the dynamically generated Linq class objects to the business object (did this by hand, but it isn't hard to do or maintain).
The program is now immune to any schema change that doesn't affect the stored procedure signatures. If the signatures do change, we just drag off the old proc from the design and drag it back to regenerate the code. A few passes through the unit tests to make changes (which usually don't go higher than the public DAL interface) and it's done. Things upstream of the DAL use Linq to Objects techniques to select, filter, and sort data that isn't in the right format straight from the stored proc calls.
We have some excellent DBAs writing the stored procedures and an entirely different group writing the other code, so maybe it is a good example of why (and how) you can use LINQ in the scenario you describe.
Some handy features are the debugger picking up sytax errors in your query, compared to writing SQL statements as strings. Mistakes that wont get picked up until runtime.
Plus I find LINQ statements easier to read than SQL.
It may be a case of convenience triumphing over performance. If you're programming at Facebook levels of uber-performance then you might think about every clock cycle but the simple truth is that the majority of applications don't need this attention and benefit from efficiencies in code maintenance and dev time (100k contractor vs. another $erver).
That said, there's a case for outsourcing as much of the query processing from the DB box in very high scale systems, else the DB is the bottle neck and you need to shard or re-architect down the line. Costly.
I think its fair to say that LINQ will scale better/easier both in terms of servers and from many core in that your LINQ codebase will get m-core for 'free' as soon as MS release C# 4.0.
I do see your point in asking and as a non-ASP.NET dev just beginning a www project for the first time, I can't see the point of 80% of ASP.NET (themes, controls etc.) - it seems I need to learn more and code more than the HTML itself! -- again, I'm sure there's an good reason for it all.
--- I haven't got the 50 pts to comment on the post I want to so I'm doing it here ---
David B suggests that writing some SQL is all there is to getting the most out of SQL Server and that using query hints is the steering mechanism. The same task can be achieved in many different ways in SQL and many with 1000s of times the performance gain. I suggest reading Inside T-SQL Querying (Itzik Ben Gan). Over and over, Itzik shows how to rethink the query and use new commands to shrink the logical reads sometimes from thousands into less than ten.
For very simple queries, the overhead of an extra layer adds to the roundtrip cost. For somewhat more complex queries in normal 'business app' scenarios, the optimizations done by the Linq-to-SQL expression->sql translation magic can often save a lot.
As an example, I recently did a 1:1 translation of a customer-supplied 1400+ (!) line stored proc to L2S. Not only did it go from 1400 lines of SQL to 500 lines of much more readable, strongly typed, and commented code. It also started hitting the database with an average of ~1500 reads instead of ~30k reads. This is before I even started looking at db-side optimizations - that saving is something I can 100% attribute to L2S's ability to eliminate predicates that can be evaluated client-side.
simple answer, there are two approaches: create exquisite Rube Goldberg contraptions, or just get the job done in a simple way. Many devs lean towards the former.
Developers get bored easily, and would often personally enjoy doing things a harder way that seems to provide a certain intellectual beauty. Are you developing an app or writing a PhD? As my msft director used to yell in the hallways, "I don't want another research project!"
please repeat after me (min 3x)
there is no silver bullet
I will not use a technology just because its the latest thing from msft
I will not use something just to get it on my resume
Not only are their competent SQL coders, any decent app programmer, especially LOB apps, should write intermediate SQL. If you don't know any SQL and are writing LINQ to SQL, how are you going to debug your data calls? How are you going to profile them to fix bottlenecks?
We're trying out LINQ to SQL and I think there are major issues with it, such as:
There is no simple way to return the query results to another object. This in itself seems insane. Microsoft created the var anonymous datatype, and recommends using it, but there is no way to move this data out of your local method, hence the oo paradigm breaks if you have to use the data in the same function that retrieved it.
Tables are NOT objects. Study up on 3rd normal form etc. Relational databases are for storing data, not using it. I don't want to be restricted or encouraged to use my tables as objects. The data I retrieve from the database will very often be joins of multiple tables, and may include SQL casts, functions, operators, etc.
There is no performance gain, and a slight loss
Now I have way more code to worry about. There are the dbml files and still a DAL to actually write the LINQ. Yes, lots of it is machine-generated, that doesn't mean its not there, its something else that can go wrong (i.e. your dbml files, etc.).
Now that I've given the background, I will attempt to answer you actual question, why do people use LINQ To SQL:
Its the latest thing from Microsoft and I want it on my resume.
Msft has convinced managers/execs that it will decrease coding time
Developers hate SQL. (no good dev environment or debugging except manually--it would be nice to have better intellisense to a sql tool.)
I encourage people not to jump on the bandwagon just because everyone else is, learn enough to put it on your resume, be willing to use it if forced to, but try and really understand the pros and cons first.

Is ORM (Linq, Hibernate...) really that useful?

I have been playing with some LINQ ORM (LINQ directly to SQL) and I have to admit I like its expressive powers . For small utility-like apps, It also works quite fast: dropping a SQL server on some surface and you're set to linq away.
For larger apps however, the DAL never was that big of an issue to me to setup, nor maintain, and more often than not, once it was set, all the programming was not happening there anyway...
My, honest - I am an ORM newbie - question : what is the big advantage of ORM over writing a decent DAL by hand?
(seems like a double, couldn't find it though)
UPDATE : OK its a double :-) I found it myself eventually :
ORM vs Handcoded Data Access Layer
Strong-typing
No need to write the DAL yourself => time savings
No need to write SQL code yourself =>
less error-prone
I've used Hibernate in the past to dynamically create quite complex queries. The logic involved to create the appropriate SQL would have been very time-consuming to implement, compared with the logic to build the appropriate Criteria. Additionally, Hibernate knew how to work with various different databases, so I didn't need to put any of that logic in our code. We had to test against different databases of course, and I needed to write an extension to handle "like" queries appropriately, but then it ran against SQL Server, Oracle and HSqldb (for testing) with no issues.
There's also the fact that it's more code you don't have to write, which is always a nice thing :) I can't say I've used LINQ to SQL in anything big, but where I've used it for a "quick and dirty" web-site (very small, rarely updated, little benefit from full layer abstraction) it was lovely.
I used JPA in a project, and at first I was extremely impressed. Gosh it saved me all that time writing SQL! Gradually, however, I became a bit disenchanted.
Difficulty defining tables without surrogate keys. Sometimes we need tables that don't have surrogate keys. Sometimes we want a multicolumn primary key. TopLink had difficulties with that.
Forced datastructure relationships. JPA uses annotations to describe the relationship between a field and the container or referencing class. While this may seem great at first site, what do you do when you reference the objects differently in the application? Say for example, you need just specific objects that reference specific records based on some specific criteria (and it needs to be high-performance with no unnecessary object allocation or record retrieval). The effort to modify Entity classes will almost always exceed the effort that would have existed had you never used JPA in the first place (assuming you are at all successful getting JPA to do what you want).
Caching. JPA defines the notion of caches for your objects. It must be remembered that the database has its own cache, typically optimized around minimizing disk reads. Now you're caching your data twice (ignoring the uncollected GC heap). How this can be an advantage is beyond me.
Data != Objects. For high-performance applications, the retrieval of data from the DB must be done very efficiently. Forcing object creation is not always a good thing. For example, sometimes you may want arrays of primitives. This is about 30 minutes of work for an experienced programmer working with straight JDBC.
Performance, debugging.
It is much more difficult to gauge the performance of an application with complex things going on in the (sub-optimal, autogenerated) caching subsystem, further straining project resources and budgets.
Most developers don't really understand the impedence mismatch problem that has always existed when mapping objects to tables. This fact ensures that JPA and friends will probably enjoy considerable (cough cough) success for the forseeable future.
Well, for me it is a lot about not having to reinvent/recreate the wheel each time I need to implement a new domain model. It is simply a lot more efficient to use for instance nHibernate (my ORM of choice) for creating, using and maintaining the data access layer.
You don't specify exactly how you build your DAL, but for me I used to spend quite some time doing the same stuff over and over again. I used to start with the database model and work my way up from there, creating stored procedures etc. Even if I sometimes used little tools to generate parts of the setup, it was a lot of repetitive coding.
Nowadays I start with the domain. I model it in UML, and for most of the time I'm able to generate everything from that model, including the database schema. It need a few tweaks here and there, but with my current setup I get 95% of the job with the data access done in no time at all. The time I save I can use to fine tune the parts that need tuning. I seldom need to write any SQL statements.
That's my two cents. :-)
Portability between different db vendors.
My, honest - i am an ORM newbie - question : what is the big advance of ORM over writing a decent DAL by hand?
Not all programmers are willing or even capable of writing "a decent DAL". Those who can't or get scared from the mere thought of it, find LINQ or any other ORM a blessing.
I personally use LINQ to manipulate collections in the code because of its expressiveness. It offers a very compact and transparent way to perform some common tasks on collections directly in code.
LINQ will stop being useful to you when you will want to create very specific and optimized queries by hand. Then you are likely to get a mixture of LINQ queries intermingled with custom stored procedures wired into it. Because of this considerations, I decided against LINQ to SQL in my current project (since I have a decent (imho) DAL layer). But I'm sure LINW will do just fine for simple sites like maybe your blog (or SO for that matter).
With LINQ/ORM there may also be a consideration of lagging for high traffic sites (since each incoming query will have to be compiled all over again). Though I have to admit I do not see any performance issues on SO.
You can also consider waiting for the Entity Framework v2. It should be more powerful than LINQ (and hopefully not that bad as v1 (according to some people)).
Transparent persistence - changes get saved (and cascaded) without you having to call Save(). At first glance this seems like a nightmare, but once you get used to working with it rather than against it, your domain code can be freed of persistence concerns almost completely. I don't know of any ORM other than Hibernate / NHibernate that does this, though there might be some...
The best way to answer the question is to understand exactly what libraries like Hibernate are actually accomplishing on your behalf. Most of the time abstractions exist for a reason, often to make certain problems less complex, or in the case Hibernate is almost a DSL for expression certain persistance concepts in a simple terse manner.
One can easily change the fetch strategy for collections by changing an annotation rather than writing up lots of code.
Hibernate and Linq are proven and tested by many, there is little chance you can achieve this quality without lots of work.
Hibernate addresses many features that would take you months and years to code.
Also, while the JPA documentation says that composite keys are supported, it can get very (very) tricky quickly. You can easily spend hours (days?) trying to get something quite simple working. If JPA really makes things simpler then developers should be freed from thinking too much about these details. It doesn't, and we are left with having to understand two levels of abstraction, the ORM (JPA) and JDBC. For my current project I'm using a very simple implementation that uses a package protected static get "constructor" that takes a ResultSet and returns an Object. This is about 4 lines of code per class, plus one line of code for each field. It's simple, high-performance, and quite effective, and I retain total control. If I need to access objects differently I can add another method that reads different fields (leaving the others null, for example). I don't require a spec that tells me "how ORMs must (!) be done". If I require caching for that class, I can implement it precisely as required.
I have used Linq, I found it very useful. I saves a lot of your time writing data access code. But for large applications you need more than DAL, for them you can easily extent classes created by it. Believe me, it really improves your productivity.

Resources