I've been using LINQ to SQL for a while, and most of the time I end up creating a complex linq expressions using SQLFunctions and DbEntities classes to mimic functions such as toString or DateTime formating (e.g. Linq to SQL Format Date time in One Take) not available in Linq to SQL.
I always thought that this is much cleaner than forcing query execution using ToList() and then using linq again but this time with objects with all the available functions. My questions is, which is a better approach, try to stuff everything in one Linq to SQL expression with efforts to format the data and everything, or go the easy way and just extract the data in one step and then process it using ForEach on the resulting list from the first query.
Related
We must create and show at runtime (asp.net mvc) some complex reports from Oracle tables data with millions of records. The reports data must be obtained from groupings and little complex calculations.
So is it better for performance and maintainability of code that do these groupings and calculations via sql query (pl/sql) or via linq?
Thanks for your kindle reply
So is it better for performance and maintainability of code that do
these groupings and calculations via sql query (pl/sql) or via linq?
It depends on what you mean by via linq. If you mean that you fetch the complete table to local memory and then use linq statements to extract the result that you want, then of course SQL statements are faster.
However, if you mean that you use Entity Framework, or something similar, then the answer is not a easy to give.
If you use Entity Framework (or some clone), your tables will be represented by IQueryable<...> instead of IEnumerable<...>. An IQueryable has an Expression and a Provider. The Expression represents the query that must be performed. The Provider knows which system must execute the query (usually a Database Management System) and how to communicate with this system. When the query must be executed, it is the task of the Provider to translate the Expression into the language that the system knows (usually something SQL-like) and to execute the SQL-query.
There are two kinds of IQueryable LINQ statements: those that return an IQueryable<...> of something, and those that return a TResult. The ones that return IQueryable only change the Expression. They are functions that use deferred execution.
Function that do not return an IQueryable, are ToList(), FirstOrDefault(), Any(), Max(), etc. Internally they will call functions that will GetEnumerator() (usually via a foreach), which orders the Provider to translate the Expression and execute the query.
Back to your question
So which one is more efficient, entity framework or SQL? Efficiency is not only the time to perform the queries, it is also the development/testing time, for the first version and for future changes in the software.
If you use an entity-framework (-clone), the SQL-queries created from the Expressions are pretty efficient, depending on the framework manufacturer. If you look at the code, then sometimes the SQL query is not the optimal one, although you'll have to be a pretty good SQL-programmer to improve most queries.
The big advantage above using Entity Framework and LINQ queries above SQL statements is that development times will be shorter. The syntax of the LINQ statements is checked at compile time, SQL statements at run-time. Development and test periods will be shorter.
It is easy to reuse LINQ statements, while SQL statements almost always have to be written especially for the query you want to execute. LINQ statements can be tested without a database on any sequence of items that represent your tables.
My Advice
For most queries you won't notice any difference in execution time between the entity framework query or the SQL query.
If you expect complicated queries and future changes, I'd go for entity framework. With main argument the shorter development time, the better testing possibilities, and the better maintainability.
If you detect some queries where you notice that the execution time is too long, you can always decide to bypass entity framework by executing a SQL query instead of using LINQ.
If you've wrapped your DbContext in a proper repository, where you hide the use cases from their implementations, the users of your repository won't notice the difference.
When using an ORM, and writing queries using LINQ, Who is responsible for transforming the LINQ espression tree to a native SQL? is it the ORM itself, or the CLR? or something else?
As Philippe said in comment - the ORM does it. CLR just compiles your LINQ expression as expression tree, and pass this object to IQueryProvider related to the ORM.
Then this IQueryProvider does parsing of this expression with custom ExpressionVisitor classes. And then based on parsing results - ORM generates pure SQL code, execute it and materialize result.
Some ORM has optimization for this process. For example Entity Framework saves parsed information in memory into something like SQL Execution Plans, and then just uses it in future queries, so it parses it just once, and then reuse data parsed from an expression tree.
If expression tree doesn't have linked parameters, then its easy. Otherwise it needs to partially reparse it again for each execution to get used parameter and generate SQL with them.
I currently have a SQL query that needs to be rewritten in C# code that interrogates 2 different sharepoint lists.
Given that this query filters using the SQL year() function, has multiple unions and subqueries, how should I be writing this in code? CAML queries or LINQ seem excessive and slow when the query will be converted back into SQL to be run anyway (the lists in question are actually tables surfaced as lists through Access Services, so it seems stupid to convert a database query into code, in order to run a database query!)
I ended up doing this by importing all the data from the relevant table's lists into an in-memory SQLite database, using the client object model, and then running a modified SQL query on the SQLite tables. As there wasn't that much data this was an acceptable method.
I have an SQL query with joins. I want to use Linq. some one suggested me to avoid joins and use Linq include. How can I combine entities using Linq include? Is there a better way than joins ?
Thanks
LINQ to SQL converts LINQ syntax to an SQL query. If the query requires a Join, then LINQ to SQL will actually execute an SQL query with joins against the database. The advantage you may gain is that the LINQ to SQL query may be more optimized than the one you have written.
If you are not familiar with using LINQ and you have a working application, you will probably not gain anything other than complication by adding an additional layer of abstraction (LINQ). That being said, LINQ is an excellent and powerful tool. However, that doesn't change the truth to the adage "If it ain't broke don't fix it."
Basically, what you are considering is converting your application's data access procedure from
app -> SQL Query with Joins -> data objects
to
app -> LINQ Query with Joins -> SQL Query with Joins -> data objects
I am pretty well versed in both, and I think writing LINQ queries is a lot easier than writing SQL queries. (You get strongly typed models and Intellisense). However, if you only know SQL, then writing an SQL query is significantly easier than learning LINQ and writing a LINQ query .
Note that LINQ also supports joins. Here is an article on using joins in LINQ: http://odetocode.com/Blogs/scott/archive/2008/03/25/inner-outer-lets-all-join-together-with-linq.aspx .
I've seen a lot of people talking about IQueryable and I haven't quite picked up on what all the buzz is about. I always work with generic List's and find they are very rich in the way you can "query" them and work with them, even run LINQ queries against them.
I'm wondering if there is a good reason to start considering a different default collection in my projects.
The IQueryable interface allows you to define parts of a query against a remote LINQ provider (typically against a database, but doesn't have to be) in multiple steps, and with deferred execution.
E.g. your database layer could define some restriction (e.g. based on permissions, security - whatever) by adding a .Where(x => x.......) clause to your query. But this doesn't get executed just yet - e.g. you're not retrieving 150'000 rows that match that criteria.
Instead, you pass up the IQueryable interface to the next level, the business layer, where you might be adding additional requirements and where clauses to your query - again, nothing gets executed just yet, you're also not tossing out 80'000 of your 150'000 rows you retrieved - you're just defining additional query criteria.
And the UI layer might do the same thing, e.g. based on user input in a form or something.
The magic is that you're passing the IQueryable interface through all the layers, adding additional critieria to it - but it doesn't get executed / evaluated until you actually force it. This also means you're not needlessly selecting and retrieving tons of data which you end up discarding afterwards.
You can't really do that with a classic static list - you have to pick the data, possibly discarding a lot of it again later on in the process - you have a static list, after all.
IQueryable allows you to make queries using LINQ, just like the LINQ to Object queries, where the queries are actually "compiled" and run elsewhere.
The most common implementations work against databases. If you use List<T> and LINQ to Objects, you load the entire "table" of data into memory, then run your query against it.
By using IQueryable<T>, the LINQ provide can "translate" your LINQ statement into actual SQL code, and run it on the database. The results can be returned to you and enumerated.
This is much, much more efficient, especially if you're working in N-Tiered systems.
LINQ queries against IEnumerable<T> produce delegates (methods) which, when invoked, perform the described query.
LINQ queries against IQueryable<T> produce expression trees, a data structure which represents the code that produced the query. LINQ providers such as LINQ to SQL interpret these data structures, generating the same query on the target platform (T-SQL in this case).
For an example of how the compiler interprets the query syntax against IQueryable<T>, see my answer to this question:
Building Dynamic LINQ Queries based on Combobox Value