I would like to apply a Criteria query to an in-memory collection
of entities, instead of on the database. Is this possible?
To have Criteria API work like LINQ? Or alternatively, convert
Criteria query to LINQ query.
Thanks!
I don't believe you can use Criteria to query against an in-memory collection and come to think about it it doesn't seem to make much sense. If I'm understanding everything correctly you've already queried against your database. I'd suggest to either tune your original query (whichever method you choose) to include all of your filters. Or you could use LINQ (as you suggested) to refine your results.
Also, what's your reasoning for wanting to query from memory?
It sounds like you're rolling your own caching mechanism. I would highly recommend checking out NHibernate's 2nd level cache. It handles many complex scenarios gracefully such as invalidating query results on updates to the underlying tables.
http://ayende.com/Blog/archive/2009/04/24/nhibernate-2nd-level-cache.aspx
Related
When it comes to performance, should i use .ToList().Distinct() or .Distinct().ToList() ?
Both extension methods generate the same SQL query or not?
It seems that the second approach should perform better but is that true?
Are there any advantages or disadvantages of using one over another?
Short Answer: .Distinct().ToList()
Explain:
ToList: It converts an IEnumerable<T> to a List<T>, It's called Immediate execution. So you should filter all data in DB Server first instead of get all data then Distinct in "client-side"
It depends. If it is a query that is executed against a List<T> or a Dictionary<K,V> then the latter (Distinct().ToList()) would be preferrable.
The reason being, that if you do .ToList().Distinct(), Distinct() returns an IEnumerable that has to be executed again to get a real collection. In essence, you create two collections, but you would never use the first one.
There is a situation however where .ToList().Distinct() can be preferrable and that is if you are working with a Object-to-Relational mapper (see: EntityFramework) and you want to fetch all rows from a database table (maybe to populate a cache in the background or to use up less CPU on the database) and then do the .Distinct() operation locally.
Your mention of SQL suggests that your datasource is a DBContext of some kind.
In that situation, by definition, once you have done .ToList() all available data has been converted to objects in .NET Memory. Doing a .Distinct() after that can only run in .NET memory - it will run as if there is no database.
The SQL query for the above is definitely not the same as for .Distinct().ToList(), which will let the database do the DISTINCT operation.
To achieve the best performance, the best thing to do is .Distinct().ToList().
I have multiple migrations (say around 10) and the corresponding models for the tables. The problem I am facing is that all the migrations/tables have multiple primary keys but the model has just a string variable to define primary key ($primaryKey), hence, when I save or update a table row using where clause it would just take one primary key and miss the other and hence end up changing multiple rows instead of one. So, basically I switched to DB Queries and it worked well.
So, my question to you is that is there any performance gain with model? Or it is just a designing paradigm? Is there any option to do the same thing (Have multiple primary key) within a model? I know by overriding the Eloquent methods we can do this but is there any other good option?
Using an ORM like Eloquent is a convenience, not a requirement. From what I've heard there's actually often a slight performance decrease when using an ORM because there's the additional overhead of translating that query into the relevant SQL.
Often, using an ORM makes for queries that are much easier to understand, and for that reason alone it's worth using. However, for more complex queries, an ORM is likely to get in the way, and so you should consider using another method to query the database. You don't have to use just an ORM - you can mix and match the different methods as you see fit.
Laravel has the Query Builder as an alternative to writing raw SQL or using the ORM, and that may be a better option for you here. I would avoid writing raw SQL if you can because both Eloquent and the Query Builder will handle escaping the parameters for you, to help avoid SQL injection vulnerabilities.
My choice would be to use the ORM where possible, and fall back to the Query Builder when the ORM gets in the way.
Is it performant wise to use greedy LINQ operators such as ToList,ToLookUp,Distinct etc?
What would be a best practice(s) for LINQ query execution?
You often use for your objects List<> or making all your objects lists to IEnumerable<>. I know the latest gives more flexibility.
When working with memory (LINQ to Objects) it's ok to always use deffered loading, cause you can access it whenever you need without fear that tha data changed, added or inserted as the reference will execute the query as soon as you need access. But this changes with database LINQ queries such LINQ to EF.
Would like a StackOverflow users opinion.
Thank you!
What would be a best practice(s) for LINQ query execution?
A List may be accessed by index, a Lookup may be accessed by Key. These types are obviously serializable across a WCF boundary. A deferred IEnumerable doesn't do these things well.
For EF or LinqToSql, one must run their queries before the DataContext or whatever holds the SqlConnection gets disposed.
In my code, I use deferred IEnumerables only for method scoped variables when convenient. I use List for properties (sometimes the property constructs the List, but usually it's just backed by an instance) and method return types. Since I'm doing comparatively expensive things (like accessing the database or using WCF), the performance of eagerly executing in-memory Linq queries has never been an issue.
The final authority on any performance question is: how does it measure?
I've seen a lot of people talking about IQueryable and I haven't quite picked up on what all the buzz is about. I always work with generic List's and find they are very rich in the way you can "query" them and work with them, even run LINQ queries against them.
I'm wondering if there is a good reason to start considering a different default collection in my projects.
The IQueryable interface allows you to define parts of a query against a remote LINQ provider (typically against a database, but doesn't have to be) in multiple steps, and with deferred execution.
E.g. your database layer could define some restriction (e.g. based on permissions, security - whatever) by adding a .Where(x => x.......) clause to your query. But this doesn't get executed just yet - e.g. you're not retrieving 150'000 rows that match that criteria.
Instead, you pass up the IQueryable interface to the next level, the business layer, where you might be adding additional requirements and where clauses to your query - again, nothing gets executed just yet, you're also not tossing out 80'000 of your 150'000 rows you retrieved - you're just defining additional query criteria.
And the UI layer might do the same thing, e.g. based on user input in a form or something.
The magic is that you're passing the IQueryable interface through all the layers, adding additional critieria to it - but it doesn't get executed / evaluated until you actually force it. This also means you're not needlessly selecting and retrieving tons of data which you end up discarding afterwards.
You can't really do that with a classic static list - you have to pick the data, possibly discarding a lot of it again later on in the process - you have a static list, after all.
IQueryable allows you to make queries using LINQ, just like the LINQ to Object queries, where the queries are actually "compiled" and run elsewhere.
The most common implementations work against databases. If you use List<T> and LINQ to Objects, you load the entire "table" of data into memory, then run your query against it.
By using IQueryable<T>, the LINQ provide can "translate" your LINQ statement into actual SQL code, and run it on the database. The results can be returned to you and enumerated.
This is much, much more efficient, especially if you're working in N-Tiered systems.
LINQ queries against IEnumerable<T> produce delegates (methods) which, when invoked, perform the described query.
LINQ queries against IQueryable<T> produce expression trees, a data structure which represents the code that produced the query. LINQ providers such as LINQ to SQL interpret these data structures, generating the same query on the target platform (T-SQL in this case).
For an example of how the compiler interprets the query syntax against IQueryable<T>, see my answer to this question:
Building Dynamic LINQ Queries based on Combobox Value
I am considering refactoring a repository I have to improve its flexibility and reduce the method count.
Where there are the following methods:
Collection GetAllUsersByRole(Role role)
User GetUserByuserName(string userName)
...I would like to have a single method taking a Linq expression:
ICollection GetUsers(Expression e)
{
//retrieve user collection from store
//apply expression to collection and return
}
Is this a reasonable approach? Presumably I'd lose some efficiency because the full users collection would need to be retrieved and filtered every time, rather than retrieving a subset of users according to some hard-coded criteria?
Edit: NHibernate provides ORM in my implementation.
You really want to take an Expression as the argument to that method.
As far as performance, it really comes down to how far you want to go with it. The simplest method is bringing all the objects into memory and then filtering with the predicate expression.
On the other hand, you mention some sort of criteria. I have no idea what your back end data system is, but you can take these passed filters and transform them into your criteria. This is essentially what Linq to SQL and Linq to Entities does, but hopefully the range of possibilities you need to support is significantly smaller. If not, it might make sense to switch to one of the ORM tools if you want to take this approach.
This is not a very reasonable approach, cos typically will cost you a lot of performance issues. If you use data access technology that accepts LINQ queries than you just can use this query (expression) with it. This can be IQueryable for LINQ to SQL or ObjectQuery for EntityFramework. It also can be ICriteria (without linq support) for nHibernate. All modern ORM tools have its own expression API, so you just need to use it. If you have custom Data Access layer you will need to write your own API for creating criterias, for example Query Object.