I'm using KendoUI grid and using its ToDataSourceResult to filter my data (as suggested by the documentation) but i'm worried about the performance impact.
Based on my understanding, the following suggested code get all records from database into memory and performm ToDataSourceResult extension method to filter the records in memory (LINQ fluent api concept). Will this have great performance impact if i have lots of records?
I suppose having a query to the database with the "where" clause upfront will have better performance... please advise.
Here's what suggested in the documentation
var countries = _database.Countries.GetAll();
return Json(countries.ToDataSourceResult(request, record => new
{
record.Id,
record.Name,
record.Currency,
record.TimeZone
}));
This is other option without using ToDataSource filter
var selectedCountries = _database.Countries.GetCountryStartWith("m")
It depends on what your GetAll method returns. If it returns IQueryable from a LINQ-enabled provider (Entity Framework, Linq to SQL or anything else) all operations will be performed at database level (not in memory). If it returns all records - well you have a problem even before using ToDataSourceResult.
Related
I am struggling a bit on what probably is a simple matter or something I misunderstand... But anyway, using Linq entity code first, I am trying to keep some of my tables to be inaccessible from the client, without success.
Using Breeze, I have made a datacontext that holds only the dbsets I want exposed, fine.
But when I write a query using .expand(). For example, let's say I have a posts table which I want to expose, and an Owner table that I want to hide.
Using a query like:
var query = EntityQuery
.from('Posts')
.expand('Owner');
I can still see all the columns from Owner.
So the question is: in Linq, how am I supposed to secure/protect/hide the tables, and/or specific columns, that I want to hide?
After some digging, all I have found is the [JsonIgnore] attribute, which seems insufficient to me.
What is the best way to do this? I feel I am missing something probably huge here, but it's the end of the day around here...
Thanks
If you are using the Breeze's WebApi implementation then Breeze also supports ODataQueryOptions ( see here and here ).
This allows you to mark up your controller methods so as to limit how the query is interpreted. For example, to only allow filtering on your 'Posts' query and therefore exclude the ability to "expand" or "select" 'Owners' from any 'Posts' request you could do the following.
[Queryable(AllowedQueryOptions=AllowedQueryOptions.Filter| AllowedQueryOptions.Top | AllowQueryOptions.Skip)]
public IQueryable<Posts> Posts() {
....
}
Ok apparently my question was already addressed here:
Risks of using OData and IQueryable
I just found it.
I would like to provide an OData interface for my application. The examples that I have seen use EF to map the LINQ queries to SQL queries.
IMHO it this approach pretty much exposes the physical database model to the world (I know EF/NH give some flexibility, but it is limited).
What I would like the be able to do, is the following:
Define my Data Contract via some DTOs.
Have a OData Service that will let users query over my Data Contract Dtos.
Have some translation layer to translate the queries over the DTOs to queries over, let's say, EF model or NH.
Execute the translated query.
Map the results back to my Data Contracts.
Am I out of my mind or is there a solution to this problem?
I have 2 models, the "contract" model and the "persisted" model. The persisted model is what Entity Framework is mapped to. The Get method that returns an IQueryable returns a IQueryable which is just something along the lines of:
return dbContext.PersistedCustomers.Select(x => new Customer(Name = x.OtherName, ...));
At least when using DbContext as opposed to ObjectContext, Where criteria based on the contract model get translated automatically into Where criteria of the PersistedModel to be executed against the database. Hopefully the differences between the two aren't that complex that you need some weird data massaging. I'm sure there's limits to the reversal it does.
One way of doing it would be to create a ViewModel that will represent your Model and then use AutoMapper to map between them. You can use like this:
var address = _Context.Addresses.Where(p => p.AddressID == addressID).Single();
AddressVM result = Mapper.Map<AddressVM>(address);
I need to perform queries in a ADO.NET model by using LINQ.
I have an EF class for the database of my web store WebStoreEntities and I need to find all the products, in the table Products, that are currently under discount. The instance of WebStoreEntities is called webStoreDB.
I have already available a method that queries all the product from the database:
public IList<Products> GetAllProducts()
{
return webStoreDB.Products.ToList();
}
In order to write a method with signature IList<Products> GetAllDiscountProducts() that retrieves all the discounted products, is it faster to write:
return webStoreDB.Products.Where(m => Equals(m.discounted, 1)).ToList();
or
return GetAllProducts().Where(m => Equals(m.discounted, 1)).ToList();
Linq uses streaming and deferred execution so until the ToList() is called your query hasn't been executed in your database context.
(You should know you're using linq-to-entities here so your linq is composed into an sql query and then executed)
GetAllProducts() calls ToList():
At this point the sql is sent to your database, executed and then it returns in memory objects. These in-memory objects are then filtered by this part:
.Where(m => Equals(m.discounted, 1)).ToList();
(And this is part now Linq-To-Objects)
It is a HUGE snag because it will Execute an sql query to return ALL products (=>SELECT * FROM PRODUCTS). And if you have a million products, they'll all be returned in memory!!
I recommend HRH Jon Skeets Edulinq series, it should give you a good understanding of deferred execution. Especially the article on .Where();
return webStoreDB.Products.Where(m => Equals(m.discounted, 1)).ToList();
This is faster because the filtering is done in the database where as
return GetAllProducts().Where(m => Equals(m.discounted, 1)).ToList();
filtering will be done in memory of the client. Basically LINQ statements issued against IQueriable interface will be translated to SQL when the provider is some relational database.
But if you use LINQ with IEnumerable interface then the query is executed in memory.
I've noticed that depending on how I extract data from my Entity Framework model, I get different types of results. For example, when getting the list of employees in a particular department:
If I pull directly from ObjectContext, I get an IQueryable<Employee>, which is actually a System.Data.Objects.ObjectQuery<Employee>:
var employees = MyObjectContext.Employees.Where(e => e.DepartmentId == MyDepartment.Id && e.SomeCondtition)
But if I use the Navigation Property of MyDepartment, I get an IEnumerable<Employee>, which is actually a System.Linq.WhereEnumerableIterator<Employee> (private class in System.Linq.Enumerable):
var employees = MyDeparment.Employees.Where(e => e.SomeCondtition)
In the code that follows, I heavily use employees in several LINQ queries (Where, OrderBy, First, Sum, etc.)
Should I be taking into consideration which query method I use? Will there be a performance difference? Does the latter use deferred execution? Is one better practice? Or does it not make a difference?
I ask this because since installing ReShaper 6, I'm getting lots of Possible multiple enumeration of IEnumerable warnings when using the latter method, but none when using direct queries. I've been using the latter method more often, simply because it's much cleaner to write, and I'm wondering if doing so has actually had a detrimental effect!
There is very big difference.
If you are using the first approach you have IQueryable = exression tree and you can still add other expressions and only when you execute the query (deferred execution) the expression tree will be converted to SQL and executed in the database. So if you use your first example and add .Sum of something you will indeed execute operation in the database and it will transfer only single number back to your application. That is linq-to-entities.
The second example uses in memory collection. Navigation property doesn't represent IQueryable (expression tree). All linq commands are treated as linq-to-objects = all records representing related data in navigation property must be first loaded from database to your application and all operations are done in memory of your application server. You can load navigation property eagerly (by using Include), explicitly (by using Load) or lazily (it is just done automatically when you access the property for the first time if lazy loading is enabled). So if you want to have sum of something this scenario requires you to load all data from database and then execute the operation locally.
I am using an IList<Employee> where i get the records more then 5000 by using linq which could be better? empdetailsList has 5000
Example :
foreach(Employee emp in empdetailsList)
{
Employee employee=new Employee();
employee=Details.GetFeeDetails(emp.Emplid);
}
The above example takes a lot of time in order to iterate each empdetails where i need to get corresponding fees list.
suggest me anybody what to do?
Linq to SQL/Linq to Entities use a deferred execution pattern. As soon as you call For Each or anything else that indirectly calls GetEnumerator, that's when your query gets translated into SQL and performed against the database.
The trick is to make sure your query is completely and correctly defined before that happens. Use Where(...), and the other Linq filters to reduce as much as possible the amount of data the query will retrieve. These filters are built into a single query before the database is called.
Linq to SQL/Linq to Entities also both use Lazy Loading. This is where if you have related entities (like Sales Order --> has many Sales Order Lines --> has 1 Product), the query will not return them unless it knows it needs to. If you did something like this:
Dim orders = entities.SalesOrders
For Each o in orders
For Each ol in o.SalesOrderLines
Console.WriteLine(ol.Product.Name)
Next
Next
You will get awful performance, because at the time of calling GetEnumerator (the start of the For Each), the query engine doesn't know you need the related entities, so "saves time" by ignoring them. If you observe the database activity, you'll then see hundreds/thousands of database roundtrips as each related entity is then retrieved 1 at a time.
To avoid this problem, if you know you'll need related entities, use the Include() method in Entity Framework. If you've got it right, when you profile the database activity you should only see a single query being made, and every item being retrieved by that query should be used for something by your application.
If the call to Details.GetFeeDetails(emp.Emplid); involves another round-trip of some sort, then that's the issue. I would suggest altering your query in this case to return fee details with the original IList<Employee> query.