My entity "TimeRecord" has a collection of "WayPoints" and two one-to-one properties "Location" and "WayData".
Each property can be null.
I need to export all Time Records with initialized properties for a specific User.
I actually had a working solution but then I started to use NHibernateProiler and first I noticed that this code results in a ridiculous number of query’s against db.
var query = (from timeRecord in Session.Query<TimeRecord>()
.Where(tr => tr.User.Id == userid)
select timeRecord);
Then I changed my code to:
var query = (from post in Session.Query<TimeRecord>()
.Fetch(x => x.Location)
.Fetch(x => x.WayData)
.FetchMany(x => x.WayPoints)
.Where(tr => tr.User.Id == userid)
select post);
Which lead me to the Cartesian product problem.
Right now I’m experimenting with this piece of code:
var sql1 = "from TimeRecord b left outer join fetch b.Location where b.User.Id=:User_id";
var sql2 = "from TimeRecord b left outer join fetch b.WayData where b.User.Id=:User_id";
var sql3 = "from TimeRecord b left inner join fetch b.WayPoints where b.User.Id=:User_id";
var result = Session.CreateMultiQuery()
.Add(Session.CreateQuery(sql1))
.Add(Session.CreateQuery(sql2))
.Add(Session.CreateQuery(sql3))
.SetParameter("User_id", userid)
.List();
But I can’t say if this is the correct approach or if this is even possible with nHibernate. Can someone help me with that?
The 1+N issue is a usual with the entity/collection mapping and ORM tools. But NHibernate has a very nice solution how to manage it properly. It is called:
19.1.5. Using batch fetching
This setting will allow:
To continue querying the root entity (TimeRecord in our case)
No fetching inside of the query (Session.Query<TimeRecord>()). That means we do have support for correct paging. (Take(), Skip() will be executed over the flat root table)
All the collections will be loaded with their own SELECT statements (could seem as disadvantage but se below)
There will be much more less SELECTs then 1+N. All of them will be batched. E.g. by 25 records
all the native mapping (lazy loading of collections) will still be in place...
The xml mapping example:
-- class level
<class name="Location" batch-size="25 ...
-- collection level
<batch name="Locations" batch-size="25" ...
I would suggest to apply that over all your collections/classes. With Fluent mapping it could be done also with conventions
The fluent mapping:
// class
public LocationMap()
{
Id(x => x....
...
BatchSize(25);
// collection
HasMany(x => x.Locations)
...
.BatchSize(25);
Related
This question already has answers here:
EntityFramework - contains query of composite key
(12 answers)
Closed 2 years ago.
I have a table as follows:
PersonalDetails
Columns are:
Name
BankName
BranchName
AccountNo
Address
I have another list that contains 'Name' and 'AccountNo'.
I have to find all the records from table that whose respective 'Name' and 'AccountNo' are present in given list.
Any suggestion will be helpful.
I have done following but not of much use:
var duplicationhecklist = dataAccessdup.MST_FarmerProfile
.Join(lstFarmerProfiles,
t => new { t.Name,t.AccountNo},
t1 => new { t1.Name, t1.AccountNo},
(t, t1) => new { t, t1 })
.Select(x => new {
x.t1.Name,
x.t1.BankName,
x.t1.BranchName,
x.t1.AccountNo
}).ToList();
where lstFarmerProfiles is a list.
You probably found out that you can't join an Entity Framework LINQ query with a local list of entity objects, because it can't be translated into SQL. I would preselect the database data on the account numbers only and then join in memory.
var accountNumbers = lstFarmerProfiles.Select(x => x.AccountNo).ToArray();
var duplicationChecklist =
from profile in dataAccessdup.MST_FarmerProfile
.Where(p => accountNumbers
.Contains(p.AccountNo))
.AsEnumerable() // Continue in memory
join param in lstFarmerProfiles on
new { profile.Name, profile.AccountNo} equals
new { param.Name, param.AccountNo}
select profile
So you will never pull the bulk data into memory but the smallest selection you can probably get to proceed with.
If accountNumbers contains thousands of items, you may consider using a better scalable chunky Contains method.
Since you have the lists in .net of values you want to find, try to use the Contains method, for sample:
List<string> names = /* list of names */;
List<string> accounts = /* list of account */;
var result = db.PersonalDetails.Where(x => names.Contains(x.Name) && accounts.Contains(x.AccountNo))
.ToList();
If MST_FarmerProfile is not super large I think you best option is to bring it into memory using AsEnumerable() and do the joining there.
var duplicationhecklist =
(from x in dataAccessdup.MST_FarmerProfile
.Select(z => new {
z.Name,
z.BankName,
z.BranchName,
z.AccountNo
}).AsEnumerable()
join y in lstFarmerProfiles
on new { x.Name, x.AccountNo} equals new { y.Name, y.AccountNo}
select x).ToList();
Since data is usually located on different machines or in separate processes at least: DB - is one and your in-memory list is your app, there is just 2 ways to do it.
Download as small data part from DB to local as possible and join locally (usually using AsEnumerable() or basically ToList()). You got many good thoughts on this in other answers.
Another one is different - upload your local data to server somehow and perform query on DB side. Uploading can be done differently: using some temp tables OR using VALUES. Fortunately there is a small extension for EF now (for both EF6 and EF Core) which you could try. It is EntityFrameworkCore.MemoryJoin (name might be confusing, but it supports both EF6 and EF Core). As stated in author's article it modifies SQL query passed to server and injects VALUES construction with data from your local list. And query is executed on DB server.
If accountNo identifies the record then you could use:
var duplicationCheck = from farmerProfile in dataAccessdup.MST_FarmerProfile
join farmerFromList in lstFarmerProfiles
on farmerProfile.AccountNo equals farmerFromList.AccountNo
select new {
farmerProfile.Name,
farmerProfile.BankName,
farmerProfile.BranchName,
farmerProfile.AccountNo
};
If you need to join on name and account then this should work:
var duplicationCheck = from farmerProfile in dataAccessdup.MST_FarmerProfile
join farmerFromList in lstFarmerProfiles
on new
{
accountNo = farmerProfile.AccountNo,
name = farmerProfile.Name
}
equals new
{
accountNo = farmerFromList.AccountNo,
name = farmerFromList.Name
}
select new
{
farmerProfile.Name,
farmerProfile.BankName,
farmerProfile.BranchName,
farmerProfile.AccountNo
};
If you are only going to go through duplicateChecklist once then leaving .ToList() out will be better for performance.
Using Linq to Entity (Entity Framework) in MVC 3 project.
My model:
Table - Users
UserID (PK)
...
Table - Clients
ClientID (PK)
Table - PropertyItems
PropertyItemID (PK)
Table - MemberContactPreference (Contains PropertyItems selected by Users- many to many)
UserID(FK)
PropertyItemID(FK)
Table ClientProperties (Contains PropertyItems that belong to Clients - many to many)
ClientID (FK)
PropertyItemID (FK)
I want to list all the distinct users that have selected all the properties selected by clients.
My Approach :
I got a list of all properties for a particular client in
Iqueryable<ClientProperty> clientProperties = GetClientProperties(ClientID)
Iqueryable<User> UsersMatchingClientProperties = GetAllUsers();
foreach (ClientProperty property in clientproperties)
{
UsersMatchingClientProperties = (from uem in UsersMatchingClientProperties
join ucp in GetAllMemberContactPreferences on
ucp.UserID == uem.UserID
where uem.MemberContactPreferences.SelectMany(
mcp => mcp.PropertyItemID == property.PropertyItemID)
select uem).Distinct;
}
It gives the right result only first time. As it doesn't reduce the number of items in UsersMatchingClientProperties with each iteration. actually it replaces the collection with new resultset. I want to filter out this collection with each iteration.
Also, any suggestions to do this in Lambda expression without using Linq.
Thanks
That generation of an iqueryable in a for loop seems like a dangerous thing, which could end up in a monster sql join being executed at once.
Anyway, I don't think you need that. How about something like this?
// for a given client, find all users
// that selected ALL properties this client also selected
Iqueryable<ClientProperty> clientProperties = GetClientProperties(ClientID)
Iqueryable<User> allUsers= GetAllUsers();
Iqueryable<MemberContactPreference> allMemberContactProperties = GetAllMemberContactPreferences();
Iqueryable<User> UsersMatchingClientProperties = allUsers
.Where(user => allMemberContactProperties
.Where(membP => membP.UserID==user.UserID)
.All(membP => clientProperties
.Select(clientP => clientP.PropertyID)
.Contains(membP.PropertyID)
)
);
Here is an alternative query in case you want the users that selected ANY property for a given client
// for a given client, find all users
// that selected ANY properties this client also selected
Iqueryable<ClientProperty> clientProperties = GetClientProperties(ClientID)
Iqueryable<User> allUsers= GetAllUsers();
Iqueryable<MemberContactPreference> allMemberContactProperties = GetAllMemberContactPreferences();
Iqueryable<User> UsersMatchingClientProperties = clientproperties
.Join(allMembersContactProperties, // join clientproperties with memberproperties
clientP => clientP.PropertyItemID,
membP => membP.PropertyItemID,
(clientP, membP) => membP)) // after the join, ignore the clientproperties, keeping only memberproperties
.Distinct() // distinct is optional here. but perhaps faster with it?
.Join(allUsers, //join memberproperties with users
membP => membP.UserID,
user => user.UserID,
(membP, user) => user)) // after the join, ignore the member properties, keeping only users
.Distinct();
I trust Hugo did a good job suggesting ways to improve your query (+1). But that does not yet explain the cause of your problem, which is the modified closure pitfall.
I think that after your loop there is some code that actually executes the query in UsersMatchingClientProperties. At that moment the query is executed with the last value of the loop variable property! (The loop variable is the closure in each query delegate that is created in an iteration, and it is modified by each iteration).
Change the loop like this:
foreach (ClientProperty property in clientproperties)
{
var property1 = property;
...
and use property1 in the query. That should solve the cause of the problem. But as said, it looks like the whole process can be improved.
I'm trying to use LINQ to query the following Entity Data Model
based on this db model
I'd like to be able to pull a list of products based on ProductFacets.FacetTypeId.
Normally, I'd use joins and this wouldn't be a problem but I don't quite understand how to query many-to-many tables under the Entity DataModel.
This is an example sql query:
select p.Name, pf.FacetTypeId from Products p
inner join ProductFacets pf on p.ProductId = pf.ProductId
where pf.FacetTypeId in(8, 12)
Presuming EF 4:
var facetIds = new [] { 8, 12 };
var q = from p in Context.Products
where p.FacetTypes.Any(f => facetIds.Contains(f.FacetTypeId))
select p;
In EF (assuming the mapping is done correctly), joins are hardly ever used; navigation properties are used instead.
Your original SQL returns a tuple with repeated Name entries. With LINQ, it's often easier
to "shape" the queries into non-tuple results.
The following should be the same as the SQL, only instead of returning (Name, FacetTypeId) pairs with repeated Names, it will return a type that has a Name and a sequence of FacetTypeIds:
var facetIds = new [] { 8, 12 };
var result = from p in db.Products
select new
{
p.Name,
FacetTypeIds = from pf in p.FacetTypes
where pf.FacetTypeId == 8 || pf.FacetTypeId == 12
select pf.FacetTypeId,
};
I have this sql that i want to have written in linq extension method returning an entity from my edm:
SELECT p.[Id],p.[Firstname],p.[Lastname],prt.[AddressId],prt.[Street],prt.[City]
FROM [Person] p
CROSS APPLY (
SELECT TOP(1) pa.[AddressId],a.[ValidFrom],a.[Street],a.[City]
FROM [Person_Addresses] pa
LEFT OUTER JOIN [Addresses] AS a
ON a.[Id] = pa.[AddressId]
WHERE p.[Id] = pa.[PersonId]
ORDER BY a.[ValidFrom] DESC ) prt
Also could this be re-written in linq extension method using 3 joins?
Assuming you have set the Person_Addresses table up as a pure relation table (i.e., with no data besides the foreign keys) this should do the trick:
var persons = model.People
.Select(p => new { p = p, a = p.Addresses.OrderByDescending(a=>a.ValidFrom).First() })
.Select(p => new { p.p.Id, p.p.Firstname, p.p.LastName, AddressId = p.a.Id, p.a.Street, p.a.City });
The first Select() orders the addresses and picks the latest one, and the second one returns an anonymous type with the properties specified in your query.
If you have more data in your relation table you're gonna have to use joins but this way you're free from them. In my opinion, this is more easy to read.
NOTE: You might get an exception if any entry in Persons have no addresses connected to them, although I haven't tried it out.
I have a piece of code that I don't know how to improve it.
I have two entities: EntityP and EntityC.
EntityP is the parent of EntityC. It is 1 to many relationship.
EntityP has a property depending on a property of all its attached EntityC.
I need to load a list of EntityP with the property set correctly. So I wrote a piece of code to get the EntityP List first.It's called entityP_List. Then as I wrote below, I loop through the entityP_List and for each of them, I query the database with a "any" function which will eventually be translated to "NOT EXIST" sql query. The reason I use this is that I don't want to load all the attached entityC from database to memory, because I only need the aggregation value of their property. But the problem here is, the looping will query the databae many times, for each EntityP!
So I am wondering if anybody can help me improve the code to query the database only once to get all the EntityP.IsAll_C_Complete set, without load EntityC to memory.
foreach(EntityP p in entityP_List)
{
isAnyNotComoplete = entities.entityC.Any(c => c.IsComplete==false && c.parent.ID == p.ID);
p.IsAll_C_Complete = !isAnyNotComoplete;
}
Thank you very much!
In EF 4, you can do:
var ids = entityP_List.Select(p => p.ID);
var q = (from p in entities.entityP
where ids.Contains(p => p.ID)
select new
{
ID = p.ID,
IsAll_C_Complete = !p.entityCs.Any(c => !c.IsComplete)
}).ToList();
foreach (var p in entityP_List)
{
p.IsAll_C_Complete = q.Where(e.ID == p.Id).Single().IsAll_C_Complete;
}
...which will do the whole thing in one DB query. For EF 1, Google BuildContainsExpression for a replacement for the .Contains( part of the above.
I would base EntityP on a SQL View instead of a table. Then I would define the relationship, and aggregate the value for child table within the view.