Entity Framework Core - Upsert entities from other database encounters tracking problems - performance

I have a flatfile from a different database. I import it and map it to my application's entities. Because the flatfile does not contain ids I cannot be sure the entries I handle are not duplicates of what has already been added to my database earlier or to my context at this moment.
The error message I get is:
The instance of entity type 'Car' cannot be tracked because another
instance with the same key value for {'Make', 'Model'} is already
being tracked. When attaching existing entities, ensure that only one
entity instance with a given key value is attached. Consider using
'DbContextOptionsBuilder.EnableSensitiveDataLogging' to see the
conflicting key values.
An example:
Data rows from flatfile
Volvo V70 Steve
Volvo V70 John
Having mapped these rows and trying to put them in db
foreach(var row in flatFileRows){
Car existingCar = null;
if(dbContext.Cars.Any(c => c.Make == row.Make && c.Model == row.Model)){
existingCar = dbContext.Cars
.SingleOrDefault(c => c.Make == row.Make && c.Model == row.Model);
}
//I also do the same for existingDriver
var car = existingCar != null
? existingCar
: new Car()
{
Make = row.Make,
Model = row.Model,
Drivers = new List<Driver>();
};
var driver = new Driver()
{
CarId = existingCar != null ? exsitingCar.Id : 0,
Name = row.Name
};
car.Drivers.Add(driver);
dbContext.Cars.Update(car); //Second time we hit this the error is thrown
}
dbContext.SaveChanges();
Make and Model are set to keys in the schema because I don't want duplicate entries of the car models.
The above example is simplified.
What I want is to check if I already put a car in the db with these attributes and then build according to my schema from that entity. I don't care to track any entries, disconnected or otherwise, because I just need to populate the database.

Related

Recursive linq expressions to get non NULL parent value?

I wrote a simple recursive function to climb up the tree of a table that has ID and PARENTID.
But when I do that I get this error
System.InvalidOperationException: 'The instance of entity type 'InternalOrg' cannot be tracked because another instance with the same key value for {'Id'} is already being tracked. When attaching existing entities, ensure that only one entity instance with a given key value is attached.
Is there another way to do this or maybe done in one LINQ expression ?
private InternalOrgDto GetInternalOrgDto(DepartmentChildDto dcDto)
{
if (dcDto.InternalOrgId != null)
{
InternalOrg io = _internalOrgRepo.Get(Convert.ToInt32(dcDto.InternalOrgId));
InternalOrgDto ioDto = new InternalOrgDto
{
Id = io.Id,
Abbreviation = io.Abbreviation,
Code = io.Code,
Description = io.Description
};
return ioDto;
}
else
{
//manually get parent department
Department parentDepartment = _departmentRepo.Get(Convert.ToInt32(dcDto.ParentDepartmentId));
DepartmentChildDto parentDepartmenDto = ObjectMapper.Map<DepartmentChildDto>(parentDepartment);
return GetInternalOrgDto(parentDepartmenDto);
}
}
Is there a way to get a top-level parent from a given child via Linq? Not that I am aware of. You can do it recursively similar to what you have done, though I would recommend simplifying the query to avoid loading entire entities until you get what you want. I'm guessing from your code that only top level parent departments would have an InternalOrg? Otherwise this method would recurse up the parents until it found one. This could be sped up a bit like:
private InternalOrgDto GetInternalOrgDto(DepartmentChildDto dcDto)
{
var internalOrgid = dcDto.InternalOrgId
?? FindInternalOrgid(dcDto.ParentDepartmentId)
?? throw new InternalOrgNotFoundException();
InternalOrgDto ioDto = _context.InternalOrganizations
.Where(x => x.InternalOrgId == internalOrgId.Value)
.Select(x => new InternalOrgDto
{
Id = x.Id,
Abbreviation = x.Abbreviation,
Code = x.Code,
Description = x.Description
}).Single();
return ioDto;
}
private int? FindInternalOrgid(int? departmentId)
{
if (!departmentId.HasValue)
return (int?) null;
var details = _context.Departments
.Where(x => x.DepartmentId == departmentId.Value)
.Select(x => new
{
x.InternalOrgId,
x.ParentDepartmentId
}).Single();
if (details.InternalOrgId.HasValue)
return details.InternalOrgId;
return findInternalOrgId(details.parentDepartmentId);
}
The key considerations here are to avoid repository methods that return entities or sets of entities, especially where you don't need everything about an entity. By leveraging the IQueryable provided by EF through Linq we can project down to just the data we need rather than returning every field. The database server can accommodate this better via indexing and help avoid things like locks. If you are using repositories to enforce low level domain rules or to enable unit testing then the repositories can expose IQueryable<TEntity> rather than IEnumerable<TEntity> or even TEntity to enable projection and other EF Linq goodness.
Another option to consider where I have hierarchal data where the relationships are important and I want to quickly find all related entities to a parent, or get to a specific level, one option is to store a breadcrumb with each record which is updated if that item is ever moved. The benefit is that these kinds of checks become very trivial to do, the risk is that anywhere/anything that can modify data relationships could leave the breadcrumb trail in an invalid state.
For example, if I have a Department ID 22 which belongs to Department 8 which belongs to Department 2 which is a top-level department, 22's breadcrumb trail would be: "2,8". If the breadcrumbs are empty we have a top-level entity. (and no parent Id) We can parse the breadcrumbs using a simple string.Split() operation. This avoids the recursive trips to the DB entirely. Though you may want a maintenance job running behind the scenes to periodically inspect recently modified data to ensure their breadcrumb trails are accurate and alerting you if any get broken. (Either by faulty code or such)

Many-To-Many Entity Framework Update

I have an object that has a many-to-many relationship with another object. I am trying to write an update statement that doesn't result in having to delete all records from the many-to-many table first.
My data is:
StoredProcedure - StoredProcedureId, Name
Parameter - ParameterId, Name
StoredProcedure_Parameter - StoredProcedureId, ParameterId, Order
I have a UI for updating a stored procedured object (adding/removing parameters or changing the order of the parameters).
When I save, I end up at:
var storedProcedure = context.Sprocs.FirstOrDefault(s => s.SprocID == sproc.StoredProcedureId);
if (storedProcedure == null)
{
//do something like throw an exception
} else
{
storedProcedure.Name = sproc.Name;
//resolve Parameters many to many here
//remove all Params that are not in sproc.Params
//Add any params that are in sproc.Params but not in storedProcedure.Params
//Update the Order number for any that are in both
}
I know I could simply call .Clear() on the table and then reinsert all of the values with their current state (ensuring that all parameters that were removed by the UI are gone, new ones are added, and updated Orders are changed). However, I feel like there must be a better way to do this. Do many-to-many updates with EF usually get resolved by deleting all of the elements and reinserting them?
Here there is my code that I use and it works. The difference is that instead o having your 3 tables( StoredProcedure, StoredProcedure_Parameter and Parameter ) I have the following 3 tables: Order, OrdersItem(this ensure the many-to-many relation) and Item. This is the procedure that I used for updating or add an order, or after I change an existing OrderItem or add a new one to the Order.
public void AddUpdateOrder(Order order)
{
using (var db = new vitalEntities())
{
if (order.OrderId == 0)
{
db.Entry(order).State = EntityState.Added;
}
else
{
foreach (var orderItem in order.OrdersItems)
{
if (orderItem.OrderItemsId == 0)
{
orderItem.Item = null;
if (order.OrderId != 0)
orderItem.OrderId = order.OrderId;
db.Entry(orderItem).State = EntityState.Added;
}
else
{
orderItem.Order = null;
orderItem.Item = null;
db.OrdersItems.Attach(orderItem);
db.Entry(orderItem).State = EntityState.Modified;
}
}
db.Orders.Attach(order);
db.Entry(order).State = EntityState.Modified;
}
SaveChanges(db);
}
}

MVC - Adding data into linker tables

I have a registration form that allows a school to register. In addition to the obvious login and general details the school can pick from a list of facilities and accreditations that they have.
My data is displayed lovely and binded correctly.
Problem Entering the data into the linker tables does not work it throws an error in both the different ways that I have tried:
Method1:
MembershipUser membershipUser = null;
if (schoolRegisterModel != null)
{
if (null != DB)
{
school SchoolUser = new school();
SchoolUser.username = schoolRegisterModel.UserName;
SchoolUser.email = schoolRegisterModel.Email;
string sPassowrdSalt = Security.Instance().CreateSalt();
SchoolUser.password = Security.Instance().CreatePasswordHash(schoolRegisterModel.Password, sPassowrdSalt);
SchoolUser.password_salt = sPassowrdSalt;
..More data etc..
foreach (var item in schoolRegisterModel.Facilities)
{
if (item.#checked)
{
school_facility sf = new school_facility();
sf.facility_id = item.facility_id;
SchoolUser.school_facility.Add(sf);
}
}
foreach (var item in schoolRegisterModel.Accreditations)
{
if (item.#checked)
{
school_accreditation sa = new school_accreditation();
sa.accreditation_id = item.accreditation_id;
SchoolUser.school_accreditation.Add(sa);
}
}
DB.schools.Add(SchoolUser);
DB.SaveChanges();
Error: {"The INSERT statement conflicted with the FOREIGN KEY constraint \"FK_school_facility_facility\". The conflict occurred in database \"MYDB\", table \"dbo.facility\", column 'facility_id'.\r\nThe statement has been terminated."}
Also - Do I need to manually retrieve the soon to be school ID that will be generated based on this insert. This method avoids entering data directly into the linker tables using only the primary table (school).
Method2:
Same code again apart from trying to update the primary tables (school) accreditation and facilities collection directly, I manually update the linker tables seperately using the latest primary key generated by the previous query, code for this is as follows:
MembershipUser membershipUser = null;
if (schoolRegisterModel != null)
{
if (null != DB)
{
school SchoolUser = new school();
SchoolUser.username = schoolRegisterModel.UserName;
SchoolUser.email = schoolRegisterModel.Email;
string sPassowrdSalt = Security.Instance().CreateSalt();
SchoolUser.password = Security.Instance().CreatePasswordHash(schoolRegisterModel.Password, sPassowrdSalt);
SchoolUser.password_salt = sPassowrdSalt;
..More data etc..
// Linker data for facilities and accreditations.
// Facilities
foreach (var item in schoolRegisterModel.Facilities)
{
if (item.#checked)
{
school_facility sf = new school_facility();
sf.facility_id = item.facility_id;
sf.school_id = SchoolUser.school_id;
DB.school_facility.Add(sf);
}
}
// Accreditations
foreach (var item in schoolRegisterModel.Accreditations)
{
if (item.#checked)
{
school_accreditation sa = new school_accreditation();
sa.accreditation_id = item.accreditation_id;
sa.school_id = SchoolUser.school_id;
DB.school_accreditation.Add(sa);
}
}
m_DB.SaveChanges();
Error: {"The INSERT statement conflicted with the FOREIGN KEY constraint \"FK_school_facility_facility\". The conflict occurred in database \"MYDB\", table \"dbo.facility\", column 'facility_id'.\r\nThe statement has been terminated."}
If you guys have any idea where I am going wrong then please do let me know. There seem to be examples of updating linker table date (which I will need at some point anyway) but can't find an example of my problem...
Thanks in advance.
Looks like I have found the answer:
MVC: The INSERT statement conflicted with the FOREIGN KEY constraint
My data being pulled through was basically not containing the correct foreign key value (0 - which didn't exist) and quite rightly my DB was throwing the error. Sorry for wasting time to whoever read and thanks for your time. I hope this can help somebody else.
Joe

Can ExecuteQuery return a DBML generated class without having it fetch all the information for that class?

I have a couple of DBML generated classes which are linked together by an id, e.g.
ClassA {
AID,
XID,
Name
}
ClassB {
AID,
ExtraInfo,
ExtraInfo2
}
In using something like db.ClassAs.Where(XID == x) and iterating through that result,
it ends up executing a query for each of the ClassAs and each of ClassBs, which is slow.
Alternatively, I've tried to use ExecuteQuery to fetch all the info I care about and have that return a ClassA. In iterating over that I end up with it doing the same, i.e. doing alot of individual fetches vs. just 1. If I store it in a ClassC (that is not associated with a DB entity) which has the fields of interest of both ClassA and ClassB, this query is much faster, but it's annoying b/c I just created IMO an unnecessary ClassC.
How can I still use ClassA, which associates to ClassB, and still use ExecuteQuery to run 1 query vs. A*B number of queries?
If you have associations you shouldn't need to use ExecuteQuery().
Here's an example using some imaginary Book Library context and anonymous types for the result:
var results =
Books
.Where(book => book.BookId == 1)
.Select(book =>
new
{
book.Name,
AuthorName = book.Author.Name, //Is a field in an associated table.
book.Publisher, //Is an associtated table.
});
EDIT: without anon types
var results =
Books
.Where(book => book.BookId == 1)
.Select(book =>
new BookResult()
{
BookName = book.Name,
AuthorName = book.Author.Name, //Is a field in an associated table.
Publisher = book.Publisher, //Is an associtated table.
});

Auditing in Entity Framework

After going through Entity Framework I have a couple of questions on implementing auditing in Entity Framework.
I want to store each column values that is created or updated to a different audit table.
Right now I am calling SaveChanges(false) to save the records in the DB(still the changes in context is not reset). Then get the added | modified records and loop through the GetObjectStateEntries. But don't know how to get the values of the columns where their values are filled by stored proc. ie, createdate, modifieddate etc.
Below is the sample code I am working on it.
// Get the changed entires( ie, records)
IEnumerable<ObjectStateEntry> changes = context.ObjectStateManager.GetObjectStateEntries(EntityState.Modified);
// Iterate each ObjectStateEntry( for each record in the update/modified collection)
foreach (ObjectStateEntry entry in changes)
{
// Iterate the columns in each record and get thier old and new value respectively
foreach (var columnName in entry.GetModifiedProperties())
{
string oldValue = entry.OriginalValues[columnName].ToString();
string newValue = entry.CurrentValues[columnName].ToString();
// Do Some Auditing by sending entityname, columnname, oldvalue, newvalue
}
}
changes = context.ObjectStateManager.GetObjectStateEntries(EntityState.Added);
foreach (ObjectStateEntry entry in changes)
{
if (entry.IsRelationship) continue;
var columnNames = (from p in entry.EntitySet.ElementType.Members
select p.Name).ToList();
foreach (var columnName in columnNames)
{
string newValue = entry.CurrentValues[columnName].ToString();
// Do Some Auditing by sending entityname, columnname, value
}
}
Here you have two basic options:
Do it at the database level
Do it in the c# code
Doing it at the data base level, means using triggers. In that case there is no difference if you are using enterprise library or another data access technology.
To do it in the C# code you would add a log table to your datamodel, and write the changes to the log table. When you do a save changes both the changes to the data and the information which you wrote to the log table would be saved.
Are you inserting the new record using a stored proc? If not (i.e. you are newing up an object, setting values, inserting on submit and then saving changes the new object id will be automatically loaded into the id property of the object you created. If you are using a stored proc to do the insert then you need to return the ##IDENTITY from the proc as a return value.
EX:
StoreDateContext db = new StoreDataContext(connString);
Product p = new Product();
p.Name = "Hello Kitty Back Scratcher";
p.CategoryId = 5;
db.Products.Add(p);
try
{
db.SaveChanges();
//p.Id is now set
return p.Id;
}
finally
{
db.Dispose;
}

Resources