I have recently developed a C# application using Linq.
I am getting from an external database a list of profiles I need to process, some are new and some are already in the database, and need to be updated.
What I do today is go over the profile list and check each profile if such exists I update otherwise I insert - this solution is working fine.
I am sure there is a way to use bulk insert/update something like UPDATE ON DUPLICATE, this way I can save time since the files I get are huge and bulk insert/update is known to have better performance. I would like to avoid the iteration I am now using.
insertall doesn't work for already stored rows, I need the combination of both update and insert
Here is my code, Your help is highly appreciated.
foreach (Profile tmpProfile in profiles)
{
try
{
var matchedProfile = (from c in db.ProfileEntities
where c.ProfileId == tmpProfile.Id
select c).SingleOrDefault();
if (matchedProfile == null)
{
//Insert
db.ProfileEntities.InsertOnSubmit(EntityMapper.ToEntity(tmpProfile));
}
else
{
//Update
EntityMapper.ToEntity(ref matchedProfile, tmpProfile);
}
}
catch (System.Data.SqlServerCe.SqlCeException sqlExec)
{
}
catch (Exception e)
{
}
}
db.SubmitChanges();
One possible optimisation would be to create a list of all the items that you have from the external application, and then read all items from the database that match at once, instead of doing multiple round trips.
You can then update all of those, insert all of the ones that are left and call SubmitChanges at the end - you will then have 2 round trips to the database instead of one per profile retreived externally.
I don't know of any bulk update or insert features in Linq to SQL
Related
I am using ADO.NET with Data Adaptor to Fill a Dataset in my .NET Core 3.1 Project.
The first run for the Fill method occurs when my program initially starts so I have an in memeory cache to start using with my business/program logic. When I then make any changes to the tables using EF Core, once the changes have been saved I then run the Data Adapter Fill method to re-populate the Dataset with the updates from the tables that were modified in SQL through EF Core..
Reading various docs for a number of days now, what I'm unclear about is whether the Data Adapter Fill method overwrites all of the existing table rows in the Dataset each time the fill method is called? i.e if I'm loading a dataset with a table from SQL that has 10k rows, is it going to overwrite all 10k rows that exist in the dataset, even if 99% of the rows have not changed?
The reason I am going down the Dataset route is that I want to keep and in memory cache of the various tables from SQL so I can query the data as fast as possible without raising queries SQL all the time.
The solution I want is something along the lines of Data Adaptor Fill method, but I don't want the Dataset to be overwritten for any rows that had not been modified in SQL since the last run.
Is this how things are working already? or do I have to look for another solution?
Below just an example of the Adaptor Fill method.
public async Task<AdoNetResult> FillAlarmsDataSet()
{
string connectionString = _config.GetConnectionString("DefaultConnection");
try
{
string cmdText1 = "SELECT * FROM [dbo].[Alarm] ORDER BY Id;" +
"SELECT * FROM [dbo].[AlarmApplicationRole] ORDER BY Id;";
dataAdapter = new SqlDataAdapter(cmdText1, connectionString);
// Create table mappings
dataAdapter.TableMappings.Add("Alarm", "Alarm");
dataAdapter.TableMappings.Add("AlarmApplicationRole", "AlarmApplicationRole");
alarmDataSet = new DataSet
{
Locale = CultureInfo.InvariantCulture
};
// Create and fill the DataSet
await Task.Run(() => dataAdapter.Fill(alarmDataSet));
return AdoNetResult.Success;
}
catch (Exception ex)
{
// Return the task with details of the exception
return AdoNetResult.Failed(ex);
}
}
This question already has answers here:
Unable to edit db entries using EFCore, EntityState.Modified: "Database operation expected to affect 1 row(s) but actually affected 0 row(s)."
(17 answers)
Closed 2 years ago.
I am using Sql Server on Linux with EC Core (2.1.0-preview1-final)
I am trying to update some data coming from a web Api service (PUT) request. The data (ticket) is passed correctly and gets deserialised into an object (ticket).
When I try to update, I use the following code:
public Ticket UpdateTicket(Ticket ticket)
{
using (var ctx = new SupportTicketContext(_connectionString))
{
ctx.Entry(ticket).State = EntityState.Modified;
ctx.SaveChanges(); // <== **BLOWS UP HERE**
var result = ctx.Tickets
.First(t => t.TicketId == ticket.TicketId);
return result;
}
}
The code throws the following error:
Database operation expected to affect 1 row(s) but actually affected 0 row(s). Data
may have been modified or deleted since entities were loaded
I am able to Insert and fetch from the database, no problem.
If I restart Visual Studio, the error occurs usually on the second time I try to update ANY ticket (i.e. any other ticketId - it seems to be on the second and subsequent requests).
The updates are unpredictably successful! Sometimes I can update another ticket and it goes through (even on the 3rd or susequent request)
I have tried a number of modifications to the code, including
ctx.Attach(ticket);
but this does not help.
How do I get it to update the database successfully?
Any ideas on how to debug this? Logging seems to be difficult to set up.
Any ideas greatly appreciated.
There were two errors. First, I tried to call the Update() method without specifying a DbSet:
_applicationDbContext.Update(user);
Correct way:
_applicationDbContext.Users.Update(user);
The second error was trying to update a model that without first pulling it from the database:
public bool UpdateUser(IdentityUser model)
{
_applicationDbContext.Users.Update(model);
_applicationDbContext.SaveChanges();
return true;
}
Nope.
I first needed to retrieve it from the database, then update it:
public bool UpdateUser(IdentityUser model)
{
var user = _applicationDbContext.Users.FirstOrDefault(u => u.Id == model.Id);
user.PhoneNumberConfirmed = model.PhoneNumberConfirmed;
_applicationDbContext.Users.Update(user);
_applicationDbContext.SaveChanges();
return true;
}
Had such problem too, but the reason was a trigger I was using.
Changed the trigger type from "INSTEAD OF INSERT" to "AFTER INSERT" and instead of modifying and inserting the data I was allowing my command to insert the data to the table and then updated it using the trigger.
Why isn't the exception triggered? Linq's "Any()" is not considering the new entries?
MyContext db = new MyContext();
foreach (string email in {"asdf#gmail.com", "asdf#gmail.com"})
{
Person person = new Person();
person.Email = email;
if (db.Persons.Any(p => p.Email.Equals(email))
{
throw new Exception("Email already used!");
}
db.Persons.Add(person);
}
db.SaveChanges()
Shouldn't the exception be triggered on the second iteration?
The previous code is adapted for the question, but the real scenario is the following:
I receive an excel of persons and I iterate over it adding every row as a person to db.Persons, checking their emails aren't already used in the db. The problem is when there are repeated emails in the worksheet itself (two rows with the same email)
Yes - queries (by design) are only computed against the data source. If you want to query in-memory items you can also query the Local store:
if (db.Persons.Any(p => p.Email.Equals(email) ||
db.Persons.Local.Any(p => p.Email.Equals(email) )
However - since YOU are in control of what's added to the store wouldn't it make sense to check for duplicates in your code instead of in EF? Or is this just a contrived example?
Also, throwing an exception for an already existing item seems like a poor design as well - exceptions can be expensive, and if the client does not know to catch them (and in this case compare the message of the exception) they can cause the entire program to terminate unexpectedly.
A call to db.Persons will always trigger a database query, but those new Persons are not yet persisted to the database.
I imagine if you look at the data in debug, you'll see that the new person isn't there on the second iteration. If you were to set MyContext db = new MyContext() again, it would be, but you wouldn't do that in a real situation.
What is the actual use case you need to solve? This example doesn't seem like it would happen in a real situation.
If you're comparing against the db, your code should work. If you need to prevent dups being entered, it should happen elsewhere - on the client or checking the C# collection before you start writing it to the db.
Within a project I have a database table with the following columns
I would like to be able to delete from this table all rows which have a matching SharingAgencyId and ReceivingAgencyId values that I can pass in.
What I have tried so far:
public static ICollection<SecurityDataShare> UpdateSharedEntites(long conAgency, long recAgency)
{
ICollection<SecurityDataShare> agShares = null;
try
{
using (var context = new ProjSecurityEntities(string.Empty))
{
agShares = (from a in context.SecurityDataShares
.Where(c => c.ReceivingAgencyId == recAgency && c.SharingAgencyId == conAgency)
select a);
}
}
catch (Exception ex)
{
//ToDo
throw;
}
}
My thought process was to retrieve the records where the id's matched the parameters passed in and then using a foreach loop iterate through (agShare) and remove each row followed by saving my changes. With the current implementation I don't seem to have access to any of the Delete methods.
Looking to the example above I'd appreciate any suggestions on how to remove the rows within the table that contained a value of 43 and 39 using dbContext.
Cheers
If I understand right, your DbContext's properties, like SecurityDataShares should be typed as IDbSet<SecurityDataShare>. If that's correct, you should be able to use this Remove method.
foreach(var agShare in agShares) {
context.SecurityDataShares.Remove(agShare);
}
context.SaveChanges();
Be aware that this creates a separate SQL statement for deleting these objects. If you expect the number of objects to be rather large, you may want to use a stored procedure instead.
Entity Framework doesn't make it easy to run a single command to delete multiple rows (that I know of). My preference is to run a SQL statement directly for multi-entity updates/deletes using native sql with the dbcontext of sorts.
you can also pass datatable to the stored procedure with database contain dynamic type table of your type and
use that table into stored procedure for deleting matching rows from Database table.
Not sure if I'm missing anything here. Basically, I am looking for Linq to Nhibernate to do the following SQL statement:
update SomeTable
set SomeInteger = (SomeInteger + 1)
where SomeInteger > #NotSoMagicNumber
Is there any way to do that?
Thanks!
Late answer but it now exists in Nhibernate 5.0.
//
// Summary:
// Update all entities selected by the specified query. The update operation is
// performed in the database without reading the entities out of it.
//
// Parameters:
// source:
// The query matching the entities to update.
//
// expression:
// The update setters expressed as a member initialization of updated entities,
// e.g. x => new Dog { Name = x.Name, Age = x.Age + 5 }. Unset members are ignored
// and left untouched.
//
// Type parameters:
// TSource:
// The type of the elements of source.
//
// Returns:
// The number of updated entities.
public static int Update<TSource>(this IQueryable<TSource> source, Expression<Func<TSource, TSource>> expression);
In your case :
session.Query<SomeObject>()
.Update(i => new SomeObject { SomeInteger = i.SomeInteger + 1 });
Thanks NHibernate team!
Linq (not Linq to NHibernate, Linq in general) does not have a bulk update verb like SQL has. If you need the efficiency of the bulk update statement like yours, I'd just stick to SQL.
Like most (if not all) LINQ providers, LINQ to NHibernate only comes in useful in reading data.
To achieve what you want to do in NHibernate with the help of LINQ, you will want to fetch all of the relevant objects & update each one. Something like:
//NHibernate session initialisation & finalisation skipped for brevity
var relevantObjects = from i in session.Linq<SomeObject>()
where i.SomeInteger > notSoMagicNumber
select i;
foreach (SomeObject item in relevantObjects)
{
i.SomeInteger++;
session.Update(item);
}
Make sure you flush your session after all this, & wrap it all in a transaction to minimise the number of database updates.
All this said, depending on the size of your data, you may run into performance issues in using NHibernate for bulk operations. Using an IStatelessSession may help for that purpose, but I haven't tried it out myself.
UPDATE Turns out if you wrap it up in a transaction, you don't need to do session.Update or flush the session.