I have a bit of linq to entities code in a web app. It basically keeps a count of how many times an app was downloaded. I'm worried that this might happen:
Session 1 reads the download count (eg. 50)
Session 2 reads the download count (again, 50)
Session 1 increments it and writes it to the db (database stores 51)
Session 2 increments it and writes it to the db (database stores 51)
This is my code:
private void IncreaseHitCountDB()
{
JTF.JTFContainer jtfdb = new JTF.JTFContainer();
var app =
(from a in jtfdb.Apps
where a.Name.Equals(this.Title)
select a).FirstOrDefault();
if (app == null)
{
app = new JTF.App();
app.Name = this.Title;
app.DownloadCount = 1;
jtfdb.AddToApps(app);
}
else
{
app.DownloadCount = app.DownloadCount + 1;
}
jtfdb.SaveChanges();
}
Is it possible that this could happen? How could I prevent it?
Thank you,
Fidel
Entity Framework, by default, uses an optimistic concurrency model. Google says optimistic means "Hopeful and confident about the future", and that's exactly how Entity Framework acts. That is, when you call SaveChanges() it is "hopeful and confident" that no concurrency issue will occur, so it just tries to save your changes.
The other model Entity Framework can use should be called a pessimistic concurrency model ("expecting the worst possible outcome"). You can enable this mode on an entity-by-entity basis. In your case, you would enable it on the App entity. This is what I do:
Step 1. Enabling concurrency checking on an Entity
Right-click the .edmx file and choose Open With...
Choose XML (Text) Editor in the popup dialog, and click OK.
Locate the App entity in the ConceptualModels. I suggest toggling outlining and just expanding tags as necessary. You're looking for something like this:
<edmx:Edmx Version="2.0" xmlns:edmx="http://schemas.microsoft.com/ado/2008/10/edmx">
<!-- EF Runtime content -->
<edmx:Runtime>
<!-- SSDL content -->
...
<!-- CSDL content -->
<edmx:ConceptualModels>
<Schema Namespace="YourModel" Alias="Self" xmlns:annotation="http://schemas.microsoft.com/ado/2009/02/edm/annotation" xmlns="http://schemas.microsoft.com/ado/2008/09/edm">
<EntityType Name="App">
Under the EntityType you should see a bunch of <Property> tags. If one exists with Name="Status" modify it by adding ConcurrencyMode="Fixed". If the property doesn't exist, copy this one in:
<Property Name="Status" Type="Byte" Nullable="false" ConcurrencyMode="Fixed" />
Save the file and double click the .edmx file to go back to the designer view.
Step 2. Handling concurrency when calling SaveChanges()
SaveChanges() will throw one of two exceptions. The familiar UpdateException or an OptimisticConcurrencyException.
if you have made changes to an Entity which has ConcurrencyMode="Fixed" set, Entity Framework will first check the data store for any changes made to it. If there are changes, a OptimisticConcurrencyException will be thrown. If no changes have been made, it will continue normally.
When you catch the OptimisticConcurrencyException you need to call the Refresh() method of your ObjectContext and redo your calculation before trying again. The call to Refresh() updates the Entity(s) and RefreshMode.StoreWins means conflicts will be resolved using the data in the data store. The DownloadCount being changed concurrently is a conflict.
Here's what I'd make your code look like. Note that this is more useful when you have a lot of operations between getting your Entity and calling SaveChanges().
private void IncreaseHitCountDB()
{
JTF.JTFContainer jtfdb = new JTF.JTFContainer();
var app =
(from a in jtfdb.Apps
where a.Name.Equals(this.Title)
select a).FirstOrDefault();
if (app == null)
{
app = new JTF.App();
app.Name = this.Title;
app.DownloadCount = 1;
jtfdb.AddToApps(app);
}
else
{
app.DownloadCount = app.DownloadCount + 1;
}
try
{
try
{
jtfdb.SaveChanges();
}
catch (OptimisticConcurrencyException)
{
jtfdb.Refresh(RefreshMode.StoreWins, app);
app.DownloadCount = app.DownloadCount + 1;
jtfdb.SaveChanges();
}
}
catch (UpdateException uex)
{
// Something else went wrong...
}
}
You can prevent this from happenning if you only query the download count column right before you are about to increment it, the longer the time spent between reading and incrementing the longer the time another session has to read it (and later rewriting - wrongly - incremented number ) and thus messing up the count.
with a single SQL query :
UPDATE Data SET Counter = (Counter+1)
since its Linq To Entities,it means delayed execution,for another session to screw up the Count (increment the same base,losing 1 count there) it would have to try to increment the app.Download count i beleive between the two lines:
else
{
app.DownloadCount += 1; //First line
}
jtfdb.SaveChanges(); //Second line
}
thats means that the window for the change to occur, thus making the previous count old, is so small that for an application like this is virtually impossible.
Since Im no LINQ pro, i dont know whether LINQ actually gets app.DownLoadCount before adding one or just adds one through some SQL command, but in either case you shouldnt have to worry about that imho
You could easily test what would happen in this scenario - start a thread, sleep it, and then start another.
else
{
app.DownloadCount = app.DownloadCount + 1;
}
System.Threading.Thread.Sleep(10000);
jtfdb.SaveChanges();
But the simple answer is that no, Entity Framework does not perform any concurrency checking by default (MSDN - Saving Changes and Managing Concurrency).
That site will provide some background for you.
Your options are
to enable concurrency checking, which will mean that if two users download at the same time and the first updates after the second has read but before the second has updated, you'll get an exception.
create a stored procedure that will increment the value in the table directly, and call the stored procedure from code in a single operation - e.g. IncrementDownloadCounter. This will ensure that there is no 'read' and therefore no possibility of a 'dirty read'.
Related
I'm using EF 5 with Oracle database.
I'm doing a select count in a table with a specific parameter. When I'm using EF, the query returns the value 31, as expected, But the result takes about 10 seconds to be returned.
using (var serv = new Aperam.SIP.PXP.Negocio.Modelos.SIP_PA())
{
var teste = (from ens in serv.PA_ENSAIOS_UM
where ens.COD_IDENT_UNMET == "FBLDY3840"
select ens).Count();
}
If I execute the simple query bellow the result is the same (31), but the result is showed in 500 milisecond.
SELECT
count(*)
FROM
PA_ENSAIOS_UM
WHERE
COD_IDENT_UNMET 'FBLDY3840'
There are a way to improve the performance when I'm using EF?
Note: There are 13.000.000 lines in this table.
Here are some things you can try:
Capture the query that is being generated and see if it is the same as the one you are using. Details can be found here, but essentially, you will instantiate your DbContext (let's call it "_context") and then set the Database.Log property to be the logging method. It's fine if this method doesn't actually do anything--you can just set a breakpoint in there and see what's going on.
So, as an example: define a logging function (I have a static class called "Logging" which uses nLog to write to files)
public static void LogQuery(string queryData)
{
if (string.IsNullOrWhiteSpace(queryData))
return;
var message = string.Format("{0}{1}",
queryData.Trim().Contains(Environment.NewLine) ?
Environment.NewLine : "", queryData);
_sqlLogger.Info(message);
_genLogger.Trace($"EntityFW query (len {message.Length} chars)");
}
Then when you create your context point to LogQuery:
_context.Database.Log = Logging.LogQuery;
When you do your tests, remember that often the first run is the slowest because the server has to actually do the work, but on the subsequent runs, it often uses cached data. Try running your tests 2-3 times back to back and see if they don't start to run in the same time.
I don't know if it generates the same query or not, but try this other form (which should be functionally equivalent, but may provide better time)
var teste = serv.PA_ENSAIOS_UM.Count(ens=>ens.COD_IDENT_UNMET == "FBLDY3840");
I'm wondering if the version you have pulls data from the DB and THEN counts it. If so, this other syntax may leave all the work to be done at the server, where it belongs. Not sure, though, esp. since I haven't ever used EF with Oracle and I don't know if it behaves the same as SQL or not.
Time to leave the shy mode behind and make my first post on stackoverflow.
After doing loads of research (plugins, performance, indexes, types of update, friends) and after trying several approaches I was unable to find a proper answer/solution.
So if possible I would like to get your feedback/help in a Microsoft Dynamics CRM 2013/2015 plugin performance issue (or coding technique)
Scenario:
Microsoft Dynamics CRM 2013/2015
2 Entities with Relationship 1:N
EntityA
EntityB
EntityB has the following columns:
Id | EntityAId | ColumnDemoX (decimal) | ColumnDemoY (currency)
Entity A has: 500 records
Entity B has: 150 records per each Entity A record. So 500*150 = 75000 records.
Objective:
Create a Post Entity A Plugin Update to "mimic" the following SQL command
Update EntityB
Set ColumnDemoX = (some quantity), ColumnDemoY = (some quantity) * (some value)
Where EntityAId = (some id)
One approach could be:
using (var serviceContext = new XrmServiceContext(service))
{
var query = from a in serviceContext.EntityASet
where a.EntityAId.Equals(someId)
select a;
foreach (EntityA entA in query)
{
entA.ColumnDemoX = (some quantity);
serviceContext.UpdateObject(entA);
}
serviceContext.SaveChanges();
}
Problem:
The foreach for 150 records in the post plugin update will take 20 secs or more.
While the
Update EntityB Set ColumnDemoX = (some quantity), ColumnDemoY = (some quantity) * (some value) Where EntityAId = (some id)
it will take 0.00001 secs
Any suggestion/solution?
Thank you all for reading.
H
You can use the ExecuteMultipleRequest, when you iterate the 150 entities, save the entities you need to update and after that call the request. If you do this, you only call the service once, that's very good for the perfomance.
If your process could be bigger and bigger, then you should think making it asynchronous as a plug-in or a custom activity workflow.
This is an example:
// Create an ExecuteMultipleRequest object.
requestWithResults = new ExecuteMultipleRequest()
{
// Assign settings that define execution behavior: continue on error, return responses.
Settings = new ExecuteMultipleSettings()
{
ContinueOnError = false,
ReturnResponses = true
},
// Create an empty organization request collection.
Requests = new OrganizationRequestCollection()
};
// Add a UpdateRequest for each entity to the request collection.
foreach (var entity in input.Entities)
{
UpdateRequest updateRequest = new UpdateRequest { Target = entity };
requestWithResults.Requests.Add(updateRequest);
}
// Execute all the requests in the request collection using a single web method call.
ExecuteMultipleResponse responseWithResults =
(ExecuteMultipleResponse)_serviceProxy.Execute(requestWithResults);
Few solutions comes to mind but I don't think they will please you...
Is this really a problem ? Yes it's slow and database update can be so much faster. However if you can have it as a background process (asynchronous), you'll have your numbers anyway. Is it really a "I need this numbers in the next second as soon as I click or business will go down" situation ?
It can be a reason to ditch 2013. In CRM 2015 you can use a calculated field. If you need this numbers only to show up in forms (eg. you don't use them in reporting), you could also do it in javascript.
Warning this is for the desesperate call. If you really need your update to be synchronous, immediate, you can't use calculated fields, you really know what your doing etc... Why not do it directly in the database? I know this is a very bad advice. There are a lot of reason not to do it this way (you can read a few here). It's unsupported and if you do something wrong it could go really bad. But if your real situation is as simple as your example (just a calculated field, no entity creation, no relation modification), you could do it this way. You'll have to consider many things: you won't have any audit on the fields, no security, caching issues, no modified by, etc. Actually I pretty much advise against this solution.
1 - Put it this logic to async workflow.
OR
2 - Don't use
serviceContext.UpdateObject(entA);
serviceContext.SaveChanges();.
Get all the records (150) from post stage update the fields and ExecuteMultipleRequest to update crm records in one time.
Don't send update request for each and every record
Why isn't the exception triggered? Linq's "Any()" is not considering the new entries?
MyContext db = new MyContext();
foreach (string email in {"asdf#gmail.com", "asdf#gmail.com"})
{
Person person = new Person();
person.Email = email;
if (db.Persons.Any(p => p.Email.Equals(email))
{
throw new Exception("Email already used!");
}
db.Persons.Add(person);
}
db.SaveChanges()
Shouldn't the exception be triggered on the second iteration?
The previous code is adapted for the question, but the real scenario is the following:
I receive an excel of persons and I iterate over it adding every row as a person to db.Persons, checking their emails aren't already used in the db. The problem is when there are repeated emails in the worksheet itself (two rows with the same email)
Yes - queries (by design) are only computed against the data source. If you want to query in-memory items you can also query the Local store:
if (db.Persons.Any(p => p.Email.Equals(email) ||
db.Persons.Local.Any(p => p.Email.Equals(email) )
However - since YOU are in control of what's added to the store wouldn't it make sense to check for duplicates in your code instead of in EF? Or is this just a contrived example?
Also, throwing an exception for an already existing item seems like a poor design as well - exceptions can be expensive, and if the client does not know to catch them (and in this case compare the message of the exception) they can cause the entire program to terminate unexpectedly.
A call to db.Persons will always trigger a database query, but those new Persons are not yet persisted to the database.
I imagine if you look at the data in debug, you'll see that the new person isn't there on the second iteration. If you were to set MyContext db = new MyContext() again, it would be, but you wouldn't do that in a real situation.
What is the actual use case you need to solve? This example doesn't seem like it would happen in a real situation.
If you're comparing against the db, your code should work. If you need to prevent dups being entered, it should happen elsewhere - on the client or checking the C# collection before you start writing it to the db.
I am using a simple form to allow people to sign up for an event. Their details are saved to a Sharepoint list. I have a quota of people who can sign up for an event (say 100 people).
How can I prevent the 100th and the 101st person from signing up concurrently, causing the quota check to allow the 101st person to sign up (because the 100th person isn't in the list yet)?
Place the ItemAdding code inside a lock statement to make sure that only one thread at a time can enter the critical section of code:
private Object _lock = new Object();
public override void ItemAdding(SPItemEventProperties properties)
{
lock(_lock)
{
// check number of the list items and cancel the event if necessary
}
}
I came up with this idea of a solution for a farm with multiple WFEs - a shared resource (a row in a table in pseudo-code above) gets locked during the time the item is added to the list:
private Object _lock = new Object();
public override void ItemAdding(SPItemEventProperties properties)
{
try
{
// 1. begin a SQL Server transaction
// 2. UPDATE dbo.SEMAPHORE
// SET STATUS = 'Busy'
// WHERE PROCESS = 'EventSignup'
lock(_lock)
{
// 3. check number of the list items and cancel the event if necessary
}
}
finally
{
// 4. UPDATE dbo.SEMAPHORE
// SET STATUS = ''
// WHERE PROCESS = 'EventSignup'
// 5. commit a SQL Server transaction
}
}
I left the lock statement because I'm not sure what will happen if the same front-end server tries to add the item #100 and #101 - will the transaction lock the row or will it not because the same connection to SQL Server will be used?
So then you can use event receivers item adding method. at item adding, your item is not created, you can calculate the current count of signed up people. if it is bigger then 100 you can cancel item adding.
but sure, more than one item adding method can be fired, to prevent that you can calculate the current count of people and increase the count +1, and keep that value somewhere else (on a field on event item perhaps) and all item adding methods can check that value before adding the item.
item added method is too late for these operations.
this would be the solution i would use.
I guess if you are updating a column, lets say - "SignUp Count", then one of the users will get the Save Conflict issue. Whoever updated the value for the first time wins and the second one will fail.
Regards,
Nitin Rastogi
My main question here is dealing with the pramas map when having a one-to-many relationship managed within one dynamic form, as well as best practices for dealing with one-to-many when editing/updating a domain object through the dynamic form. The inputs for my questions are as follows.
I have managed to hack away a form that allows me to create the domain objects shown below in one Dynamic form, since there is no point in having a separate form for creating phone numbers and then assigning them to a contact, it makes sense to just create everything in one form in my application. I managed to implement something similar to what I have asked in my Previous Question (thanks for the people who helped out)
class Contact{
String firstName
String lastName
// ....
// some other properties
// ...
static hasMany = [phones:Phone]
static mapping = {
phones sort:"index", cascade: "all-delete-orphan"
}
}
class Phone{
int index
String number
String type
Contact contact
static belongsTo = [contact:Contact]
}
I basically managed to get the values from the 'params' map and parse them on my own and create the domain object and association manually. I.e. i did not use the same logic that is used in the default scaffolding, i.e.
Contact c = new Contact(params)
etc...., i just looped through all the params and hand crafted my domain objects and saved them and everything works out fine.
My controller has code blocks that look like this (this is stripped down, just to show a point)
//create the contact by handpicking params values
def cntct = new Contact()
cntct.firstName = params.firstName
cntct.lastName = params.lastName
//etc...
//get array of values for number,type
def numbers = params['phone.number']
def types = params['phone.type']
//loop through one of the arrays and create the phones
numbers.eachWithIndex(){ num, i ->
//create the phone domain object from
def phone = new Phone()
phone.number = num
phone.type = types[i]
phone.index = i
cntct.addToPhones(phone)
}
//save
My questions are as follows:
What is the best practice of handeling such a situation, would using Command objects work in this case, if yes where can i found more info about this, all the examples I have found during my search deal with one-to-one relationships, I couldn't find an example for one-to-many?
What is the best way to deal with the relatiohsips of the phones in this case, in terms of add/removing phones when editing the contact object. I mean the creation logic is simple since I have to always create new phones on save, but when dealing with updating a contact, the user might have removed a phone and/or editing an exiting one and/or added some new phones. Right now what I do is just delete all the phones a contact has and re-create them according to what was posted by the form, but I feel that's not the best way to do it, I also don't think looping over the existing ones and comparing with the posted values and doing a manual diff is the best way to do it either, is there a best practice on how to deal with this?
Thanks, hopefully the questions are clear.
[edit] Just for more information, phone information can be added and deleted dynamically using javascript (jquery) within the form [/edit]
disclaimer: i do not know if the following approach works when using grails. Let me know later.
See better way for dynamic forms. The author says:
To add LineItems I have some js that calculates the new index and adds that to the DOM. When deleting a LineItem i have to renumber all the indexes and it is what i would like to avoid
So what i do
I have a variable which stores the next index
var nextIndex = 0;
When the page is loaded, i perform a JavaScript function which calculates how many child The collection has and configure nextIndex variable. You can use JQuery or YUI, feel free.
Adding a child statically
I create a variable which store the template (Notice {index})
var child = "<div>"
+= "<div>"
+= "<label>Name</label>"
+= "<input type="text" name=\"childList[{index}].name\"/>"
+= "</div>"
+= "</div>"
When the user click on the Add child button, i replace {index} - by using regex - by the value stored in the nextIndex variable and increment by one. Then i add to the DOM
See also Add and Remove HTML elements dynamically with Javascript
Adding a child dinamically
Here you can see The Paolo Bergantino solution
By removing
But i think it is the issue grow up when deleting. No matter how many child you remove, does not touch on the nextIndex variable. See here
/**
* var nextIndex = 3;
*/
<input type="text" name="childList[0].name"/>
<input type="text" name="childList[1].name"/> // It will be removed
<input type="text" name="childList[2].name"/>
Suppose i remove childList1 What i do ??? Should i renumber all the indexes ???
On the server side i use AutoPopulatingList. Because childList1 has been removed, AutoPopulatingList handles it as null. So on the initialization i do
List<Child> childList = new AutoPopulatingList(new ElementFactory() {
public Object createElement(int index) throws ElementInstantiationException {
/**
* remove any null value added
*/
childList.removeAll(Collections.singletonList(null));
return new Child();
}
});
This way, my collection just contains two child (without any null value) and i do not need to renumber all the indexes on the client side
About adding/removing you can see this link where i show a scenario wich can gives you some insight.
See also Grails UI plugin
Thanks,
Your answer brought some insight for me to do a wider search and I actually found a great post that covers all the inputs in my question. This is just a reference for anyone reading this. I will write a blog entry on how I implemented my case soon, but this link should provide a good source of ino with a working exmaple.
http://www.2paths.com/2009/10/01/one-to-many-relationships-in-grails-forms/
Most of the time I use ajax to manage such problem.
So when the user clicks add new phone I get the template UI from the server for manageability purpose ( the UI just same GSP template that I use to edit, update the phone), so this way you are not mixing your UI with your js code, whenever you want to change the UI you have to deal only with our GSP code.
Then after getting the UI I add it to the page using jquery DOM manipulation. Then after filling the form when they hit add(save) the request is sent to the server via ajax and is persisted immediately.
When the user clicks edit phone the same UI template is loaded from the server filled with existing phone data, then clicking update will update the corresponding phone immediately via ajax, and same thing applies to delete operation.
But one day I got an additional scenario for the use case that says, "until I say save contact no phone shall be saved on the backend, also after adding phones to the contact on the ui if navigate away to another page and come back later to the contact page the phones I added before must be still there." ugh..
To do this I started using the Session, so the above operations I explained will act on the phone list object I stored on the session instead of the DB. This is simple perform all the operation on the phonesInSession but finally dont forget to do this(delete update):
phonesToBeDeleted = phonesInDB - phonesInSession
phonesToBeDeleted.each{
contact.removeFromPhones(it)
it.delete()
}
I know I dont have to put a lot of data in session but this is the only solution I got for my scenario.
If someone has got similar problem/solution please leave a comment.
First, in all your input fields names you add an #:
<input type="text" name="references[#].name"/>
Second, add call a function before submitting:
<g:form action="save" onsubmit="replaceAllWildCardsWithConsecutiveNumbers();">
Third, this is the code for the function that you call before submitting the form:
function replaceAllWildCardsWithConsecutiveNumbers(){
var inputs = $('form').find("[name*='#']");
var names = $.map(inputs, function(el) { return el.name });
var uniqueNames = unique(names);
for (index in uniqueNames) {
var uniqueName = uniqueNames[index];
replaceWildCardsWithConsecutiveNumbers("input", uniqueName);
replaceWildCardsWithConsecutiveNumbers("select", uniqueName);
}
}
function unique(array){
return array.filter(function(el, index, arr) {
return index === arr.indexOf(el);
});
}
function replaceWildCardsWithConsecutiveNumbers(inputName, name){
counter = 0;
$(inputName + "[name='" + name + "']").each(function (i, el) {
var curName = $(this).attr('name');
var newName = curName.replace("#", counter);
$(this).attr('name', newName);
counter += 1;
});
}
Basically, what the code for replaceAllWildCardsWithConsecutiveNumbers() does, is to create a list for all input (or select) elements whose name contains an #. Removes the duplicates. And then iterates over them replacing the # with a number.
This works great if you have a table and you are submitting the values to a command object's list when creating a domain class for the first time. If you are updating I guess you'll have to change the value of counter to something higher.
I hope this helps someone else since I was stuck on this issue for a while myself.