I have a bunch of custom entity records in a List (which comes from a csv file).
What is the best way to check which records are new and create those that are?
The equality check is based on a single text field in this case, but I need to do the same thing elsewhere where the equality check is based on a lookup and 2 text fields.
For arguments sake lets say I was inserting Account records, this is what I currently have:
private void CreateAccounts()
{
var list = this.GetAccounts(); // get the list of Accounts, some may be new
IEnumerable<string> existingAccounts = linq.AccountSet.Select(account => account.AccountNumber); // get all Account numbers in CRM, linq is a serviceContextName variable
var newAccounts = list.Where(account => !existingAccounts.Contains(account.AccountNumber)); // Account numbers not in CRM
foreach (var accountNumber in newAccounts) // go through the new list again and get all the Account info
{
var account = newAccounts.Where(newAccount => newAccount.AccountNumber.Equals(accountNumber)).FirstOrDefault();
service.Create(account);
}
}
Is there a better way to do this?
I seem to be iterating through lists too many times, but it must be better than querying CRM multiple times:
foreach (var account in list)
{
// is this Account already in CRM
// if not create the Account
}
Your current method seems a bit backwards (get everything out of CRM, then compare it to what you have locally), but it may not be too bad depending on how many accounts you have ie < 5000.
For your simple example, you should be able to apply a where in statement.
Joining on multiple fields is a little more tricky. If you are running CRM > R12, you should be able to use the ExecuteMultipleRequests, creating a seperate request for each item in your list, and then batching them all up, so there is one big request "over the wire" to CRM.
Related
Time to leave the shy mode behind and make my first post on stackoverflow.
After doing loads of research (plugins, performance, indexes, types of update, friends) and after trying several approaches I was unable to find a proper answer/solution.
So if possible I would like to get your feedback/help in a Microsoft Dynamics CRM 2013/2015 plugin performance issue (or coding technique)
Scenario:
Microsoft Dynamics CRM 2013/2015
2 Entities with Relationship 1:N
EntityA
EntityB
EntityB has the following columns:
Id | EntityAId | ColumnDemoX (decimal) | ColumnDemoY (currency)
Entity A has: 500 records
Entity B has: 150 records per each Entity A record. So 500*150 = 75000 records.
Objective:
Create a Post Entity A Plugin Update to "mimic" the following SQL command
Update EntityB
Set ColumnDemoX = (some quantity), ColumnDemoY = (some quantity) * (some value)
Where EntityAId = (some id)
One approach could be:
using (var serviceContext = new XrmServiceContext(service))
{
var query = from a in serviceContext.EntityASet
where a.EntityAId.Equals(someId)
select a;
foreach (EntityA entA in query)
{
entA.ColumnDemoX = (some quantity);
serviceContext.UpdateObject(entA);
}
serviceContext.SaveChanges();
}
Problem:
The foreach for 150 records in the post plugin update will take 20 secs or more.
While the
Update EntityB Set ColumnDemoX = (some quantity), ColumnDemoY = (some quantity) * (some value) Where EntityAId = (some id)
it will take 0.00001 secs
Any suggestion/solution?
Thank you all for reading.
H
You can use the ExecuteMultipleRequest, when you iterate the 150 entities, save the entities you need to update and after that call the request. If you do this, you only call the service once, that's very good for the perfomance.
If your process could be bigger and bigger, then you should think making it asynchronous as a plug-in or a custom activity workflow.
This is an example:
// Create an ExecuteMultipleRequest object.
requestWithResults = new ExecuteMultipleRequest()
{
// Assign settings that define execution behavior: continue on error, return responses.
Settings = new ExecuteMultipleSettings()
{
ContinueOnError = false,
ReturnResponses = true
},
// Create an empty organization request collection.
Requests = new OrganizationRequestCollection()
};
// Add a UpdateRequest for each entity to the request collection.
foreach (var entity in input.Entities)
{
UpdateRequest updateRequest = new UpdateRequest { Target = entity };
requestWithResults.Requests.Add(updateRequest);
}
// Execute all the requests in the request collection using a single web method call.
ExecuteMultipleResponse responseWithResults =
(ExecuteMultipleResponse)_serviceProxy.Execute(requestWithResults);
Few solutions comes to mind but I don't think they will please you...
Is this really a problem ? Yes it's slow and database update can be so much faster. However if you can have it as a background process (asynchronous), you'll have your numbers anyway. Is it really a "I need this numbers in the next second as soon as I click or business will go down" situation ?
It can be a reason to ditch 2013. In CRM 2015 you can use a calculated field. If you need this numbers only to show up in forms (eg. you don't use them in reporting), you could also do it in javascript.
Warning this is for the desesperate call. If you really need your update to be synchronous, immediate, you can't use calculated fields, you really know what your doing etc... Why not do it directly in the database? I know this is a very bad advice. There are a lot of reason not to do it this way (you can read a few here). It's unsupported and if you do something wrong it could go really bad. But if your real situation is as simple as your example (just a calculated field, no entity creation, no relation modification), you could do it this way. You'll have to consider many things: you won't have any audit on the fields, no security, caching issues, no modified by, etc. Actually I pretty much advise against this solution.
1 - Put it this logic to async workflow.
OR
2 - Don't use
serviceContext.UpdateObject(entA);
serviceContext.SaveChanges();.
Get all the records (150) from post stage update the fields and ExecuteMultipleRequest to update crm records in one time.
Don't send update request for each and every record
This question is related to Parse.com
I want to build a push query to push a notification from cloud code to all users belonging to 'Moderator' role.
I tried below code, it failed to push as "users" is a relation, not an object.
var moderatorQuery = new Parse.Query(Parse.Role);
moderatorQuery.equalTo("name", "Moderators");
// Push to devices belonging to these moderators
var pushQuery = new Parse.Query(Parse.Installation);
pushQuery.matchesKeyInQuery("installationOwner", "users", moderatorQuery);
Another way of doing this is by fetching the Moderator role object and using the relation.query() or by calling getUsers() on moderator object.
But my requirement is to push to a query directly without fetching(get/find) any object.
How to build such a query ?
I think you need first to get the role, then use it's user relation query as the query to match in the pushQuery, like this:
var moderatorQuery = new Parse.Query(Parse.Role);
moderatorQuery.equalTo("name", "Moderators");
moderatorQuery.first().then(function(moderatorRole) {
var pushQuery = new Parse.Query(Parse.Installation);
var usersQuery = moderatorRole.relation("users").query();
pushQuery.matchesQuery("installationOwner", usersQuery);
// setup push, then send()
});
EDIT - Missed the last sentence of the question. The short answer is that there's no way to query the "many" side of the relationship in a single query.
The longer answer is that it can be done with additional data. You can make the query singular at the cost of keeping additional data up-to-date.
For example, you could keep an (a) isModerator bool on the User, or (b) more generally, a "roles" array of pointers (not a relation, because that's your root problem) on User, or (c) a separate table altogether that joins Role and User with singular pointers.
All of these ideas make the single-query easy (a) query User where isModerator == true, (b) query User where roles isEqual to moderator, (c) query TheJoinTable where role == moderator, include user and select user.
Doing this shifts the burden from the query to keeping the extra data up to date. You could accomplish this pretty simply using beforeSave on either end of the Role-User relation (probably Role).
All that said, you should examine the added constraint of a single query very carefully and make sure its worth the extra trouble.
Zoho Creator is a great system for quickly creating simple cloud applications. I've run into a problem with sub-forms, though: currently, Zoho Creator does not provide functionality for sorting sub-form records by a specified column. Instead, it sorts records in the order in which they were added.
My sub-form is a Creator Form that's linked to another Creator Form (basically, 2 different tables). The forms are linked with a bi-directional lookup relationship.
I've seen and tried implementing these "hacks", but none of them work for my situation:
[Zoho Forums, "Subforms sorting rows"][1]
[Zoho Forums, "Hack to sort rows of a subform and pre-populate row fields that I want to preset"][2]
I also called Zoho tech support, and after looking at my application, they said that sorting sub-form records is not currently possible.
Any other ideas?
My tested solution is still a hack, but until Zoho implements a method to sort sub-form records via the GUI, this will have to do.
First, create a function that you can call from anywhere (e.g. when a new sub-form record is added or changed)--for details on that, go here: http://www.zoho.com/creator/help/script/functions.html
This function will first duplicate the sub-form records by the parent record ID (sorting by the appropriate column) and then delete all sub-form records that were inserted before the script started:
int SubFormRecords_SortByAnything_ReturnCount(int ParentRecordID)
{
scriptStartTime = zoho.currenttime;
for each rSubFormRecord in SubFormRecords [ParentFieldName = input.ParentRecordID] sort by FieldName1, FieldName3, FieldName2
{
NewSubFormRecordID = insert into SubFormRecords
[
FieldName1 = rSubFormRecord.FieldName1
FieldName2 = rSubFormRecord.FieldName2
FieldName3 = rSubFormRecord.FieldName3
];
}
delete from SubFormRecords[ (Series == input.ParentRecordID && Added_Time < scriptStartTime) ];
return SubFormRecords[ParentFieldName == input.EventID].count();
}
Once the above sorting function is in place (customized for your application), call it when appropriate. I call it when adding a record associated with the sub-form, or when I change the sorting column values.
That works well, and as long as you don't have complex logic associated with adding and deleting records, it should have minimal impact on application performance.
Please let me know whether that works for you, and if you have any better ideas.
Caveat: This solution is not suitable for forms containing additional sub-form records because deleting the records will delete linked sub-form values.
Thanks.
I have a a very simple workaround:
1) You have to add a Form Workflow
2)Record Event - Create OR Edit OR Create/Edit (As per your requirement)
3)Form Event - On successful form submission
4)Let Main_Form be the link name of the Main Form
4)Let Sub_Form be the Link name of the Sub Form (Not the link name you specify in the main form for the same sub form)
4)Let Field1 and Field2 are fields of subform on which you want to sort subform records
5)Let Link_ID be lookup field of Mainform ID in the subform
Workflow
1)Sub_Records = Sub_Form[Link_ID == input.ID] sort by Field1,Field2;
(sort by multiple fields, add asc/desc as per requirement)
2)delete from Sub_Form[Link_ID == input.ID];
3)for each sub_record in Sub_Records
{
insert into Sub_Form
[
Added_User = zoho.loginuser
Link_ID = input.ID
Field1 = sub_record.Field1
Field2 = sub_record.Field2
]
}
//Now you check the results in edit view of the main form
Why isn't the exception triggered? Linq's "Any()" is not considering the new entries?
MyContext db = new MyContext();
foreach (string email in {"asdf#gmail.com", "asdf#gmail.com"})
{
Person person = new Person();
person.Email = email;
if (db.Persons.Any(p => p.Email.Equals(email))
{
throw new Exception("Email already used!");
}
db.Persons.Add(person);
}
db.SaveChanges()
Shouldn't the exception be triggered on the second iteration?
The previous code is adapted for the question, but the real scenario is the following:
I receive an excel of persons and I iterate over it adding every row as a person to db.Persons, checking their emails aren't already used in the db. The problem is when there are repeated emails in the worksheet itself (two rows with the same email)
Yes - queries (by design) are only computed against the data source. If you want to query in-memory items you can also query the Local store:
if (db.Persons.Any(p => p.Email.Equals(email) ||
db.Persons.Local.Any(p => p.Email.Equals(email) )
However - since YOU are in control of what's added to the store wouldn't it make sense to check for duplicates in your code instead of in EF? Or is this just a contrived example?
Also, throwing an exception for an already existing item seems like a poor design as well - exceptions can be expensive, and if the client does not know to catch them (and in this case compare the message of the exception) they can cause the entire program to terminate unexpectedly.
A call to db.Persons will always trigger a database query, but those new Persons are not yet persisted to the database.
I imagine if you look at the data in debug, you'll see that the new person isn't there on the second iteration. If you were to set MyContext db = new MyContext() again, it would be, but you wouldn't do that in a real situation.
What is the actual use case you need to solve? This example doesn't seem like it would happen in a real situation.
If you're comparing against the db, your code should work. If you need to prevent dups being entered, it should happen elsewhere - on the client or checking the C# collection before you start writing it to the db.
I have a table for Projects, a table for Documents (with a FK to Projects), and a table for users with a relationship to projects and documents. Users can be subscribers to a document and that is how they have a relationship to the document.
Users are considered team members on a project. A document can only assigned to one project at a time. Only users, from the "Project Team" can be subscribed to a document.
What I am struggling with, in my application, is that admins are able to add users as subscribers to a document. However, the admin can only pick from a list of unsubscribed users. Those unsubscribed users are members of the project team, like I said above.
Right now, I am creating 2 queries on my database to pull all the subscribed users and the entire list of team members. I then compare the list and only pull the team members that are not subscribed.
I am not sure if I should even index this type of data or just pull from the database directly. If I should use an index, this data needs to be updated quickly becuase the admins need that unsubscribed list rather fast.
This is what my query looks like going against Entity Framework 4.1:
var currentSubscribers = _subscriptionRepository.Get(s => s.DocumentId == documentId).Select(s => s.Contact).ToList();
if (projecTeamMembers != null)
{
var availableSubscribers = (projecTeamMembers.Where(
p => !(currentSubscribers.Select(c => c.Id)).Contains(p.Id))).ToDictionary(c => c.Id, c=> c.LastName + ", " + c.FirstName);
return availableSubscribers;
}
else
{
return null;
}
This works great in EF, but I have been thinking of indexing my data using Lucene.Net and need some advice on if I should do this or not.
Write this query indide your data repository where you have access to the Database context
var q = from m in dbContext.ProjecTeamMembers
where !(from s in dbContext.Subscribers
where s.DocumentId == documentId &&
s.Contact.Id == p.Id
select s).Any()
select m;
var availableSubscribers = q.ToDictionary(m => m.Id, c=> m.LastName + ", " + m.FirstName);
The best way I have found to do something like this is, using Lucene.net, is to keep an index of subscribers and an index of all team members. And compare the two. It's faster than pulling form the database each time.