I'm having issue with duplicates and promises.
I'm fetching GitHub pull requests for every organization repository at the same time. While doing so, I might stumble upon a GitHub profile that I did not save yet in my database.
So I need to save this profile with a specific and unique ID, that is generated with the profile GitHub ID and the organization ID.
When I try to create a new element with a unique ID already saved, my database is throwing an error (which is expected), but when I am using promises, my database does not throw any error and creates multiple elements with the same unique ID. Because I ask my DB to save them at the same time.
Do you know how to avoid creating duplicates of unique elements while using promises?
Related
I'm using the Dynamics connector in Azure Data Factory.
TLDR
Does this connector support loading child records which need a parent record key passed in? For example if I want to create a contact and attach it to a parent account, I upsert a record with a null contactid, a valid parentcustomerid GUID and set parentcustomeridtype to 1 (or 2) but I get an error.
Long Story
I'm successfully connecting to Dynamics 365 and extracting data (for example, the lead table) into a SQL Server table
To test that I can transfer data the other way, I am simply loading the data back from the lead table into the lead entity in Dynamics.
I'm getting this error:
Failure happened on 'Sink' side. ErrorCode=DynamicsMissingTargetForMultiTargetLookupField,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=,Source=,''Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=Cannot find the target column for multi-target lookup field: 'ownerid'.
As a test I removed ownerid from the list of source columns it loads OK.
This is obviously a foreign key value.
It raises two questions for me:
Specifically with regards to the error message: If I knew which lookup it needed to use, how can I specify which lookup table it should validate against? There's no settings in the ADF connector to allow me to do this.
This is obviously a foreign key value. If I only had the name (or business key) for this row, how can I easily lookup the foreign key value?
How is this normally done through other API's, i.e. the web API?
Is there an XRMToolbox addin that would help clarify?
I've also read some posts that imply that you can send pre-connected data in an XML document so perhaps that would help also.
EDIT 1
I realised that the lead.ownertypeid field in my source dataset is NULL (that's what was exported). It's also NULL if I browse it in various Xrmtoolbox tools. I tried hard coding it to systemuser (which is what it actually is in the owner table against the actual owner record) but I still get the same error.
I also notice there's a record with the same PK value in systemuser table
So the same record is in two tables, but how do I tell the dynamics connector which one to use? and why does it even care?
EDIT 2
I was getting a similar message for msauto_testdrive for customerid.
I excluded all records with customerid=null, and got the same error.
EDIT 2
This link appears to indicate that I need to set customeridtype to 1 (Account) or 2 (Contact). I did so, but still got the same error.
Also I believe I have the same issue as this guy.
Maybe the ADF connector suffers from the same problem.
At the time of writing, #Arun Vinoth was 100% correct. However shortly afterwards there was a documentation update (in response to a GitHub I raised) that explained how to do it.
I'll document how I did it here.
To populate a contact with against a parent account, you need the parent accounts GUID. Then you prepare a dataset like this:
SELECT
-- a NULL contactid means this is a new record
CAST(NULL as uniqueidentifier) as contactid,
-- the GUID of the parent account
CAST('A7070AE2-D7A6-EA11-A812-000D3A79983B' as uniqueidentifier) parentcustomerid,
-- customer id is an account
'account' [parentcustomerid#EntityReference],
'Joe' as firstname,
'Bloggs' lastname,
Now you can apply the normal automapping approach in ADF.
Now you can select from this dataset and load into contact. You can apply the usual automapping approach, this is: create datasets without schemas. Perform a copy activity without mapping columns
This is the ADF limitation with respect to CDS polymorphic lookups like Customer and Owner. Upvote this ADF idea
Workaround is to use two temporary source lookup fields (owner team and user in case of owner, account and contact in case of customer) and with parallel branch in a MS Flow to solve this issue. Read more, also you can download the Flow sample to use.
First, create two temporary lookup fields on the entity that you wish to import Customer lookup data into it, to both the Account and Contact entities respectively
Within your ADF pipeline flow, you will then need to map the GUID values for your Account and Contact fields to the respective lookup fields created above. The simplest way of doing this is to have two separate columns within your source dataset – one containing Account GUID’s to map and the other, Contact.
Then, finally, you can put together a Microsoft Flow that then performs the appropriate mapping from the temporary fields to the Customer lookup field. First, define the trigger point for when your affected Entity record is created (in this case, Contact) and add on some parallel branches to check for values in either of these two temporary lookup fields
Then, if either of these conditions is hit, set up an Update record task to perform a single field update, as indicated below if the ADF Account Lookup field has data within it
I'm basically trying to create a primary ID between CRM and QuickBooks. Figured I'd just use the existing PK in CRM for the lookup. I'd like the PK to visible to the user, but not editable in CRM.
This has presented several problems in that you can't do that out of the box. I thought I read somewhere you could either via business rule or calculated field, but I haven't had luck with that.
It sounds like it would require web resources if I were to go this route.
The other option would be to just generate unique values for every record in Accounts and Contacts.
Does this automatically populate existing records or just new records? How do I get it to populate existing records?
You can use Auto number manager for configuring an auto-number attribute in every entity. This seeds a number based on configured format for new records. Uniqueness assured by SQL sequence feature & no need of any extra plugin/workflow.
For existing records - you can design a workflow along with a temp entity to assign auto-number. Read more.
Otherwise you can use SSIS + Kingswaysoft package to generate auto-number & assign for existing records.
I am suggesting you to create a new text field on the entity and create a pre plugin that will get the record primary GUID id from context and will set this GUID into the newly added attribute. You can set this field as read-only of form as well.
OR you can generate new GUID as well into the plugin.
Simple parent/child scenario like Order and OrderLineItems. I am inserting a new Order, the OrderID is an identity column (sql server). I'm also inserting OrderLineItems in the same SaveChanges transaction. I need to get the new OrderID into the OrderLineItems, but not sure how to do it. I have the appropriate FK relationships setup properly. When I save, I get an error that OrderID is a required field in OrderLineItems.
Will I have to split this out into 2 server calls? First to insert the Order, which will return the OrderID. And then another to insert the OrderLineItems?
The Breeze documentation discusses this topic (key generation) at several points including but not limited to: http://www.breezejs.com/documentation/save-changes, http://www.breezejs.com/documentation/extending-entities and http://www.breezejs.com/documentation/add-new-entity.
The basic idea idea is that providing that your model and metadata are set up properly, breeze can assign a temporary id in place of the identity column for use in linking your order and orderlineitem entities prior to being saved. As part of the save process, Breeze updates these temporary keys to their "real" key values and updates the local cache as well upon successful completion of the save.
I have uploaded my MVC3 project , it's s simple blog , at first it works well but after couple hours! following error appears (I've made custom error to Off to see the error)
The model backing the 'SiteContext' context has changed since the database was created. Either manually delete/update the database, or call Database.SetInitializer with an IDatabaseInitializer instance. For example, the DropCreateDatabaseIfModelChanges strategy will automatically delete and recreate the database, and optionally seed it with new data.
to solve this I have to manually delete my database and create again and then restore to the backup that I have created. but after after 2 hours again I get the error!
I really don't have any idea , what caused that ??
When you create a model and ask EF to create a database from it, EF would hash the model and store the hash value with the database. Whenever the context is created, EF recomputes the hash and matches it against what is stored at the database. If the model changes in any way, the resulting hash will be different and EF will throw the exception you have just seen. This is done in order to keep the model in sync with the database.
Is there any way the model could have changed during runtime?
One thing you could do to figure out the difference is to
1.Re-create the database from the model as you are doing now and get it scripted (script1.sql).
2.Wait till the error happens and delete the db and re-create it again and script it (script2.sql)
3.Try to compare the two and see whether you can spot a difference in the schemas.
This should give you an idea of what has changed in the model.
Goodluck
I'm building off of a previous discussion I had with Jon Skeet.
The gist of my scenario is as follows:
Client application has the ability to create new 'PlaylistItem' objects which need to be persisted in a database.
Use case requires the PlaylistItem to be created in such a way that the client does not have to wait on a response from the server before displaying the PlaylistItem.
Client generates a UUID for PlaylistItem, shows the PlaylistItem in the client and then issue a save command to the server.
At this point, I understand that it would be bad practice to use the UUID generated by the client as the object's PK in my database. The reason for this is that a malicious user could modify the generated UUID and force PK collisions on my DB.
To mitigate any damages which would be incurred from forcing a PK collision on PlaylistItem, I chose to define the PK as a composite of two IDs - the client-generated UUID and a server-generated GUID. The server-generated GUID is the PlaylistItem's Playlist's ID.
Now, I have been using this solution for a while, but I don't understand why/believe my solution is any better than simply trusting the client ID. If the user is able to force a PK collison with another user's PlaylistItem objects then I think I should assume they could also provide that user's PlaylistId. They could still force collisons.
So... yeah. What's the proper way of doing something like this? Allow the client to create a UUID, server gives a thumbs up/down when successfully saved. If a collision is found, revert the client changes and notify of collison detected?
You can trust a client generated UUID or similar global unique identifier on the server. Just do it sensibly.
Most of your tables/collections will also hold a userId or be able to associate themselves with a userId through a FK.
If you're doing an insert and a malicious user uses an existing key then the insert will fail because the record/document already exists.
If you're doing an update then you should validate that the logged in user owns that record or is authorized (e.g. admin user) to update it. If pure ownership is being enforced (i.e. no admin user scenario) then your where clause in locating the record/document would include both the Id and the userId. Now technically the userId is redundant in the where clause because the Id will uniquely find one record/document. However adding the userId makes sure the record belongs to the user that's doing the update and not the malicious user.
I'm assuming that there's an encrypted token or session of some sort that the server is decrypting to ascertain the userId and that this is not supplied by the client otherwise that's obviously not safe.
A nice solution would be the following: To quote Sam Newman's "Building Microservices":
The calling system would POST a BatchRequest, perhaps passing in a
location where a file can be placed with all the data. The Customer
service would return a HTTP 202 response code, indicating that the
request was accepted, but has not yet been processed. The calling
system could then poll the resource waiting until it retrieves a 201
Created indicating that the request has been fulfilled
So in your case, you could POST to server but immediately get a response like "I will save the PlaylistItem and I promise its Id will be this one". Client (and user) can then continue while the server (maybe not even the API, but some background processor that got a message from the API) takes its time to process, validate and do other, possibly heavy logic until it saves the entity. As previously stated, API can provide a GET endpoint for the status of that request, and the client can poll it and act accordingly in case of an error.