My schema is as follows
RecordType: OwnedEvents contains an element called "Owner". This Owner element points to CKReference created from a RecordType of type "OwnedEvents".
I have N OwnedEvents objects all pointing to one Owner object.
When I execute a query to retrieve all the records that point a given Owner object, I see that, in some cases, the query does not return all the records that point to it (it does return it in most cases). It also does not return an error in the completion block. When I go to the CloudKit dashboard, I see that all OwnedEvents objects are pointing to the same and correct Owner object as expected. The code I am using to retrieve is below.
My questions are:
Is this kind of "silent" failure expected? I.e Do I need to identify this and recover? OR Am I doing something wrong in my query?
Is there any way to say the query must return an error if not all records matching the criterion are retrieved? (i.e an atomic version of fetch..I guess not, since it is tough to know ahead of time what the expected number of matching records is, but I am wondering if there is something I can specify in the fetch or in the schema that hints at min number of records expected in the fetch result).
Is it more efficient to fetch using a predicate formed using a reference compared to fetch that uses a predicate of the form "Get all records of type OwnedEvents with Column X > N1 and Column X
The reference is pointed to be same set of records, N1...N2
I.e If the reference based retrieval is going to be unreliable I might as well go away from it and take a brute force approach, if it is not any faster. At least in latter I have a easy way of identifying errors
Code used
CKReference* recordToMatch = [[CKReference alloc] initWithRecordID:eventCollectionID
action:CKReferenceActionDeleteSelf]; //eventCollectionID is recordID of Record of type Owner
NSPredicate* predicate = [NSPredicate predicateWithFormat:#"Owner == %#", recordToMatch];
// Create the query object.
CKQuery* query = [[CKQuery alloc] initWithRecordType:#"OwnedEvents" predicate:predicate];
CKQueryOperation *queryObject = [[CKQueryOperation alloc] initWithQuery:query] ;
queryObject.queryCompletionBlock = ^(CKQueryCursor * cursor, NSError * operationError) {
if(operationError)
{
DDLogError(#"Error %# occured during query",operationError) ;
}
else if(cursor)
{
DDLogVerbose(#"Not all objects satisfying the query were returned. Need to get next batch") ;
//To do fetch remaining entries
CKQueryOperation *queryObject = [[CKQueryOperation alloc] initWithCursor:cursor] ;
//Execute another query with this object
}
else
{
DDLogVerbose(#"All objects satisfying the query were returned") ;
//Take appropriate action with rcvd data
}
} ;
queryObject.recordFetchedBlock = ^(CKRecord *record) {
DDLogVerbose(#"Received record with seq Num: %#, type",record[#"SequenceNumber"],record[#"EventType"]) ;
//Check if items are in sequence
} ;
[self.publicDatabase addOperation:queryObject] ;
Related
I'm using EF Core but I'm not really an expert with it, especially when it comes to details like querying tables in a performant manner...
So what I try to do is simply get the max-value of one column from a table with filtered data.
What I have so far is this:
protected override void ReadExistingDBEntry()
{
using Model.ResultContext db = new();
// Filter Tabledata to the Rows relevant to us. the whole Table may contain 0 rows or millions of them
IQueryable<Measurement> dbMeasuringsExisting = db.Measurements
.Where(meas => meas.MeasuringInstanceGuid == Globals.MeasProgInstance.Guid
&& meas.MachineId == DBMatchingItem.Id);
if (dbMeasuringsExisting.Any())
{
// the max value we're interested in. Still dbMeasuringsExisting could contain millions of rows
iMaxMessID = dbMeasuringsExisting.Max(meas => meas.MessID);
}
}
The equivalent SQL to what I want would be something like this.
select max(MessID)
from Measurement
where MeasuringInstanceGuid = Globals.MeasProgInstance.Guid
and MachineId = DBMatchingItem.Id;
While the above code works (it returns the correct value), I think it has a performance issue when the database table is getting larger, because the max filtering is done at the client-side after all rows are transferred, or am I wrong here?
How to do it better? I want the database server to filter my data. Of course I don't want any SQL script ;-)
This can be addressed by typing the return as nullable so that you do not get a returned error and then applying a default value for the int. Alternatively, you can just assign it to a nullable int. Note, the assumption here of an integer return type of the ID. The same principal would apply to a Guid as well.
int MaxMessID = dbMeasuringsExisting.Max(p => (int?)p.MessID) ?? 0;
There is no need for the Any() statement as that causes an additional trip to the database which is not desirable in this case.
I am performing an Elasticsearch query using the high-level-rest-api for Java and expect to see records that are either active or do not have a reference id. I'm querying by name for the records and if I hit the index directly with /_search?q=, I see the results I want.
Is my logic correct (pseudo-code):
postFilters.MUST {
Should {
MustNotExist {referenceId}
Must {status = Active}
}
Should {
MustNotExist {referenceId}
Must {type = Person}
}
}
What I get are records that are active with a reference id. But, I want to include records that also do not have a referenceId, hence why I have MustNotExist {referenceId}.
For simplicity, the second Should clause can be dropped (for testing) as the first one is not working as expected by itself.
In my case, I had to use a match query instead of a term query because the value I was querying for was not a primitive or a String. For example, the part where Must, type = Person, Person was an enum, and so looking for "Person" was not quite right, whereas match allowed it to "match".
Code added.
I have searched endlessly for a solution here and cannot find one, please help!
I have three objects (A, B, and C). A has a lookup to B, and B is the master to C (detail). Both A and C have many records related to each B record.
I want to have a job run that gets a subset of records from object C (it will usually be around 5,000 records). Then go through each of those and get the records on Object A that lookup to the same Object B record, summarize an Object A number field, and put that on the C record.
I have successfully gotten this to work in small scale, <100 Object C records. But each Object C record requires a new SOQL query since I am iterating through them in a for loop after I get all the Object C records. Plus I know this it is not best practice to ever have a query in a loop.
How can I get this to work? Since the records share the relationship with Object B, is there another way to get the data from the Object A records that match? Or is there some way to pull two lists, one Object C and one Object A. Then summarize the Object A records and line the lists up some how?
Thanks in advance!
Code:
public class nightlyJob {
public static void updateNumbers(){
integer I = 29;
List<ObjectC__c> CUpdateList = new List<ObjectC__c>();
List<ObjectC__c> CpullList =
[SELECT ID, Index__c, ObjectB__r.id
FROM ObjectC__c
WHERE Index__c = :I];
for(ObjectC__c s : CpullList){
List<ObjectA__c> AList =
[SELECT ObjectB__c, Number__c
FROM ObjectA__c
WHERE ObjectB__c = :s.ObjectB__r.Id];
decimal NumSum = 0;
for(ObjectA__c a : AList){
NumSum = a.Number__c + NumSum;
}
s.Num__c = NumSum;
CUpdateList.add(s);
}
update CUpdateList;
}
}
It looks like you are really missing several fundamental concepts at the moment.
The biggest problem you are up against in SFDC development is that "database" operations are very expensive and are strictly limited. It's not just a matter of "best practice": if in a single transaction you exceed these limits -- number of SOQL calls, number of records returned, number of records updated, number of DML statements, etc. -- your transaction will fail. For details, search online for "Salesforce Execution Governors and Limits".
You can write code that works within these limitations, but there is a bit of a learning curve.
First, learn to use collections with SOQL queries to get your SOQL queries out of loops. This is a.k.a. "bulkfication" and it fundamental to SFDC development:
List<ObjectC__c> CpullList =
[SELECT ID, Index__c, ObjectB__r.id
FROM ObjectC__c
WHERE Index__c = :I];
// Create a map with the results of this query.
// key=ObjectC__c.Id, value = Object__c record
Map<Id, ObjectC__c> objCmap = Map<Id, ObjectC__c>(CpullList);
// Build a set of all the Object_B id's from this result set
Set<Id> objBids = new Set<Id>();
for (ObjectC__c record : CpullList) {
objBids.add(record.ObjectB__r.id);
}
// Now you can use only one SOQL query instead of a loop
List<ObjectA> AList = [SELECT ObjectB__c, Number__c
FROM ObjectA__c
WHERE ObjectB__c in:objBids];
Next, use "SOQL aggregate functions" whenever you can. Example: in your code here, you could use "SUM()" and "group by" instead of performing these calculations with loops:
// Get the sum of ObjectA__c.Number__c for each Object B in objBIds
AggregateResult[] groupedResults = [select ObjectB__c,
sum(Number__c) sumA
from ObjectA__c
where ObjectB__c in: objBids
group by ObjectB__c];
for (AggregateResult ar : groupedResults) {
System.debug('Object B Id' + ar.get('Objectb__c'));
System.debug('Sum of ObjectA__c.Number__c' + ar.get('sumA'));
// Here, you might want to build a Map<Id, Integer> sumAmap:
// key=Object B ID, value=sumA
// and then use it along with objCmap to build a collection of Object C's
// for your update statement...
}
You can continue this process and apply these ideas to make the code more efficient.
But even after you have your methods working as efficiently as possible, you still may run into limits due to the number of records you're dealing with. At that point, you will need to learn about the Batchable interface, the Queuable interface and #future calls (how to process a larger number of records, split across transactions) That's really too much to information to cover in a single SO answer.
I'm using LINQ to SQL to obtain data from a set of database tables. The database design is such that given a unique ID from one table (Table A) one and only one instance should be returned from an associated table (Table B).
Is there a more concise way to compose this query and ensure that only one item was returned without using the .Count() extension method like below:
var set = from itemFromA in this.dataContext.TableA
where itemFromA.ID == inputID
select itemFromA.ItemFromB;
if (set.Count() != 1)
{
// Exception!
}
// Have to get individual instance using FirstOrDefault or Take(1)
FirstOrDefault helps somewhat but I want to ensure that the returned set contains only one instance and not more.
It sounds like you want Single:
var set = from itemFromA in this.dataContext.TableA
where itemFromA.ID == inputID
select itemFromA.ItemFromB;
var onlyValue = set.Single();
Documentation states:
Returns the only element of a sequence, and throws an exception if there is not exactly one element in the sequence.
Of course that means you don't get to customize the message of the exception... if you need to do that, I'd use something like:
// Make sure that even if something is hideously wrong, we only transfer data
// for two elements...
var list = set.Take(2).ToList();
if (list.Count != 1)
{
// Throw an exception
}
var item = list[0];
The benefit of this over your current code is that it will avoid evaluating the query more than once.
I'm using NHibernate 3.2 and I have a repository method that looks like:
public IEnumerable<MyModel> GetActiveMyModel()
{
return from m in Session.Query<MyModel>()
where m.Active == true
select m;
}
Which works as expected. However, sometimes when I use this method I want to filter it further:
var models = MyRepository.GetActiveMyModel();
var filtered = from m in models
where m.ID < 100
select new { m.Name };
Which produces the same SQL as the first one and the second filter and select must be done after the fact. I thought the whole point in LINQ is that it formed an expression tree that was unravelled when it's needed and therefore the correct SQL for the job could be created, saving my database requests.
If not, it means all of my repository methods have to return exactly what is needed and I can't make use of LINQ further down the chain without taking a penalty.
Have I got this wrong?
Updated
In response to the comment below: I omitted the line where I iterate over the results, which causes the initial SQL to be run (WHERE Active = 1) and the second filter (ID < 100) is obviously done in .NET.
Also, If I replace the second chunk of code with
var models = MyRepository.GetActiveMyModel();
var filtered = from m in models
where m.Items.Count > 0
select new { m.Name };
It generates the initial SQL to retrieve the active records and then runs a separate SQL statement for each record to find out how many Items it has, rather than writing something like I'd expect:
SELECT Name
FROM MyModel m
WHERE Active = 1
AND (SELECT COUNT(*) FROM Items WHERE MyModelID = m.ID) > 0
You are returning IEnumerable<MyModel> from the method, which will cause in-memory evaluation from that point on, even if the underlying sequence is IQueryable<MyModel>.
If you want to allow code after GetActiveMyModel to add to the SQL query, return IQueryable<MyModel> instead.
You're running IEnumerable's extension method "Where" instead of IQueryable's. It will still evaluate lazily and give the same output, however it evaluates the IQueryable on entry and you're filtering the collection in memory instead of against the database.
When you later add an extra condition on another table (the count), it has to lazily fetch each and every one of the Items collections from the database since it has already evaluated the IQueryable before it knew about the condition.
(Yes, I would also like to be the extensive extension methods on IEnumerable to instead be virtual members, but, alas, they're not)