I have a Cocoa Mac application with a search field and a collection view bound to an sqlite table through CoreData. The table contains several hundred thousand records with text fields (name, place, ...) indexed by name. I'm using BEGINSWITH predicate in a search field binding to select a dozen of records for display in a collection view. Everything works fine, but the problem is that CoreData loads the whole table into memory at the first query request and only then does the necessary filtering of records for display which means considerable delay for the user.
Is there a way to set up CoreData so that the whole table does not load into memory? Ideally, I would like to fetch only the first 100 items from a range of alphabetically sorted records for every query in the search field.
On iOS, this would be achieved using a NSFetchedResultsController. The following question describes the Mac equivalent controllers (such as NSArrayController): NSFetchedResultsController Mac OSX Cocoa equivalent.
EDIT
As per my comment below, a NSFetchRequest should be created in conjunction with the array controller. This can then be configured with fetchLimit and fetchOffset to determine how many fetch results are returned.
Make sure that you have the fetch request set to fetch as faults only. That way only the objects whose attributes who are actively accessed will be fully loaded into memory.
Related
I'm in need of entering a few data points in the UI of a FileMaker app that are used either for search or for computation, but that have no relation to any field in a database (and don't need to be saved). So I want to add an input field without having it tied to a table field, and it seems that's something FileMaker just doesn't do.
Two use cases:
a) I want a custom search/filter interface instead of using the FM one. My users should see two calendars, pick two dates and the data is filtered by those (between them), as well as additional criteria, which don't directly translate to field searches. I know I can use "startdate ... enddate", but I'd like a more user-friendly interface.
b) Users enter a few data points into seperate fields which are then computed and combined into one database field by script. This is technical data that is entered by copy-and-paste and needs a bit of parsing before I put it into the database. Again, I'd like a field that isn't related to the database, put a script trigger on it, and when data is entered there, it is parsed and put into the actual DB fields.
Is it possible at all to have input fields not related to a database in FileMaker ?
If not, what's the best practice? I thought about setting up a dummy table with various fields I can use, but maybe there's a better way?
You should read up on global fields. They can be in any table and are accessible from all tables. They do not retain their value after the session is closed if the file is hosted. Use a script to perform a search based on what the user types in the global field.
I'm trying to export a large number of records from my database, but I need relationship data in order to build the export correctly. Ideally I would be able to use cursor() to get a Lazy Collection, but that won't load the relationships. I can't load the relationship within a loop, because that will create N+1 queries, and this could be hundreds of thousands of additional queries, which is unacceptable.
Here's what "works" (but runs out of memory):
Record::with('projects')->get()->map(function ($record) {
dd($record); // Shows the `projects` relationship
});
But when I use cursor()...
Record::with('projects')->cursor()->map(function ($record) {
dd($record); // Does NOT show the `projects` relationship
});
Is there a way to get a lazy collection that includes a record's relationship? I have looked in the documentation and it's not clear. Other suggestions have been to use chunk() which is unfortunately not a possibility in this situation.
EDIT: I shouldn't say chunk isn't a possibility, but it's a very expensive re-write. Currently, the data is structured with a lot of variability. So in order to construct the CSV for export, I need (for example) a header for the file. I currently grab that header by looping through all the records (the fields are stored in a JSONB field) and building out an array based on the fields present on those records.
I am also normalizing the data against those headers. So if one record has the field "address-1" but another record doesn't have that, the one that doesn't have it instead shows a blank value in the appropriate column. Otherwise, when inserting the row into the CSV, it doesn't respect the header.
These operations currently grab the entire data set and use a LazyCollection to map the header and normalize the records, and then feed it into the CSV one at a time. It would be ideal if I could grab relationships in a LazyCollection as well rather than having to rewrite the workflow.
according to this doc
cursor work in db stage, while loading relations come after method 'get' or 'first' ...
so: the code in cursor will work in db row represented as Model instance before the overall result, means that this code will run into db, without loading the relation, again db row (iterate through your database records...)
if you can't use chunk... then i think that you can use mySql to manage your data using raw-expressions
I have a NSTableView and I binded my Array Controller to it. The Array Controller is binded to my Managed Object Context. Everything in the NSTableView works but the objects in my TableView rearranges randomly when I start up my app. For example, the order of [1,2,3] will suddenly change to [3,1,2] and this happens occasionally. I want it so that things stay in the order they were put when making them. How do I stop the rearranging?
The rows in the CoreData data store are not sorted! If you want to view them by creation date you need to add your own timestamp to every object and pass a sort descriptor to the fetch request.
Edit: If you are using SQLite as your CoreData store I recommend to have a look at the SQLite file. This will give you an idea on how CoreData works and explain why certain things (like unordered rows) are the way they are.
P.S. I use Base to inspect SQLite databases. It's not cheap but quite nice.
I have a working system that lets me build a database containing instances of various entities , all linked together nicely.
Before I knew I would care, I came across a tutorial on using Core Data and bindings, and it went through a complete case where you get a table showing all the entities of some type with a column for each property. It showed both the UI side and the Data model side - not that I need the data model part at this point. Now, darned if I can find it. This is one of those things that is supposed to be easy, and requires virtually no code, but getting exactly the right connections in UIBuilder is not going to happen if I can't find instructions.
Also, I thought I came across an example of something like a query editor where the user could select which properties to sort on, which to match on, etc. Did I imagine that?
Anyone out there know where I can find such?
Sure, you can do this without code:
Add an array controller to your nib.
Bind or connect an outlet for its managed object context
Set the array controller to Entity mode, fill in the entity name, and select Prepares Content.
Bind your table view columns to array controller's arranged objects, and fill in the key name for the model key.
Regarding the query editor, open up the model, and on the Editor menu click Add Fetch Request.
I found at least a partial answer to the query editor question, in this apple tutorial. Not sure how far it will get me, as I prefer to write code where possible, since then I can leave a trail of comments.
I am doing a ajax based search which suggests the values from the database tables as the user inputs fields in the search box. It does work but its really very slow, takes more than 10 seconds minimum for any suggestions to load. how do I optimize this to make it fast..
Like Index it/save it in cache/crawl?
Right now autoSearch() js function is fired which retrieves data from the related table to load the suggestions. how do I make this faster?
I dont know if this will make a difference but I am using grails/groovy for application development.
Thanks
Priyank
Have you added an index to any searched fields (or checked in the database to make sure the examined fields are indexed)?
The Grails GORM does not automatically create indices for text fields unless you add the appropriate static mapping to your domain class.