We have some big models where we need to read properties via the model derivative api. Reading all properties leads to an out of memory of the heap. We need to check the properties of each object for a custom prop set in a cad program like revit or navisworks.
So we are exploring to fetch properties for an object, explained here:
https://forge.autodesk.com/blog/new-objectid-query-parameter-model-derivative-properties-api
But after reading the metadata for the guid, we have like 50k of objectids or more. Thats too much to fetch the properties separately per object.
Is there a possibility to:
- fetch properties for multiple object id's?
- Fetch the properties for an object id and all his children?
Or is there another recommendation on how to handle such big models where the response when reading all the properties is too big (and we don't know up front which objectIds to read properties from)?
kind regards
I completely understood your question and as I know you working with .nwm format which is basically includes all the files from project (this is the reason for such amount objects in metadata)
For such case you can use middle-ware server with custom helpers methods, please take a look on this repo from Cyrille Fauvel,
https://github.com/cyrillef/propertyServer
It can help you working with multiple id, with range of id's, some methods you can take as base for your own.
Also, as far as I understood you getting property programatically so maybe you can some how use 'name' field in metadata object which is also can be unique as 'guid'.
Related
This is just a theoretical question, I am new to PyQT. I am not able to identify classes or code examples that will show how the model updates the data (if that is done via Qt)
Right now my understanding is that the programmer is responsible for writing the methods/code for updating the data when a change is done in the model data.
Am I correct ?
Update: It seems that my question needs a drawing :-)
So it is data as in data stored in a database. By my understanding the data in this picture is the data stored in the database not the data that the model is based on which is a subset of the data in the database
Try following some of the tutorials and look at the example programs provided with PyQt or PySide.
A model is an object which stores data. This can be as simple as a list of strings. A view is an object which displays data. Qt provides some standard models for storing common types of data and some standard views for displaying common types of data. For simple cases you can use these pre-built components. These components have standard methods for adding or removing data from the model, or updating the view as the model changes. These changes are propagated from the model to the view, or from the view or control widgets to the model, using signals. Thanks to standardized naming of methods and signals this all works pretty seamlessly.
However your specific data storage or data presentation needs might be a bit different. In this case you can implement a custom model or a custom view, or if necessary both. You implement your custom model (or view) by subclassing one of the classes provided by the framework. You have to implement specific named methods on these subclasses because it's the standard naming that makes the automatic behavior with other components work, but can also add your own custom methods and such as you see fit.
Further to MiniMe's comment:
Lets look at an example. Qt Models have a xx.setData() method to update a data item, which takes parameters specifying where in the model the data goes (index) and the new data. When you implement that method, you perform the work of updating the data to whatever underlying storage mechanism you use (e.g. a list or dictionary) and emit the xx.dataChanged() signal. If any views are linked to this model, they will have subscribed to this signal and so will update themselves automatically.
def setData(self, index, value):
# Update the data in the underlying python list self.my_list
# The index object is of a data type provided by Qt. To perform
# this operation we have to extract the row number where the change
# is to be made, then use that to update the appropriate entry in
# the list.
self.my_list[index.row()] = value
# Create a PyQt modelIndex object based on the row number that was
# updated. The self.index() method is provided by Qt.
modelIndex = self.index(index.row())
# Send the modelIndex of the change to any connected views so they
# know to update themselves
self.dataChanged.emit(modelIndex)
# In reality we'd do some validation checks and return False if there
# was a problem and the data change didn't happen.
return True
Similar methods would need to be implemented for adding a new data item in the list or removing a data item from the list. These methods would make the change, send a signal indicating which part of the model had changed to any attached views, then return. I hope this is more helpful.
Lets say we have a class "Car" than has different pieces of data ( maker, model, color, fabrication date, registration date, etc). The class has no method to get data, but it knows to as for it from another object (sent via constructor, let's cal it for short DS).- and the same for when needing to update changes.
A method getColor() would be implemented like this
if(! this->loaded('color')){
this->askDS('color') // this will do the necesarry work to generate a request to DS
}
return this->information('color');
Nothing too fancy so far. No comes the part i want to find out if it has a name, or if there are libraries / frameworks that do this already.
DS has a list of methods registered dinamically based on the class that needs data. For car we have:
input: car serial number, output: method to use to read the numbers to extract raw values
input: car raw color value, output: color code
input: car color code, manufacturer, year, mode, output:human-readable color (for example navy blue)
Now, DS or any method does not have an ordered list of using command to start from serial number and return the color blue, but if can construct a chain of methods that from one set of data, it can run them in order and get the desired data.
For our example above, DS runs 1,2,3 in that order and injects the data resulted from all methods into the class object that needed it.
Now if the car needs registration info, we have method (4) that gets that from the police database with an api request.
So, given:
- a type of model (class/object)
- a list of methods that take a fixed list of input(object properties) and give out a fixed list of output (object properties)
- a class DS that can glue the methods and run the needed ones for a model to get from property A (serial) to properby B (human readable colour) without the model or DS having a preconfigured way to get this data but finding it as needed.
does this have a name or is it already implemented somewhere ?
I've implemented a very basic prototype and it works very nice and i think this implementation method has useful features:
if you have a set of methods that do sql queries and then your app switches to using an api, you only need to change the methods and don't have to touch any other part of the application
when looking for a chain of methods that resolve the 'need' the object has, you can find a method chain, run it, if it fails keep looking for another list of methods based on the currently available data - so if you have multiple sources for a piece of data, it can try multiple versions
starting from the above paragraph i could start with an app that only has sql queries for data retrieval - when i find out a part of the app overloads the sql server i could add a method to retrieve data from cache with a lower cost than the one from database (or multiple layered caches, each with different costs)
i could probably add business logi in the mix the same ways as cache, and based on the user location / options present different data
this requires less coding overall, and decouples the data source from the object, making each piece easier to mock/test
what is needed to make this fast is a caching solution for the discovered method chains, since matching hundreds of thousands of methods per model type would be time-consuming but I don't think this is very hard to do - just store all found chains in memory as you find them and some metadata to be able to resume a search from any point in time - when you update the methods, just clear the cache, take a performance hit for the first requests
Thank you for your time
What you describe sounds like a somewhat roundabout way of doing Dependency Injection. Quote:
"Passing the service to the client, rather than allowing a client to
build or find the service, is the fundamental requirement of the
pattern."
Depending on what language you're using, there should be several Dependency Injection frameworks/libraries available.
I have this idea of generating an array of user-links that will depend on user-roles.
The user can be a student or an admin.
What I have in mind is use a foreach loop to generate a list of links that is only available for certain users.
My problem is, I created a helper class called Navigation, but I am so certain that I MUST NOT hard-code the links in there, instead I want that helper class to just read an object sent from somewhere, and then will return the desired navigation array to a page.
Follow up questions, where do you think should i keep the links that will only be available for students, for admins. Should i just keep them in a text-file?
or if it is possible to create a controller that passes an array of links, for example
a method in nav_controller class -> studentLinks(){} that will send an array of links to the helper class, the the helper class will then send it to the view..
Sorry if I'm quite crazy at explaining. Do you have any related resources?
From your description it seems that you are building some education-related system. It would make sense to create implementation in such way, that you can later expand the project. Seems reasonable to expect addition of "lectors" as a role later.
Then again .. I am not sure how extensive your knowledge about MVC design pattern is.
That said, in this situation I would consider two ways to solve this:
View requests current user's status from model layer and, based on the response, requests additional data. Then view uses either admin or user templates and creates the response.
You can either hardcode the specific navigation items in the templates, from which you build the response, or the lit of available navigation items can be a part of the additional information that you requested from model layer.
The downside for this method is, that every time you need, when you need to add another group, you will have to rewrite some (if not all) view classes.
Wrap the structures from model layer in a containment object (the basis of implementation available in this post), which would let you restrict, what data is returned.
When using this approach, the views aways request all the available information from model layer, but some of it will return null, in which case the template would not be applied. To implement this, the list of available navigation items would have to be provided by model layer.
P.S. As you might have noticed from this description, view is not a template and model is not a class.
It really depends on what you're already using and the scale of your project. If you're using a db - stick it there. If you're using xml/json/yaml/whatever - store it in a file with corresponding format. If you have neither - hardcode it. What I mean - avoid using multiple technologies to store data. Also, if the links won't be updated frequently and the users won't be able to customize them I'd hardcode them. There's no point in creating something very complex for the sake of dynamics if the app will be mostly static.
Note that this question doesn't quite fit in stackoverflow. programmers.stackexchange.com would probably be a better fit
We're looking for a way to document Core Data entities. So far the only real options I've come up with are:
Document externally using UML or some other standard
Create NSManagedObject subclasses for every entity and use code comments
Use the User Info dictionary to create a key value pair that holds a string comment
Option 1 feels like too much extra work and something that will almost certainly be out of date 99% of the time.
Option 2 feels natural and more correct than option 1. The biggest con here is that those comments could potentially be lost if this model class is regenerated using Xcode.
Option 3 feels a little less correct than option 2, but has the added advantage of adding automation possibilities with regards to meta data extraction. For instance, in one of our apps we need to keep a real close eye on what we're storing locally on the device as well as syncing to iCloud. Using the user info dictionary it's pretty easy to automate the creation of some form of artefact which can be checked both internally and externally (by the client) for compliance
So my question is whether it would be inappropriate to use the user info dictionary for this purpose? And are there any other options I'm missing?
Option 2 is what I use every time. If you look at your core data model (something.xcdatamodeld or something.xcdatamodel) you will see something like the picture below.
You can tie your entity to whatever class you want and then put the comments in there. It helps if you keep your entity name the same as your class name to make it obvious what you've done.
Additionally this also gives you the ability to add automation. You can do this by creating custom getters and setters (accessor methods) and a custom description method.
I use option 2 and categories. I'll let XCode generate the NSManagedObject subclasses and use a categorie on each of these subclasses. With the categories I do not loose my changes made in the categories, can document, make custom getter and setters and I am still able to use generated subclasses.
If we speak only about documenting (i.e. writing more or less large amounts of text which is intended to be read by humans) your classes, I'd use the option 2.
If you are concerned with the possibility of Xcode overwriting your classes in the option 2, you may consider creating two classes for each entity: one which is generated by Xcode and always could be replaced (you generally do not touch this file) and one other which inherits from the generated one and in which you put all your customizations and comments.
This two-class approach is proposed by the mogenerator.
Although if you need to store some metadata with the entities which will be processed programmatically, the userInfo is perfectly suitable for this.
I'm really confused by this... still.
I asked a similar question to this a while before, but i'll ask it even simpler now.
I see this in a lot of samples and tutorials. How could you put [Bind(Exclude="ID")] on an entire Model, and expect to do Edits on the model? If you get pack all the properties of a model on a POST but not the ID, then how do you know which ID to edit?
Even if i'm using ViewModels... i'm probably creating them without IDs. So in that case... also... how do I know which ID was updated on an Edit?
Yes, i understand that there is a "security" element to this. People can hijack the ID... so we need to keep people from updating the value during a POST. But... what is the correct way to handle edits then? What's common practice?
I feel like i'm missing something VERY trivial.
In MVC requests are processed by the model binder when the client makes a request. If you include models on your controllers then, as far as I'm aware, you actually have to specify the model you wish to bind to by prefixing your arguments with the model name (unless you only have one argument which is the model)
SomeModel_ID
Now, in some cases you might want to exclude certain properties from being bound to because they pose a security risk, which you seem to be happy with as a concept. We will exclude ID on the model, preventing any client request from posting this value in plain text.
Now why might we exclude an entire model? Well not all controller arguments are pre-processed by a model binder. RedirectToAction for example does not pass through the model binder, so it is conceivable in this instance for you to create a new model in a POST controller action, and redirect to a GET controller action, passing along a sanitised model. That model cannot be populated by the client, but we are free to populate it ourselves on the server side.
The only time I bind to a model is when I have a view model and an associated editor for that model. This makes it really easy to inject a common editor into a page and to encapsulate those properties. If you have to exclude certain properties from being bound to I would argue that you are doing it wrong.
Update
Following your comments I think I can see why you might be confused. The model bind excluder prevents the client from ever setting a model property. If you need this property to do your updating then you simply can't exclude it. What this does mean then is that the user could potentially post back any ID. In this case you should check that the user has permission to be modifying any objects or database records associated with this ID before serving the requested update. Validating the arguments is a manual process. You can use data annotations for validating inputs, but this isn't likely to help very much with access permissions. It's something you should be checking for manually at some stage.
You know the ID because it's passed to you through the page address. So:
http://yoursite.com/admin/users/edit/20
Will populate your ID parameter with 20. If it's used in a POST (ie, the information is filled in), just manually fill in the ID field and pass it to the database controller in whatever manner you have devised.
This is also immune to (trivial) hijacks because if they were to write some other ID besides 20, they wouldn't be updating the user with ID 20 now would they? :)