Let's say we are structuring an application with MVC (also, Stores/Services). SQL is used as the persistence mechanism. And memory efficiency is a major concern.
Obviously, we should take advantage of SQL queries and only ask for fields of our Model in theory object when they are needed.
For example, an mobile app may need to display a list of title for articles, while the body of the article doesn't get displayed until user taps on a specific title. In this case, we ask SQL for just the titles first.
The question is, what should the model object look like?
The solutions I can think of are:
Enhance the model with some states that indicate which fields are populated. This could also be archived by using nil/NULL/None values on unpopulated fields of the model object.
Split the theoretical model to multiple classes. Following the previous example, we could have an Article class and an ArticleDetail class, with a one-to-one relation.
Forget the Store object, let each model object lazy evaluate it's costly fields. The model would have to know about its persistence mechanism.
This should be a common problem. How do the ORM in your favorite frameworks/libraries resolve it? Any best practices?
Related
I am in the midst of designing an application following the mvc paradigm. I'm using the sqlalchemy expression language (not the orm), and pyramid if anyone was curious.
So, for a user class, that represents a user on the system, I have several accessor methods for various pieces of data like the avatar_url, name, about, etc. I have a method called getuser which looks up a user in the db(by name or id), retrieves the users row, and encapsulates it with the user class.
However, should I have to make this look-up every-time I create a user class? What if a user is viewing her control panel and wants to change avatars, and sends an xhr; isn't it a waste to have to create a user object, and look up the users row when they wont even be using the data retrieved; but simply want to make a change to subset of the columns? I doubt this lookup is negligible despite indexing because of waiting for i/o correct?
More generally, isn't it inefficient to have to query a database and load all a model class's data to make any change (even small ones)?
I'm thinking I should just create a seperate form class (since every change made is via some form), and have specific form classes inherit them, where these setter methods will be implemented. What do you think?
EX: Class: Form <- Class: Change_password_form <- function: change_usr_pass
I'd really appreciate some advice on creating a proper design;thanks.
SQLAlchemy ORM has some facilities which would simplify your task. It looks like you're having to re-invent quite some wheels already present in the ORM layer: "I have a method called getuser which looks up a user in the db(by name or id), retrieves the users row, and encapsulates it with the user class" - this is what ORM does.
With ORM, you have a Session, which, apart from other things, serves as a cache for ORM objects, so you can avoid loading the same model more than once per transaction. You'll find that you need to load User object to authenticate the request anyway, so not querying the table at all is probably not an option.
You can also configure some attributes to be lazily loaded, so some rarely-needed or bulky properties are only loaded when you access them
You can also configure relationships to be eagerly loaded in a single query, which may save you from doing hundreds of small separate queries. I mean, in your current design, how many queries would the below code initiate:
for user in get_all_users():
print user.get_avatar_uri()
print user.get_name()
print user.get_about()
from your description it sounds like it may require 1 + (num_users*3) queries. With SQLAlchemy ORM you could load everything in a single query.
The conclusion is: fetching a single object from a database by its primary key is a reasonably cheap operation, you should not worry about that unless you're building something the size of facebook. What you should worry about is making hundreds of small separate queries where one larger query would suffice. This is the area where SQLAlchemy ORM is very-very good.
Now, regarding "isn't it a waste to have to create a user object, and look up the users row when they wont even be using the data retrieved; but simply want to make a change to subset of the columns" - I understand you're thinking about something like
class ChangePasswordForm(...):
def _change_password(self, user_id, new_password):
session.execute("UPDATE users ...", user_id, new_password)
def save(self, request):
self._change_password(request['user_id'], request['password'])
versus
class ChangePasswordForm(...):
def save(self, request):
user = getuser(request['user_id'])
user.change_password(request['password'])
The former example will issue just one query, the latter will have to issue a SELECT and build User object, and then to issue an UPDATE. The latter may seem to be "twice more efficient", but in a real application the difference may be negligible. Moreover, often you will need to fetch the object from the database anyway, either to do validation (new password can not be the same as old password), permissions checks (is user Molly allowed to edit the description of Photo #12343?) or logging.
If you think that the difference of doing the extra query is going to be important (millions of users constantly editing their profile pictures) then you probably need to do some profiling and see where the bottlenecks are.
Read up on the SOLID principle, paying particular attention to the S as it answers your question.
Create a single class to perform user existence check, and inject it into any class that requires that functionality.
Also, you need to create a data persistence class to store the user's data, so that the database doesn't have to be queried every time.
The small web application I am working on is becoming bigger and bigger. I've noticed that when posting forms or just calling other functions I've passed parameters that consist of IDs or a whole instance of a Model class.
In a performance stand point, is it better for me to pass the whole Model object (filled with values) or should I pass the ID, then retrieve from the database?
Thanks!
For Performance benefits, you can do lot of things, common things are
1) Fetch as many as records which are needed, e.g. customized paging, in LINQ use (skip and take methods)
2) Use Data caching in controllers and Cache dependencies for Lists which are bound with View
3) Use Compiled query to fetch records. (see here)
Apply all these and see the mark-able page load speed.
EDIt: For IDs recommendations, In this question, Both will be same performance impact if you pass only ID and fetch rest of the model from database OR pass filled model.
Do not solve problems which do not exist yet. Use a tool to measure the performance problem and then try to solve.
It is always best to consider these from the use case.
For example, if I want to get an item by ID, then I pass the ID, not the whole object with the ID filled out.
I use WCF services to host my BLL and interface to my DAL, so passing data around is a costly exercise, so I do it sparingly.
If I need to update an object, I pass the object, if I just want to perform an action on an object, such as delete or get, I use the ID.
Si
I am currently working on a project where we are rewriting software that was originally written in Visual DataFlex and we are changing it to use SQL and rewriting it into a C# client program and a C#/ASP.Net website. The current database for this is really horrible and has just had columns added to table or pipe(|) characters stuck between the cell values when they needed to add new fields. So we have things like a person table with over 200 columns because stuff like 6 lots of (addressline1, addressline2, town, city, country, postcode) columns for storing different addresses (home/postal/accountPostal/ect...).
What we would like to do is restructure the database, but we also need to keep using the current structure so that the original software can still work as well. What I would like to know is would it be possible using Linq to write a DataContext Object Model Class that could sort of interpret the data base structures so that we could continue to use the current database structure, but to the code it could look like we where using the new structure, and then once different modules of the software are rewritten we could change the object model to use the correct data structure???
First of all, since you mention the DataContext I think you're looking at Linq to SQL? I would advice to use the Entity Framework. The Entity Framework has more advanced modeling capabilities that you can use in a scenario as yours. It has the ability to construct for example a type from multiple tables, use inheritance or complex types.
The Entity Framework creates a model for you that consists of three parts.
SSDL which stores how your database looks.
CSDL which stores your model (your objects and the relationships between them)
MSL which tells the Entity Framework how to map from your objects to the database structure.
Using this you can have a legacy database and map this to a Domain Model that's more suited to your needs.
The Entity Framework has the ability to create a starting model from your database (where all tables, columns and associations are mapped) en then you can begin restructuring this model.
These classes are generated as partial so you could extend them by for exampling splitting the database piped fields into separate properties.
Have you also thought about using Views? If possible you could at views to your database that give you a nicer dataschema to work with and then base your model on the views in combination with stored procedures.
Hope this gives you any ideas.
I am wondering how the models in code ignitor are suposed to be used.
Lets say I have a couple of tables in menu items database, and I want to query information for each table in different controllers. Do I make different model classes for each of the tables and layout the functions within them?
Thanks!
Models should contain all the functionality for retrieving and inserting data into your database. A controller will load a model:
$this->load->model('model_name');
The controller then fetches any data needed by the view through the abstract functions defined in your model.
It would be best to create a different model for each table although its is not essential.
You should read up about the MVC design pattern, it is used by codeigniter and many other frameworks because it is efficient and allows code reuse. More info about models can be found in the Codeigniter docs:
http://codeigniter.com/user_guide/general/models.html
CodeIgniter is flexible, and leaves this decision up to you. The user's guide does not say one way or the other how you should organize your code.
That said, to keep your code clean and easy to maintain I would recommend an approach where you try to limit each model to dealing with an individual table, or at least a single database entity. You certainly want to avoid having a single model to handle all of your database tables.
For my taste, CodeIgniter is too flexible here - I'd rather call it vague. A CI "model" has no spec, no interface, it can be things as different as:
An entity domain object, where each instance represents basically a record of a table. Sometimes it's an "anemic" domain object, each property maps directly to a DB column, little behaviour and little or no understanding of objects relationships and "graphs" (say, foreign keys in the DB are just integer ids in PHP). Or it can also be a "rich (or true) domain object", with all the business intelligence, and also knows about relations: say instead of $person->getAccountId() (returns int) we have $person->getAccount(); perhaps also knows how to persist itself (and perhaps also the full graph or related object - perhaps some notion of "dirtiness").
A service object, related to objects persistence and/or general DB querying: be a DataMapper, a DAO, etc. In this case we have typically one single instance (singleton) of the object (little or no state), typically one per DB table or per domain class.
When you read, in CI docs or forums, about , say, the Person model you can never know what kind of patter we are dealing with. Worse: frequently it's a ungly mix of those fundamentally different patterns.
This informality/vagueness is not specific to CI, rather to PHP frameworks, in my experience.
I have a web application built on an MVC design.
I have a database which contains a large number of objects (forum threads) which I can't load into memory at once. I now want to display (part of) this collection with different filters in effect (kinda like what stackoverflow does with questions sorted by date, votes, tags etc).
Where do I implement the filtering logic? It seems to me that this must go into the model part of the application, as only models interact with the database (in my implementation). If I make the filtering a part of the view, then the view must access the database directly to get the list of filtered objects, right? I'd like to avoid this, because it exposes the database layout to the view. But at the same time, displaying different views of the same data should be implemented in the view part of the application, as they are just that -- different views of the same data.
So how do I resolve this? Do I create an additional model, say, FilteredThreadsList, and have it remember the filter to use, and then use a FilteredView to display the list of threads that FilteredThreadsList spits out?
Or do I have to build a ThreadQueryier that allows views to query the database for certain thread objects, so I can have the filtering logic in a view without exposing the database backend?
You should never query data from the view. I don't know what framework you are using in particular but as for Ruby on Rails (should be the same for other frameworks) we always pull the necessary data from the controller and store all that information into a variable. The variable will be accessed by the view which can help you avoid querying your database directly from the view.If the code to query the database gets too lengthy in the controller, insert that code into the model instead so it's more maintainable for your project in the future. Additionally, you can call this model method from multiple places in your application if needed. Good luck!
From an architectural point of view, the model should be having the code for filtering. This is so, because in many applications the code for filtering is not trivial and has a good amount of domain logic in it. (Think of filtering top gainers from a list of stocks). From your example as well, it looks the same since you might want to filter by vote or by date or by tags and then by answered or unanswered etc.
In some very simple applications that deal with search/list of entities and allows Create/Read/Update/Delete of an entity, the pagination, sorting and filtering logic is usually very generic and can be implemented in a controller base class that is inherited by all entity-specific controller classes.
The bottom line is this: if your filtering logic is generic put it in the controller else put it in the model.
Model, that's only bunch of entities.
View provides a visual representation of the data from model - use as much of views as you want. If your application is web based, you can fetch data into browser just once (AJAX) using and re-use them for different UI components rendered in the browser.
As for what entities and what view to use for their representation, I think it's work of Controller. If you need some support for it on "model layer", add it but avoid tight coupling.