I am new to caching
What should I cache
eg. Do I cache user info? eg. since they are frequently used throughout the application (like in the header saying "welcome {username}")?
But most things should be used quite frequently anyways? eg. Users have projects. These projects don't belong to everyone, but they will be frequently used by specific users do I cache them too? Won't I be caching nearly everything then?
Also regarding CRUD, with doctrine queries, I can just use $query->useResultCache(true) but what happens when I update/delete an entity? I need to somehow update my cache too? how?
The basic principle of caching is to hold frequently used data that doesnt change often in memory to reduce database work.
Its more convenient to use the php session variables to hold basic things like username.
In case of projects, if they dont change often, and retrieved by users frequently, it would be a good idea to cache them. How long a project info stays cached depends on the change frequency.
Also note that if the info you present to users is vital or time important, you should use caching cautiously.
Check this reference page for basic information on caching http://www.doctrine-project.org/docs/orm/2.0/en/reference/dql-doctrine-query-language.html#cache-related-api
Or check http://www.doctrine-project.org/docs/orm/2.0/en/reference/caching.html for detailed explanation.
Related
Correct me if I'm wrong, but from my understanding, "database caches" are usually implemented with an in-memory database that is local to the web server (same machine as the web server). Also, these "database caches" store the actual results of queries. I have also read up on the multiple caching strategies like - Cache Aside, Read Through, Write Through, Write Behind, Write Around.
For some context, the Write Through strategy looks like this:
and the Cache Aside strategy looks like this:
I believe that the "Application" refers to a backend server with a REST API.
My first question is, in the Write Through strategy (application writes to cache, cache then writes to database), how does this work? From my understanding, the most commonly used database caches are Redis or Memcached - which are just key-value stores. Suppose you have a relational database as the main database, how are these key-value stores going to write back to the relational database? Do these strategies only apply if your main database is also a key-value store?
In a Write Through (or Read Through) strategy, the cache sits in between the application and the database. How does that even work? How do you get the cache to talk to the database server? From my understanding, the web server (the application) is always the one facilitating the communication between the cache and the main database - which is basically a Cache Aside strategy. Unless Redis has some kind of functionality that allows it to talk to another database, I don't quite understand how this works.
Isn't it possible to mix and match caching strategies? From how I see it, Cache Aside and Read Through are caching strategies for application reads (user wants to read data), while Write Through and Write Behind are caching strategies for application writes (user wants to write data). Couldn't you have a strategy that uses both Cache Aside and Write Through? Why do most articles always seem to portray them as independent strategies?
What happens if you have a cluster of webs servers? Do they each have their own local in-memory database that acts as a cache?
Could you implement a cache using a normal (not in-memory) database? I suppose this would still be somewhat useful since you do not need to make an additional network hop to the database server (since the cache lives on the same machine as the web server)?
Introduction & clarification
I guess you have one misunderstood point, that the cache is NOT expclicitely stored on the same server as the werbserver. Sometimes, not even the database is sperated on it's own server from the webserver. If you think of APIs, like HTTP REST APIs, you can use caching to not spend too many resources on database connections & queries. Generally, you want to use as few database connections & queries as possible. Now imagine the following setting:
You have a werbserver who serves your application and a REST API, which is used by the webserver to work with some resources. Those resources come from a database (lets say a relational database) which is also stored on the same server. Now there is one endpoint which serves e.g. a list of posts (like blog-posts). Every user can fetch all posts (to make it simple in this example). Now we have a case where one can say that this API request could be cached, to not let all users always trigger the database, just to query the same resources (via the REST API) over and over again. Here comes caching. Redis is one of many tools which can be used for caching. Since redis is a simple in-memory key-value storage, you can just put all of your posts (remember the REST API) after the first DB-query, into the cache. All future requests for the posts-list would first check whether the posts are alreay cached or not. If they are, the API will return the cache-content for this specific request.
This is one simple example to show off, what caching can be used for.
Answers on your question
My first question is, why would you ever write to a cache?
To reduce the amount of database connections and queries.
how is writing to these key-value stores going to help with updating the relational database?
It does not help you with updating, but instead it helps you with spending less resources. It also helps you in terms of "temporary backing up" some data - but that only as a very little side effect. For this, out there are more attractive solutions (Since redis is also not persistent by default. But it supports persistence.)
Do these cache writing strategies only apply if your main database is also a key-value store?
No, it is not important which database you use. Whether it's a NoSQL or SQL DB. It strongly depends on what you want to cache and how the database and it's tables are set up. Do you have frequent changes in your recources? Do resources get updated manually or only on user-initiated actions? Those are questions, leading you to the right caching implementation.
Isn't it possible to mix and match caching strategies?
I am not an expert at caching strategies, but let me try:
I guess it is possible but it also, highly depends on what you are doing in your DB and what kind of application you have. I guess if you find out what kind of application you are building up, then you will know, what strategy you have to use - i guess it is also not recommended to mix those strategies up, because those strategies are coupled to your application type - in other words: It will not work out pretty well.
What happens if you have a cluster of webs servers? Do they each have their own local in-memory database that acts as a cache?
I guess that both is possible. Usually you have one database, maybe clustered or synchronized with copies, to which your webservers (e.g. REST APIs) make their requests. Then whether each of you API servers would have it's own cache, to not query the database at all (in cloud-based applications your database is also maybe on another separated server - so another "hop" in terms of networking). OR (what i also can imagine) you have another middleware between your APIs (clusterd up) and your DB (maybe also clustered up) - but i guess that no one would do that because of the network traffic. It would result in a higher response-time, what you usually want to prevent.
Could you implement a cache using a normal (not in-memory) database?
Yes you could, but it would be way slower. A machine can access in-memory data faster then building up another (local) connection to a database and query your cached entries. Also, because your database has to write the entries into files on your machine, to persist the data.
Conclusion
All in all, it is all about being fast in terms of response times and to prevent much network traffic. I hope that i could help you out a little bit.
I have a project that I am working where data cannot be cached on the client of certain entity types. Some are Ok, and others are not.
I don't see anything in the documentation or the API that references NOT caching an entity of a particular type, so is there a way to do this to prevent secure information from being cached?
What have I tried?
Nothing... I don't see a way to do this...
Edit
As an alternative, if anyone from IdeaBlade could lend a hand in explaining how and where the caching is saving info and where (if anywhere) that information is persisted it may be helpful in alleviating the need to NOT cache.
Edit 2
Ok no answer, yet, I changed the title -
How can I ensure no sensitive data is persisted in the local memory after the browser has been shut down?
There are two questions here.
How to keep Breeze from holding entities of a type in its in-memory cache.
How prevent caching of entities across browser sessions.
Number 2 first: Breeze in memory cache evaporates when the browser session closes and is not shared across browser sessions. You can arrange for local persistence but that's a conscious decision and some small effort. I don't think you have to worry about that.
If you want to prevent a type from being cached in memory at all (really? Why?) you could write a simple JsonResultsAdapter that zaps the $type parameter for nodes of this entity type and let's the others pass through to the default adapter. Breeze only makes cachable entities out of nodes with a recognized $type. Look that adapter up in the doc.
I'm looking at the cache function in the findAll function of cfWheels. I'm a little apprehensive about using it. My queries are not taking that long that I absolutely need them, but a bit of a speed boost is always welcome. I'm getting 10ms from a queried cache that otherwise takes about 100ms. The thing I'm wondering about is when an entry changes, I'd like the cache to be cleared on the next run. It doesn't seem like there's any mechanism or flag in the framework that would allow that, so I'd have to set and clear the flags myself, which would most likely end up having to read from the database anyway. I was hoping that I could set the cache for a full day and update when needed, is this horribly misguided? I'm most likely not going to go down the road of developing any of the functionality to allow the caching for this application, but am curious if it is worth while revisiting.
More precisely whenever you make a new entry in the database, use the cfhttp tag to reload the application.
Caching can be cleared through reloading an application. It would not be the answer you are seeking but is a solution, here is an another approach. You can reload an application through <CFHTTP> by sending URL through <CFHTTP> after adding your new database record. If you are adding record through management site, then you can reload your Public site using <cfhttp>.
:)
I have an application that was built using Fusebox 4 with ColdFusion. Can anyone recommend a good caching solution, that is a plugin, which works directly with this older version of the framework?
Another idea I've been tinkering around with is to take the most commonly used queries in the system and applying cachedWithin. The value would be a variable stored in the application scope. Basically anytime we update any of the most commonly accessed tables in the db, we update the application.cachedwithin variable as well. So whenever these tables are updated the data is refreshed. Anything else that isn't used frequently will simply query the DB to get the content.
Also to add to this very simple caching methodology would be to simply store strings, or other frequently used content, directly within the application scope.
This bulk of this application is around 30 pages, comprised of approximately 200 products. So its quite a small website.
Can anyone recommend a good Fusebox 4 cache plugin or confirm if this simple caching methodology is a good idea? If not, could you recommend a simple alternative?
thanks in advance
I would suggest you to use cfcache to store all pages output into statistics HTML files.
Then on any update, you can clear the cache of the updated pages or all the cache:
<cfcache action="flush" />
<cfobjectcache action="clear" />
make sure to disable the urlSessionFormat() in URL.
I'm not sure that you even need to be caching given the size of the site, unless you are getting a huge amount of traffic. If you are currently having performance problems, the first thing to do is make sure that Fusebox is in production mode, so that it isn't recreating the parsed files on each request.
Caching the queries should certainly aid performance - how long are the queries currently taking to execute? With Fusebox 4, it can be problematic to have "Report execution times" turned on in CF when debugging, as it can significantly affect the time the request takes to execute.
I have developed application using ZF.The app is little big with a lots of features.
I use Zend_Application(already using autoloader in constructor),Zend_Layout,Zend_view,Zend_form,etc. My current issue is, the page loading is very slow and that too in localhost with XAMP.
I have enabled xdebug, to investigate the issue, got a cachegrind file in "tmp" folder and tried to view it with WinCachegrind software. There i can a see a lot of processes and functions being run for each and every request or page load.
Also, i have installed YSlow add-on for firefox and observed the speed of page loads in seconds...I have compare the speed with ZF and non ZF applications. And from the comparison, the pages for non zf app takes less than 1 sec to load and for the ZF app, it takes atleast 6-7 seconds. What a huge difference.
Main Things happen in the app are :
1) Database connection happens for each request.
2) Im not adding the view to layout explicitly,ZF just appends it automatically, to layout.phtml, based on the action name.
3) Some windows have forms with few drop down boxes which fetches data from the database.
4) Have menus with ACL implimented, before it was loading the privilges from DB for each and every request, but now i have optimized it, so that it will work only duiring the login and rest of the time it will take from the Zend_Registry.
I would like to attach the cachegrind file so that some one can see whats happening in the background, but i cant see an option here for attaching.
Someone please help me to find a solution for this. Any kind of help is really appreciated. Thanks a lot
Let's try to give some hints.
First database connection should happen only once (except if you use several privileges access on the database or several databases). So check that you use Singleton patterns with you Zend_Db_Tables object
Then you do not use Zend_Cache. You should really start to use Zend_Cache and build several cache objects. Let's say for example a File cach, with long term storage, and a memcache or Apc Cache, storing objects. Then use these cache in several layers:
gives the FileCache to Zend_Db_Table (defaultMetaDataCache), this way you will avoid a loot of metadata queries, queries that ask for description of each columns of the tables you use.
Store one or more Acl object (depends on how you use Acl, if you have one big Acl with all rules or several with subsets). And store them in mid-duration caches when they are built.
Think of other usages, detect heavy loops, semi-static contents (like you select lists, how many time should they be considered static?)
Finally, get a whole mental image of how your application engine works, and how your data will grow and be used.You will need that step to use application levels caches in the very best way (for example should some elements be cached for groups of users?, should Acl objects be build for groups, for each user, for everybody, is ther some blocks in the layout that should be rendered the same for everybody?).