Is there a standard way to calculate TTL for different types of cache? this's more of a generic question so lets assume we're designing a system from scratch and we have the following requirements/specs:
static resources served by CDNs are rarely updated e.g.(privacy
policy, about, images and maps)
application cache is used to
serve a- sessions b- recently used reads regardless of the type
client side cache (previously requested files), as well as lets say
images or posts a client can see (something similar to
Instagram/twitter in this case)
Calculate TTL for the following types based on the little to no information provided above:
Client cache
CDN
Webserver cache (used for media)
Application caache (sessions and recent reads of some data)
TTLs are mostly defined using historical data, use cases, and experience. There are no predefined rules/theories that tell you about the cache expiry. Cache TTL should have some tolerance like if you set TTL too high then you might see expired(stale) data, what's the impact of stale data in your application? In some cases, stale data is not accepted at all but in other cases, it's ok to use stale data for SOME TIME.
Still, you'll observe each caching system has some predefined TTL for example AWS CDN has 24 hours expiry, Google CDN has 1 hour. Etag is another thing, that's used in CDN.
CDN can catch data for a week but depending on the data some data can change hourly as well so in that case expiry is set to a lower value, similar things apply to other use cases.
The session should be cached for a week or so, but some applications cache the session for a longer period. Of course, there're pros and cons of using low/high TTL.
Application data cache has similar characters as CDN data, the data can change any time and change must reflect in the cache. Again depending on the use case the TTL should be used, my experiences say you can cache some data for one day or one week but some data can not be cached for more than 15 minutes since it might get updated within 15 minutes.
Depending on the nature of the data you can always find some optimal TTL, finding optimal TTL takes time as you would have to monitor cache hit/miss and stale data ratio.
Refers
https://martinfowler.com/bliki/TwoHardThings.html
https://www.stevesouders.com/blog/2012/03/22/cache-them-if-you-can/
I tried googling something similar , but wasn't habel to find something on the topic
I'm just curious, does it matter how big the number of seconds are set in a key impact performance in redis?
For example:
set mykey "foobarValue" EX 100 VS set mykey "foobarValue" EX 2592000
To answer this question, we need to see how Redis works.
Redis maintains tables of a key, value pair with an expiry time, so each entry can be translated to
<Key: <Value, Expiry> >
There can be other metadata associated with this as well. During GET, SET, DEL, EXPIRE etc operations Redis calculates the hash of the given key(s) and tries to perform the operation. Since it's a hash table, it needs to prob during any operation, while probing it may encounter some expired keys. If you have subscribed for "Keyspace notification" then notification would be sent and the given entry is removed/updated based on the operation being performed. It also does rehashing, during rehashing it might find expired keys as well. Redis also runs background tasks to cleanup expire keys, that means if TTL is too small then more keys would be expired, as this process is random, so more event would be generated.
https://github.com/antirez/redis/blob/a92921da135e38eedd89138e15fe9fd1ffdd9b48/src/expire.c#L98
It does have a small performance issue when TTL is small since it needs to free the memory and fix some pointers. But it can so happen that you're running out of memory since expired keys are also present in the database. Similarly, if you use higher expiry time then the given key would present in the system for a longer time, that can create memory issue.
Setting smaller TTL has also more cache miss for the client application, so client will have performance issues as well.
I am using Community-Auth with Codeingniter V3 to do authentication and to store authorization levels, etc.
The problem I am having is that my users are sometimes being redirected to the login page, even though they have not been inactive. I cannot seem to isolate a particular behavior or pattern to duplicate the problem.
The problem occurs when a controller calls the verify_min_level routine which should just verify that they are logged on. But it returns FALSE, which means Community-Auth believes they are not logged in, and the code redirects to the login screen.
Since it seems to happen randomly and for no apparent reason (the user was not inactive for a while, etc) it is driving my users crazy.
Has anyone else seen this kind of behavior?
I seem to have identified the problem. This particular client wanted sessions that would only end when they logged out or closed their browser window. So I set the session expiration to zero (0).
I thought that the garbage collection would only delete sessions occasionally (given that in codeigniter I understand that 0 means the session ends in two years) and that I would catch up with it with my own garbage collection. However I started noticing that the ci_sessions table (I moved session data to database from file system to help debug this issue) would have multiple sessions removed frequently, even though none of the sessions were anywhere near two years old.
What seems to have solved the problem is to turn off the garbage collection completely by setting the PHP parameter sessions.gc_probability to 0.
No garbage collection, no premature deletion of session variables.
I am implementing a nightly CRON job to do garbage collection of the ci_sessions table.
The default setting on an application I am working on was 30 mins. I would like to increase to an hour and half, 2 hours. What are the drawbacks for doing this?
Thanks in advance for your answers.
The big issue would be memory, but it's not necessarily a significant one. With a long timeout if you have a lot of sessions and a lot of data in each, then you'll use more memory than you should because without an explicit logout or other action that ends the session when it is no longer needed, it will take 90-120 minutes to free up that memory that's not being used.
But all of that is only a problem if you store a lot of data in the session which is in general a bad idea, since even 30 minutes is a long time to wait to reclaim unused session data. Often data that is stored in the session causes data staleness issues, e.g. storing domain class instances means that they're detached from their original Hibernate session and must be reattached. But that just reloads the instance from the database using the instance's id, so storing the id would have been sufficient. Usually you'll find that there are much more efficient caches than the HTTP session.
I was asked this question in an interview:
For a high traffic website, there is a method (say getItems()) that gets called frequently. To prevent going to the DB each time, the result is cached. However, thousands of users may be trying to access the cache at the same time, and so locking the resource would not be a good idea, because if the cache has expired, the call is made to the DB, and all the users would have to wait for the DB to respond. What would be a good strategy to deal with this situation so that users don't have to wait?
I figure this is a pretty common scenario for most high-traffic sites these days, but I don't have the experience dealing with these problems--I have experience working with millions of records, but not millions of users.
How can I go about learning the basics used by high-traffic sites so that I can be more confident in future interviews? Normally I would start a side project to learn some new technology, but it's not possible to build out a high-traffic site on the side :)
The problem you were asked on the interview is the so-called Cache miss-storm - a scenario in which a lot of users trigger regeneration of the cache, hitting in this way the DB.
To prevent this, first you have to set soft and hard expiration date. Lets say the hard expiration date is 1 day, and the soft 1 hour. The hard is one actually set in the cache server, the soft is in the cache value itself (or in another key in the cache server). The application reads from cache, sees that the soft time has expired, set the soft time 1 hour ahead and hits the database. In this way the next request will see the already updated time and won't trigger the cache update - it will possibly read stale data, but the data itself will be in the process of regeneration.
Next point is: you should have procedure for cache warm-up, e.g. instead of user triggering cache update, a process in your application to pre-populate the new data.
The worst case scenario is e.g. restarting the cache server, when you don't have any data. In this case you should fill cache as fast as possible and there's where a warm-up procedure may play vital role. Even if you don't have a value in the cache, it would be a good strategy to "lock" the cache (mark it as being updated), allow only one query to the database, and handle in the application by requesting the resource again after a given timeout
You could probably be better of using some distributed cache repository, as memcached, or others depending your access pattern.
You could use the Cache implementation of Google's Guava library if you want to store the values inside the application.
From the coding point of view, you would need something like
public V get(K key){
V value = map.get(key);
if (value == null) {
synchronized(mutex){
value = map.get(key);
if (value == null) {
value = db.fetch(key);
map.put(key, value);
}
}
}
return value;
}
where the map is a ConcurrentMap and the mutex is just
private static Object mutex = new Object();
In this way, you will have just one request to the db per missing key.
Hope it helps! (and don't store null's, you could create a tombstone value instead!)
Cache miss-storm or Cache Stampede Effect, is the burst of requests to the backend when cache invalidates.
All high concurrent websites I've dealt with used some kind of caching front-end. Bein Varnish or Nginx, they all have microcaching and stampede effect suppression.
Just google for Nginx micro-caching, or Varnish stampede effect, you'll find plenty of real world examples and solutions for this sort of problem.
All boils down to whether or not you'll allow requests pass through cache to reach backend when it's in Updating or Expired state.
Usually it's possible to actively refresh cache, holding all requests to the updating entry, and then serve them from cache.
But, there is ALWAYS the question "What kind of data are you supposed to be caching or not", because, you see, if it is just plain text article, which get an edit/update, delaying cache update is not as problematic than if your data should be exactly shown on thousands of displays (real-time gaming, financial services, and so on).
So, the correct answer is, microcache, suppression of stampede effect/cache miss storm, and of course, knowing which data to cache when, how and why.
It is worse to consider particular data type for caching only if data consumers are ready for getting stale date (in reasonable bounds).
In such case you could define invalidation/eviction/update policy to keep you data up-to-date (in business meaning).
On update you just replace data item in cache and all new requests will be responsed with new data
Example: Stocks info system. If you do not need real-time price info it is reasonable to keep in cache stock and update it every X mils/secs with expensive remote call.
Do you really need to expire the cache. Can you have an incremental update mechanism using which you can always increment the data periodically so that you do not have to expire your data but keep on refreshing it periodically.
Secondly, if you want to prevent too many users from hiting the db in one go, you can have a locking mechanism in your stored proc (if your db supports it) that prevents too many people hitting the db at the same time. Also, you can have a caching mechanism in your db so that if someone is asking for the exact same data from the db again, you can always return a cached value
Some applications also use a third service layer between the application and the database to protect the database from this scenario. The service layer ensures that you do not have the cache miss storm in the db
The answer is to never expire the Cache and have a background process update cache periodically. This avoids the wait and the cache-miss storms, but then why use cache in this scenario?
If your app will crash with a "Cache miss" scenario, then you need to rethink your app and what is cache verses needed In-Memory data. For me, I would use an In Memory database that gets updated when data is changed or periodically, not a Cache at all and avoid the aforementioned scenario.