The simplest-possible cache works like this:
It makes sure it has got enough memory to hold the entire data structure to be cached, which lives on some slower medium.
It eagerly copies the entire data structure to memory.
It serves all requests from memory, without caring about cache misses and thus without having to access the original data structure any more.
Is there a common name for this trivial kind of cache?
I've searched for terms like "mirroring cache", "full cache", "eager cache", and "memory proxy", but didn't find anything.
Related
On the Wikipedia page about cache replacement policies, there is a small section about LIFO/FILO policy:
Last in first out (LIFO) or First in last out (FILO)
Using this algorithm the cache behaves in the same way as a stack and opposite way as a FIFO queue. The cache evicts the block added most recently first without any regard to how often or how many times it was accessed before.
I tried to look a bit for an application of this policy but didn't find any example. In my opinion if you discard the most recently added entry, then it defeats the purpose of caching. When there is a cache miss, you'll fetch the data, save it in the cache but it will likely be the first one discarded on the next cache miss, so why did we cache it at all? The only reason I see is that each entry will be likely fetched only one, but then why implement caching then?
An excerpt from Wiki on Cache invalidation -
"Cache invalidation is a process in a computer system whereby entries in a cache are replaced or removed." But, why on earth do we need to invalidate Cache?
I can think of only possible scenario -
If for some reason cache and the database go out of sync, the data in cache will be stale. To sync it, we will need to invalidate cache. But, the cache and DB going of sync(except for a short period of time when the data is yet to be written into both) is not a desirable behaviour. So, cache invalidation acts as a remedy if we discover that the cache does not contain the correct data. Is this its sole purpose?
Cache invalidation exists because most caches operate based upon a trade-off of performance vs capacity.
Consider a solid state drive vs a hard drive. The performance of the SSD will be better but the amount of data you can store will be worse at the same cost level. Often people will combine them to get the performance of an SSD for frequently accessed files (such as the operating system), and a HDD for raw storage capacity.
CPUs are structured in a similar hierarchy, where the closest to the CPU is the fastest but also the smallest. The costs in this case are not necessarily just monetary cost but also physical space, power usage, heat production etc.
CPU registers - fastest, very small
CPU caches (also have their own hierarchy) - fast, small
RAM - medium, large
To keep the caches performing at their best, the most frequently accessed items must be maintained so that there is a better ratio of cache hits to misses. We want to be fetching from our slower sources as infrequently as possible. Similarly, because of the limited size constraint, we need to evict the items which are accessed least frequently.
Cache invalidation is the strategy which we will utilise in order to decide which items to evict and when, in order to make space for newer items which have a higher likelihood of being required again. It is not applicable if your cache contains a full representation of some other data source.
There are plenty of reasons. Probably one of the the most common ones: a cache is (often by nature) much smaller compared to the overall amount of data that needs to be stored.
In other words: if you just keep adding and adding elements to your cache, it becomes a full copy of your data. Respectively, you run out of memory quickly.
In other words: the nature of a cache is this: it is limited (somehow) in size. Thus, sooner or later you are facing a decision like: "I can't just add a new element to the cache, I have to make room first". And then you have to do exactly that: invalidate one of the entries in your cache so that there is room for that "newer" entry.
And given the comment by the OP: often invalidating a whole cache is seen similar to "restart" your program, or "re-install your app", or "restart your device". It is often seen as "generic" mean to ensure the program/application gets reset to a known good state.
Is there any benefit to use cache if there is read miss for every access?
My question aims at a better understanding of caches.
Read miss for every access can also happen during cold start, am I right?
If you are doing only reads and you miss all levels of cache on every read, then by definition caches didn't help. You paid extra in power and latency to check each level of cache, and extra power and (probably) latency to load a whole cache line of data which will never be used (by definition) except for the data you read since all your accesses miss.
You didn't say you are doing only reads though. So of course caches can help in the case that your reads all miss but some of your writes hit.
Perhaps you meant that the read-portion of all accesses, both reads and writes, misses - where the read-portion of a write is the read-for-ownership access present on most cache-based systems where a cache-line must be read into cache before (part of it) can be written. In that case, the cache probably also didn't help and probably hurt.
Read miss for every access can also happen during cold start, am I right?
No, almost never. Some reads will miss, but those will bring in adjacent data on the same cache line which will often result in later hits (spatial locality). Many reads will go back to the same location even in a cold start (temporal locality) which will also often hit. Even beyond the dynamics of a single cache line, modern CPUs often offer hardware prefetching which will recognize certain access patterns and will bring in data before you need it which can result in hits even the first time you access a cache line.
Finally, on most general purpose hardware there you usually cannot decide to simply "not use the caches" so as a practical matter you pay the built-in costs of caching even if your hit rate is low.
That said, sometimes, when you know your access pattern you can provide hints to the CPU. For example, x86 CPUs provide "non-temporal store" instructions which essentially bypass the caches when used - meaning that the stored cache line won't be cached. This is useful not necessarily to speed up the store itself (which still largely pays the price of the cache hierarchy which is baked into the hardware), but to avoid polluting the cache with data the developer knows will not soon be accessed.
I'm using perf as basic event counter. I'm working on a program which suffers from data cache store misses. Which as as high as ratio of %80.
I know how caches in principle work. It loads from memory on various miss cases, removes data from cache when it pleases. What I don't understand is , what is difference between store - load misses. How does it differ loading and storing. How can you store-miss ?
A load-miss (as you know) is referring to when the processor needs to fetch data from main memory, but data does not exist in the cache. So whenever the processor wants some data from the main memory, it esquires the cache, and if the data is already loaded you get a load-hit and otherwise you get a load-miss.
A store-miss is related to when the processor wants to write back the newly calculated data to the main memory.When it wants to write-back the data to the main memory, it hasto make sure that the content of the cache and main memory are in sync with each other. It can happen with two different policies that you can find here: Writing Policies.
So no matter what policy you choose, you first need to check whether the data is already in the cache so you can store it to cache first (since it's faster), and if the data block you are looking for has been evicted from the cache, you get a store-miss related to that cache.
You can check the applet here, to get a better idea of what happens in different scenarios.
I'm not fully familiar with how perf define these events, but given the common definition I believe load/store miss is just a way to break down the overall miss rate counting, so that you may tell which accesses miss more often. Note that loads are usually performed speculatively (at least in modern x86 cpus), while stores are performed much later along the pipeline, after the commit point, so even a piece of code with both loads and stores to the same region can have different miss rates.
In MESI-based cache protocols a load would hit the cache, or miss and fetch the line from the memory or next cache levels, either exclusively if it's not owned by anyone else, or in a shared state if it is. It would write the data to the caches along the way in the process.
A store would fetch a line in the same manner, but use an RFO (read-for-ownership) request which grants it exclusive ownership and the right to modify the line. The line would still get cached, but once the new data is written to it locally (usually in your L1 cache), it would become modified. The hit/miss process would look the same though.
What Saman referred to in his answer is the breakdown between reads and writes. Loads and stores (and other forms of access like code-read) all form the "read" part, and writebacks (or intentional write-throughs using special command or mem types like uncacheable) form the "write part.
I want to cache data on the client. What is the best algorithm/data structure that can be employed?
Case 1. The data to be stored requires extremely fast string searching capability.
Case 2. The cached data set can be large. I don't want to explode the client's memory usage and also I don't want to make a network and disk access calls which slows down my processing time on the client side
Solutions:
Case 1: I think suffix tree/Tries provides you with a good solution in this case.
Case 2: The two problems to consider here are:
To store large data with minimum memory consumption
Not to make any network calls to access any data which is not available in the cache.
LRU caching model is one solution I can think of but that does not prevent me from bloating the memory.
Is there any way to write down to a file and access without compromising the data (security aspect)?
Let me know if any point is not clear.
EDIT:
Josh, I know my requirements are non-realistic. To narrow down my requirement, I am looking for something which stores using LRU algorithm. It will be good if we can have dynamic size configuration for this LRU with a maximum limit to it. This will reduce the number of calls going to the network/database and provide a good performance as well.
If this LRU algorithm works on a compressed data which can be interpreted with a slight overhead (but less than a network call), it will be much better.
Check out all the available caching frameworks/libraries - I've found Ehcache to be very useful. You can also have it keep just some (most recent) in memory and failover to disk at a specified memory usage. The disk calls will still be a lot faster then network calls and you avoid taking all the memory.
Ehcache
Unfortunately, I think your expectations are unrealistic.
Keeping memory usage small, but also not making disk access calls means that you have nowhere to store the data.
Furthermore, to answer your question about security, there is no client side data storage (assuming you are talking about a web-application) that is "secure". You could encrypt it, but this will destroy your speed requirements as well as require server-side processing. Everything stored at and sent from the client is suspect.
Perhaps if you could describe the problem in greater detail we can suggest some realistic solutions.