Does the Linq2DB ORM support query result caching - first level or second level? I noticed the namespace LinqToDB.Common.Internal.Cache, does it mean caching has to be built by the consuming application through a custom caching manager?
Well, linq2db is designed to work with queries not object management. Caching is a very specific case which introduces a lot of side effects. For example if you change some field and the cache contains such records - you have to invalidate the cache, otherwise the system will return obsolete data. Invalidating caches is the most difficult part here.
Anyway there are third party libraries which can do that for you:
LinqCache
Probably there are other libraries which can do similar things.
Related
I am developing some SPA with a backend written in Java (Spring Boot). In relational DB that backend connects to, there is a table with some dictionary values. Values can edited by users of the app, but it's done really, really rarely (almost never).
Those dictionary values are used in a lot of pages on UI and because of that I would like to "cache" them in a way. What I want to achieve is that I want to load dictionary values on startup to avoid asking DB for values during every request between UI and Backend.
Firstly, I thought about just loading it on the UI part of the app, when user enters the page for the first time. Then I ruled it out, since when one of the users changes the values, it should be reloaded.
What I think might work is just loading them on startup of Backend into some collection (that can be safely used in concurent environment, probably ConcurrentMap) and then during some GET requests asking that collection for the values (instead of DB). When the values are changed, that request just updates the DB table and reloads them into collection.
Then I thought that the collection solution won't be enough, when my backend would be scaled up to more than one instance. In that case, only one of instances will be updated and the second one will provide outdated data. We can avoid it and force refreshes i.e. every 15 minutes (instead of on demand during values update).
But what I think is the best solution is to start some redis service on a side, load dictionary values into it and after every DB update of the values just update the redis instance with the new ones. Every instance of backend would use the same instance of redis, which seems quicker than executing query (select * from _ where _ = _) on DB.
What do you think? Is my thought process is correct? Do you have any ideas that can help solve my issue?
If you are using Spring you could check out Spring Cache Abstraction. That way your cache will be up-to-date whenever some change occurs.
Out of the box few implementations are supported by Spring:
Spring provides a few implementations of that abstraction: JDK java.util.concurrent.ConcurrentMap based caches, Ehcache 2.x, Gemfire cache, Caffeine, and JSR-107 compliant caches (such as Ehcache 3.x). See Plugging-in Different Back-end Caches for more information on plugging in other cache stores and providers.
If you decide to use Memcached implementation you can check out this library (uses Xmemcached under the hood) here.
You could also check a small demo app of how to use Spring Cache Abstraction in your project (link).
I think your in the right path with your approach in terms of 'caching'. I suggest you also check Memcached for it simplicity. Redis is a good choice but still it depends on your requirements and if you need that much feature. just my 2cent
https://aws.amazon.com/elasticache/redis-vs-memcached/
https://devcenter.heroku.com/articles/spring-boot-memcache#add-caching-to-spring-boot
Thanks,
I want to use GraphQL to access data from different data sources (CSV, SQL Server, Web Server) . I want to know if caching mechanism is supported, so that when the connection is lost the data is still available? I see the data loader which is data batching to increase the performance of the query, but I do not know how data loader can do caching?
Thanks.
There's no built-in support, but you need to remember that graphql-js (which I assume you're using) isn't a framework, it's just a library. You'd implement caching exactly the same as you'd do it with anything else - by surrounding your data-fetching functions with caching get/set behaviour.
I don't use caching very heavily with graphql-js, but if you've ever implemented caching before, it's exactly the same principle.
I want to disable doctrine's default caching for a specific function(page), on all other pages, it should work as usual. Also I don't even want to clear the current caching inside that function. Just need that, no caching will be considered for that particular function call and its inside workflow.
Is there any easy way to achieve this? Thanks.
Rana, I believe you can use $query->useResultCache(false); to disable the cache for the page you want. Take a look at the documentation if needed.
Cheers,
You could use another entity manager for that specific page with caching turned off. As you didn't mention what kind of framework you use, I am unable to make any further assumptions.
According to the documentation (and personal experience) you shouldn't use doctrine without a cache:
Do not use Doctrine without a metadata and query cache! Doctrine is optimized for working with caches. The main parts in Doctrine that are optimized for caching are the metadata mapping information with the metadata cache and the DQL to SQL conversions with the query cache. These 2 caches require only an absolute minimum of memory yet they heavily improve the runtime performance of Doctrine. The recommended cache driver to use with Doctrine is APC. APC provides you with an opcode-cache (which is highly recommended anyway) and a very fast in-memory cache storage that you can use for the metadata and query caches as seen in the previous code snippet.
Would you mind sharing that specific need of yours to be able to provide you with a better answer/solution to your problem?
I am new to caching
What should I cache
eg. Do I cache user info? eg. since they are frequently used throughout the application (like in the header saying "welcome {username}")?
But most things should be used quite frequently anyways? eg. Users have projects. These projects don't belong to everyone, but they will be frequently used by specific users do I cache them too? Won't I be caching nearly everything then?
Also regarding CRUD, with doctrine queries, I can just use $query->useResultCache(true) but what happens when I update/delete an entity? I need to somehow update my cache too? how?
The basic principle of caching is to hold frequently used data that doesnt change often in memory to reduce database work.
Its more convenient to use the php session variables to hold basic things like username.
In case of projects, if they dont change often, and retrieved by users frequently, it would be a good idea to cache them. How long a project info stays cached depends on the change frequency.
Also note that if the info you present to users is vital or time important, you should use caching cautiously.
Check this reference page for basic information on caching http://www.doctrine-project.org/docs/orm/2.0/en/reference/dql-doctrine-query-language.html#cache-related-api
Or check http://www.doctrine-project.org/docs/orm/2.0/en/reference/caching.html for detailed explanation.
I have a series of code books in my database, and I am using plain JDBC calls to fetch them and store them in a collection. I would like to put these in some kind of a cache at application startup time in order to save time later.
I don't need any fancy stuff like automatic object invalidation, TTL etc - the code books change rarely, so I'll trigger the update myself and just reload the whole cache when the need arises.
The project where I need this uses Spring, and this is my first project using it. Is there a standard/elegant way to do this in Spring?
Thanks.
Check out Spring-cache.
Supports EHCache, OSCache and a memory cache, but allows pluggable cache providers too.