I was experimenting with using Redis for caching and got a bit confused with persistence. I would like to create a service that automatically writes into a PostgreSQL database every time the Redis cache is updated.
My initial thought was to use the Pub/Sub feature to listen to events and then trigger the appropriate queries that write into my RDBMS. Though, after looking some more into the Pub/Sub feature it appears that it is probably not be meant to be used this way.
It seems that the standard practice is to keep that logic inside of the REST API that queries Redis. I would however like to have a service that automatically does this in the background. Is there a simpler way of doing this that I am missing ?
Thank you in advance.
I guess the common way is to Save data to MongoDB and Redis manually.
for example when a user post new message:
newPost = new Post("Hello world!");
// save to db and get model ID
newPost->save();
// save model to Redis
newPost->saveToCache();
This approach will guarantee that all new Models will store in database even when Redis is down.
An other approach is to save data in Redis first, and add a flag to a Redis List, Then(anytime later) read flags from list and save related data to database.
Related
I am developing some SPA with a backend written in Java (Spring Boot). In relational DB that backend connects to, there is a table with some dictionary values. Values can edited by users of the app, but it's done really, really rarely (almost never).
Those dictionary values are used in a lot of pages on UI and because of that I would like to "cache" them in a way. What I want to achieve is that I want to load dictionary values on startup to avoid asking DB for values during every request between UI and Backend.
Firstly, I thought about just loading it on the UI part of the app, when user enters the page for the first time. Then I ruled it out, since when one of the users changes the values, it should be reloaded.
What I think might work is just loading them on startup of Backend into some collection (that can be safely used in concurent environment, probably ConcurrentMap) and then during some GET requests asking that collection for the values (instead of DB). When the values are changed, that request just updates the DB table and reloads them into collection.
Then I thought that the collection solution won't be enough, when my backend would be scaled up to more than one instance. In that case, only one of instances will be updated and the second one will provide outdated data. We can avoid it and force refreshes i.e. every 15 minutes (instead of on demand during values update).
But what I think is the best solution is to start some redis service on a side, load dictionary values into it and after every DB update of the values just update the redis instance with the new ones. Every instance of backend would use the same instance of redis, which seems quicker than executing query (select * from _ where _ = _) on DB.
What do you think? Is my thought process is correct? Do you have any ideas that can help solve my issue?
If you are using Spring you could check out Spring Cache Abstraction. That way your cache will be up-to-date whenever some change occurs.
Out of the box few implementations are supported by Spring:
Spring provides a few implementations of that abstraction: JDK java.util.concurrent.ConcurrentMap based caches, Ehcache 2.x, Gemfire cache, Caffeine, and JSR-107 compliant caches (such as Ehcache 3.x). See Plugging-in Different Back-end Caches for more information on plugging in other cache stores and providers.
If you decide to use Memcached implementation you can check out this library (uses Xmemcached under the hood) here.
You could also check a small demo app of how to use Spring Cache Abstraction in your project (link).
I think your in the right path with your approach in terms of 'caching'. I suggest you also check Memcached for it simplicity. Redis is a good choice but still it depends on your requirements and if you need that much feature. just my 2cent
https://aws.amazon.com/elasticache/redis-vs-memcached/
https://devcenter.heroku.com/articles/spring-boot-memcache#add-caching-to-spring-boot
Thanks,
I need to create a food ordering service, using microservices, scalable , cluster, several steps to order. Need to store user data between steps / requests.
What is an approach to keep state and user data? Store it in DB? Cache? Shared memory?
Are there any tutorials for the best practice of it?
(I gonna use spring / springboot and modules)
Anything that you cannot afford to lose (usually the business data) will go in DB and can be parallelly cached in an in-memory DB like Redis that has a cache eviction algorithm inbuilt.
Anything that, if lost, is not a big deal (usually the technical things that are not directly linked with the business data) can go only in an in-memory DB.
Since you are using Spring, you could probably use something like Redis with Spring Data Redis. There are already known Spring solutions (such as this) to fall back on api calls to fetch data from DB if the Redis server goes down. You can also run multiple Redis instances behind Redis Sentinel to provide failover. Redis Cluster provides a way to run a Redis installation where data is automatically sharded across multiple Redis nodes. Also, you can configure Redis to persist the data in file system once daily or so to backup the cache data for disaster recovery.
If you are looking for a fully managed service, AWS provides "Step Functions" to satisfy your stateful requirements: https://stackoverflow.com/questions/tagged/aws-step-functions
I have Oracle as my main RDBMS for read and write, but I want to use couchbase as caching layer as it has map-reduce as can be used as memcache. Any idea as to how i can implement that, and how to transfer and update data in the caching layer, when Oracle is updated or inserted etc.
You are not telling anything about your current performance issues.
I have seen too many applications which do not really take advantage of RDBMS/SQL features, especially if an ORM sits in between.
The cure is to put another cache on top of a database, and to synchronize this in a cluster manually using IP multicasts (SwarmCache for example), message queues (JMS) or nightly import jobs. It could create more problems in the end. And it increases system complexity.
So my answer to your question is: I would not do it, as long as there is room for improvement regarding your data model and/or queries.
I believe your question is about Database synchronization. This can be done through a combination of using DB dependencies and "right-thru" features that I am not too sure about whether couchbase offers. So with DB dependency you have cached items dependent upon Db items and if the DB items are updated or deleted the corresponding dependent item in the cache is removed and at the same time you can write a "right-thru" handler executed at the server level; and the main purpose of this handler is loading fresh copies of the removed items in the cache. So, basically, you'll write the handler once and registerit with the cache server and the cache server will execute it when needed to sync. new items in the DB with the cache. This reading on Db synchronization can be useful . Its based on a product Ncache.
So your question is not directly related to Couchbase, but as other stated more about how you can be alerted when data are changing into your Oracle instance.
One thing that is not well known is the Oracle Database Change Notification feature that is quite cool for this:
http://docs.oracle.com/cd/E11882_01/java.112/e16548/dbchgnf.htm
So you can create an application that is listening to your changes and pushes the data into Couchbase.
Say I have some objects that need to be created only once in the application yet accessed from within multiple requests. The objects are immutable. What is the best way to do this?
Store them in the session.
If you don't want to lose them after a server's restart, then use a database (SQLite, which is a single file, for example).
You want to persist your objects. Normally you'd do it with some ORM like Active Record or Datamapper. Depending on what is available to you. If you want something dead simple without migrations and you have access to a MongoDB use mongomapper.
If that object is used only for some time, then it is discarded (and if needed again, then recreated), use some caching mechanism, like memcached or redis.
If setting up such services is heavy and you want to avoid it, and - say - you are using Debian/Ubuntu then save your objects into files (with Marshal-ing) in the /shm device which is practically memory.
If the structure of the data is complex, then go with SQLite as suggested above.
I have a series of code books in my database, and I am using plain JDBC calls to fetch them and store them in a collection. I would like to put these in some kind of a cache at application startup time in order to save time later.
I don't need any fancy stuff like automatic object invalidation, TTL etc - the code books change rarely, so I'll trigger the update myself and just reload the whole cache when the need arises.
The project where I need this uses Spring, and this is my first project using it. Is there a standard/elegant way to do this in Spring?
Thanks.
Check out Spring-cache.
Supports EHCache, OSCache and a memory cache, but allows pluggable cache providers too.