I am relatively new to REDIS. I am working with Spring Boot application with Redis running in cluster mode.
In my application, in order to implement a certain use case, I need to consume expire events for a set of keys.
From Redis documentation on key expiry: key expiry in REDIS, it seems expired events are generated when the Redis server deletes the key and this may not be exactly when the time to live theoretically reaches the value of zero.
Also, as I mentioned above, Redis is running in cluster mode and from the article, it seems, to receive all keyspace events of a cluster, clients need to subscribe to each of the nodes.
Basically, my spring boot application is running on three k8s pods and I would like to consume the expired events for a key exactly when latter gets expired, by each of the pods.
In this regard, I have been able to implement Redis pub/sub in multiple pod set up, but I am not able to understand how should I do this for expiry event of keys in Redis cluster set up.
I have searched for this and found a quite a few posts as follows:
Expired key trigger event - Spring data Redis
How to Enable Keyspace Notifications for Expired Keys using Spring Data Redis
Spring data redis - listen to expiration event
But, in these articles, I did not find any clue regarding how to achieve this.
Could anyone please give some pointers regarding this ? Thanks.
Related
Currently using Spring Session Redis in a Spring Boot 2.5 web application.
The application correctly receives web session expiration notifications from Redis, but AFAIK only if the application is up and running at that time. It seems that if the backend is stopped at that time (crash, planned maintenance, ...), the notifications are not "kept" or "delayed" until the backend is up again, they are simply lost.
In https://redis.io/topics/notifications from "if your Pub/Sub client disconnects, and reconnects later, all the events delivered during the time the client was disconnected are lost" I understand that this is the standard Redis behavior.
Is this indeed the current Spring Session Redis behavior ?
Are there recommended solutions to "delay" the notifications ?
As you mentioned, this is the standard behavior of Redis Keyspace notifications.
If a key expires while the application is down, the expiry event cannot be processed, there is no way to delay the notification.
In the context of session expiration in Spring Session Data Redis, this means that the SessionExpiredEvent/SessionDeletedEvent will not fire if the application is down (crashed etc) when the Redis Keyspace notification is sent.
This will cause problems if you have a task dependent on these events.
It also means that resources may not get cleaned up properly.
However, from a security standpoint, the session will still expire correctly and the user will not be able to continue using the expired session.
If an application is performing a critical task on expiry, a different data store may be a better fit, given the "fire and forget" nature of Redis Pub/Sub.
Alternatively, having multiple nodes will make it unlikely that they will all be down at the time of receiving a notification.
I am working on Microservice architecture. One of my service is exposed to source system which is used to post the data. This microservice published the data to redis. I am using redis pub/sub. Which is further consumed by couple of microservices.
Now if the other microservice is down and not able to process the data from redis pub/sub than I have to retry with the published data when microservice comes up. Source can not push the data again. As source can not repush the data and manual intervention is not possible so I tohught of 3 approaches.
Additionally Using redis data for storing and retrieving.
Using database for storing before publishing. I have many source and target microservices which use redis pub/sub. Now If I use this approach everytime i have to insert the request in DB first than its response status. Now I have to use shared database, this approach itself adding couple of more exception handling cases and doesnt look very efficient to me.
Use kafka inplace if redis pub/sub. As traffic is low so I used Redis pub/sub and not feasible to change.
In both of the above cases, I have to use scheduler and I have a duration before which I have to retry else subsequent request will fail.
Is there any other way to handle above cases.
For the point 2,
- Store the data in DB.
- Create a daemon process which will process the data from the table.
- This Daemon process can be configured well as per our needs.
- Daemon process will poll the DB and publish the data, if any. Also, it will delete the data once published.
Not in micro service architecture, But I have seen this approach working efficiently while communicating 3rd party services.
At the very outset, as you mentioned, we do indeed seem to have only three possibilities
This is one of those situations where you want to get a handshake from the service after pushing and after processing. In order to accomplish the same, using a middleware queuing system would be a right shot.
Although a bit more complex to accomplish, what you can do is use Kafka for streaming this. Configuring producer and consumer groups properly can help you do the job smoothly.
Using a DB to store would be a overkill, considering the situation where you "this data is to be processed and to be persisted"
BUT, alternatively, storing data to Redis and reading it in a cron-job/scheduled job would make your job much simpler. Once the job is run successfully, you may remove the data from cache and thus save Redis Memory.
If you can comment further more on the architecture and the implementation, I can go ahead and update my answer accordingly. :)
I have a consumer that reads and writes messages to a time-series database. We have multiple instances of the time series database running as a cluster on multiple physical machines.
Our plan is to deploy the consumer on Kubernetes so I can scale if I need more instance with load-balance they all point to a single time series service that is running.
Now I getting an Issue where it's come to my mind that if I have 5 instances which consume the same topic then they work individually means( they all get message payload and save like any one instance is doing )
What we want is
we want if one consumer is busy then it will go to the next free instance but not be subscribed to by all instance running. To scale or load-balance means I want like normal load-balancing application or how spring-boot app works when you scale on Kubernetes
so is there any way to make it like a load-balancing consumer and processing only one, even consume by 1st or 2nd or 3rd like normal app work as loadbanlacer?
if anyone has ideas about this, how it going to behave and what kind of output we are going to get if doing this with Kafka Spring boot application?
We have a microservice which is developed using spring boot. couple of the functionalities it implements is
1) A scheduler that triggers, at a specified time, a file download using webhdfs and process it and once the data is processed, it will send an email to users with the data process summary.
2) Read messages from kafka and once the data is read, send an email to users.
We are now planning to make this application high available either in Active-Active or Active-passive set up. The problem we are facing now is if both the instances of the application are running then both of them will try to download the file/read the data from kafka, process it and send emails. How can this be avoided? I mean to ensure that only one instance triggers the download and process it ?
Please let me know if there is known solution for this kind of scenarios as this seems to be a common scenario in most of the projects? Is master-slave/leader election approach a correct solution?
Thanks
Let the service download that file, extract the information and publish them via kafka.
Check beforehand if the information was already processed by querying kafka or a local DB.
You also could publish an DataProcessed-Event that triggers the EmailService, that sends the corresponding E-Mail.
I am currently implementing a spring boot microservice, which is persisting data to a single Cassandra database node. I need to be able to set the number of retries if the connection to the database is lost and the number of milliseconds between the retries in the microservice config file. I am using "spring-boot version 1.5.6" and spring-data-cassandra version 1.5.6". I was able to set the number of milliseconds between retries by creating cluster of type CassandraCqlClusterFactoryBean and passing a custom reconnection policy in the cluster.setReconnectionPolicy() method. But I am not able to set the number of retries with a custom retry policy. If understood correctly the retry policy handles only the case in which a query is made, but in my case I need to set the number of retries in all times no matter if a query is made or not. After a couple of days of research I was able to produce an ugly hack which basically uses a custom ReconnectionSchedule and stops the spring boot application after certain conditions are met in the nextDelayMs() method. Nevertheless I continued to look in the source code in debug mode and I saw that a NoHostAvailableException exception is thrown by the ControlConnection. So I checked the datastax official documentation regarding Control connection, and I found
Coming soon…
So could someone please show me how to correctly implement a way of stopping my cassandra driver of trying to reconnect to the node after a predefined number of retries.
Thanks in advance.
Look here at 9.3.1.
Maybe you can do something like trying to open a session each x second until a timeout expired or a session is successful created.