I have a application using Redis. This system implemented with java spring used jedis package for connection to the redis with the configuration as follow
jedis.pool.host=redisServer-IP
so the application connect to redis server on the redisServer-IP and works fine but, for the lack of memory on a single server and and HA capability I need to use a redis cluster I used docker compose to create a redis cluster using the here.
Also redis cluster working fine with three masters and three replicas.
I just need to understand, the Redis Cluster can work with the single endpoint, because I can only set single endpoint in the above jedis.pool.host configuration, or I need to have a proxy to deal with the redis cluster ?
NOTE: I can not make any changes in my application
Related
Currently we are exploring Redis as distributed in-mem cache which would be hosted on AWS via Amazon elasticache.
For Redis cluster connection from application, we are using Jedis client.
What are the cache property settings supported from client side?
org.springframework.data.redis.cache.RedisCacheConfiguration does not give much options other than TTL(time-to-live). IS there any other way to set more cache properties from client-side like enabling ssl, memory size etc.?
I am trying to deploy a spring-security server, with Redis as token store.
In order to have some redundancy in Redis, we want to deploy it as a cluster.
The problem is Jedis, which is used by spring security as underlying library, doesn't support pipelining in cluster mode, but spring security uses pipelining.
My question is how can I solve this situation. More precisely:
1- Should I use another mode of deployment form Redis? What actually works.
2- Can I somehow force spring security to use reddison for connecting to Resid?
Please adivse.
If you want redundancy, use replication (master/slave) not cluster.
If you have more data than RAM on a machine, use cluster.
If you have more data than RAM on a machine and want redundancy, use cluster with replication.
Jedis supports replication with sentinel, so give that a go unless you have a lot of data. Some more info on usage here: https://github.com/xetorthio/jedis/issues/725
In my project we have a requirement to run two instances of spring cloud config server so if one instance goes down, other will take care the config server responsibilities.
Currently, you would need to put config server behind a load balancer. It is stateless, so that wouldn't hurt. There is an open issue to configure multiple config server url's in the client, so it could do failover there.
If you are running multiple instances of the config server, you can have them all register themselves in Eureka, and maybe do a lookup to the config server with it's application name via Eureka in all the other microservices. This way, Zuul (and Ribbon) will take care of the load balancing.
Edit:
I guess spencergibb is right. It's best to use a load balancer, for eg: ELB, if you're going to deploy on AWS.
Consider multiple spring-cloud-config-uris for high availability
I want to create a distributed cluster in spring xd.
I am able to create a cluster with single admin, one zookeeper, one instance of redis and hsqldb.
But when i'm trying to do that with multiple instance of zookeeper , hsqldb, redis ,i'm not able to configure it correctly.
You should only have a single instance of zookeeper, hsqldb and redis. All xd-admins should be configured to connect to the same instance of each of these services and so should the xd-containers be.
Like Thomas has mentioned, the idea is that you have your (multiple) instances of admin and containers deployed, and all connect to the same zk,redis, hsqldb & rabbitmq.
Why do you want to start multiple instances of these applications?
Zookeeper provides the topology of the cluster and manages deployments. Also, it makes sure to note when nodes go up and down - avoiding single point of failures when you have many xd-admin instances (one is leader and the others replicate, they will become leader if the current one fails).
Or are you talking about making those instance parallel to avoid a SPOF? In that case, you should try to dedicate an entire VM for each of those applications.
I am working on Spring XD and GemFire XD. I want to understand how Spring XD's distributed environment works. I know spring xd uses either redis or rabittmq as the transport.
I am clear about this, I have install spring xd and rabittmq on one machine. I changed the redis.properties file and added hostnames.
Do I need to install spring xd on all the machines? If so, after installing, how to bring those up.
On the master machine, I will do ./xd-admin and ./xd-container
How do you start up the nodes (spring xd instances/workers) so that they can listen for instructions from xd-admin?
Please help me on this.
Thanks,
-Suyodhan
Redis is used for analytics as only supported platform. For transport, you need either Redis or Rabbit.
Basically you just need to install Redis and RabbitMQ per their respective documentation. They can be in same or different servers, Ideally you would use their high availability option. For example Redis Sentinal. YOu don't need RabbitMQ unless you want to change the default transport from Redis to Rabbit. Once you install Redis and Rabbit, bring them up and provide their host:port info (and any additional as applicable) to the servers.yml in XD install (in all nodes) and bring up admin and containers. Evrything should work automatically by using zookeeper as the means to manage the distributed runtime.
If you use Spring XD in distributed mode, I assume you have set up zookeeper as well. (If not check this http://docs.spring.io/spring-xd/docs/1.0.0.M7/reference/html/#_setting_up_zookeeper )
Admin and Container instances register themselves with Zookeeper as they come up. Admin queries zookeeper for available containers and assign tasks like deploying modules. Zookeeper is the trick behind Distributed mode.
Hope this helps.
You will install Spring xd one time on one machine, Spring XD will be connected to your hdfs distributed scaled out environment.
You need to start the followings:
1. redis or rappitMQ in your case
2. hsqldb server
3. container
4. admin
when you start spring xd, you need to register the name node firstly using the command:
hadoop config fs --name hdfs://serverip:8020
then you can use any module defined in spring xd (using stream or batch) by specifying its parameters directly without specifying those in the server.yml file.
Moha.