Redis cluster client creation in spring with spark - spring

Question

I have a project which was connecting to standalone redis with client creation as :
public RedisConnectionFactory clientCreation() {
JedisPoolConfig poolConfig = new JedisPoolConfig();
poolConfig.setMaxTotal(5);
poolConfig.setTestOnBorrow(true);
poolConfig.setTestOnReturn(true);
JedisConnectionFactory client = new JedisConnectionFactory(poolConfig);
client.setHostName(redisEastEndpoint);
client.setPort(redisEastPort);
client.setTimeout(redisTimeout);
return client;
the library version used for jedis and spring-data-redis are:
[INFO] +- redis.clients:jedis:jar:2.9.3:compile
[INFO] | \- org.apache.commons:commons-pool2:jar:2.6.2:compile
[INFO] +- org.springframework.data:spring-data-redis:jar:2.1.8.RELEASE:compile
[INFO] | +- org.springframework.data:spring-data-keyvalue:jar:2.1.8.RELEASE:compile
[INFO] | +- org.springframework:spring-oxm:jar:5.1.7.RELEASE:compile
[INFO] | +- org.springframework:spring-aop:jar:5.1.7.RELEASE:compile
[INFO] | \- org.springframework:spring-context-support:jar:5.1.7.RELEASE:compile
Now I have a requirement to move to cluster redis, and changed the client creation as
public RedisConnectionFactory clientCreation() {
JedisPoolConfig poolConfig = new JedisPoolConfig();
poolConfig.setMaxTotal(5);
poolConfig.setTestOnBorrow(true);
poolConfig.setTestOnReturn(true);
RedisClusterConfiguration redisClusterConfiguration = new RedisClusterConfiguration(redisEastEndpoint);
if(StringUtils.isNotBlank(redisPassword)) {
redisClusterConfiguration.setPassword(redisPassword);
}
JedisClientConfiguration.JedisClientConfigurationBuilder jedisClientConfiguration = JedisClientConfiguration.builder();
jedisClientConfiguration.connectTimeout(Duration.ofSeconds(redisTimeout));
jedisClientConfiguration.usePooling().poolConfig(poolConfig);
return new JedisConnectionFactory(redisClusterConfiguration,
jedisClientConfiguration.build());
with this code change I am getting No reachable node in cluster as :
Caused by: org.springframework.data.redis.RedisConnectionFailureException: No reachable node in cluster; nested exception is redis.clients.jedis.exceptions.JedisNoReachableClusterNodeException: No reachable node in cluster
at org.springframework.data.redis.connection.jedis.JedisExceptionConverter.convert(JedisExceptionConverter.java:67)
at org.springframework.data.redis.connection.jedis.JedisExceptionConverter.convert(JedisExceptionConverter.java:41)
at org.springframework.data.redis.PassThroughExceptionTranslationStrategy.translate(PassThroughExceptionTranslationStrategy.java:44)
at org.springframework.data.redis.FallbackExceptionTranslationStrategy.translate(FallbackExceptionTranslationStrategy.java:42)
at org.springframework.data.redis.connection.jedis.JedisClusterConnection.convertJedisAccessException(JedisClusterConnection.java:760)
at org.springframework.data.redis.connection.jedis.JedisClusterStringCommands.convertJedisAccessException(JedisClusterStringCommands.java:556)
at org.springframework.data.redis.connection.jedis.JedisClusterStringCommands.setNX(JedisClusterStringCommands.java:186)
at org.springframework.data.redis.connection.DefaultedRedisConnection.setNX(DefaultedRedisConnection.java:288)
at org.springframework.data.redis.core.DefaultValueOperations.lambda$setIfAbsent$10(DefaultValueOperations.java:296)
at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:224)
at org.springframework.data.redis.core.RedisTemplate.execute(RedisTemplate.java:184)
at org.springframework.data.redis.core.AbstractOperations.execute(AbstractOperations.java:95)
at org.springframework.data.redis.core.DefaultValueOperations.setIfAbsent(DefaultValueOperations.java:296)
at org.springframework.data.redis.support.atomic.RedisAtomicLong.initializeIfAbsent(RedisAtomicLong.java:147)
at org.springframework.data.redis.support.atomic.RedisAtomicLong.<init>(RedisAtomicLong.java:92)
at org.springframework.data.redis.support.atomic.RedisAtomicLong.<init>(RedisAtomicLong.java:62)
at com.capitalone.api.customers.customerapi.service.config.RedisConfig.westESCID(RedisConfig.java:126)
at com.capitalone.api.customers.customerapi.service.config.RedisConfig$$EnhancerBySpringCGLIB$$88e1d043.CGLIB$westESCID$4(<generated>)
at com.capitalone.api.customers.customerapi.service.config.RedisConfig$$EnhancerBySpringCGLIB$$88e1d043$$FastClassBySpringCGLIB$$7889c704.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244)
at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:363)
at com.capitalone.api.customers.customerapi.service.config.RedisConfig$$EnhancerBySpringCGLIB$$88e1d043.westESCID(<generated>)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154)
... 39 more
Caused by: redis.clients.jedis.exceptions.JedisNoReachableClusterNodeException: No reachable node in cluster
at redis.clients.jedis.JedisSlotBasedConnectionHandler.getConnection(JedisSlotBasedConnectionHandler.java:61)
at redis.clients.jedis.JedisSlotBasedConnectionHandler.getConnectionFromSlot(JedisSlotBasedConnectionHandler.java:78)
at redis.clients.jedis.JedisClusterCommand.runWithRetries(JedisClusterCommand.java:113)
at redis.clients.jedis.JedisClusterCommand.runBinary(JedisClusterCommand.java:58)
at redis.clients.jedis.BinaryJedisCluster.setnx(BinaryJedisCluster.java:273)
at org.springframework.data.redis.connection.jedis.JedisClusterStringCommands.setNX(JedisClusterStringCommands.java:184)
... 59 more
}
since we have a spark application running in spark-2.1.3,due to version dependency I need to stick with the same spring-data-redis.
How would I be able to connect to cluster redis without the version upgrade of jedis and spring-data-redis?

Answer 1

It was spark 2.1.3 which was the bottleneck. Once spark was upgraded to 2.4.4, it worked as expected.