I have a multi-master redis cluster configuration hosted in Kubernetes cluster. We connect to the redis via hostname of load balancer using Jedis Library. hostname is prod-redis-cluster.xyz.com. The following is the code I use to connect to the Redis environment.
When there is a patching to redis nodes, IP address of the load balancer does not change whereas the IP address of the actual Redis nodes behind the load balancer changes. However, application loses connectivity to the cluster and it re-establishes only when we restart the application. Is there any other configuration needed here?
#Configuration
public class CacheConfig {
#Value("${jedis.nodes}")
private String hostName;
#Value("${jedis.password}")
private String password;
#Value("${jedis.timeout}")
private long timeout;
#Bean
public RedisClusterConfiguration redisClusterConfiguration() {
RedisClusterConfiguration redisClusterConfiguration = new RedisClusterConfiguration();
String[] hostOrPort = hostName.split(":");
Set<RedisNode> nodeList = new HashSet<>();
nodeList.add(new RedisNode(hostOrPort[0], Integer.parseInt(hostOrPort[1])));
redisClusterConfiguration.setClusterNodes(nodeList);
redisClusterConfiguration.setPassword(password);
return redisClusterConfiguration;
}
#Bean
public JedisPoolConfig jedisPoolConfig() {
JedisPoolConfig poolConfig = new JedisPoolConfig();
poolConfig.setMaxIdle(50);
poolConfig.setMaxTotal(50);
poolConfig.setTestOnCreate(true);
poolConfig.setTestOnBorrow(true);
poolConfig.setTestOnReturn(true);
poolConfig.setTestWhileIdle(true);
return poolConfig;
}
#Bean("myJedisConnectionFactory")
public JedisConnectionFactory jedisConnectionFactory(RedisClusterConfiguration redisClusterConfiguration,
JedisPoolConfig jedisPoolConfig) {
Duration duration = Duration.ofMillis(timeout);
JedisClientConfiguration clientConfiguration = JedisClientConfiguration.builder().
connectTimeout(duration)
.readTimeout(duration).usePooling()
.poolConfig(jedisPoolConfig).build();
JedisConnectionFactory jedisConnectionFactory = new JedisConnectionFactory(
redisClusterConfiguration, clientConfiguration);
return jedisConnectionFactory;
}
#Bean("redisTemplate")
public RedisTemplate<String, String> getInfo(#Qualifier("myJedisConnectionFactory")
JedisConnectionFactory jedisConnectionFactory) {
RedisTemplate<String, String> template = new RedisTemplate<String, String>();
template.setDefaultSerializer(new StringRedisSerializer());
template.setHashKeySerializer(new StringRedisSerializer());
template.setKeySerializer(new StringRedisSerializer());
template.setConnectionFactory(jedisConnectionFactory);
template.afterPropertiesSet();
return template;
}
}
Related
Folks,
I have a redis cache connection for my application. Which recently got stalled due to connections increase. So I have implimented connection pooling for my redis connection in my application. The configuration is as below:
#Configuration
public class CacheConfig {
#Value("${cache.host.name}")
private String cacheHostName;
#Value("${cache.database.number}")
private Integer database;
#Value("${cache.port.number}")
private Integer portNumber;
#Value("${cache.pool.max-total}")
private Integer maxTotal;
#Value("${cache.pool.max-idle}")
private Integer maxIdle;
#Value("${cache.pool.min-idle}")
private Integer minIdle;
#Bean
JedisConnectionFactory jedisConnectionFactory() {
RedisStandaloneConfiguration redisConfig =
new RedisStandaloneConfiguration(cacheHostName, portNumber);
redisConfig.setDatabase(database);
JedisConnectionFactory redisConnectionFactory =
new JedisConnectionFactory(redisConfig, jedisClientConfiguration());
return redisConnectionFactory;
}
#Bean
JedisClientConfiguration jedisClientConfiguration() {
JedisPoolConfig jedisPoolConfig = new JedisPoolConfig(); // GenericObjectPoolConfig
jedisPoolConfig.setMaxTotal(maxTotal);
jedisPoolConfig.setMaxIdle(maxIdle);
jedisPoolConfig.setMinIdle(minIdle);
return JedisClientConfiguration.builder().usePooling().poolConfig(jedisPoolConfig).build();
}
#Bean
public RedisTemplate<String, Object> redisTemplate() {
RedisTemplate<String, Object> template = new RedisTemplate<>();
template.setConnectionFactory(jedisConnectionFactory());
return template;
}
}
But I'm not able to validate if my configuration is actually in effect. Any suggestion to validate this is appreciated!
I have two projects that use Spring boot with Redis. In the first project, I have saved data into Redis cache.
Then, in the second project, when I get data from Redis that I saved in project one it is always null, and project one is also null when I try to get data from Redis that I have saved in project two.
Both projects use the dto as below.
#Data
#EqualsAndHashCode(callSuper = false)
#NoArgsConstructor
#AllArgsConstructor
#JsonInclude(content = Include.NON_NULL)
public class AccountDto implements Serializable {
#JsonIgnore
private static final long serialVersionUID = -2006838697507278668L;
#JsonProperty("account_no")
private String accountNo;
#JsonProperty("account_type")
private String accountType;
#JsonProperty("customer_no")
private String customerNO;
#JsonProperty("entity_code")
private String entityCode;
#JsonProperty("auth_stat")
private String authStat;
#JsonProperty("create_at")
#Temporal(TemporalType.TIMESTAMP)
private Date createAt;
}
Redis configuration
#Configuration
#RefreshScope
#EnableRedisRepositories
public class RedisConfiguration {
#Value("${spring.redis.host: ''}")
private String redisHost;
#Value("${spring.redis.port: ''}")
private int redisPort;
#Bean
public JedisConnectionFactory connectionFactory() {
RedisStandaloneConfiguration configuration = new RedisStandaloneConfiguration();
configuration.setHostName(redisHost);
configuration.setPort(redisPort);
return new JedisConnectionFactory(configuration);
}
#Bean
public RedisTemplate<String, Object> redisTemplate() {
RedisTemplate<String, Object> template = new RedisTemplate<>();
template.setConnectionFactory(connectionFactory());
template.setKeySerializer(new StringRedisSerializer());
template.setValueSerializer(new StringRedisSerializer());
return template;
}
}
This class I have created two methods, one for saving and getting data from Redis
#RequiredArgsConstructor
#Service
public class AccountDtoCache {
private static final String HASH_KEY_ACCOUNT = "Account";
private static HashOperations<String, String, AccountDto> hashOperations;
private final RedisTemplate<String, Object> redisTemplate;
#PostConstruct
private synchronized void init() {
hashOperations = redisTemplate.opsForHash();
}
public void save(AccountDto account) {
hashOperations.put(HASH_KEY_ACCOUNT, account.getAccountNo(), account);
}
public static AccountDto getByAccountNo(String account) {
return hashOperations.get(HASH_KEY_ACCOUNT, account);
}
}
I try to get data from Redis as below.
When I debug, it's always null in other projects, but it's working with the same project that I have saved data into Redis.
AccountDtoCache.getByAccountNo("accountNo");
I fixed the issue above with two steps as below
I changed both of the projects package names to the same package name
Redis config
#Bean
public RedisTemplate<String, Object> redisTemplate(RedisConnectionFactory redisConnectionFactory) {
RedisTemplate<String, Object> template = new RedisTemplate<>();
GenericJackson2JsonRedisSerializer serializer = new GenericJackson2JsonRedisSerializer();
template.setConnectionFactory(redisConnectionFactory);
template.setKeySerializer(new StringRedisSerializer());
template.setValueSerializer(new StringRedisSerializer());
template.setHashKeySerializer(serializer);
template.setHashValueSerializer(serializer);
return template;
}
I have implemented Redis session management using LettuceConnectionFactory on my Spring Boot java application. I couldn't figure out a way to capture Redis connection failure. Is there a way to capture the connection failure?
Spring Boot version: 2.2.6
Code:
#Configuration
#PropertySource("classpath:application.properties")
#EnableRedisHttpSession
public class HttpSessionConfig extends AbstractHttpSessionApplicationInitializer {
Logger logger = LoggerFactory.getLogger(HttpSessionConfig.class);
#Value("${spring.redis.host}")
private String redisHostName;
#Value("${spring.redis.port}")
private int redisPort;
#Value("${spring.redis.password}")
private String redisPassword;
private #Value("${spring.redis.custom.command.timeout}")
Duration redisCommandTimeout;
private #Value("${spring.redis.timeout}")
Duration socketTimeout;
#Bean
LettuceConnectionFactory lettuceConnectionFactory() {
final SocketOptions socketOptions = SocketOptions.builder().connectTimeout(socketTimeout).build();
final ClientOptions clientOptions = ClientOptions.builder()
.socketOptions(socketOptions)
.build();
LettuceClientConfiguration clientConfig = LettuceClientConfiguration.builder()
.commandTimeout(redisCommandTimeout)
.clientOptions(clientOptions)
.readFrom(ReadFrom.SLAVE_PREFERRED)
.build();
RedisStandaloneConfiguration serverConfig = new RedisStandaloneConfiguration(redisHostName,
redisPort);
final LettuceConnectionFactory lettuceConnectionFactory = new LettuceConnectionFactory(serverConfig,
clientConfig);
return lettuceConnectionFactory;
}
#Bean
public RedisTemplate<Object, Object> sessionRedisTemplate() {
final RedisTemplate<Object, Object> template = new RedisTemplate<>();
template.setConnectionFactory(lettuceConnectionFactory());
return template;
}
#Bean
public ConfigureRedisAction configureRedisAction() {
return ConfigureRedisAction.NO_OP;
}
}
I have configured Redis in my spring boot application and I put some properties for that in application.properties file. but its taking default properties (localhost:6379)
here is config
#Configuration
#EnableRedisRepositories("com.demo.redis.repository")
public class RedisDataSourceConfig {
#Bean
public JedisConnectionFactory jedisConnectionFactory() {
log.debug("Create Jedis Connection Factory");
return new JedisConnectionFactory();
}
#Bean
public RedisTemplate<String, Object> redisTemplate() {
RedisTemplate<String, Object> template = new RedisTemplate<>();
template.setConnectionFactory(jedisConnectionFactory());
template.setKeySerializer(new StringRedisSerializer());
template.setValueSerializer(new JdkSerializationRedisSerializer());
template.setHashKeySerializer(template.getKeySerializer());
template.setHashValueSerializer(template.getValueSerializer());
return template;
}
#Bean
public RedisTemplate<String, Object> jsonRedisTemplate() {
RedisTemplate<String, Object> redisTemplate = new RedisTemplate<>();
redisTemplate.setConnectionFactory(jedisConnectionFactory());
redisTemplate.setKeySerializer(new StringRedisSerializer());
redisTemplate.setValueSerializer(new Jackson2JsonRedisSerializer<>(Object.class));
redisTemplate.setHashKeySerializer(redisTemplate.getKeySerializer());
redisTemplate.setHashValueSerializer(redisTemplate.getValueSerializer());
return redisTemplate;
}
}
here is my properties in property file
spring.cache.type=redis
spring.redis.host=192.168.10.226
spring.redis.port=6379
spring.cache.redis.time-to-live=600000
I have used RedisStandaloneConfiguration to set properties. it's working fine now
#Bean
JedisConnectionFactory jedisConnectionFactory() {
RedisStandaloneConfiguration redisStandaloneConfiguration = new RedisStandaloneConfiguration();
redisStandaloneConfiguration.setHostName("localhost");
redisStandaloneConfiguration.setPort(6379);
redisStandaloneConfiguration.setDatabase(0);
redisStandaloneConfiguration.setPassword(RedisPassword.of("password"));
JedisClientConfigurationBuilder jedisClientConfiguration = JedisClientConfiguration.builder();
jedisClientConfiguration.connectTimeout(Duration.ofSeconds(60));// 60s connection timeout
JedisConnectionFactory jedisConFactory = new JedisConnectionFactory(redisStandaloneConfiguration,
jedisClientConfiguration.build());
return jedisConFactory;
}
Add the PropertySource Annotation with your config
#Configuration
#EnableRedisRepositories("com.demo.redis.repository")
#PropertySource("classpath:application.properties")
public class RedisDataSourceConfig {
#Bean
public JedisConnectionFactory jedisConnectionFactory() {
log.debug("Create Jedis Connection Factory");
return new JedisConnectionFactory();
}
#Bean
public RedisTemplate<String, Object> redisTemplate() {
RedisTemplate<String, Object> template = new RedisTemplate<>();
template.setConnectionFactory(jedisConnectionFactory());
template.setKeySerializer(new StringRedisSerializer());
template.setValueSerializer(new JdkSerializationRedisSerializer());
template.setHashKeySerializer(template.getKeySerializer());
template.setHashValueSerializer(template.getValueSerializer());
return template;
}
#Bean
public RedisTemplate<String, Object> jsonRedisTemplate() {
RedisTemplate<String, Object> redisTemplate = new RedisTemplate<>();
redisTemplate.setConnectionFactory(jedisConnectionFactory());
redisTemplate.setKeySerializer(new StringRedisSerializer());
redisTemplate.setValueSerializer(new Jackson2JsonRedisSerializer<>(Object.class));
redisTemplate.setHashKeySerializer(redisTemplate.getKeySerializer());
redisTemplate.setHashValueSerializer(redisTemplate.getValueSerializer());
return redisTemplate;
}
}
Instead of using the #Bean annotation, you should use #Autowired for JedisConnectionFactory. Use the following code:
#Autowired
private JedisConnectionFactory jedisConnectionFactory;
And then in your redisTemplate() method just call:
template.setConnectionFactory(jedisConnectionFactory);
Now, jedisConnectionFactory will use your application.properties as long as it is available in the standard path: src/main/resources.
Trying to use SpringBoot with SpringData with Elasticache:
application.properties:
spring.redis.host=XXXX-dev.XXXX.clusXXXcfg.XXX.cache.amazonaws.com
spring.redis.port=6379
CacheConfiguration:
#Configuration
#PropertySource("classpath:application.properties")
public class CacheConfiguration {
#Value("${spring.redis.host}")
private String redisHostName;
#Bean
public RedisTemplate<String, Company> redisTemplate() {
RedisTemplate<String, Company> template = new RedisTemplate();
template.setConnectionFactory(jedisConnectionFactory());
return template;
}
#Bean
JedisConnectionFactory jedisConnectionFactory() {
JedisConnectionFactory factory = new JedisConnectionFactory();
factory.setHostName(redisHostName);
factory.setUsePool(true);
return factory;
}
#Bean
public static PropertySourcesPlaceholderConfigurer propertySourcesPlaceholderConfigurer() {
return new PropertySourcesPlaceholderConfigurer();
}
}
Service call:
#Autowired
RedisTemplate<String, Company> redisTemplate;
private ValueOperations valueOperations;
#PostConstruct
private void init() {
valueOperations = redisTemplate.opsForValue();
}
#Override
public String createOtp(Company company) {
String token = UUID.randomUUID().toString();
valueOperations.set(token, company);
valueOperations.getOperations().expire(token, 5, TimeUnit.MINUTES);
return token;
}
Error:
org.springframework.data.redis.ClusterRedirectException: Redirect: slot 7228 to 10...:6379.*
redis.clients.jedis.exceptions.JedisMovedDataException: MOVED 7228 10...:6379.*
The question is - what is wrong with configuration?
You're running your Elasticache in Redis Cluster mode (only Redis Cluster responds with MOVED) but the connection factory is configured in standalone mode.
Spring Boot can auto-configure all the things you've set up manually for you. Basically, remove your CacheConfiguration class (or at least remove the majority of code):
#Configuration
public class CacheConfiguration {
#Bean
public RedisTemplate<String, Company> redisTemplate(RedisConnectionFactory connectionFactory) {
RedisTemplate<String, Company> template = new RedisTemplate();
template.setConnectionFactory(connectionFactory);
return template;
}
}
And then configure the following properties in your application.properties file:
spring.redis.cluster.nodes=<node_host>:<port> # Comma-separated list of "host:port" pairs to bootstrap from.
Spring Boot loads application.properties by default and the Redis auto-config configures a RedisTemplate<Object, Object> bean by default. Specializing beans is a valid use-case – do not duplicate what's already provided by the auto-config, especially if you want to achieve what auto-config does.
See also:
Common application properties
Externalized Configuration