Folks,
I have a redis cache connection for my application. Which recently got stalled due to connections increase. So I have implimented connection pooling for my redis connection in my application. The configuration is as below:
#Configuration
public class CacheConfig {
#Value("${cache.host.name}")
private String cacheHostName;
#Value("${cache.database.number}")
private Integer database;
#Value("${cache.port.number}")
private Integer portNumber;
#Value("${cache.pool.max-total}")
private Integer maxTotal;
#Value("${cache.pool.max-idle}")
private Integer maxIdle;
#Value("${cache.pool.min-idle}")
private Integer minIdle;
#Bean
JedisConnectionFactory jedisConnectionFactory() {
RedisStandaloneConfiguration redisConfig =
new RedisStandaloneConfiguration(cacheHostName, portNumber);
redisConfig.setDatabase(database);
JedisConnectionFactory redisConnectionFactory =
new JedisConnectionFactory(redisConfig, jedisClientConfiguration());
return redisConnectionFactory;
}
#Bean
JedisClientConfiguration jedisClientConfiguration() {
JedisPoolConfig jedisPoolConfig = new JedisPoolConfig(); // GenericObjectPoolConfig
jedisPoolConfig.setMaxTotal(maxTotal);
jedisPoolConfig.setMaxIdle(maxIdle);
jedisPoolConfig.setMinIdle(minIdle);
return JedisClientConfiguration.builder().usePooling().poolConfig(jedisPoolConfig).build();
}
#Bean
public RedisTemplate<String, Object> redisTemplate() {
RedisTemplate<String, Object> template = new RedisTemplate<>();
template.setConnectionFactory(jedisConnectionFactory());
return template;
}
}
But I'm not able to validate if my configuration is actually in effect. Any suggestion to validate this is appreciated!
Related
I have two projects that use Spring boot with Redis. In the first project, I have saved data into Redis cache.
Then, in the second project, when I get data from Redis that I saved in project one it is always null, and project one is also null when I try to get data from Redis that I have saved in project two.
Both projects use the dto as below.
#Data
#EqualsAndHashCode(callSuper = false)
#NoArgsConstructor
#AllArgsConstructor
#JsonInclude(content = Include.NON_NULL)
public class AccountDto implements Serializable {
#JsonIgnore
private static final long serialVersionUID = -2006838697507278668L;
#JsonProperty("account_no")
private String accountNo;
#JsonProperty("account_type")
private String accountType;
#JsonProperty("customer_no")
private String customerNO;
#JsonProperty("entity_code")
private String entityCode;
#JsonProperty("auth_stat")
private String authStat;
#JsonProperty("create_at")
#Temporal(TemporalType.TIMESTAMP)
private Date createAt;
}
Redis configuration
#Configuration
#RefreshScope
#EnableRedisRepositories
public class RedisConfiguration {
#Value("${spring.redis.host: ''}")
private String redisHost;
#Value("${spring.redis.port: ''}")
private int redisPort;
#Bean
public JedisConnectionFactory connectionFactory() {
RedisStandaloneConfiguration configuration = new RedisStandaloneConfiguration();
configuration.setHostName(redisHost);
configuration.setPort(redisPort);
return new JedisConnectionFactory(configuration);
}
#Bean
public RedisTemplate<String, Object> redisTemplate() {
RedisTemplate<String, Object> template = new RedisTemplate<>();
template.setConnectionFactory(connectionFactory());
template.setKeySerializer(new StringRedisSerializer());
template.setValueSerializer(new StringRedisSerializer());
return template;
}
}
This class I have created two methods, one for saving and getting data from Redis
#RequiredArgsConstructor
#Service
public class AccountDtoCache {
private static final String HASH_KEY_ACCOUNT = "Account";
private static HashOperations<String, String, AccountDto> hashOperations;
private final RedisTemplate<String, Object> redisTemplate;
#PostConstruct
private synchronized void init() {
hashOperations = redisTemplate.opsForHash();
}
public void save(AccountDto account) {
hashOperations.put(HASH_KEY_ACCOUNT, account.getAccountNo(), account);
}
public static AccountDto getByAccountNo(String account) {
return hashOperations.get(HASH_KEY_ACCOUNT, account);
}
}
I try to get data from Redis as below.
When I debug, it's always null in other projects, but it's working with the same project that I have saved data into Redis.
AccountDtoCache.getByAccountNo("accountNo");
I fixed the issue above with two steps as below
I changed both of the projects package names to the same package name
Redis config
#Bean
public RedisTemplate<String, Object> redisTemplate(RedisConnectionFactory redisConnectionFactory) {
RedisTemplate<String, Object> template = new RedisTemplate<>();
GenericJackson2JsonRedisSerializer serializer = new GenericJackson2JsonRedisSerializer();
template.setConnectionFactory(redisConnectionFactory);
template.setKeySerializer(new StringRedisSerializer());
template.setValueSerializer(new StringRedisSerializer());
template.setHashKeySerializer(serializer);
template.setHashValueSerializer(serializer);
return template;
}
I have a multi-master redis cluster configuration hosted in Kubernetes cluster. We connect to the redis via hostname of load balancer using Jedis Library. hostname is prod-redis-cluster.xyz.com. The following is the code I use to connect to the Redis environment.
When there is a patching to redis nodes, IP address of the load balancer does not change whereas the IP address of the actual Redis nodes behind the load balancer changes. However, application loses connectivity to the cluster and it re-establishes only when we restart the application. Is there any other configuration needed here?
#Configuration
public class CacheConfig {
#Value("${jedis.nodes}")
private String hostName;
#Value("${jedis.password}")
private String password;
#Value("${jedis.timeout}")
private long timeout;
#Bean
public RedisClusterConfiguration redisClusterConfiguration() {
RedisClusterConfiguration redisClusterConfiguration = new RedisClusterConfiguration();
String[] hostOrPort = hostName.split(":");
Set<RedisNode> nodeList = new HashSet<>();
nodeList.add(new RedisNode(hostOrPort[0], Integer.parseInt(hostOrPort[1])));
redisClusterConfiguration.setClusterNodes(nodeList);
redisClusterConfiguration.setPassword(password);
return redisClusterConfiguration;
}
#Bean
public JedisPoolConfig jedisPoolConfig() {
JedisPoolConfig poolConfig = new JedisPoolConfig();
poolConfig.setMaxIdle(50);
poolConfig.setMaxTotal(50);
poolConfig.setTestOnCreate(true);
poolConfig.setTestOnBorrow(true);
poolConfig.setTestOnReturn(true);
poolConfig.setTestWhileIdle(true);
return poolConfig;
}
#Bean("myJedisConnectionFactory")
public JedisConnectionFactory jedisConnectionFactory(RedisClusterConfiguration redisClusterConfiguration,
JedisPoolConfig jedisPoolConfig) {
Duration duration = Duration.ofMillis(timeout);
JedisClientConfiguration clientConfiguration = JedisClientConfiguration.builder().
connectTimeout(duration)
.readTimeout(duration).usePooling()
.poolConfig(jedisPoolConfig).build();
JedisConnectionFactory jedisConnectionFactory = new JedisConnectionFactory(
redisClusterConfiguration, clientConfiguration);
return jedisConnectionFactory;
}
#Bean("redisTemplate")
public RedisTemplate<String, String> getInfo(#Qualifier("myJedisConnectionFactory")
JedisConnectionFactory jedisConnectionFactory) {
RedisTemplate<String, String> template = new RedisTemplate<String, String>();
template.setDefaultSerializer(new StringRedisSerializer());
template.setHashKeySerializer(new StringRedisSerializer());
template.setKeySerializer(new StringRedisSerializer());
template.setConnectionFactory(jedisConnectionFactory);
template.afterPropertiesSet();
return template;
}
}
I have implemented Redis session management using LettuceConnectionFactory on my Spring Boot java application. I couldn't figure out a way to capture Redis connection failure. Is there a way to capture the connection failure?
Spring Boot version: 2.2.6
Code:
#Configuration
#PropertySource("classpath:application.properties")
#EnableRedisHttpSession
public class HttpSessionConfig extends AbstractHttpSessionApplicationInitializer {
Logger logger = LoggerFactory.getLogger(HttpSessionConfig.class);
#Value("${spring.redis.host}")
private String redisHostName;
#Value("${spring.redis.port}")
private int redisPort;
#Value("${spring.redis.password}")
private String redisPassword;
private #Value("${spring.redis.custom.command.timeout}")
Duration redisCommandTimeout;
private #Value("${spring.redis.timeout}")
Duration socketTimeout;
#Bean
LettuceConnectionFactory lettuceConnectionFactory() {
final SocketOptions socketOptions = SocketOptions.builder().connectTimeout(socketTimeout).build();
final ClientOptions clientOptions = ClientOptions.builder()
.socketOptions(socketOptions)
.build();
LettuceClientConfiguration clientConfig = LettuceClientConfiguration.builder()
.commandTimeout(redisCommandTimeout)
.clientOptions(clientOptions)
.readFrom(ReadFrom.SLAVE_PREFERRED)
.build();
RedisStandaloneConfiguration serverConfig = new RedisStandaloneConfiguration(redisHostName,
redisPort);
final LettuceConnectionFactory lettuceConnectionFactory = new LettuceConnectionFactory(serverConfig,
clientConfig);
return lettuceConnectionFactory;
}
#Bean
public RedisTemplate<Object, Object> sessionRedisTemplate() {
final RedisTemplate<Object, Object> template = new RedisTemplate<>();
template.setConnectionFactory(lettuceConnectionFactory());
return template;
}
#Bean
public ConfigureRedisAction configureRedisAction() {
return ConfigureRedisAction.NO_OP;
}
}
Trying to use SpringBoot with SpringData with Elasticache:
application.properties:
spring.redis.host=XXXX-dev.XXXX.clusXXXcfg.XXX.cache.amazonaws.com
spring.redis.port=6379
CacheConfiguration:
#Configuration
#PropertySource("classpath:application.properties")
public class CacheConfiguration {
#Value("${spring.redis.host}")
private String redisHostName;
#Bean
public RedisTemplate<String, Company> redisTemplate() {
RedisTemplate<String, Company> template = new RedisTemplate();
template.setConnectionFactory(jedisConnectionFactory());
return template;
}
#Bean
JedisConnectionFactory jedisConnectionFactory() {
JedisConnectionFactory factory = new JedisConnectionFactory();
factory.setHostName(redisHostName);
factory.setUsePool(true);
return factory;
}
#Bean
public static PropertySourcesPlaceholderConfigurer propertySourcesPlaceholderConfigurer() {
return new PropertySourcesPlaceholderConfigurer();
}
}
Service call:
#Autowired
RedisTemplate<String, Company> redisTemplate;
private ValueOperations valueOperations;
#PostConstruct
private void init() {
valueOperations = redisTemplate.opsForValue();
}
#Override
public String createOtp(Company company) {
String token = UUID.randomUUID().toString();
valueOperations.set(token, company);
valueOperations.getOperations().expire(token, 5, TimeUnit.MINUTES);
return token;
}
Error:
org.springframework.data.redis.ClusterRedirectException: Redirect: slot 7228 to 10...:6379.*
redis.clients.jedis.exceptions.JedisMovedDataException: MOVED 7228 10...:6379.*
The question is - what is wrong with configuration?
You're running your Elasticache in Redis Cluster mode (only Redis Cluster responds with MOVED) but the connection factory is configured in standalone mode.
Spring Boot can auto-configure all the things you've set up manually for you. Basically, remove your CacheConfiguration class (or at least remove the majority of code):
#Configuration
public class CacheConfiguration {
#Bean
public RedisTemplate<String, Company> redisTemplate(RedisConnectionFactory connectionFactory) {
RedisTemplate<String, Company> template = new RedisTemplate();
template.setConnectionFactory(connectionFactory);
return template;
}
}
And then configure the following properties in your application.properties file:
spring.redis.cluster.nodes=<node_host>:<port> # Comma-separated list of "host:port" pairs to bootstrap from.
Spring Boot loads application.properties by default and the Redis auto-config configures a RedisTemplate<Object, Object> bean by default. Specializing beans is a valid use-case – do not duplicate what's already provided by the auto-config, especially if you want to achieve what auto-config does.
See also:
Common application properties
Externalized Configuration
I need to cache multiple types like:
public Country findCountry(String countryName)
and:
public List<Destination> findAllDestinations(String countryName)
I am using RedisCacheManager and RedisTemplate only support only one serializer.
It is solved now after some research.
change spring-data-redis to 1.4.2.RELEASE
extend RedisCacheManager with your class with cache map to serializer (cacheName->serializer) and caches names
overrides the getCache method(Cache getCache(String name)) and based on cache name, set the serializer name in the redis template
use your customized cache manager
Example -
public class CustomRedisCacheManager extends RedisCacheManager
{
public static final String CACHE_NAME_DEFAULT = "default";
public static final String CACHE_NAME_COUNTRY = "country";
public static final String CACHE_NAME_DESTINATIONS = "destinations";
private Map<String, RedisCache> redisCaches = new HashMap<>();
public CustomRedisCacheManager(Map<String, RedisTemplate> redisTemplates)
{
super(redisTemplates.get(CACHE_NAME_DEFAULT), redisTemplates.keySet());
redisTemplates.keySet().stream().forEach(cacheName -> redisCaches.put(cacheName, new RedisCache(cacheName, null, redisTemplates.get(cacheName), 0)));
}
#Override
public Cache getCache(String cacheName)
{
return redisCaches.get(cacheName);
}
}
#Configuration
#EnableCaching
public class RedisConfiguration extends CachingConfigurerSupport
{
#Bean
public JedisConnectionFactory jedisConnectionFactory()
{
JedisConnectionFactory factory = new JedisConnectionFactory();
factory.setHostName(redisHostName);
factory.setPort(redisPort);
factory.setTimeout(100);
return factory;
}
#Bean
public CacheManager cacheManager()
{
Map<String, RedisTemplate> templates = new HashMap<>();
templates.put(CACHE_NAME_DEFAULT, getDefaultRedisTemplate());
templates.put(CACHE_NAME_COUNTRY, getMetadataRedisTemplate());
templates.put(CACHE_NAME_DESTINATIONS, getDestinationsRedisTemplate());
SabreRedisCacheManager sabreRedisCacheManager = new SabreRedisCacheManager(templates);
return sabreRedisCacheManager;
}
#Bean
public RedisTemplate<Object, Object> getDefaultRedisTemplate()
{
return getBaseRedisTemplate();
}
#Bean
public RedisTemplate<Object, Object> getCountryRedisTemplate()
{
RedisTemplate<Object, Object> redisTemplate = getBaseRedisTemplate();
redisTemplate.setValueSerializer(jsonRedisSerializer(Country.class));
return redisTemplate;
}
#Bean
public RedisTemplate<Object, Object> getDestinationsRedisTemplate()
{
RedisTemplate<Object, Object> redisTemplate = getBaseRedisTemplate();
redisTemplate.setValueSerializer(jsonRedisSerializer(TypeFactory.defaultInstance().constructCollectionType(List.class, Destination.class)));
return redisTemplate;
}
private RedisTemplate<Object, Object> getBaseRedisTemplate()
{
RedisTemplate<Object, Object> redisTemplate = new RedisTemplate<>();
redisTemplate.setConnectionFactory(jedisConnectionFactory());
redisTemplate.setKeySerializer(stringRedisSerializer());
redisTemplate.setHashKeySerializer(stringRedisSerializer());
redisTemplate.setValueSerializer(jsonRedisSerializer(Object.class));
return redisTemplate;
}
private Jackson2JsonRedisSerializer jsonRedisSerializer(Class type)
{
return jsonRedisSerializer(TypeFactory.defaultInstance().constructType(type));
}
private Jackson2JsonRedisSerializer jsonRedisSerializer(JavaType javaType)
{
Jackson2JsonRedisSerializer jackson2JsonRedisSerializer = new Jackson2JsonRedisSerializer(javaType);
jackson2JsonRedisSerializer.setObjectMapper(new JsonObjectMapper());
return jackson2JsonRedisSerializer;
}
}