How to enable distributed/clustered cache when using redis with spring data cache - spring-boot

How to enable distributed/clustered cache when using Redis with spring-boot cache.
Especially when using Redis through spring-boot-starter-data-redis

Enable caching in the spring boot app is very simple. You would need to just follow three steps.
Define cache configuration
Add EnableCaching to any configuration class
Provide a CacheManager bean
For Redis, we've RedisCacheManager that can be configured and created.
Cache Configuration
#Configuration
#Getter
#Setter
#ConfigurationProperties(prefix = "cache")
public class CacheConfigurationProperties {
// Redis host name
private String redisHost;
// Redis port
private int redisPort;
// Default TTL
private long timeoutSeconds;
// TTL per cache, add enties for each cache
private Map<String, Long> cacheTtls;
}
Set their values via properties or yaml file like
cache.redisHost=localhost
cache.redisPort=6379
cache.timeoutSeconds=1000
cache.cacheTtls.cach1=100
cache.cacheTtls.cach2=200
Once you have created configuration, you can create cache config for RedisCacheManger by builder.
#Configuration
#EnableCaching
public class CacheConfig {
private static RedisCacheConfiguration createCacheConfiguration(long timeoutInSeconds) {
return RedisCacheConfiguration.defaultCacheConfig()
.entryTtl(Duration.ofSeconds(timeoutInSeconds));
}
#Bean
public LettuceConnectionFactory redisConnectionFactory(CacheConfigurationProperties properties) {
RedisStandaloneConfiguration redisStandaloneConfiguration = new RedisStandaloneConfiguration();
redisStandaloneConfiguration.setHostName(properties.getRedisHost());
redisStandaloneConfiguration.setPort(properties.getRedisPort());
return new LettuceConnectionFactory(redisStandaloneConfiguration);
}
#Bean
public RedisCacheConfiguration cacheConfiguration(CacheConfigurationProperties properties) {
return createCacheConfiguration(properties.getTimeoutSeconds());
}
#Bean
public CacheManager cacheManager(
RedisConnectionFactory redisConnectionFactory, CacheConfigurationProperties properties) {
Map<String, RedisCacheConfiguration> cacheConfigurations = new HashMap<>();
for (Entry<String, Long> cacheNameAndTimeout : properties.getCacheTtls().entrySet()) {
cacheConfigurations.put(
cacheNameAndTimeout.getKey(), createCacheConfiguration(cacheNameAndTimeout.getValue()));
}
return RedisCacheManager.builder(redisConnectionFactory)
.cacheDefaults(cacheConfiguration(properties))
.withInitialCacheConfigurations(cacheConfigurations)
.build();
}
}
If you're using Redis cluster than update cache properties as per that. In this some beans would become primary if you want cache specific bean than make these methods private.

Related

Multiple Redis For Spring Cache And JWT

I have actually an application that is using Redis for Cache and keeping the JWT token into redis.
My goal is to have one redis for cache, and another one for jwt token.
I dont understand how i can achieve this.
How can i say to spring to use a specific redis (here "redis-cache") for caching ?
Actually i only put #EnableCaching and it is working properly
Thanks for any help
spring:
redis:
port: 7000
password: password123
host: 127.0.0.1
redis-cache: # New One
port: 7001
password: password123
host: 127.0.0.1
I'm using redisTemplate to keep jwt token into redis
#Configuration
public class GenericBeanConfig {
#Bean
public RedisTemplate<String, Object> redisTemplate(RedisConnectionFactory connectionFactory) {
RedisTemplate<String, Object> template = new RedisTemplate<>();
Jackson2JsonRedisSerializer<String> jrs = new Jackson2JsonRedisSerializer<String>(String.class);
template.setKeySerializer(jrs);
template.setConnectionFactory(connectionFactory);
return template;
}
}
...
#EnableCaching
public class ProjectsApplication {
public static void main(String[] args) {
SpringApplication.run(ProjectsApplication .class, args);
}
}
Caching some endpoint
#Cacheable(value = "users-rbac")
public UserResponseDTO search(#PathVariable String email) {
return userService.search(email);
}
You can define your custom CacheManager as well, you define your cache manager as follows. Customize this as you see, for now host, port and password are fixed but you can read them from application config file using #Value .
#Bean
public CacheManager cacheManager() {
// create redis configuration
RedisStandaloneConfiguration configuration =
new RedisStandaloneConfiguration("127.0.0.1", 7001);
configuration.setPassword(RedisPassword.of("password"));
// create a connection factory, use other connection factory if you want
LettuceConnectionFactory factory = new LettuceConnectionFactory(configuration);
factory.afterPropertiesSet();
RedisCacheWriter writer = RedisCacheWriter.nonLockingRedisCacheWriter(factory);
// define cache cnfiguration
RedisCacheConfiguration cacheConfiguration = RedisCacheConfiguration.defaultCacheConfig();
// use Jackson serdes do not use JDK one
cacheConfiguration.serializeValuesWith(
SerializationPair.fromSerializer(new GenericJackson2JsonRedisSerializer()));
// create cache manager
return new RedisCacheManager(writer, cacheConfiguration);
}

Java JobRunr when using Spring Boot Redis Starter

How do I create and use the Redis connection that spring-boot-starter-data-redis creates? It doesn't seem like there is a Bean for RedisClient created by the default auto configuration so I'm not sure of the best way to do this.
The documentation does state that in this case you need to create the StorageProvider yourself which is fine, but can you reuse what Spring Boot has already created. I believe this would need to be a pooled connection which you would also need to enable through Spring Boot.
RedisTemplate offers a high-level abstraction for Redis interactions:
https://docs.spring.io/spring-data/data-redis/docs/current/reference/html/#redis:template
Redis autoconfiguration :
#AutoConfiguration
#ConditionalOnClass({RedisOperations.class})
#EnableConfigurationProperties({RedisProperties.class})
#Import({LettuceConnectionConfiguration.class, JedisConnectionConfiguration.class})
public class RedisAutoConfiguration {
public RedisAutoConfiguration() {
}
#Bean
#ConditionalOnMissingBean(
name = {"redisTemplate"}
)
#ConditionalOnSingleCandidate(RedisConnectionFactory.class)
public RedisTemplate<Object, Object> redisTemplate(RedisConnectionFactory redisConnectionFactory) {
RedisTemplate<Object, Object> template = new RedisTemplate();
template.setConnectionFactory(redisConnectionFactory);
return template;
}
#Bean
#ConditionalOnMissingBean
#ConditionalOnSingleCandidate(RedisConnectionFactory.class)
public StringRedisTemplate stringRedisTemplate(RedisConnectionFactory redisConnectionFactory) {
return new StringRedisTemplate(redisConnectionFactory);
}
}
Here you can find the corresponding configuration properties(including connection pool default configuration).
Simple implementation example :
https://www.baeldung.com/spring-data-redis-tutorial

Register Caffeine Cache in Spring Actuator (CacheManager)

We're using Spring Boot 2 and Spring Actuator. When creating a cache like the following:
#Bean
public CaffeineCache someCache() {
return new CaffeineCache("my-cache",
Caffeine.newBuilder()
.maximumSize(1000)
.expireAfterWrite(10, TimeUnit.SECONDS)
.build());
}
it is registered into Spring Actuator and can be accessed and handle via endpoints:
❯ http GET localhost:8080/actuator/caches
{
"cacheManagers": {
"cacheManager": {
"caches": {
"my-cache": {
"target": "com.github.benmanes.caffeine.cache.BoundedLocalCache$BoundedLocalManualCache"
}
}
}
}
}
However, this is valid when using the annotation #Cacheable - but I would like to create a cache and use it as a map.
Therefore, I can create:
#Bean
public com.github.benmanes.caffeine.cache.Cache<String, MyObject> customCache(QueryServiceProperties config) {
return Caffeine.newBuilder()
.maximumSize(10)
.expireAfterAccess(10, TimeUnit.SECONDS)
.build();
}
And it works but it cannot be discovered by Spring Actuator. Is there any way to register this kind of cache?
Addapted from this Answer I did the following:
#Autowired
private CacheMetricsRegistrar cacheMetricsRegistrar;
private LoadingCache<Key, MyObject> cache;
#PostConstruct
public void init() {
cache = Caffeine.newBuilder()
.maximumSize(10_000)
.refreshAfterWrite(cacheDuration)
.recordStats()
.build(this::loadMyObject);
// trick the compiler
Cache tmp = cache;
cacheMetricsRegistrar.bindCacheToRegistry(new CaffeineCache(CACHE_NAME, tmp), Tag.of("name", CACHE_NAME));
}
The Cache should now show up in the cache actuator endpoints, e.g. "http://localhost:8080/metrics/cache.gets"
use CacheManager
add your custom cache into CacheManager, inject CacheManager and get that cache out for your usage.
see https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-caching.html for some further details

Redis cache docker container with spring boot not working on my local Machine

I am using redis cache on my local Machine as a docker image. I have enabled the cache for one of my method using the cacheable anothion. Application is not able to cache the same when i am using on aws it is working instead of localhost
public class RedisConfig {
#Autowired
private JedisConnectionFactory jedisConnectionFactory;
#Bean
public RedisTemplate<Object, Object> redisTemplate() {
System.out.println("localhost")
System.out.println("6379");
jedisConnectionFactory.getHostName();
jedisConnectionFactory.getPort();
RedisTemplate<Object, Object> template = new RedisTemplate<>();
template.setConnectionFactory(jedisConnectionFactory);
template.setValueSerializer(new GenericToStringSerializer<Object>(Object.class));
return template;
}
#Bean
public CacheManager cacheManager(RedisTemplate redisTemplate) {
RedisCacheManager cacheManager = new RedisCacheManager(redisTemplate);
// Number of seconds before expiration. Defaults to unlimited (0)
cacheManager.setDefaultExpiration(60);
cacheManager.setUsePrefix(true);
return cacheManager;
}
}
#Cacheable(value="sgcode" , cacheManager ="cacheManager")
public String getSegmentCode(String aname ) {
Logger.info("code ", "##### SEGMENT METHOD CALLED ##### {}", aname);
return segmentCodeMap.get(aname);
}
Logger line will be printed only once after that it should fetch from cache.
Finally after lot of struggle found the root cause. use the enablecaching anotation in application class

Spring Cloud Task - specify database config

I have Spring Cloud Task that loads data from SQL Server to Cassandra DB which will be run on Spring Cloud Data Flow.
One of the requirement of Spring Task is to provide relational database to persist metadata like task execution state. But I don't want use either of the above databases for that. Instead, I have to specify third database for persistence. But it seems like Spring Cloud Task flow automatically picks up data source properties of SQL Server from application.properties. How can I specify another db for task state persistence?
My Current properties:
spring.datasource.url=jdbc:sqlserver://iphost;databaseName=dbname
spring.datasource.username=user
spring.datasource.password=password
spring.datasource.driverClassName=com.microsoft.sqlserver.jdbc.SQLServerDriver
spring.jpa.show-sql=false
#spring.jpa.hibernate.dialect=org.hibernate.dialect.SQLServer2012Dialect
spring.jpa.hibernate.naming.physical-strategy=org.hibernate.boot.model.naming.PhysicalNamingStrategyStandardImpl
spring.jpa.hibernate.ddl-auto=none
spring.data.cassandra.contact-points=ip
spring.data.cassandra.port=9042
spring.data.cassandra.username=username
spring.data.cassandra.password=password
spring.data.cassandra.keyspace-name=mykeyspace
spring.data.cassandra.schema-action=CREATE_IF_NOT_EXISTS
Update: 1
I added below code to point to 3rd database as suggested by Michael Minella. Now Spring Task is able to connect to this DB and persist state. But now my batch job source queries are also connecting to this database. Only thing I changed was to add datasource for task.
spring.task.datasource.url=jdbc:postgresql://host:5432/testdb?stringtype=unspecified
spring.task.datasource.username=user
spring.task.datasource.password=passwrod
spring.task.datasource.driverClassName=org.postgresql.Driver
#Configuration
public class DataSourceConfigs {
#Bean(name = "taskDataSource")
#ConfigurationProperties(prefix="spring.task.datasource")
public DataSource getDataSource() {
return DataSourceBuilder.create().build();
}
}
#Configuration
public class DDTaskConfigurer extends DefaultTaskConfigurer{
#Autowired
public DDTaskConfigurer(#Qualifier("taskDataSource") DataSource dataSource) {
super(dataSource);
}
}
Update #2:
#Component
#StepScope
public class MyItemReader extends RepositoryItemReader<Scan> implements InitializingBean{
#Autowired
private ScanRepository repository;
private Integer lastScanIdPulled = null;
public MyItemReader(Integer _lastIdPulled) {
super();
if(_lastIdPulled == null || _lastIdPulled <=0 ){
lastScanIdPulled = 0;
} else {
lastScanIdPulled = _lastIdPulled;
}
}
#PostConstruct
protected void setUpRepo() {
final Map<String, Sort.Direction> sorts = new HashMap<>();
sorts.put("id", Direction.ASC);
this.setRepository(this.repository);
this.setSort(sorts);
this.setMethodName("findByScanGreaterThanId");
List<Object> methodArgs = new ArrayList<Object>();
System.out.println("lastScanIdpulled >>> " + lastScanIdPulled);
if(lastScanIdPulled == null || lastScanIdPulled <=0 ){
lastScanIdPulled = 0;
}
methodArgs.add(lastScanIdPulled);
this.setArguments(methodArgs);
}
}
#Repository
public interface ScanRepository extends JpaRepository<Scan, Integer> {
#Query("...")
Page<Scan> findAllScan(final Pageable pageable);
#Query("...")
Page<Scan> findByScanGreaterThanId(int id, final Pageable pageable);
}
Update #3:
If I add config datasource for Repository, I now get below exception. Before you mention that one of the datasource needs to be declared Primary. I already tried that.
Caused by: java.lang.IllegalStateException: Expected one datasource and found 2
at org.springframework.cloud.task.batch.configuration.TaskBatchAutoConfiguration$TaskBatchExecutionListenerAutoconfiguration.taskBatchExecutionListener(TaskBatchAutoConfiguration.java:65) ~[spring-cloud-task-batch-1.0.3.RELEASE.jar:1.0.3.RELEASE]
at org.springframework.cloud.task.batch.configuration.TaskBatchAutoConfiguration$TaskBatchExecutionListenerAutoconfiguration$$EnhancerBySpringCGLIB$$baeae6b9.CGLIB$taskBatchExecutionListener$0(<generated>) ~[spring-cloud-task-batch-1.0.3.RELEASE.jar:1.0.3.RELEASE]
at org.springframework.cloud.task.batch.configuration.TaskBatchAutoConfiguration$TaskBatchExecutionListenerAutoconfiguration$$EnhancerBySpringCGLIB$$baeae6b9$$FastClassBySpringCGLIB$$5a898c9.invoke(<generated>) ~[spring-cloud-task-batch-1.0.3.RELEASE.jar:1.0.3.RELEASE]
at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228) ~[spring-core-4.3.14.RELEASE.jar:4.3.14.RELEASE]
at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:358) ~[spring-context-4.3.14.RELEASE.jar:4.3.14.RELEASE]
at org.springframework.cloud.task.batch.configuration.TaskBatchAutoConfigu
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(
entityManagerFactoryRef = "myEntityManagerFactory",
basePackages = { "com.company.dd.collector.tool" },
transactionManagerRef = "TransactionManager"
)
public class ToolDbConfig {
#Bean(name = "myEntityManagerFactory")
public LocalContainerEntityManagerFactoryBean
myEntityManagerFactory(
EntityManagerFactoryBuilder builder,
#Qualifier("ToolDataSource") DataSource dataSource
) {
return builder
.dataSource(dataSource)
.packages("com.company.dd.collector.tool")
.persistenceUnit("tooldatasource")
.build();
}
#Bean(name = "myTransactionManager")
public PlatformTransactionManager transactionManager(
#Qualifier("myEntityManagerFactory") EntityManagerFactory
entityManagerFactory
) {
return new JpaTransactionManager(entityManagerFactory);
}
}
#Configuration
public class DataSourceConfigs {
#Bean(name = "taskDataSource")
#ConfigurationProperties(prefix="spring.task.datasource")
public DataSource getDataSource() {
return DataSourceBuilder.create().build();
}
#Primary
#Bean(name = "ToolDataSource")
#ConfigurationProperties(prefix = "tool.datasource")
public DataSource dataSource() {
return DataSourceBuilder.create().build();
}
}
You need to create a TaskConfigurer to specify the DataSource to be used. You can read about this interface in the documentation here: https://docs.spring.io/spring-cloud-task/1.1.1.RELEASE/reference/htmlsingle/#features-task-configurer
The javadoc can be found here: https://docs.spring.io/spring-cloud-task/docs/current/apidocs/org/springframework/cloud/task/configuration/TaskConfigurer.html
UPDATE 1:
When using more than one DataSource, both Spring Batch and Spring Cloud Task follow the same paradigm in that they both have *Configurer interfaces that need to be used to specify what DataSource to use. For Spring Batch, you use the BatchConfigurer (typically by just extending the DefaultBatchConfigurer) and as noted above, the TaskConfigurer is used in Spring Cloud Task. This is because when there is more than one DataSource, the framework has no way of knowing which one to use.

Resources