I'm currently having troubles with my redis cache configuration.
I was previously using a Redis CRUD repository with RedisHash objects. Everything was working fine.
I need to use #cacheable annotation for stuff which aren't linked to my crud repository.
So I had cache configuration with #EnableCaching annotation
#Bean
public RedisCacheManagerBuilderCustomizer redisCacheManagerBuilderCustomizer() {
return (builder) -> builder
.withCacheConfiguration("default",
RedisCacheConfiguration.defaultCacheConfig())
.withCacheConfiguration("ttlCache",
RedisCacheConfiguration.defaultCacheConfig().entryTtl(Duration.ofSeconds(10)));
}
Everything about my cache configuration is OK.
But now there is some trouble to store entities in the redis.
After searching it seems that my RedisHash object have to implements Serializable. Ok I do it.
But now all my methods in the repository are doing some strange thing (essentialy the get return empty). When i look in my redis repository i see some new items indicating that a cache is used for my repository.
My question is, is there a way to disable usage of cache for my repository ?
Thanks in advance.
Related
Let's assume we have a standard Spring Boot application with JPA. We have repositories, services and REST controllers. On the service layer, we have this:
#Service
public class UserService {
#Autowired
private UserRepository userRepo;
#Transactional(readOnly = true)
public User getUserById(userId: String) {
return this.userRepo.findById(userId).orElse(null);
}
#Transactional(readOnly = false)
public User saveUser(User user){
this.userRepo.save(user);
}
}
We want to cache the result of getUserById, either via #Cacheable or via an explicit cache. We have the following two options:
If a call to saveUser(...) occurs, we call repo.save(user) and then we immediately put the saved user into our cache (write-through).
If a call to saveUser(...) occurs, we invalidate the corresponding entry in the cache. Once the user is requested by ID again, we read it from the database and put it into the cache.
Both methods have issues:
If we write-through, we risk that the database transaction fails at some point after the repo.save(user) call and is rolled back. We may have written a version of the user into our cache that never hit the database. Our cache is out-of-sync with the database.
If we only invalidate the cache, there is a time period between the invalidation and the transaction commit where a concurrent transaction may call getUserById(...). This transaction will still read the old version of the user and write it into the cache. The result is that we have outdated data in our cache.
Is the built-in spring cache susceptible to these issues as well? How do you avoid such problems in your application?
Ok so I got confused here. There is the Spring Boot side of caching, which is supposed to cache method results, and then there is the 2nd level cache of the JPA provider. These are different things with different purposes. My bad for mixing them up.
I am using RedisTemplate for caching purpose in my spring boot service. Now I want to check cache hit/cache miss through end point actuator/prometheus. But can not see cache hit/cache miss for the cache.
The code I have written is something like below
#EnableCaching
#Configuration
public class CachingConfiguration {
#Bean
public RedisTemplate<String, SomeData> redisTemplate(LettuceConnectionFactory connectionFactory, ObjectMapper objectMapper)
{
RedisTemplate<String, SomeData> template = new RedisTemplate<>();
template.setConnectionFactory(connectionFactory);
var valueSerializer = new Jackson2JsonRedisSerializer<SomeData>(SomeData.class);
valueSerializer.setObjectMapper(objectMapper);
template.setValueSerializer(valueSerializer);
return template;
}
}
Now am doing like below to get and save into cache
to get:-
redisTemplate.opsForValue().get(key);
And to save:-
redisTemplate.opsForValue().set(key, obj, some_time_limit);
My cache is working properly, am getting able to save into cache and getting proper data.
But I don't see cache hit/miss related data inside actuator/prometheus.
In my application.yml file I have added below
cache:
redis:
enable-statistics: 'true'
I would assume that in order for Springboot Cache Monitoring to apply (Including Hits/Misses), you would need to depend on AutoConfiguration.
In your case you are creating the RedisTemplate yourself, and probably enable-statistics is not actually applied.
Can you remove the redistemplate creation and use #Cacheable annotation abstraction? That way any supported Cache library will work out of the box, without you having to create #Bean and manually configuring it.
Otherwise, generally if you wanted to enable statistics on a cache manager manually, you will need to call RedisCacheManager.RedisCacheManagerBuilder enableStatistics():
https://docs.spring.io/spring-data/redis/docs/current/api/org/springframework/data/redis/cache/RedisCacheManager.RedisCacheManagerBuilder.html
For Reference:
Auto-configuration enables the instrumentation of all available Cache
instances on startup, with metrics prefixed with cache. Cache
instrumentation is standardized for a basic set of metrics.
Additional, cache-specific metrics are also available.
Metrics are tagged by the name of the cache and by the name of the
CacheManager, which is derived from the bean name.
Only caches that are configured on startup are bound to the registry. For caches not
defined in the cache’s configuration, such as caches created on the
fly or programmatically after the startup phase, an explicit
registration is required. A CacheMetricsRegistrar bean is made
available to make that process easier.
I had exactly the same question and spent a good number of hours trying to figure out how to enable cache metrics for my manually created RedisTemplate instance.
What I eventually realised is that it's only RedisCache class which collects and exposes CacheStatistics through getStatistics() method. As far as I can see there is nothing like that for RedisTemplate, which means you either need to switch to using RedisCache through RedisCacheManager and #Cacheable annotation or implement your custom metrics collection.
My system is a dynamic telemetry system. We have hundreds of different spiders all sending telemetry back to the SpringBoot server, Everything is dynamic, driven by json files in Mongo, including the UI. We don't build the UI, as opposed to individual teams can configure their own UI for their needs, all by editing json docs.
We have the majority of the UI running and i began the middleware piece. We are using Spring Boot for the first time along with Spring Data Mongo with several MQ listeners for events. The problem is Spring Data. I started reading the docs on it and I realized the docs do not address using it without POJO's. I have this wonderfully dynamic model that changes per user per minute if the telemetry spiders dictate, I couldn't shackle this to a POJO if I tried. Is there a way to use Spring Data with a Map?
It seems from my experiments that the big issue is there is no way to tell the CRUD routines of the repository class what collection to query without a POJO.
Are my suspicions correct in that this won't work and am I better off ditching Spring Data and using the Mongo driver directly?
I don't think you can do it without a pojo when using spring-data. The least you could do is this
public interface NoPojoRepository extends MongoRepository<DummyPojo, String> {
}
and create a dummy pojo with just id and a Map.
#Data
public class DummyPojo {
#Id
private String id;
private Map<String, Object> value;
}
Since this value field is a map, you can store pretty much anything.
I am working on a project that has a requirement of Pivotal GemFire.
I am unable to find a proper tutorial about how to configure gemFire with Spring Boot.
I have created a partitioned Region and I want to configure Locators as well, but I need only server-side configuration as client is handled by someone else.
I am totally new to Pivotal GemFire and really confused. I have tried creating a cache.xml but then somehow a cache.out.xml gets created and there are many issues.
#Priyanka-
Best place to start is with the Guides on spring.io. Specifically, have a look at...
"Accessing Data with GemFire"
There is also...
"Cache Data with GemFire", and...
"Accessing GemFire Data with REST"
However, these guides focus mostly on "client-side" application concerns, "data access" (over REST), "caching", etc.
Still, you can use Spring Data GemFire (in a Spring Boot application even) to configure a GemFire Server. I have many examples of this. One in particular...
"Spring Boot GemFire Server Example"
This example demonstrates how to bootstrap a Spring Boot application as a GemFire Server (technically, a peer node in the cluster). Additionally, the GemFire properties are specified Spring config and can use Spring's normal conventions (property placeholders, SpEL expression) to configure these properties, like so...
https://github.com/jxblum/spring-boot-gemfire-server-example/blob/master/src/main/java/org/example/SpringBootGemFireServer.java#L59-L84
This particular configuration makes the GemFire Server a "GemFire Manager", possibly with an embedded "Locator" (indicated by the start-locator GemFie property, not to be confused with the "locators" GemFire property which allows our node to join and "existing" cluster) as well as a GemFire CacheServer to serve GemFire cache clients (with a ClientCache).
This example creates a "Factorials" Region, with a CacheLoader (definition here) to populate the "Factorials" Region on cache misses.
Since this example starts an embedded GemFire Manager in the Spring Boot GemFire Server application process, you can even connect to it using Gfsh, like so...
gfsh> connect --jmx-manager=localhost[1099]
Then you can run "gets" on the "Factorial" Region to see it compute factorials of the numeric keys you give it.
To see more advanced configuration, have a look at my other repos, in particular the Contacts Application RI (here).
Hope this helps!
-John
Well, I had the same problem, let me share with you what worked for me, in this case I'm using Spring Boot and Pivotal GemFire as cache client.
Install and run GemFire
Read the 15 minutes quick start guide
Create a locator(let's call it locator1) and a server(server1) and a region(region1)
Go to the folder where you started the 'Gee Fish'(gfsh) and then go to the locator's folder and open the log file, in that file you can get the port your locator is using.
Now let's see the Spring boot side:
In you Application with the main method add the #EnablegemFireCaching annotation
In the method(wherever it is) you want to cache, add the #Cacheable("region1") annotation.
Now let's create a configuration file for the caching:
//this is my working class
#Configuration
public class CacheConfiguration {
#Bean
ClientCacheFactoryBean gemfireCacheClient() {
return new ClientCacheFactoryBean();
}
#Bean(name = GemfireConstants.DEFAULT_GEMFIRE_POOL_NAME)
PoolFactoryBean gemfirePool() {
PoolFactoryBean gemfirePool = new PoolFactoryBean();
gemfirePool.addLocators(Collections.singletonList(new ConnectionEndpoint("localhost", HERE_GOES_THE_PORT_NUMBER_FROM_STEP_4)));
gemfirePool.setName(GemfireConstants.DEFAULT_GEMFIRE_POOL_NAME);
gemfirePool.setKeepAlive(false);
gemfirePool.setPingInterval(TimeUnit.SECONDS.toMillis(5));
gemfirePool.setRetryAttempts(1);
gemfirePool.setSubscriptionEnabled(true);
gemfirePool.setThreadLocalConnections(false);
return gemfirePool;
}
#Bean
ClientRegionFactoryBean<Long, Long> getRegion(ClientCache gemfireCache, Pool gemfirePool) {
ClientRegionFactoryBean<Long, Long> region = new ClientRegionFactoryBean<>();
region.setName("region1");
region.setLookupEnabled(true);
region.setCache(gemfireCache);
region.setPool(gemfirePool);
region.setShortcut(ClientRegionShortcut.PROXY);
return region;
}
That's all!, also do not forget to serialize(implements Serializable) the class is being cached(The class your cached method is returning)
I am using spring cache mechanism with SimpleCacheManager/ConcurrentMapCache.
And I am using a web service to clear the cache and the following is the code .
for(String cacheName : cacheManager.getCacheNames()){
Cache cache =cacheManager.getCache(cacheName);
if(cache!=null){
cache.clear();
}
}
When I called this code from a Rest webservice on local vm , I can see its clearing the cache and can see the changes that we done in the database with other service , However on the production environment , the webservice returning 200 status in the logs. but it still shows the old data.
On production we have 2 servers
We have to restart our application to refresh the cache and get the latest data from the database.
I used to do this creating a void method annotated with #CacheEvict(allEntries=true), this annotation is similar to #CacheRemoveAll from JSR-107.
Something like that:
#CacheEvict(allEntries=true)
public void evictAll() {
// Do nothing
}
I know, it's ugly, but works to me.
My two cents, avoid to use the default spring cache manager in production, use a cache manager more sophisticated instead like Guava or EhCache.
Cheers.