I have Spring Boot and cacheable method:
#Cacheable("ids")
public List<String> getValidIds() {
return list_from_another_service;
}
I have the cache eviction config:
#Scheduled(fixedRateString = "PT60M")
#CacheEvict(value = "ids", allEntries = true)
public void evictCache() {
//Do nothing.
}
Question:
The cache is evicted each 1 hr. What should the cache return if getValidIds() method receives an error during cache refreshing?
Will the cache be empty for 1 hour or will it still store and return the data from the previous successful call?
Related
I am trying to cache Kafka Records within 3 minutes of interval post that it will get expired and removed from the cache.
Each incoming records which is fetched using kafka consumer written in springboot needs to be updated in cache first then if it is present i need to discard the next duplicate records if it matches the cache record.
I have tried using Caffeine cache as below,
#EnableCaching
public class AppCacheManagerConfig {
#Bean
public CacheManager cacheManager(Ticker ticker) {
CaffeineCache bookCache = buildCache("declineRecords", ticker, 3);
SimpleCacheManager cacheManager = new SimpleCacheManager();
cacheManager.setCaches(Collections.singletonList(bookCache));
return cacheManager;
}
private CaffeineCache buildCache(String name, Ticker ticker, int minutesToExpire) {
return new CaffeineCache(name, Caffeine.newBuilder().expireAfterWrite(minutesToExpire, TimeUnit.MINUTES)
.maximumSize(100).ticker(ticker).build());
}
#Bean
public Ticker ticker() {
return Ticker.systemTicker();
}
}
and my Kafka Consumer is as below,
#Autowired
CachingServiceImpl cachingService;
#KafkaListener(topics = "#{'${spring.kafka.consumer.topic}'}", concurrency = "#{'${spring.kafka.consumer.concurrentConsumers}'}", errorHandler = "#{'${spring.kafka.consumer.errorHandler}'}")
public void consume(Message<?> message, Acknowledgment acknowledgment,
#Header(KafkaHeaders.RECEIVED_TIMESTAMP) long createTime) {
logger.info("Recieved Message: " + message.getPayload());
try {
boolean approveTopic = false;
boolean duplicateRecord = false;
if (cachingService.isDuplicateCheck(declineRecord)) {
//do something with records
}
else
{
//do something with records
}
cachingService.putInCache(xmlJSONObj, declineRecord, time);
and my caching service is as below,
#Component
public class CachingServiceImpl {
private static final Logger logger = LoggerFactory.getLogger(CachingServiceImpl.class);
#Autowired
CacheManager cacheManager;
#Cacheable(value = "declineRecords", key = "#declineRecord", sync = true)
public String putInCache(JSONObject xmlJSONObj, String declineRecord, String time) {
logger.info("Record is Cached for 3 minutes interval check", declineRecord);
cacheManager.getCache("declineRecords").put(declineRecord, time);
return declineRecord;
}
public boolean isDuplicateCheck(String declineRecord) {
if (null != cacheManager.getCache("declineRecords").get(declineRecord)) {
return true;
}
return false;
}
}
But Each time a record comes in consumer my cache is always empty. Its not holding the records.
Modifications Done:
I have added Configuration file as below after going through the suggestions and more kind of R&D removed some of the earlier logic and now the caching is working as expected but duplicate check is failing when all the three consumers are sending the same records.
`
#Configuration
public class AppCacheManagerConfig {
public static Cache<String, Object> jsonCache =
Caffeine.newBuilder().expireAfterWrite(3, TimeUnit.MINUTES)
.maximumSize(10000).recordStats().build();
#Bean
public CacheLoader<Object, Object> cacheLoader() {
CacheLoader<Object, Object> cacheLoader = new CacheLoader<Object, Object>() {
#Override
public Object load(Object key) throws Exception {
return null;
}
#Override
public Object reload(Object key, Object oldValue) throws Exception {
return oldValue;
}
};
return cacheLoader;
}
`
Now i am using the above cache as manual put and get.
I guess you're trying to implement records deduplication for Kafka.
Here is the similar discussion:
https://github.com/spring-projects/spring-kafka/issues/80
Here is the current abstract class which you may extend to achieve the necessary result:
https://github.com/spring-projects/spring-kafka/blob/master/spring-kafka/src/main/java/org/springframework/kafka/listener/adapter/AbstractFilteringMessageListener.java
Your caching service is definitely incorrect: Cacheable annotation allows marking the data getters and setters, to add caching through AOP. While in the code you clearly implement some low-level cache updating logic of your own.
At least next possible changes may help you:
Remove #Cacheable. You don't need it because you work with cache manually, so it may be the source of conflicts (especially as soon as you use sync = true). If it helps, remove #EnableCaching as well - it enables support for cache-related Spring annotations which you don't need here.
Try removing Ticker bean with the appropriate parameters for other beans. It should not be harmful as per your configuration, but usually it's helpful only for tests, no need to define it otherwise.
Double-check what is declineRecord. If it's a serialized object, ensure that serialization works properly.
Add recordStats() for cache and output stats() to log for further analysis.
I have a simple cache intended for storing Guava RateLimiter instances by IP. See code block below. The put() call does not put anything into cache. Is there some limitation against storing a RateLimiter in a Guava Cache? Is there something obvious I'm missing?
#Component
public class MyRateLimiter {
public static final Logger LOGGER = LoggerFactory.getLogger(MyRateLimiter.class);
public static long CACHE_SIZE = 1000L;
public static long TIMEOUT = 10L;
private static Cache<String, RateLimiter> cache = CacheBuilder.newBuilder()
.maximumSize(CACHE_SIZE)
.expireAfterWrite(TIMEOUT, TimeUnit.MINUTES)
.build();
public boolean tryAcquire(String key, double secondsBeforeNextOperation) {
RateLimiter rateLimiter = cache.getIfPresent(key);
if (rateLimiter == null) {
rateLimiter = getNewRateLimiter(secondsBeforeNextOperation);
cache.put(key, rateLimiter); // <- This executes..
}
return rateLimiter.tryAcquire(); // <- But cache is still empty at breakpoint here
}
private RateLimiter getNewRateLimiter(double secondsBeforeNextOperation) {
return RateLimiter.create(1 / secondsBeforeNextOperation);
}
}
This code happens to run in a Spring Component but it is singleton-scoped by default and the cache is static. Furthermore, I set a breakpoint on the return rateLimiter.tryAcquire() line and cache is still empty, even one line of code after the cache.put() line just executed.
JVM is Java 8 and I'm running in Spring Boot.
---UPDATE---
Here is my tryAcquire() method where I use get(K, Callable<V>):
public boolean tryAcquire(String key, double secondsBeforeNextOperation) {
RateLimiter rateLimiter = null;
try {
rateLimiter = cache.get(key, () ->
getNewRateLimiter(secondsBeforeNextOperation));
} catch (ExecutionException e) {
LOGGER.warn("Throttling cache was not able to be read.");
return false;
}
return rateLimiter.tryAcquire(); // <-- cache still empty at this breakpoint
}
Faced with the same problem and decided to debug it, the deletion occurs at the end of the logic of the put method in the "evictEntries" method because maxSegmentWeight = 0, add a custom weigher which return 0
private final Cache<Long, Object> cache =
CacheBuilder.newBuilder()
.expireAfterWrite(60, TimeUnit.SECONDS)
.weigher((key, value) -> 0)
.maximumWeight(1000L)
.build();
Exactly how are you determining that the cache is empty on the line with your breakpoint comment? I'd be very interested to see the result of printing/logging the value of cache.asMap() at that same place.
This was happening to me because the expireAfterWrite was being set as 0
So, I was looking at caching methods in Java (Spring). And Guava looked like it would solve the purpose.
This is the usecase -
I query for some data from a remote service. Kind of configuration field for my application. This field will be used by every inbound request to my application. And it would be expensive to call the remote service everytime as it's kind of constant which changes periodically.
So, on the first request inbound to my application, when I call remote service, I would cache the value. I set an expiry time of this cache as 30 mins. After 30 mins when the cache is expired and there is a request to retrieve the key, I would like a callback or something to do the operation of calling the remote service and setting the cache and return the value for that key.
How can I do it in Guava cache?
Here i give a example how to use guava cache. If you want to handle removal listener then need to call cleanUp. Here i run a thread which one call clean up every 30 minutes.
import com.google.common.cache.*;
import org.springframework.stereotype.Component;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
#Component
public class Cache {
public static LoadingCache<String, String> REQUIRED_CACHE;
public Cache(){
RemovalListener<String,String> REMOVAL_LISTENER = new RemovalListener<String, String>() {
#Override
public void onRemoval(RemovalNotification<String, String> notification) {
if(notification.getCause() == RemovalCause.EXPIRED){
//do as per your requirement
}
}
};
CacheLoader<String,String> LOADER = new CacheLoader<String, String>() {
#Override
public String load(String key) throws Exception {
return null; // return as per your requirement. if key value is not found
}
};
REQUIRED_CACHE = CacheBuilder.newBuilder().maximumSize(100000000)
.expireAfterWrite(30, TimeUnit.MINUTES)
.removalListener(REMOVAL_LISTENER)
.build(LOADER);
Executors.newSingleThreadExecutor().submit(()->{
while (true) {
REQUIRED_CACHE.cleanUp(); // need to call clean up for removal listener
TimeUnit.MINUTES.sleep(30L);
}
});
}
}
put & get data:
Cache.REQUIRED_CACHE.get("key");
Cache.REQUIRED_CACHE.put("key","value");
I am working on a local classifieds website and right now every time a page loads the database gets queried.
I have noticed that other popular classifieds websites serve a cached version of their site, which would greatly reduce the load time and server load.
How can I achieve this with Spring Boot or Tomcat? I want the website's cache to update every X minutes.
I am using Thymeleaf as my template engine
First you should add org.springframework.boot:spring-boot-starter-cache to your dependencies in build.gradle or pom.xml.
Let's say you're using DataService to get data to feed your view. You can put #Cacheable annotation on it.
#Service
class DataService {
#Cacheable("cache")
String compute() {
return "something"
}
}
Then you should add the following configuration:
#EnableCaching
#Configuration
public class CacheConfiguration {
public static final String CACHE_NAME = "cache";
#Bean
public CacheManager cacheManager() {
ConcurrentMapCacheManager cacheManager = new ConcurrentMapCacheManager(CACHE_NAME);
return cacheManager;
}
#CacheEvict(allEntries = true, value = CACHE_NAME)
#Scheduled(fixedDelay = 10* 60 * 1000 , initialDelay = 500)
public void evictCache() {}
}
Every 10 minutes cache will be cleared.
I am having trouble using caching in Grails. Caching is working fine in that the second time I request an object, it gets served from the cache.
However, I am unable to get timeToLive to work. I have set it to 15 seconds and I expect the body of expensiveGetMethod() to be executed when I call it after more than 15 seconds.
But no matter how long I wait between calls, the object is always being served from the cache.
What am I doing wrong here? Why is my object never getting evicted from the cache?
My service class, with the expensive method:
class MyObjectService {
#Cacheable(value='myCache')
public expensiveGetMethod(String id) {
println "+++++++++++++++++ This is an expensive method call for $id"
new MyObject(id: id)
}
}
My domain object:
class MyObject implements Serializable {
private static final long serialVersionUID = 1
String id
String name
}
My service.
grails-app\conf\Config.groovy looks like this:
springcache {
defaults {
overflowToDisk = false
memoryStoreEvictionPolicy = "LRU"
eternal = false
maxElementsInMemory = 10
}
caches {
myCache {
timeToLive = 15
timeToIdle = 15
}
}
}