I follow the instructions on this tutorial (https://www.baeldung.com/spring-boot-redis-cache)
#Cacheable(value = "itemCache")
public UserInfo getUserInfo(String id) {
// without explicit manipulate cache, the value can't be retrieved from Redis
UserInfo res = cacheManager.getCache("itemCache").get(id, UserInfo.class);
if (res != null) {
return res;
}
try {
... retrieve from database ...
res = convertMapToUserInfo(id, userInfoMap);
// without explicit manipulate cache, the value can't be stored in Redis
cacheManager.getCache("itemCache").put(id, res);
return res;
}
} catch (Exception e) {
...
} finally {
...
}
return null;
}
The weird thing is, I have to put/get items from Cache manualy, even I use Cacheable annotation. Without explicit manipulate cache, the returned value of getUserInfo can't be cached.
The RedisConfiguration contains code
#Bean
public RedisCacheManagerBuilderCustomizer redisCacheManagerBuilderCustomizer() {
return (builder) -> builder
.withCacheConfiguration("itemCache",
this.cacheConfiguration());
}
#Bean
public RedisCacheConfiguration cacheConfiguration() {
return RedisCacheConfiguration.defaultCacheConfig()
.disableCachingNullValues()
.serializeValuesWith(RedisSerializationContext.SerializationPair.fromSerializer(new GenericJackson2JsonRedisSerializer()));
}
And I also add #EnableCaching to my Application class. Could anyone help me figure out why my cache doesn't take effect. Thanks!
I've recently found out that if we use just one entry with the builder it does'nt work. But if you have more than one cache in your project and include all these entries in this method or even if you add a 'dummy' entry, it works. Maybe a bug or a known issue in Spring implementation, I don't know.
#Bean
public RedisCacheManagerBuilderCustomizer redisCacheManagerBuilderCustomizer() {
return (builder) -> builder
.withCacheConfiguration("itemCache", this.cacheConfiguration())
.withCacheConfiguration("dummy", this.cacheConfiguration());
}
I hope helps.
Related
I am trying to cache Kafka Records within 3 minutes of interval post that it will get expired and removed from the cache.
Each incoming records which is fetched using kafka consumer written in springboot needs to be updated in cache first then if it is present i need to discard the next duplicate records if it matches the cache record.
I have tried using Caffeine cache as below,
#EnableCaching
public class AppCacheManagerConfig {
#Bean
public CacheManager cacheManager(Ticker ticker) {
CaffeineCache bookCache = buildCache("declineRecords", ticker, 3);
SimpleCacheManager cacheManager = new SimpleCacheManager();
cacheManager.setCaches(Collections.singletonList(bookCache));
return cacheManager;
}
private CaffeineCache buildCache(String name, Ticker ticker, int minutesToExpire) {
return new CaffeineCache(name, Caffeine.newBuilder().expireAfterWrite(minutesToExpire, TimeUnit.MINUTES)
.maximumSize(100).ticker(ticker).build());
}
#Bean
public Ticker ticker() {
return Ticker.systemTicker();
}
}
and my Kafka Consumer is as below,
#Autowired
CachingServiceImpl cachingService;
#KafkaListener(topics = "#{'${spring.kafka.consumer.topic}'}", concurrency = "#{'${spring.kafka.consumer.concurrentConsumers}'}", errorHandler = "#{'${spring.kafka.consumer.errorHandler}'}")
public void consume(Message<?> message, Acknowledgment acknowledgment,
#Header(KafkaHeaders.RECEIVED_TIMESTAMP) long createTime) {
logger.info("Recieved Message: " + message.getPayload());
try {
boolean approveTopic = false;
boolean duplicateRecord = false;
if (cachingService.isDuplicateCheck(declineRecord)) {
//do something with records
}
else
{
//do something with records
}
cachingService.putInCache(xmlJSONObj, declineRecord, time);
and my caching service is as below,
#Component
public class CachingServiceImpl {
private static final Logger logger = LoggerFactory.getLogger(CachingServiceImpl.class);
#Autowired
CacheManager cacheManager;
#Cacheable(value = "declineRecords", key = "#declineRecord", sync = true)
public String putInCache(JSONObject xmlJSONObj, String declineRecord, String time) {
logger.info("Record is Cached for 3 minutes interval check", declineRecord);
cacheManager.getCache("declineRecords").put(declineRecord, time);
return declineRecord;
}
public boolean isDuplicateCheck(String declineRecord) {
if (null != cacheManager.getCache("declineRecords").get(declineRecord)) {
return true;
}
return false;
}
}
But Each time a record comes in consumer my cache is always empty. Its not holding the records.
Modifications Done:
I have added Configuration file as below after going through the suggestions and more kind of R&D removed some of the earlier logic and now the caching is working as expected but duplicate check is failing when all the three consumers are sending the same records.
`
#Configuration
public class AppCacheManagerConfig {
public static Cache<String, Object> jsonCache =
Caffeine.newBuilder().expireAfterWrite(3, TimeUnit.MINUTES)
.maximumSize(10000).recordStats().build();
#Bean
public CacheLoader<Object, Object> cacheLoader() {
CacheLoader<Object, Object> cacheLoader = new CacheLoader<Object, Object>() {
#Override
public Object load(Object key) throws Exception {
return null;
}
#Override
public Object reload(Object key, Object oldValue) throws Exception {
return oldValue;
}
};
return cacheLoader;
}
`
Now i am using the above cache as manual put and get.
I guess you're trying to implement records deduplication for Kafka.
Here is the similar discussion:
https://github.com/spring-projects/spring-kafka/issues/80
Here is the current abstract class which you may extend to achieve the necessary result:
https://github.com/spring-projects/spring-kafka/blob/master/spring-kafka/src/main/java/org/springframework/kafka/listener/adapter/AbstractFilteringMessageListener.java
Your caching service is definitely incorrect: Cacheable annotation allows marking the data getters and setters, to add caching through AOP. While in the code you clearly implement some low-level cache updating logic of your own.
At least next possible changes may help you:
Remove #Cacheable. You don't need it because you work with cache manually, so it may be the source of conflicts (especially as soon as you use sync = true). If it helps, remove #EnableCaching as well - it enables support for cache-related Spring annotations which you don't need here.
Try removing Ticker bean with the appropriate parameters for other beans. It should not be harmful as per your configuration, but usually it's helpful only for tests, no need to define it otherwise.
Double-check what is declineRecord. If it's a serialized object, ensure that serialization works properly.
Add recordStats() for cache and output stats() to log for further analysis.
I have a simple cache intended for storing Guava RateLimiter instances by IP. See code block below. The put() call does not put anything into cache. Is there some limitation against storing a RateLimiter in a Guava Cache? Is there something obvious I'm missing?
#Component
public class MyRateLimiter {
public static final Logger LOGGER = LoggerFactory.getLogger(MyRateLimiter.class);
public static long CACHE_SIZE = 1000L;
public static long TIMEOUT = 10L;
private static Cache<String, RateLimiter> cache = CacheBuilder.newBuilder()
.maximumSize(CACHE_SIZE)
.expireAfterWrite(TIMEOUT, TimeUnit.MINUTES)
.build();
public boolean tryAcquire(String key, double secondsBeforeNextOperation) {
RateLimiter rateLimiter = cache.getIfPresent(key);
if (rateLimiter == null) {
rateLimiter = getNewRateLimiter(secondsBeforeNextOperation);
cache.put(key, rateLimiter); // <- This executes..
}
return rateLimiter.tryAcquire(); // <- But cache is still empty at breakpoint here
}
private RateLimiter getNewRateLimiter(double secondsBeforeNextOperation) {
return RateLimiter.create(1 / secondsBeforeNextOperation);
}
}
This code happens to run in a Spring Component but it is singleton-scoped by default and the cache is static. Furthermore, I set a breakpoint on the return rateLimiter.tryAcquire() line and cache is still empty, even one line of code after the cache.put() line just executed.
JVM is Java 8 and I'm running in Spring Boot.
---UPDATE---
Here is my tryAcquire() method where I use get(K, Callable<V>):
public boolean tryAcquire(String key, double secondsBeforeNextOperation) {
RateLimiter rateLimiter = null;
try {
rateLimiter = cache.get(key, () ->
getNewRateLimiter(secondsBeforeNextOperation));
} catch (ExecutionException e) {
LOGGER.warn("Throttling cache was not able to be read.");
return false;
}
return rateLimiter.tryAcquire(); // <-- cache still empty at this breakpoint
}
Faced with the same problem and decided to debug it, the deletion occurs at the end of the logic of the put method in the "evictEntries" method because maxSegmentWeight = 0, add a custom weigher which return 0
private final Cache<Long, Object> cache =
CacheBuilder.newBuilder()
.expireAfterWrite(60, TimeUnit.SECONDS)
.weigher((key, value) -> 0)
.maximumWeight(1000L)
.build();
Exactly how are you determining that the cache is empty on the line with your breakpoint comment? I'd be very interested to see the result of printing/logging the value of cache.asMap() at that same place.
This was happening to me because the expireAfterWrite was being set as 0
I've been trying to make a simple backend for user login.
It works fine if I don't use threads, but when I try to implement multi-threading it breaks for some reason.
For now, I'm just trying to do something as simple as retrieving all users' info from the User table in JSON format. The problem is that the Rest controller returns nothing to Postman even though dbActions.getAll() returns the correct List.
There's no error, no exception, nothing. I could really use some help. The parts of my code I'm trying to get to work are below.
Rest controller:
#Async
#RequestMapping(value="/view", method =RequestMethod.GET)
public List<User> viewAll() {
try {
List<User> list = new ArrayList<>();
list = dbActions.getAll();
return list;
}catch(Exception e) {
e.printStackTrace();
return null;
}
}
dbActions service:
public List<User> getAll() {
List<User> results = new ArrayList<>();
CompletableFuture<Void> future;
try {
future = CompletableFuture.runAsync(new Runnable() {
public void run() {
userRepo.findAll().forEach(results::add);
}
});
future.get();
return results;
} catch (Exception e) {
e.printStackTrace();
return results;
}
}
Try removing the #Async annotation from your viewAll() method
So I've been working on an extremely simple website over the past 2 days, but I figured it would be neat to somehow link the website with a Discord bot. For the Discord bot part, I've been using the JDA library.
The issue I'm running in to is that I seem to be unable to use the save method. However, the findById and findAll seem to work perfectly fine. The way I have my code setup can be found below.
#Controller
public class IndexController extends ListenerAdapter {
private static boolean botStarted = false;
#Autowired
private NewsPostRepository newsPostRepository;
#GetMapping("/")
public String getIndex(ModelMap map) {
// TODO: add news post images.
NewsPost savedNewsPost = newsPostRepository.save(new NewsPost("Controller", "Posted through controller",
new byte[]{}, new Date(), true));
System.out.println(savedNewsPost);
return "index";
}
#GetMapping("/start")
public String startBot() {
if (!botStarted) {
try {
JDA jda = new JDABuilder(AccountType.BOT)
.setToken("my-token")
.addEventListener(this)
.buildBlocking(); // Blocking vs async
} catch (LoginException e) {
e.printStackTrace();
} catch (RateLimitedException e) {
e.printStackTrace();
} catch (InterruptedException e) {
e.printStackTrace();
} finally {
botStarted = true;
}
}
return "redirect:/";
}
#Override
public void onMessageReceived(MessageReceivedEvent messageReceivedEvent) {
User author = messageReceivedEvent.getAuthor();
MessageChannel channel = messageReceivedEvent.getChannel();
Message message = messageReceivedEvent.getMessage();
if (!author.isBot()) {
if (message.getContent().startsWith("!news")) {
NewsPost savedNewsPost = newsPostRepository.save(new NewsPost("Discord", "Posted through Discord",
new byte[]{}, new Date(), true));
System.out.println(savedNewsPost);
}
}
}
}
The repository:
public interface NewsPostRepository extends CrudRepository<NewsPost, String> {
}
The weird thing is, that when I go to the index page, the NewsPost saves perfectly fine, and is visible in the database.
When I try to use the Discord bot to add a NewsPost, it returns an object in the same way it would in the method for the index, with an ID that is not null and should be usable to find it in the database, however, this entry is nowhere to be found. No exception appears either. Keep in mind that both of these save() calls are identical.
I've tried to use a service and adding #Transactional but so far nothing has worked.
I am working with Spring and EhCache
I have the following method
#Override
#Cacheable(value="products", key="#root.target.PRODUCTS")
public Set<Product> findAll() {
return new LinkedHashSet<>(this.productRepository.findAll());
}
I have other methods working with #Cacheable and #CachePut and #CacheEvict.
Now, imagine the database returns 100 products and they are cached through key="#root.target.PRODUCTS", then other method would insert - update - deleted an item into the database. Therefore the products cached through the key="#root.target.PRODUCTS" are not the same anymore such as the database.
I mean, check the two following two methods, they are able to update/delete an item, and that same item is cached in the other key="#root.target.PRODUCTS"
#Override
#CachePut(value="products", key="#product.id")
public Product update(Product product) {
return this.productRepository.save(product);
}
#Override
#CacheEvict(value="products", key="#id")
public void delete(Integer id) {
this.productRepository.delete(id);
}
I want to know if is possible update/delete the item located in the cache through the key="#root.target.PRODUCTS", it would be 100 with the Product updated or 499 if the Product was deleted.
My point is, I want avoid the following:
#Override
#CachePut(value="products", key="#product.id")
#CacheEvict(value="products", key="#root.target.PRODUCTS")
public Product update(Product product) {
return this.productRepository.save(product);
}
#Override
#Caching(evict={
#CacheEvict(value="products", key="#id"),
#CacheEvict(value="products", key="#root.target.PRODUCTS")
})
public void delete(Integer id) {
this.productRepository.delete(id);
}
I don't want call again the 500 or 499 products to be cached into the key="#root.target.PRODUCTS"
Is possible do this? How?
Thanks in advance.
Caching the collection using the caching abstraction is a duplicate of what the underlying caching system is doing. And because this is a duplicate, it turns out that you have to resort to some kind of duplications in your own code in one way or the other (the duplicate key for the set is the obvious representation of that). And because there is duplication, you have to sync state somehow
If you really need to access to the whole set and individual elements, then you should probably use a shortcut for the easiest leg. First, you should make sure your cache contains all elements which is not something that is obvious. Far from it actually. Considering you have that:
//EhCacheCache cache = (EhCacheCache) cacheManager.getCache("products");
#Override
public Set<Product> findAll() {
Ehcache nativeCache = cache.getNativeCache();
Map<Object, Element> elements = nativeCache.getAll(nativeCache.getKeys());
Set<Product> result = new HashSet<Product>();
for (Element element : elements.values()) {
result.add((Product) element.getObjectValue());
}
return Collections.unmodifiableSet(result);
}
The elements result is actually a lazy loaded map so a call to values() may throw an exception. You may want to loop over the keys or something.
You have to remember that the caching abstraction eases the access to the underlying caching infrastructure and in no way it replaces it: if you had to use the API directly, this is probably what you would have to do in some sort.
Now, we can keep the conversion on SPR-12036 if you believe we can improve the caching abstraction in that area. Thanks!
I think something like this schould work... Actually it's only a variation if "Stéphane Nicoll" answer ofcourse, but it may be useful for someone. I write it right here and haven't check it in IDE, but something similar works in my Project.
Override CacheResolver:
#Cacheable(value="products", key="#root.target.PRODUCTS", cacheResolver = "customCacheResolver")
Implement your own cache resolver, which search "inside" you cached items and do the work in there
public class CustomCacheResolver implements CacheResolver{
private static final String CACHE_NAME = "products";
#Autowired(required = true) private CacheManager cacheManager;
#SuppressWarnings("unchecked")
#Override
public Collection<? extends Cache> resolveCaches(CacheOperationInvocationContext<?> cacheOperationInvocationContext) {
// 1. Take key from query and create new simple key
SimpleKey newKey;
if (cacheOperationInvocationContext.getArgs().length != null) { //optional
newKey = new SimpleKey(args); //It's the key of cached object, which your "#Cachable" search for
} else {
//Schould never be... DEFAULT work with cache if something wrong with arguments
return new ArrayList<>(Arrays.asList(cacheManager.getCache(CACHE_NAME)));
}
// 2. Take cache
EhCacheCache ehCache = (EhCacheCache)cacheManager.getCache(CACHE_NAME); //this one we bringing back
Ehcache cache = (Ehcache)ehCache.getNativeCache(); //and with this we working
// 3. Modify existing Cache if we have it
if (cache.getKeys().contains(newKey) && YouWantToModifyIt) {
Element element = cache.get(key);
if (element != null && !((List<Products>)element.getObjectValue()).isEmpty()) {
List<Products> productsList = (List<Products>)element.getObjectValue();
// ---**--- Modify your "productsList" here as you want. You may now Change single element in this list.
ehCache.put(key, anfragenList); //this method NOT adds cache, but OVERWRITE existing
// 4. Maybe "Create" new cache with this key if we don't have it
} else {
ehCache.put(newKey, YOUR_ELEMENTS);
}
return new ArrayList<>(Arrays.asList(ehCache)); //Bring all back - our "new" or "modified" cache is in there now...
}
Read more about CRUD of EhCache: EhCache code samples
Hope it helps. And sorry for my English:(
I think there is a way to read the collection from underlying cache structure of spring. You can retrieve the collection from underlying ConcurrentHashMap as key-value pairs without using EhCache or anything else. Then you can update or remove an entry from that collection and then you can update the cache too. Here is an example that may help:
import com.crud.model.Post;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.cache.Cache;
import org.springframework.cache.CacheManager;
import org.springframework.cache.interceptor.CacheOperationInvocationContext;
import org.springframework.cache.interceptor.CacheResolver;
import org.springframework.cache.interceptor.SimpleKey;
import org.springframework.stereotype.Component;
import java.util.*;
#Component
#Slf4j
public class CustomCacheResolver implements CacheResolver {
private static final String CACHE_NAME = "allPost";
#Autowired
private CacheManager cacheManager;
#SuppressWarnings("unchecked")
#Override
public Collection<? extends Cache> resolveCaches(CacheOperationInvocationContext<?> cacheOperationInvocationContext) {
log.info(Arrays.toString(cacheOperationInvocationContext.getArgs()));
String method = cacheOperationInvocationContext.getMethod().toString();
Post post = null;
Long postId = null;
if(method.contains("update")) {
//get the updated post
Object[] args = cacheOperationInvocationContext.getArgs();
post = (Post) args[0];
}
else if(method.contains("delete")){
//get the post Id to delete
Object[] args = cacheOperationInvocationContext.getArgs();
postId = (Long) args[0];
}
//read the cache
Cache cache = cacheManager.getCache(CACHE_NAME);
//get the concurrent cache map in key-value pair
assert cache != null;
Map<SimpleKey, List<Post>> map = (Map<SimpleKey, List<Post>>) cache.getNativeCache();
//Convert to set to iterate
Set<Map.Entry<SimpleKey, List<Post>>> entrySet = map.entrySet();
Iterator<Map.Entry<SimpleKey, List<Post>>> itr = entrySet.iterator();
//if a iterated entry is a list then it is our desired data list!!! Yayyy
Map.Entry<SimpleKey, List<Post>> entry = null;
while (itr.hasNext()){
entry = itr.next();
if(entry instanceof List) break;
}
//get the list
assert entry != null;
List<Post> postList = entry.getValue();
if(method.contains("update")) {
//update it
for (Post temp : postList) {
assert post != null;
if (temp.getId().equals(post.getId())) {
postList.remove(temp);
break;
}
}
postList.add(post);
}
else if(method.contains("delete")){
//delete it
for (Post temp : postList) {
if (temp.getId().equals(postId)) {
postList.remove(temp);
break;
}
}
}
//update the cache!! :D
cache.put(entry.getKey(),postList);
return new ArrayList<>(Collections.singletonList(cacheManager.getCache(CACHE_NAME)));
}
}
Here are the methods that uses the CustomCacheResolver
#Cacheable(key = "{#pageNo,#pageSize}")
public List<Post> retrieveAllPost(int pageNo,int pageSize){ // return list}
#CachePut(key = "#post.id",cacheResolver = "customCacheResolver")
public Boolean updatePost(Post post, UserDetails userDetails){ //your logic}
#CachePut(key = "#postId",cacheResolver = "customCacheResolver")
public Boolean deletePost(Long postId,UserDetails userDetails){ // your logic}
#CacheEvict(allEntries = true)
public Boolean createPost(String userId, Post post){//your logic}
Hope it helps to manipulate your spring application cache manually!
Though I don't see any easy way, but you can override Ehcache cache functionality by supplying cache decorator. Most probably you'd want to use EhcahceDecoratorAdapter, to enhance functions used by EhCacheCache put and evict methods.
Simple and rude solution is :
#Cacheable(key = "{#pageNo,#pageSize}")
public List<Post> retrieveAllPost(int pageNo,int pageSize){ // return list}
#CacheEvict(allEntries = true)
public Boolean updatePost(Post post, UserDetails userDetails){ //your logic}
#CacheEvict(allEntries = true)
public Boolean deletePost(Long postId,UserDetails userDetails){ // your logic}
#CacheEvict(allEntries = true)
public Boolean createPost(String userId, Post post){//your logic}