In my code I am using EhCache to store an object. Code is below
#Resource(name = "ehCacheManager")
private CacheManager manager;
private Ehcache cache;
public void testCache(){
cache = manager.getCache("ehSubjects");
for loop to read subjects data{ //This loops runs 15051 times
final Element element = new Element(subject.subjectId(), subject);
cache.put(element);
}
}
System.out.println("Cache Size:"+cache.getSize()); //15000
The issue is there are 15051 subjects objects but when after the loop I try to print the cache size it always gives me 15000.
So is there a limit on the amount of objects we can keep in EhCache?
The issue was with the ehcache.xml defined by the other team mate. He has given the maxitemsincache value as 15000 due to which cahce was ignoring other items.
Related
Hello guys i'm having a big big dilema, i have a methode cacheable with a hashSet, i want to delete this cache for all the cacheable data that contains a specific value in the HashSet. is it possible with cache evict ?
i want only the HashSet for example containing the value 25486 to be deleted. the allEntry=true deletes everyting.
my HashSet are random so i can't generate the same HashSet everytime.
do someone Knows how to resolve this problem ?? thank you
#Cacheable(value = "myCache", keyGenerator= MethodAndParamsKeyGenerator.KEY_GENERATOR_NAME)
public void getCutsom(Set<Long> ids){
//...
}
#CacheEvict(value = "myCache", keyGenerator= MethodAndParamsKeyGenerator.KEY_GENERATOR_NAME)
public void getCutsom(Set<Long> ids){
}
I'm implementing Spring Boot App which reads some data from kafka to provide it for all requesting clients. Let's say I have a following class:
#Component
public class DataProvider {
private Prices prices;
public DataProvider() {
this.prices = Prices.of();
}
public Prices getPrices() {
return prices;
}
}
Each client may perform GET /api/prices to get info about newest prices. Live updates about prices are consumed from kafka. Due to the fact, that update comes every 5 seconds, which is not super often, the topic has only one partition.
I tried the very basic option using Kafka Listener:
#Component
public class DataProvider {
private Prices prices;
public DataProvider() {
this.prices = Prices.of();
}
public Prices getPrices() {
return prices;
}
#KafkaListener(topics = "test-topic")
public void consume(String message) {
Prices prices = Prices.of(message);
this.prices = prices;
}
}
Is this approach safe?
The prices must be volatile. But again: you need to be sure that an actual data for prices is OK to be dispersed. One HTTP request may return one data, but another concurrent may return other. Just because it has been just update by the Kafka consumer.
You may have your consume() and getPrices() as synchronized. So, every one is going to get an actual data at the same moment. However they are not going to be parallel since synchronized ensures only one thread can get access to the object.
Another way for consistency is to look into a ReadWriteLock barrier. So, getPrices() calls can be parallel, but as long as consume() takes a WriteLock, everyone is blocked until it is done.
So, technically your code is really safe. Only the problem if it is safe from a business purpose.
I am experiencing some sort of thread issue with the #Async method annotation where one argument contains a List of enum and is dropping items. The list is very small, 2 items. The dropping of items is not immediate, but sometimes takes hours or days to appear.
This is the general flow of our program:
A Controller generates the said List in its #RequestMapping method, passes the list to a Service class, which makes a call to a database for batching and triggers an event for each item from the database, passing the list. This list eventually gets passed into an #Async method which then drops either the first item or both items.
Controller.methodA()
-> Creates list with two items in it
-> Calls void Service.methodX(list)
-> Load batch from database
-> Iterate over batch
-> Print items from list --- list in tact
-> Calls void AsyncService.asyncMethod(list)
-> Print items from list --- eventually drops items here always the first item, sometimes both.
Code configuration and bare-bones sample:
We configured it to have 2 threads:
#Configuration
#EnableAsync
public class AsyncConfig implements AsyncConfigurer {
#Override
public Executor getAsyncExecutor() {
ThreadPoolTaskExecutor threadPoolTaskExecutor = new ThreadPoolTaskExecutor();
threadPoolTaskExecutor.setMaxPoolSize(5); // Never actually creates 5 threads
threadPoolTaskExecutor.setCorePoolSize(2); // Only 2 threads are ever created
threadPoolTaskExecutor.initialize();
return threadPoolTaskExecutor;
}
}
This is a local replica to try to trigger the core issue, but no luck:
#RestController
public class ThreadingController {
private final ThreadingService threadingService;
public ThreadingController(ThreadingService threadingService) {
this.threadingService = threadingService;
}
#GetMapping("/test")
public void testThreads() {
List<SomeEnum> list = new ArrayList<>();
list.add(SomeEnum.FIRST_ENUM);
list.add(SomeEnum.SECOND_ENUM);
for (int i = 0; i < 1000; i++) {
this.threadingService.workSomeThreads(i, list);
}
}
}
public enum SomeEnum {
FIRST_ENUM("FIRST_ENUM"),
SECOND_ENUM("SECOND_ENUM");
#Getter
private String name;
SomeEnum(String name) {
this.name = name;
}
}
#Slf4j
#Service
public class ThreadingService {
#Async
public void workSomeThreads(int i, List<SomeEnum> list) {
try {
Thread.sleep(100L); // Add some delay to slow things down to trigger GC or other tests during processing
} catch (InterruptedException e) {
e.printStackTrace();
}
log.info("Count {} ; Here are the list items: {}", i, list.toString());
assert(list.size() == 2);
}
}
If we look through this, I have one controller simulating both the Controller and Service mentioned earlier. It spins through a batch of data, sending the same list over and over. There's an aync method in another class to test that the list is the same. I was not able to replicate the issue locally, but this is the core problem.
To my knowledge, Java is pass-by-reference and every variable passed into a method gets its own pointer in the stack to that reference in memory, but don't think it would cause us to run out of memory. We are running in PCF and don't see any memory spikes or anything during this time. Memory is constant around 50%. I also tried using a CopyOnWriteArrayList (thread safe) instead of ArrayList and still the problem exists.
Questions:
Any idea why the #Async method would drop items in the method argument? The list is never modified after construction, so why would items disappear? Why would the first item always disappear? Why not the second item? Why would both disappear?
Edit: So this question had little to do with #Async in the end. I found deeply nested code that removed items from the list, causing items to go missing.
What you said is correct, Java is indeed pass-by-reference. The change in your list must be definitely due to some other code that is modifying this list while the threads are executing. There is no other way the object would change its values.
You must investigate the code in the below section to identify if there is something that is modifying the list.
-> Print items from list --- eventually drops items here always the first item, sometimes both.
-> code following this might be changing the list.
As the AsyncService would execute its code asynchronously and in the mean time, some other code modifies the list.
You may as well make the method params to be final.
I have a requirement for which I need to execute a bunch of random complex queries with multiple joins for reporting purposes. So I am planning to use entitymanager native query feature directly. I just tried and it seems to work.
#Service
public class SampleService {
#Autowired
private EntityManager entityManager;
public List<Object[]> execute(String sql){
Query query = entityManager.createNativeQuery(sql);
return query.getResultList();
}
}
This code is invoked once in every 30 seconds. Single threaded - scheduled process.
Question:
Should I be using entity manager or entity manager factory?
Should I close the connection here? or is it managed automatically?
How to reduce the DB connection pool - as it is not multi threaded app or Should I not be worried about that?
Any other suggestions!?
Should I be using entity manager or entity manager factory?
Injecting EntityManager Vs. EntityManagerFactory
EntityManager looks fine in this instance.
Should I close the connection here? or is it managed automatically?
No I dont think you need to as the manager handles this.
How to reduce the DB connection pool - as it is not multi threaded app or Should I not be worried about that?
I doubt you need concern yourself with the connection pools unless you are expecting large volumes and your application is running slowly under load. Try doing some bench marking you may have much more capacity than you need and be prematurely optimising your app.
It more likely you would you increase it number of connections rather than decrease. To increase the number of connections you do that in the application.properties (or application.yml)
Any other suggestions!?
Rather than a generic method I would consider having a separate repository class outside of the service and have that repository method do something specific. Make a method return a specific result or thing rather than pass in any sql.
As a rough outline of two seperate classes (files) something like this
#Service
public class SampleService {
#Autowired
private MyAuthorNativeRepository myAuthorNaviveRepository;
public List<Author> getAuthors(){
return myAuthorRepository.getAuthors();
}
}
#Service
public class MyAuthorNativeRepository {
#Autowired
private EntityManager entityManager;
public List<Author> getAuthors(){
Query q = entityManager.createNativeQuery("SELECT blah blah FROM Author");
List<Author> authors = new ArrayList();
for (Object[] row : q.getResultList()) {
Author author = new Author();
author.setName(row[0]);
authors.add(author);
}
return authors;
}
}
I am calling fetchCatchAndClear method where I am passing List and it consists of cache names. Can someone help me how to iterate over the list and clear the cache based on the Cache Name coming from List of String. Also if the list is empty I should clear all the caches present.
A rather simple approach which sticks to the org.springframework.cache.CacheManager could be the following:
List<String> cacheNames = List.of("aCache", "anotherCache"); // the list you are passing in
CacheManager cacheManager = new SimpleCacheManager(); // any cache manager you are injecting from anywhere
// a simple iteration, exception handling omitted for readability reasons
cacheNames.forEach(cacheName -> cacheManager.getCache(cacheName).clear());
Evicting all caches is also that simple, except that you have to query the relevant cache names from the very same cache manager:
CacheManager cacheManager = new SimpleCacheManager();
Collection<String> cacheNames = cacheManager.getCacheNames();
cacheNames.forEach(cacheName -> cacheManager.getCache(cacheName).clear());
If you just want to evict a single cache entry you could either do it programmatically like:
cacheManager.getCache(cacheName).evict(cacheKey); or annotation-based like
#CacheEvict(value = "yourCacheName", key = "#cacheKey")
public void evictSingleCacheValue(String cacheKey) {
}
#CacheEvict(value = "yourCacheName", allEntries = true)
public void evictAllCacheValues() {
}