I have a method on which I added a cache by adding the #CacheResult annotation (I actual created a proxy because I can't change the original implementation of SomethingService):
#Service
public class SomethingServiceProxyImpl implements SomethingService {
#Autowired
#Qualifier("somethingService")
SomethingService somethingService;
#Override
#CacheResult(cacheName = "somethingCache", exceptionCacheName = "somethingExceptionCache", cachedExceptions = { SomeException.class })
public SomePojo someMethod(String someArg) {
return somethingService.someMethod(someArg);
}
}
What I need now, is to be able to log cache hits, meaning cases where the result returned was the one from the cache. I've looked at Spring Cache, at JCache and EHCache (the implementation I use) and I've only found way to listen (with listeners) to the following events: CREATED, UPDATED, REMOVED, EVICTED, EXPIRED but none of them have an event for when the cache returned a result (not null).
I don't really want to have to change the implementation to use the cache programatically instead of using the annotations (I actually have a lot of services to change, not just the one), is there a good way to log those events anyway?
Thoughts about that topic. Probably, the first two are the most relevant:
Don't: The code that gets executed in Spring and the respective cache on a cache hit, is the most performance critical one. That's why it is not so clever to let call additional code in that case, or even have an option for that. Wiring in a log will impact your performance massively. Usually there is already logging in an application for everything that leads to a cache request (e.g. incoming web requests). To get an idea whether the cache is working correctly, a counter of the hits is enough. That is available via the JCache JMX Statistics.
Logging adapter: Using Spring, you can write a Cache adapter which does the logging as you need it and wire it in via configuration. Rough idea: Look at the CacheManager and Cache interfaces. Wrap the CacheManager create cache method and return a wrapped cache with logging.
Hack via ExpiryPolicy: When a custom ExpiryPolicy is specified a JCache implementation calls the method getExpiryForAccess on every cache access. However, you don't get any information on the actual key being requested. I also recommend staying away from own ExpiryPolicy implementations, because of performance reasons. So this is just for completeness.
Logging cache / log every access: In case you specify multiple caches, Spring calls them one after another. You could wire in a dummy cache as first cache, which just logs the access.
Related
I have a spring webflux application.
I am loading some list from database into bean. I have two ways of implementing the loading of this bean.
Approach 1: Reactive Way
#Bean
public List<Item> getItemList() throws IOException {
List<Item> itemList = new ArrayList<>();
itemRespository.findAll().collectList().subscribe(itemList::addAll);
return itemList;
}
Approach 2 : Blocking way
#Bean
public List<Item> getItemList() throws IOException {
List<Item> itemList = itemRespository.findAll().collectList().block();
return itemList;
}
Now as I want my application to be reactive, I don't want to use the blocking way.
But the endpoints which I am exposing through my controller depends on this bean's data.
#RestController
public class SomeController{
#Autowired
private List<items> getItemList;
#GetMapping('/endpoint')
public void process(){
List list = getItemList; //this may not get initialzed as the bean loading is reactive
//some more code
}
}
So in case of reactive approach, it may happen that somebody may call my endpoint(as application has already started and ready to serve requests), while due to some reason it may happened that my list has yet not bean retrieved from database(may be any reason ex: slowness of database server etc.), producing inconsistent results for the users calling my endpoint(which in turns depend on this bean).
I am looking for a solution for this scenario.
EDIT : More precise question is that should I load those beans reactively in my application, on which my exposed endpoints are dependent?
The current application architecture solution presented is a typical example on a design that is inherently blocking.
If the first request made to the api needs the items to be in place, then we must sure that they are there before we can take on requests. And the only way to ensure that is to block until the items de facto have been fetched and stored.
Since the design is inherently blocking, we need to rethink our approach.
What we want is to make the service available for requests as quick as possible. We can solve this by using a cache, that will get filled when the first request is made.
Which means application starts up with an empty cache. This cache could for instance be a #Component as spring beans are singletons by default.
the steps would be:
service starts up, cache is empty
service receives its first request
checks if there is data in the cache
if data is stale, evict the cache
if cache is empty, fetch the data from our source
fill the cache with our fetched data
set a ttl (time to live) on the data placed in the cache
return the data to the calling client
Second request:
request comes in to the service
checks if there is data in the cache
checks if the data is stale
if not grab the data and return it to the calling subscriber
There are several cache solutions out there, spring has their #Cachable annotation, which by default is just a key value store, but can be paired with an external solution like redis etc.
Other solutions can be Google guava which has a very good read on their github.
This type of solution is called trading memory for cpu we gain startup time and fast requests (cpu), but the cost is we will spend some more memory to hold data in a cache.
While implementing Hazelcast for the first time in set of web APIs, the usage of Map and Cache is inconsistent.
For example, creating a cache using SpringCacheManager results in the creation of a map
var sCache = springCacheManager.getCache("testCache");
sCache.putIfAbsent("test", "test2");
However, creating a cache using the CachingProvider CacheManager results in the creation of an actual cache that must be opened and closed (as per the documentation)
try (var cache = Caching.getCachingProvider().getCacheManager(null, null,
HazelcastCachingProvider.propertiesByInstanceName("hazelcache")).createCache("actualCache", config)) {
cache.putIfAbsent("test", "test");
}
Another example, using the #Cacheable annotation will create a map, even though the documentation outlines the usage of a Cache. The following code will successfully return the first computed value using a Map in hazelcast. A cache is never used.
#Cacheable(value = "counter")
public Boolean test(Integer addTo) {
counter += addTo;
return counter % 2 != 0;
}
Is there a formal definition within Hazelcast of a cache vs a map? Are both usable for the same purpose?
The image below contains a view into a test Hazelcast Management Center that shows the above components, namely the maps and caches. These are all generated by the same client.
test
There are Cache, Spring Cache and Map to consider here.
For Cache, Hazelcast is an implementation provider for the Java caching standard, JSR107.
These show as "Cache" on the Management Center, and if you run hazelcastInstance.getDistributedObjects() they'll be of type ICache. It's in the Hazelcast documentation here.
For Map, Hazelcast provides a data structure IMap which is mostly a superset of java.util.Map. These show as "Map" on the Management Center.
Spring also provides caching, and you can set CacheType for JSR107 or directly with Hazelcast, or allow Spring to pick. When Spring uses Hazelcast directly, it will use IMap not ICache for storage.
If you pick JCache or configure Spring to use JCache, then you get standards compliant behaviour. You have caching, and can easily swap caching provider from Hazelcast to something else should you want to.
Map gives you operations such as executeOnKey to update one or more entries in situ. If the entry is a compound object but on a small part is changing, this can be a more efficient way to update it than sending the whole value.
I have below use case for Cache. I am using #Cacheable annotation and it looks something like this:
#Cacheable(cacheNames = "sampleCache", key = "#customerId", cacheResolver = "sampleCacheResolver")
public #ResponseBody SampleResponse getSampleMethod(String customerId) {
}
TTL is set in sampleCacheResolver. However I have a requirement where I have to change the TTL based on the response object. For example, in the response object if say SampleResponse.month is current month, I want to set the ttl to 1 min, else I want to leave the default value of 3 mins.
Since the sampleCacheResolver is getting called at the request level, I am not sure how I can update the ttl based on the response.
This is not possible with Spring's Cache Abstraction out of the box.
Spring is very clear that neither TTL (Time To Live) nor TTI (Idle Timeout) expiration policies (or even Eviction policies for that matter) are handled by the abstraction, and for good reason.
Expiration and Eviction implementations are caching provider specific and many have different possible actions when entries expire or are evicted.
It's rather peculiar to apply TTL to the response, particular given that TTL is a set timeframe.
At any rate, you have 2 options.
First, some caching providers and their supporting Spring bits offer capabilities close to what you are seeking.
For instance, I work on Spring Boot, Session and Data for Apache Geode, which can serve as a caching provider in Spring's Cache Abstraction.
1 of the features I support is entry level (that is, entity) expiration policies via an annotation-based configuration model. See here. Of course, this is made possible by Apache Geode itself given it's ability to express custom expiration policies. That is, the implementation of entry level TTL using annotations is based on Apache Geode's CustomExpiry interface.
Another way to handle entry/entity or response TTL is to override/extend Spring's Cache Abstraction. The 2 primary interfaces in the abstraction is the CacheManager and Cache interfaces.
In your case, you would implement the Cache interface to include the TTL logic you need. You Cache (TTL) implementation would wrap the actual caching provider implementation (e.g. EhCache Cache). Then you must implement the CacheManager interface to wrap the caching provider CacheManager implementation (again, the EhCache CacheManager). Your CacheManager implementation exists soley to create instances of your TTL enabled Cache implementation wrapping the provider's Cache implementation, when called upon by the framework. Essentially, your TTL enabled Cache implementation is enhancing, or decorating the caching provider's Cache implementation with the TTL logic you require.
Yet another possible solution would be to use Spring AOP. Indeed, many of Spring's features like Caching or Transaction Management (demarcation) is handled by Spring AOP. A carefully crafted AOP Aspect could wrap the Caching Aspect and contain the TTL logic you require. You must be sure that the TTL AOP Aspect comes after Spring's Caching Aspect, which can be handled by bean ordering.
I did not provide an example for your exact use case this time. However, I have answered questions that required a "custom" implementation of Spring's Cache and CacheManager interfaces before. For your reference, you can refer to my example to see how you might implement your own Cache/CacheManager combination.
Before going down the paths I presented, I encourage you to explore your caching provider to see if it affords you the TTL functionality you are after. You should choose an appropriate caching provider based on your needs and requirements.
I know this is a lot of information, but I simply wanted you to have options and to think about the problem from multiple angles.
We would like to define our caches in our (spring boot) application similar to the concept varnish is doing with it's "grace mode". Right now we are using spring's #Cacheable annotation together with EhCache (but we are not bound to EhCache, if there's another cache backend able to to this would be fine, too).
Ideally I am searching for something like:
#Cacheable(cacheNames = "name", evictionAfter = "5 minutes", graceTimeAfterEviction = "1 minute")
public myCachedMethod() { ... }
What I mean is: If a value in the cache is already evicted (in this example 5+ minutes old) and a new request is asking for it, I want to return the already evicted value (to this request and all following until the new value is stored -- or the grace time passed, too), but still run the method asynchronously filling the cache with a new value.
Most of the approaches I have seen so far solve this by refreshing the cache programmatically with an asynchronous, scheduled task. But that's not what I want, as my cache entries depend on the arguments, and I don't want to keep old values if no one asks for them any more.
Of course, I know that this is not possible in this way, as spring is not aware of the cache management itself. And it can't be configured in the cache backend neither, because the cache backend has no idea of how to refresh the cache.
But maybe there is an approach out there, I did not find (I searched a lot for it already for a few years/months).
According to the documentation of caffeine cache this seems to be supported (https://github.com/ben-manes/caffeine, "https://github.com/ben-manes/caffeine"), but the documentation is, again, only about scheduled refreshes, no grace period. And it does not work with spring's annoations.
There is a similar question here: Spring cacheable asynchronous update while returns old cache, but more focussed on the sync=true feature.
As it stands I am using a JSF request scoped bean to do all my CRUD operations. As I'm sure you most likely know Tomcat doesn't provide container managed persistence so in my CRUD request bean I am using EnityManagerFactory to get fold of enity manager. Now about the validity of my choice to use request scoped bean for this task, it's probably open for a discussion (again) but I've been trying to put it in the context of what I've read in the articles you gave me links to, specifically the first and second one. From what I gather EclipseLink uses Level 2 cache by default which stored cached entity. On ExlipseLink Examples - JPA Caching website it says that:
The shared cache exists for the duration of the persistence unit ( EntityManagerFactory, or server)
Now doesn't that make my cached entities live for a fraction of time during the call that is being made to the CRUD request bean because the moment the bean is destroyed and with it EntityManagerFactory then so is the cache. Also the last part of the above sentence "EntityManagerFactory, or server" gets me confused .. what precisely is meant by or server in this context and how does one control it. If I use the #Cache annotation and set appropriate amount of expire attribute, will that do the job and keep the entities stored on the servers L2 cache than, regardless of whether my EntityManagerFactory has been destroyed ?
I understand there is a lot of consideration to do and each application has specific requirements . From my point of view configuring L2 cache is probably the most desirable (if not only, on Tomcat) option to get things optimized. Quoting from your first link:
The advantages of L2 caching are:
avoids database access for already loaded entities
faster for reading frequently accessed unmodified entities
The disadvantages of L2 caching are:
memory consumption for large amount of objects
stale data for updated objects
concurrency for write (optimistic lock exception, or pessimistic lock)
bad scalability for frequent or concurrently updated entities
You should configure L2 caching for entities that are:
read often
modified infrequently
not critical if stale
Almost all of the above points apply to my app. At the heart of it, amongst other things, is constant and relentless reading of entities and displaying them on the website (the app will serve as a portal for listing properties). There's also a small shopping cart being build in the application but the products sold are not tangible items that come as stock but services. In this case stale entities are no problem and also, so I think, isn't concurrency as the products (here services) will never be written to. So the entities will be read often, and they will be modified infrequently (and those modified are not part of the cart anyway, an even those are modified rarely) and therefore not critical if stale. Finally the first two points seem to be exactly what I need, namely avoidance of database access to already loaded entities and fast reading of frequently accessed unmodified enteties. But there is one point in disadvantages which still concerns me a bit: memory consumption for large amount of objects. Isn't it similar to my original problem?
My current understanding is that there are two options, only one of which applies to my situation:
To be able to delegate the job of longer term caching to the persistence layer than I need to have access to PersistenceContext and create a session scoped bean and set PersistenceContextType.EXTENDED. (this options doesn't apply to me, no access to PersistenceContext).
Configure the L2 #Cache annotation on entities, or like in option 1 above create a session scoped bean that will handle long term caching. But aren't these just going back to my original problem?
I'd really like to hear you opinion and see what do you think could be a reasonable way to approach this, or perhaps how you have been approaching it in your previous projects. Oh, and one more thing, just to confirm.. when annotating an entity with #Cache all linked entities will be cached along so I don't have to annotate all of them?
Again all the comments and pointers much appreciated.
Thanks for you r answer .. when you say
"In Tomcat you would be best to have some static manager that holds onto the EntityManagerFactory for the duration of the server."
Does it mean I could for example declare and initialize static EntityManagerFactory field in an application scoped been to be later used by all the beans throughout the life of the application ?
EclipseLink uses a shared cache by default. This is shared for all EntityManagers accessed from an EntityManagerFactory. You do not need to do anything to enable caching.
In general, you do not want to be creating a new EntityManagerFactory per request, only a new EntityManager. Creating a new EntityManagerFactory is quite expensive, so not a good idea, even ignoring caching (it has its own connection pool, must initialize the meta-data, etc.).
In Tomcat you would be best to have some static manager that holds onto the EntityManagerFactory for the duration of the server. Either never close it, or close it when a Servlet is destroyed.