Does Sprin-Cache support evicting selected entries based on fewer keys or different keys than those used while creating the entry?
User may be fetched by username and withTransactions=true
User may be fetched by username and withTransactions=false
User may be fetched by clientId and withTransactions=true
User may be fetched by clientId and withTransactions=false
Fetch
#Cacheable(value = USER_CACHE, key = "{#userRequest.username, #userRequest.clientId, #userRequest.withTransactions}")
public Optional<PlatformUser> getUser(userRequest userRequest) {
return userUsecase.getUser(userRequest);
}
Create
#Caching(evict = {#CacheEvict(value = USER_CACHE,key = "{#subscriptionDto.username}")})
public Optional<PlatformUser> createUser(userRequest userRequest) {
return userUsecase.createUser(userRequest);
}
Let assume I have following cases:
Get with parameter as UserRequest[user1, null, false]: Returns null and the result is cached (user1 does not exist) -- cache miss
Create with parameter as UserRequest[user1, 1234567, false]: Evict previous entry only based on username. -- evict
Get with parameter as UserRequest[user1, null, false]: Returns PlatformUser Object and the result is cached (user1 now exists). -- cache miss, cached
Get with parameter as UserRequest[user1, null, false]: Returns PlatformUser Object and the result is cached (user1 now exists). -- cache miss, cached
Get with parameter as UserRequest[user1, null, true]: Returns PlatformUser Object and the result is cached (user1 now exists). -- cache miss, cached
Create a Transaction against user1:
a. Evict entry(only 1) having username=user1 and withTransactions=false
b. Evict entry(only 1) having clientId=1234567 and withTransactions=false
Is there any way to achieve such eviction? Any lead would be appreciated.
Related
Nifi 1.2.0.
In a custom processor, an LSN is used to fetch data from a SQL Server db table.
Following are the snippets of the code used for:
Storing a key-value pair
final StateManager stateManager = context.getStateManager();
try {
StateMap stateMap = stateManager.getState(Scope.CLUSTER);
final Map<String, String> newStateMapProperties = new HashMap<>();
String lsnUsedDuringLastLoadStr = Base64.getEncoder().encodeToString(lsnUsedDuringLastLoad);
//Just a constant String used as key
newStateMapProperties.put(ProcessorConstants.LAST_MAX_LSN, lsnUsedDuringLastLoadStr);
if (stateMap.getVersion() == -1) {
stateManager.setState(newStateMapProperties, Scope.CLUSTER);
} else {
stateManager.replace(stateMap, newStateMapProperties, Scope.CLUSTER);
}
}
Retrieving the key-value pair
final StateManager stateManager = context.getStateManager();
final StateMap stateMap;
final Map<String, String> stateMapProperties;
byte[] lastMaxLSN = null;
try {
stateMap = stateManager.getState(Scope.CLUSTER);
stateMapProperties = new HashMap<>(stateMap.toMap());
lastMaxLSN = (stateMapProperties.get(ProcessorConstants.LAST_MAX_LSN) == null
|| stateMapProperties.get(ProcessorConstants.LAST_MAX_LSN).isEmpty()) ? null
: Base64.getDecoder()
.decode(stateMapProperties.get(ProcessorConstants.LAST_MAX_LSN).getBytes());
}
When a single instance of this processor is running, the LSN is stored and retrieved properly and the logic of fetching data from SQL Server tables works fine.
As per the NiFi doc. about state management :
Storing and Retrieving State State is stored using the StateManager’s
getState, setState, replace, and clear methods. All of these methods
require that a Scope be provided. It should be noted that the state
that is stored with the Local scope is entirely different than state
stored with a Cluster scope. If a Processor stores a value with the
key of My Key using the Scope.CLUSTER scope, and then attempts to
retrieve the value using the Scope.LOCAL scope, the value retrieved
will be null (unless a value was also stored with the same key using
the Scope.CLUSTER scope). Each Processor’s state, is stored in
isolation from other Processors' state.
When two instances of this processor are running, only one is able to fetch the data. This has led to the following question:
Is the StateMap a 'global map' which must have unique keys across the instances of the same processor and also the instances of different processors? In simple words, whenever a processor puts a key in the statemap, the key should be unique across the NiFi processors(and other services, if any, that use the State API) ? If yes, can anyone suggest what unique key should I use in my case?
Note: I quickly glanced at the standard MySQL CDC processor code class(CaptureChangeMySQL.java) and it has a similar logic to store and retrieve the state but then am I overlooking something ?
The StateMap for a processor is stored underneath the id of the component, so if you have two instances of the same type of processor (meaning you can see two processors on the canvas) you would have something like:
/components/1111-1111-1111-1111 -> serialized state map
/components/2222-2222-2222-2222 -> serialized state map
Assuming 1111-1111-1111-1111 was the UUID of processor 1 and 2222-2222-22222-2222 was the UUID of processor 2. So the keys in the StateMap don't have to be unique across all instances because they are scoped per component id.
In a cluster, the component id of each component is the same on all nodes. So if you have a 3 node cluster and processor 1 has id 1111-1111-1111-1111, then there is a processor with that id on each node.
If that processor is scheduled to run on all nodes and stores cluster state, then all three instances of the processor are going to be updating the same StateMap in the clustered state provider (ZooKeeper).
I have a service that pulls statistics for a sales region. The service computes the stats for ALL regions and then caches that collection, then returns only the region requested.
public object Any(RegionTotals request)
{
string cacheKey = "urn:RegionTotals";
//make sure master list is in the cache...
base.Request.ToOptimizedResultUsingCache<RegionTotals>(
base.Cache, cacheKey, CacheExpiryTime.DailyLoad(), () =>
{
return RegionTotalsFactory.GetObject();
});
//then retrieve them. This is all teams
RegionTotals tots = base.Cache.Get<RegionTotals>(cacheKey);
//remove all except requested
tots.Records.RemoveAll(o => o.RegionID != request.RegionID);
return tots;
}
What I'm finding is that when I use a MemoryCacheClient (as part of a StaticAppHost that I use for Unit Tests), the line tots.Records.RemoveAll(...) actually affects the object in the cache. This means that I get the cached object, delete rows, and then the cache no longer contains all regions. Therefore, subsequent calls to this service for any other region return no records. If I use my normal Cache, of course the Cache.Get() makes a new copy of the object in the cache, and removing records from that object doesn't affect the cache.
This is because an In Memory cache doesn't add any serialization overhead and just stores your object instances in memory. Whereas when you use any of the other Caching Providers your values are serialized first then sent to the remote Caching Provider then when it's retrieved it's deserialized back so it's never reusing the same object instances.
If you plan on mutating cached values you'll need to clone the instances before mutating them, if you don't want to manually implement ICloneable you can serialize and deserialize them with:
var clone = TypeSerializer.Clone(obj);
I am using the the following code for redis lock and release
var key = "test-x";
RedisValue token = (RedisValue) Guid.NewGuid().ToString();
if(db.LockTake(key, token, duration)) {
try {
// you have the lock do work
} finally {
db.LockRelease(key, token);
}
}
My problem:
In a unit test I am calling this method 2 times. The first time always work, but the second time I want to obtain the lock on this specific key, it does not work. From my understanding the db.LockRelease should release the lock, making it available for the second request. I did notice that db.LockRelease returns false.
Any idea what might be happening?
The lock key needs to be unique. You are probably using the same lock key as the cache key in you code. From https://stackoverflow.com/a/25138164:
the key (the unique name of the lock in the database)
How to use Cache.getOrElse(java.lang.String key, java.util.concurrent.Callable block, int expiration)
Could someone give me a example？
My point is how to use “expiration",I know it means expire time.
By the way:
I want save some object to cache,and set a expire time.
when the expire time,I can reset the object to the cache.
Thanks.
Let's assume that, you want to set User object on cache, for that you set userId as key and user object as value. If need set expiration time, for sample i set it as 30secs.
cache.set(userId, userObject, 30);
At some point of time, if you want to get user object from cache, which you set earlier using userId as key, you might try the following way to get the user object from cache.
User user = cache.get(userId);
Above will return you the user object, if you access within 30secs, otherwise it will return NULL. This will be perfect for case like validating the session.
In some case, you frequently need to retrieve value from cache, for that following is the best approach.
User user = cache.getOrElse(userId, () -> User.get(userId), 30);
cache will check, whether it has given userId as key, if available then straight away return the user object and update the expiration time to 30secs further.
If given userId not available, then callable block gets invoked and set userId as key, user object fetched from db as value and expiration time as 30secs.
Expiration is the number of seconds that the Object would be hold in the Cache. If you pass 0 as expiration the Cache doesn't expire and you would have to control it by hand.
What getOrElse does is check the Cache, if the Object is not there then call the callable block that you are passing and adds the result to the cache for the number of seconds that you are passing as expiration time.
I based my comment in the Play Framework Cache Javadoc.
I use getOrElse in controllers when I have dynamic and static content to display. Cache the static and then render it together with the dynamic part:
try {
Html staticHtml = Cache.getOrElse("static-content", () -> staticView.render(), 60 * 60);
Html rendered = dynamicPage.render(arg1, arg2, staticHtml);
return ok(rendered);
} catch (Exception e) {
e.printStackTrace();
return internalServerError();
}
staticView.render() returns some html from a view. This view should not call any other pages which are dynamic or you stash something you do not really want to stash.
60*60 means I want to store it for one hour (60 seconds times 60 minutes... ok you can write 3600 if you want)
I should add that getOrElse gets the Object from the cache with the specified key (in this example the key is static-content) but if it cannot find it, then it calls the function which returns an object which is then stored for the specified amount of time in the cache with that key. Pretty neat.
Then you can call some other (dynamic) page and pass the html to it.
The dynamic stuff will stay dynamic :)
I have a cache thats populated via #Cacheable as follows
#Cacheable(value = "accountGroupCache")
public List<Acc> getAccInfo(int groupId, String user)
I would like to know what will be the key value pair for this cache? I am using ehcahe to do the caching.
The key will be based on the parameters to your getAccInfo() method. The value is going to be the List<Acc> returned by your method.
According to the docs:
28.3.1.1 Default Key Generation
Since caches are essentially key-value stores, each invocation of a
cached method needs to be translated into a suitable key for cache
access. Out of the box, the caching abstraction uses a simple
KeyGenerator based on the following algorithm:
If no params are given, return 0.
If only one param is given, return that instance.
If more the one param is given, return a key computed from the hashes of all parameters.
Looking at the source code for DefaultKeyGenerator, this is how it computes "a key computed from the hashes of all parameters":
int hashCode = 17;
for (Object object : params) {
hashCode = 31 * hashCode +
(object == null ? NULL_PARAM_KEY : object.hashCode());
}