I have a cache thats populated via #Cacheable as follows
#Cacheable(value = "accountGroupCache")
public List<Acc> getAccInfo(int groupId, String user)
I would like to know what will be the key value pair for this cache? I am using ehcahe to do the caching.
The key will be based on the parameters to your getAccInfo() method. The value is going to be the List<Acc> returned by your method.
According to the docs:
28.3.1.1 Default Key Generation
Since caches are essentially key-value stores, each invocation of a
cached method needs to be translated into a suitable key for cache
access. Out of the box, the caching abstraction uses a simple
KeyGenerator based on the following algorithm:
If no params are given, return 0.
If only one param is given, return that instance.
If more the one param is given, return a key computed from the hashes of all parameters.
Looking at the source code for DefaultKeyGenerator, this is how it computes "a key computed from the hashes of all parameters":
int hashCode = 17;
for (Object object : params) {
hashCode = 31 * hashCode +
(object == null ? NULL_PARAM_KEY : object.hashCode());
}
Related
Nifi 1.2.0.
In a custom processor, an LSN is used to fetch data from a SQL Server db table.
Following are the snippets of the code used for:
Storing a key-value pair
final StateManager stateManager = context.getStateManager();
try {
StateMap stateMap = stateManager.getState(Scope.CLUSTER);
final Map<String, String> newStateMapProperties = new HashMap<>();
String lsnUsedDuringLastLoadStr = Base64.getEncoder().encodeToString(lsnUsedDuringLastLoad);
//Just a constant String used as key
newStateMapProperties.put(ProcessorConstants.LAST_MAX_LSN, lsnUsedDuringLastLoadStr);
if (stateMap.getVersion() == -1) {
stateManager.setState(newStateMapProperties, Scope.CLUSTER);
} else {
stateManager.replace(stateMap, newStateMapProperties, Scope.CLUSTER);
}
}
Retrieving the key-value pair
final StateManager stateManager = context.getStateManager();
final StateMap stateMap;
final Map<String, String> stateMapProperties;
byte[] lastMaxLSN = null;
try {
stateMap = stateManager.getState(Scope.CLUSTER);
stateMapProperties = new HashMap<>(stateMap.toMap());
lastMaxLSN = (stateMapProperties.get(ProcessorConstants.LAST_MAX_LSN) == null
|| stateMapProperties.get(ProcessorConstants.LAST_MAX_LSN).isEmpty()) ? null
: Base64.getDecoder()
.decode(stateMapProperties.get(ProcessorConstants.LAST_MAX_LSN).getBytes());
}
When a single instance of this processor is running, the LSN is stored and retrieved properly and the logic of fetching data from SQL Server tables works fine.
As per the NiFi doc. about state management :
Storing and Retrieving State State is stored using the StateManager’s
getState, setState, replace, and clear methods. All of these methods
require that a Scope be provided. It should be noted that the state
that is stored with the Local scope is entirely different than state
stored with a Cluster scope. If a Processor stores a value with the
key of My Key using the Scope.CLUSTER scope, and then attempts to
retrieve the value using the Scope.LOCAL scope, the value retrieved
will be null (unless a value was also stored with the same key using
the Scope.CLUSTER scope). Each Processor’s state, is stored in
isolation from other Processors' state.
When two instances of this processor are running, only one is able to fetch the data. This has led to the following question:
Is the StateMap a 'global map' which must have unique keys across the instances of the same processor and also the instances of different processors? In simple words, whenever a processor puts a key in the statemap, the key should be unique across the NiFi processors(and other services, if any, that use the State API) ? If yes, can anyone suggest what unique key should I use in my case?
Note: I quickly glanced at the standard MySQL CDC processor code class(CaptureChangeMySQL.java) and it has a similar logic to store and retrieve the state but then am I overlooking something ?
The StateMap for a processor is stored underneath the id of the component, so if you have two instances of the same type of processor (meaning you can see two processors on the canvas) you would have something like:
/components/1111-1111-1111-1111 -> serialized state map
/components/2222-2222-2222-2222 -> serialized state map
Assuming 1111-1111-1111-1111 was the UUID of processor 1 and 2222-2222-22222-2222 was the UUID of processor 2. So the keys in the StateMap don't have to be unique across all instances because they are scoped per component id.
In a cluster, the component id of each component is the same on all nodes. So if you have a 3 node cluster and processor 1 has id 1111-1111-1111-1111, then there is a processor with that id on each node.
If that processor is scheduled to run on all nodes and stores cluster state, then all three instances of the processor are going to be updating the same StateMap in the clustered state provider (ZooKeeper).
I'm very new both on Spring and Redis. I would like to know if there is a way to get the KEY by value?
My KEY is patterned like this: "{typeOfFile}:{id}:{filename}"
typeOfFile could either be "image", "html", or "pdf".
For instance, I want to get the get the KEY of an image type of file with a given fileHash and content. I'm doing it with this kind of idea:
private String getKeyByVal(final String givenFileHash, final String content) {
// get all keys that starts with "image"
Set<String> keys = redisTemplate.keys("image*");
if (keys != null) {
for (String key : keys) {
Map<Object, Object> val = redisTemplate.opsForHash().entries(key);
// check if the value of KEY is equal to the given fileHash
if (val.get("fileHash").equals(givenFileHash) && val.get("content").equals(content)) {
return key;
}
}
}
}
However, I was told that this is quite costly since I'm getting all the keys that starts with "image", and manually check all of them.
Now I'm thinking, maybe it would be much better if I can get the KEY by value. So that it would be easier to get all of its properties. Is that possible in Redis?
No, this is not possible in Redis. You can however store a reverse map in simultaneous as below:
fileHash -> "{typeOfFile}:{id}:{filename}"
This solution assumes that the file hash is unique. If the hash is not unique then you can store a set of ids with the same hash, retrieve the content for each of them and compare. Still a lot faster than the original solution.
I have seen this class that looks like this:
/// <summary>
/// Provides an internal structure to sort the query parameter
/// </summary>
protected class QueryParameter
{
public QueryParameter(string name, string value)
{
Name = name;
Value = value;
}
public string Name { get; private set; }
public string Value { get; private set; }
}
/// <summary>
/// Comparer class used to perform the sorting of the query parameters
/// </summary>
protected class QueryParameterComparer : IComparer<QueryParameter>
{
public int Compare(QueryParameter x, QueryParameter y)
{
return x.Name == y.Name
? string.Compare(x.Value, y.Value)
: string.Compare(x.Name, y.Name);
}
}
Then there is a call later in the code that does the sort:
parameters.Sort(new QueryParameterComparer());
which all works fine.
I decided that it was a waste of time creating a QueryParameter class that only had name value and it would probably be better to use Dictionary. With the dictionary, rather than use the Sort(new QueryParameterComparer()); I figured I could just do this:
parameters.ToList().Sort((x, y) => x.Key == y.Key ? string.Compare(x.Value, y.Value) : string.Compare(x.Key, y.Key));
The code compiles fine, but I am unsure whether it is working because the list just seems to output in the same order it was put in.
So, can anyone tell me if I am doing this correctly or if I am missing something simple?
Cheers
/r3plica
The List<T>.Sort method is not part of LINQ.
You can use OrderBy/ThenBy extension methods before calling ToList():
parameters = parameter.OrderBy(x => x.Key).ThenBy(x => x.Value).ToList();
From your code, I surmise that parameters is your dictionary, and you're calling
parameters.ToList().Sort(...);
and then carrying on using parameters.
ToList() creates a new list; you are then sorting this list and discarding it. You're not sorting parameters at all, and in fact you can't sort it because it's a dictionary.
What you need is something along the lines of
var parametersList = parameters.ToList();
parametersList.Sort(...);
where ... is the same sort as before.
You could also do
var parametersList = parameters.OrderBy(...).ToList();
which is a more LINQ-y way of doing things.
It may even be appropriate to just do e.g.
foreach(var kvp in parameters.OrderBy(...))
(or however you plan on using the sorted sequence) if you're using the sorted seqence more often than you're changing the dictionary (i.e. there's no point caching a sorted version because the original data changes a lot).
Another point to note - a dictionary can't contain duplicate keys, so there's no point checking x.Key == y.Key any more - you just need to sort via (x, y) => string.Compare(x.Key, y.Key)
I'd be careful here, though - by the look of it, the original code did support duplicate keys, so by switchnig to a dictionary you might be breaking something.
Dictionary are only equivalent to two hash map, and allow you to access to any alement (given the key) with costant time O(1) (because the make a lookup search on an hashtable).
So if you would order the elements because you intended to do a dicotomic search later, you do not need that you should use directly dictionary (or if you would query for both the value in dictionary you could use a couple of dictionary with the same elements but switching key value pairs).
As somebody write before me, if you question is how to order a list with linq you should work with linq and with orderby thenby.
I want to add some calculated properties to an EntityObject without loosing the possibility of querying it agains the database.
I created a partial class and added the fields I need in the object. Than I wrote a static function "AttachProperties" that should somehow add some calculated values. I cannot do this on clientside, since several other functions attach some filter-conditions to the query.
The functions should look like this:
return query.Select(o =>
{
o.HasCalculatedProperties = true;
o.Value = 2;
return o;
});
In my case the calculated value depends on several lookups and is not just a simple "2". This sample works with an IEnumerable but, of course, not with an IQueryable
I first created a new class with the EntityObject as property and added the other necessary fields but now I need this extended class to be of the same basetype.
First, in my opinion changing objects in a Select() is a bad idea, because it makes something else happen (state change) than the method name suggests (projection), which is always a recipe for trouble. Linq is rooted in a functional programming (stateless) paradigm, so this kind of usage is just not expected.
But you can extend your class with methods that return a calculation result, like:
partial class EntityObject
{
public int GetValue()
{
return this.MappedProp1 * this.MappedProp2;
}
}
It is a bit hard to tell from your question whether this will work for you. If generating a calculated value involves more than a simple calculation from an object's own properties it may be better to leave your entities alone and create a services that return calculation results from an object graph.
Try something like this:
return from o in collection
select new O()
{
OtherProperty = o.OtherProperty,
HasCalculatedProperties = true,
Value = 2
};
This will create a copy of the original object with the changes you require and avoid all the messiness that come with modifying an entity in a select clause.
I've thrown myself headfirst into C# and .Net 2.0 using Linq, and I'm having a few problems debugging some of the problems, namely the following:
I have a ComboBox control (cmbObjects) I want to populate with a set of objects retrieved using Linq. I've written a helper method to populate a List<T> generic:
class ObjectProvider
{
public static List<T> Get<T>(bool includeNull) where T : class, new()
{
List<T> list = new List<T>();
LutkeDataClassesDataContext db = ConnectionManager.GetConnection();
IQueryable<T> objects = db.GetTable<T>().AsQueryable();
if (includeNull) list.Add(null);
foreach (T o in objects) list.Add(o);
return list;
}
public static List<T> Get<T>() where T : class, new()
{
return Get<T>(false);
}
}
I verified the results when calling the function with true or false - the List does contain the right values, when passing true, it contains null as the first value, followed by the other objects.
When I assign the DataSource to the ComboBox however, the control simply refuses to display any items, including the null value (not selectable):
cmbObjects.DataSource = ObjectProvider.Get<Car>(true);
Passing in false (or no parameter) does work - it displays all of the objects.
Is there a way for me to specify a "null" value for the first object without resorting to magic number objects (like having a bogus entry in the DB just to designate a N/A value)? Something along the lines of a nullable would be ideal, but I'm kind of lost.
Also, I've tried adding new T() instead of null to the list, but that only resulted in an OutOfMemoryException.
The combo box control has an option to append data bound items to the hard-coded items in the list. So you hard-code your n/a value, and data bind the real values.
Okay, it seems the DataSource becomes invalid if you try to add a null value. The solution was to just add the items via a simple foreach loop with an empty string at the start instead of assigning the List<>.