I am currently toying around with clojure.core.cache, and I can not figure out how to do an atomic set-and-get operation with it when having a cache miss in a cache stored in an Atom.
I am trying to introduce time-based caching into my clojure service to reduce the amount of expensive db-calls on the one hand and to avoid a memory overflow some time in the future if I just use memoize (since this should be a long-running service).
Now as it seems to me, the correct way to use the library is to create a cache via one of the factory functions, store it in an atom, and then swapping in the new cache whenever we update it. (see example below from their docu)
(require '[clojure.core.cache :as cache])
(def cache (atom (cache/fifo-cache-factory {:a 1, :b 2})))
(swap! cache cache/through-cache :d (constantly 13))
;=> {:a 1, :b 3, :d 13}
(swap! cache cache/evict :b)
;=> {:a 1, :d 13}
(get #cache :a)
;=> 1
Now what I do not understand is how to do an atomic set-and-get operation, i.e. how to retrieve a value and the cache being filled transparently if necessary.
If I do it via 2 operations it might cause race conditions from my understanding, so it would be unsafe to use in production.
Is there something I am missing?
Addition:
I now realized that the get-and-set operation cache/through is done atomically and that consequentially every get operation will return the same value.
Atoms guarantee that there won't be a race condition.
Every call to swap! uses a CAS operation behind the scenes. That's why the function you need to provide in swap! should be side-effect free, because it might be re-tried if another thread got in the way of updating the atom.
Retrieving-and-filling with through-cache in a swap! is the correct approach, that said, if you need to guarantee that a slow retrieval operation for a value is only done exactly once, you might want to look at agents.
See https://clojure.org/reference/agents
esp. this part:
"If during the function execution any other dispatches are made (directly or indirectly), they will be held until after the state of the Agent has been changed."
Related
I have some module-level state I want to write once and then never modify.
Specifically, I have a set of strings I want to use to look things up in later. What is an efficient and ordinary way of doing this?
I could make a function that always returns the same set:
my_set() -> sets:from_list(["a", "b", "c"]).
Would the VM optimize this, or would the code for constructing the set be re-run every time? I suspect the set would just get GCd.
In that case, should I cache the set in the process dictionary, keyed on something unique like the module md5?
Key = proplists:get_value(md5, module_info()), put(Key, my_set())
Another solution would be to make the caller to call an init function to get back an opaque chunk of state, then pass that state into each function in the module.
A compile-time constant, like your example list ["a", "b", "c"], will be stored in a constant pool on the side when the module is loaded, and not rebuilt each time you run the expression. (In the old days, the list would have been reconstructed from its elements for each new call.) This goes for all constants no matter how complicated (like lists of lists of tuples). But when you call out to a function like sets:from_list/1, the compiler cannot assume anything about the representation used by the sets module, and the set will be constructed dynamically from that constant list.
While an ETS table would work, it is less efficient for larger constants (like, say, a set or map containing many entries), because an ETS table has the same memory model as a process - data is written and read by copying, as if by sending messages. If the constants are small, the difference between copying them and recreating them locally would be negligible, and if the constants are large, you waste time copying them.
What you want instead is a fairly new feature called Persistent Term Storage: https://erlang.org/doc/man/persistent_term.html (since Erlang/OTP 21). It is similar to the way compile time constants are handled, so there is no copying when looking up a value. (The key could be the name of your module.) Persistent Term is intended as pretty much a write-once-read-many storage - you can update the stored entry, but that's a more expensive operation which may trigger a global GC.
I want to remove few cache entries if the key in the cache matches some pattern.
For example, I've the following key-value pair in the cache,
("key-1", "value-1"), ("key-2", "value-2"), ("key-3", "value-3"), ("key-4", "value-4")
Since cache implements a map interface, i can do like this
cache.entrySet().removeIf(entry -> entry.getKey().indexOf("key-") > 0);
Is there a better way to do this in infinispan (may be using functional or cache stream api)?
The removeIf method on entrySet should work just fine. It will be pretty slow though for a distributed cache as it will pull down every entry in the cache and evaluate the predicate locally and then perform a remove for each entry that matches. Even in a replicated cache it still has to do all of the remove calls (at least the iterator will be local though). This method may be changed in the future as we are updating the Map methods already [a].
Another option is to instead use the functional API as you said [1]. Unfortunately the way this is implemented is it will still pull all the entries locally first. This may be changed at a later point if the Functional Map APIs become more popular.
Yet another choice is the cache stream API which may be a little more cumbersome to use, but will provide you the best performance of all of the options. Glad you mentioned it :) What I would recommend is to apply any intermediate operations first (luckily in your case you can use filter since your keys won't change concurrently). Then use the forEach terminal operation which passes the Cache on that node [2] (note this is an override). Inside the forEach callable you can call the remove command just as you wanted.
cache.entrySet().parallelStream() // stream() if you want single thread per node
.filter(e -> e.getKey().indexOf("key-") > 0)
.forEach((cache, e) -> cache.remove(e.getKey()));
You could also use indexing to avoid the iteration of the container as well, but I won't go into that here. Indexing is a whole different beast.
[a] https://issues.jboss.org/browse/ISPN-5728
[1] https://docs.jboss.org/infinispan/9.0/apidocs/org/infinispan/commons/api/functional/FunctionalMap.ReadWriteMap.html#evalAll-java.util.function.Function-
[2] https://docs.jboss.org/infinispan/9.0/apidocs/org/infinispan/CacheStream.html#forEach-org.infinispan.util.function.SerializableBiConsumer-
I am controlling the Consumer that get's to this forEach so it may or may not be asked to perform an action.
list.parallelStream().forEach( x-> {} );
Streams being lazy Streams won't iterate, right? Nothing will happen is what i expect. Tell me if i am wrong, please.
It will traverse the whole stream, submitting tasks to fork-join pool, splitting the list to parts and passing all the list elements to this empty lambda. Currently it's impossible to check in runtime whether the lambda expression is empty or not, thus it cannot be optimized out.
Similar problem appears in using Collector. All collectors have the finisher operation, but in many cases it's an identity function like x -> x. In this case sometimes the code which uses collectors can be greatly optimized, but you cannot robustly detect whether the supplied lambda is identity or not. To solve this an additional collector characteristic called IDENTITY_FINISH was introduced instead. Were it possible to robustly detect whether supplied lambda is identity function, this characteristic would be unnecessary.
Also look at JDK-8067971 discussion. This proposes creating static constants like Predicate.TRUE (always true) or Predicate.FALSE (always false) to optimize operations like Stream.filter. For example, if Predicate.TRUE is supplied, then filtering step can be removed, and if Predicate.FALSE is supplied, then stream can be replaced with empty stream at this point. Again were it possible to detect in runtime that the supplied predicate is always true, then it would be unnecessary to create such constants.
Question 1
I am building/searching for a RAM memory cache layer for my server. It is a simple LRU cache that needs to handle concurrent requests (both Gets an Sets).
I have found https://github.com/pmylund/go-cache claiming to be thread safe.
This is true as far as getting the stored interface. But if multiple goroutines requests the same data, they are all retrieving a pointer (stored in the interface) to the same block of memory. If any goroutine changes the data, this is no longer very safe.
Are there any cache-packages out there that tackles this problem?
Question 1.1
If the answer to Question 1 is No, then what would be the suggested solution?
I see two options:
Alternative 1
Solution: Storing the values in a wrapping struct with a sync.Mutex so that each goroutine needs to lock the data before reading/writing to it.
type cacheElement struct { value interface{}, lock sync.Mutex }
Drawbacks: The cache becomes unaware of changes made to data or might even have dropped it out of the cache. One goroutine might also lock others.
Alternative 2
Solution: Make a copy of the data (assuming the data in itself doesn't contain pointers)
Drawbacks: Memory allocation every time a cache Get is performed, more garbage collection.
Sorry for the multipart question. But you don't have to answer all of them. If you have a good answer to Question 1, that would be sufficient for me!
Alternative 2 sounds good to me, but please note that you do not have to copy the data for each cache.Get(). As long as your data can be considered immutable, you can access it with many multiple readers at once.
You only have to create a copy if you intend to modify it. This idiom is called COW (copy on write) and is quite common in concurrent software design. It's especially well suited for scenarios with a high read/write ratio (just like a cache).
So, whenever you want to modify a cached entry, you basically have to:
create a copy of the old cached data, if any.
modify the data (after this step, the data should be considered immutable and must not be changed anymore)
add / replace the existing element in the cache. You could either use the go-cache library you have pointed out earlier (which is based on locks) for that, or write your own lock-free library that simply swaps the pointers to the data element atomically.
At this point any goroutine that performs a cache.Get operation will get the new data. Existing goroutines however, might still be reading the old data. So, your program might operate on many different versions of the same data at once. But don't worry, as soon as all goroutines have finished accessing the old data, the GC will collect it automatically.
tux21b gave a good answer. I'll just point out that you don't have to return pointers to data. you can store non pointer values in your cache and go will pass by value which will be a copy. Then your Get and Set methods will be safe since nothing can actually modify the cache contents.
There are always several ways to do the same thing in Mathematica. For example, when adapting WReach's solution for my recent problem I used Condition:
ClearAll[ff];
SetAttributes[ff, HoldAllComplete];
ff[expr_] /; (Unset[done]; True) :=
Internal`WithLocalSettings[Null, done = f[expr],
AbortProtect[If[! ValueQ[done], Print["Interrupt!"]]; Unset[done]]]
However, we can do the same thing with Block:
ClearAll[ff];
SetAttributes[ff, HoldAllComplete];
ff[expr_] :=
Block[{done},
Internal`WithLocalSettings[Null, done = f[expr],
AbortProtect[If[! ValueQ[done], Print["Interrupt!"]]]]]
Or with Module:
ClearAll[ff];
SetAttributes[ff, HoldAllComplete];
ff[expr_] :=
Module[{done},
Internal`WithLocalSettings[Null, done = f[expr],
AbortProtect[If[! ValueQ[done], Print["Interrupt!"]]]]]
Probably there are several other ways to do the same. Which way is the most efficient from the point of view of memory and CPU use (f may return very large arrays of data - but may return very small)?
Both Module and Block are quite efficient, so the overhead induced by them is only noticable when the body of a function whose variables you localize does very little. There are two major reasons for the overhead: scoping construct overhead (scoping constructs must analyze the code they enclose to resolve possible name conflicts and bind variables - this takes place for both Module and Block), and the overhead of creation and destruction of new symbols in a symbol table (only for Module). For this reason, Block is somewhat faster. To see how much faster, you can do a simple experiment:
In[14]:=
Clear[f,fm,fb,fmp];
f[x_]:=x;
fm[x_]:=Module[{xl = x},xl];
fb[x_]:=Block[{xl = x},xl];
Module[{xl},fmp[x_]:= xl=x]
We defined here 4 functions, with the simplest body possible - just return the argument, possibly assigned to a local variable. We can expect the effect to be most pronounced here, since the body does very little.
In[19]:= f/#Range[100000];//Timing
Out[19]= {0.063,Null}
In[20]:= fm/#Range[100000];//Timing
Out[20]= {0.343,Null}
In[21]:= fb/#Range[100000];//Timing
Out[21]= {0.172,Null}
In[22]:= fmp/#Range[100000];//Timing
Out[22]= {0.109,Null}
From these timings, we see that Block is about twice faster than Module, but that the version that uses persistent variable created by Module in the last function only once, is about twice more efficient than Block, and almost as fast as a simple function invokation (because persistent variable is only created once, and there is no scoping overhead when applying the function).
For real functions, and most of the time, the overhead of either Module or Block should not matter, so I'd use whatever is safer (usually, Module). If it does matter, one option is to use persistent local variables created by Module only once. If even this overhead is significant, I'd reconsider the design - since then obviously your function does too little.There are cases when Block is more beneficial, for example when you want to be sure that all the memory used by local variables will be automatically released (this is particularly relevant for local variables with DownValues, since they are not always garbage - collected when created by Module). Another reason to use Block is when you expect a possibility of interrupts such as exceptions or aborts, and want the local variables to automatically be reset (which Block does). By using Block, however, you risk name collisions, since it binds variables dynamically rather than lexically.
So, to summarize: in most cases, my suggestion is this: if you feel that your function has serious memory or run-time inefficiency, look elsewhere - it is very rare for scoping constructs to be the major bottleneck. Exceptions would include not garbage-collected Module variables with accumulated data, very light-weight functions used very frequently, and functions which operate on very efficient low-level structures such as packed arrays and sparse arrays, where symbolic scoping overhead may be comparable to the time it takes a function to process its data, since the body is very efficient and uses fast functions that by-pass the main evaluator.
EDIT
By combining Block and Module in the fashion suggested here:
Module[{xl}, fmbp[x_] := Block[{xl = x}, xl]]
you can have the best of both worlds: a function as fast as Block - scoped one and as safe as the one that uses Module.