If that's the case, then is it best to store stuff as JSON?
I looked in the documentation, but its not explicitly acknowledged.
Dalli uses Marshal.dump to serialize values you can store any thing that can be dumped (for example procs can't be dumped on most ruby implementations.
Personally I prefer only to store arrays, hashes, strings, numbers and combinations thereof.
Storing arbitrary objects can be inefficient (for example an activerecord object has several copies of its attributes in its instance variables).
Another potential problem is if you store an instance of a class and you later rename that class - you'll no longer be able to retrieve that value from the cache because the cached data still has the old class name in it.
memcache can cache everything that is serializable so even Dalli do.
Related
How do I get a List of objects from redis cache based on the key passed?
I am exploring cachemanager.net for redis cache. I have gone through the examples. But I could not find any example related to getting the List of objects based on the key passed.
var lst =cache.Get("Key_1");
It is returning only one object.
But I would like it like this. I have stored 1000 objects in cache with key name like Key_1, Key_2, Key_3..... Key_1000. I want to get list of 1000 objects if I pass Key_* as Key.
CacheManager does not provide any functionality to search keys or get many keys via wildcard. That's simply not how caches work.
As Karthikeyan pointed out, in Redis you could use the keys operator, but that's not a good solution and should only be used for debugging manually. Other cache systems don't even have something like that, therefore CacheManager also cannot provide that feature. Hope that makes sense ;)
With CacheManager, you can either store all your objects in one cache key and cache the list. That might have some limitations if you use redis because serialization might be become an issue.
Or, you store each object separately and retrieve them in a loop. The redis client will optimize certain things, also, in CacheManager, if you have 2 layers of caching, the performance will get better over time.
You can use redis hash instead. And you can use hgetall command to retrieve all the values in that hash.
http://redis.io/commands#hash
Or if you want to use a normal key Value pair you have to write a lua script to achieve it.
local keys = redis.call('keys','key_*')
return redis.call('mget',keys)
Keys is not advisable in production as it is blocking.
You can use scan command instead of keys to get all the keys matching that pattern and then follow the same procedure to achieve the same.
I'm wondering why the SimpleKeyGenerator in Spring 4+ doesn't just return an Integer representing a hash of the method parameters? Since the returned SimpleKey stores the actual method parameters, it was leading to excessive memory usage and garbage collection in my web application, bringing the whole thing down every two days or so. (Some of these parameters were fairly large collections of strings.) I've since had to revert to my own KeyGenerator implementation, which is basically a mishmash of code I've seen elsewhere.
I'm using EhCache 2.9 as my cache implementation.
From the API docs of SimpleKeyGenerator:
Unlike DefaultKeyGenerator, no collisions will occur with the keys generated by this class.
If SimpleKey would only store the hash code and not the actual parameters itself, this could not be guaranteed. A hash code is not a unique key, you can have different objects (or combinations of parameters, in this case) which have the same hash code (in fact, that's unavoidable, because there are more possible objects than possible hash codes - see pigeonhole principle).
In other words, without storing the parameters themselves, the equals() method of SimpleKey cannot be implemented correctly - it needs to call the equals() methods of all parameters and not just compare hash codes.
See also the API docs of DefaultKeyGenerator - that implementation of KeyGenerator indeed only stores hash codes and it's deprecated precisely because it's possible that collisions can occur, leading to incorrect behaviour.
If this causes an excessive memory usage problem, then configure your cache differently so that it doesn't grow too large or make sure you don't pass large objects to SimpleKeyGenerator for generating keys.
I have a class that reads from a DB on startup. I'd prefer to be able to store it in the session, but I get the following error when trying to do so:
ERROR TypeError: no marshal_dump is defined for class Mutex
Is what I'm doing possible/reasonable? If so how should I go about doing it? If not, whats a good alternative to storing the class instance in the session? Currently my workaround is just instantiating it whenever I need to use it, but that doesn't strike me as a good solution or one that will be able to scale.
A good alternative is to store the id of the record in the session. Then when you need that data again you'd use a helper to return the data either from memory or from the database. A perfect example is the pattern used in current_user helper methods found in many ruby authentication gems. You could modify this helper to use a cache layer if you find it to be a bottleneck, but I'd leave that as an optimization after the fact.
Issues of having to get the object into a marshaled format that will live happily in a session, there are issues with storage space, stale data and possibly unintentional exposure to confidential data.
OK, Ruby gurus, this is a hard one to describe in the title, so bear with me for this explanation:
I'm looking to pass a string that represents a variable: not an instance, not the collection of properties that make up an object, but the actual variable: the handle to the object.
The reason for this is that I am dealing with resources that can be located on the filesystem, on the network, or in-memory. I want to create URI handler that can handle each of these in a consistent manner, so I can have schemes like eg.
file://
http://
ftp://
inmemory://
you get the idea. It's the last one that I'm trying to figure out: is there some way to get a string representation of a reference to an object in Ruby, and then use that string to create a new reference? I'm truly interested in marshalling the reference, not the object. Ideally there would be something like taking Object#object_id, which is easy enough to get, and using it to create a new variable elsewhere that refers to the same object. I'm aware that this could be really fragile and so is an unusual use case: it only works within one Ruby process for as long as there is an existing variable to keep the object from being garbage collected, but those are both true for the inmemory scheme I'm developing.
The only alternatives I can think of are:
marshal the whole object and cram it into the URI, but that won't work because the data in the object is an image buffer - very large
Create a global or singleton purgatory area to store a variable for retrieval later using e.g. a hash of object_id:variable pairs. This is a bit smelly, but would work.
Any other thoughts, StackOverflowers?
There's ObjectSpace._id2ref :
f = Foo.new #=> #<Foo:0x10036c9b8>
f.object_id #=> 2149278940
ObjectSpace._id2ref(2149278940) #=> #<Foo:0x10036c9b8>
In addition to the caveats about garbage collection ObjectSpace carries a large performance penalty in jruby (so much so that it's disabled by default)
Variables aren't objects in Ruby. You not only cannot marshal/unmarshal them, you can't do anything with them. You can only do something with objects, which variables aren't.
(It would be really nice if they were objects, though!)
You could look into MagLev which is an alternative Ruby implementation built on top of VMware's Gemstone. It has a distributes object model wiht might suit your use-case.
Objects are saved in the central Gemstne instance (with some nifty caching) and can be accessed by any number of remote worker instances. That way, any of the workers act on the same object space and can access the very same objects simultaneously. That way, you can even do things like having the global Garbage Collector running on a single Ruby instance or seamlessly moving execution at any point to different nodes (while preserving all the stack frames) using Continuations.
I would like a persistent hash; an object that act as a hash, but that can persist between program runs.
Ideally, it would only load in memory the value that are accessed.
Since persistent key/value storage is kind of everyones requirement, as it happens there are a large number of solutions.
YAML is probably the easiest way to persist Ruby objects.
JSON works as well but doesn't directly handle symbols.
MySQL and other SQL databases such as sqlite3 also solve this problem, of course. Usually, access is encapsulated within the ActiveRecord ORM library.
The Ruby core has a Marshaling library.
Using sdbm
require 'sdbm'
SDBM.open("/mypath/myfile.dbm") do |myMap|
[...]
myMap[key] = avalue
[...]
myvar = myMap[anotherKey]
[...]
end
create to files : myfile.dbm.dir and myfile.dbm.pag
I would consider using redis-rb, which has a hash datatype. This would not only persist your hash across program runs, but across multiple machines. It's super fast, in memory, and you can have it up and running in < 5 minutes.
in IRB (assuming you've installed and are running redis-server and have installed redis-rb:
require "redis"
redis = Redis.new
The important operations are:
redis.hset(key, field, value)
and
redis.hget(key,field)