Is it currently only possible to expire an entire key/value pair? What if I want to add values to a List type structure and have them get auto removed 1 hour after insertion. Is that currently possible, or would it require running a cron job to do the purging manually?
There is a common pattern that solves this problem quite well.
Use sorted sets, and use a timestamp as the score. It's then trivial to delete items by score range, which could be done periodically, or only on every write, with reads always ignoring the out of range elements, by reading only a range of scores.
More here: https://groups.google.com/forum/#!topic/redis-db/rXXMCLNkNSs
Is it currently only possible to expire an entire key/value pair?
As far as I know, and also according to key commands and document about expiration, currently you can set expiration only to specific key and not to it's underlying data structure. However there is a discussion on google groups about this functionality with outlined alternative solutions.
I came upon a different method of handling this, don't know if it's helpful to any of you,but here goes:
The hash and the sorted set are linked by a guid.
I have a hash that is set to expire in 'x' seconds
I have a sorted set that is used for ranged queries
The data for both is added in a transaction, so if one fails, they both fail.
Upon a ranged query, use 'EXISTS' to see if the hashed value exists as the results are iterated over
If it does not exist, it has expired, so delete the item from the sorted set
What about creating two seperate sorted sets?
Main sorted set which is key = value.
Expire sorted set which is key = expire_timestamp.
If you only want to expire a single score you can set as key:unique_id = expire_timestamp.
With mercy of zrangebyscore we can get expired keys. Then all we need to do is check periodically and zrem.
If you only want to expire a single score: ​zincrby -1.
Related
In redis I'm planning to store key as a unique string and value will be a list.
I have a use case where I need to do 2 things.
First, I need to get all the values associated with a key by providing the key as input.
Second, I want to get all the keys associated with a value by providing one of the value in the values list.
Second part is where I need the advice, how we can achive this ?
I cannot get all the keys or key value pair and loop through because I will have millions of entries in Redis.
As mentioned in the comment above the retrieving of all keys with associated value at will probably sometimes create a performance issue as this will be a run through large entries.As also suggested in the official documentation about retrieving data from the memory caches you can try and use the following Redis command to get the value and see if that is what can solve your purpose.
GET
MGET
I currently have a scenario where we are using REDIS to store string field-value pairs within a hashed set HSET.
The original reasoning behind using hashed sets instead of just sets is ease of retrieving the records using HSCAN inside a GUI Search Bar as opposed to just SCAN because it's easier to get the length of a hash to use in the COUNT field.
I read in the Redis documentation that both GET and HGET commands execute with O(1) time complexity, but a member of my team thinks that if I store all the values inside a single key then it basically returns the entire hash during HGET instead of the singular field-value that I need.
So for a made up but similar example:
I have a Redis instance with a single Hashed Set called users.
The hashed set has 150,000 field:value pairs of username:email
If when I execute hget users coolguy, is the entire hash getting returned or just the email for user coolguy?
First of all, HSET is not a hash set, it creates a hash table. The mechanism behind the hash table and set (which is indeed a hash set) in redis is the same, the difference is mainly that the hash table has values.
To answer your question:
If when I execute hget users coolguy, is the entire hash getting returned or just the email for user coolguy?
Just the email for that user. You can also use HMGET to get the emails of multiple users at once. It's O(1) for each user you fetch, or O(n) for n users.
I'm not expert in redis so does anyone knows how can I create a key that can have subkeys, and these subkeys must have an expire time each one.
Is this possible in Redis??
It would be something like this:
[:keyX]
|
V
[:keyZ][:value]
|
V
EXPIRE keyZ 100
PS. the app is in ruby.
Thanks!
Redis does not have nested keys, although the Hash data type could work for you. Also, Redis expiry is only for keys - Hash fields, List elements or Sorted and regular Sets members can not be assigned with an independent TTL.
Your question does not detail why you're looking to do that (i.e. store keys under a "root" key and have each key expire on its own). You can get the per-key expiration effect by using plain ol' regular keys, or use a Hash to aggregate all the fields under one common key - but not both at the same time.
That said, if you really need this sort of functionality you can always try implementing it yourself - see here for a possible direction: Redis: To set timeout for a key value pair in Set
I have an map function in a view in CouchDB that emits non-unique two array keys, for documents of type message, e.g.
The first position in the array key is a user_id, the second position represents whether or not the user has read the message.
This works nicely in that I can set include_docs=true and retrieve the actual documents. However, I'm retrieving duplicate documents in that case, as you can see above in the view results. I need to be able to write a view that can be queried to return unique messages that have been read by a given user. Additionally, I need to be able to efficiently paginate the resultset.
notice in the image above that [66, true] is emitted twice for doc id 26a9a271de3aac494d37b17334aaf7f3. As far as I can tell, with the keys in my map function, I cannot reduce in such a way that unique documents will be returned.
the next idea I had was to emit doc._id also in the map function and reduce with group_level=exact the result being:
now I am able to get unique document ids, but I cannot get the documents without doing a second query. And even in the case of a second query, it will require a lot of complexity to do pagination like this (at least I think so).
the last idea I came up with is to emit the entire document rather than the doc._id in the third position in the array key, then I can access the entire document and likely paginate. This seems really brutish.
So my question is:
Is #3 above a terrible idea? Is there something I'm missing? Is there a better approach?
Thanks in advance.
See #WickedGrey's comment to the question. The solution is to ensure that I never emit the same key twice for one document. I do this in the map function by keeping track of the keys as I emit them in an array, then skipping the emit if the key exists in the array.
I am trying to build a comment system using Redis database, I am currently using hashes to store the comment data, but the problem I am facing is that after 10 or 12 comments, comments lose their order and start appearing randomly, anyone know what data type should be used for building a commenting system using Redis, currently my hashes are of the form.
postid:comments commentid:userid "Testcomment"
Thanks, Any help will be appreciated.
Hashes are set up for quick access by key rather than retrieval in order. If you need items in a particular order, try a list or sorted set.
The reason it appears to work at first is an optimization for small sets - when you only have a small number of items a list is the most efficient structure, so that is what redis uses internally. When you get more items, an actual hashmap is needed for efficient querying and redis rearranges the data so that it is ordered by hash rather than by insertion order.
With my web app, I am using a format like this.
(appname):(postid):(comment id) - The hash of the posts
(appname):(postid):count - The latest comment id
And then I query the (appname):(postid):count key to get the amount of times I should run a loop that gets the contents of the (appname):(postid):(comment id) hash.
Example Code
$c = $redis->get('(appname):(postid):count');
for($i = 0; $i<$c; $i++) {
var_dump($redis->hgetall('(appname):(postid):'.$i));
}