What are the different use cases of Rack::Session::Pool and Rack::Session::Cookie?
As far as I understand (correct me if I'm wrong):
Cookie stores all the session key:value pairs directly within the cookie (marshalled)
Pool only stores an id in the cookie, and maintains the rest of the session hash within #pool
So: what are the implications/reasons for choosing one over the other? what's #pool? Why does Pool need to expose a different public interface from Cookie? Why is the documentation so lacking?
You are right, Session::Cookie marshaling and store sessions in cookies.
Session::Pool instead keeps sessions in memory.
Pool has some advantages:
- faster, no marshaling needed
- you can keep any objects with it(read ones that can not be marshaled)
But when you restart your app all sessions are lost.
With Cookie instead you will have restart-persistent sessions at the price of marshaling.
Alternatives - Session::Memcache, Session::Mongo
Related
EDIT2: Clarification: The code ALREADY has refresh cache on miss logic. What I'm trying to do is reducing the number of missed cache hits.
I'm using Redis as a cache for an API. The idea is that when the API receives a call it first checks the cache and if the data isn't in cache the API will fetch it and cache it afterwards for next time.
At the moment the configuration is the following:
maxmemory 50mb
maxmemory-policy allkeys-lru
That is, use at most 50mb memory, keep trying keys in there and when memory is full start by deleting the least recently used keys (lru).
Now I want to introduce a second category of keys. For this second category I'm going to set a certain expiry time. Now I would like to set up a mechanism such that when these keys expiry this mechanism kicks in and refreshes them (and sets new expiry).
How do I do this?
EDIT:
Some progress. It turns out that Redis has a pub/sub messaging system which in particular can dispatch messages on event. One of them is expiring keys, which can be enabled as such:
notify-keyspace-events Ex
I found this code can describes a blocking python process subscribing to Redis' messaging system. It can easily be changed to detect keys expiring and make a call to the API when a key expires, and the API will then refresh the key.
def work(self, item):
requests.get('http://apiurl/?q={param}'.format(param=item['data']))
So this does precisely what I was asking about.
Often, this feels way too dangerous and out of control. I can imagine a bunch of different situations under which this will very quickly fail.
So, what's a better solution?
http://redis.io/topics/notifications
Keyspace notifications allows clients to subscribe to Pub/Sub channels
in order to receive events affecting the Redis data set in some way.
Examples of the events that is possible to receive are the following:
All the keys expiring in the database 0. (e.g)
...
EXPIRE generates an expire event when an expire is set to the key, or
a expired event every time setting an expire results into the key
being deleted (see EXPIRE documentation for more info).
To expire keys, just use Redis' built-in expiry mechanism. You don't need to refresh the cache contents on expiry, the simplest is to do it when the code experiences a cache miss.
I understand if one saves sessions to cookies, you need to encrypt them with a secret else malicious clients could modify their session at will. This design is still bad for many reasons popularly discussed.
However, if one saves sessions on the server (I happen to use a Memcache store via Rack:Session:Dalli) , I understand all the client gets is a cookie with a key the server uses to lookup their session from the store. I still set a session secret. But I don't understand what it does anymore.
Encrypting a large random number results in essentially another large random number. In other words, if there is no meaning ascribed to the information (its just a random number), then there is no security benefit to encryption. If the ID you're storing has some information embedded in it, like a certain bit set or only a certain subset of IDs are used, then encryption is useful.
The length of the session ID is important. Obviously, the longer the ID, the more resistant it is to brute forcing. The expected number of simultaneous user sessions is also a factor, as the number of sessions reduces the number of brute force attempts needed to find a valid session ID. For example, two simultaneous sessions reduces the effective strength of the ID by one bit (a 128 bit key becomes as effective as a 127 bit key would be with one session only). An Amazon-scale website with (say) 1,000,000 simultaneous sessions would effectively lose 20 bits of its session key strength.
If you need to defend against brute force attacks, implement a middleware to check for that. Adding information to the session id, like an application-unique string, can make detecting a brute-force attack easier (and requires session id encryption). Note that this does not enhance the security of the key itself, and is basically wasted effort unless the app takes some action when presented with an improper session id.
Whatever you do, just make sure to use SSL and set the cookie to https only. Time out the session server-side, and don't rely on cookie expiration and the good will of the client browser.
TL;DR: If only using cookies for session ID storage, encryption is not necessary if a good RNG is used. Use SSL and set the cookie secure attribute.
I'm implementing a PAS plugin that handles authentications against mailservers. Actually only DBMail is implemented.
I realized, that the enumerateUsers function from the PAS plugin is called numerous times per request and requires my plugin to open/close an SQL connections for every (subsequent) request. Of course, this is very expensive.
The connections itself are handled in a plone tool, which is able to handle multiple different mailservers and delegeates the enumerateUsers call to wrapper objects that represent registered servers.
My question is now, what sort of cache (OOBTree, Session?) I should use to provide a temporary local storage for repeating enumerations and avoid subsequent SQL connections?
Another idea was, to hook into the user creation process that takes place on the first login, an external user issues and completely "localize" the users.
Third idea was, to store the needed data in the specific member, if possible.
What would be best practice here?
I'd cache the query results, indeed. You need to make a decision on how long to cache the results, and if stored long term, how to invalidate that cache or check for changes.
There are no best practices for these decisions, as they depend entirely on the type of data stored and the APIs of the backends. If they support some kind of freshness query, for example, then you store everything forever and poll the backend to see if the cache needs updating.
You can start with a simple request cache; query once per request, store it on the request object. Your cache will automatically be invalidated at the end of the request as the request object is cleaned up, the next request will be a clean slate.
If your backend users rarely change, you can cache information for longer, in a local cache. I'd use a volatile attribute on the plugin. Any attribute starting with _v_ is ignored by the persistence machinery. Thus, anything stored in a _v_ volatile attribute is both thread-local and only exists for the lifetime of the process, a restart of the server clears these automatically.
At the very least you should use an _v_ volatile attribute to store your backend SQL connections. That way they can stay open between requests, and can be re-used. Something like the following method would do nicely:
def _connection(self):
# Return a backend connection
if getattr(self, '_v_connection', None) is None:
# Create connection here
self._v_connection = yourdatabaseconnection
return self._v_connection
You could also use a persistent attribute on your plugin to store your cache. This cache would be committed to the ZODB and persist across restarts. You then really need to work out how to invalidate the contents; store timestamps and evict data when to old, etc.
Your cache datastructure depends entirely on your application needs. If you don't persist information, a dictionary (username -> information) could be more than enough. Persisted caches could benefit from using a OOBTree instead of a dictionary as they reduce chances of conflicts between different threads and are more efficient when it comes to large sets of data.
Whatever you do, you do not need to use a Session. Sessions are prone to conflicts, do not scale well, and are in any case not the place to store a cache of this kind.
I've got a cfc to handle the user object. My question is: is it better to store just the user_id in the session and create the user object anew with each request? Or is is better to store the whole user object in the session?
Here are my thoughts either way:
If I store the whole object in the session:
There will be potentially less processor overhead
There will be potentially more memory overhead
all of the methods/functions are stored in the actual object, and new functions that I update in the cfc will not be available unless users logout and back in, or if I devise some way to make it refresh itself.
There could potentially be mutex or lock problems if I'm messing with the object via concurrent ajax calls
If I store just the user_id in the session:
I'll have to create the user object with each page request (potentially more processor overhead)
There will be potentially less memory overhead
There won't be a chance for mutex/lock/race conditions since each request will have its own copy of the user object
Updates to the CFC model itself will be immediately recognized across the system and users wouldn't have to log out and back in
Is there a normal practice for this sort of thing? Am I over-thinking it?
All of the CF apps I've written were targeted at high traffic levels and high availability, so we never had the luxury of being able to think about single-server practices.
So, in my experience, I always had to a) allow for multiple load-balanced servers, and b) avoid sticky-sessions on the load balancer for a number of reasons. Therefore, we needed to, at the very least, have a server become part of a cluster on the fly and pick up mid-session traffic.
So, we always pulled "session" data from a shared datastore on every request.
My suggestion is to implement a session facade.
This affords you the option to change how you persist session data (like the user record) without changing the rest of your app.
You can choose, behind the scenes, to store everything in the session scope, load it up for every request, do a hybrid, use a key-value store, whatever.
You can choose whether to eager-load data, or lazy-load data, or any mix in between, and the rest of the app doesn't need to be aware of what you've done.
On Race Conditions
If you're concerned about race conditions then I would suggest using named locks around data commit and access. This is another bonus of using a facade - your application code doesn't need to know about this, and you can choose to put locks around certain objects, as opposed to locking the whole session.
You haven't indicated whether you're using an ORM, so this is a general answer.
For typical applications, I recommend instantiating the user object into the session scope. There's a big downside to creating the object anew with each request that you didn't include in your list: changes to the user object's properties and state will not persist across requests unless you intend to flush the user object's state to your persistence layer (e.g. database) on every hit. That is likely to be a much more expensive operation than object instantiation, and it doesn't necessarily insulate you from the kinds of problems you're thinking about with respect to ajax calls, race conditions, etc -- it just transfers the manifestation of those problems to the persistence layer, where your object's data could be in an unpredictable state.
Since every new request would be an "implicit save", you would also have to design your "ephemeral" object to be able to persist itself regardless of whether it's in a valid state (imagine the case of a multi-page form that modifies some aspect of the user object).
For session-stored objects, your concerns about memory can be mitigated by careful design practices. For instance, if your user has many tasks, and each task has many items, it might be a bad idea to instantiate and compose all those objects into your user object (i.e., lazy loading would be a better approach than eager loading).
If you really must to be able to change your CFCs on the fly, you can achieve that goal even with session-stored objects. One way is to store a version flag in both the application and session. With each request, your app would compare those flags. When they differ, the app would run a session-reload routine that snapshots current properties, rebuilds the session-stored objects, and finally updates the session flag to match the application flag.
This is piggy-backing partially off Ken Redler's answer but I don't have enough reputation to comment.
The way we do it, and the way I prefer, is to store the user data in Session as a struct. Then on request start, our Auth Model creates the user object in the Request scope and overrides any default values with the Session data. There are a few advantages to this:
Less hits to the database, less CPU
Always run newest code without a complex custom system ensuring that
Clustered environment friendly (complex objects in Session can't be clustered)
Can add or remove properties without corruption (assuming your User object only updates dirty columns)
Also, if you're using CF9, one of the features they were really proud of is how much they optimized object instantiation. If you haven't, test it yourself!
It depends.
If you have a lot of traffic - in the thousands of unique visitors per minute range - the memory overhead of storing your User.cfc in the session will eventually weigh you down. This can be easily overcome by throwing hardware at it (more memory for a while, eventually more servers and a hardware load balancer). Of course popularity is a good problem to have.
If you seem to have a CPU, network or other bottleneck in your database space, you may want to have the object cached in session memory so that you have fewer hits to the database.
Why do I mention these scenarios? You may be prematurely optimizing - don't fix a problem that you don't have. Don't optimize your memory, CPU and database access until those are, or soon will be, problems.
Now from an architectural best practice - not from an optimized "what's best for my processor" - well, I can only say: It depends.
Truthfully, neither way is wrong. If you are going to find yourself needing to check credentials against your database on every request, don't cache it. If you like the feel of an object in the session, then cache it. Because you know your own domain, you can probably go back and forth all day on why you should or should not cache the user object in the session. If it's going to make it easier, do it. If it's going to make it harder, don't.
I would just warn you against doing something incredibly convoluted or anything that is not immediately obvious to a developer looking at your application - the more you write, the more you have to maintain forever, the more your co-workers will associate your name with evil.
Finally, last note, if this is a vote - I say you cache it. It makes sense and always feels good to call session.user.hasRole("xyz") or the like.
How I can store an instance object foreach user session?
I have a class to modeling a complex algorithm. This algorithm is designed to run step-by-step. I need to instantiate objects of this class for each user. Each user should be able to advance step by step their instance.
You can only store the objects in the Cache. The objects must be serializable for this. In the session you can store a key (which must be a String) to the Cache. Make sure that your code still works if the object was removed from the cache (same as a session-timeout). It's explained in http://www.playframework.org/documentation/1.0.3/cache.
Hope that solve your problem.
To store values in the session:
//first get the user's session
//if your class extends play.mvc.Controller you can access directly to the session object
Session session = Scope.Session.current();
//to store values into the session
session.put("name", object);
If you want to invalidate / clear the session object
session.clear()
from play documentation: http://www.playframework.org/documentation/1.1.1/cache
Play has a cache library and will use Memcached when used in a distributed environment.
If you don’t configure Memcached, Play will use a standalone cache that stores data in the JVM heap. Caching data in the JVM application breaks the “share nothing” assumption made by Play: you can’t run your application on several servers, and expect the application to behave consistently. Each application instance will have a different copy of the data.
You can put any object in the cache, as in the following example (in this example from the doc http://www.playframework.org/documentation/1.1.1/controllers#session, you use session.getId() to save messages for each user)
public static void index() {
List messages = Cache.get(session.getId() + "-messages", List.class);
if(messages == null) {
// Cache miss
messages = Message.findByUser(session.get("user"));
Cache.set(session.getId() + "-messages", messages, "30mn");
}
render(messages);
}
Because it's a cache, and not a session, you have to take into account that the data might no longer be available, and have some mean to retrieve it once again from somehere (the Message model, in this case)
Anyway, if you have enough memory and it involves a short interaction with the user the data should be there, and in case it's not you can redirect the user to the beginning of the wizard (you are talking about some kind of wizard page, right?)
Have in mind that play, with it's stateless share-nothing approach, really have no sessión at all, underneath it just handles it through cookies, that's why it can only accept strings of limited size
Here's how you can save "objects" in a session. Basically, you serialize/deserialize objects to JSON and store it in the cookie.
https://stackoverflow.com/a/12032315/82976