I have been given a requirement to persist user data once the user has authenticated initially. We don't want to hit the database to look up the user every time they navigate to a new view etc...
I have a User class that is [Serializable] so it could be stored in a session. I am using SQL server for session state as well. I was thinking of storing the object in session but I really hate doing that.
How are developers handling this type of requirement these days?
Three ways:
Encrypting data in cookies and sending it to client, decrypting it whenever you need it
Storing it server side by an Id (e.g UserId) in Cache, Session, or any other storage(which is safer than cookie).
Use second level caching strategy if you used an ORM
Assuming your user object is not huge and does not change often i think it is acceptable to store it in the session.
Since you already have a sql server session you will be making SP calls to pull/push the data already and adding a small object to that should have minimal perf issues compared to other options like persisting it down to the client and sending it back on every request.
I would also consider the server a much more secure place to keep this info.
You want to minimize the number of times you write to the session(request a lock) when it is stored in sql as it is implemented in a sealed class that exclusivity locks the session. If any of your other requests in this session require write access to the SQL session they will be blocked by the initial request until it releases the session lock. (there are some new hooks in .NET 4 for allowing you to change the SessionStateBehavior in the pipeline before the session is accessed)
You might consider a session state server (appfabric) if perf of your SQL session store is an issue.
Related
I'd like to have InProc session state on servers in my webfarm and only share the unique SessionID (which I believe is transferred via browser cookie).
Is this possible? We're planning to have custom object cache based on this SessionID that needs to be loaded only in certain scenarios. If possible, I'd like to avoid creating custom session provider
You really should avoid relying on internal details of ASP.NET (how it manages Session ID) - this behavior could change in the next .NET version or even before that if someone discovers a vulnerability in the current approach.
Instead create your own cookie - both reading and writing that cookie is a one-liner (HttpResponse.SetCookie() and HttpRequest.Cookies[]).
HttpSession is a high level interface built on top of cookies and url-rewriting, which means that there is only a session ID is stored in client side and the data associated with it is stored in server side.
Where is the HttpSession data actually stored in the server side? In the JVM memory or somewhere else? Can I change the place where to store it, e.g. save them into an in-memory database?
If it's not in a database, is there any concurrency problem when many clients work on the same session data at the same time?
It's up to the server where to store session data; the ones I'm familiar with allow some level of configuration as to where (disk, DB, memory, ...) session data is stored.
Different clients shouldn't be working on the same session data--session data is per-client. That said, a single client (like a web browser) could have multiple windows or tabs open, and yes, that can cause issues.
Clustering adds a layer of complexity/headache as session data is shared between servers.
In my application I want to keep a large amount of data in memory specific to a user currently accessing my web application in a user specific session. As for as I know play framework uses cookie to store session data which has a limit of 4k. How can I have much larger session data? Does ehacache memcache help here? This session has expiration time from last activity of the user.
If a session data is cache'ble its better to keep it in Cache with key as userid and clear it when user logs off. Get it reloaded from DB on relevant DB update/delete. Keeping the content in external cache like memcache, will help you to scale well and will enable you to move to distributed cache in the long run, if required. Check this interesting article on Share Nothing.
The idea with Play is to dispel the need for the session and the keeping of lots of information in memory. The problem with the in-memory approach, is that you tie the user to the specific server that their data is held, where-as the play share nothing approach means you can scale horizontally easily without worry of sticky sessions and the like.
The options you have are
- store transient data in a temporary database that can be accessed via a userId or other unique idenifier of the users session. This database would be the equivalent of your server side session.
- use a cache. However the idea of a cache is that if the information is not in the cache, it can be retrieved from the database (or other source) instead. A cache should not have to guarantee that the data will be available. If in the case of an in memory cache (like ehcache) if you have a load balanced set of servers, you may not be able to guarantee that all requets go back to the same server, so data in the cache may not be available on all servers for a particular session.
The answer to your question depends on your use case, but I think the database is your best approach based on the information you have supplied.
I'm just wondering why storing session in database? Is there any advantage to storing session in database?
The advantage to the database or memcached is that session data cannot be tampered with on the client side and that you can store a larger amount of data than you would with cookies (4kB).
If your session is stored in cookies or the database and the web service is restarted then the session data is not lost. It may only be lost if it is stored in memcached.
If the server is load balanced then the session data is passed to the web server that is serving the request, so this is not an issue with cookies, database, or memcached sessions.
The advantage of cookies over memcached or the database is that the client stores the session data, so the server is not responsible for it.
Keep in mind that either way cookies will be passed to and from the client because a session reference still needs to be maintained.
The two reasons I can think of are that:
1) If the web service is restarted, the session data is not lost
2) In a load balanced environment, the session data is stored in a central location, meaning any server can serve the request and have access to the session data.
There are at least three reasons I can think of. If you save the session in the DB you can:
access it easily on any Rails instance you execute. So if you have more than one machine, you don't have to worry about distributing the session data.
You don't have the 4kb session limit session that only aplies when using cookie session store. Although you are not supposed to use the session to store objects, you may that functionality some day.
When using and RDBM (and not Memcached, or any other non persisted storage) you don't have to worry about loosing session data.
one less obvious and small advantage to having the sessions in the database is that if you need to count current sessions and see the names of other logged in users it is easier to implement than if you were using cookies only to store session data or memcached.
another advantage is to handle session expiry on the server side as described in section 2.9:
http://guides.rubyonrails.org/security.html
"However the client can edit cookies that are stored in the web browser so expiring sessions on the server is safer."
class Session < ActiveRecord::Base
def self.sweep(time = 1.hour)
if time.is_a?(String)
time = time.split.inject { |count, unit| count.to_i.send(unit) }
end
delete_all "updated_at < '#{time.ago.to_s(:db)}' OR
created_at < '#{2.days.ago.to_s(:db)}'"
end
end
Currently our DNS routes the user to the correct datacenter and then we have a round-robin situation for the servers. We currently store the session information in the cookie but it's grown too large so we want to move it out of the browser and into a database. I'm worried that if we create a midteir box that they all hit that the response times will be affected. It's not feasible to store the session info all all machines because we're talking about 200M+ unique sessions a month. Any suggestions, thoughts?
A job for memcached or, if you want to save session data to disk, memcacheddb
Memached is a free & open source, high-performance,
distributed memory object caching
system, generic in nature, but
intended for use in speeding up
dynamic web applications by
alleviating database load.
Memcached is an in-memory key-value
store for small chunks of arbitrary
data (strings, objects) from results
of database calls, API calls, or page
rendering.
Memcached is simple yet powerful. Its
simple design promotes quick
deployment, ease of development, and
solves many problems facing large data
caches. Its API is available for most
popular languages.
Let's understand the role of browser-based cookies
Cookies are stored per browser
profile.
The same user logged on from different computers or browsers is
considered different users.
State cookies are mixed with user cookies
Segregate the cookies.
Long-term state cookies, e.g. the currently-remembered userId.
session state cookies
user cookies
Reading that your site is only beginning to consider server-side cookies implies that a segregation of cookies has not yet been done. User cookies should be stored on server as much as possible, so that when a user logs on at another computer or browser, the preferences and shopping carts are preserved. Your development team has to decide for some cookies, for example shopping carts, to be between being session-state or user info cookies.
User cookies
Need to be accessible across the web site, regardless where the user logs in. Your developers have to decide, when a user updates a preference or shopping cart, how immediate should that change be visible if the same userId is logged in at another location.
Which means you have to implement a distributed database system. You have a master db server. Let us say you have 20 web servers, each server with its own database.
Store only frequently changed cookies on the local db and leave the infrequently changing cookies on the master.
Everytime a cookie is updated at a local db, a updated flag is queued for update to the master. The cookie record in the master is not updated, only marked as stale with the location number where the fresh data resides. So that if that userid somehow gets activated 3000 miles away simultaneously, that session would find out the stale records and trigger a request to copy from those records from the fresh location to its own local db and the master db and the records no longer marked as stale on the master db.
Then you schedule a regular sync of most frequently used cookies. The frequency of sync could be nightly or depends on the result of characterization of cookie modification.
First, your programmers would need to write a routine to log all cookie read/writes. You should collect a week's worth of cookie read/write activity to perform your initial component analysis.
You perform simple statistical characterization per cookie, userid and frequency of change. Then you slide along your preferences deciding which cookie is pushed to all the local dbs and which stays on the master. The decision balances between the size of the cookie block on the local dbs and the frequency of database sync you are willing to allow. Which means not every user have the same set of cookies propagated. of course, your programmers would need to write routines to automate the regular recharacterization. Rather than per user, you might wish to lighten the processing load of cookie propagation by grouping your users using cluster analysis. May be the grouping of users for your site is so obvious that you need not perform cluster analysis.
You might be surprised to find that most of the cookies could fall into the longer-than-weekly-update bucket. Or the worse case, daily-update. and the worst case you should accept is hourly update for cookie fields which are not pushed onto the local dbs. You want to increase the chances that a cookie access occurs on the local db rather than being pulled from the master database. So when a user decides to click on "preferences" which is seldom changed, you preemptively pull the preferences records from the master while distracting the user with some frills like "have you considered preview our new service?", "would you like to answer our usability survey?", "new Gibson rant, would you comment?", etc until the "preferences" cookies are copied over.
The characterization of cookies could be done per userid, or per cluster of users to decide which cookie field to push around to local dbs.
It is more simplistic to characterize per userid because it barely involves any statistical analysis skills on the part of the programmer. The disadvantage is that the web server would have to perform decisions for each of 200 million users. The database cookie table would be
Cookie[id, param, value, expectedMutationInterval].
You web server would decide per user which cookie push regularly by the threshold time.
SELECT param, value
WHERE expectedMutationInterval < $thresholdTime
AND id = UserId
You have to perform a regular recharacterization of cookies to update expectedMutationInterval per user per cookie. A simple SQL query would be able to perform the update of expectedMutationInterval. A more complex analysis could be performed to produce the value expectedMutationInterval.
If each cookie field change is logged by time, userid and ipaddr then your Cookie log table would be
CookieLog[id, time, ipaddr, param, value].
which would help your automated recharacterization routine decide what fields to push depending on the dayofweek/month/season and location/region/ipaddr.
Then after removing user info cookies from the browser, if you still find your sessison cookies overflowing, you now decide which session cookies to push to the browser and which stays on the local server. You use the same master-local db analysis technique but now used to decide between local db and pushing to browser. You leave your least frequently accessed session cookies on the local server, either as session attributes or on in-memory db. So when a client finds a cookie is missing, it makes are request to the server for the cookie while sacrificing some least recently/frequently used cookie space on the browser to accommodate placing of that fresh cookie.
Since these are session cookies, they need be propagated to other locations because if a same userid is logged on 3000 miles away, it should have its own set of session cookies.
Characterization of browser cookies are an irony because, for AJAX apps, the client accesses the cookies without letting the server know. Letting the server know might defeat the purpose of placing the cookies in the browser in the first place. So you would have to choose idle times to send cookie accesses to the server to log - for characterization purposes.
Such level of granularity is good for cookies that are short in lengths (parameter value + parameter name), be it session based or user based cookies.
Therefore, if your parameter names and values of cookie fields are long, you might seek to quantize them.
However, quantization is a little more complex. Browser cookies have a lot of commonality. Just like any quantization/compression method, you look for the clusters of commonalities and assign each commonality block a signature. Then the cookies are stored in terms of the quantized signature.
How do you facilitate quantization of browser-based cookies? Using GWT as an example, use the Dictionary or Map class.
e.g., the cookie "%1"="^$Kdm3i" might translate to LastConnectedFriend=MohammadAli#jinnah.
You should not need to perform characterization, for example, why store your cookie as "LastConnectedFriend" when you could map it to "%1"? When a user logs in, why not map the most frequently accessed friends, etc, and place that map on the GWT/AJAX launching page? In that way you could shorten your session cookie lengths.
So, is your company looking for a statistical programmer? Disclaimer is, this is written off-the-cuff and might need some factual realignment.