I went through this video - Scalability Harvard Web Development David Malan
This is where I got stuck. Explaining the problem -
Lets assume LB is using round robin kind of approach.
As per First image, all servers are storing session in their local space, that is not accessible to other servers. If same request comes next time, and if LB redirects this request to another server, then that server will ask about authentication. That is very irritating from user point of view.
As per second image, all servers are sharing sessions. In this case, when next request comes from same client, and LB redirects to another server. Now, instead of asking for authentication, it will fetch information from Session host.
This is mentioned in above video link.
Question -
Now session host become single point of failure. If the host is down, it will impact availability badly. How can we avoid such case ?
You have these options (assuming session is something which cannot be lost at any cost)
1) The session data store is a highly available data store. For eg: You can use MongoDB replica set for such a session store. It consists of three nodes of MongoDB with a master and two slaves (minimum) and when the master goes down one of the nodes is promoted as the master. This election may take a few seconds but the session would not be lost.
2) Use an in memory data sharing library which does data partioning as well as replication. An example would be Hazelcast for Java. It gives you object level sharing across the web tier and here you can store the session which is shared. Please note AFAIK there is no data persistence in this case (on disk).
3) The most scalable approach that I have used till now is to have client side session and no server side data/session storage. What you can do in this case is to have a very long secret key stored in each app server and you set all the data in the cookie after encrypting the data with this secret key. The only problem with this approach is that you need to be very selective with what you store in the session as there is a limit to the data size on cookie. This encryption is a 2-way. Most of the SAAS based tools use this approach.
Implementing Session host as a replicated data store helps remove single point of failure. Example, using a replicated cache like Hazelcast will keep the cache replicated and distributed thus eliminating the single point of failure. There are others like Memcached and Mongo. Automatic fail over on these can be achieved via virtual ip addresses.
For this exact reason, usually session hosts (eg memcache) are fronted with a VIP (virtual IP) and have more than one hosts. In a distributed architecture you generally want to have 1-N hosts. Most companies that operate at scale generate use data storage like Couchbase (memcahce buckets) to store session state because it's fast, redundant, and highly scalable.
Do I really need Session Replication?
I am working on a number of web projects for a firm. Most of the projects are about one or two pages of input and then doing a save to a mysql database. Very Basic projects. My SA's are pushing to try to get session replication working in JBoss but I don't really see any need for it and all of its overhead.
We need load balancing and clustering so if the server does go down we can move the new requests to the backup service but I am not to big in session replication.
This is very low volume projects. I my eyes what is the odds of a user being in the project as the server goes down on the one or two pages.
I need to convince the SAs that session replication is an un-necessary complication in this instance. I am looking for pros and cons of session replication so that I can better structure my argument.
Well, the "pro" is that you have session failover, either in deliberate cluster member restarting or in inadvertent cluster-member failure. That's it.
Some of the "cons" are:
Session objects and their included objects have to be Serializable
You have to choose Session persistence or replication and manage their configurations and/or datastore
You have to think about Session persistence/replication policies (e.g. every write, request end, time scheduled) and still risk losing the session or losing the most current state of it if a failure occurs before recent changes have been stored/replicated
Non-zero performance impact of replicating or or persisting, inversely related to how robust the replication policy is. (That is, the more likely that you'll get every session change replicated promptly, the worse the performance.)
We do session replication because we considered failover to be an absolute requirement years ago when we started this, but I think if I had it to do over again I'd suggest we don't bother for the majority of our applications.
I am working on a webapp project and we are considering deploying it on multiple servers.
What solution do you advise for clustering/load-balancing with Spring?
What are the issues to take into account?
For example: How do singletons behave in a cluster of machines? What about session replication? Are there any other issues to take into account?
Here is the list of possible issues (not necessarily Spring-related):
stateful beans - if your beans have state, like collections accumulating something or counters, you need to think whether this state should be replicated or not. E.g. should this counter be local to one JVM or global in the whole cluster? In the latter case consider terracotta and hazelcast
filesystem - as long as all instances use the same database, everything is fine. But if one node writes to disk, other instance can't read it. Solutions? Either use database for all storage or distributed file system
HTTP sessions - either use sticky session or replicate sessions. If you go for replication, keep sessions as small as possible.
asynchronous jobs - if you have a job running every hour, should it run on every machine, or just on a dedicated one (or maybe on random)?
I was trying to decide what is better in a Tomcat+Apache reverse proxy mode for session replication. What is more common on deployments? session replication or stick session? Are there any drawbacks for session replication?
Thanks
I can point out the following considerations if you go for session replication.
Performance
The main drawback will be on performance. Replicated sessions involve copying of session data over to all the servers in the cluster. The more servers you have in the cluster, the additional overheads involved.
Tomcat helps with this overhead by definining two modes for session replication.
DeltaManager (default) and BackupManager
From this URL http://tomcat.apache.org/tomcat-6.0-doc/cluster-howto.html
Using the above configuration will
enable all-to-all session replication
using the DeltaManager to replicate
session deltas. By all-to-all we mean
that the session gets replicated to
all the other nodes in the cluster.
This works great for smaller cluster
but we don't recommend it for larger
clusters(a lot of tomcat nodes). Also
when using the delta manager it will
replicate to all nodes, even nodes
that don't have the application
deployed.
To get around this problem,
you'll want to use the BackupManager.
This manager only replicates the
session data to one backup node, and
only to nodes that have the
application deployed. Downside of the
BackupManager: not quite as battle
tested as the delta manager
Read this URL for good design tips for the cluster if enabling session replication.
Memory
How many concurrent users will be hitting the application? the more users, the more data gets stored into sessions, and hence an overload for session replication.
Code considerations
Additionally you need to ensure the data being put into the session by the application is serializable. Serializing session data has some overhead for replicating the session state. It's a good idea to keep the session size reasonably small, so the developers need to check the amount of data being put into the session.
Sticky Sessions
Given these considerations, it actually depends on the criticality of the use cases. If you go for sticky sessions alone, then there is a chance of loss of user data during a critical journey.
Do you have means to recover from that - eg: by persisiting critical data into database at each step of a order or payment journey? If not the user has to login and start again. This is fine for websites which are not transactional, but browse brochureware type of data or filling out forms to capture data which is not payment etc.
Session variables are normally keept in the web server RAM memory.
In a cluster, each request made by a client can be handled by a different cluster node. right?!
So, in this case...
What happens with session variables? Aren't they stored in the nodes RAM memory?
How the other nodes will handled my request correctly if it doesn't have my session variables, or at least all of it?
This issue is treated by the web server (Apache, IIS) or by the language runtime (PHP, ASP.NET, Ruby, JSP)?
EDIT: Is there some solution for Classic ASP?
To extend #yogman's answer.
Memcached is pure awesomeness! It's a high performance and distributed object cache.
And even though I mentioned distributed it's basically as simple as starting one instance on one of your spare/idle servers, you configure it as in ip, port and how much ram to use and you're done.
memcached -d -u www -m 2048 -l 10.0.0.8 -p 11211
(Runs memcached in daemon mode, as user www, 2048 MB (2 GB) of RAM on IP 10.0.0.8 with port 11211.)
From then on, you ask memcached for data and if the data is not yet cached you pull it from the original source and store it in memcached. I'm sure you are familiar with cache basics.
In a cluster environment you can link up your memcached's into a cluster and replicate the cache across your nodes. Memcached runs on Linux, Unix and Windows, start it anywhere you have spare RAM and start using your resources.
APIs for memcached should be generally available. I'm saying should because I only know of Perl, Java and PHP. But I am sure that e.g. in Python people have means to leverage it as well. There is a memcached wiki, in case you need pointers, or let me know in the comments if I was raving too much. ;)
There are 3 ways to store session state in ASP.NET. The first is in process, where the variables are stored in memory. The second is to use a session state service by putting the following in your web.config file:
<sessionState
mode="StateServer"
stateConnectionString="tcpip=127.0.0.1:42424"
sqlConnectionString="data source=127.0.0.1;user id=sa;password="
cookieless="false"
timeout="20" />
As you can see in the stateConnectionString attribute, the session state service can be located on a different computer.
The third option is to use a centralized SQL database. To do that, you put the following in your web.config:
<sessionState
mode="SQLServer"
stateConnectionString="tcpip=127.0.0.1:42424"
sqlConnectionString=
"data source=SERVERHAME;user id=sa;password="
cookieless="false"
timeout="20"
/>
More details on all of these options are written up here: http://www.ondotnet.com/pub/a/dotnet/2003/03/24/sessionstate.html
Get a Linux machine and set up http://www.danga.com/memcached . Its speed is unbeatable compared to other approaches. (for example, cookies, form hidden variables, databases)
As with all sorts of thing, "it depends".
There are different solutions and approaches.
As mentioned, there's the concept of a centralized store for session state (database, memcached, shared file system, etc.).
There are also cluster wide caching systems available that make local data available to all of the machines in the cluster. Conceptually it's similar to the centralized session state store, but this data isn't persistent. Rather it lives within the individual nodes and is replicated using some mechanism provided by your provider.
Another method is server pinning. When a client hits the cluster the first time, some mechanism (typically a load balancer fronting the cluster) pins the client to a specific server. In a typical client lifespan, that client will spend their entire time on a single machine.
For the failover mechanism, each machine of the cluster is paired with another machine, and so any session changes are shared with the paired machine. Should the clients pinned machine encounter an issue, the client will hit another machine. At this point, perhaps due to cookies, the new machine sees that it's not the original machine for the client, so it pings both the original machine, and the paired machine for the clients session data.
At that point the client may well be pinned to the new machine.
Different platforms do it in different ways, including having no session state at all.
With Hazelcast, you can either use Hazelcast distributed map to store and share sessions across the cluster or let Hazelcast Webapp Manager do everything for you. Please check out the docs for details. Hazelcast is a distributed/partitioned, super lite and easy, free data distribution solution for Java.
Regards,
-talip
http://www.hazelcast.com
To achieve load balancing for classic ASP, you may store the user specific values in the database and pass a reference unique id in the URL as follows.
Maintain a session table in the database which generates a unique id for each record. The first time you want to store session specific data, generate a record in your session table and store the session values in it. Obtain the unique id of the new session record and re-write all links in your web application to send the unique id as part of querystring.
In every subsequent page where you need the session data, query the session table with the unique id passed in the querystring.
Example:
Consider your website to have 4 pages: Login.asp, welcome.asp, taskList.asp, newtask.asp
When the user logs in using login.asp page, after validating the user, create a record in session table and store the required session specific values (lets say user's login date/time for this example). Obtain the new session record's unique id (lets say the unique id is abcd).
Append all links in your website with the unique id as below:
welcome.asp?sessionId=abcd
tasklist.asp?sessionId=abcd
newtask.asp?sessionId=abcd
Now, if in any of the above web pages you want to show the user's login date/time, you just have to query your session table with the sessionID parameter (abcd in this case) and display to the user.
Since the unique value identifying the session is a part of the URL, any of your web servers serving the user will be able to display the correct login date/time value.
Hope this helps.
In ASP.NET you can persist session data to an SQL Server database which is common to all web servers in the cluster.
Once configured (in the web.config for your site), the framework handles all of the persistance for you and you can access the session data as normal.
As Will said, most load-balancing approaches will use some sort of stickiness in the way the distribute forthcoming requests from the same client, meaning, a unique client will hit the same server unless that actual server goes down.
That minimizes the need of distribution of session-data, meaning that only in the eventual failure of a server, a client would loose his session. Depending on your app, this is more or less critical. In most cases, this is not a big issue.
Even the simplest way of loadbalacing (round-rubin the DNS-lookups) will do some sort of stickiness since most browsers will cache the actual lookup and therefor keep going to the first record it received, AFAIK.
It's usually the runtime that is responsible for the sessiondata, in for exampla PHP it's possible to define your own session-handler, which can persist the data into a database for instance. By default PHP stores sessiondata on files, and it might be possible to share these files on a SAN or equivalent in order to share session-data. This was just a theory I had but never got around to test since we decided that loosing sessions wasn't critical and didn't want that single point of failure.