I am evaluating the case of using sticky sessions with Session replication in tomcat. From my initial evaluation, I thought that if we enable Session replication, then session started in one tomcat node will be copied to all other tomcat nodes and thus we do not need sticky session to continue sessions and the request can be picked up by any node.
But it seems that session replication is in general used with sticky sessions, otherwise the session id needs to be changed whenever the request goes to some other node. ref: http://tomcat.apache.org/tomcat-6.0-doc/cluster-howto.html#Bind_session_after_crash_to_failover_node
Can anyone explain what is the real use of session replication if you have to enable sticky session? Because then you would be unnecessarily copying the session on each node, when the request with a given session id is always going to the same node. It could be beneficial in case of a node crashing, but then that does not happen frequently and using session replication only for that seems like an overkill.
As Mindas explained it before :
When you use loadbalancing it means you have several instances of tomcat and you need to divide loads.
If you're using session replication without sticky session : Imagine you have only one user using your web app, and you have 3
tomcat instances. This user sends several requests to your app, then
the loadbalancer will send some of these requests to the first tomcat
instance, and send some other of these requests to the secondth
instance, and other to the third.
If you're using sticky session without replication : Imagine you have only one user using your web app, and you have 3 tomcat
instances. This user sends several requests to your app, then the
loadbalancer will send the first user request to one of the three
tomcat instances, and all the other requests that are sent by this
user during his session will be sent to the same tomcat instance.
During these requests, if you shutdown or restart this tomcat
instance (tomcat instance which is used) the loadbalancer sends the
remaining requests to one other tomcat instance that is still
running, BUT as you don't use session replication, the instance
tomcat which receives the remaining requests doesn't have a copy of
the user session then for this tomcat the user begin a session : the
user loose his session and is disconnected from the web app although
the web app is still running.
If you're using sticky session WITH session replication : Imagine you have only one user using your web app, and you have 3 tomcat
instances. This user sends several requests to your app, then the
loadbalancer will send the first user request to one of the three
tomcat instances, and all the other requests that are sent by this
user during his session will be sent to the same tomcat instance.
During these requests, if you shutdown or restart this tomcat
instance (tomcat instance which is used) the loadbalancer sends the
remaining requests to one other tomcat instance that is still
running, as you use session replication, the instance tomcat which
receives the remaining requests has a copy of the user session then
the user keeps on his session : the user continue to browse your web
app without being disconnected, the shutdown of the tomcat instance
doesn't impact the user navigation.
I think the only real benefit is to be able to shut down Tomcat instances without much thinking. Especially this applies today in cloud world (think Amazon AWS spot instances) when nodes can go on and off really often. Alternative to this would be to buy a decent load balancer which supports node draining. But decent load balancers are expensive, and draining takes time.
Another scenario I can think of is a (poor implementation of) shopping cart where items are kept in the HttpSession and shutting down would require the user to re-purchase them (which would likely lead to a lost sale).
But in most cases you're right - benefit of having both sticky sessions and session replication is very negligible.
Just to clarify configuration issues on JBoss 5.X in "all" base configuration with mod_jk. Setting sticky sessions in workers.properties file
worker.list=loadbalancer
... nodes configuration omitted
worker.loadbalancer.balance_workers=node1,node2
worker.loadbalancer.sticky_session=True
does not prevent session replication. In order to switch off session replication on JBoss we need to set in $JBOSS_HOME\server\YOUR_NODE_NAME\deploy\cluster\jboss-cache-manager.sar\META-INF\jboss-cache-manager-jboss-beans.xml cacheMode parameter to LOCAL.
Usually in sticky session scenario we don't want session replication, since we do not want additional overhead connected with significant amount of I/O operations needed to replicate sessions.
In fact, if go with sticky sessions, we do not need to run JBoss in "all" configuration, we might use "default" or "standard" based configuration.
The only thing that has to be done is change in $JBOSS_HOME/server/YOUR_NODE_NAME/deploy/jbossweb.sar/server.xml:
<Engine name="jboss.web" defaultHost="localhost" jvmRoute="YOUR_NODE_NAME">
Related
Assumptions -
There are 4 servers sitting behind a reverse proxy which acts as a load balancer
Load Balancer is purely load balancing and sends a request to any of the 4 servers depending on their current load
Users need to be authenticated for accessing this application, and some space should hold state of all users, as reverse-proxy is only load balancing
Application needs to scale beyond 4 servers, say to 4000 servers.
Question -
In a large scale multi-server system who holds state of all the users - load balancer, each server, separate server?
Is the state of all users saved on all servers so that the load balancer can send a request to any server? How does this scale to 100m users?
You can use sticky sessions. It enables the load balancer to bind a user's session to a specific instance. This ensures that all requests from the user during the session are sent to the same instance. Read Sticky and NON-Sticky sessions.
Also let's say the instance gets killed due to some reason, in order to maintain the state, the authentication token and other information can also be saved on a separate redis cache, which is much faster to query. Read Session Management in microservices
In a stateless multi-server system, a separate server (authentication server) or a separate server cluster (authentication api) holds state of all users. If its a single authentication server for a large application, you can expect it to have RAM in the range of 100's of GBs, maybe more.
Nope, state of all users is usually not replicated on all application servers, it will be a huge waste of resources. Authentication server (or server cluster) may act as load balancer itself or forward all requests to a separate load balancer - true for a stateless application.
In a stateful application, individual servers hold state of users through sticky sessions.
If possible, try to keep your application stateless. A stateless application will have better performance and will be easier to scale out than a stateful application!
I am using redirectAttributes to pass success or failure messages to redirected url. So that I can show the success or failure message on the redirected page only once. If the same page is refreshed again then the message will not come up again. This is ok and working fine in normal deployment on tomcat.
Now we have setup a clustered environment where we have deployed the web application. But in this case the redirectAttributes are working weirdly. Sometimes it works and sometimes not.
Following is the line of code I am using to add flashAttribute to the redirect attributes.
redirectAttributes.addFlashAttribute("successMsg", message);
I am using Spring 3.1.0.RELEASE version and Tomcat 7 for clustered environment.
I want to know whether there is any workaround for this problem. Does any newer Spring version supports the use of redirectAttributes in clustered environment?
Also you can let me know if there is another way to perform this kind of stuff.
Thanks in advance.
It sounds like your HTTP sessions for a client may not be shared between the Tomcat servers. Since the Spring Flash attributes are stored in the session, you may be experiencing the following:
Initial request goes to serverA, and a flash attribute is set in the session on serverA
A redirect occurs, and the request gets sent to serverB. serverA and serverB have different HTTP sessions for the user (assuming you have no mechanism to share them), so serverB does not see the flash attribute (it has it's own separate HTTP session)
You could experience this problem intermittently if the server to which a client request gets sent is non-deterministic. For example, if both of the above described requests happen to go to serverA, then the flash attribute would work correctly, since the session would be the same.
If this is the case, then you need a mechanism to either:
Provide a 'sticky' session -- guarantee that all requests for a given client get routed to the same Tomcat server. Usually this is accomplished via a load-balancer / routing mechanism (example: nginx ip hash routing)
Implement session replication -- make sessions shared across all Tomcat servers, so that regardless of which Tomcat serves the request for a client, the HTTP session will be the same.
I want to know the difference between sticky- and non-sticky sessions. What I understood after reading from internet:
Sticky : only single session object will be there.
Non-sticky session : session object for each server node
When your website is served by only one web server, for each client-server pair, a session object is created and remains in the memory of the web server. All the requests from the client go to this web server and update this session object. If some data needs to be stored in the session object over the period of interaction, it is stored in this session object and stays there as long as the session exists.
However, if your website is served by multiple web servers which sit behind a load balancer, the load balancer decides which actual (physical) web-server should each request go to. For example, if there are 3 web servers A, B and C behind the load balancer, it is possible that www.mywebsite.com is served from server A, www.mywebsite.com is served from server B and www.mywebsite.com/ are served from server C.
Now, if the requests are being served from (physically) 3 different servers, each server has created a session object for you and because these session objects sit on three independent boxes, there's no direct way of one knowing what is there in the session object of the other. In order to synchronize between these server sessions, you may have to write/read the session data into a layer which is common to all - like a DB. Now writing and reading data to/from a db for this use-case may not be a good idea. Now, here comes the role of sticky-session.
If the load balancer is instructed to use sticky sessions, all of your interactions will happen with the same physical server, even though other servers are present. Thus, your session object will be the same throughout your entire interaction with this website.
To summarize, In case of Sticky Sessions, all your requests will be directed to the same physical web server while in case of a non-sticky load balancer may choose any webserver to serve your requests.
As an example, you may read about Amazon's Elastic Load Balancer and sticky sessions here : http://aws.typepad.com/aws/2010/04/new-elastic-load-balancing-feature-sticky-sessions.html
I've made an answer with some more details here :
https://stackoverflow.com/a/11045462/592477
Or you can read it there ==>
When you use loadbalancing it means you have several instances of tomcat and you need to divide loads.
If you're using session replication without sticky session : Imagine you have only one user using your web app, and you have 3
tomcat instances. This user sends several requests to your app, then
the loadbalancer will send some of these requests to the first tomcat
instance, and send some other of these requests to the secondth
instance, and other to the third.
If you're using sticky session without replication : Imagine you have only one user using your web app, and you have 3 tomcat
instances. This user sends several requests to your app, then the
loadbalancer will send the first user request to one of the three
tomcat instances, and all the other requests that are sent by this
user during his session will be sent to the same tomcat instance.
During these requests, if you shutdown or restart this tomcat
instance (tomcat instance which is used) the loadbalancer sends the
remaining requests to one other tomcat instance that is still
running, BUT as you don't use session replication, the instance
tomcat which receives the remaining requests doesn't have a copy of
the user session then for this tomcat the user begin a session : the
user loose his session and is disconnected from the web app although
the web app is still running.
If you're using sticky session WITH session replication : Imagine you have only one user using your web app, and you have 3 tomcat
instances. This user sends several requests to your app, then the
loadbalancer will send the first user request to one of the three
tomcat instances, and all the other requests that are sent by this
user during his session will be sent to the same tomcat instance.
During these requests, if you shutdown or restart this tomcat
instance (tomcat instance which is used) the loadbalancer sends the
remaining requests to one other tomcat instance that is still
running, as you use session replication, the instance tomcat which
receives the remaining requests has a copy of the user session then
the user keeps on his session : the user continue to browse your web
app without being disconnected, the shutdown of the tomcat instance
doesn't impact the user navigation.
Let's say the user sends a request to get its profile, there won't be anything in the memory of our web application instance. we get the user profile from DB nit before sending the response, we save the data in the memory of let's say Instance3. But the next request from the same user can go to any instance.
When the request first comes to Instance3, that time it will create a session that will have a session id. when the response is sent to the client, the client is supplied with a cookie. so next time this client makes a request, this cookie will be attached to the request, the load balancer will look at the cookie, and the load balancer will know that that request has to be forwarded to Instance3. This is sticky session solution. Its downside is what if Instance3 goes down? the load balancer will route the request to other instances but they do not have a cache. All the users stored in Instance3 will have high latency. This will impact the reliability of your system.
If you store sessions in all instances, now you would have memory issues. Let's say if an instance could store 100 user sessions and you have 3 instances, you would be able to store 300 sessions. But if each instance stores each session, you will be able to store only 100 sessions in all of your 3 instances. So this will impact the scalability of your application.
sticky and non-sticky sessions are components of stateful replication. If you want higher scalability you do not cache anything on your web application instance, your web instance will hit the DB with every request but this will cause high latency.
A better way is stateless replication where you do not store anything on your application instance but instead, you use server-side caching (memcached/redis)
How do I make multiple tomcats on different servers sharing sessions?
I have a balancer sitting in front of 2 tomcat on two different servers, so when user starts a session, it's only associated with one of the tomcats. He might or might not get the session in the next request.
i think you can tell it to do sticky sessions, so the LB keeps requests from a client on one app server...
edit --
Difference between session affinity and sticky session?
What is the difference between session affinity and sticky session in context of load balancing servers?
I've seen those terms used interchangeably, but there are different ways of implementing it:
Send a cookie on the first response and then look for it on subsequent ones. The cookie says which real server to send to.
Bad if you have to support cookie-less browsers
Partition based on the requester's IP address.
Bad if it isn't static or if many come in through the same proxy.
If you authenticate users, partition based on user name (it has to be an HTTP supported authentication mode to do this).
Don't require state.
Let clients hit any server (send state to the client and have them send it back)
This is not a sticky session, it's a way to avoid having to do it.
I would suspect that sticky might refer to the cookie way, and that affinity might refer to #2 and #3 in some contexts, but that's not how I have seen it used (or use it myself)
As I've always heard the terms used in a load-balancing scenario, they are interchangeable. Both mean that once a session is started, the same server serves all requests for that session.
Sticky session means that when a request comes into a site from a client all further requests go to the same server initial client request accessed. I believe that session affinity is a synonym for sticky session.
They are the same.
Both mean that when coming in to the load balancer, the request will be directed to the server that served the first request (and has the session).
Sticky session means to route the requests of particular session to the same physical machine who served the first request for that session.
This article clarifies the question for me and discusses other types of load balancer persistence.
Dave's Thoughts: Load balancer persistence (sticky sessions)
difference is explained in this article:
https://www.haproxy.com/blog/load-balancing-affinity-persistence-sticky-sessions-what-you-need-to-know/
Main part from this link:
Affinity: this is when we use an information from a layer below the application layer to maintain a client request to a single server. Client's IP address is used in this case. IP address may change during same session and then connection may switch to different server.
Persistence: this is when we use Application layer information to stick a client to a single server. In this case, loadbalancer inject some cookie in response and use same cookie in subsequent request to route to same server.
sticky session: a sticky session is a session maintained by persistence
The main advantage of the persistence over affinity is that it’s much more accurate, but sometimes, Persistence is not doable(when client dont allow cookies like cookie less browser), so we must rely on affinity.
Using persistence, we mean that we’re 100% sure that a user will get redirected to a single server.
Using affinity, we mean that the user may be redirected to the same server…
They are Synonyms.
No Difference At all
Sticky Session / Session Affinity:
Affinity/Stickiness/Contact between user session and, the server to which user request is sent is retained.