How do I make multiple tomcats on different servers sharing sessions?
I have a balancer sitting in front of 2 tomcat on two different servers, so when user starts a session, it's only associated with one of the tomcats. He might or might not get the session in the next request.
i think you can tell it to do sticky sessions, so the LB keeps requests from a client on one app server...
edit --
Difference between session affinity and sticky session?
Related
Assumptions -
There are 4 servers sitting behind a reverse proxy which acts as a load balancer
Load Balancer is purely load balancing and sends a request to any of the 4 servers depending on their current load
Users need to be authenticated for accessing this application, and some space should hold state of all users, as reverse-proxy is only load balancing
Application needs to scale beyond 4 servers, say to 4000 servers.
Question -
In a large scale multi-server system who holds state of all the users - load balancer, each server, separate server?
Is the state of all users saved on all servers so that the load balancer can send a request to any server? How does this scale to 100m users?
You can use sticky sessions. It enables the load balancer to bind a user's session to a specific instance. This ensures that all requests from the user during the session are sent to the same instance. Read Sticky and NON-Sticky sessions.
Also let's say the instance gets killed due to some reason, in order to maintain the state, the authentication token and other information can also be saved on a separate redis cache, which is much faster to query. Read Session Management in microservices
In a stateless multi-server system, a separate server (authentication server) or a separate server cluster (authentication api) holds state of all users. If its a single authentication server for a large application, you can expect it to have RAM in the range of 100's of GBs, maybe more.
Nope, state of all users is usually not replicated on all application servers, it will be a huge waste of resources. Authentication server (or server cluster) may act as load balancer itself or forward all requests to a separate load balancer - true for a stateless application.
In a stateful application, individual servers hold state of users through sticky sessions.
If possible, try to keep your application stateless. A stateless application will have better performance and will be easier to scale out than a stateful application!
I understand that two sessions cannot use a connection at the exact same time. But I had thought it was possible for multiple sessions to share the same connection or pipe, similar in principle to a threaded application, where processing of execution is time-sliced.
The reason I bring this up is I'm somewhat perplexed by how the F5 load balancer manages connections, in relation to application sessions. I have a web server which connects to the F5, which load balances 2 application servers:
Client (i.e. laptop) --> Web Server --> F5 Load balancer for app servers --> (app server 1, app server 2)
I was looking at the number of connections on the F5 load balancer the other day and it showed 7 connections to app server 1 and 3 connections to app server 2; 10 total connections. But the actual application had hundreds of sessions. How could this be? If there are 1000 sessions, let's say, that averages out to 1 connection per 100 sessions. Something doesn't add up here, I'm thinking.
Can and does the F5 load balancer distinguish between inbound connections and outbound connections? If so, how can I see both inbound and outbound connection? I'm thinking that, perhaps, there are 10 inbound connections from the web server to the load balancer and 1000 outbound connections (because 1000 sessions) to the app servers.
I'm thinking it should be possible to queue or share multiple sessions per connections but maybe that's not how it works, particularly with load balancers. Any help making sense of all of this would be most appreciated. Thank you.
If you were using the OneConnect feature, this is exactly what it's intended for. The BIG-IP manages an internal session state for these connections and can reuse and maintain server-side sessions for multiple external connections.
Useful for potentailly bulky applications but can cause issues if you have internal applications that reuse unique keys for session state (Java is a good example).
SOL7208 - Overview of the Oneconnect Profile
I want to know the difference between sticky- and non-sticky sessions. What I understood after reading from internet:
Sticky : only single session object will be there.
Non-sticky session : session object for each server node
When your website is served by only one web server, for each client-server pair, a session object is created and remains in the memory of the web server. All the requests from the client go to this web server and update this session object. If some data needs to be stored in the session object over the period of interaction, it is stored in this session object and stays there as long as the session exists.
However, if your website is served by multiple web servers which sit behind a load balancer, the load balancer decides which actual (physical) web-server should each request go to. For example, if there are 3 web servers A, B and C behind the load balancer, it is possible that www.mywebsite.com is served from server A, www.mywebsite.com is served from server B and www.mywebsite.com/ are served from server C.
Now, if the requests are being served from (physically) 3 different servers, each server has created a session object for you and because these session objects sit on three independent boxes, there's no direct way of one knowing what is there in the session object of the other. In order to synchronize between these server sessions, you may have to write/read the session data into a layer which is common to all - like a DB. Now writing and reading data to/from a db for this use-case may not be a good idea. Now, here comes the role of sticky-session.
If the load balancer is instructed to use sticky sessions, all of your interactions will happen with the same physical server, even though other servers are present. Thus, your session object will be the same throughout your entire interaction with this website.
To summarize, In case of Sticky Sessions, all your requests will be directed to the same physical web server while in case of a non-sticky load balancer may choose any webserver to serve your requests.
As an example, you may read about Amazon's Elastic Load Balancer and sticky sessions here : http://aws.typepad.com/aws/2010/04/new-elastic-load-balancing-feature-sticky-sessions.html
I've made an answer with some more details here :
https://stackoverflow.com/a/11045462/592477
Or you can read it there ==>
When you use loadbalancing it means you have several instances of tomcat and you need to divide loads.
If you're using session replication without sticky session : Imagine you have only one user using your web app, and you have 3
tomcat instances. This user sends several requests to your app, then
the loadbalancer will send some of these requests to the first tomcat
instance, and send some other of these requests to the secondth
instance, and other to the third.
If you're using sticky session without replication : Imagine you have only one user using your web app, and you have 3 tomcat
instances. This user sends several requests to your app, then the
loadbalancer will send the first user request to one of the three
tomcat instances, and all the other requests that are sent by this
user during his session will be sent to the same tomcat instance.
During these requests, if you shutdown or restart this tomcat
instance (tomcat instance which is used) the loadbalancer sends the
remaining requests to one other tomcat instance that is still
running, BUT as you don't use session replication, the instance
tomcat which receives the remaining requests doesn't have a copy of
the user session then for this tomcat the user begin a session : the
user loose his session and is disconnected from the web app although
the web app is still running.
If you're using sticky session WITH session replication : Imagine you have only one user using your web app, and you have 3 tomcat
instances. This user sends several requests to your app, then the
loadbalancer will send the first user request to one of the three
tomcat instances, and all the other requests that are sent by this
user during his session will be sent to the same tomcat instance.
During these requests, if you shutdown or restart this tomcat
instance (tomcat instance which is used) the loadbalancer sends the
remaining requests to one other tomcat instance that is still
running, as you use session replication, the instance tomcat which
receives the remaining requests has a copy of the user session then
the user keeps on his session : the user continue to browse your web
app without being disconnected, the shutdown of the tomcat instance
doesn't impact the user navigation.
Let's say the user sends a request to get its profile, there won't be anything in the memory of our web application instance. we get the user profile from DB nit before sending the response, we save the data in the memory of let's say Instance3. But the next request from the same user can go to any instance.
When the request first comes to Instance3, that time it will create a session that will have a session id. when the response is sent to the client, the client is supplied with a cookie. so next time this client makes a request, this cookie will be attached to the request, the load balancer will look at the cookie, and the load balancer will know that that request has to be forwarded to Instance3. This is sticky session solution. Its downside is what if Instance3 goes down? the load balancer will route the request to other instances but they do not have a cache. All the users stored in Instance3 will have high latency. This will impact the reliability of your system.
If you store sessions in all instances, now you would have memory issues. Let's say if an instance could store 100 user sessions and you have 3 instances, you would be able to store 300 sessions. But if each instance stores each session, you will be able to store only 100 sessions in all of your 3 instances. So this will impact the scalability of your application.
sticky and non-sticky sessions are components of stateful replication. If you want higher scalability you do not cache anything on your web application instance, your web instance will hit the DB with every request but this will cause high latency.
A better way is stateless replication where you do not store anything on your application instance but instead, you use server-side caching (memcached/redis)
I am evaluating the case of using sticky sessions with Session replication in tomcat. From my initial evaluation, I thought that if we enable Session replication, then session started in one tomcat node will be copied to all other tomcat nodes and thus we do not need sticky session to continue sessions and the request can be picked up by any node.
But it seems that session replication is in general used with sticky sessions, otherwise the session id needs to be changed whenever the request goes to some other node. ref: http://tomcat.apache.org/tomcat-6.0-doc/cluster-howto.html#Bind_session_after_crash_to_failover_node
Can anyone explain what is the real use of session replication if you have to enable sticky session? Because then you would be unnecessarily copying the session on each node, when the request with a given session id is always going to the same node. It could be beneficial in case of a node crashing, but then that does not happen frequently and using session replication only for that seems like an overkill.
As Mindas explained it before :
When you use loadbalancing it means you have several instances of tomcat and you need to divide loads.
If you're using session replication without sticky session : Imagine you have only one user using your web app, and you have 3
tomcat instances. This user sends several requests to your app, then
the loadbalancer will send some of these requests to the first tomcat
instance, and send some other of these requests to the secondth
instance, and other to the third.
If you're using sticky session without replication : Imagine you have only one user using your web app, and you have 3 tomcat
instances. This user sends several requests to your app, then the
loadbalancer will send the first user request to one of the three
tomcat instances, and all the other requests that are sent by this
user during his session will be sent to the same tomcat instance.
During these requests, if you shutdown or restart this tomcat
instance (tomcat instance which is used) the loadbalancer sends the
remaining requests to one other tomcat instance that is still
running, BUT as you don't use session replication, the instance
tomcat which receives the remaining requests doesn't have a copy of
the user session then for this tomcat the user begin a session : the
user loose his session and is disconnected from the web app although
the web app is still running.
If you're using sticky session WITH session replication : Imagine you have only one user using your web app, and you have 3 tomcat
instances. This user sends several requests to your app, then the
loadbalancer will send the first user request to one of the three
tomcat instances, and all the other requests that are sent by this
user during his session will be sent to the same tomcat instance.
During these requests, if you shutdown or restart this tomcat
instance (tomcat instance which is used) the loadbalancer sends the
remaining requests to one other tomcat instance that is still
running, as you use session replication, the instance tomcat which
receives the remaining requests has a copy of the user session then
the user keeps on his session : the user continue to browse your web
app without being disconnected, the shutdown of the tomcat instance
doesn't impact the user navigation.
I think the only real benefit is to be able to shut down Tomcat instances without much thinking. Especially this applies today in cloud world (think Amazon AWS spot instances) when nodes can go on and off really often. Alternative to this would be to buy a decent load balancer which supports node draining. But decent load balancers are expensive, and draining takes time.
Another scenario I can think of is a (poor implementation of) shopping cart where items are kept in the HttpSession and shutting down would require the user to re-purchase them (which would likely lead to a lost sale).
But in most cases you're right - benefit of having both sticky sessions and session replication is very negligible.
Just to clarify configuration issues on JBoss 5.X in "all" base configuration with mod_jk. Setting sticky sessions in workers.properties file
worker.list=loadbalancer
... nodes configuration omitted
worker.loadbalancer.balance_workers=node1,node2
worker.loadbalancer.sticky_session=True
does not prevent session replication. In order to switch off session replication on JBoss we need to set in $JBOSS_HOME\server\YOUR_NODE_NAME\deploy\cluster\jboss-cache-manager.sar\META-INF\jboss-cache-manager-jboss-beans.xml cacheMode parameter to LOCAL.
Usually in sticky session scenario we don't want session replication, since we do not want additional overhead connected with significant amount of I/O operations needed to replicate sessions.
In fact, if go with sticky sessions, we do not need to run JBoss in "all" configuration, we might use "default" or "standard" based configuration.
The only thing that has to be done is change in $JBOSS_HOME/server/YOUR_NODE_NAME/deploy/jbossweb.sar/server.xml:
<Engine name="jboss.web" defaultHost="localhost" jvmRoute="YOUR_NODE_NAME">
What is the difference between session affinity and sticky session in context of load balancing servers?
I've seen those terms used interchangeably, but there are different ways of implementing it:
Send a cookie on the first response and then look for it on subsequent ones. The cookie says which real server to send to.
Bad if you have to support cookie-less browsers
Partition based on the requester's IP address.
Bad if it isn't static or if many come in through the same proxy.
If you authenticate users, partition based on user name (it has to be an HTTP supported authentication mode to do this).
Don't require state.
Let clients hit any server (send state to the client and have them send it back)
This is not a sticky session, it's a way to avoid having to do it.
I would suspect that sticky might refer to the cookie way, and that affinity might refer to #2 and #3 in some contexts, but that's not how I have seen it used (or use it myself)
As I've always heard the terms used in a load-balancing scenario, they are interchangeable. Both mean that once a session is started, the same server serves all requests for that session.
Sticky session means that when a request comes into a site from a client all further requests go to the same server initial client request accessed. I believe that session affinity is a synonym for sticky session.
They are the same.
Both mean that when coming in to the load balancer, the request will be directed to the server that served the first request (and has the session).
Sticky session means to route the requests of particular session to the same physical machine who served the first request for that session.
This article clarifies the question for me and discusses other types of load balancer persistence.
Dave's Thoughts: Load balancer persistence (sticky sessions)
difference is explained in this article:
https://www.haproxy.com/blog/load-balancing-affinity-persistence-sticky-sessions-what-you-need-to-know/
Main part from this link:
Affinity: this is when we use an information from a layer below the application layer to maintain a client request to a single server. Client's IP address is used in this case. IP address may change during same session and then connection may switch to different server.
Persistence: this is when we use Application layer information to stick a client to a single server. In this case, loadbalancer inject some cookie in response and use same cookie in subsequent request to route to same server.
sticky session: a sticky session is a session maintained by persistence
The main advantage of the persistence over affinity is that it’s much more accurate, but sometimes, Persistence is not doable(when client dont allow cookies like cookie less browser), so we must rely on affinity.
Using persistence, we mean that we’re 100% sure that a user will get redirected to a single server.
Using affinity, we mean that the user may be redirected to the same server…
They are Synonyms.
No Difference At all
Sticky Session / Session Affinity:
Affinity/Stickiness/Contact between user session and, the server to which user request is sent is retained.