I have a really simple setup: An azure load balancer for http(s) traffic, two application servers running windows and one database, which also contains session data.
The goal is being able to reboot or update the software on the servers, without a single request being dropped. The problem is that the health probe will do a test every 5 seconds and needs to fail 2 times in a row. This means when I kill the application server, a lot of requests during those 10 seconds will time out. How can I avoid this?
I have already tried running the health probe on a different port, then denying all traffic to the different port, using windows firewall. Load balancer will think the application is down on that node, and therefore no longer send new traffic to that specific node. However... Azure LB does hash-based load balancing. So the traffic which was already going to the now killed node, will keep going there for a few seconds!
First of all, could you give us additional details: is your database load balanced as well ? Are you performing read and write on this database or only read ?
For your information, you have the possibility to change Azure Load Balancer distribution mode, please refer to this article for details: https://learn.microsoft.com/en-us/azure/load-balancer/load-balancer-distribution-mode
I would suggest you to disable the server you are updating at load balancer level. Wait a couple of minutes (depending of your application) before starting your updates. This should "purge" your endpoint. When update is done, update your load balancer again and put back the server in it.
Cloud concept is infrastructure as code: this could be easily scripted and included in you deployment / update procedure.
Another solution would be to use Traffic Manager. It could give you additional option to manage your endpoints (It might be a bit oversized for 2 VM / endpoints).
Last solution is to migrate to a PaaS solution where all this kind of features are already available (Deployment Slot).
Hoping this will help.
Best regards
I have to perform load testing on a load balanced (clustered) system composed of three servers.
Is it a good practice to test via JMeter each server ? Or maybe it would be better testing the whole cluster calling the load balancer dedicated endpoint ?
Thanks !
Well behaved load test needs to mimic real life application usage as close as possible therefore if the load balancer acts as a single entry point to the system - JMeter needs to hit this endpoint only so the whole system will be like a "black box".
With regards to distributed systems testing best practices you can also consider the following couple of areas:
Load balancers may route requests depending on the origin so it might be a good idea to implement IP Spoofing so each JMeter virtual user could have its own source IP address.
Load Balancer endpoint host(s) may have multiple IP addresses so consider DNS Cache Manager to your Test Plan so each JMeter virtual user could resolve endpoint address on its own as due to caching of DNS calls on OS or JVM level your test can hit one node only while others will be idle.
Testing the whole system is always better. I mean you can find out problem with the load balancing management. It's always better to be in the same condition as the production environment.
After your assessment of the first test. You can regulate and find out that maybe one server is slowing down the chain with another test.
The answer is both (and you should find more cases.), You need to test your system on load as much as close to real environment to know its capabillities.
But also ,for example, when upgrading a version, sometimes only a few or even one server remain online and you need to know what load it can sustain .
I have 2 web servers (IIS 8.5) behind a hardware firewall and our application uses SignalR for some real-time updates. We are using SQL Server as the backplane to help us work in this load balanced environment. Additionally we are using sticky sessions on the load balancer to help us keep the users on the same web server during their session. When we are running in this hardware configuration we lose at least 1/3 of our messages. Sometimes we get all the expected messages but more often than not we are missing plenty.
When we are running on a single web server all messages are received. Does anyone have any suggestions for troubleshooting this problem? We've turned on logs (both client & server) and nothing looks like it's missing or broken. We're really stumped.
EDIT---
Some additional details that I hope will shed light on the situation.
Server to Client messages are getting lost. Pretty much all our communication is Server to Client.
We are using sticky session just based on IP and limited to 5 minutes but we're losing messages within that 5 minutes.
This is some old SignalR code that has been only minimally touched since SignalR 1 (or even older). We are keeping an in memory list of users along with their connections and we use that list to send notices back to the client. It seems most likely that this is the cause of the troubles but with Sticky sessions the user should be stuck to the same server for at least the 5 minutes right?
This list of users maps Username to connection id. This is useful when our backend services (on another machine) sends a message back with the username not the connection id.
Finally resolved this. There were 2 issues really. The first is that we were using an in memory list of users as mentioned in the edit above. Once we realized that wasn't going to work across machines we removed it. It also led us to the second issue which was how SignalR 2 uses the IUserIdProvider and our call should have been
Clients.User(userId).send(message)
instead of
context.Clients.Client(connection)
This code had existed since we first started using SignalR many years ago and never got properly updated as we upgraded SignalR versions
Have the same machineKey specified in your web.config on both servers.
I've been building some apps that connect to a SQL backend. I use ajax calls to hit WebMethods, a WebAPI, etc.
I notice that the first initial call to the SQL backend retrieves the data fairly slow. I can only assume that this is because it must first negotiate credentials first before retrieving the data. It probably caches this somewhere, and thus, any calls made afterwards come back very fast.
I'm wondering if there's an ideal, or optimal way, to initialize this connection.
My thought was to make a simple GET call right when the page loads (grabbing something very small, like a single entry). I probably wouldn't be using the returned data in any useful way, other than to ensure that any calls afterwards come back faster.
Is this an okay way to approach fixing the initial delay? I'd love to hear how others handle this.
Cheers!
There are a number of reasons that your first call could be slower than subsequent ones
Depending on your server platform, code may be compiled when first executed
You may not have an active DB connection in your connection pool
The database may not have cached indices or data on the first call
Some VM platforms may take a while to allocate sufficient resources to your server if it has been idle for a while.
One way I deal with those types of issues on the server side is to add startup code to my web service that fetches data likely to be used by many callers when the service first initializes (e.g. lookup tables, user credential tables, etc).
If you only control the client, consider that you may well wish to monitor server health (I use the open source monitoring platform Zabbix. There are also many commercial web-based monitoring solutions). Exercising the server outside of end-user code is probably better than making an extra GET call from a page that an end user has loaded.
Session variables are normally keept in the web server RAM memory.
In a cluster, each request made by a client can be handled by a different cluster node. right?!
So, in this case...
What happens with session variables? Aren't they stored in the nodes RAM memory?
How the other nodes will handled my request correctly if it doesn't have my session variables, or at least all of it?
This issue is treated by the web server (Apache, IIS) or by the language runtime (PHP, ASP.NET, Ruby, JSP)?
EDIT: Is there some solution for Classic ASP?
To extend #yogman's answer.
Memcached is pure awesomeness! It's a high performance and distributed object cache.
And even though I mentioned distributed it's basically as simple as starting one instance on one of your spare/idle servers, you configure it as in ip, port and how much ram to use and you're done.
memcached -d -u www -m 2048 -l 10.0.0.8 -p 11211
(Runs memcached in daemon mode, as user www, 2048 MB (2 GB) of RAM on IP 10.0.0.8 with port 11211.)
From then on, you ask memcached for data and if the data is not yet cached you pull it from the original source and store it in memcached. I'm sure you are familiar with cache basics.
In a cluster environment you can link up your memcached's into a cluster and replicate the cache across your nodes. Memcached runs on Linux, Unix and Windows, start it anywhere you have spare RAM and start using your resources.
APIs for memcached should be generally available. I'm saying should because I only know of Perl, Java and PHP. But I am sure that e.g. in Python people have means to leverage it as well. There is a memcached wiki, in case you need pointers, or let me know in the comments if I was raving too much. ;)
There are 3 ways to store session state in ASP.NET. The first is in process, where the variables are stored in memory. The second is to use a session state service by putting the following in your web.config file:
<sessionState
mode="StateServer"
stateConnectionString="tcpip=127.0.0.1:42424"
sqlConnectionString="data source=127.0.0.1;user id=sa;password="
cookieless="false"
timeout="20" />
As you can see in the stateConnectionString attribute, the session state service can be located on a different computer.
The third option is to use a centralized SQL database. To do that, you put the following in your web.config:
<sessionState
mode="SQLServer"
stateConnectionString="tcpip=127.0.0.1:42424"
sqlConnectionString=
"data source=SERVERHAME;user id=sa;password="
cookieless="false"
timeout="20"
/>
More details on all of these options are written up here: http://www.ondotnet.com/pub/a/dotnet/2003/03/24/sessionstate.html
Get a Linux machine and set up http://www.danga.com/memcached . Its speed is unbeatable compared to other approaches. (for example, cookies, form hidden variables, databases)
As with all sorts of thing, "it depends".
There are different solutions and approaches.
As mentioned, there's the concept of a centralized store for session state (database, memcached, shared file system, etc.).
There are also cluster wide caching systems available that make local data available to all of the machines in the cluster. Conceptually it's similar to the centralized session state store, but this data isn't persistent. Rather it lives within the individual nodes and is replicated using some mechanism provided by your provider.
Another method is server pinning. When a client hits the cluster the first time, some mechanism (typically a load balancer fronting the cluster) pins the client to a specific server. In a typical client lifespan, that client will spend their entire time on a single machine.
For the failover mechanism, each machine of the cluster is paired with another machine, and so any session changes are shared with the paired machine. Should the clients pinned machine encounter an issue, the client will hit another machine. At this point, perhaps due to cookies, the new machine sees that it's not the original machine for the client, so it pings both the original machine, and the paired machine for the clients session data.
At that point the client may well be pinned to the new machine.
Different platforms do it in different ways, including having no session state at all.
With Hazelcast, you can either use Hazelcast distributed map to store and share sessions across the cluster or let Hazelcast Webapp Manager do everything for you. Please check out the docs for details. Hazelcast is a distributed/partitioned, super lite and easy, free data distribution solution for Java.
Regards,
-talip
http://www.hazelcast.com
To achieve load balancing for classic ASP, you may store the user specific values in the database and pass a reference unique id in the URL as follows.
Maintain a session table in the database which generates a unique id for each record. The first time you want to store session specific data, generate a record in your session table and store the session values in it. Obtain the unique id of the new session record and re-write all links in your web application to send the unique id as part of querystring.
In every subsequent page where you need the session data, query the session table with the unique id passed in the querystring.
Example:
Consider your website to have 4 pages: Login.asp, welcome.asp, taskList.asp, newtask.asp
When the user logs in using login.asp page, after validating the user, create a record in session table and store the required session specific values (lets say user's login date/time for this example). Obtain the new session record's unique id (lets say the unique id is abcd).
Append all links in your website with the unique id as below:
welcome.asp?sessionId=abcd
tasklist.asp?sessionId=abcd
newtask.asp?sessionId=abcd
Now, if in any of the above web pages you want to show the user's login date/time, you just have to query your session table with the sessionID parameter (abcd in this case) and display to the user.
Since the unique value identifying the session is a part of the URL, any of your web servers serving the user will be able to display the correct login date/time value.
Hope this helps.
In ASP.NET you can persist session data to an SQL Server database which is common to all web servers in the cluster.
Once configured (in the web.config for your site), the framework handles all of the persistance for you and you can access the session data as normal.
As Will said, most load-balancing approaches will use some sort of stickiness in the way the distribute forthcoming requests from the same client, meaning, a unique client will hit the same server unless that actual server goes down.
That minimizes the need of distribution of session-data, meaning that only in the eventual failure of a server, a client would loose his session. Depending on your app, this is more or less critical. In most cases, this is not a big issue.
Even the simplest way of loadbalacing (round-rubin the DNS-lookups) will do some sort of stickiness since most browsers will cache the actual lookup and therefor keep going to the first record it received, AFAIK.
It's usually the runtime that is responsible for the sessiondata, in for exampla PHP it's possible to define your own session-handler, which can persist the data into a database for instance. By default PHP stores sessiondata on files, and it might be possible to share these files on a SAN or equivalent in order to share session-data. This was just a theory I had but never got around to test since we decided that loosing sessions wasn't critical and didn't want that single point of failure.