https://learn.microsoft.com/en-us/azure/azure-cache-for-redis/cache-planning-faq The document mentions the Throughput numbers for GETS. But there are multiple client connections possible and there is also a limit based on the Pricing tier.
Question: Is the given number on "GET Requests per second" per client connection OR after creating a max possible connection with Redis cache and running GET operations from each client?
That's the total GETs/second regardless of the number of connections. I believe we tested with 50 connections. With lower numbers of connections, you may hit bottlenecks in the throughput of client instances or network connections before hit the limits of the server.
We always recommend benchmarking throughput with your application's actual architecture and workload to find actual cache capabilities for your use case: https://learn.microsoft.com/en-us/azure/azure-cache-for-redis/cache-best-practices-performance
Related
In Sequelize.js you should configure the max connection pool size (default 5). I don't know how to deal with this configuration as I work on an autoscaling platform in AWS.
The Aurora DB cluster on r3.2xlarge allows 2000 max connections per read replica (you can get that by running SELECT ##MAX_CONNECTIONS;).
The problem is I don't know what should be the right configuration for each server hosted on our EC2s. What should be the right max connection pool size as I don't know how many servers will be launched by the autoscaling group? Normally, the DB MAX_CONNECTIONS value should be divided by the number of connection pools (one by server), but I don't know how many server will be instantiated at the end.
Our concurrent users count is estimated to be between 50000 and 75000 concurrent users at our release date.
Did someone get previous experience with this kind of situation?
It has been 6 weeks since you asked, but since I got involved in this recently I thought I would share my experience.
The answer various based on how the application works and performs. Plus the characteristics of the application under load for the instance type.
1) You want your pool size to be > than the expected simultaneous queries running on your host.
2) You never want your a situation where number of clients * pool size approaches your max connection limit.
Remember though that simultaneous queries is generally less than simultaneous web requests since most code uses a connection to do a query and then releases it.
So you would need to model your application to understand the actual queries (and amount) that would happen for your 75K users. This is likely a lot LESS than 75K/second db queries a second.
You then can construct a script - we used jmeter - and run a test to simulate performance. One of the items we did during our test was to increase the pool higher and see the difference in performance. We actually used a large number (100) after doing a baseline and found the number made a difference. We then dropped it down until it start making a difference. In our case it was 15 and so I set it to 20.
This was against t2.micro as our app server. If I change the servers to something bigger, this value likely will go up.
Please note that you pay a cost on application startup when you set a higher number...and you also incur some overhead on your server to keep those idle connections so making larger than you need isn't good.
Hope this helps.
Is it even possible to fake the traffic, considering your network connection will have a bandwidth limit? If I create a test for 1000 users, visiting 3 pages a second, will the results really represent this scenario if done in real life (not simulation)?
Should I create an executable to perform the load test, and run it from separate network connections?
It looks like you have a concern that the network bandwidth limit of the test machine will invalidate the accuracy of the test. To determine if your concern is legit, perform the following rough assessment. Let say your three pages have sizes 20k, 40k and 60k respectively. To load test them you would need to generate the following maximum bandwidth:
1000x (20k+40k+60k)= 120Mbps
Make an adjustment for compression if you use it. Factor in the requests sizes if they are significant. If you test with browser caching emulation enabled, increase the size of the first requests by the size of static resources. Then compare the maximum bandwidth with your network adapter bandwidth. If it is 100 Mbps, then the network adapter will be a bottleneck. If it is 1 Gps or more, then the network adapter will not be a bottleneck.
Alternatively, skip calculations and just run the test. On the bandwidth graph in VSTS find the max value. If it is by far smaller than your network adapter limit, then you can run this test from one machine.
Keep in mind that requesting 1000 x 3 pages per second does not guarantee that you will receive responses from 1000 X 3 pages per second due to the server speed limit.
Is there an upper limit on how many concurrent connections that MarkLogic can process? For example, ASP.NET is restricted to processing 10 requests concurrently, regardless of infrastructure and hardware. Is there a similar restriction in MarkLogic server? If not, are there any benchmarks that give some indication as to how many connections a typical instance can handle?
Given a large enough budget there is no practical limit on the number of concurrent connections.
The basic limit is the application server thread count, although excess requests will also pile up in the backlog queue. According to groups.xsd each application server is limited to at most 256 threads. The backlog seems to have no maximum, but most operating systems will silently limit it to something between 256-4096. So depending on whether or not you count the backlog, a single app server on a single host could have 256-4352 concurrent connections.
After that you can use multiple app servers, and add hosts to the cluster. Use a load balancer if necessary. Most operating systems will impose a limit of around 32,000 - 64,000 open sockets per host, but there is no hard limit on the number of hosts or app servers. Eventually request ids might be a problem, but those are 64-bit numbers so there is a lot of headroom.
Of course none of this guarantees that your CPU, memory, disk, and network can keep up with the demand. That is a separate problem, and highly application-specific.
When creating an AmazonDynamoDBClient in the Java API we can specify the maximum connection pool size using setMaxConnections on ClientConfiguration. Is there a hard / recommended limit on this? For example although the default limit is 50 connections a linux client should be able to sustain 5,000 open connections, will Amazon allow this?
If there is a maximum limit does it only apply to a single client instance? What about if there are several machines using dynamo through the same account, will they share a connection limit?
Thanks!
Since the connections to Amazon DynamoDB is http(s) based, the concept of open connections is limited to your tcp max open connections at once. I highly doubt there's a limit on Amazons end at all as it is load balanced close to infinity.
Naturally, the exception is your read and write capacity limits. Note that they want you to contact them if you will exceed a certain amount of capacity units which depend on your region.
You've probably already read them, but the limitations of DynamoDB is found here:
http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Limits.html
My testing shows that amazon load balancer rest connection with its instance when it has about 10k concurrent connections into it. Is that a limit of Amazon load balancer? If not, is there a setting for it? I need to support upto 1M concurrent connections for my testing.
Thanks,
Sean Nguyen
The ELB should scale way beyond that, but you need to be testing from multiple test clients that appear to come from unique source IPs. This will cause multiple ELB instances to spawn multiple instances behind the scenes (this can be detected by DNS lookups). This is explained in the whitepaper that Rightscale published:
http://blog.rightscale.com/2010/04/01/benchmarking-load-balancers-in-the-cloud/
Note that it takes a little while for ELB resources to scale out, so tests need to run for 20 minutes or more.
You also need to be sure that you have enough resources behind the load balancer. EC2 instances (as shown in the white paper mentioned above) seem to hit a throughput limit of around 100k packets per second which limits the number of concurrent connections that can be served (bear in mind the overhead of TCP and HTTP). You will need a lot of instances to be able to cope with 1M concurrent connections, and I'm not sure at what point you will hit the limit of ELB; in RightScale's test they only hit 19k.
Also you need to be clear about exactly what you mean by 1M concurrent connections, do you mean total keep-alive connections (assuming keep-alive enabled), or do you mean 1M transactions per second?