Is there a way to limit download speed for end user in Envoy?
Ideally it should be dynamic (different download speed limit for different user).
There is some link https://www.envoyproxy.io/docs/envoy/latest/api-v2/config/filter/fault/v2/fault.proto#config-filter-fault-v2-faultratelimit-fixedlimit but not sure how it's applicable for that task.
Related
In basic terms, I want to make sure that our Livestreaming Shows can without issues have 10,000 viewers at one time. That the following things are working well:
Video Quality
Video Resolution
Video Latency
Do this can be done using local machine , I read local machine cannot produce such huge number of requests.
Do I needed to purchase addition premium platform or it can be done using jmeter alone.
Do this can be done using local machine - we don't know, it depends on your machine hardware specifications. I would take the following steps:
set up monitoring of the machine's resources consumption like CPU, RAM, etc. If you don't have better alternatives - you can go for JMeter PerfMon Plugin
make sure to follow JMeter Best Practices
start with 1 virtual user and gradually increase the load till 10 000 at the same time looking at the resources consumption
when any of monitored resources starts exceeding reasonable threshold, i.e. 90% of total available capacity stop the test and check how many users were online at this stage via i.e. Active Threads Over Time plugin
this is how many users you can simulate for particular this test from particular this machine. If it's 10 000 - you're good to go with a single one, if it's less - divide 10 000 to the number of users you were able to mimic and this will be the number of machines of that hardware specifications you will need for the test
JMeter out of box can be run in the clustered mode so given you have the machines you can use as the load generators there is no need to purchase anything else. If you don't - you can rent VMs from i.e. MS Azure or AWS EC2 or whatever is your favorite cloud provider. In this case you will need to pay for the machine/computing time according to the vendor price list
There are companies which offer "JMeter as a service", normally they charge more than cloud VM vendors but you won't need to worry about JMeter distributed configuration, results collection, etc. They are BlazeMeter, Flood.io, Redline13, etc.
I have a 32GB, i7 core processor running on windows 10 and I am trying to generate 10kVU concurrent load via jmeter. For some reason I am unable to go beyond 1k concurrent and I start getting BindException error or Socket connection error. Can someone help me with the settings to achieve that kind of load? Also if someone is up for freelancing I am happy to consider that as well. Any help would be great as I am nearing production and am unable to load test this use case. If you guys have any other tools that I can use effectively, that would also help.
You reach the limit of 1 computer, thus you must execute in distributed environment of multiple computers.
You can setup JMeter's distributed testing on your own environment or use blazemeter or other cloud based load testing tool
we can use BlazeMeter, which provides us with an easy way to handle our load tests. All we need to do is to upload our JMX file to BlazeMeter. We can also upload a consolidated CSV file with all the necessary data and BlazeMeter will take care of splitting it, depending on the amount of engines we have set.
On BlazeMeter we can set the amount of users or the combination of engines (slave systems) and threads that we want to apply to our tests. We can also configure additional values like multi locations.
1k concurrent sounds low enough that it's something else ... it's also the default amount of open file descriptor limits on a lot of Linux distributions so maybe try to raise the limit.
ulimit -Sn
will show you your current limit and
ulimit -Hn
will show you the hard limit you can go before you have to touch configuration files. Editing /etc/security/limits.conf as root and setting something like
yourusername soft nofile 50000
yourusername hard nofile 50000
yourusername - will have to be the username of the user which with you run jmeter.
After this you will probably have to restart in order for the changes to take effect. If not on Linux I don't know how to actually do this you will have to google :D
Recommendation:
As a k6 developer I can propose it as an alternative tool, but running 10k VUs on a single machine will be hard with it as well. Every VU will take some memory - like at least 1-3mb and this will go up the larger your script is. But with 32gb you could still run upto 1-2kVUs and use http.batch to make concurrent requests which might simulate the 10k VUs depending on what your actual workflow is like.
I managed to run the stages sample with 300VUs on a single 3770 i7 CPU and 4gb of ram in a virtual machine and got 6.5k+ rps to another virtual machine on a neighboring physical machine (the latency is very low) so maybe 1.5-2kVUs with a a somewhat more interesting script and some higher latency as this will give time to the golang to actually run GC while waiting for tcp packets. I highly recommend using discardResponseBodies if you don't need them and even if you need some to actually get the response for them. This helps a lot with the memory consumption a each VU
I am looking at load testing of Progressive download video files with 100 user load. The testing tool I am looking at is Jmeter, Load Runner and NeoLoad. Though the script required for creating the load is very simpler, it consist of couple of request and it is able to make the connection with server and start the downloading of the file. Though I understand that the progressive technology is pretty old, but still it is used in many website. The question I have is around the strategy.
Do we need to download the complete file(i.e. 1.3 GB in my case)?
Even we looked at saving the response as file, the resources such as Network and disk I/O are at max? Does this strategy suits here?
Can we have some another strategy where we can engage the server for the duration and test for issues underlying with connection issues and transmission speed?
Depending on your use case, there is Seeking feature so theoretically you should be able to specify start offset so you will not have to get the whole file. Also you can consider using HTTP Header Manager to send Range header
If your target is to verify that the file has been downloaded fully and it is not broken you can tick "Save Response as MD5 Hash" box on "Advanced" tab of the HTTP Request sampler - this way you will save at least 130 GB of disk space. MD5 checksum can be verified using i.e. MD5Hex Assertion
The main idea of the load testing is simulating real application usage with 100% accuracy. Not knowing the requirements of your product it is impossible to come up with suggestions, however JMeter can be configured to behave pretty much like real browser does so it is a viable option.
See Load Testing Video Streaming with JMeter: Learn How article for more information if needed.
I have webportal for testing, our product owner want us to do measure performace of website by using chrome's network tab. Is it correct way to measure performance. My product owner wants me to mesaure Finish time and consider that time for performance of web portal. So my question is, is it a correct way of performance testing.Chrome Network tab finish time
This tool reports client-side performance, you can rely on it up to certain extent as it reports how long did page load take from browser's perspective.
However the test is not very representative as for example I'm getting the page loaded in ~3 seconds (and I'm using flaky mobile internet) while you have ~13 seconds.
So I would recommend considering a better tool, for example YSLow which not only measures rendering time but also can detect and report related issues.
Another point is, given application response time is 3 seconds while 1 user is accessing it, how do you know what response time will be when 100 users will be concurrently working with the application? 500 users? 1000 users? That's why you should not limit your performance testing to measuring DOM load and rendering speed only, you need to mimic at least anticipated number of your application users simultaneously accessing the application - this will give you some initial performance metrics to see whether your application behaviour is acceptable or not.
I have requirement of test api with 10k users. What I choose is :
Jmeter
Linux server to install Jmeter and throw load with 10k users
One node for now
Will perform API with operations like :
Login
Book hotel with post parameters
Update Booking Details
Save Booking
I am thinking to use above for API testing with 10k users, Above tools are enough or I should look for other options like loadimpact, loader or blazemeter?
If you are talking about API you should be rather considering "requests per second" rather than "users" as I strongly doubt that end users will be sending requests to API endpoints via curl or Postman.
No matter whether your goal is "users" or "requests per second" it is only you who can answer as it depends on many parameters like:
your machine hardware specifications
software specifications (OS/JVM/JMeter version and architecture)
nature of your test (request/response size, number of pre/post processors, assertions, etc)
So you should act as follows:
Make sure you're following JMeter Best Practices
Make sure you monitor baseline health metrics of the node which is running JMeter (CPU, RAM, Swap, Network, Disk), you can use JMeter PerfMon Plugin for that .
Start with minimal load (1 virtual user or 1 request per second) and increase the load until your machine starts swapping or any other health metric exceeds, say, 80% of maximum available capacity. Once it happens take a look into active threads (Active Threads Over Time listener) or throughput (Transactions Per Second) - this is the maximum number of users or hits per second you can produce on particular this host for particular this test. If it is enough - you're good to go, if not - you will have to switch to Distributed testing
See What’s the Max Number of Users You Can Test on JMeter? article for more details.
The whole answer is an elaborate it depends.
The best thing you can do is to run PerfMon agents on the server generating the load as well as on the server running the system under test.
This way you should see (in the CPU utilization and free memory statistics) whether you had maxed what the server providing the API can do or whether it is your load generator running out of steam. In the first case you got some readline based on the hardware and configuration you had run with. In the second one you have an indication to employ more than 1 box to generate the load or to investigate settings and options.
Have a closer look at PerfMon JMeter plugin for exact details.