I'm trying to stress test my Spring Boot application, but when I run the following command, what ab is doing is that trying to give out a result the the maximum my application could holds. But what I need is to check whether my application could hold at a specific request per second.
ab -p req.json -T application/json -k -c 1000 -n 500000 http://myapp.com/customerTrack/v1/send
The request per second given from above command is 4000, but actually, a lot of records are buffered in my application which means it can't hold that much rps. Could anyone tell me how to set a specific request per second in ab tools? Thanks!
I don't think you can get what you want from ab. There are a lot of other tools out there.
Here's a simple one that might do exactly what you want.
https://github.com/rakyll/hey
For rate limiting to 100 requests per second the below command should work.
hey -D req.json -T application/json -c 1000 -q 100 -n 500000 http://myapp.com/customerTrack/v1/send
Apache Bench is single threaded program that can only take advantage of one processor on your client’s machine. In extreme conditions, the tool could misrepresent results if the parameters of your test exceed the capabilities of the environment the tool is running in. Accorading to your description, the rps has already reach your hardware limitation.
A lot of records are buffered in my application which means it can't hold that much rps
It is very hard to control request per second in single machine.
You can find better performacne testing tools from here HTTP(S) Benchmark Tools
If you have budget you can try goad, which is an AWS Lambda powered, highly distributed, load testing tool built in Go for the 2016 Gopher Gala. Goad allows you to load test your websites from all over the world whilst costing you the tiniest fractions of a penny by using AWS Lambda in multiple regions simultaneously.
Related
In Jmeter I have a scenario like
Load tested with 4000 users and 1 hour duration
759965 requests made and out of which one request failed on an average 18894.13 requests made per second.
This was the earlier scenario and I want to make the same scenario again with the above information. Can someone guide me how to set up the environment and also the results. I have designed my script using Co-relation with the help regular expression extractor.enter image description here
For the normal Thread Group the configuration would be something like:
It would also be a good idea to use some ramp-up period so the load would increase gradually and you could correlate increasing load with other metrics like response time or transactions per second.
You might also want to use one of Custom Thread Groups which can be installed as JMeter Plugins, they provide easy visual way to define the number of threads, test duration, ramp-up, ramp-down, time to hold the load, eventual spikes, etc.
Once you define your desired workload you should run your test in command-line non-GUI mode, with regards to the test results the easiest option is to generate HTML Reporting Dashboard
ULTIMATE GOAL: publish 100 "heartbeat" messages/second over MQTT using Mosquitto as a broker.
To send a single heartbeat, I can easily do mosquitto_pub -t "ems/heartbeat" -m 0.
I am aware of the watch utility in Unix systems but is not fast enough for the goal. My first approach to the problem was to scale this up by using the command while sleep 0.01; do mosquitto_pub -t "ems/heartbeat" -m 0; done. By subscribing to the "ems/heartbeat" topic (mosquitto_sub -t "ems/heartbeat") I am fairly sure that the messages published to the topic are way less than the 100 expected in the timeframe of 1 second. So here is my question: how can I run a shell script - the heartbeat above - 100 times a second, or even better, how can I publish 100 messages/sec over a certain MQTT topic?
The title might seem a bit misleading compared to what is my ultimate goal, but finding a way to run a shell script 100 times/sec should do the trick. If there are different ways to tackle the problem, they are of course welcome! Thanks!
EDIT & ADDITIONAL INFORMATION:
The receiver of these messages is just a microcontroller that needs to check that the connection with the Electronic Monitoring System (a laptop) is alive. The need for 100 messages/sec is given by the fact that the micro is providing control to highspeed actuators and when a connection loss occurs, everything needs to be in a safe state.
The basic assumption is that the 100 messages are spread over a 1-second time span and are coming from a single entity, the EMS.
By running the time command this is the output, so no way it possible to use mosquitto_pub to send 100 messages a second, as pointed out.
time mosquitto_pub -t "ems/heartbeat" -m 0
real 0m0.039s
user 0m0.006s
sys 0m0.011s
The short answer is don't.
Use a proper load generating tool, e.g. jmeter has MQTT support.
Otherwise use a proper MQTT client library and connect once and then just publish in a loop. That way you don't have the overhead of setting up and tearing down a new connection to the broker for each message (which is what driving mosquitto_pub this way is doing)
I'm trying to test my https endpoint with JMeter. I want to make at least 10000 requests per second, but when I set the number of threads to 10000 I get way less rps, around 500.
I've tried setting the number of threads to 1000 and 100, surprisingly I get this same number of rps. I'm using HTTP Sampler and "use Keep-Alive" is set to true. When I look in the statistics I see that when using 100 threads, it makes use of Keep-Alive and connect_time is around 100 ms, but when the number of threads is higher connect_time grows, it's like it stops reusing the connections.
I know this isn't a server issue, because I've tried testing that same endpoint with Yandex.Tank and phantom and it can easily maintain 10 000 requests per second, the problem is it can't use response data to make furhter requests, that's why I have to use JMeter for this task.
This can be done by using "Stepping thread group". It will allow you to send 10000 request per second upto specified time. Refer below image.
Stepping Thread Group
Download jar from below link.
https://jmeter-plugins.org/wiki/SteppingThreadGroup/
I hope you are trying to achieve this using one machine. Try with multiple machine or jmeter distributed mode.
https://jmeter.apache.org/usermanual/jmeter_distributed_testing_step_by_step.pdf
https://www.blazemeter.com/blog/how-to-perform-distributed-testing-in-jmeter/
https://blazemeter.com/blog/3-common-issues-when-running-jmeter-scripts-and-how-solve-them/
I am assuming that it is the issue with machine which is not able to generate that much load. Usually, i have use max 300 threads per machine but it depend on the machine config. Just check if the machine is having issue and multiple machine is able to generate more load, considering server is not having any issue.
Hope this helps.
Update:-Usually 200-500 can be handled my modern machines.
Please check the below link to have some more info:-
1.How do threads and number of iterations impact test and what is JMeter’s max. thread limit
2.https://www.blazemeter.com/blog/what%e2%80%99s-the-max-number-of-users-you-can-test-on-jmeter/ .
If suppose I will run load testing with 5000 threads or may be more ,
Will the main server under test crash at a certain level.
As EJP said, it is JMeter purpose to find the limit of tested application and how it will react under performance.
So yes it is perfectly possible.
You should read JMeter Manual.
If you are doing any kind of performance tests (load, stress, soak etc) you will want to know at what point your application server falls over i.e. its breaking point.
Once you've found out what your upper limit is, start dialing back the number of threads until you find your application's "sweet spot" for example CPU usage <80% & >70%
I want to load test a URL by hitting it few hundred times at same millisecond . I tried JMeter but I could hit 2 request at same millisecond. This seems to be problem that my machine cant create threads fast enough . Is there any solution to the issue ?
In JMeter you can use synchronizing timer setting it to 100, this way all threads will wait until there are 100 available and hit the server:
http://jmeter.apache.org/usermanual/component_reference.html#Synchronizing_Timer
Another solution is to increase the number of Threads so that you hit this throughput.
In next coming version (2.8) of JMeter you will be able to create threads on demand (created once needed).
Anyway hitting few hundred times at same millisecond is a high load so you will have to tune JMeter correctly.
Regards
Philippe
JMeter uses blocking HTTP client, in order to hit the server at the exact same time with 100 reqeusts you need 100 threads in JMeter. Even providing that, you still don't have 100 cores to actually run such code at the same time. Even if you had 100 cores, it takes some time to start a thread, so you would have to start them in advance and synchronize on some sort of barrier. And that is not supported in JMeter.
Why do you really want to run your server "at same millisecond"? An ordinary load test just calls the server with as many connections as possible, but not necessarily at the same time. Moreover, sometimes you are even adding random sleep between requests to simulate so-called think time.
As per Philippe's answer, JMeter does in fact support synchronous requests. But maybe for what you want something like Apache Bench using -c100 (or tune it to whatever works) is a better option? It's pretty basic stuff but then the overhead is a lot smaller which might help in this situation.
But I would also steal from Tomasz's answer and echo his concern that perhaps this is not really the best way to approach load testing. If you're trying to replicate real life traffic then do you really need such a high level of concurrency?
You need to use Jmeter-server and a host of client machines for load generation. Your single machine is not enough to generate the load itself.