How to properly load test JMS with JMeter? - jmeter

Question

I've set up a thread group with JMS Point-to-Point sampler and it works ok with our application.
I send an xml message, using ${__UUID()} function for some fields in order to guarantee the message is unique, and expect a response with a timeout of 60s and the content should contain a simple pattern (Response Assertion).
I tried to do a simple load test sending 1000 messages but I'm confused about how I should configure the threads. I get different results with different configurations.
Case 1:
- Threads: 1000
- Ramp-up: 1
- Loop: 1
- Avg time/sample: ~80s
- Total time: 02:41
Case 2:
- Threads: 1000
- Ramp-up: 10
- Loop: 1
- Avg time/sample: ~60s
- Total time: 01:43
- Errors: 3%
Case 3:
- Threads: 1000
- Ramp-up: 100
- Loop: 1
- Avg time/sample: ~12s
- Total time: 02:13
Case 4:
- Threads: 1
- Ramp-up: 1
- Loop: 1000
- Avg time/sample: ~1.2s
- Total time: >16min
Case 5:
- Threads: 10
- Ramp-up: 1
- Loop: 100
- Avg time/sample: ~1.1s
- Total time: 02:12
Case 6:
- Threads: 100
- Ramp-up: 1
- Loop: 10
- Avg time/sample: ~7.3s
- Total time: 01:30
How I'm supposed to interpret those results? Which configuration should I use?

Answer 1

It depends on what you're trying to achieve, the main performance testing types are:
Load Testing - when you put your system under anticipated load and see if response times/number of transactions per second are expected in terms of non-functional requirements or SLAs. If this is the case - just configure JMeter to exactly replicate the expected usage of your application and that would be your "configuration"
Stress Testing - when you're identifying the limits of your system and looking for the bottleneck. In this case I would recommend starting from 1 thread and gradually increasing the load at the same time looking into the following charts:
Active Threads Over Time
Response Times Over Time
Transactions Per Second
Response Codes Per Second
Ideally the number of transactions per second should increase as you increase the number of users and response time should remain the same, however I expect that at some point you will see that response time will go up and transactions per second will go down - that would indicate the saturation point - the point of maximum system performance. You can record the number of active threads or requests per second at this stage and report it. Additionally you can look for the root cause of the performance problem and try to fix it.