I have to generate load for my performance testing using JMeter. I am having .Net code to send data to IOT hub which we want to use.
I can easily create REST endpoints or executable based on requirement. To integrate with JMeter which Sampler I should prefer OS Sampler or HTTP request Sampler to integrate with existing .Net code.
We will be use distributed testing in JMeter so if we go with REST Endpoints then scaling will be required but scaling is not required in case of executable so I think OS Sampler is better over HTTP request Sampler.
Do you see any concern using OS Sampler over HTTP request Sampler?
Please suggest performance comparison also.
Thanks in advance.
If you have a .NET executable which performs an IOT broker load testing - you don't need JMeter.
If you choosing the way to proceed I would recommend going for HTTP Request samplers as with the OS Process sampler you will have only .NET application execution time while with the HTTP Request samplers you will see:
connect time
latency also known as time to first byte
"pure" response time for each and every HTTP request
ability to correlate increasing load with changing other metrics like throughput, server hits per second, response time, etc. using HTML Reporting Dashboard
Related
For performance testing testing Jmeter UI - endpoint - response time deferring from developer tool loading time
Ex Jmeter - endpoint response time is 615 Ms where in developer tool shows (Load: 13.30 s) Finish: 18.98 s
Note : Jmeter capturing 35 backend call/ Developer tool capturing 104 backend call
why jmeter couldn't capture all developer tool backend calls?
why jmeter couldn't capture all developer tool backend calls? do we have any solution for this issue
As per JMeter project main page:
JMeter is not a browser, it works at protocol level. As far as web-services and remote services are concerned, JMeter looks like a browser (or rather, multiple browsers); however JMeter does not perform all the actions supported by browsers. In particular, JMeter does not execute the Javascript found in HTML pages. Nor does it render the HTML pages as a browser does (it's possible to view the response as HTML etc., but the timings are not included in any samples, and only one sample in one thread is ever displayed at a time).
So apart from configuring JMeter to behave like a real browser when it comes to handling embedded resources, cookies, think times and so on you need to pay attention to the nature of the requests which are being called when you open the page in the browser.
If the request is created by JavaScript, i.e. it's an AJAX call JMeter won't make the request automatically when you open the page. However you can record it using JMeter's HTTP(S) Test Script Recorder and run after the main request. The cumulative response time can be measured using Transaction Controller
Check the Retrieve All Embeded Resources in Advanced tab of the http sampler and see what is the difference.
Times will be always slightly different between dev tools and Jmeter or between JMeter and other performance tools like Gatling for example.
I am load testing a scenario in JMeter. I'm trying to achieve a certain amount of r/s and then sustain that for x amount of time. However, with the testing tool, it looks like falling short of r/s for each endpoint. Do I just need more virtual users from the system? or is an additional fix needed for the test plan below?
HTTP Header Manager
User Defined Variables
DNS Cache Manager
HTTP Authorization Manager
HTTP Cache Manager
Thread Group
HTTP Cookie Manager
CSV Data Set Config # 1
HTTP Request #1
HTTP Header Manager
HTTP Request #2
HTTP Header Manager
JSON Extractor
HTTP Request #3
HTTP Header Manager
CSV Data Set Config # 2
HTTP Request #4
JSR2223 PreProcessor
HTTP Header Manager
The CSV Data Set Config # 1 is CSV of username and passwords that are passed to HTTP Requests #1 and #2. The JSON Extractor extracts values from HTTP Request #2 to the HTTP Header Manager of HTTP Request #3.
I can think of 2 situations:
JMeter isn't capable of sending requests fast enough. If this is the case:
Make sure to follow JMeter Best Practices
Try increasing the number of threads/ virtual users
If you cannot conduct the required load from a single load generator - consider switching to Distributed Testing
Your application under test isn't capable of responding fast enough. The reasons could be:
Your application lacks resources (CPU, RAM, Network, Disk, Swap, etc.). Consider monitoring its baseline health metrics using i.e. JMeter PerfMon Plugin
Your application middleware configuration isn't suitable for high loads. Inspect configuration of load balancer, application server, database, whatever to see if they're set up good enough for the anticipated load. Check out Web Performance Optimization tips to get initial idea. Normally the vendors of application servers, databases, etc. provide scaling instructions.
Your application code is not efficient enough, i.e. uses bad algorithms. You can use profiling tools to check what are the slowest and/or frequently called functions, largest objects, etc.
I am looking for a way to performance and stress test my SPA.
I prefer an open source(free) tool to do so.
I have searched quite a lot(https://www.blazemeter.com/blog/how-load-test-ajaxxhr-enabled-sites-jmeter) about the possibility to use Jmeter but i am still not sure if it is possible as this is an SPA and Jmeter works on the protocol level.
JMeter is not a browser as such it will not:
play URLs called through javascript.
report time taken in UI by javascript to display it
But the good news is that you can record all requests at HTTP protocol level,
all requests will be nested inside a Transaction Controller.
JMeter will play them sequentially and not in parallel.
So:
response time will be sum of Main + child requests response time
Load simulation will not be highly impacted as parallelism due to load testing will be simulated
There is also JMeter Webdriver Sampler that allows you to use a Real Browser
So by combining both approach you should be able to:
- Simulate load on your target application
- Have an idea of User Experience response times through JMeter Webdriver Sampler
You can install both plugins using JMeter-Plugins plugin manager plugin.
Note there is a 3rd party plugin that provides parallel execution if you want the parallel execution but I'm not sure it's needed:
https://github.com/Blazemeter/jmeter-bzm-plugins/blob/master/parallel/Parallel.md
How can I execute those JMeter recorded script in parallel as we do while creating HTTP SAMPLER (Embedded HTTP Resource) ?
While recording I also checked Retrieve All Embedded Resource with pool of 6. Because of this I getting incorrect response time(Varies from browser timing of page).
Is there any way that we could execute our recorded HTTP Samplers in parallel?
As of JMeter version 3.2 it is not possible to kick off extra threads to run a specific sampler(s) group in parallel, each thread (virtual user) executes samplers upside down.
So you should be very careful with what you are recording. For example, you must not be recording any embedded resources calls, there is `URLs Patterns to Exclude" input on the "Requests Filtering" tab of the HTTP(S) Test Script Recorder where you can define which resources need to be excluded from the recording.
Remember that well-behaved JMeter test should be producing response time similar to real browser, but you need to configure JMeter to mimic real browser, to wit:
Add HTTP Header Manager to represent browser headers
Add HTTP Cookie Manager to represent browser cookies and deal with cookie-based authentication
Add HTTP Cache Manager to act like browser's memory and disk caches
See How to make JMeter behave more like a real browser guide for more information on JMeter fine tuning.
When I have given 500 concurrent users load via jmeter my server throwing error message but the same time I have called same request via browser showing proper response. How it is possible? Is there any settings in jmeter for avoiding same.
It is hard to say what can go wrong without seeing your JMeter configuration, full server response, JMeter and application under test logs and network dump for browser and JMeter.
The whole idea of performance testing is mimicking real user as close as possible, so you need at least
Add HTTP Request Defaults and set JMeter to:
Download embedded resources
Use concurrent pool of 2-5 threads
Add HTTP Cookie Manager
Add HTTP Cache Manager
Add HTTP Header Manager
Correlate any dynamic parameters
Simulate any specific application behaviour (i.e. AJAX calls)
etc.
In addition to above recommendations: ideally given "good" JMeter you shouldn't see any "response messages", you should see a number of errors in final report so double check you:
Run JMeter in non-GUI mode
Storing only those metrics which are absolutely required
Follow other recommendations from 9 Easy Solutions for a JMeter Load Test “Out of Memory” Failure
beside, what Dmitri described above, I would also check the actual throughput the server returns in either cases.
Throughput depends a lot on the timers you configured in Jmeter to simulate think time.
Jmeter has no rendering and no javascript engine, so each thread is much much faster than a real browser.