There is difference I see in the Jmeter response time versus manually captured response time.
I record the execution using blazemeter and make necessary token updates to execute the scripts.
Result is as below :
Launch : Jmeter-8000ms : Manual-2000ms
Login : Jmeter-10000ms : Manual-3000ms
Search : Jmeter-8000ms : Manual-1000ms
Logout : Jmeter-4000ms : Manual- 500ms
Why is Jmeter adding extra time when compared to the browser execution?
How can I mimic and get the response time as that of manual execution.
Try cleaning your browsing history, especially browser cache, it might be the case your application relies on "heavy" content like large images or pieces of JavaScript code. JMeter simulates a "clean" session each time and browser retrieves some content from disk or memory cache. Also consider adding HTTP Cache Manager to your Test Plan
If you recorded calls to embedded resources (images, scripts, styles, fonts) - remove these HTTP Request samplers and instead instruct JMeter to download them in parallel using HTTP Request Defaults
If your application is based on AJAX technology these requests are also executed in parallel by real browsers. JMeter cannot parse JavaScript so if you recorded AJAX-driven requests you need to execute them in parallel in order to produce the same network footprint as real browser does. It can be achieved using Parallel Controller which can overcome JMeter's Thread Groups limitations and execute its children at the same time
Related
I recorded a jmeter script, it has all the resources urls in the script. so if I put Parallel Controller with 6 connection and run the script, the script reports higher response time THAN if I put only main html URL and check download embedded resouces with 6 parallel connection. ..so why it reports higher response time with Parallel Controller?
Given both approaches execute the same requests you should be getting the same or at least similar results. If you see differences - capture the traffic from both runs using a 3rd-party tool like Fiddler or Wireshark and compare the network footprint for both cases.
I can only think of one possible reason for difference: cache. If you have HTTP Cache Manager in your test plan then JMeter respects Cache-Control headers and doesn't re-request the content on subsequent calls/iterations.
I have been using Jmeter for performance testing my web application. I have recorded the jmeter script by excluding js,css and other static content files.
While running the script, Jmeter doesnt execute javascript files so ajax XHR request are not sent. To overcome this i have recorded the script with js, css and other static content and it recorded all the Ajax xhr request too. But the performance results seems to be different from the browser loading time. Also i need to use cache during my performance testing.
Below is how my test plan will look like,
Included Retrieve all embedded resources in HTTP Request manager.
Concurrent pool size is 6
I have added HTTP cookie manager and Cache manager.
I have added a loop controller (This is for caching, jmeter will cache the files on first iteration and it will use the cached files
after that)
The problem i am facing is that the time taken for rest call are double the time shown in the browser console for single user. I have tried all other combinations but always i am getting higher time than the browser console.
I have tried to use the Selenium webdriver plugin to simulate the browser behavior but it doesnt seems to be using the cache. (https://www.blazemeter.com/blog/how-load-test-ajaxxhr-enabled-sites-jmeter)
Is there any other way to solve this problem? I want to take the metrics with cache so kindly suggest me any solution that must include cache. Or is there any other tool similar to jmeter that could solve this issue. My goal is to take web page load time with cache for 'n' number of users.
PS : I am even interested to write any scripts in jmeter but the scripts should not overload the performance of jmeter.
Thanks in advance.
You should not be recording calls to embedded resources (images, scripts, styles, fonts, etc.) as if you record them - they will be executed sequentially while real browsers do this in parallel. So remove recorded requests for the embedded resources and "tell" JMeter to download them (and do it in parallel) using HTTP Request Defaults
You should be recording AJAX requests, however real browsers execute them in parallel while JMeter runs them sequentially. In order to make JMeter's behavior closer to real browser you need to put these AJAX calls under the Parallel Controller
You can install the Parallel Controller extension using JMeter Plugins Manager
Jmeter and caching are unrelated - everything that happens after data went over the wire is out of scope by design. You should only ever simulate requests that you expect NOT to be cached. So this is the feasible part: drop all requests that you expect the browser to cache from the Jmeter script (or move them outside the loop). On the load time of XHR: the browser will most certainly use HTTP keepAlive. Result is that all requests except the very first skip the setup and teardown phase of TCP sockets and are much faster - esp. when the request itself is small and quick. You can simulate this in JMeter also by checking the KeepAlive option AND selecting http commons as implementation. You can read up on this in the docs here: http://jmeter.apache.org/usermanual/component_reference.html#HTTP_Request
How can I execute those JMeter recorded script in parallel as we do while creating HTTP SAMPLER (Embedded HTTP Resource) ?
While recording I also checked Retrieve All Embedded Resource with pool of 6. Because of this I getting incorrect response time(Varies from browser timing of page).
Is there any way that we could execute our recorded HTTP Samplers in parallel?
As of JMeter version 3.2 it is not possible to kick off extra threads to run a specific sampler(s) group in parallel, each thread (virtual user) executes samplers upside down.
So you should be very careful with what you are recording. For example, you must not be recording any embedded resources calls, there is `URLs Patterns to Exclude" input on the "Requests Filtering" tab of the HTTP(S) Test Script Recorder where you can define which resources need to be excluded from the recording.
Remember that well-behaved JMeter test should be producing response time similar to real browser, but you need to configure JMeter to mimic real browser, to wit:
Add HTTP Header Manager to represent browser headers
Add HTTP Cookie Manager to represent browser cookies and deal with cookie-based authentication
Add HTTP Cache Manager to act like browser's memory and disk caches
See How to make JMeter behave more like a real browser guide for more information on JMeter fine tuning.
There is Jmeter response time difference between Jmeter run results and manually captured the response time from the local system using stop watch on the web application.
Browse the web app from local windows system and use stop watch to see the response time to load the page.
Run the Jmeter in non-gui/gui mode and observe the response time (used Listeners only to debug, when run the script no listener was added)
Can see there is a difference in both. Please suggest how to know if Jmeter has captured the correct response time.
Given you properly configure JMeter you should get the same or similar response time for the same request. "Proper" configuration stands for:
You should configure JMeter to retrieve embedded resources and use parallel thread pool of ~5 threads for this
This options "tells" JMeter to fetch images, styles and scripts referenced in the main HTML page and do it in parallel like real browsers do
Add HTTP Cache Manager to your Test Plan. Real browsers download embedded resources but do it only once, on subsequent requests the resources are being returned from disk cache, no actual request is being made. HTTP Cache Manager enables cache simulation and cache-control headers handling.
Add HTTP Cookie Manager to represent cookies/sessions and deal with cookie-based authentication
Add HTTP Header Manager to your test plan to represent browser headers. It might be important as i.e. missing Accept-Encoding header may disable server-side compression therefore client will receive much more data and it will take more time.
Assuming "good" JMeter configuration you should see more or less same behavior
If there are still differences - capture the requests sent by JMeter and the browser using a sniffer tool like Wireshark and amend JMeter configuration to eliminate the differences
JMeter have three basic measurements it captures per request:
Elapsed Time (which is overall timespan from the point when it just starts sending request to the last bit received)
Latency (which starts the same point in time and ends when server starts responding)
And Connect time (which is included in latency and is basically the time for handshakes with server, including SSL/TLS negotiations)
So if you set a data writer among your listeners (e.g SimpleDataWriter, although AggregateReport & SummaryReport can do it as well), you'll see these metrics in your data file (while standard listeners/visualisers/aggregators stuck to elapsed time only).
But mind that these metrics doesn't include response rendering, and especially any code to be executed by browser.
JMeter just doesn't do it at all: obviously, it measures just the combined performance of Server + Network, skipping everything on the client side (except for bare necessities, like protocol negotiations).
That might explain the difference you've experienced.
As well as the difference between logged server processing times & the response times measured by JMeter: server just doesn't count what the network brings in.
PS And you don't have to sit and click on stopwatch with your browser: modern ones have a Dev Tools capable of showing you the precision timings divided by stages. E.g., just call Ctrl+Shift+I in Chrome, switch to network tab & behold the timings right there as you doing your requests.
We are using Apache JMeter for the performance testing of web application. Apparently response time is too high in comparison to loading page in browser during load. When we open the page during load it opens in 2 seconds however JMeter reports 70 seconds. I understand browser memory cache and disk cache are used in browser however isn't JMeter cache manager does same. How to assert it, comparing response header is one option. Any thoughts on this will be appreciated.
It may be wrong configuration in the script. There won't be much difference between web browser and Jmeter response times (browser rendering time is ignored in Jmeter, not a big factor but must be considered)
If you are using single Http Sampler for a web page and retrieving all resources in tha page, then select "Parallel Downloads" option to '6' in Http sampler advanced section. So, you are simulating the browser behaviour of parallel downloading of the resources like .js, .css, images etc.
If you recorded the script using Test Script Recorder, then there will be an Http Sampler for each resource requet for that page which will be sent sequentially, hence increase in response time. You might be in this case, as of now, there is no feature/option to send the http samplers in parallel. so I suggest using the approach of adding one sampler, use parallel download of resources option in Http Sampler advanced section.
Also, caching is an important factor which decides the response time. Adding Http Cache manager can solve the issue in jmeter. This simulates browser behaviour of caching. I don't think there will be huge difference b/w browser and jmeter in implementing caching althought may not be perfect.
The reason could be the lack of resources on JMeter machine or JMeter not being properly configured for producing high loads. JMeter default configuration is good for load tests development and debugging, you can run load tests up to certain amount of virtual users, but if you need to conduct a really high loads - you need to perform some configuration changes.
First of all, double check my guess using jvisualvm and your operating system monitoring tools (there is PerfMon JMeter Plugin which can be used for monitoring different metrics on application under test and JMeter load generators sides). If it is the case - take the next steps to get the most performance of your JMeter installation:
Increase JVM Heap Size, NewSize, switch to ConcurrentMarkSweep Garbage Collector.
Run your test in non-GUI mode.
Disable all the listeners during the test.
Follow other recommendations from 9 Easy Solutions for a JMeter Load Test “Out of Memory” Failure article.