I recorded a jmeter script, it has all the resources urls in the script. so if I put Parallel Controller with 6 connection and run the script, the script reports higher response time THAN if I put only main html URL and check download embedded resouces with 6 parallel connection. ..so why it reports higher response time with Parallel Controller?
Given both approaches execute the same requests you should be getting the same or at least similar results. If you see differences - capture the traffic from both runs using a 3rd-party tool like Fiddler or Wireshark and compare the network footprint for both cases.
I can only think of one possible reason for difference: cache. If you have HTTP Cache Manager in your test plan then JMeter respects Cache-Control headers and doesn't re-request the content on subsequent calls/iterations.
Related
I am working on migration of scripts from performance center to Jmeter5.2.1.
As part of this migration , we are using same functional flow which we did in performance center.
My scenario consists of users logging in to the web application perform 10-15 iterations and then logout.
This is my Testplan.
TestPlan
--ThreadGroup1
--Once Only Controller (login of users)
--Loop Controller (10 Iterations)
HTTP1
HTTP2
HTTP3
.
.
--Once only Controller (logout of users)
--csv Config data ( username/password)
--csv config data( unique data for the loop controller)
With this approach I am noticing that the time taken to complete the test in Jmeter is much more than what we have in performance center ( I took care of think times and added the similar values)
Why is my test run slow in Jmeter?
Is loop controller sequential? Meaning at a given time it can run only one request?
If not loop controller what other options we have to satisfy my scenario.
If I include different thread groups , carrying JSESSIONIDs needs to be done across thread groups which is not a best practice to do so.
Update:
Comparison between performance center and Jmeter settings
Below are the settings in Jmeter.
Thread Group settings:
TestPlan :
HTTP Cookie manager in Thread Group
CSV data files in Test plan
Once Only counters for Login and Logout
Loop Controller for Iterations.
HTTP request Defaults: ( Even with out checking retrieve all embedded and parallel downloads its taking more than an hour for 3 users)
TestPlan
Performance Center results :
Every Sampler has HTTP Header manager
Entire Test Plan
Given you send the same requests you should have the same response times, no matter which tool is being used under the hood.
It's hard to say what the differences are without seeing the full scripts from the both tools so generic advice is to use a third-party sniffer tool like Wireshark or Fiddler in order to identify the differences and configure JMeter to behave exactly like the "performance center" (whatever it is)
For example I fail to see HTTP Cache Manager and it will cause JMeter to download embedded resources (images, scripts, styles, sounds, fonts) for each and every HTTP request while real browser does it only once.
I also don't see HTTP Header Manager which might be very important, for example if you send Accept header the server will be aware that the client can understand gzipped responses which will greatly reduce network traffic.
More information: How to make JMeter behave more like a real browser
I have been using Jmeter for performance testing my web application. I have recorded the jmeter script by excluding js,css and other static content files.
While running the script, Jmeter doesnt execute javascript files so ajax XHR request are not sent. To overcome this i have recorded the script with js, css and other static content and it recorded all the Ajax xhr request too. But the performance results seems to be different from the browser loading time. Also i need to use cache during my performance testing.
Below is how my test plan will look like,
Included Retrieve all embedded resources in HTTP Request manager.
Concurrent pool size is 6
I have added HTTP cookie manager and Cache manager.
I have added a loop controller (This is for caching, jmeter will cache the files on first iteration and it will use the cached files
after that)
The problem i am facing is that the time taken for rest call are double the time shown in the browser console for single user. I have tried all other combinations but always i am getting higher time than the browser console.
I have tried to use the Selenium webdriver plugin to simulate the browser behavior but it doesnt seems to be using the cache. (https://www.blazemeter.com/blog/how-load-test-ajaxxhr-enabled-sites-jmeter)
Is there any other way to solve this problem? I want to take the metrics with cache so kindly suggest me any solution that must include cache. Or is there any other tool similar to jmeter that could solve this issue. My goal is to take web page load time with cache for 'n' number of users.
PS : I am even interested to write any scripts in jmeter but the scripts should not overload the performance of jmeter.
Thanks in advance.
You should not be recording calls to embedded resources (images, scripts, styles, fonts, etc.) as if you record them - they will be executed sequentially while real browsers do this in parallel. So remove recorded requests for the embedded resources and "tell" JMeter to download them (and do it in parallel) using HTTP Request Defaults
You should be recording AJAX requests, however real browsers execute them in parallel while JMeter runs them sequentially. In order to make JMeter's behavior closer to real browser you need to put these AJAX calls under the Parallel Controller
You can install the Parallel Controller extension using JMeter Plugins Manager
Jmeter and caching are unrelated - everything that happens after data went over the wire is out of scope by design. You should only ever simulate requests that you expect NOT to be cached. So this is the feasible part: drop all requests that you expect the browser to cache from the Jmeter script (or move them outside the loop). On the load time of XHR: the browser will most certainly use HTTP keepAlive. Result is that all requests except the very first skip the setup and teardown phase of TCP sockets and are much faster - esp. when the request itself is small and quick. You can simulate this in JMeter also by checking the KeepAlive option AND selecting http commons as implementation. You can read up on this in the docs here: http://jmeter.apache.org/usermanual/component_reference.html#HTTP_Request
There is Jmeter response time difference between Jmeter run results and manually captured the response time from the local system using stop watch on the web application.
Browse the web app from local windows system and use stop watch to see the response time to load the page.
Run the Jmeter in non-gui/gui mode and observe the response time (used Listeners only to debug, when run the script no listener was added)
Can see there is a difference in both. Please suggest how to know if Jmeter has captured the correct response time.
Given you properly configure JMeter you should get the same or similar response time for the same request. "Proper" configuration stands for:
You should configure JMeter to retrieve embedded resources and use parallel thread pool of ~5 threads for this
This options "tells" JMeter to fetch images, styles and scripts referenced in the main HTML page and do it in parallel like real browsers do
Add HTTP Cache Manager to your Test Plan. Real browsers download embedded resources but do it only once, on subsequent requests the resources are being returned from disk cache, no actual request is being made. HTTP Cache Manager enables cache simulation and cache-control headers handling.
Add HTTP Cookie Manager to represent cookies/sessions and deal with cookie-based authentication
Add HTTP Header Manager to your test plan to represent browser headers. It might be important as i.e. missing Accept-Encoding header may disable server-side compression therefore client will receive much more data and it will take more time.
Assuming "good" JMeter configuration you should see more or less same behavior
If there are still differences - capture the requests sent by JMeter and the browser using a sniffer tool like Wireshark and amend JMeter configuration to eliminate the differences
JMeter have three basic measurements it captures per request:
Elapsed Time (which is overall timespan from the point when it just starts sending request to the last bit received)
Latency (which starts the same point in time and ends when server starts responding)
And Connect time (which is included in latency and is basically the time for handshakes with server, including SSL/TLS negotiations)
So if you set a data writer among your listeners (e.g SimpleDataWriter, although AggregateReport & SummaryReport can do it as well), you'll see these metrics in your data file (while standard listeners/visualisers/aggregators stuck to elapsed time only).
But mind that these metrics doesn't include response rendering, and especially any code to be executed by browser.
JMeter just doesn't do it at all: obviously, it measures just the combined performance of Server + Network, skipping everything on the client side (except for bare necessities, like protocol negotiations).
That might explain the difference you've experienced.
As well as the difference between logged server processing times & the response times measured by JMeter: server just doesn't count what the network brings in.
PS And you don't have to sit and click on stopwatch with your browser: modern ones have a Dev Tools capable of showing you the precision timings divided by stages. E.g., just call Ctrl+Shift+I in Chrome, switch to network tab & behold the timings right there as you doing your requests.
We are using Apache JMeter for the performance testing of web application. Apparently response time is too high in comparison to loading page in browser during load. When we open the page during load it opens in 2 seconds however JMeter reports 70 seconds. I understand browser memory cache and disk cache are used in browser however isn't JMeter cache manager does same. How to assert it, comparing response header is one option. Any thoughts on this will be appreciated.
It may be wrong configuration in the script. There won't be much difference between web browser and Jmeter response times (browser rendering time is ignored in Jmeter, not a big factor but must be considered)
If you are using single Http Sampler for a web page and retrieving all resources in tha page, then select "Parallel Downloads" option to '6' in Http sampler advanced section. So, you are simulating the browser behaviour of parallel downloading of the resources like .js, .css, images etc.
If you recorded the script using Test Script Recorder, then there will be an Http Sampler for each resource requet for that page which will be sent sequentially, hence increase in response time. You might be in this case, as of now, there is no feature/option to send the http samplers in parallel. so I suggest using the approach of adding one sampler, use parallel download of resources option in Http Sampler advanced section.
Also, caching is an important factor which decides the response time. Adding Http Cache manager can solve the issue in jmeter. This simulates browser behaviour of caching. I don't think there will be huge difference b/w browser and jmeter in implementing caching althought may not be perfect.
The reason could be the lack of resources on JMeter machine or JMeter not being properly configured for producing high loads. JMeter default configuration is good for load tests development and debugging, you can run load tests up to certain amount of virtual users, but if you need to conduct a really high loads - you need to perform some configuration changes.
First of all, double check my guess using jvisualvm and your operating system monitoring tools (there is PerfMon JMeter Plugin which can be used for monitoring different metrics on application under test and JMeter load generators sides). If it is the case - take the next steps to get the most performance of your JMeter installation:
Increase JVM Heap Size, NewSize, switch to ConcurrentMarkSweep Garbage Collector.
Run your test in non-GUI mode.
Disable all the listeners during the test.
Follow other recommendations from 9 Easy Solutions for a JMeter Load Test “Out of Memory” Failure article.
I am trying to setup a JMeter load test, using the Recording Controller.
This results in duplication of all the calls the web browser would make - including css files, js files and images.
Given that a web browser would get these concurrently (performing approximately 10 web requests concurrently). The jmeter documentation tends to equate 1 user to 1 thread - but this does not line up with the way in which modern web browsers work.
Do I need to use 10 threads for each user when setting up my load test?
Strictly Speaking, JMeter can't simulate a browser. JMeter has its own limitations, like not parsing .js files (as per 3.0 version), sending ajax requests from events etc.
But, for your question, there is support in JMeter.
So, you can specify the concurrent pool size as per your requirements, say concurrent pool size value set to 10, which would solve your issue.
Today browser normally sends requests concurrently and each browser has its own max value for maximum connections that can be made concurrently.
Refer following links:
Max parallel http connections in a browser?
and How to solve Chrome's 6 connection limit when using xhr polling.
When you use "Retrieve Embedded Resources" and set Concurrent Pool Size, you ONLY need to add parent sampler, which triggers resource requests like .css, .js files by Jmeter itself. (Eg: add only stackoverflow.com and it will load all the resources like .css, .js automatically). During recording , all these requests are seperately recorded, so you need to remove all of them or create a new plan/thread group containing only parent sampler.
Note: As I mentioned, requests triggered from .js, .csv won't be sent as Jmeter won't parse them like Browser. You need to add ONLY these requests as samplers explicitly. Add View Results Tree and compare the requests triggered by Jmeter and Browser (F12 -> Network tab) to know which request are missing by Jmeter.
First of all, don't record requests to embedded resources. If you click Add Suggested Excludes button the HTTP(S) Test Script Recorder will automatically populate a regular expression to filter images, scripts and styles.
Second. To simulate browser behaviour: 1 main request to the page and several parallel requests to retrieve content you can use "Advanced" tab of the HTTP Request Sampler (or even better HTTP Request Defaults). You can also limit embedded resources to your application under test domain there
Real browsers download images, styles and scripts. However well-behaved browsers do it only once, on subsequent requests these entities are being returned from cache. So make sure you add HTTP Cache Manager to your Test Plan to represent browser cache so you could avoid overwhelming your server with extra requests which don't happen in the reality.
See How To Make JMeter Behave More Like A Real Browser for more detailed explanation and instructions on using aforementioned Test Elements