I am trying to setup a JMeter load test, using the Recording Controller.
This results in duplication of all the calls the web browser would make - including css files, js files and images.
Given that a web browser would get these concurrently (performing approximately 10 web requests concurrently). The jmeter documentation tends to equate 1 user to 1 thread - but this does not line up with the way in which modern web browsers work.
Do I need to use 10 threads for each user when setting up my load test?
Strictly Speaking, JMeter can't simulate a browser. JMeter has its own limitations, like not parsing .js files (as per 3.0 version), sending ajax requests from events etc.
But, for your question, there is support in JMeter.
So, you can specify the concurrent pool size as per your requirements, say concurrent pool size value set to 10, which would solve your issue.
Today browser normally sends requests concurrently and each browser has its own max value for maximum connections that can be made concurrently.
Refer following links:
Max parallel http connections in a browser?
and How to solve Chrome's 6 connection limit when using xhr polling.
When you use "Retrieve Embedded Resources" and set Concurrent Pool Size, you ONLY need to add parent sampler, which triggers resource requests like .css, .js files by Jmeter itself. (Eg: add only stackoverflow.com and it will load all the resources like .css, .js automatically). During recording , all these requests are seperately recorded, so you need to remove all of them or create a new plan/thread group containing only parent sampler.
Note: As I mentioned, requests triggered from .js, .csv won't be sent as Jmeter won't parse them like Browser. You need to add ONLY these requests as samplers explicitly. Add View Results Tree and compare the requests triggered by Jmeter and Browser (F12 -> Network tab) to know which request are missing by Jmeter.
First of all, don't record requests to embedded resources. If you click Add Suggested Excludes button the HTTP(S) Test Script Recorder will automatically populate a regular expression to filter images, scripts and styles.
Second. To simulate browser behaviour: 1 main request to the page and several parallel requests to retrieve content you can use "Advanced" tab of the HTTP Request Sampler (or even better HTTP Request Defaults). You can also limit embedded resources to your application under test domain there
Real browsers download images, styles and scripts. However well-behaved browsers do it only once, on subsequent requests these entities are being returned from cache. So make sure you add HTTP Cache Manager to your Test Plan to represent browser cache so you could avoid overwhelming your server with extra requests which don't happen in the reality.
See How To Make JMeter Behave More Like A Real Browser for more detailed explanation and instructions on using aforementioned Test Elements
Related
I am working on migration of scripts from performance center to Jmeter5.2.1.
As part of this migration , we are using same functional flow which we did in performance center.
My scenario consists of users logging in to the web application perform 10-15 iterations and then logout.
This is my Testplan.
TestPlan
--ThreadGroup1
--Once Only Controller (login of users)
--Loop Controller (10 Iterations)
HTTP1
HTTP2
HTTP3
.
.
--Once only Controller (logout of users)
--csv Config data ( username/password)
--csv config data( unique data for the loop controller)
With this approach I am noticing that the time taken to complete the test in Jmeter is much more than what we have in performance center ( I took care of think times and added the similar values)
Why is my test run slow in Jmeter?
Is loop controller sequential? Meaning at a given time it can run only one request?
If not loop controller what other options we have to satisfy my scenario.
If I include different thread groups , carrying JSESSIONIDs needs to be done across thread groups which is not a best practice to do so.
Update:
Comparison between performance center and Jmeter settings
Below are the settings in Jmeter.
Thread Group settings:
TestPlan :
HTTP Cookie manager in Thread Group
CSV data files in Test plan
Once Only counters for Login and Logout
Loop Controller for Iterations.
HTTP request Defaults: ( Even with out checking retrieve all embedded and parallel downloads its taking more than an hour for 3 users)
TestPlan
Performance Center results :
Every Sampler has HTTP Header manager
Entire Test Plan
Given you send the same requests you should have the same response times, no matter which tool is being used under the hood.
It's hard to say what the differences are without seeing the full scripts from the both tools so generic advice is to use a third-party sniffer tool like Wireshark or Fiddler in order to identify the differences and configure JMeter to behave exactly like the "performance center" (whatever it is)
For example I fail to see HTTP Cache Manager and it will cause JMeter to download embedded resources (images, scripts, styles, sounds, fonts) for each and every HTTP request while real browser does it only once.
I also don't see HTTP Header Manager which might be very important, for example if you send Accept header the server will be aware that the client can understand gzipped responses which will greatly reduce network traffic.
More information: How to make JMeter behave more like a real browser
During performance testing "Retrieve All Embedded Resources" should be checked or unchecked?
My analysis: "Retrieve All Embedded Resources" should not be checked. And cache also should not be cleared automatically.
Note:
a. "Retrieve All Embedded Resources" option is not checked in Jmeter default recording template.
Which is the correct response time, if 500 user are iterating 2 scenarios 50 times.
My analysis: Need to refer "Transaction Controller - Sample Time [sum value]" in "View Results in Table" listener. Note, need to user parallel controller in order to handle parallel requests and need to refer max response time in that.
If you're testing a web application you need to configure JMeter to behave like a real browser. Real browsers download these "embedded resources" like images, scripts, styles, fonts, sounds, etc. and doing this in parallel (one main request followed by 6 threads to download the embedded resources). So you should:
Configure JMeter to download embedded resources
Do it using parallel downloads with i.e. 6 threads
Limit the resources location to the domain(s) you're testing to exclude 3rd-party resources like CDNs for scripts, banners from external sites, etc)
Cache: if you want to simulate "new" user each iteration you should clear the cache, if you want to simulate "returning" user - he should already have the resources in his cache and shouldn't download them.
More information: Web Testing with JMeter: How To Properly Handle Embedded Resources in HTML Responses
We don't know what you're measuring, the response time for a single request or for the whole business transaction. Personally I would go for the single request (but including all the embedded resources and eventually AJAX requests simulated by the parallel controller if any) and look not into the average response time but rather at 90% percentile as average can mask the problem while 90% percentile is the response time which 90% of the users will get.
We need to test our web application with 100+ users with JMeter tool.
Should we use JMeter webdriver plugin to launch the real browsers or can we use the blazemeter plugin to generate the script and run in JMeter? Will the script created with blazemeter simulate the real browser load?
Theoretically you can use real browsers, but be aware that browsers are very resource intensive so my expectation is that you will need at least one CPU core and at 1 GB of RAM per browser instance in order to have enough resources.
This means you will have to go for distributed testing and on average you will have one slave per 5 browsers. If this is something you can afford - go for it. If don't - be aware that you can configure JMeter to behave like a real browser, to wit:
Tell JMeter to Retrieve All Embedded Resources from HTML Files and Use concurrent pool of 6 threads. This can be done using HTTP Request Defaults, the settings live under "Advanced" tab
Add HTTP Cache Manager to simulate browser cache
Add HTTP Cookie Manager to mimic browser cookies
Add HTTP Header Manager to send browser-like headers (i.e. User-Agent, Accept-Encoding, etc.)
If your application uses AJAX requests - put the relevant HTTP Request samplers under the Parallel Controller
I have been using Jmeter for performance testing my web application. I have recorded the jmeter script by excluding js,css and other static content files.
While running the script, Jmeter doesnt execute javascript files so ajax XHR request are not sent. To overcome this i have recorded the script with js, css and other static content and it recorded all the Ajax xhr request too. But the performance results seems to be different from the browser loading time. Also i need to use cache during my performance testing.
Below is how my test plan will look like,
Included Retrieve all embedded resources in HTTP Request manager.
Concurrent pool size is 6
I have added HTTP cookie manager and Cache manager.
I have added a loop controller (This is for caching, jmeter will cache the files on first iteration and it will use the cached files
after that)
The problem i am facing is that the time taken for rest call are double the time shown in the browser console for single user. I have tried all other combinations but always i am getting higher time than the browser console.
I have tried to use the Selenium webdriver plugin to simulate the browser behavior but it doesnt seems to be using the cache. (https://www.blazemeter.com/blog/how-load-test-ajaxxhr-enabled-sites-jmeter)
Is there any other way to solve this problem? I want to take the metrics with cache so kindly suggest me any solution that must include cache. Or is there any other tool similar to jmeter that could solve this issue. My goal is to take web page load time with cache for 'n' number of users.
PS : I am even interested to write any scripts in jmeter but the scripts should not overload the performance of jmeter.
Thanks in advance.
You should not be recording calls to embedded resources (images, scripts, styles, fonts, etc.) as if you record them - they will be executed sequentially while real browsers do this in parallel. So remove recorded requests for the embedded resources and "tell" JMeter to download them (and do it in parallel) using HTTP Request Defaults
You should be recording AJAX requests, however real browsers execute them in parallel while JMeter runs them sequentially. In order to make JMeter's behavior closer to real browser you need to put these AJAX calls under the Parallel Controller
You can install the Parallel Controller extension using JMeter Plugins Manager
Jmeter and caching are unrelated - everything that happens after data went over the wire is out of scope by design. You should only ever simulate requests that you expect NOT to be cached. So this is the feasible part: drop all requests that you expect the browser to cache from the Jmeter script (or move them outside the loop). On the load time of XHR: the browser will most certainly use HTTP keepAlive. Result is that all requests except the very first skip the setup and teardown phase of TCP sockets and are much faster - esp. when the request itself is small and quick. You can simulate this in JMeter also by checking the KeepAlive option AND selecting http commons as implementation. You can read up on this in the docs here: http://jmeter.apache.org/usermanual/component_reference.html#HTTP_Request
By default JMeter sends all the requests sequentially. Is there any
methodology to send the requests concurrently at the same time for a single user i.e.
something similar to web_concurrent_start () and web_concurrent_end ()
functionality in Load Runner. Any thoughts / ideas in this regard?
JMeter's equivalent of the web_concurrent_start () would be Synchronizing Timer, however depending on what you're trying to achieve you may need to use a different approach.
For simulating browser's behaviour with regards to images, scripts and styles it would be using "Retrieve All Embedded Resources" and "Parallel downloads" in the HTTP Request sampler "Advanced" tab
For simulating AJAX requests you may need to do some scripting using JSR223 Test Elements and classes from i.e. java.util.concurrent namespace as JMeter Thread Groups are not designed to kick off more threads than defined.
It sounds like you need a custom sampler to fire asynchronous requests. I added some info here regarding one I am using:
Performance Testing of AJAX calls via JMeter
If your requirement is simulating browser behaviour of sending requests concurrently to load images, .css and .js files, You should have a look at the following answer:
Does a Jmeter thread really approximate a user?
You can send multiple requests in parallel by only specifying the parent URL (let's say stackoverflow.com) and enable the checkbox "Retrieve All Embedded Resources" to send requests in parallel and define concurrent pool size to specify the number of parallel requests to be sent (usually 6).
Note: JMeter parses the HTML response and triggers the requests for the resources specified in HTML response. In JMeter 3.0, it can parses .css files also, but can't parse .js files (a limitation). so, you have to take care of those requests manually (by adding HTTP samplers for those requests)