I need to do performance testing where my scenario is 100 different user logging per day
You don't need any performance testing for 100 users as 100 users per day = 12 users per hour = one user each 5 minutes, any server will handle this "load", even if it is running on a Raspberry Pi.
In general if at some point of time you would like to know the limits of your application, i.e. what is the maximum amount of users it can handle - you will need to go for so called Stress Testing when you start with 1 user and gradually increase the load until response time starts exceeding acceptable thresholds or errors start occurring or website crashes whatever comes the first.
When it comes to using JMeter for web applications performance testing normally steps look like:
Record your test scenario(s) using JMeter's Proxy Server
Perform correlation if required (most probably you will need it as modern web applications widely use dynamic parameters for i.e. user state management or CSRF protection)
Given your test assumes > 1 virtual user you should be using different credentials for every user. Most often JMeter users go for CSV Data Set Config to read the data from external file.
References:
Building a Web Test Plan
Building an Advanced Web Test Plan
There is also an example test script available via File -> Templates -> Building an Advanced Web Test Plan entry of the JMeter main menu which you can use as a "skeleton" for your test
Related
This is for e-commerce project where the number of users login will be more. I have been given a benchmark 8000 concurrent users need to login and response time should be 3 minutes
#abi , hi .
Let me provide couple notes here.
Depending upon Your connection bandwidth , from my experience as performance test engineer, I'd say jmeter single instance usually holds up to 1k(1000)- 2k(2000) in best case users load.
Considering You have a requirement for 8k (8000 users) load, You need to launch jmeter in distributed mode ( master <-> slaves).
For this config setup I'd recommend to go with 1 master node and 4slaves. For that - You will need 5 machines (aws/azure, whatever) in the same sub-network.
Re more technical details on distributed setup, please take a look:
in public jmeter documentation
please also look into this step-by-step setup manual
Also, when i've been doing set-up for 10k load for one of my recent projects - I did couple notes for myself in g-doc . Let me know if it opens fine for You.
Last note, If You need to do some load/performance tests on APIs that require AUTHZ, I'd recommend to split authorize (IDP bypass) and performance scenario itself - in different thread groups. As usually IDP in DEVs/Stagings does not hold much load .
So at first You need to authorize w/o any load (1st Thread group).
And in 2nd Thread group - start calling target APIs under the test.
It depends on:
Your machine specifications (CPU, RAM, NIC card, hard drive, etc.)
The nature of your Test Plan (number of requests, size of requests/responses, number of pre/post processors, assertions, timers, etc.)
Response time of your application
So if your test is a simple GET request which returns small text response - you might simulate 10 000 of users on a mid-range modern laptop. And if your test is connected with heavy requests, large responses, file uploads, etc. - it might be 1000 users.
Make sure to follow recommendations from JMeter Best Practices
Make sure to have monitoring of resources usage of your system (CPU, RAM, Swap, etc.). You can use JMeter PerfMon Plugin for this.
Make sure that your test behaves like a real browser
Start with 1 virtual user and gradually increase the load until you reach 8000 virtual users or JMeter starts lacking resources, whatever comes the first. If you can simulate 8000 users from a single machine - you're good to go. If not - you will have to consider Distributed Testing.
I want to test 400 Concurrency Users Which allow us to pass our load testing scenario as I am using below configuration setting in Apache JMeter which will through us lots of errors.
Number of Thread (Users): 400
Ramp-Up Time: 1
Loop Count: Forever Until ( Period of 1 minutes )
We are not telepathic enough to tell what's wrong with your setup without seeing the configuration and the nature of errors.
Several generic hints:
Run your test with 1-2 users/iterations to ensure it works fine and does what it is supposed to be doing. Check requests and responses details using View Results Tree listener
Make sure to run your test in command-line non-GUI mode and disable all the Listeners while your test is running.
It is better to increase and decrease the load gradually so consider using longer ramp-up time and increase test duration accordingly. I.e.
During the first minute virtual users arrive
They then hold the load for another minute
During the last minute virtual users leave
This way you will be able to tell what was the load when the errors started occurring, what is the maximum number of users your application can support, where is the saturation point, does it recover when the load gets back to normal, etc. See JMeter Ramp-Up - The Ultimate Guide article for more details.
It might be the case you found the bottleneck, i.e. your application fails to support 400 concurrent users, now you need to find the reason which may be in:
incorrect middleware configuration (wrong web server, database, load balancer settings)
your application simply lacks resources (CPU, RAM, Network, Swap, etc.). You can check this using JMeter PerfMon Plugin
if infrastructure configuration is OK and there is enough headroom for the application to operate most probably the reason is in the application code, you need to inspect what it is doing using APM or Profiler tools and report the issue.
I need to perform load test on our web application that involves user interaction to the web pages. So i have written selenium scripts to
handle the click events.For,example we have functionalities like some users has to perform signup/registering to our site,some has to login and perform click actions and logout, and a set of users has to visit our home page, click on multiple URL links available on the home page.
In JMeter,i have added one threadgroup each for above mentioned functionalities.Each threadgroup has "JUnit Request Sampler" which calls the selenium methods that has code for performing click actions.
I have setup the "Thread Properties" for the threadgroups to run 200 threads per minute over a span of 5 to 10 mins[target is for 15 mins]
Default browser i am using is "Firefox".I have also set the JMeter properties such that i wouldn't landup into any memory issues.
I am running the scripts in non-gui mode and collecting the result into a jtl file.
The problem i am facing here is, while scripts are executing multiple browsers openup and as each page involves click events,some of the clicks are not happening correctly resulting in increase in error count. If i use very minimal number of threads with time delay then no errors are seen.
I have tried with distributed mode testing as well but there was no major reduce in error count.
I am looking forward for a solution or rather suggestions that can help me out to have minimal or zero errors while running JMeter scripts that involves user interactions and achieve intended load or thereafter increase the load.
Regards
Praveena
As per WebDriver Tutorial it is not recommended to use WebDriver to conduct the load onto the web application
From experience, the number of browser (threads) that the reader creates should be limited by the following formula:
C = B + 1
where C = Number of Cores of the host running the test
and N = Number of Browser (threads).
Looking into Firefox browser requirements I strongly doubt you have a computer with 200 cores and 400 GBs of RAM therefore my expectation is that you should use JMeter's HTTP Request samplers to create the load.
Browsers don't do any magic, they send HTTP Requests and render the responses, so well-behaved protocol-based JMeter tests will look just like a real browser for the application under test. So I would recommend converting your Selenium tests into "pure" JMeter tests, you can leave 1-2 browsers to measure end-user experience when the application is under the load.
I have requirement of test api with 10k users. What I choose is :
Jmeter
Linux server to install Jmeter and throw load with 10k users
One node for now
Will perform API with operations like :
Login
Book hotel with post parameters
Update Booking Details
Save Booking
I am thinking to use above for API testing with 10k users, Above tools are enough or I should look for other options like loadimpact, loader or blazemeter?
If you are talking about API you should be rather considering "requests per second" rather than "users" as I strongly doubt that end users will be sending requests to API endpoints via curl or Postman.
No matter whether your goal is "users" or "requests per second" it is only you who can answer as it depends on many parameters like:
your machine hardware specifications
software specifications (OS/JVM/JMeter version and architecture)
nature of your test (request/response size, number of pre/post processors, assertions, etc)
So you should act as follows:
Make sure you're following JMeter Best Practices
Make sure you monitor baseline health metrics of the node which is running JMeter (CPU, RAM, Swap, Network, Disk), you can use JMeter PerfMon Plugin for that .
Start with minimal load (1 virtual user or 1 request per second) and increase the load until your machine starts swapping or any other health metric exceeds, say, 80% of maximum available capacity. Once it happens take a look into active threads (Active Threads Over Time listener) or throughput (Transactions Per Second) - this is the maximum number of users or hits per second you can produce on particular this host for particular this test. If it is enough - you're good to go, if not - you will have to switch to Distributed testing
See What’s the Max Number of Users You Can Test on JMeter? article for more details.
The whole answer is an elaborate it depends.
The best thing you can do is to run PerfMon agents on the server generating the load as well as on the server running the system under test.
This way you should see (in the CPU utilization and free memory statistics) whether you had maxed what the server providing the API can do or whether it is your load generator running out of steam. In the first case you got some readline based on the hardware and configuration you had run with. In the second one you have an indication to employ more than 1 box to generate the load or to investigate settings and options.
Have a closer look at PerfMon JMeter plugin for exact details.
Here is the scenario
We are load testing a web application. The application is deployed on two VM servers with a a hardware load balancer distributing the load.
There are tow tools used here
1. HP Load Runner (an expensive tool).
2. JMeter - free
JMeter was used by development team to test for a huge number of users. It also does not have any licensing limit like Load Runner.
How the tests are run ?
A URL is invoked with some parameters and web application reads the parameter , process results and generates a pdf file.
When running the test we found that for a load of 1000 users spread over period of 60 seconds, our application took 4 minutes to generate 1000 files.
Now when we pass the same url through JMeter, 1000 users with a ramp up time of 60 seconds,
application takes 1 minutes and 15 seconds to generate 1000 files.
I am baffled here as to why this huge difference in performance.
Load runner has rstat daemon installed on both servers.
Any clues ?
You really have four possibilities here:
You are measuring two different things. Check your timing record structure.
Your request and response information is different between the two tools. Check with Fiddler or Wireshark.
Your test environment initial conditions are different yielding different results. Test 101 stuff, but quite often overlooked in tracking down issues like this.
You have an overloaded load generator in your loadrunner environment which is causing all virtual users to slow. For example you may be logging everything resulting in your file system becoming a bottleneck for the test. Deliberately underload your generators, reduce your logging levels and watch how you are using memory for correlations so you don't create a physical memory oversubscribed condition which results in high swap activity.
As to the comment above as to JMETER being faster, I have benchmarked both and for very complex code the C based solution for Loadrunner is faster upon execution from iteration to iteration than the Java based solution in JMETER. (method: complex algorithm for creating data files on the fly for upload for batch mortgage processing. p3: 800Mhz. 2GB of RAM. LoadRunner 1.8 million iterations per hour ungoverned for a single user. JMETER, 1.2 million) Once you add in pacing it is the response time of the server which is determinate to both.
It should be noted that LoadRunner tracks its internal API time to directly address accusations of the tool influencing the test results. If you open the results set database set (.mdb or Microsoft SQL server instance as appropriate) and take a look at the [event meter] table you will find a reference for "Wasted Time." The definition for wasted time can be found in the LoadRunner documentation.
Most likely the culprit is in HOW the scripts are structured.
Things to consider:
Think / wait time: When recording,
Jmeter does not automatically put in
waits.
Items being requested: Is
Jmeter ONLY requesting/downloading
HTML pages while Load runner gets all
embedded files?
Invalid Responses:
are all 1000 Jmeter responses valid?
If you have 1000 threads from a
single desktop, I would suspect you
killed Jmeter and not all your
responses were valid.
Dont forget that the testing application itself measures itself, since the arrival of the response is based on the testing machine time. So from this perspective it could be the answer, that JMeter is simply faster.
The second thing to mention is the wait times mentioned by BlackGaff.
Always check results with result tree in jmeter.
And always put the testing application onto separate hardware to see real results, since testing application itself loads the server.