I have this project in where I want to do load tests on a web page.
I initially learned Selenium but then realized it is more for how a web page works. I also tried Jmeter, but I want to do load tests including the user's behavior, such as clicking, uploading files, retrieving files, and so on.
I tried as well with the plugin for Selenium in Jmeter but it seems to open a navigator for each thread. My test will consist of a maximum of 14000 threads, so I wanted to know if you know some tools or what do you recommend in these cases. Thank you
There are no free solution for this. But you can run your JMeter tests with Selenium Plugin in the cloud (running a 5000 threads with Selenium WebDriver test will cost about $10 for the AWS).
Check this Webdriver Example in Java for more information
Btw, you can upload files via JMeter (without Selenium), you may simulate clicks (if they do requests).
Related
I need some input from loadrunner/jmeter experts.
I'm having automation scripts which is written in selenium+ java .
Can i use same scripts to do performance testing using loadrunner/jemeter.?
why i want do like this : to maintain single code base. if any changes happen so i will able to correct at one place and use it for UI Automation and performance testing.
Any help appreciate.
Selenium automates browsers that it!
Jmeter is specifically to use for Performance test. to be more specific either stress testing or load testing.
JMeter and Selenium can be used together for load testing of a website. Selenium is used to perform load testing and JMeter is used to measure the performance of the same. One can measure how fast HTML pages take to load, and also check javascript and CSS performances. This is done using a combination of the graphical JMeter interface and some scripting to invoke Selenium code.
Please read more about it here
JMeter and Selenium are both used for web application testing. But the main difference is that JMeter is mostly used to test the performance of web applications under load and stress. On the other hand, Selenium is best suited for automation and cross browser testing of a website. Based on what type of testing is necessary for the project, one can choose JMeter or Selenium or integrate both if that serves their purpose.
Theoretically yes, it is possible, but it is not recommended even by Selenium developers, be informed of the following additional constraints:
Browsers are quite resource intensive, for example Firefox 94 needs 1 CPU core and 2 GB of RAM per instance so you can calculate how much RAM you will need to simulate, say, 100 users.
You won't get metrics and KPIs normally available as the result of a performance test run
JMeter does not support the client side performance testing (verifying the content rendering time / page loading time) out of the box.
In single page applications requests are sent to the server as API calls. I am interested in verifying the client side page loading time for concurrent users with JMeter.
What is the best way to test the SPA with JMeter ?
Do you recommend Selenium WebDriver plugin?
Is there any other tools that can be used to verify the client side
load testing for concurrent users?
As per JMeter project main page:
JMeter is not a browser, it works at protocol level. As far as web-services and remote services are concerned, JMeter looks like a browser (or rather, multiple browsers); however JMeter does not perform all the actions supported by browsers. In particular, JMeter does not execute the Javascript found in HTML pages. Nor does it render the HTML pages as a browser does (it's possible to view the response as HTML etc., but the timings are not included in any samples, and only one sample in one thread is ever displayed at a time).
So I doubt whether it makes sense even to consider JMeter for pure frontend operation, most probably you're looking for Performance tab of your browser developer tools or more sophisticated solutions like Lighthouse or WebPageTest
With regards to WebDriver Sampler plugin, it just provides JMeter integration with Selenium browser automation framework so you will be able to launch a real browser per JMeter thread (virtual user) and the rest depends on your implementation, most probably you will want to query window.performance APIs values to get the metrics you're looking for via executeScript() function
I'm trying to build a simulator that simulates hundreds of users on a web application.
I'm usually using Microsoft Load Simulator and WebTests.
If the webpage has some complexity I'm using WebTest plugins to adjust it correctly.
Now, I have a web page that does tons of ajax requests per url I visit, the ajax requests are based on complex calculations done in the browser.
If I'll just browse to the url and record all the traffic, even after I'll make some dynamic parameters I still won't be able to simulate it correctly since there may be different requests that could be sent based on the server previous responses.
If I'll build a webtest that simulates it correctly it will require a lot of webtest plugins and will be impossible to maintaince since the site will probably change each few weeks.
I thought about using selenium but if I'll use it I will need far too much hardware resources to run hundreds of users.
I came up with the idea of using a headless browser such as PhantomJs, SimpleBrowser, HtmlUnit and etc.
Both SimpleBrowser and HtmlUnit does not support executing javascript/AJAX which makes them useless for me.
I tried using PhantomJs but I had a problem with running multiple users in parallel since the localStorage is the same for all so it keeps the same session for all so I can't simulate different users in parallel.
Does anyone had any experience with loadtesting complex ajax web applications?
I will Love you for eternity if you would help me with this issue.
P.S
I'm usually coding in C# but I'm open for new languages\technologies.
Using Selenium for performance testing is not logical. I recommend you to use Locust for real performance testing. For getting and using a dynamic data you can check the this answer. You need to write simple Python script for simulating users.
I am investigating www.loadbooster.com that can import a Selenium script and run headless borwoser with PhantomJS to run the script as a load test.
It is still work in progress for me, so I cant comment on how good it is, but you could investigate it.
I am working on a project where the core application needs a harness to send messages which will process the message and feed to a portal web application. I am writing the automated testing for the portal application using cucumber/ruby/capybara.
I have never used Jmeter but through a colleague i just understand the basics of the Jmeter GUI tool. I have been asked me to use Jmeter as harness tool so that I can later extend the functionality to do performance testing as well. How can I do this ? How can i use Jmeter in my automated scripts using cucumber/ruby/selenium or capybara to do my testing ?
I dont want to use GUI as the whole test case should be automated end to end, I have to figure out a way to integrate Jmeter functionality to do test data set up and then run my browser test to check whether the data exists.
i hope my question is clear, ask me if you are not. Hoping to get a solution as i dont think it is a uncommon question.
Note: i have seen the ruby-jmeter gem, i think it does partly what i need but it is actually a DSL to use jmeter in ruby.the problem with that is I have to stick the syntax of gem and have to figure all the options in gem. I have also read a blog that this can be done using Jmeter recording proxy and running the browser tests at the proxy which I am not sure how to do that. here is the link where this is mentioned:
https://groups.google.com/forum/#!topic/cukes/4ZGkf3a234Y
As per my understanding of the problem, please go through the below links
for recording the script using Jmeter:
http://jmeter.apache.org/usermanual/jmeter_proxy_step_by_step.pdf
for integrating selenium in Jmeter:
There are actually 2 ways of achieving this:
exporting all your selenium scripts as *.jar and copying it to /junit folder under /lib folder of Jmeter.
Use JUnit Sampler to execute your test cases:
http://intensetesting.wordpress.com/2013/09/25/integrating-jmeter-with-selenium-code/
use WebDeriver plugin for Jmeter and write your selenium test cases in Jmeter:
http://jmeter-plugins.org/wiki/WebDriverTutorial/
Also you can run Jmeter via command line.
We are using watir and integrated with VS 2008 using ruby in steel and we have automated our web application and it awsome.
Is there way to use the same script to do the performance testing or is there any better tool.
It's hard to tell if you want something that analyzes the performance of your website (ie: profiler) or a load/stress testing tool. I'm going to assume you want a load testing tool and not a profiler, given that you're talking about script reuse.
All load testing tools, except for one (disclaimer: my company is that one), work by recording HTTP traffic and then replaying it. The script is very different from a functional testing script like one you'd have for Watir.
You can either record the HTTP traffic generated by your Watir script or try to run your functional tests directly.
If you're also using FireWatir, you can use Firebug, which is an excellent web developer tool and shows you the recorded traffic for each page. If you're using IE primarily, check out HttpWatch. It's commercial, but provides great network timings for IE and can export to various data formats. Alternatively, many load testing tools provide a proxy that can record traffic and generate a load script for you.
Once you've got the network data, you can likely quickly turn it in to a script that Pylot, Grinder, JMeter, etc can understand. The problem with this method is that you need to re-record your script whenever any part of the site or the test changes. And if your app is anything more than basic HTML (ie: Ajax, .NET viewstate, etc) then you may have to use some advanced parts of your load testing tool. See my article on ajax load testing for more info.
Shameless plug: if you were using Selenium (or were willing to convert a couple Watir scripts to Selenium scripts), which is another open source functional testing tool, you could use BrowserMob, which provides a load testing service that uses real browsers to play back the load and functional test scripts (Selenium) to drive them. It uses a lot more resources, but thanks to cloud computing the price point is still very low.
There's rawk that you could run over the log files. This gives a pretty comprehensive summary of what's taking so long.
Alternatively there's NewRelic which provides monitoring for your rails app and gives you a detailed breakdown of what every request is doing.
And finally there's FiveRuns which does things very similar to NewRelic.
Have a look at LoadWise, you could reuse existing functional test scripts for performance testing.
With the same load test scripts without change, You can either preview it in Firefox (via FireWatir) or hit your web sites with X number of virtual users (via Celerity).
http://testwisely.com/en/loadwise
dIf you are truely talking just 'performance' then you could alter the scripts to start capturing and recording the page load tines. Any time you so some action that causes a page to load (like navigating to a link) Watir returns the time to load the page.
You just need to have the scripts implement some kind of simple logging method to be able to record the time to load each page, and then alter the scripts so that it return value is captured ala
loadtime = browser.goto(someurl)
perflogger(someurl, loadtime)