I am looking to run the one test on multiple URLs using the selenium ide plugin for Firefox. My environment is load balanced, so I have the same website working on a number of servers. I will be testing internally, so I can access each server via their internal IP address (e.g. 192.168.1.1, 192.168.1.2, etc.). the purpose of the test I do is to check that servers are synchronized, by confirming that UI elements are there.
Is there a selenium command that will allow me to open a URL (e.g. 192.168.1.1), run a set of UI checks, then open the next URL (e.g. 192.168.1.2), and run the same UI checks again?
I currently change the base URL before every test to achieve this, but if I could automate this entirely, it will save me a lot of time (I have lots of different servers to hit).
Not with Selenium IDE. But you can generate code for various languages, the Selenium API in those languages do allow opening multiple, independent browser instances.
Related
I have this project in where I want to do load tests on a web page.
I initially learned Selenium but then realized it is more for how a web page works. I also tried Jmeter, but I want to do load tests including the user's behavior, such as clicking, uploading files, retrieving files, and so on.
I tried as well with the plugin for Selenium in Jmeter but it seems to open a navigator for each thread. My test will consist of a maximum of 14000 threads, so I wanted to know if you know some tools or what do you recommend in these cases. Thank you
There are no free solution for this. But you can run your JMeter tests with Selenium Plugin in the cloud (running a 5000 threads with Selenium WebDriver test will cost about $10 for the AWS).
Check this Webdriver Example in Java for more information
Btw, you can upload files via JMeter (without Selenium), you may simulate clicks (if they do requests).
Most of the tools out there do not seem to be interpreting jQuery / Javascript code that is loaded on the page that I want to test. This is however important because that code will, in turn initiate a number of calls to other endpoints in my webapp which adds up to load in a real life scenario.
I've looked at JMeter already and am a bit reluctant to give it a try as the landing page mentions explicitly that it does not do Javascript interpreting at all.
What would be some recommendation of tools that can provide a more accurate measurement of load testing by including Javascript code?
None of the load testing tools really executes JavaScript, they all act on protocol level and JavaScript is being executed by browser.
There are 3 options how you could proceed if you want 100% realistic testing:
HP LoadRunner has TrueClient protocol which is basically headless browser with JavaScript capabilities.
JMeter with WebDriver Sampler plugin - the way to kick off real browsers from JMeter test
Selenium Grid (or other way to kick off several Selenium instances at once)
All 3 above options are very resource intensive, you will required at least 1 CPU core and a couple of gigabytes or RAM for a single browser instance so I would recommend a little bit different approach. JMeter cannot execute JavaScript, but it can execute JavaScript-driven HTTP Requests so create main load using JMeter (or equivalent) and while the load test is running use Selenium to automatically check real browsing experience or YSLow to do the same manually.
Load testing tools doesn't execute JavaScript.
You know which endpoints your JavaScript code is using so just add these endpoints to your JMeter scenario.
The thick client problem has been around since the early 1990s with traditional thick client-server applications. In fact, the earliest commercial tools were all driving full clients on X Windows before adding API level virtual user capabilities. On the commercial front this ability to drive full, thick clients is still expressed as GUI virtual users (Mercury/HP/Microfocus LoadRunner, Segue/Borland/Microfocus Silk Performer, Rational/IBM Performance Tester) allowing you to measure the weight between API and user level if needed.
Here is the thing, for a given business process and data set the end client is predictable in its behavior, and in the requests that it makes to the servers that are feeding it data. There may be a few odd conditions where you need to reproduce an algorithm from JavaScript to C (or even run it directly in some tools) for branching on a type of API/HTTP request, but these are not numerous.
I'm trying to build a simulator that simulates hundreds of users on a web application.
I'm usually using Microsoft Load Simulator and WebTests.
If the webpage has some complexity I'm using WebTest plugins to adjust it correctly.
Now, I have a web page that does tons of ajax requests per url I visit, the ajax requests are based on complex calculations done in the browser.
If I'll just browse to the url and record all the traffic, even after I'll make some dynamic parameters I still won't be able to simulate it correctly since there may be different requests that could be sent based on the server previous responses.
If I'll build a webtest that simulates it correctly it will require a lot of webtest plugins and will be impossible to maintaince since the site will probably change each few weeks.
I thought about using selenium but if I'll use it I will need far too much hardware resources to run hundreds of users.
I came up with the idea of using a headless browser such as PhantomJs, SimpleBrowser, HtmlUnit and etc.
Both SimpleBrowser and HtmlUnit does not support executing javascript/AJAX which makes them useless for me.
I tried using PhantomJs but I had a problem with running multiple users in parallel since the localStorage is the same for all so it keeps the same session for all so I can't simulate different users in parallel.
Does anyone had any experience with loadtesting complex ajax web applications?
I will Love you for eternity if you would help me with this issue.
P.S
I'm usually coding in C# but I'm open for new languages\technologies.
Using Selenium for performance testing is not logical. I recommend you to use Locust for real performance testing. For getting and using a dynamic data you can check the this answer. You need to write simple Python script for simulating users.
I am investigating www.loadbooster.com that can import a Selenium script and run headless borwoser with PhantomJS to run the script as a load test.
It is still work in progress for me, so I cant comment on how good it is, but you could investigate it.
I need to deploy an application onto some Windows machines for purposes of data collection from a group of people (i.e. the application will be used to gather responses to a series of survey questions). The process is interactive, alternating between displays of text and images with specific timing requirements. I have put together a prototype application using HTML and JavaScript that implements the survey. However, there are some unique constraints on the deployment environment that have me stuck:
While the machine is Internet-connected, the client requires that the survey application must run fully local to the PC that it runs on. Therefore, sending the survey results to a remote server is not permissible. Obviously, saving to a local file from a Web browser is typically not permitted for security reasons.
Installation of applications onto the machines that will run the survey is not permitted.
The configuration of the machines is not known specifically a priori, but I can assume some recent version of Windows with IE8+.
The "no remote access" requirement was a late comer, and has thrown a wrench into the plan of just writing a simple Web application that could post results to an HTTP server. I'm now looking for the easiest way forward. Two main approaches come to mind:
Use a GUI framework that provides a control that can display HTML/JavaScript; running a full-blown application on the PC would allow me to save the results to the filesystem. I've never done this, but it seems like in this day and age it shouldn't be too difficult. This would allow me to reuse much of my existing prototype implementation, but I would need some way of transferring the results (which would be stored in a JavaScript data structure) outside of the Web control to where the rest of the application could access it.
Reimplement the entire application using some GUI framework (I've used PyQt successfully before, although not on Windows). This approach is obviously less desirable than #1 due to the lack of reuse. However, it may be necessary if #1 isn't feasible.
Any recommendations for the best way to go? Ideally, I'm looking for a solution that can be run in a "portable" manner from a USB thumbdrive or similar.
Have you looked at HTML Applications (HTA)? They work in IE5+ and can use Windows Scripting Host to write to local drives and UNC shares...
Maybe you can use a portable web server with a scripting language on the server side. http://code.google.com/p/mongoose/ Mongoose, for example, you can run PHP, CGI, etc. .. scripts. Then, simply create a script to save a file to your hard drive. And let the rest of the application in the same manner.
Use a script to start the web server, and perhaps a portable web browser like K-Meleon to start the application http://kmeleon.sourceforge.net/ This is highly configurable. Or start the system explorer to your localhost URL.
The only problem may be that the user has to modify the firewall for the first time you run the server?
We are using watir and integrated with VS 2008 using ruby in steel and we have automated our web application and it awsome.
Is there way to use the same script to do the performance testing or is there any better tool.
It's hard to tell if you want something that analyzes the performance of your website (ie: profiler) or a load/stress testing tool. I'm going to assume you want a load testing tool and not a profiler, given that you're talking about script reuse.
All load testing tools, except for one (disclaimer: my company is that one), work by recording HTTP traffic and then replaying it. The script is very different from a functional testing script like one you'd have for Watir.
You can either record the HTTP traffic generated by your Watir script or try to run your functional tests directly.
If you're also using FireWatir, you can use Firebug, which is an excellent web developer tool and shows you the recorded traffic for each page. If you're using IE primarily, check out HttpWatch. It's commercial, but provides great network timings for IE and can export to various data formats. Alternatively, many load testing tools provide a proxy that can record traffic and generate a load script for you.
Once you've got the network data, you can likely quickly turn it in to a script that Pylot, Grinder, JMeter, etc can understand. The problem with this method is that you need to re-record your script whenever any part of the site or the test changes. And if your app is anything more than basic HTML (ie: Ajax, .NET viewstate, etc) then you may have to use some advanced parts of your load testing tool. See my article on ajax load testing for more info.
Shameless plug: if you were using Selenium (or were willing to convert a couple Watir scripts to Selenium scripts), which is another open source functional testing tool, you could use BrowserMob, which provides a load testing service that uses real browsers to play back the load and functional test scripts (Selenium) to drive them. It uses a lot more resources, but thanks to cloud computing the price point is still very low.
There's rawk that you could run over the log files. This gives a pretty comprehensive summary of what's taking so long.
Alternatively there's NewRelic which provides monitoring for your rails app and gives you a detailed breakdown of what every request is doing.
And finally there's FiveRuns which does things very similar to NewRelic.
Have a look at LoadWise, you could reuse existing functional test scripts for performance testing.
With the same load test scripts without change, You can either preview it in Firefox (via FireWatir) or hit your web sites with X number of virtual users (via Celerity).
http://testwisely.com/en/loadwise
dIf you are truely talking just 'performance' then you could alter the scripts to start capturing and recording the page load tines. Any time you so some action that causes a page to load (like navigating to a link) Watir returns the time to load the page.
You just need to have the scripts implement some kind of simple logging method to be able to record the time to load each page, and then alter the scripts so that it return value is captured ala
loadtime = browser.goto(someurl)
perflogger(someurl, loadtime)