So I'm trying to get an environment set up that acts like a selenium grid in that:
1) It runs the tests in parallel across a distributed set of clients
2) I can specify tests to run on specific clients (which are running on different OS's, with different browsers), or tests to run once on all the clients or any combination thereof.
The problem is that capybara actually currently uses WebDriver, not selenium, so as far as I know I can't specify a selenium grid for the tests to hook up to and have it work.
I've looked into things such as DeepTest and Testjour, but neither scratches both itches above. I'd love for there to be a way I can tag a cucumber feature with what browsers I want it run on, and it just does it, like selenium grid currently does. Is there an easy way I can do this without a lot of hackery on my part, or do I need to wait for Selenium 2 to be released?
Oh and I should add I've love to keep capybara's ability to swap out web drivers at will - running on selenium (or WebDriver I guess I should say), htmlunit, etc.
There is currently a version of grid for webdriver under development.
You can find info about it here
So one potential solution I'm digging into here is Hydra. It seems to work for parallel execution of cucumber scenarios across multiple machines simply using ssh, thus scratching itch #1 above. I've set it up and have it running properly on two macs and a windows VM, and things are running smoothly. However, itch #2 remains unscratched. So I forked it, and my first pass is to simply set up a way to run all features on all machines in a parallel manner, ensuring that all features get tested on every supported browser we have. My next pass may be to hack in a way to to be able to specify which features run on which machines, if time permits and the need is great enough. We'll see how it goes.
Related
I'm currently working on writing a suite of test scripts using watir webdriver. Is there something out there that would make script generation easier than looking directly at the HTTP and manually putting the script together? Maybe something captures user interactions with the browser elements and then writes that to a script.
I could just write them manually, but I may as well ask and see if there is a better way.
There are a couple record and playback tools that are available for Selenium (like IDE), and several non-open source solutions as well. Most of the Selenium and Watir development communities actively discourage their usage for writing test suites as they create very brittle tests that are difficult to maintain over time.
Watir does allow you to locate elements based on text or regular expressions, which can make it easier to find many elements without looking at the html. In general, though, you the tester have a better idea of the structure of your website, what id elements are there, and what css elements are unique on a page, or unlikely to change with future site updates, etc.
I have a scenario where I need to do performance test on a web application to check when multiple users login to the app and use, then the UIs are getting rendered in time (fast response). I don't want to use record & replay. Is there a way i can run my existing selenium UI tests from Loadrunner with multiple users ?
Thanks in advance
Amit
It may be possible.
First hurdle is language. Loadrunner supports a few languages. The two I know for which there are also Selenium bindings are Java and C#. If your Selenium scripts are in either one or can be packaged and invoked from JVM or CLR (e.g. Python) it could be possible.
Second hurdle is hardware to support user load. Running the browsers will take a lot of resources. I think you could feasibly run three browsers (Chrome, FF and IE) on a single box. That limits you to three users per agent. If you have hardware to run the number of agents to meet your user load it could be possible. UI rendering times will be meaningless.
The loadrunner script will be a C# or Java project. Add whatever references you need to run your Selenium tests and invoke from there.
However, my gut reaction is this may be rickety. Error handling will add complexity. These alternatives may give a better outcome:
Ditch the idea of using Loadrunner and use a service like Neustar to run multiple Selenium tests. If user load is low, Selenium Grid may also be an option.
Ditch the idea of using Selenium and code in the Loadunner web API.
There is an Eclipse addin for LoadRunner.
You can install this addin, import LoadRunner references and add transaction to your selenium scripts.
Here is a link, that has detailed description about above procedure.
http://www.joecolantonio.com/2014/05/08/how-to-run-a-selenium-script-in-loadrunner/
But yes as earlier said you have to take memory usage into considerations.
No. If you wish to run GUI virtual users in a LoadRunner context then your model is through QuickTest Professional.
The act of rendering (actual drawing of pixels on the screen after an item is downloaded or rasterized) will remain the same for a given speed of video card independent of load on the system as the actual draw to the screen is a local client side activity.
I suspect you mean something different than rendering (drawing on the screen).
Is there a way to detect whether someone is conducting web testing using watir-webdriver on your site? I have read somewhere that it is fairly easy to detect watir/selenium, but I never managed to get more details about it.
I have tried UserAgent detection, but that's not something very useful as far as it's easy to change it.
Right, I will make my comments into an answer as requested.
I doubt if it's possible. The idea of Selenium is to automate browsers by simulating actions like real users. You can't possibly detect it from server side, unless Selenium fails to simulate (e.g. click really fast, but if the Selenium code is written deliberately to simulate a real user in a slow fashion, then I'd say it will be difficult to detect).
On the other hand, User Agent approach won't work if someone runs it using common browsers with default UA.
How to install different versions of IE, without using Virtual PCs.
Is there any simulators ... ??
The answer is to use VMs. Even if that's not the answer you want to hear, that is the answer.
Running (some of) them on the same machine is just about possible, but you shouldn't do it, because they rely on external components, which you also can't have multiple versions of, so yes, you'll end up with (for example) IE6 running alongside other versions, but it will not be the same as the IE6 you started with. In particular, the activeX components are suceptible to this, so if your site uses any filter styles for example, you may get unexpected side effects.
The only reason you'd want multiple versions of IE is for testing purposes, but the above should be enough to prove that even if you do achieve it, it won't give you valid tests.
So we go back to the original point, which is to use a VM.
You can use IETester (http://www.my-debugbar.com/wiki/IETester/HomePage) This works quite nicely for testing multiple versions of IE.
We are considering using Cucumber for testing web applications (not in rails, most of them are asp.net actually).
The applications are in production, our main goal is to test if everything is fine with the services, from time to time, infra people would run it.
We have two questions:
1) Is this a good use for cucumber? Do community people encourage this use of cucumber feature definitions?
2) We have some captchas in our applications. Is there an widely adopted solution for this common problem?
Cucumber looks relatively new. I am a Java person and have used Selenium, HtmlUnit, JWebUnit, etc. Selenium runs in .net, ruby, java, and some other scripting languages.
Selenium has been around for a while (2004 whereas Cucumber is relatively new, 2007).
Selenium has an IDE so you can easily record tests in firefox, save them, and then run them in your integration tests.
I am biased towards Selenium, but it does a good job and allows you to test your applications in several browsers (firefox, safari, IE). It also has support for distributing tests across several servers (if your environment is that large, it supports it).
Ideally, you would have developers or the infrastructure people writing the tests. Then if you have a CI server, you could automatically run the tests you have recorded/written, and then continuously check your application still works as a whole. This works really great for catching errors as soon as they happen so if the developer makes a change and breaks something out of his scope, it will be fresh in his or her head.
As far as CAPTCHA goes, there are various libraries out there. I am unfortunately more knowledgeable with the Java equivalent and not so much with the .Net. Don't write your own, there should be a library you can use.
Walter
ad 1. in my opinion cucumber is great, also we were able to convince our customers to actually understand and verify the tests we wrote in cucumber. We used cucumber + watir for webtesting
ad 2. as far as captcha goes, do you mean how to ignore the captcha for testing? we do not show captcha for our own ip addresses, you could also always accept a specific value for the captcha if the request comes from your development or test environments ip
I can recommend Cucumber. I trained a team of developers and managers to use it on my last project (a PHP application). It worked very well in most cases.
I think your two questions are mutually exclusive. Captcha is designed to prevent something automated so you're going to have to solve that problem for which ever automated test runner you use. You can probably mock something up or work out how to disable it in your test environment. I would opt for the latter. I don't imagine it would be critical to cover your captcha in your test suite.