I have a scenario where I need to do performance test on a web application to check when multiple users login to the app and use, then the UIs are getting rendered in time (fast response). I don't want to use record & replay. Is there a way i can run my existing selenium UI tests from Loadrunner with multiple users ?
Thanks in advance
Amit
It may be possible.
First hurdle is language. Loadrunner supports a few languages. The two I know for which there are also Selenium bindings are Java and C#. If your Selenium scripts are in either one or can be packaged and invoked from JVM or CLR (e.g. Python) it could be possible.
Second hurdle is hardware to support user load. Running the browsers will take a lot of resources. I think you could feasibly run three browsers (Chrome, FF and IE) on a single box. That limits you to three users per agent. If you have hardware to run the number of agents to meet your user load it could be possible. UI rendering times will be meaningless.
The loadrunner script will be a C# or Java project. Add whatever references you need to run your Selenium tests and invoke from there.
However, my gut reaction is this may be rickety. Error handling will add complexity. These alternatives may give a better outcome:
Ditch the idea of using Loadrunner and use a service like Neustar to run multiple Selenium tests. If user load is low, Selenium Grid may also be an option.
Ditch the idea of using Selenium and code in the Loadunner web API.
There is an Eclipse addin for LoadRunner.
You can install this addin, import LoadRunner references and add transaction to your selenium scripts.
Here is a link, that has detailed description about above procedure.
http://www.joecolantonio.com/2014/05/08/how-to-run-a-selenium-script-in-loadrunner/
But yes as earlier said you have to take memory usage into considerations.
No. If you wish to run GUI virtual users in a LoadRunner context then your model is through QuickTest Professional.
The act of rendering (actual drawing of pixels on the screen after an item is downloaded or rasterized) will remain the same for a given speed of video card independent of load on the system as the actual draw to the screen is a local client side activity.
I suspect you mean something different than rendering (drawing on the screen).
Related
I've got a super tedious task to do for work that I'm hoping to get some guidance on semi-automating.
We've got a listbox in one program with a few hundred entries. I need to compare that against another listbox in a web app and figure out which values are missing or extra. I don't have access to the code from either.
Any advice on the best way to accomplish that would be appreciated.
For this sort of problem, you could try using a single Python script that fetches and compares your data.
For the web app portion, you can use Selenium webdriver - there are a lot of other ways to retrieve your web content (another Python web client library, or the curl command-line program), but webdriver should give you access to the contents of the listbox quite easily.
And for the Windows GUI app portion, you can try using pywinauto, swapy, or go directly to the MS UI Automation framework.
You could of course approach this with a different language, but I feel Python gives you the best of both worlds (web and native Windows app automation) without too much fuss. It can be nice being able to perform both types of tasks from one script.
For about a year a half, I've been working with SilkTest, which is a GUI automation tool, for both desktop and web applications. It simulates mouse and keyboard inputs, which eventually simulate end user behaviour. However, I find that it is a bit flaky; Button.Click() or DialogBox.Close() method calls that work just fine 9 times in a row seem to fail on a 10th call, only to go back working on the 11th. Normally I would just chalk this up to a quirk with SilkTest (or the application under Test, or the OS, or what have you) but then I see that there are similar issues with other GUI automation tools like Selenium:
Selenium Click() fails with Anchor Elements
Selenium Click() fails clicking button object
I know that for desktop apps, each GUI control/dialog has a tag element associated with it (at least in Windows-based GUIs) and that for web pages there is the domain object model hierarchy of page elements. My guess is that these tools sometimes run into issues navigating these hierarchies and finding unique elements and controls. But what is going on here? SilkTest is a relatively old, commercial software package while selenium is relatively new, open source and constantly evolving. The fact that they both can have similar problems raises a couple of flags with me.
Also, is this the case with other GUI test tools? Or have I just had a somewhat unusual experience?
There are 2 things here that you are talking about, first the concept of finding an object in the application under test that you want to automate. Your description of how SilkTest (and other tools) does this is quite accurate, i.e. as long as there is something that the automation software can use to identify the control then you are fine.
The second thing is why does the automation itself fails randomly, since the tool has not reported that it could not find the control then it must think that it sent the appropriate action to the application, e.g. a Click or a Type. This could be that the application is not ready to accept the action that you are sending it, this is similar to you attempting to click on something "before it was ready", in this case the application can decide to buffer the input or to discard the input.
So, how do you fix this? One way would be to use the capabilities of the tool to try to work out when the application is ready for input rather than sending it a stream of input blindly. SilkTest has capabilities that allow for you to do this (as does TestPartner). I cannot comment on Selenium as it is something I have not used.
A simple way of testing this would be to insert a pause for a couple of seconds before the offending action, then run this in a loop to see whether this solves the problem, if this is the case then it is your problem. If this does not fix the issue then there is something else going on that you need to contact the vendor of the testing tool.
Remember that applications are getting more and more complex, i.e. multi-threading, communications, any one of these could cause the automatic syncronisation to fail causing actions to fail.
Hope that helps.
So I'm trying to get an environment set up that acts like a selenium grid in that:
1) It runs the tests in parallel across a distributed set of clients
2) I can specify tests to run on specific clients (which are running on different OS's, with different browsers), or tests to run once on all the clients or any combination thereof.
The problem is that capybara actually currently uses WebDriver, not selenium, so as far as I know I can't specify a selenium grid for the tests to hook up to and have it work.
I've looked into things such as DeepTest and Testjour, but neither scratches both itches above. I'd love for there to be a way I can tag a cucumber feature with what browsers I want it run on, and it just does it, like selenium grid currently does. Is there an easy way I can do this without a lot of hackery on my part, or do I need to wait for Selenium 2 to be released?
Oh and I should add I've love to keep capybara's ability to swap out web drivers at will - running on selenium (or WebDriver I guess I should say), htmlunit, etc.
There is currently a version of grid for webdriver under development.
You can find info about it here
So one potential solution I'm digging into here is Hydra. It seems to work for parallel execution of cucumber scenarios across multiple machines simply using ssh, thus scratching itch #1 above. I've set it up and have it running properly on two macs and a windows VM, and things are running smoothly. However, itch #2 remains unscratched. So I forked it, and my first pass is to simply set up a way to run all features on all machines in a parallel manner, ensuring that all features get tested on every supported browser we have. My next pass may be to hack in a way to to be able to specify which features run on which machines, if time permits and the need is great enough. We'll see how it goes.
I would like to know how to go about doing regression testing effectively for a web application.
I m spending a lot of time in regression testing for every build.
I also hear that automated GUI testing is bad and not worth it.
Is there any better way of automating regression testing. if so any good tools to automate GUI tests.
My web app is designed for stupid IE. cant help it, but that how it is.Any good tool that can help me?
I know that we must keep GUI layer thin and all, but I m not a GUI expert or anything , but i need to test it properly and effectively.
Thanks all
Try looking into Cucumber or Selenium
There is also a screencast on it over at railscasts.
That should really help.
You can use standard unit tests and integration tests for the non-GUI components, of course. For the GUI, it's worth looking at UI automation packages.
Many of them are pretty terrible, but a lot of the horror stories come from people using them incorrectly. If you can stay away from specifying click locations by (x,y) coordinates, and you can detect when state transitions happen, rather than using delay(), GUI testing can be effectively automated.
As long as your webapp works with Firefox, too, I recommend using Selenium for that scenario. It allows you to record your test cases inside the browser (by means of a Firefox plugin) but has Internet Explorer support for playback.
If Firefox for recording tests is not an option, you could have a look at the WebDriver project. It uses a code based approach to test specification an works well with TDD, too.
You might also want to have a look at CubicTest, which is a Eclipse plugin for graphically defining Selenium and Watir test cases.
I've been looking at ways people test their apps in order decide where to do caching or apply some extra engineering effort, and so far httperf and a simple sesslog have been quite helpful.
What tools and tricks did you apply on your projects?
I use httperf for a high level view of performance.
Rails has a performance script built in, that uses the ruby-prof gem to analyse calls deep within the Rails stack. There is an awesome Railscast on Request Profiling using this technique.
NewRelic have some seriously cool analysis tools that give near real-time data.
They just made it a "Lite" version available for free.
I use jmeter for session-based testing - it allows very fine-grained control over pages you want to hit, parameters to inject, loops to go through, etc. It's great for simulating how many real users your site can handle, rather than just performance testing a set of static urls. You can distribute tests over multiple machines quite easily by loading up the jmeter-server on computers with publicly accessible IP's. I have found some limitations in the number of users/threads any one machine can throw at a server at once (it depends on the test), but jmeter has helped my team improve our apps capacity for users to 6x.
It doesn't have any fancy graphing -- I actually use my own in-house graphing with gruff that can do performance analysis on request time for certain pages and actions.
I'm evaluating a new opensource web page instrumentation and measurement suite called Jiffy. It's not particularly for ruby, it works for all kind of webapps
There's also a Jiffy Firebug Extension for rendering the metrics inside the browser.
I also suggest you look at Browser Mob for load testing.
A colleague of mine has also posted some interesting thoughts on this.