I am using Selenium for automation with Eclipse and TestNG.
Whenever required I execute script from testng.xml.
Why do I need a Build tool loke Maven or Ant because Compiling and execution work taken care by Eclipse.
Only possible reason that I can think of are:
1. Scheduling using Windows 'Task Schedular'
2. Ant allows creating a batch file for execution, so execution gets simplified.
2. Craeting ReportNG reports that are triggered through ANT
Is there any other good reason?
Also why do I need Jenkins?
Jenkins can be really helpful when you want Continuous Integration
builds to be Tested frequently.
In the environment where continuous development and Continuous
Integration goes hand in hand we can use Jenkins to test the build
regularly.
With Selenium Automation we can build/develop the automation script
for the developed builds. Once the Automation is developed we can
create a bat file and use Jenkins to run the bat file.
With Jenkins we can also set the time duration when to execute the
test builds like every day at 12:00 pm IST. Also we can get easily
notified if any test build execution fails because Jenkins also
provide features to send email notification on failure.
Also in some odd cases where we need to test the AUT on Linux/Unix
environment we can easily deploy the Automation on the such Test
system and use ANT to run the Selenium Automation from command
prompt.
Related
When I run my selenium test (mvn test) from jenkins (windows) I see only the console output. I don't see the real browsers getting opened. How can I configure jenkins so that I can see the browsers running the test?
I think you're mis-understanding the reason you would use Jenkins, i believe it's designed to do exactly what it is doing, you need to install a different test-framework along with Maven and Selenium.
Jenkins is an open source DevOps tool which is basically use to automate repetitive tasks like code deployment on server.
SERVER being the main operating word here, you are trying to run your automation scripts from your desktop.
https://www.youtube.com/watch?v=peDWh9q_d0s
Above is a video tutorial of how to setup Maven and a very popular automation test framework, testNG.
Give me a shout if you need any more from me,
All the best,
Jack
Recently we moved to Contentious Integration for build management using Teamcity and also configured a few automated junit tests as ant tasks. To extend it further I'm looking at running a static code analysis tool like FindBugs as a step whenever a build triggers. I would appreciate any helpful suggestion.
You can run FindBugs or other command line tool from TeamCity easy. Just create a command line task, point to your exe and off you go. Sonar is also another very good tool, here is a sample instance http://nemo.sonarqube.org/
We are using TFS/VS 2010 to run Selenium tests which are scheduled in the TFS controller. After the build and tests are finished I would like to run the failed tests from that build.
Currently I am doing this by using a Windows Scheduled Task and executing a batch file which calls a powershell script which gets the latest build version (and failed tests) and then executes them (using mstest) and finally publishes the results back to build.
I just want this to happen without a windows scheduled task, it is too fickle. I believe I need to edit ProcessTemplate.xaml and add an event (InvokeProcess) to achieve this, I just can't find much on it.
Thanks in advance!
The group that I work in has standardized on Jenkins for Continuous Integration builds. Code check-in triggers a standard build, Cobertura analysis and publish to an Artifactory SNAPSHOT repo. I've just finished adding a new target to the master build file that'll kick off a Sonar run but I don't want that running on every check-in.
Is there a way to schedule a nightly build of a specific build target in Jenkins? Jenkins obviously facilitates scheduled builds but it'll run the project's regular build every time. I'd like to be able to schedule the Sonar build target to run nightly.
I could, of course, create a separate Jenkins project just to run the Sonar target on a schedule but I'm trying to avoid that if I can. Our Jenkins server already has several hundred builds on it; doubling that for the sake of scheduling nightly builds isn't very desirable. I looked for a Jenkins plug-in that might facilitate this but I couldn't find anything. Any suggestions?
Here's one way to do it, if you are ok with triggering the build using cron or some other scheduling tool:
Make the build parameterized, and use a parameter in your build file to decide if the Sonar build target should run or not.
Trigger the build remotely by HTTP POST:ing the parameter values as a form to http://[jenkins-host]/jobs/[jobname]/buildWithParameters. Depending on your Jenkins version and configuration, you might need to add an Authentication Token and include this in your url.
Authenticate your POST using a username and password.
wget --auth-no-challenge --http-user=USERNAME --http-password=PASSWORD "https://[jenkins-host]/job/[jobname]/buildWithParameters?token=<token defined in job configuration>&<param>=<value>&<param2>=<value2>"
I am also looking for a solution for this. My current solution in my mind is to create 2 triggers in the regular build, one is the nightly build, another one is Polling SCM
In the sonar plugin configuration, it has the options to skip the builds triggered by the SCM change. Therefore, only the nightly build will start a sonar analysis.
I didn't get a chance to test it now, but I suppose this will work.
Updated on 12/19/2011
The above solution doesn't work if the sonar analysis is invoked as a standalone build step. To make the sonar analysis run conditionally, you could use the following 2 plugins:
Conditional BuildStep Plugin - this allows the sonar analysis to be run conditionally
Jenkins Environment Injector Plug-in - this allows you to inject the variables to indicate how the build is triggered.
I've just installed Hudson and it is running beautifully. It builds, runs JUnit-tests and also CheckStyle analysis.
Next step for us would be to create an installation, install it and then run automated tests on the actual installation. I would then like to fail the build if the tests fail or at least publish the results somehow. I think we would set it up so that part runs periodically or manually triggered.
We use InstallAnywhere for installation and IBM Rational Functional Tester for automated tests.
So questions are: anyone created a similar setup? are there any plugins, tutorials or other resource that could help me along. Or do you have any tips or advice in general.
The command line reference for Rational Functional Tester:
http://publib.boulder.ibm.com/infocenter/rfthelp/v8r0m0/index.jsp?topic=/com.ibm.rational.test.ft.doc/topics/RobotJCommandLine.html
Sample command for running a test:
java -classpath "C:\IBM\RFT\FunctionalTester\bin\rational_ft.jar"
com.rational.test.ft.rational_ft -datastore \\My_project\AUser\RobotJProjects -user admin -project
\\My_project\AUser\TestManagerProjects\Test.rsp -build "Build 1" -logfolder "Default" -log
"Al_SimpleClassicsA#1" -rt.log_format "TestManager" -rt.bring_up_logviewer true -playback
basetests.SimpleClassicsA_01
An additional note, you'll want to configure windows properly on your agent machine which will be running the tests. This is not advice specific to Hudson or RFT, but rather all GUI automation tools on Windows. RFT will require an interactive desktop environment for it to be able to click buttons, etc. If you have your Hudson agent running as a Windows service, there will be no desktop. See the following: Silverlight tests not working unless RDP connection open
We have run a fairly complicated distributed build on Hudson, it is a process that basically follows:
Test on Windows.
Test on OSX, run code coverage & push results to server.
Test on OSX Tiger.
Package for OSX Leopard & push build to server.
Package for Windows & push build to server.
Update product website.
We don't use InstallAnywhere or Rational Functional Tester, but have similar sorts of mechanisms in their place. The key we found to making it all sing in Hudson was being able run our various steps from the command line. Maven and appropriate plugins made short work of this task. So my advice would be just that, using whatever build tool you are using (ant, maven, ?) configure them so that you can run your rational functional tester and install anywhere from the command line with a simple goal passed to your build tool (i.e. mvn test or mvn assembly:assembly).
After that, make sure whatever machine Hudson is running on has everything installed (i.e. Rational Functional Tester) and configured, so that you can open up the command line and type in the goal and have your tests correctly execute.
Hooking it up in Hudson from that point on is fairly simple - just pass in the goal when you configure the build.
I believe the best answer is that integrating RFT with Hudson/Jenkins is a useless endeavor.
As this IBM FAQ says, to make RFT work you must:
be logged in the machine;
the screen can't be locked;
if you are remotely connected, you can't minimize the connection screen.
So you can't run Jenkins/Hudson as a service, making it not very useful. You must run it from your logged account. If you are in a corporate computer (very probable if you are using RFT), you probably must use a hack to prevent the screen saver to start. If the screen is locked, your tests will always fails.
It isn't very difficult to configure your tests to run from the command line, you just have to take care of the return codes when the tests fail and succeed.
Jenkins/Hudson would also give you some advantages, like integrating the tests with your version control, probably automatically running the tests when a commit is made. It would also help sending emails when the tests fail.
But you still would have to integrate the RFT logs with some kind of JUnit plugin to have a nice report. You also would have to have script to run the tests using the command line.
I think it is not worth the trouble to use an continuous integration server with RFT. Better just have your tests running every day in Windows Task Scheduler. It is a simpler solution with less failure points.
Or use my final solution: quit RFT and use the free Selenium with a headless web driver.
I have some general advice on this because I have not yet implemented this myself.
I am assuming you want to have Hudson run the RFT scripts automatically for you via a build or Hudson process?
I want to implement something similar in my organisation as well.
I have not yet been able to implement this because of organisational constraints but here is what I have thought out/done so far:
Downloaded Windows process viewer, got the command for running the tests.
Made shell Script out of it, separated out the variables etc
The future plan is to setup a Windows Slave machine which would have all the tools in it that would be required once the Tests are kicked off, for eg. the correct versions of browsers, and environment variables, and other tools that are required.
Hudson would kick off a process which runs the shell scripts created which runs all the RFT Scripts and performs necessary operations on the slave machine.