I am using Jenkins as my CI build server. After a build of our software is complete it starts up the automated tests on a slave machine. The tests that use a web browser run at a very good pace, the tests that run on two different local applications run very slowly. It takes about 3 seconds between each keystroke.
If I start the tests manually through Visual Studio 2010 on the same slave machine the local application test run just fine (fast keystrokes).
Any idea why the local apps are so slow when run through Jenkins?
I'm not sure if this is the same case, but we had similar issues with an automated UI test, and found that we must have a real session open on the slave running the test. NOT RDP.
We did this by using VNC to login to the slave and leave the session open.
I hope this helps.
Related
We have a testing framework made in AutoIt for our Windows apps (older legacy apps that we will continue supporting). These have never been run on a schedule or part of CI (always been run manually). I tried to get some kind of auto run (or even just status reporting) out of the tests, with minimal luck.
I have a VM where the tests can run. I experimented with my own web app, which works okay locally for running and status reporting. But when set up on the server, AutoIt reports it cannot open the application. Same thing happens if I try to run the tests from a .bat file.
My current solution is to have AutoIt call my web app to report status (working okay locally, untested on the server), or to see if I can get AutoIt to report results back to TeamCity. I have the agent installed but when I run the build from TeamCity, AutoIt reports it can't launch the application. I tried this while logged into the VM, logged out, with RDS open; no luck.
Is it possible to run the tests manually from the VM and send results back to TeamCity? When I run them from TeamCity it reads the AutoIt output (which is in the expected format), but I need to let TeamCity know to update the results (so we can use TeamCity rather than my web app to show the results).
I may need to find a way to let TeamCity know a build has been started, which might then let it know to keep an eye on the process' output, but I'm not positive. Any ideas?
I solved this so it could be done more traditionally.
If anyone is confused by what "running the agent from the console" meant, it just means installing the agent without selecting the "as a service" checkbox, and then manually starting the agent by cd'ing to the BuildAgent/bin directory and running the command agent start. I also created a batch file that will do this automatically (but you must run it as admin).
Further, I found AutoIt couldn't do anything if the test doesn't run from the right directory, so I had to devise a solution to this.
The only issue now is that I have to have an old laptop always connected to the virtual machine the tests and app run on (since the AutoIt tests won't work without the VM desktop being interactive).
To test our desktop application we have a Master project that runs the Slave project on different VMs.
We are using TeamCity to start our tests. On a TestManagement machine (VM) we have a Build Agent that is running as Service. This Build Agent starts the Master project with TestExecute.
This project connects with RDP to different VM to run our UI Tests (Slave project) of our App. We are using Network Suite and run our tests on Hosts (VMs) like it is suggest here: Using Network Suite.
But we have an issue with the screen resolution that is used to run our UI Tests. The resolution is too low.
This page Running tests via RDP gives an explanation:
"When running tests on a remote computer that participates in distributed testing, TestComplete creates a Remote Desktop session and automatically sets the master computer's screen resolution on the remote computer. This is done to avoid possible problems with test running."
So TestExecute will use the master computer's screen resolution.
But TeamCity Agent is run as a service on a VM (TestManagement machine) and there is no screen resolution because it's an headless machine and not like a real user that will connect with RDP to this VM and start the Master Project.
I assume TestExecute will then use the default screen resolution (something like 640x480) as the resolution for the RDP connection. But this is too low to run our tests, some object are not on the screen and we have many issues...
Is there a way to change the resolution used by the RDP connection that TestExecute / TestComplete will create ?
The tricky part is that the Master project is started from a service on an headless machine...
Thank you for your answers,
Camille
In the past, we experienced a similar problem as yours and we were unable to solve it by this way. As solution (right now working for us) we installed into our test environments the TightVNC. This able us to forget the screen connection (RDP issues) because you configure the screen resolution by RDP the first time, then access via IP, log in and you will be able to launch whatever you want without open any application.
Maybe its not the best solution but for our testing environments we can launch tests from Jenkins on demand without screen resolution problems.
My Window Tester UI Tests(All the tests which has error dialogs and error messages in in RCP Application) are failing in Jenkins server, but they are passing in my local machine.
Is there anything to do with machine configurations, like based one executors?
if the machine configurations doesn't matter why they are not failing in my local machine?
You are probably not running your Jenkins server in a desktop session, meaning that it does not have access to your GUI (e.g. it can't launch anything that pops-up a window or a dialog).
So you either need to start the Jenkins master manually from the command line or allow it to access the GUI if it's run as a service.
See also these two related topics:
How to run GUI tests on a jenkins windows slave without remote desktop connection?
https://serverfault.com/questions/285065/gui-tests-in-hudson-jenkins-on-windows
Im working with VS2010 and TFS and the Coded UI tests (CUIT).
I created some CUIT and added them to several Test Cases in Microsoft Test Manager 2010.
These tests are executed in our build-deploy-test steps.
This looks as follows (simplified):
Build application
Deploy
2a. Set snapshot of the Virtual Machine to which the application will be deployed to the snapshot in which the application is not installed
2b. Install application
Run tests
Step 3 contains Coded UI tests, but they cannot be executed (will fail), since there is no active remote desktop connection, which they need to be executed.
Now I would like to add a build step 2c. That automatically connects to the Remote Desktop so that the Coded UI Tests kan be executed.
How should I create this step in the Process Template?
I attempted this myself by running a PowerShell script to connect to a machine via Remote Desktop (see here to see how to use the Remote Desktop command-line utility).
This didn't really work for me personally because I was running this script from the build machine - which I could only connect to via Remote Desktop - and so when I close Remote Desktop it would close the session on the build machine and thus I would not be able to obtain a remote desktop connection with my test environment (plus it wasn't ideal to have to launch Remote Desktop from the build machine).
Therefore, the solution I went with instead was to enable auto-logon on the test environment (there are various ways to do this, I can't remember how I did it, but it's easily Googled).
So in my PowerShell script, which launched from our build machine, I used a command to remotely restart the test environment. With auto-logon enabled, the test environment would start back up with an active session, then I could carry on with Visual Studio Coded UI testing. This worked for me without any issues.
The command I used was something like this:
Shutdown.exe /r \\mytestenvironment
Here is some documentation for it.. If you intend on using this command to remotely restart a computer, don't forget to give your machine permissions on the test environment, otherwise this command won't work.
I then launched a command to sleep for a few minutes, to give time for the remote test environment to start back up again (with an active session), then I launched the tests.
Personally, I used a PowerShell script but you could also use a DOS/Batch script if you are more familiar with that.
Also, are you trying to run the automated tests as part of a build process template?
Let me know if you need any further help.
I'm trying to run Watin from within a TeamCity build, using nUnit.
All tests run fine locally. I know you cannot run the full Watin tests (i.e. POST) from TeamCity if it is running as a Windows Service. You must start the build agent from a .bat file.
But, I don't want to have to login to the server for it to start.
I've tried getting a Scheduled Task (in Windows Server 2008) to fire the agent.bat file on StartUp (not Login), but with no luck.
Has anyone else got Watin/TeamCity running from a Scheduled Task?
Thanks,
Pete
Check out the solution to doing this using CC.NET.
Watin Tests fail on CC.Net
Brett and I worked on this for awhile. The problem is that Windows Services running as a user cannot interact with the desktop, but because WatiN runs a real instance of the browser it requires a desktop to interact with. The solution in the link provided can show you how to do this.