I've got some CodedUI tests I'd like to run on a remote machine from visual studio. Now it used to be that you'd install a test controller somewhere, a test agent on the box you want to run on, get them talking, then use a test settings file that was pointed to that controller. I was going through this and installed agents for 2015 update 3 on my run box. I noticed that there is no controller configuration option. After some googling I found this: https://vstfsalm.wordpress.com/2015/08/18/tfs-2015-test-agent-has-not-been-configured-run-the-test-agent-configuration-tool-to-configure-the-test-agent/. Very cool. So I go in to tfs to set up a machine group. Great. The part I don't get is: how can I now tell VS to run at that group? Test settings files require a controller when they are set to remote run. Do I have to downgrade the agents back to 2013? Does anyone know if Microsoft just didn't implement this work style in 2015 (on purpose or otherwise) or if it just isn't implemented yet?
To run the code UI test via vNext build on TFS update3. First make sure the code UI test can be executed successfully in your Visual Studio on your remote machine locally. Also don't forget to configure the build agent to interact with desktop.
Then you just need to add Deploy TestAgent on RemoteTestMachine task and run Functional test. When you queue a build, it will deploy your code ui test solution to the drop folder on the build server machine. Then use MSTEST to run the test assembly. It's same with running code UI test via MSTEST command line on your local machine.
Update
Q: Will the Test Agent 2015 support all the scenarios supported by
Test Controller and Test Agent of Visual Studio 2013?
A: We recommend you use Agents for Visual Studio 2015 in all the new automated testing scenarios. You can use the Deploy Test Agents task in a build definition to download and install the test agents on your machine. The following table shows the scenarios supported by Agents for Visual Studio 2013 and the alternatives for Team Foundation Server (TFS) 2015 and Team Services (TS).
Source Link: Install and configure test agents
Currently, there is no alternative for this scenarion. You may have to use test agent 2013 to achieve it.
Related
I have created a continuous integration in dev.azure pipeline. I have created a build>>release>> added Agent as VSTS Test Platform Installer >> added Visual Studio Test runner which will use vstest.console.exe to execute webtests.
I have a solution>project>webtest scripts and supported files which i ran locally successfully in Visual Studio 2017. Now i am trying to move these web tests to pipeline such that i can run them as CI.
I have added everything as below in dev azure pipeline:
Added Visual Studio Test Platform Installer. vsTestVersion: 15.0
Configured assembly test files as ***test*.dll
***.WebTests
***.testsettings.
Search folder as: $(System.DefaultWorkingDirectory)/_Performance/drop/
Version as: Visual Studio 2017
Settings file as: $(System.DefaultWorkingDirectory)/_Performance/drop/Local.testsettings
Build platform: AnyCPU and rest settings as runInParallel: false
codeCoverageEnabled: true
testRunTitle: 'LoadTest_$(rev:r)'
platform: AnyCPU
configuration: Release
continueOnError: true
Now when i am saving my release and running it. It runs the whole solution and copy all files successfully in the virtual drive under D:\a\r1\a_Performance\drop
But as a next step once the test runs after VSTS Installer it shows.......................................
............
No test is available in D:\a\r1\a_Performance\drop\WebAndLoadTestProject1\bin\Release\Microsoft.VisualStudio.QualityTools.WebTestFramework.dll D:\a\r1\a_Performance\drop\WebAndLoadTestProject1\bin\Release\WebAndLoadTestProject1.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
2019-03-04T20:37:35.7477518Z
enter image description here
I figured out a quick solution. Convert the .webtest file to a code test file and checkin that code in git. Run the same solution now in azure DevOps release. It is running the same test successfully in the assigned vsts2017 VM. So as per my understanding it seems Visual Studio webtests cannot run directly in pipeline release.
To convert the webtests into code open the vsts webtest on the toolbar and there is a Generate Code button in bottle shape. Clicking this button prompts for a test name, and then generates code of the same webtest. Both the webtest and code are separate from each other. Changes in one will not added into other.
I have a situation : I have two servers (server A and server B) and my TFS making build of my application on server A and should deploy the results on server B. After that it should run UI tests remotely.
For doing that I use TFS 2015 and I added a few steps for my build definition (if you understand what i mean ))) ).
Firs step is "Visual Studio Build" step which is build my project. -> is working fine
The second step is "Publish Build Artifacts" for publishing the build results from server A to server B (maybe it is not best solution for publishing something between machines but is working for me. If you have some advice for that, I will be appreciate to hear it).-> is working fine
Third step is "Visual Studio Test Agent Deployment" which should deploy Test Agent on server B. When this step is runed it shows 'agentlocation' is not a valid path. Ensure that the account with which build agent is running has access to this share. And the all if fails on this step but root to file vstf_testagent.exe is acceseble by everyone (For sharing of this folder was set Everyone Read/Write).
Fourth step is "Visual Studio test using Test Agent" which should run my UI tests. I hope that this step can run my UI tests.
on server A I installed Test Controller and on server B I installed Test Agent and gave him name of Test Controller. On server A I use Visual Studio 2015 but on server B I use Visual Studio 2013.
Please help me to understand problem with third step. What can it be and how to resolve it?
And can step fourth run my UI tests?
Thanks a lot for any help!
The field of "Test Agent Location" is optionally, which is used to supply the path to vstf_testagent.exe from network or local location. If no path is provided, it will be downloaded from network. You can leave this field blank and let it download from network, maybe you can deploy a test agent successfully.
I'm trying to set up a build server script to run VS Load Tests. My preferred build server is Team City, but I would accept VSO's build server as well.
Thanks!
You should not use build server to run load test.
Visual Studio Online provides cloud-based load test, you can use it for your testing.
Instruction about VSO Load Test: https://www.visualstudio.com/en-us/get-started/test/load-test-your-app-vs
The NUnit Test Adapter for VS 2012 allows you to run NUnit tests from within the Visual Studio 2012 IDE via the Test Explorer. However, the reason I have NUnit tests in the first place is because I have tests written against SharePoint 2010 (I know, these are Integration Tests, not Unit Tests), which, due to the requirement that the SharePoint API be accessed through a 64-bit client means that MSTest/VS can't run them.
I was hoping that with the NUnit Test Adapter, however, I would be able to run the SharePoint tests from within Test Explorer in Visual Studio but I keep getting the error message:
SetUp failed for test fixture xyz
SetUp : System.IO.FileNotFoundException : The Web application at http://sp could not be found. Verify that you have typed the URL correctly. If the URL should be serving existing content, the system administrator may need to add a new request URL mapping to the intended application.
This indicates to me that the NUnit Test Adapter is running as a 32-bit process (I suspect the process is vstest.executionengine.x86.clr20.exe).
Is there any way to run NUnit tests that target SharePoint 2010 using the NUnit Test Adapter and Test Explorer in Visual Studio 2012?
To run tests in 64-bit process:
In Visual Studio 2012, select TEST->Test Settings->Default Processor Architecture->x64 menu item
With vstest.console.exe, specify /Platform:x64 command line option.
You may also be interested in SharePoint Emulators, which allow you run SharePoint integration tests in isolation.
I have a test project that I want to automate on a test server. For now, I have installed the Visual Studio agents (Test Controller, Test Agent and Test Lab) on my local machine, as I hope to try it out here before touching anything on the test server.
The problem I'm having is that I can't find adequate documentation on automating tests using the Visual Studio agent tools. I have successfully set up and configured the test agent and test controller on my local machine, but I'm not sure where I go from here. The test controller is connected to Team Foundation Server. I have a test project contained within TFS - I want to run the tests within it.
I tried launching the VS test manager and attempted to find a way to run tests from there, but I was unsuccessful.
How do I go about doing this? (in simple steps)
My previous experience is with NUnit, and launching automated tests was as simple is launching the command line NUnit program, providing it with the path of the test project and a few other parameters.
Once the Visual Studio agents are set up and configured correctly, it's simply a matter of running tests against a .testsettings file which is connected to the test controller.
When the tests are run (either via the Visual Studio GUI or the mstest.exe command-line program), the active testsettings file will run the tests on any test agent which is connected to the test controller.
So in terms of automating tests, it's just a matter of specifying the correct testsettings file as one of the command-line switches to the MSTest.exe.
A testsetting file can either be created within the Visual Studio solution or alternatively they can be created within Microsoft Test Manager (automated tests can also be scheduled using the MS Test Manager command-line program, tcm.exe).
To select an active testsetting file to use when running tests from Visual Studio 2010, select the 'Test' menu on the toolbar, then 'Select Active Test Settings'.