All of the load tests documentation on MSDN describes a computer (running visual studio) connecting to a test controller, which connects to the test agents. Is it possible to run Visual Studio on the Test Controller, and run tests from there?
Running Visual Studio on a Controller with agents is more or less like running local test. You can run load tests from a single (developer?) machine, but you usually can't generate enough traffic to really see how the application responds, especially if the target is hosted on the same machine. It's also very depending on your context.
Using Controllers & Agents in a test rig, provides a distributed architecture to generate a lot of load and activity.
Related
I've got some CodedUI tests I'd like to run on a remote machine from visual studio. Now it used to be that you'd install a test controller somewhere, a test agent on the box you want to run on, get them talking, then use a test settings file that was pointed to that controller. I was going through this and installed agents for 2015 update 3 on my run box. I noticed that there is no controller configuration option. After some googling I found this: https://vstfsalm.wordpress.com/2015/08/18/tfs-2015-test-agent-has-not-been-configured-run-the-test-agent-configuration-tool-to-configure-the-test-agent/. Very cool. So I go in to tfs to set up a machine group. Great. The part I don't get is: how can I now tell VS to run at that group? Test settings files require a controller when they are set to remote run. Do I have to downgrade the agents back to 2013? Does anyone know if Microsoft just didn't implement this work style in 2015 (on purpose or otherwise) or if it just isn't implemented yet?
To run the code UI test via vNext build on TFS update3. First make sure the code UI test can be executed successfully in your Visual Studio on your remote machine locally. Also don't forget to configure the build agent to interact with desktop.
Then you just need to add Deploy TestAgent on RemoteTestMachine task and run Functional test. When you queue a build, it will deploy your code ui test solution to the drop folder on the build server machine. Then use MSTEST to run the test assembly. It's same with running code UI test via MSTEST command line on your local machine.
Update
Q: Will the Test Agent 2015 support all the scenarios supported by
Test Controller and Test Agent of Visual Studio 2013?
A: We recommend you use Agents for Visual Studio 2015 in all the new automated testing scenarios. You can use the Deploy Test Agents task in a build definition to download and install the test agents on your machine. The following table shows the scenarios supported by Agents for Visual Studio 2013 and the alternatives for Team Foundation Server (TFS) 2015 and Team Services (TS).
Source Link: Install and configure test agents
Currently, there is no alternative for this scenarion. You may have to use test agent 2013 to achieve it.
I'm trying to set up a build server script to run VS Load Tests. My preferred build server is Team City, but I would accept VSO's build server as well.
Thanks!
You should not use build server to run load test.
Visual Studio Online provides cloud-based load test, you can use it for your testing.
Instruction about VSO Load Test: https://www.visualstudio.com/en-us/get-started/test/load-test-your-app-vs
I have a solution with a WebAPI project and a VS Test project.
My tests make calls to the API using RestSharp via a Url rather than instantiating the controller itself and injecting things (context etc).
My question, is there a way to tell the test project to launch the web project via IIS Express when a test run begins? Currently I just run two instances of VS, one with the web projected with debugging started and the other running the test package
If you are trying to debug both the test and the service, consider doing this
Launch to debug your Web service in Visual Studio.
Click Debug -> Detach All. IIS Express will keep running.
Set a break point and start Debuging your Unit Test.
(Skip this if you don't need to debug the web service) Click Debug -> Attach to Process. Find iisexpress.exe and attach.
However, you lose Edit and Continue on your web service which was detached.
I wouldn't recommend using the network to unit test Web API. You open yourself up to potential flakiness in the test and you end up testing a whole lot more than the service itself.
But if you really must do so, maybe to test that your client can exchange information with the API, then I'd suggest you look into self-hosting the service:
http://www.asp.net/web-api/overview/hosting-aspnet-web-api/self-host-a-web-api
Self-hosting lets you start up a server for Web API with just a few lines of code and you could have your tests start up this server in the right places. In most cases, this should behave the same as having your service hosted in IIS Express. But there are some important distinctions. For example, you won't be able to use some System.Web concepts you may be used to (like HttpContext.Current).
Update:
I've written a blog post about testing Web API services that might help -
https://learn.microsoft.com/en-us/archive/blogs/youssefm/writing-tests-for-an-asp-net-web-api-service
Hope that helps.
I know this is an old post but I was just faced with this issue. I can provide a more detailed response if / when anyone reads this.
In short.. I created a console app referencing the unit test assembly and via reflection and a simple menu system you can run any one of your tests.
I then set multiple startup projects to be the Web project and the console project.
I can then F5 and debug both the unit test and the Web project from within the same session. No attaching to a process of multiple solutions needed.
Running your web service or site in release non-debug by CTRL-F5 makes it run independent from Visual Studio and you are free to run your Tests from inside VS
I know that this is late but I use this solution:
Start two instances of Visual Studios. Start debugging the API in one VS and then debug the test in another.
I have a test project that I want to automate on a test server. For now, I have installed the Visual Studio agents (Test Controller, Test Agent and Test Lab) on my local machine, as I hope to try it out here before touching anything on the test server.
The problem I'm having is that I can't find adequate documentation on automating tests using the Visual Studio agent tools. I have successfully set up and configured the test agent and test controller on my local machine, but I'm not sure where I go from here. The test controller is connected to Team Foundation Server. I have a test project contained within TFS - I want to run the tests within it.
I tried launching the VS test manager and attempted to find a way to run tests from there, but I was unsuccessful.
How do I go about doing this? (in simple steps)
My previous experience is with NUnit, and launching automated tests was as simple is launching the command line NUnit program, providing it with the path of the test project and a few other parameters.
Once the Visual Studio agents are set up and configured correctly, it's simply a matter of running tests against a .testsettings file which is connected to the test controller.
When the tests are run (either via the Visual Studio GUI or the mstest.exe command-line program), the active testsettings file will run the tests on any test agent which is connected to the test controller.
So in terms of automating tests, it's just a matter of specifying the correct testsettings file as one of the command-line switches to the MSTest.exe.
A testsetting file can either be created within the Visual Studio solution or alternatively they can be created within Microsoft Test Manager (automated tests can also be scheduled using the MS Test Manager command-line program, tcm.exe).
To select an active testsetting file to use when running tests from Visual Studio 2010, select the 'Test' menu on the toolbar, then 'Select Active Test Settings'.
I have to implement VSTS 2010 for Unit and Load Testing.
But I didn't know the process of implementing VSTS 2010 but I know the process of performing load testing
I have to implement VSTS 2010 for Unit and Load on my present TFS environment.
Can anyone provide some advice and guidance with regard to implementing VSTS 2010 for unit and load along with TFS, so that any one can use VSTS like TFS.
You can use Visual Studio to create and run unit tests.
http://msdn.microsoft.com/en-us/library/ms182532.aspx
You can use TFS Team Build to automate running unit tests.
http://geekswithblogs.net/jakob/archive/2009/06/03/tfs-team-build-2010-running-unit-tests.aspx
But running LOAD tests with TFS? hmmmm. I don't know if anyone has done it before but I don't like the idea. Load test environments need to be very controlled environments and I don't see a reason to do LOAD testing as a part of continuous integration.
I believe it can be done by digging into TFS 2010 Lab Management with virtual machines or something but it not worth the effort.