Now on my project we use TFS as TestCase management tool, issue tracking system and code management. But we want to move to JIRA and Bitbucket. In VisualStudio we have ability to bind TC to automation test and then run test suite with automation test. The question is: Is it possible to bind JIRA test case with automation test(UI tests, API tests used MSTest framework)? Is it possible to run automation test and see if test case pass\failed? Some reports?
Maybe there is possibility to develop some extension for this?
Unfortunately, right now there are no built-in JIRA importer for data from Microsoft Team Foundation Server for Visual Studio.
However, there are complete third party solutions available which might help you between JIRA and TFS such as below:
TFS4JIRA
MS TFS Connector for JIRA
UseTFS
Related
I need to somehow associate the automated tests with manual test plan in Azure. I already installed Visual Studio and open my automated test cases in VS. But when I am trying to click "Associate to Test Case" I got the error that functionality is applicable only with MSTest, XUNIT etc. And my framework Pytest is out of this list.
Does that mean I have no possibility to associate my automated test cases with manual one using this functionality? But maybe I can do it in other way? Maybe something can be done with yaml file? Or I can simply tagged automated test cases directly in Test Plan? Or maybe I can install third party plugin for pytest? Or maybe there is a way to build a matrix with manual/automated test cases?
I have no idea, but I need to have the visual picture of test cases matrix.
Does that mean I have no possibility to associate my automated test
cases with manual one using this functionality
Yes, we cannot associate framework Pytest to test case in Azure DevOps. It's not supported.
See What types of tests are supported? for details.
A: These are the limitations for each type of test:
Coded UI test, Selenium tests, and unit tests written using Version 1
of the MSTest framework can be associated with a test case.
Tests that use MSTest v2, NUnit, and xUnit frameworks can be associated with a test case workitem when using Visual Studio 15.9 Preview 2 or later. However, these tests cannot be run using Microsoft Test Manager and XAML builds.
Tests that use the .NET core framework can be associated with a test case workitem when using Visual Studio 15.9 Preview 2 or later. To run the .NET core tests the appropriate target framework must be specified in a runsettings file. However, these tests cannot be run using Microsoft Test Manager and XAML builds.
Tests that use other test frameworks such as Chutzpah (for JavaScript tests such as Mocha or QUnit), or Jest cannot be associated with a test case.
Associating generic tests may work, but running these tests is not supported.
However, you can reference the following blogs to run Python test with Azure DevOps pipeline:
Run Python test with Azure DevOps pipeline
Selenium Python Tests & Azure DevOps Integration – Made for each other!
I have a generic Selenium Test which needs to run over all projects how would i do that with Visual Studio online? I can get it to work on one project at a test but not sure how to make it so this test can run on all projects?
Generally you need to write the specific Selenium test for the specific project. However if all the projects you mentioned have the same structure/components/features, then you can reuse the Selenium test project for them.
I don't think it's this is applicable if you mean create tests together which want to apply for all the different projects.
You can reference below articles about the Selenium Test in TFS:
Get started with Selenium testing in a CD pipeline
Parallel Testing in a Selenium Grid with VSTS
Is there a possibility to add a custom file (e.g. custom Selenium report for all tests) into the body of Run summary itself (Runs -> Run Summary -> Attachments).
Tests are executed using the vNext "Run Functional Tests" task, the unit framework used in tests is a TestTools.UnitTesting.
I know that there is a TestContext.AddTestResult but it attaches file to a specific test, while I want to use some sort of a summary file for the whole run.
Test Run Summary in question
If you want to display your custom result in the test run summary or build result summary, seems you will have to write your own extensions. Help link: Overview of extensions for Visual Studio Team Services
Base for this were a lot of examples provided by MS on github:
Visual Studio Team Services Tasks
Visual Studio Team Services Sample Extensions
To get a first feeling what places on your TFS Web Portal can be extended/customized you can download and install this extension Contributions Guide from the Visual Studio Marketplace.
Hello I recently started with a new company that has developing a internal website for their finance and have been using TFS with Vis studio for quite some time. However they recently decided they wanted to expand into Automated UI testing and pair it with MTM for the QA dept.. The issue I am having is that the Microsoft Test Manager 2013 is not set up with a build to start using the automated/ lab environment features. Im at a loss of even where to go to merge that build with MTM so that I can then use the Test configuration tool to start creating a lab environment. Is there any documentation or resources that might help with the process to.
1)Create a build in MTM (using the web app I have in Vis studio)
2)Setup a Lab environment in MTM using the above build.
3)Being link my automated test cases(although that is pretty straight forward the above to I cannot seem to grasp.
Anything advice would be greatly!! appreciated.
Thank you for your time
Chris
(using Enterprise/Highest level of TFS and Vis stuido 2013)
For (1) please refer the link from MSDN: Working with builds in MTM
Go to Plan->Properties.
you can use the drop down to set the filter for the build (filter by build definition or build quality)
And finally use the Modify hyperlink to assign a new build.
For (2)(3) please refer the paragraph 'Build and install your application in a lab environment ' & 'View and Update the Test Results ' from MSDN Link: How to: Run Automated Tests In a Lab Environment Using Microsoft Test Manager
I have to implement VSTS 2010 for Unit and Load Testing.
But I didn't know the process of implementing VSTS 2010 but I know the process of performing load testing
I have to implement VSTS 2010 for Unit and Load on my present TFS environment.
Can anyone provide some advice and guidance with regard to implementing VSTS 2010 for unit and load along with TFS, so that any one can use VSTS like TFS.
You can use Visual Studio to create and run unit tests.
http://msdn.microsoft.com/en-us/library/ms182532.aspx
You can use TFS Team Build to automate running unit tests.
http://geekswithblogs.net/jakob/archive/2009/06/03/tfs-team-build-2010-running-unit-tests.aspx
But running LOAD tests with TFS? hmmmm. I don't know if anyone has done it before but I don't like the idea. Load test environments need to be very controlled environments and I don't see a reason to do LOAD testing as a part of continuous integration.
I believe it can be done by digging into TFS 2010 Lab Management with virtual machines or something but it not worth the effort.