I am unable to associate test methods to test cases in the Test Explorer (the "Associate to Test Case" option is greyed out), or via MTM, or the VSTS website. I simply cannot find a way to associate tests to the test cases for automated testing.
I am using .NET core app 1.1, ASP.NET using MSTest (though I have tried Xunit as well, to no avail). Even a basic test case that always passes cannot be associated.
In MTM, I have created a test plan that has the Automation Status set to "Plan".
In VSTS, I have a project using a Git repository, in which I have a CI build created that successfully discovers, runs and reports the tests.
I cannot find where to associate these specific tests to test runs. How may I accomplish this?
Refer to these steps to associate test method to test case:
Install MTM 2017 (Run VS 2017 install app (vs_Enterprise.exe)>Modify)
Open VS 2017>Tools>Options>Work Items>General> Select Visual Studio (Compatibility mode) for Open work items in:
Open your test project in VS 2017 and build
Open Team explorer and connect to your team project
Type test case id in Search Work items box> Press enter to open test case
Select Associated Automation tab and click …
Select a test method > OK
Save test case work item
Another way is that you can associate the test method with test case through Update a field REST API.
For example:
PATCH https://[account].visualstudio.com/DefaultCollection/_apis/wit/workitems/[testcaseid]?api-version=1.0
Content-Type: application/json-patch+json
Body:
[
{
"op": "add",
"path": "/fields/Microsoft.VSTS.TCM.AutomatedTestName",
"value": "[namespace.classname.methodname (e.g. UnitTestProject1.UnitTest1.TestMethod2)]"
},
{
"op": "add",
"path": "/fields/Microsoft.VSTS.TCM.AutomatedTestStorage",
"value": "[assembly name(e.g. unittestproject1.dll)"
},
{
"op": "add",
"path": "/fields/Microsoft.VSTS.TCM.AutomatedTestId",
"value": "[guid id]"
},
{
"op": "add",
"path": "/fields/Microsoft.VSTS.TCM.AutomatedTestType",
"value": "Unit Test"
},
{
"op": "add",
"path": "/fields/Microsoft.VSTS.TCM.AutomationStatus",
"value": "Automated"
}
]
The AutomatedTestId is a Guid value, so you can generate a new Guid by using this C# code:
Guid g = Guid.NewGuid();
string s = g.ToString();
Check this extension that I have created https://github.com/JanuszNowak/janono.ado.testcase.associate.cli . It allows associating in automatic manner. So if yours selenium test are using xunit, nunit or mstest you can associate them automatically.
Code sample:
namespace ExampleTestProject
{
[TestClass]
[janono.ado.testcase.associate.Organization("janono-pub")]
public class UnitTest1
{
[TestMethod]
[janono.ado.testcase.associate.TestCase(5)] //<---
public void TestMethod1()
{
//yours test method content
//...
//
}
}
}
Related
I need to configure the following params:
environment, trend, history, executors, retries,
etc. I need these params for cucumber to work with ruby. I searched in a lot of places and I did not find much. I would appreciate if you can provide these params.
History
Allure stores history information to allure-report/history folder during report generation. So you need to copy such folder from previous launch into your allure-results before you generate the report.
History features are handled out of the box by Allure CI plugins
Executor info
To add an information about your test executor create a file executor.json in your allure-results:
{
"name": "Jenkins",
"type": "jenkins",
"url": "http://example.org",
"buildOrder": 13,
"buildName": "allure-report_deploy#13",
"buildUrl": "http://example.org/build#13",
"reportUrl": "http://example.org/build#13/AllureReport",
"reportName": "Demo allure report"
}
In the report such information will be displayed like this:
Allure CI plugins will add executor information for you
Categories
You can also add custom problem categories in the report. Create file categories.json and place it into your results folder:
[
{
"name": "Ignored tests",
"messageRegex": ".*ignored.*",
"matchedStatuses": [
"skipped"
]
},
{
"name": "Infrastructure problems",
"messageRegex": ".*RuntimeException.*",
"matchedStatuses": [
"broken"
]
},
{
"name": "Outdated tests",
"messageRegex": ".*FileNotFound.*",
"matchedStatuses": [
"broken"
]
},
{
"name": "Regression",
"messageRegex": ".*\\sException:.*",
"matchedStatuses": [
"broken"
]
}
]
Retries
Retries are handled by default. For each test Allure generates historyId (usually it is md5(fullName + parameterValues)). If report contains few results with the same historyId Allure only displays latest, other are marked as retries and available from Retries section on test result page.
For example, if you run your test 3 times (first 2 are broken, latest is passed, Allure will display such test as passed)
Using VS2015 community update 3 on Windows 7, with last NUnit3. I can only "Run All" tests, other options (Ctrl+R,Ctrl+T or Run Selected tests) in most cases execute unrelated tests, even from other class.
I also noticed that test name in tooltip in TestExplorer may be not equals to test name in hovered item in list.
project.json:
{
"dependencies": {
"NUnit": "3.4.1",
"NUnitLite": "3.4.1",
"dotnet-test-nunit": "3.4.0-alpha-2"
},
"frameworks": {
"net461": {
}
},
"testRunner": "nunit",
"buildOptions": {
"emitEntryPoint": true
}
}
I am running into some performance issues trying to run the chutzpah.console.exe with my chutzpah.json file. For this project, I am using TypeScript and pre-compiling the TypeScript files into .js files before running the tests. Each test file that I have generally references one other TypeScript file that it depends on - this is so Visual Studio is happy and so that the Chutzpah extensions work. For example, my drawerBusiness.spec.ts file contains this reference:
/// <reference path="../../../../app/business/documents/drawerBusiness.ts"/>
My chutzpah.json file is as follows:
{
"Compile": {
"Mode": "External",
"Extensions": [ ".ts" ],
"ExtensionsWithNoOutput": [ ".d.ts" ]
},
"Framework": "jasmine",
"TestHarnessReferenceMode": "Normal",
"TestHarnessLocationMode": "SettingsFileAdjacent",
"References": [
{ "Path": "./lib/jquery/jquery.js" },
{ "Path": "./lib/angular/angular.js" },
{ "Path": "./lib/angular-mocks/angular-mocks.js" },
{ "Path": "./lib/jasmine-jquery/jasmine-jquery.js" },
{ "Path": "./lib/underscore/underscore.js" },
{ "Path": "./lib/angular-bootstrap/ui-bootstrap-tpls.js" },
{ "Path": "./tests/dependencies.js" }
],
"Tests": [
{ "Include": "*/tests/specs/*.ts", "ExpandReferenceComments": "true" }
]
}
As you can see from above, I need to load some external libraries. I presume these get loaded once and then are used for each of the tests?
When I was using pure JavaScript, this configuration was blazing fast. It completed quite quickly in Visual Studio and through running the tests from the chutzpah.console.exe. However, since I have switched over to Chutzpah, the tests are much slower:
Tests complete: 263
=== 263 total, 0 failed, took 74.64 seconds ===
These results are even worse on our build machine, and it's timing out. It seems like there's a huge delay between each spec file that is run, and I think that is causing the overall slowness - once the spec file starts running, it completes in a few one-hundredths of a second. Does anyone have any ideas about what could possibly be configured incorrectly?
Here are a few things you can try to help get to the bottom of this issue:
Can you try enableding "EnableTestFileBatching": true in your chutzpah.json file to see if it helps? There can be slowness if you have MANY test files and this setting tells chutzpah to batching them into one harness.
Can you provide the output of the /trace option so I can see all the timings?
Can you provide a repro that I can run locally?
I'm using Jasmine and Chutzpah to test my TypeScript project in Visual Studio 2013 with PhantomJS. If I run the tests using the Chutzpah context menu extension they work fine but when I run them using the ReSharper test explorer, they fail if there are any external dependencies. The error message is ReferenceError: $ is not defined.
My project structure looks like:
- Scripts
- MyApp
- Components
- DatePicker.ts
- Tests
- Components
- DatePickerTest.ts
- typings
- jquery
- jquery.d.ts
- jquery-1.10.2.js
Chutzpah.json
Chutzpah.json looks like
{
"Framework": "jasmine",
"Compile": {
"Mode": "External",
"Extensions": [ ".ts" ],
"ExtensionsWithNoOutput": [ ".d.ts" ]
},
"References": [
{
"Includes": [
"*/Scripts/jquery-1.10.2.js"
],
"Excludes": [ "*/Scripts/*.d.ts" ]
}
],
"Tests": [
{
"Includes": [ "*/Scripts/Tests/*.ts" ],
"Excludes": [ "*/scripts/Tests/*.d.ts" ]
}
]
}
DatePickerTest.ts looks like
/// <reference path="../../myapp/components/datepicker.ts" />
import DatePicker = MyApp.Components.DatePicker;
describe("Datepicker", () => {
var datePicker: DatePicker
beforeEach(() => {
datePicker = new DatePicker();
});
// tests
});
Instantiating the DatePicker fails with the message ReferenceError: $ is not defined because the class uses $.extend() in the constructor.
I've tried declaring the reference to jQuery at the top of the test by adding the line /// <chutzpah_reference path="../../jaquery-1.10.2.js" /> but that doesn't make any difference. Plus my understanding is that adding references in the Chutzpah.json means I shouldn't have to do this anyway.
As suggested in this answer, I have debugged my test in the browser and sure enough jQuery isn't being loaded in the index.html page, so what am I doing wrong?
The problem came down to how ReSharper attempts to resolve dependencies based on the type definition references it finds. The ReSharper unit testing docs say:
If your test file references and (sic) external JavaScript library using a declaration file (*.d.ts), ReSharper will search the original JavaScript file by the name of the *.d.ts file.
My class under test was referencing the jQuery definition file:
/// <reference path="../../typings/jquery/jquery.d.ts" />
The problem was that I also have jquery.cookie.js included in my project (as part of Foundation), and ReSharper was incorrectly including that instead of jquery-1.10.2.js.
Renaming or removing jquery.cookie.js allowed ReSharper to find the correct jQuery file and solved the problem.
I am very new to the Titanium Alloy framework. I am creating a CRUD app. So I need a database. I created a sample.sqlite in app/asset folder using Mozilla Sqlite add on. Then I created a model given below. But after adding the records through the application, I am not able to see the records in db using Mozilla Addon. But in application the added records are showing.So I am confused that the appication is using the sample.sqlite file we created or some other file(s). Please rectify the point. If it uses other file(s) How can I see the database in Mozilla addon? I followed the link but unable to find the Window > Open Perspective > Other > DDMS option in Titanium Studio.
I am using Windows 7 and Genymotion Android Emulator
Here is the Model
exports.definition = {
config: {
columns: {
"Id": "INTEGER",
"Name": "TEXT",
"Age": "INTEGER"
},
adapter: {
type: "sql",
collection_name: "sample",
db_name: "sample",
}
},
extendModel: function(Model) {
_.extend(Model.prototype, {
// extended functions and properties go he
});
return Model;
},
extendCollection: function(Collection) {
_.extend(Collection.prototype, {
// extended functions and properties go here
});
return Collection;
}
};
You want to pull the data from db using Mozilla add on.
For that first you need to open the DDMS that you can find at android-sdk-win > tools > DDMS
Once there go File Explorer > Data > Data > package name > .sqlite file
and save it
You can then use mozilla add on to connect to the .sqlite file.