I'm new to Microsoft Test Manager, but have found a way to generate test plans and doing testing relatively efficiently. Now, I'm trying to learn how to use Test Scribe.
I need the reports to contain passed tests, together with test steps. Right now I'm able to create reports from "plan" with test steps included, and reports from "test" with success/no success included - but how do I get both? I sort of want the data from the "test plan summary" report, containing "step successful" / "step not successful".
Does anyone know how to do that..?
You need to amke some changes in the template of the summary report file generated.
Follow the URL here
Related
How can i retrieve Test settings using Rest API from TFS.
The below mentioned URL is the best i can get . but this is for Test result retention setting...... and what i am looking for is regular/ general Test settings.
Can some one please provide a sample URL for it.
Sample URL:- https://fabrikam-fiber-inc.visualstudio.com/DefaultCollection/Fabrikam/_apis/test/resultretentionsettings?api-version=2.0-preview"
If you mean the test settings file (See Specifying Test Settings for Visual Studio Tests), then you can get and open the file from web portal to check the settings.
If you mean Test configurations, you can refer to this link for the REST API : Test configurations:
e.g.: To Get a test configuration:
GET https://{instance}/DefaultCollection/{project}/_apis/test/configurations/{configurationId}?api-version={version}
Whatever, all the available test REST APIs are all here: Test Management
UPDATE:
According to your clarification (below answer), you just want to get the specific test run details. You can use below REST API to Get a test run:
GET http://{instance}/DefaultCollection/{project}/_apis/test/runs/{run ID}?api-version={version}
Then you will see all the settings/properties of the specific test run.
No what I really meant for was the "Test Settings" information. In MTM, we can specify the TestSettngs via TestSettings Manager.( See the below image.)
While creating a test run, we need to provide this "Test Settings" information.
Next, coming to REST API, we can create a test run using REST API too.
The below is the API reference url.
https://www.visualstudio.com/en-us/docs/integrate/api/test/overview
Under "Create a Test Run" section, you can see "test settings" under the content type. I wanted to know how to get that information via REST call(
Check see image 2 below)
(I am unable to embed pictures on the site as i have not earned 10 reputation points. please open the links provided below to view the images)
Went through the serenity documented for extracting the test outcomes
below isthe code, it didn't work
OutcomeFormat format = OutcomeFormat.XML; TestOutcomes outcomes =
TestOutcomeLoader.loadTestOutcomes().inFormat(format)
Tried with below code and its working,
OutcomeFormat format = OutcomeFormat.JSON; TestOutcomeLoaderBuilder
outcomes= TestOutcomeLoader.loadTestOutcomes().inFormat(format);
TestOutcomes out =outcomes.from(new File(""));
Issue is i need the test outcomes in #AfterScenario, but the thing is serenity reports gets generated after the entire execution tried changing the pom but didn't help. Is there any other way using which we can extract the test results?
Serenity uses JSON format by default now. Why are you trying to obtain the test outcomes? (i.e. what problem are you trying to solve?)
Created a separate java class for report extracting and added that in maven plugin which will get executed after serenity report is generated.
As #John smart has mentioned JSON and HTML are default output formats.
Still if you want to access Outcomes after test execution.
You can create a custom listener and listen to serenity event bus.
TestRunFinished event will be published with Outcome as a parameter.
You can use the outcome for getting required details.
For creating custom listener you can follow this page
I used Microsoft Test Manager to create a test for a log on page (not locally hosted) and recorded this. All steps were executed without errors.
Then I've created a Coded UI test project in Visual Studio 2013.
I added a Coded UI Test with the existing recording.
I ran the test and I got the message below:
"FailedToPerformActionOnHiddenControlException: Cannot perform 'SetProperty of Text with value " on the username field.
I received this message when using the existing recording, but also if I record using the recording function in Visual Studio.
Has someone got experience with Coded UI Tests and maybe give me an example as to how to set it up.
If I understand correctly, you're basically trying to have Coded UI input a value into an HtmlInput control that's a text box on a web page. The error message "FailedToPerformActionOnHiddenControlException: Cannot perform 'SetProperty of Text with value" normally means that the value you're trying to submit isn't able to be taken by the control, either because a value isn't accepted (example, numeric values in a name field) or because it's more characters than can be accepted by the field. It could also be that the recording isn't defining the object properly, and that the property Text isn't an option on the control that was found.
So, confirm these three things:
1. The control you're tying to send value to is actually an HtmlInput. You can't send the Text property to, for example, an `HtmlDiv`.
2. The value you're submitting is a valid value for your control (you're not trying to submit numbers, for example, into a field that won't accept them).
3. The value you're submitting is within the length limit for the field.
If I were a betting man, I would say that you probably don't have the object defined properly. Take a look at the SearchProperties of the control in question first and make sure that it matches the HTML on the page itself.
Do I need to set a standard environment for testing on an other server.
I develop the Coded UI test project in a workspace on a development environment, but I need the tests to be run on a acceptation environment (the recorded tests are executed on the acceptation environment).
I read it's easy to setup this environment though, but what is best practise?
Is it better to execute it on the same environment?
Here's a code snippet:
[TestMethod]
public void TestLogonToAccount()
{
// To generate code for this test, select "Generate Code for Coded UI Test" from the shortcut menu and select one of the menu items.
this.UIMap.Enterusername();
this.UIMap.Enterpassword();
this.UIMap.Clickonlogin();
this.UIMap.ClickonCentral();
this.UIMap.Searchforemailaddress();
this.UIMap.Clickonlogin1();
}
I forgot an important part regarding the login recording test in MTM.
I just tried to play the steps again and got this error:
Playback of the selected sections of the action recording could not be completed The playback failed to find the control with the
given search properties. Additional Details: TechnologyName: 'MSAA'
Name: '' ClassName: 'MozillaWindowClass' ControlType: 'Window'
I am trying to figure out how to run unit tests, using Google Test, and send the results to TeamCity.
I have run my tests, and output the results to an xml, using a command-line argument --gtest_output="xml:test_results.xml".
I am trying to get this xml to be read in TeamCity. I don't see how I can get XML Reports passed to TeamCity during build/run...
Except through XML report Processing:
I added XML Report Processing, added Google Test, then...
it asks me to specify monitoring rules, and I added the path to the xml file... I don't understand what monitoring rules are, or how to create them...
[Still, I can see nowhere in the generated xml, the fact that it intends to talk to TeamCity...]
In the log, I have:
Google Test report watcher
[13:06:03][Google Test report watcher] No reports found for paths:
[13:06:03][Google Test report watcher] C:\path\test_results.xml
[13:06:03]Publishing internal artifacts
And, of course, no report results.
Can anyone please direct me to a proper way to import the xml test results file into TeamCity ? Thank you so much !
Edit: is it possible that XML Report Processing only processes reports that were created during build ? (which Google Test doesn't do?) And is ignoring the previously generated reports, as "out of date", while simply saying that it can't find them - or are in the wrong format, or... however I should read the message above ?
I found a bug report that shows that xml reports that are not generated during the build are ignored, making a newbie like me believe that they may not be generated correctly.
Two simple solutions:
1) Create a post build script
2) Add a build step that calls the command line executable with the command-line argument. Example:
Add build step
Add build feature - XML report processing
I had similar problems getting it to work. This is how I got it working.
When you call your google test executable from the command line, prepend %teamcity.build.checkoutDir% to the name of your xml file to set the path to it like this:
--gtest_output=xml:%teamcity.build.checkoutDir%test_results.xml
Then when configuring your additional build features on the build steps page, add this line to your monitoring rules:
%teamcity.build.checkoutDir%test_results.xml
Now the paths match and are in the build directory.
So lets say I am doing a Test Complete keyword test. If something fails in it the text stops . Actually what I have founded out is that if i have 8 checkpoints if the 4th one fails the rest will always fail after it. So i get a "test execution was interrupted" error. Thats fine but it doesnt finish out the test and close the application. The reason this is an issue is because any tests after it will fail because the application is still left open. I could rewrite these tests so that the application is open when they start but is there a way to kill and application after your tests fail? If the tests pass the application is closed.
You need to organize your tests with test items. In this case, you create at least 3 test items: the first one starts the application, the second performs the test and the third closes the application. If an error occurs during execution of the second test, this second test execution is ended and TestComplete runs the third finalization test item.
Information on test items can be found in the Tests and Test Items help topic. Please note that you need to specify the Test Item value in the Stop on error column for the needed test item (the second one in the above example). Information on this and other columns can be found here. The column is hidden by default and you need to add it: right-click the header of the test items list and select Field Chooser. After this, drag the needed column to the header from the Field Chooser dialog.
Find more information on this solution in Stopping Tests on Errors and Exceptions.
Alternative solution is using the OnLogError or OnStopTest event handlers. Find description of how to handle standard TestComplete events in the Creating Event Handlers for TestComplete Events help topic.
Perhaps I'm oversimplifying, but could it be the setting for the test playback? Pls check the following page and let me know if it helps: http://support.smartbear.com/viewarticle/28751/.
If that doesn't work feel free to repost in the SmartBear Forum: http://community.smartbear.com/
The support team is monitoring the forum and I'm sure they'll be happy to help.