MTM : Is there a way to find test run completion status using command line utility tcm.exe - microsoft-test-manager

I use MTM (Microsoft test manager) for running my automated test cases.
I schedule test runs using tcm /create command (trigerred from a powershell script) and once the test run is completed I need to copy the trx (result) file on my local machine. So I want to wait till the test run is completed in some sort of polling mechanism.
Hence, I need a command to fetch the current test run status using test runid. Is there a way to get MTM test run status in this way?

I don't think this is possible. The available switches for run option are:
delete
abort
export
list
create
publish
The only data you can take about runs using /list is
Id
Title
Owner
Date Completed
you can see this by running:
tcm run /list /planid:<plainId> /collection:<CollectionUrl> /teamproject:<TeamProject>
Moreover, you don't have the runId yet so even if there was an option to get the completion status, in your case that would not be easy.
So, I think you should start looking for another solution. Perhaps the TFS Api is what you need.
Check these links:
Automation test run creation using tfs api
TFS 2010 API - Get Results of a Test Run

You can actually get the test id - its what is printed out as a result of tcm.exe run /create command
in powershell it would be something like this:
$testRunSubmitResult = .$tcmPath run /create ......
$testID = $testRunSubmitResult -match "[0-9]{1,1000}"
(i excluded the error handling logic which needs to be present in order to verify that the run was submitted)
after that you can do the following thing - you can export the test run with the used id, and if the test didnt finish yet, you will get and error.
do
{
Start-Sleep -s 60
$testResults = .$tcmPath run /export /id:$testID /resultsfile:$args /collection ....
if(Test-Path $args[0])
{
break
}
if($testResults.GetType() -eq #("1").GetType())
{
if($testResults[1].Contains("Completed export"))
{
break
}
}
if ($testResults.Contains("Completed export"))
{
break
}
}
while ($True)
This is not perfect as it might fail in test runs with big attachments (such as ones produced by the Video data collector) but it might be a solution for some of you
Or alternatively from powerscript you can just use the TFS API like this:
Add-Type -AssemblyName "Microsoft.TeamFoundation.Client, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a",
"Microsoft.TeamFoundation.Common, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a",
"Microsoft.TeamFoundation.TestManagement.Client, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a"
$tfs = [Microsoft.TeamFoundation.Client.TfsTeamProjectCollectionFactory]::GetTeamProjectCollection("http://tfs:8080/tfs/Collection")
$tfs.EnsureAuthenticated()
$testManagementService = $tfs.GetService([Microsoft.TeamFoundation.TestManagement.Client.ITestManagementService])
$testManagementTeamProject = $testManagementService.GetTeamProject('Project');
do
{
Start-Sleep -s 60
$testRun = $testManagementTeamProject.TestRuns.Find($testId);
if($testRun.State -eq 'Completed')
{
break
}
if($testRun.State -eq 'NeedsInvestigation')
{
break
}
if($testRun.State -eq 'Aborted')
{
break
}
}

Related

Issue with Set-CMTaskSequenceDeployment

Seems that New-CMTaskSequenceDeployment / Set-CMTaskSequenceDeployment cmdlet option -DeploymentOption does not work as expected.
I'm trying to automate a Task Sequence Deployment via Powershell. I use New-CMTaskSequenceDeployment cmdlet to create the deployment. The content of the TS should be downloaded before the start of the TS.
Works ok, but the -DeploymentOption DownloadAllContentLocallyBeforeStartingTaskSequence seems not to have any effect, when I check the deployment after the script ran, the option "pre-download content for this task sequence" isn't checked.
Same issue when I try Set-CMTaskSequenceDeployment.
Any hint from the community what I'm doing wrong?
...
# Create deployment for all waves now
foreach ($StrCollectionName in $ArrCollectionName)
{
$SchedulePhase2 = New-CMSchedule -Nonrecurring -Start $DateScheduleStartPhase2
Try {
$Deployment = New-CMTaskSequenceDeployment -CollectionName $StrCollectionName -TaskSequencePackageId $StrTaskSequenceID -DeployPurpose Required -AvailableDateTime $DateAvailablePhase1 -DeploymentOption DownloadAllContentLocallyBeforeStartingTaskSequence -SoftwareInstallation $False -SystemRestart $False -Schedule $SchedulePhase2 -RerunBehavior RerunIfFailedPreviousAttempt -AllowUsersRunIndependently $True -SendWakeUpPacket $True
Write-Host "Success - Deployment $Deployment created!"
}
Catch {
Write-Host "Error - Exception caught in creating deployment : $error[0]"
Exit
}
}
...
Looks like unfortunately (and unexpected) the pre-download behavior is different for package/program deployment and task sequence deployment.
In case of a package/program deployment the content download will start after start time in case the deployment has a mandatory time defined.
A TS deployment behaves different. It will start download when mandatory time (schedule) has been reached. Start time will be ignored.
This difference is independently from how the deployment has been started (Console or powershell cmdlet) therefore it is not an issue of the cmdlet.
First of all, you can check the picture below to make sure not to confuse these two options.
Difference between Preload content checkbox and Download all content locally before starting TS
Once done Here is my proposition :
Just by clicking, try to retrieve the property of you TSDeployment before and after you clicked the checkbox. You will see that one property changed. AdvertFlags
PS MUZ:\> (Get-CMTaskSequenceDeployment -DeploymentID MUZ200C5).AdvertFlags
[Convert]::ToString((Get-CMTaskSequenceDeployment -DeploymentID MUZ200C5).AdvertFlags,2)
Output :
34275328
10000010110000000000000000
From there, you can read from the MS doc : https://learn.microsoft.com/en-us/configmgr/develop/reference/core/servers/configure/sms_advertisement-server-wmi-class
From this, I learn that I need to change the 12th bit like this :
$advertflag = Get-CMTaskSequenceDeployment -DeploymentID MUZ200C5
$advertflag.AdvertFlags = $advertflag.AdvertFlags -bor "0x00001000"
$advertflag.put()
I hope it will help someone someday :)

Cannot find a way to monitor uninstallation of preinstalled Win10 Apps

Currently working on automating the building of some images. I want to have it all documented and have a way to ensure nothing is ever skipped when an image needs created, updated, etc. It will save me a lot of headaches. Below is only the relevant code for this question
Part of uninstalling 'Preinstalled Win10 Apps' is not giving me what I am looking for.
I am not able to accurately see when an 'App' is truly uninstalled.
When it executes Remove-AppxPackage for an App, it will add a progress bar to the top of the PS Window showing its progress and when it completes. It disappears after a while, even if it does not show completed. It will continue to execute behind the scenes but I cannot tell when they are completed. Some are taking longer to uninstall than others.
My goal is find a way to monitor the removal. I cannot find anything built into Remove-AppxPackage for me to do that. I cannot find anything to check against.
I tried wrapping them in jobs using Start-Job, Wait-Job, Remove-Job but that didn't change anything. It was being marked as complete while the uninstall hadn't completed.
Any help or pointing me to a possible solution would be appreciated. Currently I either assume it is done after a lengthy period of time or I rerun the code to see if it tries to process the uninstall.
Testing on a clean install of Win10 Build 1709
List of Apps I am wanting to remove
[System.Collections.ArrayList]$Win10_Apps_To_Uninstall = #(
"Microsoft.BingWeather"
"Microsoft.GetHelp"
"Microsoft.Getstarted"
"Microsoft.Microsoft3DViewer"
"Microsoft.MicrosoftOfficeHub"
"Microsoft.MicrosoftSolitaireCollection"
"Microsoft.Office.OneNote"
"Microsoft.OneConnect"
"Microsoft.People"
"Microsoft.Print3D"
"Microsoft.SkypeApp"
"Microsoft.Wallet"
"microsoft.windowscommunicationsapps"
"Microsoft.WindowsFeedbackHub"
"Microsoft.XboxApp"
"Microsoft.XboxGameOverlay"
"Microsoft.XboxIdentityProvider"
"Microsoft.XboxSpeechToTextOverlay"
"Microsoft.ZuneMusic"
"Microsoft.ZuneVideo"
)
Code to loop through the removal and unprovisioning
foreach ($Win10_App in $Win10_Apps_To_Uninstall) {
try {
Get-AppxPackage -Name $Win10_App -AllUsers | Remove-AppxPackage -Verbose
}
catch {
}
try {
Get-AppXProvisionedPackage -Online |
Where-Object DisplayName -EQ $Win10_App |
Remove-AppxProvisionedPackage -Online -Verbose | Out-Null
}
catch {
}
}

How to fix failed tests count in teamcity for re-executed tests?

We are using TeamCity for Selenium tests execution with Specflow+Specrun, the problem is that TeamCity count re-executed tests.
For example if some test failed for first time he will be re-executed two times more, in teamcity we will see that three tests failed, but probably it was one test.
Also if first re-execution will fail, but other two will success, this will be reported in teamcity as two failed, one passed, but I need to be reported that only one test has been passed.
It is possible to configure this in TeamCity using service messages or something else?
UPDATED:
Based on answer we can gather logs using powershell script and change build status using teamcity service messages:
$path = Get-ChildItem %teamcity.build.checkoutDir%\ProjectFolder\bin\Remote\TestResults\specrun.log
$file = Get-content $path
$total = $file | select-string "Total:"
$passed = $file | select-string "Succeeded:"
$failed = $file | select-string "Failed:"
write-host $( "##teamcity[buildStatus text='Tests {0}, {1}, {2}']" -f $total, $passed, $failed )
TeamCity is behaving as expected. TeamCity reports what Testing Framework tells him. Roughly the process is:
Execute step that run tests.
Gather the output logs.
Extract stats: number of test executed, passed and failing.
Fail the build if tests fail if configured to do that.
You have to configure your Testing Framework (specflow+specrun in your case) to change the way it reports the re-execution. You should configure specflow to log the re-executions in a different way... if it can.
In general you should avoid re-execution of tests. The tests should work at the first attempt. They should prepare the state if needed, do their test, and clean the state.
The need of re-execution indicates probably some dependency with the other tests. Something like:
Test 001: Search for user Foo1. (FAILS)
Test 002: Adds user Foo1. (OK)
Re-execution of Test 001: Search for user Foo1. (OK)
The correct way is that TEST001 does not have a dependency of TEST002, adds the user Foo1, test search and clean user Foo1.

Protractor - run test again if fails

I'm using shell script to run protractor tests.
I want to make sure that if the test fails (exit code != 0) then it will run again - three times most.
I'm already using Teamcity, but Teamcity sends the 'FAIL' email and only then tries again. I want the test will run three times before sending a message.
this is part of my script:
if [ "$#" -eq 0 ];
then
/usr/local/bin/protractor proactor-config.js --suite=sanity
now I want to somehow check whether the Exit Code was 0 and of not - run again.
Thanks.
I wrote a small module to do this called protractor flake. It can be used via the cli
# defaults to 3 attempts
protractor-flake -- protractor.conf.js
Or programatically.
One nice thing here is that it will only re-run failed spec files, instead of your test suite.
There is a long standing feature request for this in the protractor issue queue. It probably won't be baked into the core of the framework.
function to check status
function test {
"$#"
local status=$?
if [ $status -ne 0 ]; then
echo "error with $1" >&2
fi
return $status
}
test command1
test command2
If you use protractor with cucumber-js, you can choose to give each scenario (or all scenarios tagged as unstable) a number of retries:
./node_modules/cucumber/bin/cucumber-js --help
...
--retry <NUMBER_OF_RETRIES> specify the number of times to retry failing test cases (default: 0) (default: 0)
--retryTagFilter <EXPRESSION> only retries the features or scenarios with tags matching the expression (repeatable).
This option requires '--retry' to be specified. (default: "")
Unfortunately if every failed scenario has been successfully retried, Protractor will still return with exit code 1:
https://github.com/protractor-cucumber-framework/protractor-cucumber-framework/issues/176
As a workaround, when starting the protractor I append the following to its command line:
const directory = 'build';
ensureDirSync(directory);
const cucumberSummary = join(directory, 'cucumberSummary.log');
protractorCommandLine += ` --cucumberOpts.format=summary:${cucumberSummary} \
|| grep -P "^(\\d*) scenarios? \\(.*?\\1 passed" ${cucumberSummary} \
&& rm ${cucumberSummary}`;

Best Way to Trigger an Event When a Specific Error is Thrown in Windows 2003 Server?

I have a Windows 2003 Server that uses IIS to host a legacy ASP.NET web service that connects to a database on a remote Oracle database server that I have no control over. The problem is that the database server goes down every week or two, but then comes back up after about 5 minutes. I have to then restart IIS to remove any corrupt connections before my web service works again.
What is the best way to trigger an event (i.e. email myself and/or reset IIS) when a specific error code occurs (in this case it will be an ORA- type error, but I can get the Windows error code)?
IIS Setting?
Task Scheduler? (limited to scheduled tasks only I believe on Windows 2003 server, eg. per day/week/month etc)
Powershell Script?
Other options?
I know in Windows 2008 Server that you can configure the Task Scheduler to trigger an event when the server experiences certain error codes in its Error Log... but I can't find anything like this in the Task Scheduler of Windows 2003 Server.
Thanks.
I wrote a powershell shell script to monitor the sql server errorlog and report on specific errors. I stored where I last stop reading and then continued on from that point the next time I ran the script. This is the part that actually reads the log. Then you just need to store the position in some temp file and run this as a scheduled task. And send an email if the error occurs or even restart some service.
$path = $logs.file
Write-Host $path
if($currentLog.lastpos -ne $null){$pos = $currentLog.lastpos}
else{$pos = 0}
if($logs.enc -eq $null){$br = New-Object System.IO.BinaryReader([System.IO.File]::Open($path, [System.IO.FileMode]::Open, [System.IO.FileAccess]::Read, [System.IO.FileShare]::ReadWrite))}
else{
$encoding = $logs.enc.toUpper().Replace('-','')
if($encoding -eq 'UTF16'){$encoding = 'Unicode'}
$br = New-Object System.IO.BinaryReader([System.IO.File]::Open($path, [System.IO.FileMode]::Open, [System.IO.FileAccess]::Read, [System.IO.FileShare]::ReadWrite), [System.Text.Encoding]::$encoding)
}
$required = $br.BaseStream.Length - $pos
if($required -lt 0){
$pos = 0
$required = $br.BaseStream.Length
}
if($required -eq 0){$br.close(); return $null}
$br.BaseStream.Seek($pos, [System.IO.SeekOrigin]::Begin)|Out-Null
$bytes = $br.ReadBytes($required)
$result = [System.Text.Encoding]::Unicode.GetString($bytes)
$split = $result.Split("`n")
foreach($s in $split)
{
if($s.contains(" Error:"))
{
#Filter events here
}
}
$currentLog.lastpos = $br.BaseStream.Position
$br.close()

Resources