Mstest not setting the errorlevel correctly - visual-studio-2010

I have a test project that builds fine. I am trying to run the tests using mstest. Here is the command I am executing:
Command:
echo mstest.exe /testcontainer:SecurityLogging.Tests\Logging.Tests.dll /resultsfile:%TEST_LOG_ROOT%\Execute_%TestListNode%_1.trx /detail:Owner /detail:Description /nologo /testsettings:Logging.Tests\Local.testsettings /category:"^!E2E&^!Stress&^!Perf"
My tests FAILED. If I then check the error level. It is still 0. I did:
if %errorlevel%==0 (
continue;
)
and it always continues.
I am using Visual Studio 2010. I started digging and I found this:
http://www.mail-archive.com/ccnet-user#googlegroups.com/msg03027.html
Thanks

MSTest is not failing. It executes the tests fine. Your tests are failing and their errors are handled by MStest. So, I think you will not get any %errorlevel%.

I've just confirmed that mstest.exe for Visual Studio 2010 does in fact exit with errorlevel 1 when it runs tests and some of them fail. When all the tests pass, it returns errorlevel 0.
You can verify this with the following lines in your batch file after running mstest:
echo %errorlevel%
pause
If your batch file is being called by another batch file, and you want to pass the errorlevel back, then you can use if not %errorlevel% == 0 exit 1 to shortcut your batch file.
Otherwise, as #Magoo indicates, you may have an unnecessary echo at the front of your call to mstest.exe.

Related

Rest of script not running after calling a batch file to run?

I have the following script batch script:
call standalone.bat
"C:\Program Files (x86)\Notepad++\notepad++.exe" "C:\Program Files\jboss-eap-6.2\jboss-eap-6.2\standalone\log\server.log"
The first command runs as expected but the script never seems to call notepad to open the server.log file.
What is the issue here?
Edit: ending of standalone.bat is:
if ERRORLEVEL 10 goto RESTART
:END
if "x%NOPAUSE%" == "x" pause
:END_NO_PAUSE
Stephan likely diagnosed the problem in his question comment - standalone.bat probably terminates with an exit command. If that is correct, then you can work around the problem without modifying standalone.bat by changing
call standalone.bat
to
cmd /c standalone.bat.

PostBuild Event fails in Visual Studio after SignTool.exe error

We have a project in Visual Studio 2010 that runs a batch file in the post-build event. That batch calls to signtool.exe from Microsoft SDK to sign and timestamp the binary.
Timestamp servers (we use http://timestamp.verisign.com/scripts/timstamp.dll), however, tend to be unreliable for some reason, failing sometimes. This caused build to fail.
We implemented a more advanced batch script then (based on this code), splitting signing and timestamping, and allowing to retry the timestamp operation, if it failed.
Here is a simplified version of the batch script (signfile.bat):
#echo off
REM sign the file...
signtool.exe /f Authenticode.pfx /p PASS %1
if %errorlevel% neq 0 exit /b %errorlevel%
set timestamp_server=http://timestamp.verisign.com/scripts/timstamp.dll
for /L %%a in (1,1,10) do (
REM try to timestamp the file...
signtool.exe timestamp /t %timestamp_server% %1
if errorlevel 0 if not errorlevel 1 GOTO succeeded
REM wait 2 seconds...
ping -n 2 127.0.0.1 > nul
)
REM return an error code...
echo signfile.bat exit code is 1.
exit /b 1
:succeeded
REM return a successful code...
echo signfile.bat exit code is 0.
exit /b 0
And the post-build event code would look like:
signfile.bat "$(OutDir)$(TargetName)$(TargetExt)"
So, if the timestamping fails, it retries 10 times with 2-seconds intervals.
But, what we observed was, if the timestamping went fine from the first attempt, everything was OK. However, if the first attempt failed, then the whole post-build event failed with code -1, even though the timestamping succeeded on the next try.
1>------ Build started: Project: myproject, Configuration: NonOptimized x64 ------
1> Done Adding Additional Store
1> Successfully signed: E:\tfs\MySolution\bin\x64\NonOptimized\myproject.dll
1>
1>EXEC : SignTool error : The specified timestamp server either could not be reached
1> or returned an invalid response.
1> This may happen if you specify an RFC 3161 timestamp URL but used
1> the /t option or you specified a legacy Authenticode timestamp URL
1> but used the /tr option.
1>EXEC : SignTool error : An error occurred while attempting to timestamp: E:\tfs\MySolution\bin\x64\NonOptimized\myproject.dll
1>
1>
1> Number of errors: 1
1>
1> Successfully timestamped: E:\tfs\MySolution\bin\x64\NonOptimized\myproject.dll
1>
1> signfile.bat exit code is 0.
1>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\Microsoft.CppCommon.targets(113,5): error MSB3073: The command "signfile.bat "E:\tfs\MySolution\bin\x64\NonOptimized\myproject.dll"
1>C:\Program Files (x86)\MSBuild\Microsoft.Cpp\v4.0\Microsoft.CppCommon.targets(113,5): error MSB3073: :VCEnd" exited with code -1.
========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========
So, as you can see, even though the error code returned from signfile.bat is 0, Visual Studio thinks it is -1 and fails the event.
All attempts to clear the error flag, like adding ver>nul here and there, or adding exit 0 in the end (certainly with adding "call" before signfile.bat) didn't help since it seemed like Visual Studio checked not just for errorlevel but also for something else. In fact, the batch as well as signfile.bat only return 0 or 1 in case of error, but not -1. And if signtool.exe returns an error once, it seems like there is no way to convince Visual Studio not to fail the post-build event.
After spending much time experimenting and searching, found an article, mentioned here in a comment. It looks like Visual Studio scans the output, searching for some special keywords. Signtool.exe outputs among the other EXEC : SignTool error : An error occurred, which seems like enough to alert Visual Studio that there was an error.
So, the solution proposed was to redirect output and error streams to nul as 2>nul 1>nul. Errorlevel will still be set, so you will be able to figure out if error occured. But you may have to print some extra messages to see the status:
REM try to timestamp the file...
signtool.exe timestamp /t %timestamp_server% %1 2>nul 1>nul
if errorlevel 0 if not errorlevel 1 (
echo Successfully timestamped: %1
GOTO succeeded
)
echo Timestamping failed for %1
Now Visual Studio is happy:
1>------ Build started: Project: myproject, Configuration: NonOptimized x64 ------
1> Done Adding Additional Store
1> Successfully signed: E:\tfs\MySolution\bin\x64\NonOptimized\myproject.dll
1>
1> Timestamping failed for "E:\tfs\MySolution\bin\x64\NonOptimized\myproject.dll"
1> Successfully timestamped: "E:\tfs\MySolution\bin\x64\NonOptimized\myproject.dll"
1> signfile.bat exit code is 0.
========== Build: 1 succeeded, 0 failed, 0 up-to-date, 0 skipped ==========
In fact, just adding 2>nul would be enough to fix it. Error stream will still be printed: Number of errors: 1, but it does not cause a problem.
I experienced a very similar problem, but with the added difficulty that rather than calling SignTool directly, our post-build step calls a PowerShell script which itself calls SignTool. I set up some retry logic in the PowerShell script (to try various timestamp servers, with delays), but found that if the first attempt failed then the whole build would fail, even though the error was being caught and handled properly and a subsequent signing attempt was succeeding.
I appreciate this isn't exactly what the original question is asking, but I wasted hours and hours on this before eventually working out a solution (having stumbled upon this Q&A - thank you!), so I'm posting this info in the hope that it will help others.
From PowerShell, the standard call to SignTool would be something like this:
$output = & $signToolPath sign `
/n $companyName `
/d $productName `
/du $productWebsite `
/t $timestampServer `
/sha1 $shaHash `
$filePath
If SignTool returns an error code, this will immediately fail the build (as shown in the TeamCity screenshot), which is not desirable if the script is capable of automatically retrying:
To prevent the build failing, the PowerShell error stream can be suppressed (similarly to the solution in the accepted answer) with the PowerShell redirect statement 2>&1 as follows:
$output = & $signToolPath sign `
<switches and args as above>
$filePath 2>&1
However, this means you can't see the error output when SignTool returns an error code, which is obviously unhelpful. You can address this by writing the output to the host like this:
$output = & $signToolPath sign `
<switches and args as above>
$filePath 2>&1
$output | Write-Host
but then Visual Studio / MSBuild will resume trying to be "clever" and fail the build again!
The only solution I have found which allows SignTool error messages to be displayed without failing the build is both to redirect the error stream and to garble the error output before writing to the host, something like this:
$output = & $signToolPath sign `
<switches and args as above>
$filePath 2>&1
$output -split "([a-z0-9])" -join " " | Write-Host
The error message is then human-readable but doesn't fail the build, as shown here:
Of course, if all the retry attempts fail then you need to return an appropriate non-zero exit code from the PowerShell script so that the build does fail.
Thanks again for the original Q&A, and I hope this extra info on PowerShell helps someone else.
(PS: one of the links in the accepted answer is broken, but the referenced article is archived here: http://web.archive.org/web/20180729111947/http://blog.robertromito.com/2010/08/ignore-error-from-visual-studio-post.html )

Visual Studio Pre-Build Event: Batch File Exit Code is -1

In order to work around the issue IIS Express Blocks VS 2010SP1 Builds I have created a simple batch file
taskkill /IM iisexpress.exe
exit 0
and set that batch file as a pre-build event.
If IIS Express is actually running, it works great. However, if IIS Express is not running the build fails with the following output:
The process "iisexpress.exe" not found.
The command "E:\Software\Util\KillIisExpress.bat" exited with code -1.
If instead of the batch file I use the taskkill command as the pre-build event, the error changes to
The command "taskkill /IM iisexpress.exe" exited with code 128.
How can I modify the batch file so that, no matter what exit code taskkill returns, the batch file returns with an exit code of 0 so that the VS build succeeds?
You can use
taskkill /FI "IMAGENAME eq iisexpress.exe"
which will output an info message if iisexpress.exe isn't running, but will return 0. You don't need the batch file with this, just add the command as the Pre-Build Event Command Line.

Event Build pre & post to stop and restart Windows service

I've been experimenting with using build events to start and stop Windows service that are being built in my project. However for the pre & post builds fail with an error level 255. I've tried catching this with the pre-build with no luck.
Pre-build
if "$(ConfigurationName)" == "Debug"
(
net stop myService
if errorlevel 2
if errorlevel 255
:exit
:exit
)
Post-build
if "$(ConfigurationName)" == "Release"
(
copy $(TargetDir) C:\Media\Bin\$(ProjectName)
if errorlevel 1 BuildEventFailed
:BuildEventFailed
mkdir C:\Media\Bin\$(ProjectName)
copy $(TargetDir) C:\Media\Bin\$(ProjectName)
)
else if "$(ConfigurationName)" == "Debug"
(
net start myService
)
The following weblog by Joel Varty has a solution which I use:
Use Build Events to rebuild a Windows Service without having to manually stop/start it
The only problem is when you do a rebuild. Visual Studio cleans up the files before the pre-build event fires. This then off course fails because the service is still running. But regular builds work great. Hope this helps.
Try using the opening parenthese on the first line of your pre-build code
The conditional statement doesn't require double-qoute ("")
It should be like
if $(ConfigurationName) == Debug (
net stop myService
...
)
This is how I got it to work:
(this solution was a part of an enterprise software where some dll files are reused by another app)
Model is a project which is referenced in Service project and it is built before Service. Which is why we write these codes in Model's Pre-Build Events:
Model Pre-Build Event:
if not exist "$(SolutionDir)UI\bin\Debug\ServiceFolder" mkdir "$(SolutionDir)UI\bin\Debug\ServiceFolder"
net start | find "[Service Name]"
if ERRORLEVEL 0 (
net stop "Service Name"
"C:\Windows\Microsoft.NET\Framework\v4.0.30319\InstallUtil.exe" -u "$(SolutionDir)UI\bin\Debug\ServiceFolder\Service.exe"
)
exit 0
creates a directory in output folder
finds the service by name
stops it
uninstalls it
exit 0 causes the build process to continue if error occurs here
Service Post-Build Event:
xcopy /E /Y "$(ProjectDir)bin\Debug\*" "$(SolutionDir)UI\bin\Debug\ServiceFolder"
"C:\Windows\Microsoft.NET\Framework\v4.0.30319\InstallUtil.exe" "$(SolutionDir)UI\bin\Debug\ServiceFolder\Service.exe"
net start "Service Name"
copy everything needed for the service to a different folder
installs service
starts service
About permissions?
visual studio will ask for elevated permission automatically

msbuild fails to register output

I'm trying to get our build scripts (which use MSBuild) working correctly on Vista and am finding that projects that have the Register Output (in linker options) option set to True fail to build from the command line with something like this:
Project : error PRJ0050: Failed to register output. Please try enabling Per-user Redirection or register the component from a command prompt with elevated permissions.
Although I can easily fix this for a single machine, by running as admin or whatever I want the build script to "just work" for any dev machine.
Even to just fail the registration but have the build continue would be satisfactory. Any suggestions?
Brad
You could create cmd-file that will be contain the following text:
#echo off
call regsvr32.exe /s %1
if %errorlevel% EQU 0 goto ok
echo Fail to register %1
goto exit
:ok
echo Register successful %1
:exit
After that you should switch off registering output and one should add Custom Build Step with command <pathtocmdscript> $(TargetPath). Output one should set to $(TargetPath) for Custom Build Step.
Finally you'll got message about registering progress, but compilation will not stop on that step.

Resources