When I build a Visual Studio 2010 project, I want to run unit tests with NUnit and display test results only when some tests have failed.
I have setup a post-build event in Visual Studio to call a batch file like below:
$(ProjectDir)RunUnitTest.bat "$(SolutionDir)packages\NUnit.Runners.2.6.0.12051\tools\nunit-console.exe" "$(TargetPath)"
Then in RunUnitTest.bat, I call nunit-console.exe and pass in the test project dll.
#echo off
REM runner is the full path to nunit-console.exe
set runner=%1
REM target is the full path to the dll containing unit tests
set target=%2
"%runner%" "%target%"
if errorlevel 1 goto failed
if errorlevel 0 goto passed
:failed
echo some tests failed
goto end
:passed
echo all tests passed
goto end
:end
echo on
After that, NUnit generates TestResult.xml containing test results, so how do I display it in user friendly way? It'll be the best if it displays inside Visual Studio, but other options are open too.
You might want to consider XSLT to perform a transformation and display the results from TestResult.xml.
I ended up using nunit-summary to generate all pass summary html reports and nunit-results to create failed test reports in html.
This approach is quiet easy to setup.
First, download nunit-summary and nunit-results from launchpad and put them in TestRunner folder under the test project.
Then, add a post-build event to call a batch file.
Lastly, add the batch file to TestRunner folder under test project. It should contain the following files at the least:
nunit-results.exe
nunit-results.tests.dll
nunit-results.tests.pdb
nunit-summary.exe
`nunit-core.dll
nunit.util.dll
RunUnitTests.bat
Post-build event for the project containing unit tests:
"$(ProjectDir)TestRunner\RunUnitTests.bat" "$(SolutionDir)packages\NUnit.Runners.2.6.0.12051\tools\nunit-console.exe" "$(TargetPath)" "$(TargetDir)"
Scripts in RunUnitTest.bat
REM This batch file does the followings:
REM 1. runs unit test with nunit-console.exe and produces a TestResult.xml
REM 2. if one or more tests failes, it calls unit-results.exe to convert TestResult.xml to
REM Usage: RunUnitTests.bat "path-to-nunit.exe" "path-to-test.dll" "path-to-output-folder"
#echo off
REM get input arguments
set runner=%1
set target=%2
set output=%3
REM remove double quotes
set runner=%runner:"=%
set target=%target:"=%
set output=%output:"=%
REM prepare and clean up TestResult folder
if not exist "%output%TestResults\nul" md "%output%TestResults"
del "%output%\TestResults\*.*" /q
"%runner%" "%target%"
if errorlevel 1 goto failed
if errorlevel 0 goto passed
:failed
echo some tests failed
"%~dp0nunit-results.exe" "%output%TestResult.xml"
"%output%TestResults\index.html"
exit 1
:passed
echo all tests passed
"%~dp0nunit-summary.exe" "%output%TestResult.xml" -out=TestResults\TestSummary.html
"%output%TestResults\TestSummary.html"
exit 0
Related
I like the Qt Creator development environment and now need to compile some C code using the Keil C51 compiler/tools.
Is it possible to use the Qt Creator IDE with Keil C51 tools? Can someone describe in detail how to set that up?
OR - is than another 8052 chip compiler that can be integrated with Qt Creator?
I can confirm, that I'm able to use QT with C51.
Environment
I use QT as Editor.
And Keil C51 as compiler. I write a program in a pure C (not C++) for the STC microcontroller.
TLDR
I use the Windows Batch file to compile all *.c files in the working directory and then link it together into a one hex file. Then I setup the QT (Projects -> Build Settings -> Build Steps -> Custom) to use this Batch during a building stage (don't forget to setup the working directory correctly).
How to get a needed Batch file
Simple way
Keil can generate a batch for the particular project (see Project->Options for Target->Output->Create Batch File). Then use this file in the QT as described above.
The main flaw of this way is - when you change the project configs or add a new file to the project, you need to regenerate a Batch from the Keil.
Better way
Using generated Batch from Keil, I write my own Bath, that doesn't have the flaws, described above.
Even better way
I guess using a Makefile is better for this than Batch, as it gives QT more control and info about the errors during the compilation. But I dont't have an "on-shelf" example right now.
Batch file description
In a nutshell, this Batch:
Setups the settings and directories
Calls compiler for every *.c file in the working dir
Builds *.obj list and call linker to obtain a firmware image in a binary format
Converts the obtained imge into a HEX format
The listing:
REM "Folders"
SET C51FLDR=d:\Keil\C51
SET OUTFLDR=.\Objects
SET LSTFLDR=.\Listings
SET HEXNAME=PREP_FIRMWARE
SET C51INC=%C51FLDR%\Inc;%C51FLDR%\Inc\STC
SET C51LIB=%C51FLDR%\Lib
REM "Variables"
SET C51EXE=%C51FLDR%\BIN\C51.EXE
SET C51LNK=%C51FLDR%\BIN\BL51.EXE
SET C51HEX=%C51FLDR%\BIN\OH51.EXE
set OBJLIST=
REM "Defines"
SET CPU_TYPE=STC15W408S
SET CPU_VENDOR=STC
SET UV2_TARGET=Target 1
SET CPU_XTAL=0x02160EC0
SET SETTINGS="COMPACT ROM(COMPACT) OPTIMIZE (9,SIZE) BROWSE DEBUG OBJECTEXTEND LISTINCLUDE SYMBOLS TABS (2)"
echo ===================== Compile =====================
for %%f in (*.c) do (
echo %errorlevel%
setlocal EnableDelayedExpansion
%C51EXE% %%f %SETTINGS% "PRINT(%LSTFLDR%\%%~nf.lst) OBJECT(%OUTFLDR%\%%~nf.obj)"
if !errorlevel! neq 0 exit /b !errorlevel!
)
echo ===================== Link =====================
call :obj_list
echo %OBJLIST%
%C51LNK% %OBJLIST% TO "%OUTFLDR%\%HEXNAME%" PRINT("%LSTFLDR%\%HEXNAME%.m51") RAMSIZE(256)
echo %errorlevel%
if %errorlevel% GTR 1 exit /b %errorlevel%
echo ===================== Hex =====================
%C51HEX% "%OUTFLDR%\%HEXNAME%"
if %errorlevel% neq 0 exit /b %errorlevel%
echo ==========================================
echo Build Success
echo ==========================================
goto :eof
:obj_list
for %%f in (.\Objects\*.obj) DO call :concat %%f
set OBJLIST=%OBJLIST:~0,-1%
goto :eof
:concat
set OBJLIST=%OBJLIST%"%1",
goto :eof
Right now, it is possible to use QtCreator with Qbs (since 1.15). Also, it is possible to use VSCode with the Qbs extension.
Is there any way to execute multiple feature files within a project using specrun.exe
eg: I have a project Student, and I have the below feature files:
a.Maths
b.Biology
C.Chemistry
d.Physics.
Now I need to execute Maths and Physics together. How to do it?
For single feature file (Eg:Maths) the below command is working fine.
#pushd %~dp0
%windir%\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe "Student.csproj"
#if ERRORLEVEL 1 goto end
#cd ..\packages\SpecRun.Runner.*\tools
#set profile=%1
#if "%profile%" == "" set profile=Default
SpecRun.exe run %~dp0\%profile%.srprofile "/baseFolder:%~dp0\bin\Debug" /log:specrun.log %2 /filter:testpath:Feature:Maths %4 %5
:end
#popd
Finally, I got the answer.
we have to exclude the feature files which we don't want to run.
The command that worked fine for me is as below
SpecRun.exe run %~dp0\%profile%.srprofile "/baseFolder:%~dp0\bin\Debug" /log:specrun.log "/filter:!testpath:Feature:Biology& !testpath:Feature:Chemistry"
I have a VS2008 solution file named MySolution.sln containing three projects ProjectName1, ProjectName2, ProjectName3 and ProjectName4. I am using jenkins to build the solution(to built only ProjectName1/2/3 and not ProjectName4). Below is the batch which has been provided to jenkins.
The issue is whenever lets say, the statement "%VSDIR%\devenv.com" "MySolution.sln" /Build "Release|x64" /Project ProjectName1 failed to build the particular project my %errorlevel% is not updated to non negative integer (>0). Irrespective of pass/fail the %errorlevel% is always "0"
The way jenkins calls this is
cmd /c call C:\Users\John\AppData\Local\Temp\hudson343434346343.bat
Any ideas on this?
Batch script:
#ECHO OFF
set VSDIR=C:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE
"%VSDIR%\devenv.com" "MySolution.sln" /Clean "Release|x64"
"%VSDIR%\devenv.com" "MySolution.sln" /Build "Release|x64" /Project ProjectName1
IF %ERRORLEVEL% EQU 0 (
"%VSDIR%\devenv.com" "MySolution.sln" /Build "Release|x64" /Project ProjectName2
IF %ERRORLEVEL% EQU 0 (
"%VSDIR%\devenv.com" "MySolution.sln" /Build "Release|x64" /Project ProjectName3
IF %ERRORLEVEL% EQU 0 (
) ELSE (
echo "Failed to build ProjectName3"
)
) ELSE (
echo "Failed to build ProjectName2"
)
) ELSE (
echo "Failed to build ProjectName1"
)
unset VSDIR
just use MSBuild plug-in. You are all set.
I have several suggestions to make this simpler and more reliable.
You do not need to do a separate call to devenv.com in order to do a /Clean. Simply replace the /Build command in subsequent devenv.com calls with /Rebuild, which will clean before building from scratch.
After each call to devenv.com immediately do:
if ERRORLEVEL 1 echo Error building Solution X & goto some-label
This will detect any ERRORLEVEL of 1 or higher. This approach will not require any parentheses or ELSE clauses, and avoid the need to nest the subsequent IF statements.
You can also build the 3 (out of 4) projects with a single devenv.com call. This would require you to specify a solution build configuration that excludes (skips) the fourth project. For example:
"%VSDIR%\devenv.com" "MySolution.sln" /Rebuild "Release-Jenkins|x64"
The above approach requires first loading the solution in the Visual Studio IDE, going into the Configuration Manager, and creating a new solution build configuration (based on a copy of the existing Release build configuration). Name the new solution build configuration Release-Jenkins (or whatever), then make sure the project you wish to exclude is unchecked for each of the variations of Win32/x64/AnyCPU. Also, be sure to UNCHECK the checkbox to Create new project configurations, as you do not require any new project configurations. Also create a corresponding solution build configuration for Debug builds, i.e. Debug-Jenkins.
The unset command works in bash, but not for cmd.exe shells. To undefine the variable, do:
set VSDIR=
with nothing following the = sign. Or even better, add the command SETLOCAL on its own line immediately after #ECHO OFF. The SETLOCAL statement will ensure any environment variable changes are discarded when the batch file exits.
Background: I have a post-build process that copies a file to another location. It looks like this:
copy $(TargetPath) "%programfiles%\mypath"
This step can fail if the another process is using the file. The step is not critical, so if possible I would like to ignore the failure. To do this I need the script to check to determine if the file is being used by another process.
Question: Is there a way of testing a file in a DOS script to determine if it is being used by another process?
You can see if the file exists, then rename a .dll/.exe even if it is being executed. Might want to do .pdb files, too.
IF EXIST $(TargetName).deleted del $(TargetName).deleted
IF EXIST $(TargetName).pdb.deleted del $(TargetName).pdb.deleted
IF EXIST "%programfiles%\mypath\$(TargetName)$(TargetExt)" REN "%programfiles%\mypath\$(TargetName)$(TargetExt)" $(TargetName).deleted
IF EXIST "%programfiles%\mypath\$(TargetName)$(TargetExt)" REN "%programfiles%\mypath\$(TargetName)$.pdb" $(TargetName).pdb.deleted
copy $(TargetPath) "%programfiles%\mypath"
Ok, so I needed to check the errorlevel after performing the copy, so that I could handle the exit properly. The solution is below:
copy $(TargetPath) "%programfiles%\mypath"
if errorlevel 1 goto BuildProcessFailed
goto BuildProcessOK
:BuildProcessFailed
echo BUILDPROCESS FAILED FOR PROJECT $(ProjectName)
goto ExitBuildProcess
:BuildProcessOK
echo BUILDPROCESS OK FOR PROJECT $(ProjectName)
:ExitBuildProcess
I am trying to build multiple .sln files inside a batch file. Everything works great so far. I am trying to add a check inside the batch file, so if number of errors is greater than 0 then the batch file stops executing and doesn't build the next .sln files. How can I do that? Basically something like:
msbuild test.sln
(check if build error > 0
stop)
msbuild test2.sln
MSBUILD will set the ERRORLEVEL, so something along the lines of:
msbuild test.sln
IF NOT ERRORLEVEL 0 exit 1
Edit: Apparently it should be:
msbuild test.sln
IF ERRORLEVEL 1 exit 1
msbuild.exe test.sln
if errorlevel 1 goto :errors
msbuild.exe test2.sln
if errorlevel 1 goto :errors
:: ...
:: Everything was fine.
echo Build completed without errors.
goto :eof
:error
echo Build failed.
In my opinion it's much easier to use a custom msbuild file here and use the msbuild task with your set of solutions. See here for the details.