We recently updated to Visual Studio 2010, and as part of our upgrade we started using Gallio 3.2 prerelease builds. Everything runs fine in Visual Studio (through resharper) but I'm having problems with TeamCity integration. The tests seem to run during TeamCity builds just fine (our build takes long enough to run all our tests), but the tests are not showing up in TeamCity's test area. Here is the test target from our NANT build file (this hasn't changed in our upgrade at all). Is there a trick to getting the tests to show up in TeamCity or is this something that's broken in the latest builds of Gallio?
<target name="runTests">
<gallio
result-property="exitCode"
failonerror="false">
<runner-extension value="TeamCityExtension,Gallio.TeamCityIntegration" />
<assemblies>
<include name="..\Source\Tests\${testProject}\bin\Debug\${testProject}.dll" />
</assemblies>
</gallio>
</target>
Try setting echo-results="true" like this:
<target name="runTests">
<gallio
result-property="exitCode"
echo-results="true"
failonerror="false">
<runner-extension value="TeamCityExtension,Gallio.TeamCityIntegration" />
<assemblies>
<include name="..\Source\Tests\${testProject}\bin\Debug\${testProject}.dll" />
</assemblies>
</gallio>
<fail if="${exitCode != '0'}" >One or more tests failed. Please check the log for more details</fail>
</target>
Related
I am new to MSBuild and busy automating tests of Visual Studio solutions.
I previously worked with the command line of Devenv, which provides a convenient /Runexit mode of operation. From the manual:
/Runexit (devenv.exe)
Compiles and runs the specified solution, minimizes the IDE when the solution is run,
and closes the IDE after the solution has finished running.
This is exactly the functionality that I need. I am now migrating to MSBuild. I have discovered that the project files in the Solution can be directly used for building, as the default target is precisely Build.
What can I do to handle a different target, that will have the same effect as /Runexit ? Can you help me through the maze ?
This is the most basic Target which runs a projects' output file:
<Target Name="RunTarget">
<Exec Command="$(TargetPath)" />
</Target>
For c++ unittests I use something like this; it's a property sheet so it's easy to add to any project without needing to manually modify it. It automatcially runs the output after the build so there is no need to specify an extra target and it works the same for VS and from the command line. Moreover in VS you'll get unittest errors from frameworks like Unittest++ or Catch displayed right away in the error list, so you can doubleclick them. Also the UnitTestExtraPath property can be set elsewhere just in case (e.g. on a buildserver we always want to keep the PATH clean but sometimes we do need to modify it to run built exes).
<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<ImportGroup Label="PropertySheets" />
<PropertyGroup Label="UserMacros" />
<PropertyGroup />
<ItemDefinitionGroup />
<ItemGroup />
<!--Used to be AfterTargets="AfterBuild", but that is unusable since a failing test marks the build as unsuccessful,
but in a way that VS will always try to build again. As a consequence debugging in VS is impossible since
VS will build the project before starting the debugger but building fails time and time again.-->
<Target Name="RunUnitTests" AfterTargets="FinalizeBuildStatus">
<Exec Condition="$(UnitTestExtraPath)!=''" Command="(set PATH="%PATH%";$(UnitTestExtraPath)) & $(TargetPath)" />
<Exec Condition="$(UnitTestExtraPath)==''" Command="$(TargetPath)" />
</Target>
</Project>
I'm having some trouble getting a rake build script to work locally (it runs fine on our CI server), because it for some reason uses an incorrect version of msbuild. I can also build it fine from inside Visual Studio 2013.
The following existed in my web project's .csproj file when I started working on this:
<PropertyGroup>
<VisualStudioVersion Condition="'$(VisualStudioVersion)' == ''">10.0</VisualStudioVersion>
<VSToolsPath Condition="'$(VSToolsPath)'==''">$(MSBuildExtensionsPath32)\Microsoft\VisualStudio\v$(VisualStudioVersion)</VSToolsPath>
</PropertyGroup>
<!--<Import Project="$(VSToolsPath)\WebApplications\Microsoft.WebApplication.targets" Condition="'$(VSToolsPath)' != ''" />-->
<!-- Above commented out because it fails - keep reading for why... -->
I added the following target for debugging purposes:
<Target Name="ShowVersions">
<Message Text="VS version: $(VisualStudioVersion)" />
<Message Text="VSToolsPath: $(VSToolsPath)" />
</Target>
and the accompanying rake task:
msbuild :debug_versions do |msb|
msb.solution = "src/NCVIB.WebMVC/NCVIB.WebMVC.csproj"
msb.targets :ShowVersions
end
Now, if I run the ShowVersions target from my terminal directly using MSBuild MyWebProject.csproj /t:ShowVersions it correctly identifies my VS version as 12.0. If I run rake debug_versions, it says the VS version is 11.0, and consequentially the import of WebApplications.targets fails (the path doesn't exist).
How do I make the rake task pick up the same VS version as plain msbuild does?
I am using TFS 2010 and Visual Studio 2010. I have a build definition which points to my solution. The build runs overnight
I have a set of test case files(*.tst) and i would like my current build to include these as part of the build and to execute them overnight. The test case files are in source control
I read that i have to use MsTest.exe but unsure how to get started?
Can anyone point me please how i can get started on running the test case files as part of the build? Any examples please?
Thanks in advance,
There are several ways you can have your test cases run. One way is to add an invoke process or Exec Command to your build project file or workflow.
Place the call to mstest in the AfterBuild target or workflow step. The other way would be to separate out the build and test cases into different builds.
Here are a couple of links to get you started:
How to: Configure and Run Scheduled Tests After Building Your Application
Example of MSBuild with MSTest
*Based on your comments here is an update of something you can do to get more information about the error or to continue if you encounter an error. You do not need to check for the error code if you just want to stop on any error but if you do want to check the error code then you would do something like this:
<Target Name="AfterBuild">
<Message Text="Running tests..." />
<PropertyGroup Label="TestSuccessOrNot">
<TestSuccessOrNot>5</TestSuccessOrNot>
</PropertyGroup>
<!-- Run MSTest exe-->
<Exec Command="cd ." IgnoreExitCode="False" ContinueOnError="ErrorAndContinue">
<Output TaskParameter="ExitCode" PropertyName="TestSuccessOrNot" />
</Exec>
<Message Text="ExitCode = $(TestSuccessOrNot)" />
<Error Condition="'$(TestSuccessOrNot)' != '0'" Text="Unit tests fail!" /> </Target>
My company already has a Team Foundation Server as a Continuous Integration platform. However, what I am looking to setup is a build configuration that a developer can run on their own development machine.
Let's say I have a Visual Studio solution that contains a .NET C# Class Library project (I'll call this the Library Project). It also contains another project containing the Unit Testing classes for Library Project (I'll call this the Testing Project).
I have the normal Debug and Release build configurations for each project and at the solution level. For both of these configurations, I have set it to only build the Library Project (so Testing Project does not get built).
What I would like to do is set up 2 new build configurations called Debug With Testing and Release With Testing. They will each be the same as the Debug and Release, respectively but I need them to have the following extra features:
Builds the Testing Project.
Run all test cases in the Testing Project.
Run Code Analysis on Library Project.
Generate report for testing and code analysis.
Save report in a specific location.
Doing item 1 is easy. However, I can't figure out how to do items 2 to 5. Can anyone point me in the right direction?
Any help will be greatly appreciated. TIA
You will need to write custom MS build code, I already do some similar task as the following:
Get the latest change from TFS
Build the solution including all projects
Deploy the Main Database locally
Deploy the Test Database locally which hold the test data used in the
data driven test
Run the sanity test or BVT (Build Verification Test) which has
belong to category 1 (Test the integration between DB and code)
Check-in the pending change
And hear the code of this tasks
<Target Name="GetLatestFromTFS2010" AfterTargets="build" >
<Message Importance="high" Text ="start GetLatest for the project "></Message>
<Exec Command='"C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\TF.exe" get $/AutoDBand/AutomateDatabaseAndTest/AutomateDatabaseAndTest /recursive /login:YourUsername,YourPassword' ContinueOnError='false'/>
</Target>
<!--===========Deploy Database============-->
<Target Name="DeployDatabase" AfterTargets="GetLatestFromTFS2010" Condition="'$(Configuration)' == 'DebugForCheck-in'">
<Message Importance="high" Text="-------------------------------- Deploying Database according to the connection string -------------------------------- " />
<Message Importance="high" Text=" "/>
<MSBuild Projects="..\DB\DB.dbproj" Targets="Build;Deploy" />
</Target>
<!--============Run the Test==================-->
<Target Name="UnitTests" AfterTargets="DeployDatabase" Condition="'$(Configuration)' == 'DebugForCheck-in'">
<Message Importance="high" Text="-------------------------------- Running Unit Tests for category 1 only--------------------------------" />
<Message Importance="high" Text=" "/>
<Exec Command='"C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\mstest.exe" /testcontainer:"..\BLTest\bin\Debug\BLTest.dll" /category:cat1' />
</Target>
<Target Name="Chekin-pendingChange" AfterTargets="UnitTests" >
<Message Importance="high" Text ="-------------------------------- start Check-in process-------------------------------- "></Message>
<Message Importance="high" Text=" "/>
<Exec Command='"C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\TF.exe" checkin $/AutoDBand/AutomateDatabaseAndTest/AutomateDatabaseAndTest /recursive /login:YourUsername,YourPassword' ContinueOnError='false'/>
</Target>
For more information you can see this article with source code
http://mohamedradwan.wordpress.com/2010/11/13/automate-the-best-practice-for-check-in-including-get-latest-deploy-db-run-test-check-in/
Have a look at something like:
TeamCity
Jenkins
Team Foundation Server
all are Continous Integration Servers, which are good in doing the jobs you like to have done.
I am trying to use Gallio (v3.1)/MbUnit/NCover to run a unit test in my C# code, as part of the build process for my continuous integration system.
I can get Gallio.Echo.exe to execute the tests and output an XML file (albeit it does seem to be checking all .dll files in the folder == approx. 6.5MB .xml file!!), but when I try to get NCover to link in also, it goes bang.
THEN: I tried to use the NAnt task using instructions from here, such as:
<gallio result-property="testrunner.exit-code"
application-base-directory="bin/debug"
runner-type="NCover"
failonerror="false"
report-name-format="gallio-MyTestProject"
report-types="xml"
report-directory="bin/debug">
<runner-property value="NCoverArguments='//q //ea CoverageExcludeAttribute //a MyTestProject.dll'" />
<runner-property value="NCoverCoverageFile='coverage-MyTestProject.xml'" />
<assemblies>
<include name="bin/debug" />
</assemblies>
</gallio>
but I get the following error on my command-line:
Element Required! There must be a least one 'files' element for <gallio ... />.
I have tried to specify the .dll file that I'd like to check, but it still comes up with this message. Any suggestions are most appreciated!
<assemblies> has been changed to <files>