I am trying to use Gallio (v3.1)/MbUnit/NCover to run a unit test in my C# code, as part of the build process for my continuous integration system.
I can get Gallio.Echo.exe to execute the tests and output an XML file (albeit it does seem to be checking all .dll files in the folder == approx. 6.5MB .xml file!!), but when I try to get NCover to link in also, it goes bang.
THEN: I tried to use the NAnt task using instructions from here, such as:
<gallio result-property="testrunner.exit-code"
application-base-directory="bin/debug"
runner-type="NCover"
failonerror="false"
report-name-format="gallio-MyTestProject"
report-types="xml"
report-directory="bin/debug">
<runner-property value="NCoverArguments='//q //ea CoverageExcludeAttribute //a MyTestProject.dll'" />
<runner-property value="NCoverCoverageFile='coverage-MyTestProject.xml'" />
<assemblies>
<include name="bin/debug" />
</assemblies>
</gallio>
but I get the following error on my command-line:
Element Required! There must be a least one 'files' element for <gallio ... />.
I have tried to specify the .dll file that I'd like to check, but it still comes up with this message. Any suggestions are most appreciated!
<assemblies> has been changed to <files>
Related
VS 2017 sets LastActiveSolutionConfig for a web application on its project load, thus triggering subsequent build of that project, because:
The property is set in the respective .csproj.user file, which is created, if needed.
The .csproj.user file is part of project dependencies
So by creating it, VS causes the project to be built the next time.
Imagine building it all on the command line with msbuild after cleaning up the workspace, then switching back to VS and hitting the build button. And it is building again!
So, there are these stupid auto generated CS files related to workflows, that are generated only by VS, not msbuild (TemporaryGeneratedFile_036C0B5B-1481-4323-8D20-8F5ADCB23D92.cs and friends) - our msbuild generates them on purpose to satisfy VS. Check.
Next we ensure all the Copy to Output Directory files use PreserveNewest - check.
I already forgot what else we had to do to make sure VS does not rebuild code redundantly when switching from msbuild to IDE. Now this one, which is new.
How can I prevent VS 2017 from adding this property? Is it absolutely necessary to have it?
In the mean-time, I will modify the .csproj files by adding it and see if it helps. Nobody builds Release locally at our place and it is always AnyCPU platform, so I do not care about other configurations, except Debug|AnyCPU.
This is what I would do:
Set the build verbosity to Diagnostic.
Build and look for where the msbuild file is located that generates this file:
TemporaryGeneratedFile_036C0B5B-1481-4323-8D20-8F5ADCB23D92.cs
Look for some conditions that you can alter to prevent the file from being
generated.
Set some property to alter the condition and prevent that file from being generated.
My solution is to generate the .csproj.user files if needed with the expected property. Which is incredibly annoying that one has to do it. Here is the build code that can go into your Directory.Build.Targets:
<Target Name="EnsureCSProjUserForWebApplications"
Condition="'$(IsWebApplication)' == True And !Exists('$(MSBuildProjectFullPath).user')">
<ItemGroup>
<CSProjUserContent Include="<?xml version="1.0" encoding="utf-8"?>" />
<CSProjUserContent Include="<Project ToolsVersion="15.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">" />
<CSProjUserContent Include="<PropertyGroup>" />
<CSProjUserContent Include="<LastActiveSolutionConfig>Debug|Any CPU</LastActiveSolutionConfig>" />
<CSProjUserContent Include="</PropertyGroup>" />
<CSProjUserContent Include="</Project>" />
</ItemGroup>
<WriteLinesToFile File="$(MSBuildProjectFullPath).user" Lines="#(CSProjUserContent)" ContinueOnError="true" />
</Target>
IsWebApplication is computed like this:
<IsWebApplication>$(ProjectTypeGuids.Contains('349c5851-65df-11da-9384-00065b846f21'))</IsWebApplication>
Finally, the target is part of a larger series of targets that run at the beginning and validate the csproj matches our requirements or ensure certain conditions:
<PropertyGroup>
<EnsureXyzProjectSettingsDependsOn>
AssertIISExpress;
AssertNoAssemblyInfo;
AssertDebugSymbols;
AssertLocalApplicationHostFile;
AssertImportsDFVersioning;
EnsureSharedBinLink;
AssertSharedBinOutputPath;
AssertHintPaths;
EnsureCSProjUserForWebApplications
</EnsureXyzProjectSettingsDependsOn>
</PropertyGroup>
<Target Name="EnsureXyzProjectSettings"
DependsOnTargets="$(EnsureXyzProjectSettingsDependsOn)"
BeforeTargets="BeforeBuild"
Condition="'$(SuppressStrictXyzTargetsChecks)' != true" />
Has anyone had any success with running StyleCop from TeamCity?
I know StyleCop supports a command line mode, however i am not sure how this will integrate into the report output by TeamCity.
I've checked out this plugin found here: https://bitbucket.org/metaman/teamcitydotnetcontrib/src/753712db5df7/stylecop/
However could not get it running.
I am using TeamCity 6.5.1 (latest).
I don't know how familiar you are with MSBuild, but you should be able to add a new Build Step in TC 6 and above, and set MSBuild as the build runner, and point it to a .proj file which does something similar to the following:
<Target Name="StyleCop">
<!-- Create a collection of files to scan -->
<CreateItem Include="$(SourceFolder)\**\*.cs">
<Output TaskParameter="Include" ItemName="StyleCopFiles" />
</CreateItem>
<StyleCopTask
ProjectFullPath="$(MSBuildProjectFile)"
SourceFiles="#(StyleCopFiles)"
ForceFullAnalysis="true"
TreatErrorsAsWarnings="true"
OutputFile="StyleCopReport.xml"
CacheResults="true" />
<Xslt Inputs="StyleCopReport.xml"
RootTag="StyleCopViolations"
Xsl="tools\StyleCop\StyleCopReport.xsl"
Output="StyleCopReport.html" />
<XmlRead XPath="count(//Violation)" XmlFileName="StyleCopReport.xml">
<Output TaskParameter="Value" PropertyName="StyleCopViolations" />
</XmlRead>
<Error Condition="$(StyleCopViolations) > 0" Text="StyleCop found $(StyleCopViolations) broken rules!" />
</Target>
If you don't want to fail the build on a StyleCop error, then set the Error task to be Warning instead.
You'll also need to add the following to your .proj file:
<UsingTask TaskName="StyleCopTask" AssemblyFile="$(StyleCopTasksPath)\Microsoft.StyleCop.dll" />
Microsoft.StyleCop.dll is included in the StyleCop installation, and you'll need to set your paths appropriately.
To see the outputted StyleCop results in TeamCity, you will need to transform the .xml StyleCop report to HTML using an appropriate .xsl file (called StyleCopReport.xsl in the script above).
To display the HTML file in TeamCity, you'll need to create an artifact from this .html output, and then include that artifact in the build results.
The Continuous Integration in .NET book is a great resource.
Did you know that teamcity provides specific properties just from msbuild?
No need for the service messages, see:
http://confluence.jetbrains.net/display/TCD65/MSBuild+Service+Tasks
So you dont have to add a custom report page.
Use the build stats e.g.
<TeamCitySetStatus Status="$(AllPassed)" Text="Violations: $(StyleCopViolations)" />
you can then log the statistic too:
<TeamCityReportStatsValue Key="StyleCopViolations" Value="$(StyleCopViolations)" />
And then create a custom graph to display, and you already have the violations in your msbuild output.
edit main-config.xml and add:
<graph title="Style Violations" seriesTitle="Warning">
<valueType key="StyleCopViolations" title="Violations" buildTypeId="bt20"/>
</graph>
Where buildTypeId="bt20" bt20 is your style build.
I'm late to the show here but a very easy way to achieve this is to install the StyleCop.MSBuild NuGet package in any project which you want to analyse with StyleCop.
After installing the package, StyleCop analysis will run on every build you do, regardless of where or how it is invoked, e.g VS, command line, msbuild, psake, rake, fake, bake, nant, build server, etc. No special actions are required.
If you want the build to fail when StyleCop rules are broken you just need to add the following element to your project file under each appropriate build configuration, E.g.
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
<StyleCopTreatErrorsAsWarnings>false</StyleCopTreatErrorsAsWarnings>
...
Again, this will work on every build, regardless of where and how it is invoked.
There's a (new?) third-party TeamCity plugin for StyleCop here, (though I haven't tried it yet).
UPDATE: as far as I can tell, the latest version only works with TeamCity 7 (or I did something wrong). Also, I have a very slow (virtual) build server, so even after the services were restarted, it took a while for the StyleCop runner to appear in the web interface.
Another stupid thing I did was not read the readme properly: you have to unzip the downloaded zip, and use the zip inside.
I also originally started with just a list of .cs files in the "Include" option (for the build step), but that didn't work; links to sln or csproj files do work though.
We recently updated to Visual Studio 2010, and as part of our upgrade we started using Gallio 3.2 prerelease builds. Everything runs fine in Visual Studio (through resharper) but I'm having problems with TeamCity integration. The tests seem to run during TeamCity builds just fine (our build takes long enough to run all our tests), but the tests are not showing up in TeamCity's test area. Here is the test target from our NANT build file (this hasn't changed in our upgrade at all). Is there a trick to getting the tests to show up in TeamCity or is this something that's broken in the latest builds of Gallio?
<target name="runTests">
<gallio
result-property="exitCode"
failonerror="false">
<runner-extension value="TeamCityExtension,Gallio.TeamCityIntegration" />
<assemblies>
<include name="..\Source\Tests\${testProject}\bin\Debug\${testProject}.dll" />
</assemblies>
</gallio>
</target>
Try setting echo-results="true" like this:
<target name="runTests">
<gallio
result-property="exitCode"
echo-results="true"
failonerror="false">
<runner-extension value="TeamCityExtension,Gallio.TeamCityIntegration" />
<assemblies>
<include name="..\Source\Tests\${testProject}\bin\Debug\${testProject}.dll" />
</assemblies>
</gallio>
<fail if="${exitCode != '0'}" >One or more tests failed. Please check the log for more details</fail>
</target>
I need to know how to tell MSTEST to run all test projects in a solution file. This needs to be done from the command line. Right now I have to pass it a specific project file, I'm trying to get it to run from a SOLUTION file.
I'm hoping this is possible, because in Visual Studio, hitting Ctrl+R, A, runs ALL tests in the currently opened solution.
The way I've interpretted the help files, you have to pass in each DLL specifically.
I want to run this from the command line from my CruiseControl.NET server, so I can write other utilities to make this happen. If there is a wierd way of getting this to happen through some OTHER method, let me know.
How do I tell MSTEST to run all test projects for a solution?
<exec>
<!--MSTEST seems to want me to specify the projects to test -->
<!--I should be able to tell it a SOLUTION to test!-->
<executable>mstest.exe</executable>
<baseDirectory>C:\projects\mysolution\</baseDirectory>
<buildArgs>/testcontainer:testproject1\bin\release\TestProject1.dll
/runconfig:localtestrun.Testrunconfig
/resultsfile:C:\Results\testproject1.results.trx</buildArgs>
<buildTimeoutSeconds>600</buildTimeoutSeconds>
</exec>
To elaborate on VladV's answer and make things a bit more concrete, following the suggested naming convention running your tests can be easily be automated with MSBuild. The following snippet from the msbuild file of my current project does exactly what you asked.
<Target Name="GetTestAssemblies">
<CreateItem
Include="$(WorkingDir)\unittest\**\bin\$(Configuration)\**\*Test*.dll"
AdditionalMetadata="TestContainerPrefix=/testcontainer:">
<Output
TaskParameter="Include"
ItemName="TestAssemblies"/>
</CreateItem>
</Target>
<!-- Unit Test -->
<Target Name="Test" DependsOnTargets="GetTestAssemblies">
<Message Text="Normal Test"/>
<Exec
WorkingDirectory="$(WorkingDir)\unittest"
Command="MsTest.exe #(TestAssemblies->'%(TestContainerPrefix)%(FullPath)',' ') /noisolation /resultsfile:$(MSTestResultsFile)"/>
<Message Text="Normal Test Done"/>
</Target>
Furthermore integrating MsBuild with CruiseControl is a piece of cake.
Edit
Here's how you can 'call' msbuild from your ccnet.config.
First if you do not already use MSBuild for your build automation add the following xml around the snippet presented earlier:
<Project DefaultTargets="Build"
xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
..... <insert snippet here> .....
</Project>
Save this in e.g. RunTests.proj next to your solution in your source tree. Now you can modify the bit of ccnet.config above to the following:
<msbuild>
<executable>C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\MSBuild.exe</executable>
<workingDirectory>C:\projects\mysolution\</workingDirectory>
<baseDirectory>C:\projects\mysolution\</baseDirectory>
<projectFile>RunTests.proj</projectFile>
<targets>Test</targets>
<timeout>600</timeout>
<logger>C:\Program Files\CruiseControl.NET\server\ThoughtWorks.CruiseControl.MsBuild.dll</logger>
</msbuild>
This is an old thread, but I have been struggling with the same issue and I realized that you can really just run MSTest on every dll in the whole solution and it doesn't really cause any problems. MSTest is looking for methods in the assemblies marked with the [TestMethod] attribute, and assemblies that aren't "test" assemblies just won't have any methods decorated with that attribute. So you get a "No tests to execute." message back and no harm done.
So for example in NAnt you can do this:
<target name="default">
<foreach item="File" property="filename">
<in>
<items>
<include name="**\bin\Release\*.dll" />
</items>
</in>
<do>
<echo message="${filename}" />
<exec program="C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\MSTest.exe">
<arg value="/testcontainer: ${filename}" />
<arg value="/nologo" />
</exec>
</do>
</foreach>
</target>
and it will run all the test methods in every dll in every bin\Release folder in the solution. Those which are not test dlls will return a "No tests to execute." and those that have tests will have the tests run. The only part I haven't figured out yet is that (in NAnt) execution stops the first time a command returns a non-zero value. So if any unit tests fail it doesn't keep going to execute any tests in subsequent assemblies. That is not great, but if all the tests pass, then they will all run.
I just resolve this problem recently. Here is my proposal: Use testmetadata + testlist option of mstest
First you should create a testlist in testmetadata file(vsmdi)
the commandline should be mstest /testmetadata:....vsmdi /testlist:<name>
Then use ccnet config to run mstest
i know this thread is quite Old, but its still high on Google so i thought i might help one or two.
Anyway, since there is no satisfactory solution for this.
I've written an msbuild task for this.
details can be found here:
http://imistaken.blogspot.com/2010/08/running-all-tests-in-solution.html
You could enforce some convention on the naming and location of test projects, then you could run MSTest on, say, all *Test.dll below the location of your solution.
As far as I know, there is no way to tell a test project from a 'normal' DLL project based soleley on a solution file. So, an alternative could be to analyze the project files and/or .vsmdi files to find the test projects, but that could be rather tricky.
I don't know directly but this is where VSMDI [fx:spits in a corner] can help. In your solution add all the tests to the VSMDI. And then pass the VSMDI to mstest using /testmetadata.
However I would suggest that you follow the conventions above. And use a naming convention and dump that out from the SLN file using say a for loop in the command script
I would just write a target that calls it the way you want, then whip up a batch file that calls the target that contains all the DLL's to be tested.
Unless you're adding test projects all the time, you'll very rarely need to modify it.
Why not just have msbuild output all the test assemblies to a folder.
Try setting OutputPath,OutputDir,OutDir properties in msbuild to accomplish this.
then have mstest execute against all assemblies in that folder.
I've added a pre-build action for an ASP.NET web control (server control) project, that runs jsmin.exe on a set of Javascript files. These output files are part of the source control tree and are embedded into the assembly.
The problem is when the pre-build runs, jsmin can't write the file as it's readonly. Is it possible to check the file out before hand? Or am I forced to set the file's attributes in the command line.
Any improved solution to the problem is welcome.
Update
One small issue with Mehmet's answer -you need to prepend the VS directory:
"$(DevEnvDir)tf" checkout /lock:none "$(ProjectDir)myfile"
If you're using Team Foundation Server, you can use team foundation command line utility (tf.exe) to check out the file(s) during pre-build and then check them back in during post-build. If you're using something else for source control, you can check if they have a command line tool like tf.exe.
If you do not want to check the files in as part of the build (which you normally wouldn't for this sort of thing) then I would simply set the attributes of the .js files before running jsmin on them. The easiest way of setting the files read-writeable is to use the the Attrib task provided by the MSBuild community extensions. The same community extensions also provide a JSCompress task for easily calling JSMin from MSBuild.
Therefore you'd have something like the following (not tested):
<Import Project="$(MSBuildExtensionsPath)\MSBuildCommunityTasks\MSBuild.Community.Tasks.Targets" />
<!-- rest of TFSBuild.proj file -->
<Target Name="AfterGet">
<Message Text="Compressing Javascript files under "$(SolutionRoot)"." />
<CreateItem Include="$(SolutionRoot)\**\*.js">
<Output TaskParameter="Include" ItemName="JsFiles"/>
</CreateItem>
<Attrib Files="#(JsFiles)" ReadOnly="false"/>
<JSCompress Files="#(JsFiles)" />
</Target>
Note that by modifying the files after getting them may well cause issues if you tried to move to an incremental build.