I followed sonarqube doc on how to import a xunit test execution report. I don't have any errors and the script runs fine. However, I don't see any unit tests within sonarQube.
You're likely facing a regression. The workaround is
simply adding /d:sonar.cs.vscoveragexml.reportsPaths="**\*.coveragexml" into the Additional Settings of the SonarQube Scanner for MSBuild - Begin Analysis (new) task.
Related
Is it possible to analyze C# project using Sonar-runner with sonar-c#-plugin-4.2. I saw the MSBuild SonarQube Runner to be used for C# analysis on the documentation.
Does sonar-runner support C# analysis on sonarqube 5 with C# 4.2
The use of the sonar-runner is no longer supported to analyze C# projects. Indeed, analysis through the MSBuild SonarQube Runner will be of much higher quality because it fully integrates with MSBuild and knows everything about your project, and at the same time also much easier to setup, again because most manual configuration required before is now obtained through MSBuild itself.
At least I managed to run the analysis with the sonar-runner. I consider the new begin-build-end workflow with MSBuild SonarQube Runner a bit complicated while using Jenkins and having plenty of existing scripts in play at least. Or I could not get it working due to slightly TFS-oriented documentation with Visual Studio command prompt being also involved.
I finally, after intensive googling, got the analysis running with my Visual Studio Pro + Jenkins + OpenCover + Mstest + Nunit combination by defining below properties in Invoke Standalone SonarCube Analysis in Jenkins job. I still do not know how to separate integration tests from unit tests. It does not say anywhere is this even supported for C# projects.
Workflow is that I first build everything, then run OpenCover to collect results with coverage and then invoke the runner. The below properties can of course be as file in the root folder of the project as well, but in my initial proof of concept I set these in Jenkins plugin.
sonar.projectName=projectname
sonar.projectKey=key
sonar.ProjectVersion=1.1
sonar.visualstudio.enable=true
sonar.language=cs
sonar.sources=.
sonar.dotnet.visualstudio.solution.file=code.sln
#not sure if the line below is correct or relevant...
sonar.dotnet.visualstudio.testProjectPattern=*Test
#two frameworks producing tests.
sonar.cs.vstest.reportsPaths=relativepath/results.trx
sonar.cs.nunit.reportsPaths=relativepath/testresults.xml
sonar.cs.opencover.reportsPaths=1.xml,2.xml
Here are some of the steps i followed:-
I am using SonarQube 4.3 with C# 3.2
1) I run a command on one of our solutions to produce the Coverage XML. This coverage XML is produced by running Coverage.exe (Visual Studio Coverage tool) on the test dll.
2) Then, I run sonar with the command to import this coverage XML into sonar as per new rules.
3) At the end,Sonar Analysis is successful but coverage never comes up.
Can someone tell me why the coverage doesn't come up? The new Sonar wants Coverage XML to be produced independently and then to be imported into the dashboard. C# 3.2 plugin supports this type of import and coverage XML.
I'm attempting to run dotCover on an xunit unit test. No matter which test I attempt to cover, they all give me the same result:
Inconclusive: Test wasn't run
So no matter what, dotCover doesn't find the coverage for any of my tests.
I am using ReSharper's unit test runner, and I am able to run the unit tests within VS inside the Unit Test Sessions window, so all of my unit tests run successfully.
When I attempt to cover a unit test (I've tried using the icon to the left of the test, the test all button in the Unit Test Sessions window, and right-clicking a unit test and saying Cover Unit Tests from within the same window), I always get the same message: Inconclusive: Test wasn't run.
My setup:
Visual Studio 2010
ReSharper 7.1
dotCover 2.2
xunit 1.9.1.1600
I've also set up both (http://xunitcontrib.codeplex.com/)
xunitcontrib-resharper 0.7 (RS 7.1, 6.1.1)
xunitcontrib-dotcover 0.7 (dotCover 2.0 - 2.2)
so that I should be able to use the xunit test runner from within VS.
Any help in getting the code coverage to work would be greatly appreciated.
"Inconclusive Test" means that test execution was scheduled in task runner, but didn't get any response from it.
It can be a problem with xUnitContrib itself. Can I ask you for dotCover logs to check if dotCover was actually called from the UTR with xUnit plugin?
I'm trying to run MSpec tests in Visual Studio's test system using Gallio, but it appears to be ignoring the MSpec plugin from the IDE. What I did:
Installed Gallio
Added the MSpec NuGet package to my test project
Added the Gallio NuGet package to my test project
Copied the three files required for the MSpec Gallio integration to an "MSpec" folder in Gallio's installation directory
Wrote an MSpec test
Ran "All Tests in Solution" in the IDE
This yielded the following error
"No tests were run because no tests are loaded or the selected tests are disabled"
To check Gallio itself, I added a reference to MbUnit and created a simple MbUnit test. I also ran Gallio's other tools to check if the plugin was being picked up at all. I even restarted VS for good measure.
Results
Running the tests using Gallio.Echo or Gallio.Icarus successfully executes both MbUnit and MSpec tests, so the plugin is being picked up. Running the tests from Visual Studio runs the MbUnit test but not the MSpec test, so Gallio integration appears to be working (just not the MSpec plugin). Any thoughts?
Before anyone mentions it: yes, I'd rather be using ReSharper but licenses are not an option for my situation at the moment.
Update: The is now a project to integrate MSpec into VS2012's test system. It's available on the Visual Studio Gallery and the source is on GitHub.
After starting to debug the problem with both project's sources, it appears there are two underlying issues causing the incompatibility between MSpec and Gallio's VS runner. However, both issues stem from the fact that the VS runner uses Cecil to load type information from assemblies (presumably to avoid loading the test assembly into the main VS AppDomain).
I'll update this answer as I uncover more (hopefully resulting in one or more patch to fix the issues).
Issue 1: Nested types are ignored
I had been using nested types for my tests (SubjectSpec+when_something), which are ignored by the Cecil-based reflection used by Gallio's VS runner to avoid loading the test assembly into the app domain.
This ended up being a relatively simple fix, which I've submitted as a patch to the Gallio developers. I'll update this answer if/when I get confirmation of which release it will be part of.
Issue 2: MSpec causes "An exception occurred while invoking a test driver"
If the container type is removed to avoid issue 1, MSpec throws this error. The reason for this is that MSpec is trying to create an instance of the test object, but Gallio is only supplying it a wrapper Type object that can't actually be created.
I'm trying to find a way to migrate from mstest to xunit and still benefit from the IDE integration which made me choose mstest in the first place. Gallio seems to accomplish this noble goal, plus it's free (don't want Reshaper or TestDriven.net). But I can't get code coverage to work.
My solution contains of two projects: project SUT (the assembly I need to test) and project SUT.Tests which is a VS test project (this allows Gallio to display xunit tests in VS's Test View). I have code coverage enabled in Local.testSettings for SUT.dll and instrumentation is in place. After the test run completes there is no code coverage. Code Coverage Results window reports: Empty results generated: none of the instrumented binary was used. Look at test run details for any instrumentation problems. Unfortunately the test run details do not contain any "instrumentation problems". I tried unchecking the instrument assemblies in place checkbox and re-running the unit test; same result.
Any idea what's wrong?
My setup:
- Windows 7 x64
- VS 2010 Premium (SP1)
- xUnit 1.8
- Gallio 3.3.1 x64 (installer, not zip)
Apologies for answering my own question:
Turns out Gallio loads SUT.dll from SUT.Tests\Bin\Debug rather than SUT\Bin\Debug. I added this path to code coverage details, selected it instead of the default SUT.dll path and now it works! This is better described at here, towards the end of the post.