I'm working with sonar having Gallio using OpenCover, it gives me a code coverage of 0% although there are test classes and my machine takes about 15 seconds to try to run the tests (at least it seems like that), after which they're summed up with the line 0 run, 0 passed, 0 failed, 0 inconclusive, 0 skipped.
So, I took a look around the internet / stackoverflow and found out that this could have to do with the code being compiled on another machine. As that's the case here as well, I opened the solution in Visual Studio, built it and I can see that my PDB files are recreated. After which I tried sonar with opencover again.
It didn't help
I took the line of code with which OpenCover is called out of the admin console, went to the OpenCover directory and executed the same line, without a filter as my coverage-report.xml told me that the test modules were skippedDueTo="Filter". After which they were not skipped anymore.
Still, 0% test coverage
I registered OpenCover profiler library with regsvr32 OpenCover.Profiler.dll in the same Admin console (and tried it twice, as I'm on windows 7 but am using a 32 bit sonar).
Didn't help
I tried the same line with adding the -oldStyle argument
Didn't help either
Now I feel like I'm out of options ...
The latest call I tried in the admin console:
C:\Program Files\OpenCover>OpenCover.Console.exe -register:user -target:"C:\Program Files\Gallio\bin\Gallio.Echo.exe" -targetdir:D:\projecten\udbdrm\source\CDP\CDPBackoffice\Develop\CDPBackoffice.root\CDPBackoffice\.sonar "-targetargs:\"/r:IsolatedAppDomain\" \"/report-directory:D:\projecten\udbdrm\source\CDP\CDPBackoffice\Develop\CDPBackoffice.root\CDPBackoffice\.sonar\" \"/report-name-format:gallio-report\" \"/report-type:Xml\" \"D:\projecten\udbdrm\source\CDP\CDPBackoffice\Develop\CDPBackoffice.root\CDPBackoffice\CDPBackoffice.BusinessLogic.Tests\bin\Debug\CDPBackoffice.BusinessLogic.Tests.dll\" \"D:\projecten\udbdrm\source\CDP\CDPBackoffice\Develop\CDPBackoffice.root\CDPBackoffice\CDPBackoffice.Utility.Tests\bin\Debug\CDPBackoffice.Utility.Tests.dll\" \"D:\projecten\udbdrm\source\CDP\CDPBackoffice\Develop\CDPBackoffice.root\CDPBackoffice\CDPBackoffice.DataAccessLayer.Tests\bin\Debug\CDPBackoffice.DataAccessLayer.Tests.dll\"" -mergebyhash -output:D:\projecten\udbdrm\source\CDP\CDPBackoffice\Develop\CDPBackoffice.root\CDPBackoffice\.sonar\coverage-report.xml -oldStyle
The solution does have several test classes, which run when executing them in Visual Studio and they do call source classes. Some of them fail some of them succeed.
Anyone who still has a clue on how to get test results?
Preferrably in sonar, but hey, as executing OpenCover with the call above puts the results in the coverage-report.xml and sonar uses that as input I'm fine with a working call ...
Apparently all I needed to add to the sonar-project.properties was this line:
sonar.gallio.runner=IsolatedProcess
Related
I recently added xxx.runsettings file to my solution to pass parameters from TFS variables to my solution(url). Now I'm not able to generate .trx result file after running TFS build in Test summary/results page, only code coverage is generated.
[.runsettings code taken from here-https://msdn.microsoft.com/en-us/library/jj635153.aspx
Can anyone here tell me how to edit the runsettings file in order to show .trx result file in my test summary instead of code coverage?
TFS 2015 Update 3,TFS ms build
Can see this window once build is completed.
Run functional test Logs
Publish results log -
Logs
2017-06-21T17:20:49.9138829Z Executing the powershell script: C:\agent\tasks\PublishTestResults\1.0.22\PublishTestResults.ps1
2017-06-21T17:20:50.0628925Z ##[warning]No test result files were found using search pattern 'C:\agent_work\2\s**\TestResults\xyz*.trx'.
Just remove <ResultsDirectory>.\TestResults</ResultsDirectory> section from the runsettings file should fix your issue.
By the way, Publish Test Result task does not work in your scenario since it can only publish the test result files on the build agent while the trx file is usually generated on the test agent with Run Functional Test task.
Since you are running the VStest. Which will not generate the .trx result file.
VStest step is actually using VSTest.Console.exe command, it will use the /logger:TfsPublisher which not generate the .trx file. So if you use the built-in tasks such as Visual Studio Test or Run Functional Tests to run tests, results are automatically published and you do not need a separate publish test results task.
To log results into a Visual Studio Test Results File (TRX) use /Logger:trx. More details please refer this command. To generate a .trx file is not related to runsettings file.
For TFS , the test result is automatically published, you could click test run for more details.
I'm trying to run SoapUI tests as unit tests as described here:
http://blog.simplecode.eu/post/Soap-UI-testing-with-MsTest
Everything works nice when I'm running test locally in visual studio.
But when I'm trying to run those tests during the build process on tfs2015 i get "Error: Could not create the Java Virtual Machine":
My other unit tests during that build are performed correctly.
Anyone had similar issue?
It's not a really build error in TFS. You can see the result in your screenshot, build partially succeeded.
Currently, by default the build result is "Failed" if anything failed to compile, "Partially Succeeded" if there are any unit test failures, and "Succeeded" otherwise.
Build is reported as "Partially Succeeded" when the "Test Success" property is set to "False" and "CompilationSuccess"="True"
So your error is more like something in your code fails the test or some configuration related to test in the build definition or build agent environment. Since everything works nice when you are running test locally in visual studio.
Seems disable code coverage fixed the issue.
Manage, to fix that - problem was enabled code coverage. Disabling it fixed to problem.
I’m integrating SonarQube in our build system – I installed sonar-runner-2.4 on our build agents and added the sonar-runner.properties for each solution (to the solution’s folder on TFS).
When running the build I’m executing the sonar-runner after the solution has been compiled on the build agent.
Everything seem to work except for the unit test:
On the build’s log I see that almost 200 tests ran and were completed successfully and in the sonar-runner log I see the following:
14:23:29.808 INFO - 583/583 source files analyzed
14:23:30.809 INFO - Sensor org.sonar.plugins.csharp.squid.CSharpSquidSensor#1a50b87 done: 14937 ms
14:23:30.809 INFO - Sensor org.sonar.plugins.csharp.core.CSharpUnitTestResultsProvider$CSharpUnitTestResultsImportSensor#97edbc...
14:23:30.821 INFO - Sensor org.sonar.plugins.csharp.core.CSharpUnitTestResultsProvider$CSharpUnitTestResultsImportSensor#97edbc done: 12 ms
Which seems ok I guess, but when logging into the sonar server it shows that 0 tests ran.
On the sonar-runner.properties file I set the following value to
sonar.cs.vstest.reportsPaths:
sonar.cs.vstest.reportsPaths=TestResults/*.trx
when in this case there are 3 vstest trx files located in the following local path on the build agent: `
D:\sTFS\22965\Sources\TestResults
` (see TestResults.jpg attached).
Attached is the sonar-runner.properties file.
I also attached a screen capture from the sonarqube server (see SonarServer.jpg attached).
Can you please advise what might be the problem?
You should be seeing some messages like:
INFO - Parsing the Visual Studio Test Results file ...
for each unit test result file that is being parsed, see VisualStudioTestResultsFileParser.java#L34
Can you try to pass an absolute path pattern to sonar.cs.vstest.reportsPaths? My guess is that the issue comes from the relative path.
By the way, the use of the sonar-runner to analyze .NET projects is being deprecated. You'll want to have a look at the MSBuild SonarQube Runner that offers very good integration with Team Foundation Server. See the new C# plugin documentation on SonarSource's Wiki: http://docs.sonarqube.org/display/PLUG/C%23+Plugin
EDIT
I just noticed the package name from your logs org.sonar.plugins.csharp.core.CSharpUnitTestResultsProvider. The .core. was present only in outdated versions of the C# plugin (in the 3.x series), and these versions might not support wildcards in report paths. Please upgrade to the latest version.
I have a number of unit tests written for my project, executed with CTest. I would like to integrate the results into my TeamCity build. I've downloaded and set up the plugin for my testing framework (Boost Test).
The problem that I have run into is that the tests run with CTest output to Testing/Temporary/LastTest.log, whereas TeamCity is trying to read the results from standard out. To get around this, my testing step is.
make test
cat Testing/Temporary/LastTest.log
which works, but feels like a hack.
Is there any way to get TeamCity to read from this file in addition to standard out? Alternatively, is there any way to tell ctest to output to standard out in addition to this LastTest.log file?
This question is similar, but I would like it to work for all output rather than just on failure: CMake: setting an environmental variable for ctest (or otherwise getting failed test output from ctest/make test automatically)
Teamcity has additional build features which allow to process CTest reports. I am not sure if it'll work or not but you could try adding an additional build feature in your build step to read CTest report.
I installed SonarQube and sonar-runner, and the web server seems to work just fine as I can browse it. I tried to keep things as simple as I could, I put the "sonar-project.properties" file in my project's source folder and gave . as the sonar.sources path. After doing so, the console output reported success and the web page listed my project and last analysis date, but when I went into the "Compare" tool to look at statistics it showed all items as blank, except for issues for which is displayed the number 0.
I edited the sonar-project.properties file to give the full path (instead of a relative current directory path of .) to my project's source code, replacing backslashes with forward slashes, and ran the sonar-runner command again from the command line while the working directory was the path of my source folder. It again reported success and when I reloaded the web page, it gave an updated last analysis date, but again no issues were reported and the statistics were all blank.
I have no idea what I'm doing wrong, the Sonar log only reports the installation and starting of the Windows service, and it doesn't indicate any problems. The StartNTService script starts the service without error, but the StartSonar script reports "jvm 1 | WrapperSimpleApp: Encountered an error running main: org.apache.catalina.LifecycleException: Failed to initialize component [StandardServer[-1]]". I don't know if that's relevant for the way I'm using SonarQube as a Windows service. I'm using pretty much all default configuration except I edited the sonar-runner.properties file by un-commented these lines:
sonar.host.url=http://localhost:9000
...
sonar.sourceEncoding=UTF-8
If anybody can help me figure out how to get this to work, I would greatly appreciate it. I am running Windows XP Pro x64 SP2 and Java 1.7.0_51 (32 bit). On my first attempt I did incorrectly use the 64 bit service install/launch scripts but when the console output indicated it didn't match the JVM architecture, I stopped and uninstalled the service and installed and launched the 32 bit service.
Edit: I'm using Sonarqube version 4.2 and Sonar-runner version 2.3.
By default, SonarQube is packaged with the Java ecosystem (Java, Checkstyle, PMD), but for C/C++ you will have to install one of the plugins that support analysis of these languages.
There is a commercial C++ plugin from SonarSource and a free Community Plugin.