I am using Fastlane for building and testing my ObjC project. I use scan action to run Unit Test cases and slather action to generate Code coverage report.
I am able to generate cobertura.xml report using slather action, but unable to publish the report to SonarQube.
I am using SonarQube 6.4 and fastlane 2.64.0.
FastFile
scan(
workspace: "Sample.xcworkspace",
scheme: "SampleTests",
code_coverage: true,
output_types: "html"
)
slather(
cobertura_xml: true,
output_directory: "./reports",
proj: "Sample.xcodeproj",
workspace: "Sample.xcworkspace",
scheme: "SampleTests",
)
sonar
Analysis is published to Sonar but Code Coverage report is not updated. Please let me know where i miss the key.
From your comments on your question, it seems that you haven't tried configuring the path to the report, so it's natural that no coverage data is imported. The analysis cannot intuit where reports are or that it should read them.
Having said that, you also indicate that you're generating a cobertura.xml file, but that's not one of the formats currently supported by SonarCFamily for Objective-C. So you'll need to get your coverage data into the Generic Coverage format, and then include the path to that report using the sonar.coverageReportPaths analysis property.
Related
I've got a Java/Maven project that uses RestDocs to generate documentation of our endpoints.
For the uninitiated of RestDocs, what it does is monitor tests during the build and use information from the tests to generate "snippets" that get used to create API documenation using asciidoc during the mvn package phase.
The problem I'm facing is that if the asciidoc (.adoc) file is referencing a non-existent snippet, the doc gets generated and will say something like:
"Unresolved directive in myDoc.adoc - include::{snippets}/error-example/response-fields.adoc[]"
Since this happens after the tests, I don't know how to test for something like this so that the build can stop and let the developer know that they need to update either the .adoc or the snippet.
Any guidance would be much appreciated.
Adding this to the configuration of the asciidoctor plugin fails the build when a snippet is not found that the .adoc is expecting:
<logHandler>
<outputToConsole>true</outputToConsole>
<failIf>
<containsText>include file not found</containsText>
</failIf>
</logHandler>
We utilize the pipeline and after the build completes successfully we are running the following:
bat "mvn sonar:sonar -B -s ${buildSettings} -Dsonar.analysis.mode=preview -Dsonar.skipDesign=true -Dsonar.report.export.path=sonar-report.json"
sonarToGerrit(severity: 'Major', postScore: true, category: 'Code-Review', newIssuesOnly: true, issuesScore: '0', noIssuesScore: '0', changedLinesOnly: true)
The below build log shows that it found a good number of issues but yet the issues to be commented is 0.
Build log
Other posts suggest that it may not be finding the report but I don't believe it's the case as it found a number of issues. Any pipeline configuration advice would be much appreciated.
We are using Sonar Gerrit plugin version 2.2.1, Gerrit Trigger 2.27.3 and Jenkins Enterprise version is 2.60.3.1.
The most common case regarding to your question is related to the fact that SonarQube checks the whole project, regardless of amount of changes that were done in particular change. When creating a new report, it compares a result with a result that is stored to it's database (with previously found issues that were found in a mode other than preview). Thus sonar marks as new all the issues that it was not aware about before (and if you don't store this information in SonarQube at all - all the issues will be marked as new). But sonar-gerrit plugin can only post issues to the files that were affected by the change it is verifying. So, even if you set to "false" the settings "newIssuesOnly" and "changedLinesOnly", all the issues in files not affected by the change will be ignored.
Shortly, check that the issues marked with "isNew"="true" in your sonar report were actually changed (for changedLinesOnly=true) or located in the changed files (for changedLinesOnly = false) in the commit you are trying to check.
Another possible reason is the project configuration settings. If your files are part of a submodule, you'll need to include the submodule name to project base directories set. Or you may want to try a feature "allow auto match" instead. The feature tries to match SonarQube modules to Gerrit names automatically (available since 2.1).
Not related to your question advises regarding your pipeline code:
At the moment settings recognition for severity (and other enum values) is case-sensitive. In fact, the plugin ignores your "Major" setting as it cannot recognize it, and replaces it with default "INFO" value.
Another thing, I don't see why do you set your "postScore" to true as you set "issuesScore" = 0 and "noIssuesScore"=0. You can just set postScore=false and skip these settings along with "category" for sake of simplicity.
Also, if you use the version of plugin above 2.0, be aware that API has slightly changed and now uses the next structure:
sonarToGerrit (
reviewConfig: [
issueFilterConfig: [
severity: 'MAJOR',
newIssuesOnly: false,
changedLinesOnly: false
],
noIssuesTitleTemplate: 'Your text here',
someIssuesTitleTemplate: 'Your text here',
issueCommentTemplate: 'Your text here'
]
)
Though your code should also work (plugin does support previous versions), there is a bigger chance that there may be a bug.
I am also facing the same issue with Sonar-Gerrit jenkins plugin. Downloaded it from Jenkins plugins site.Using Sonar-Gerrit plugin 2.2.1, and analysing sonar scan against jenkins workspace.
For a sample,have changed just one file and provided the project base directory to the path of that file, and ran the sonar analysis in issues mode.
Issues are not loaded in Gerrit with logs
Report has loaded and contains 759 issues
Issues to be commented: 0
Issues to be involved in score calculation: 0
Review has been sent
Went through the serenity documented for extracting the test outcomes
below isthe code, it didn't work
OutcomeFormat format = OutcomeFormat.XML; TestOutcomes outcomes =
TestOutcomeLoader.loadTestOutcomes().inFormat(format)
Tried with below code and its working,
OutcomeFormat format = OutcomeFormat.JSON; TestOutcomeLoaderBuilder
outcomes= TestOutcomeLoader.loadTestOutcomes().inFormat(format);
TestOutcomes out =outcomes.from(new File(""));
Issue is i need the test outcomes in #AfterScenario, but the thing is serenity reports gets generated after the entire execution tried changing the pom but didn't help. Is there any other way using which we can extract the test results?
Serenity uses JSON format by default now. Why are you trying to obtain the test outcomes? (i.e. what problem are you trying to solve?)
Created a separate java class for report extracting and added that in maven plugin which will get executed after serenity report is generated.
As #John smart has mentioned JSON and HTML are default output formats.
Still if you want to access Outcomes after test execution.
You can create a custom listener and listen to serenity event bus.
TestRunFinished event will be published with Outcome as a parameter.
You can use the outcome for getting required details.
For creating custom listener you can follow this page
I am trying to figure out how to run unit tests, using Google Test, and send the results to TeamCity.
I have run my tests, and output the results to an xml, using a command-line argument --gtest_output="xml:test_results.xml".
I am trying to get this xml to be read in TeamCity. I don't see how I can get XML Reports passed to TeamCity during build/run...
Except through XML report Processing:
I added XML Report Processing, added Google Test, then...
it asks me to specify monitoring rules, and I added the path to the xml file... I don't understand what monitoring rules are, or how to create them...
[Still, I can see nowhere in the generated xml, the fact that it intends to talk to TeamCity...]
In the log, I have:
Google Test report watcher
[13:06:03][Google Test report watcher] No reports found for paths:
[13:06:03][Google Test report watcher] C:\path\test_results.xml
[13:06:03]Publishing internal artifacts
And, of course, no report results.
Can anyone please direct me to a proper way to import the xml test results file into TeamCity ? Thank you so much !
Edit: is it possible that XML Report Processing only processes reports that were created during build ? (which Google Test doesn't do?) And is ignoring the previously generated reports, as "out of date", while simply saying that it can't find them - or are in the wrong format, or... however I should read the message above ?
I found a bug report that shows that xml reports that are not generated during the build are ignored, making a newbie like me believe that they may not be generated correctly.
Two simple solutions:
1) Create a post build script
2) Add a build step that calls the command line executable with the command-line argument. Example:
Add build step
Add build feature - XML report processing
I had similar problems getting it to work. This is how I got it working.
When you call your google test executable from the command line, prepend %teamcity.build.checkoutDir% to the name of your xml file to set the path to it like this:
--gtest_output=xml:%teamcity.build.checkoutDir%test_results.xml
Then when configuring your additional build features on the build steps page, add this line to your monitoring rules:
%teamcity.build.checkoutDir%test_results.xml
Now the paths match and are in the build directory.
I am generating custom report(HTML) for soapUi and I am using TestNG and Maven.
How do i generate Report?
->I manually create a Reports Folder and create HTML Report file after every run.(I am NOT
using surefire report) i.e, I have added this action in setup and teardown method.
Mentioned below is my project overview.
Main Proj(Maven)
| - src/main/java/tests/classes
| - test-output(TestNG)
| - Reports
| - SampleReport.html
Now How to integrate with Hudson? I mean This report should be displayed in Hudson and sent as email report.
There is a TestNG plugin for Jenkins that you can use to parse TestNG reults, ie.
TestNG XML report pattern: **/testng-results.xml
We are using it and this presents results in a very nice way in Jenkins.
Then, to email your html report, you can use Email-ext plugin. It will let you change email contents to HTML and attach your result, ie.
${FILE, path="target/surefire-reports/emailable-report.html"}
Hope this helps somewhat