I am trying to run a sonar maven analysis on my multilanguage project which contains many languages like *.java, *.groovy, *.js etc. I have installed all the languages plugin in my sonar and configured my pom sonar.sources parameter as src/main,src/test but still it picks up only java files. In the console output.
I get the following lines in the console indicating that it only scans the folders with pattern src/main/java and src/test/java
[INFO] [05:13:57.140] ------------- Scan myapp
[INFO] [05:13:57.140] Load module settings
[INFO] [05:13:57.187] Initializer FindbugsMavenInitializer
[INFO] [05:13:57.187] Initializer FindbugsMavenInitializer (done) | time=0ms
[INFO] [05:13:57.187] Base dir: C:\myapp
[INFO] [05:13:57.187] Working dir: C:\myapp\target\sonar
[INFO] [05:13:57.187] Source paths: pom.xml, src/main/java
[INFO] [05:13:57.187] Test paths: src/test/java
I am currently using SonarQube 5.1, Java 7u80
Note: If analysis is done using Sonar Runner, It scans all the files.
Since SonarQube 4.2, it is possible to run an analysis on a multi-language project. To do so, the sonar.language property just has to be removed. Conversely, if for some reason you want to perform a single language-only analysis, make sure sonar.language is specified.
By default the sonar.sources property is set to the value of the Maven sourceDirectory property (by default it is src/main/java) plus pom.xml (and also src/main/webapp is automatically added for war modules). Therefore, for a multi-language project, the property usually has to be overridden to: sonar.sources=src/main,pom.xml.
Since it was a multi module project, I had to include the properties sonar.sources and sonar.tests in the parent module's pom.xml file.
<properties>
<sonar.sources>pom.xml,src/main,src/test</sonar.sources>
<sonar.tests></sonar.tests>
</properties>
sonar.tests parameter is empty since it is incompatible with Maven.
Related
Using SonarQube 6.3, sonar-maven-plugin 3.3.0.660, Java 8, on a multi-module Maven project.
On the source files, code coverage (with JaCoCo) is working well, as are the analysis plugins, duplication detection, test executions and duration etc. But, we have no metrics for test files. The files are being loaded into SonarQube, but Lines of Code and Duplications are blank (as is Coverage but that one makes sense), while Bugs, Vulnerabilities, and Code Smells are all 0. We may have something configured incorrectly but I can't figure out what.
Example: one module contains Java and XML source and test files. The sonar properties in the POM are:
<sonar.sources>.</sonar.sources>
<sonar.inclusions>src/main/**/*,src/app/**/*,**/*.xml,**/*.properties,**/*.json</sonar.inclusions>
<sonar.tests>.</sonar.tests>
<sonar.test.inclusions>src/test/**/*,e2e/**/*</sonar.test.inclusions>
In SonarQube --> Administration --> Analysis Scope, we have Import Unknown Files enabled. We also have the property sonar.global.exclusions set to target/**/*
When running an analysis with the Maven plugin, we see messages like this in logs:
[INFO] 13:01:35.726 Base dir: /JENKINS_WS/project/module
[INFO] 13:01:35.726 Working dir: /JENKINS_WS/project/module/target/sonar
[INFO] 13:01:35.726 Source paths: .
[INFO] 13:01:35.726 Test paths: .
[INFO] 13:01:35.726 Source encoding: UTF-8, default locale: en_US
[INFO] 13:01:35.726 Index files
[INFO] 13:01:35.726 Included sources:
[INFO] 13:01:35.726 src/main/**/*
[INFO] 13:01:35.726 src/app/**/*
[INFO] 13:01:35.726 **/*.xml
[INFO] 13:01:35.726 **/*.properties
[INFO] 13:01:35.727 **/*.json
[INFO] 13:01:35.727 Excluded sources:
[INFO] 13:01:35.727 target/**/*
[INFO] 13:01:35.727 src/test/**/*
[INFO] 13:01:35.727 e2e/**/*
[INFO] 13:01:35.727 Included tests:
[INFO] 13:01:35.727 src/test/**/*
[INFO] 13:01:35.727 e2e/**/*
...
It appears that files in the test directories are being indexed properly:
[DEBUG] 13:01:35.778 'src/test/resources/log4j.xml' indexed as test with language 'xml'
[DEBUG] 13:01:35.779 'src/test/resources/appContext-test.xml' indexed as test with language 'xml'
[DEBUG] 13:01:35.779 'src/test/java/com/company/module/EnvEchoTest.java' indexed as test with language 'java'
And the Java scanner reports that the single test file was analyzed too:
[INFO] 13:01:38.520 Java Test Files AST scan
[INFO] 13:01:38.520 1 source files to be analyzed
[DEBUG] 13:01:38.521 'src/test/java/com/company/module/EnvEchoTest.java' generated metadata as test with charset 'UTF-8'
[DEBUG] 13:01:38.529 ----- Classpath analyzed by Squid:
[DEBUG] 13:01:38.529 /JENKINS_WS/project/module/target/test-classes
[DEBUG] 13:01:38.529 /JENKINS_WS/project/module/target/classes
... snip printing all the jar file debug msgs ....
[INFO] 13:01:38.567 1/1 source files have been analyzed
[INFO] 13:01:38.567 Java Test Files AST scan (done) | time=47ms
But the metrics are not populated in SonarQube. Can anyone provide pointers?
All is well. Or at least working as designed.
SonarQube is designed to report on the quality of your code, and tests aren't really considered to be code. They're... supporting files.
That said, there is a set of rules written specifically to examine the quality of your tests as tests. If included in your profile, they might raise issues on your tests. But so-called normal rules won't be run against tests, and metrics will not be calculated.
If you really want those things, you'll need to set up a separate, second analysis of your project, where the sonar.sources property points to the tests directory.
I have created a Jenkins job which performs a SONAR QUBE analysis.
I have added the SONAR scan as a generic build step.
I have not made any modifications to the project to enable the SONAR integration since the project should not have to care about SONAR. (i.e. no changes to POM.xml and no properties file(s) added)
My SONAR integration is working, without any mods necessary to the project, which is great.
However, it is a multi module project and now I want to exclude one of the modules from the scan, since this module contains auto generated code.
Here are the Maven commands that I’m using to trigger the SONAR scan in Jenkins:
sonar:sonar
-Dsonar:host.url=http://url.com:9000/sonar
-Dsonar.login=myusername
-Dsonar.password=mypassword
Here is my project structure:
root
Project A
Project B
Project C
I want to exclude ProjectB from the scan.
I have tried -Dsonar.exclusions=/ProjectB and it did not work.
Here is an excerpt from the log:
[INFO] ------------- Scan Project B
[INFO] Base dir: /workspace/jenkins/…/workspace/CI-JOB-NAME/projectB
[INFO] Working dir: /workspace/jenkins/…/workspace/CI-JOB-NAME/projectB/target/sonar
[INFO] Source paths: pom.xml, src/main/java
[INFO] Binary dirs: target/classes
[INFO] Source encoding: UTF-8, default locale: en_US
[INFO] Index files
[INFO] Excluded sources:
[INFO] /projectB
[INFO] 44 files indexed
[INFO] 0 files ignored because of inclusion/exclusion patterns
As shown in the log, the excluded sources property is set but not working since it says 0 files ignored...
Question: How do I exclude everything inside the project B directory from the scan?
(Preferably using a command in the Jenkins job and not adding any properties files or other modifications to the project itself)
Is the “Excluded sources path” relative to the “Base dir” path or the Root??
Should I define the exclusion path as an absolute path?
The documentation has a dedicated section for this.
The options that seem suitable for your case:
use build profiles to exclude some module (like for integration tests)
use Advanced Reactor Options (such as -pl). For example mvn sonar:sonar -pl !module2
For example using this last one, you could do:
sonar:sonar -pl "!projectB"
-Dsonar:host.url=http://url.com:9000/sonar
-Dsonar.login=myusername
-Dsonar.password=mypassword
Is the “Excluded sources path” relative to the “Base dir” path or the Root?? Should I define the exclusion path as an absolute path?
According to the documentation on exclusions,
patterns are relative to the base directory, so not absolute paths.
In any case, I don't think this option is useful in your case,
because in a multi-module maven project,
the files of a sub-module are indexed with the path relative to the module's base directory. So a pattern like projectB/** will not match anything, as projectB is not part of the paths used. If you have some unique package name in the module, let's say someuniquepackage, located at src/main/java/someuniquepackage, then the pattern src/main/java/someuniquepackage/** would work.
In anycase, I recommend the -pl option above, or using Maven's profile feature.
I'm looking for a (supported) mvn based command, which will give me a list of all the GroupID:ArtifactID:Version (GAV) for all artifacts that running a mvn package command would produce.
For a single module Maven project, with no parent pom, this is trivial: you can look inside the pom.
For a single module Maven project, with a parent pom, you could use help:effective-pom and it will present a pom file with the <version> element present.
For a multi module Maven project (reactor), you could actually do the same (didn't think so, learned so just now by trying it out). This will allow parsing the file for (multiple) <project> elements.
Anything else to consider?
The overall goal of this, is to be able to feed a downstream Continuous Delivery (http://go.cd/) stage/step/job with information on what version of it's upstream dependencies should be used.
In general you can't produce a list before the build has run...The problem is that based on the pom model not all artifacts are described, cause some plugins can produce supplemental artifacts (maven-assembly-plugin, maven-shade-plugin, maven-jar-plugin via test-jar etc.)
What you can make is to get a list of produced artifacts after a build has run..(installed). The question of yours inspired me to implement an EventSpy which produces such list at the end of the build...which looks like this:
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4.423 s
[INFO] Finished at: 2016-05-08T13:22:10+02:00
[INFO] Final Memory: 24M/310M
[INFO] ------------------------------------------------------------------------
[INFO] -- Maven Artifact Collector Summary --
[INFO] ------------------------------------------------------------------------
[INFO] test.maven.plugin.profiler:parse-pom:0.1.0-SNAPSHOT:jar
[INFO] test.maven.plugin.profiler:parse-pom:0.1.0-SNAPSHOT:pom
[INFO] test.maven.plugin.profiler:parse-pom:0.1.0-SNAPSHOT:jar:jar-with-dependencies
What i can do is to enhance that and write a file which contains the information (in more or less any format)...At the moment it's just a PoC...May be you can give some more information or create issues or PR's and request what might be needed...may be this is also interesting for others...
Furthermore your downstream part must have those artifacts within a repository cache available (either on a file system or via a repository manager or docker data container)...
I have Maven 3.x.x and defined MAVEN VERSIONS Plugin in ~/.m2/settings.xml (global user level file) where I have valid .. and plugin configuration set for maven-versions-plugin and maven-enforcer-plugin etc.
(Maven inheritance and aggregation is used among these subprojects referring parent project)
Project structure :
pom.xml
module1/pom.xml
module1/src/main/resources/module1/pom.xml
module1/src/main/resources/module1/src/somemorefileshere
module2.pom.xml
module3/pom.xml
module3/src/main/resources/module3/pom.xml
module3/src/main/resources/module3/src/somemorefileshere
When I run the following command in this multi-module project setup:
$ mvn versions:set -DnewVersion=0.0.1
It successfully changes all occurrences of 0.0.1-SNAPSHOT in parent (project root level pom.xml) and sub-project/modulesX level pom.xml files BUT it didn't make the changes in the following files:
module1/src/main/resources/module1/pom.xml
module3/src/main/resources/module3/pom.xml
Questions:
1) Is it because these files sit under "src/main/resources" folder??
2) If the plugin info is configured in ~/.m2/settings.xml then do I still need to update the same in pom.xml (I don't think so).
3) Do I have to specify ... section where we specify "url", "connection", "developerConnection" values in root level project pom.xml OR it has to be placed in every sub-project/moduleX pom.xml? I think, root level pom.xml should suffice.
4) Even though each sub-project/moduleX pom.xml is successfully changing from 0.0.1-SNAPSHOT to 0.0.1 (as the version I'm setting), I get the following output once versions:set is complete. Why it's showing "SKIPPED" for those modulesX when it actually replaced the values successfully. Does this means, I have to have ... section in those sub-project/moduleX's pom.xml file as well??
19:54:40 [INFO] ------------------------------------------------------------------------
19:54:40 [INFO] Reactor Summary:
19:54:40 [INFO]
19:54:40 [INFO] main-parent ........................................ SUCCESS [ 3.082 s]
19:54:40 [INFO] module1 ................................... SKIPPED
19:54:40 [INFO] module2 .......................................... SKIPPED
19:54:40 [INFO] module3 .......................................... SKIPPED
5) Do I need to include "maven-enforcer-plugin" configuration in every root level / sub-project level (moduleX) level pom.xml?? --OR ~/.m2/setting.xml level pom should suffice?
because those looks like archetype pom.xml files (Template files) it doesn't process them, it just touches those are connected in maven POMdel
why does maven ear plugin runs the war:war (plugin goal) when I try to build the ear project from netbeans
my process:
I right click on the ear project (which has the war dependency listed) and first Clean and then right click on it and select Build with Dependencies. Then it builds the war again using the war:war and it takes time. even though there was no change in the war but will re create it again.
this is what it says :
------------------------------------------------------------------------ Building finweb 1.0-SNAPSHOT
------------------------------------------------------------------------ The POM for org.netbeans.external:jdom-1.0:jar:RELEASE71 is missing,
no dependency information available The POM for
com.ibm:com.ibm.mq:jar:6.0.2.5 is missing, no dependency information
available The POM for com.ibm:com.ibm.mqbind:jar:6.0.2.5 is missing,
no dependency information available The POM for
net.sf.saxon:saxon:jar:10.0-b19 is missing, no dependency information
available
[dependency:copy]
[resources:resources] Using 'UTF-8' encoding to copy filtered
resources. skip non existing resourceDirectory
C:\Beta\fin\finweb\src\main\resources
[compiler:compile] Nothing to compile - all classes are up to date
[resources:testResources] Using 'UTF-8' encoding to copy filtered
resources. skip non existing resourceDirectory
C:\Beta\fin\finweb\src\test\resources
[compiler:testCompile] Nothing to compile - all classes are up to date
[surefire:test] No tests to run. Surefire report directory:
C:\Beta\fin\finweb\target\surefire-reports
------------------------------------------------------- T E S T S
------------------------------------------------------- There are no tests to run.
Results :
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
[war:war] Packaging webapp Assembling webapp [finweb] in
[C:\Beta\fin\finweb\target\finweb-1.0-SNAPSHOT] Processing war
project Copying webapp resources
[C:\Beta\fin\finweb\src\main\webapp] Webapp assembled in [109467
msecs] Building war:
C:\Beta\fin\finweb\target\finweb-1.0-SNAPSHOT.war WEB-INF\web.xml
already added, skipping
[install:install] Installing
C:\Beta\fin\finweb\target\finweb-1.0-SNAPSHOT.war to C:\Documents
and
Settings.m2\repository\com\comp\finweb\1.0-SNAPSHOT\finweb-1.0-SNAPSHOT.war
Installing C:\Beta\fin\finweb\pom.xml to C:\Documents and
Settings\5510041.m2\repository\com\comp\finweb\1.0-SNAPSHOT\finweb-1.0-SNAPSHOT.pom
So if there is nothing to compile why would it build a war again if it was already made and there are no changes. is this the default behaviour, looking at compile : compile when it doesnt compile if there is nothing needed then why does war:war does it, it shud be intelligent enuff to do it right?
pls correct me if I am wrong
Thanks in advance..
Syed.
This may be considered as workaround.
Following configs will speed up your build time.
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<version>2.4</version>
<configuration>
<useCache>true</useCache>
<recompressZippedFiles>false</recompressZippedFiles>
</configuration>
</plugin>
Reference :https://maven.apache.org/plugins/maven-war-plugin/war-mojo.html
Also am looking forward for easy way to skip war:war phase from maven default build lifecycle.
Your concern can be understood. But in principle maven-war-plugin does not know that the resulting WAR artifact will be identical to one which was build a minute ago because:
You might define MANIFEST entry that contains the current build time.
You might enable different profile, which will change the picture completely.
Other plugins (like maven-ant-plugin, maven-dependency-plugin) might add something to target directory depending on some changing conditions (e.g. property from command line).
So it is safer to rebuild war each time. On modern platforms this is really negligible.