I have 2 Jenkins shared libraries which contain groovy files to be scanned. Both projects are using the same sonarqube call using maven but in one project the groovy files are scanned, whereas in the other they are not
Sonarqube call
...
String scanCmd = "mvn org.sonarsource.scanner.maven:sonar-maven-plugin:3.4.0.905:sonar " +
"-f pom.xml " +
"-Dsonar.branch=%GIT_BRANCH% " +
"-Dsonar.test.inclusions=**/*Test*/** " +
"-Dsonar.exclusions=**/*Test*/** " +
"-Dsonar.skipDesign=true "
bat scanCmd
...
We are using version 5.6.6, if that matters
Sonar scanner config
The sonar configuration is in the parent pom which contains the sonar properties
....
<properties>
<!-- Sonar -->
<sonar.sources>src,vars</sonar.sources>
<sonar.inclusions>**/**</sonar.inclusions>
</properties>
....
Both projects (pom.xml) use the this parent pom i.e. the same sonar properties
Project 1: 0 files indexed
The project structure looks like this:
D:.
│ .gitignore
│ Jenkinsfile
│ pom.xml
│ README.md
│
├───src
│ └───ch
│ └───swisscard
│ └───jenkins
│ └───pipeline
│ └───util
│ PipelineRabbitMQ.groovy
│
└───vars
ciPipeline.groovy
So there are two groovy files which should be scanned but the logfile shows 0 files indexed:
[INFO] Base dir: D:\jenkins\tools\ci-home\workspace\rabbitmq-ci-pipeline_master-2CYM2JXOI6CJM2T4MSNQTNPWY7LRY7PTSC3OG74TSX2Q2WNKXUSQ
[INFO] Working dir: d:\jenkins\tools\ci-home\workspace\rabbitmq-ci-pipeline_master-2CYM2JXOI6CJM2T4MSNQTNPWY7LRY7PTSC3OG74TSX2Q2WNKXUSQ\target\sonar
[INFO] Source paths: src, vars
[INFO] Binary dirs: target/classes
[INFO] Source encoding: UTF-8, default locale: en_US_US
[INFO] Index files
[INFO] Included sources:
[INFO] **/**
[INFO] Excluded sources:
[INFO] **/target/**/*
[INFO] **/*Test*/**
[INFO] **/*Test*/**
[INFO] Included tests:
[INFO] **/*Test*/**
[INFO] 0 files indexed
[INFO] 0 files ignored because of inclusion/exclusion patterns
Project 2: Files properly scanned
The second project structure looks like this:
D:.
│ .gitignore
│ Jenkinsfile
│ pom.xml
├───src
│ └───main
│ │ Utils.groovy
└───vars
checkQualityGate.groovy
defaultBuildApplication.groovy
ciPipeline.groovy
And here the log file indicates that all files were scanned:
[INFO] Load server rules (done) | time=98ms
[INFO] Base dir: D:\jenkins\tools\ci-home\workspace\pipeline_feature_sonar-scan-D6GRAK5PLQOMGCOOH2T6QP6BP4LA6O3LP23UC7VBSSL3RORI4ZXQ
[INFO] Working dir: d:\jenkins\tools\ci-home\workspace\pipeline_feature_sonar-scan-D6GRAK5PLQOMGCOOH2T6QP6BP4LA6O3LP23UC7VBSSL3RORI4ZXQ\target\sonar
[INFO] Source paths: src, vars
[INFO] Source encoding: UTF-8, default locale: en_US_US
[INFO] Index files
[INFO] Included sources:
[INFO] **/**
[INFO] Excluded sources:
[INFO] **/target/**/*
[INFO] **/*Test*/**
[INFO] **/*Test*/**
[INFO] Included tests:
[INFO] **/*Test*/**
[INFO] 5 files indexed
[INFO] 0 files ignored because of inclusion/exclusion patterns
Problem/Question
So apparently both projects have the same configuration and similar structure, however one is scanned and another not. At the moment I don't understand why?
I have checked other topics which did not help me
I do not have the sonar.basedir explicily set, but according to the log the base dir is fine
I had similar issue for my project. I added sonar.scm.exclusions.disabled=true in the sonar properties and the files were indexed and scanned fine.
Related
A vulnerability scan highlighted the presence of Maven JARs in the Docker image for our Quarkus app which surprised me. These are being picked up from quarkus-app/lib/main. Running mvn quarkus:dependency-tree shows the route by which they're getting included:
[INFO] --- quarkus-maven-plugin:2.2.3.Final:dependency-tree (default-cli) # my-app ---
[INFO] Quarkus application PROD mode build dependency tree:
[INFO] my-group:my-app:pom:1.x-SNAPSHOT
...
[INFO] ├─ io.quarkus:quarkus-undertow-deployment:jar:2.2.3.Final (compile)
...
[INFO] │ ├─ io.quarkus:quarkus-core-deployment:jar:2.2.3.Final (compile)
...
[INFO] │ │ ├─ io.quarkus:quarkus-bootstrap-core:jar:2.2.3.Final (compile)
...
[INFO] │ │ │ ├─ io.quarkus:quarkus-bootstrap-maven-resolver:jar:2.2.3.Final (compile)
[INFO] │ │ │ │ ├─ org.apache.maven:maven-embedder:jar:3.8.1 (compile)
Again, I was surprised to see that it was pulling in deployment JARs as, based on my limited understanding, I would have not expected these to be needed at runtime for a non-mutable production app.
EDIT: The POM just references the extension and then there is a parent POM that references the Quarkus BOM.
<dependency>
<groupId>io.quarkus</groupId>
<artifactId>quarkus-undertow</artifactId>
</dependency>
If the deployment JARs are needed, why, and, if not, is there a way to exclude them?
As #Ladicek suggested, the cause of this issue was that one of dependencies was referencing the deployment artifact directly (just not the one I posted in the question!). This was then causing the deployment JARs for all of the extensions to be packaged in quarkus-app/lib/main.
Correcting quarkus-logging-json-deployment to quarkus-logging-json fixed the problem.
First of all, I'm entirely new to Maven.
I've imported a big complex maven project to IntelliJ. It contains multiple pom.xml, and it also depends on another local project (that I've already imported and works well).
The import process went well; on IntelliJ, nothing is red underlined. It seems to find every dependency (whether local or not). However when I try to build the .jar of one of the project via the Maven menu:
I get this output:
/Library/Java/JavaVirtualMachines/jdk1.8.0_161.jdk/Contents/Home/bin/java -Dmaven.multiModuleProjectDirectory=/Users/bastiannanchen/Programmation/Backend/backend/Reconfigure "-Dmaven.home=/Applications/IntelliJ IDEA.app/Contents/plugins/maven/lib/maven3" "-Dclassworlds.conf=/Applications/IntelliJ IDEA.app/Contents/plugins/maven/lib/maven3/bin/m2.conf" "-javaagent:/Applications/IntelliJ IDEA.app/Contents/lib/idea_rt.jar=62186:/Applications/IntelliJ IDEA.app/Contents/bin" -Dfile.encoding=UTF-8 -classpath "/Applications/IntelliJ IDEA.app/Contents/plugins/maven/lib/maven3/boot/plexus-classworlds-2.5.2.jar" org.codehaus.classworlds.Launcher -Didea.version=2018.3.4 compile
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Reconfigure 3.3.0
[INFO] ------------------------------------------------------------------------
[WARNING] The POM for <local_project> is missing, no dependency information available
[...]
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.276 s
[INFO] Finished at: 2019-02-12T11:08:44+01:00
[INFO] Final Memory: 10M/309M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project Reconfigure: Could not resolve dependencies for project <project_name> The following artifacts could not be resolved: <local_project> [...]
[...]
Do you know how to resolve this problem? I suppose I'm not proceeding correctly with the building process, but I am having a hard time to find an answer on the internet.
Thanks!
UPDATE
The structure of the project in summary:
├── backend
│ ├── project_1
│ ├── project_2
│ ├── project_3
│ └── project_4
└── utils
├── project_utils_1
├── project_utils_2
└── project_utils_3
I'm trying to build, e.g., the project_2 that has dependencies with all the projects in backend and some projects in utils.
I finally find a way to build the .jar file. Before building the package, I needed to use the Maven install plugin for the dependencies. FYI The Install Plugin is used during the install phase to add artifact(s) to the local repository.
Thank you to everybody that tried to help me :)
Using SonarQube 6.3, sonar-maven-plugin 3.3.0.660, Java 8, on a multi-module Maven project.
On the source files, code coverage (with JaCoCo) is working well, as are the analysis plugins, duplication detection, test executions and duration etc. But, we have no metrics for test files. The files are being loaded into SonarQube, but Lines of Code and Duplications are blank (as is Coverage but that one makes sense), while Bugs, Vulnerabilities, and Code Smells are all 0. We may have something configured incorrectly but I can't figure out what.
Example: one module contains Java and XML source and test files. The sonar properties in the POM are:
<sonar.sources>.</sonar.sources>
<sonar.inclusions>src/main/**/*,src/app/**/*,**/*.xml,**/*.properties,**/*.json</sonar.inclusions>
<sonar.tests>.</sonar.tests>
<sonar.test.inclusions>src/test/**/*,e2e/**/*</sonar.test.inclusions>
In SonarQube --> Administration --> Analysis Scope, we have Import Unknown Files enabled. We also have the property sonar.global.exclusions set to target/**/*
When running an analysis with the Maven plugin, we see messages like this in logs:
[INFO] 13:01:35.726 Base dir: /JENKINS_WS/project/module
[INFO] 13:01:35.726 Working dir: /JENKINS_WS/project/module/target/sonar
[INFO] 13:01:35.726 Source paths: .
[INFO] 13:01:35.726 Test paths: .
[INFO] 13:01:35.726 Source encoding: UTF-8, default locale: en_US
[INFO] 13:01:35.726 Index files
[INFO] 13:01:35.726 Included sources:
[INFO] 13:01:35.726 src/main/**/*
[INFO] 13:01:35.726 src/app/**/*
[INFO] 13:01:35.726 **/*.xml
[INFO] 13:01:35.726 **/*.properties
[INFO] 13:01:35.727 **/*.json
[INFO] 13:01:35.727 Excluded sources:
[INFO] 13:01:35.727 target/**/*
[INFO] 13:01:35.727 src/test/**/*
[INFO] 13:01:35.727 e2e/**/*
[INFO] 13:01:35.727 Included tests:
[INFO] 13:01:35.727 src/test/**/*
[INFO] 13:01:35.727 e2e/**/*
...
It appears that files in the test directories are being indexed properly:
[DEBUG] 13:01:35.778 'src/test/resources/log4j.xml' indexed as test with language 'xml'
[DEBUG] 13:01:35.779 'src/test/resources/appContext-test.xml' indexed as test with language 'xml'
[DEBUG] 13:01:35.779 'src/test/java/com/company/module/EnvEchoTest.java' indexed as test with language 'java'
And the Java scanner reports that the single test file was analyzed too:
[INFO] 13:01:38.520 Java Test Files AST scan
[INFO] 13:01:38.520 1 source files to be analyzed
[DEBUG] 13:01:38.521 'src/test/java/com/company/module/EnvEchoTest.java' generated metadata as test with charset 'UTF-8'
[DEBUG] 13:01:38.529 ----- Classpath analyzed by Squid:
[DEBUG] 13:01:38.529 /JENKINS_WS/project/module/target/test-classes
[DEBUG] 13:01:38.529 /JENKINS_WS/project/module/target/classes
... snip printing all the jar file debug msgs ....
[INFO] 13:01:38.567 1/1 source files have been analyzed
[INFO] 13:01:38.567 Java Test Files AST scan (done) | time=47ms
But the metrics are not populated in SonarQube. Can anyone provide pointers?
All is well. Or at least working as designed.
SonarQube is designed to report on the quality of your code, and tests aren't really considered to be code. They're... supporting files.
That said, there is a set of rules written specifically to examine the quality of your tests as tests. If included in your profile, they might raise issues on your tests. But so-called normal rules won't be run against tests, and metrics will not be calculated.
If you really want those things, you'll need to set up a separate, second analysis of your project, where the sonar.sources property points to the tests directory.
I have two wars say War-A and War - B. I need to build war - B re-using the src and resources directory of War - A with some minor config changes in web.xml of war-B. Hence I given the below code in war - B
<srcPath>${project.parent.basedir}/War-A/src/main/java</srcPath>
<resourcespath>${project.parent.basedir}/War-A/src/main</resourcespath>
But while building the war-B, Maven not referring the proper path of parent.basedir instead it is showing below logs
[INFO] skip non existing resourceDirectory C:\..\war-B\${project.parent.basedir}\war-A\src\main
[INFO]
[INFO] --- maven-compiler-plugin:2.3.2:compile (default-compile) #
war-B ---
[INFO] No sources to compile
[INFO]
However with the same configuration
<webapp>${project.parent.basedir}/War-A/src/main</webapp>
it is able to generate the war with webapp directory. Please suggest some solution and I am using eclipse + maven
Try this
../War-A/src/main/java
I followed the instructions to the letter in this blog post by Sonatype on writing a Maven plugin in Ruby.
When I try running mvn install (Maven 3.0.4), the build is successful, but maven-plugin-plugin cannot find any mojo descriptors:
[INFO] ------------------------------------------------------------------------
[INFO] Building Example Ruby Mojo - firstruby-maven-plugin 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[WARNING] The POM for org.jruby:jruby:jar:0.9.9 is invalid, transitive dependencies (if any) will not be available, enable debug logging for more details
[INFO]
[INFO] --- maven-plugin-plugin:2.4:descriptor (default-descriptor) # firstruby-maven-plugin ---
[INFO] Using 3 extractors.
[INFO] Applying extractor for language: java
[INFO] Extractor for language: java found 0 mojo descriptors.
[INFO] Applying extractor for language: jruby
[INFO] Extractor for language: jruby found 0 mojo descriptors.
[INFO] Applying extractor for language: bsh
[INFO] Extractor for language: bsh found 0 mojo descriptors.
If I try bumping maven-plugin-plugin version to anything higher than 2.4, the build fails with this error message:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-plugin-plugin:2.5:descriptor (default-descriptor) on project firstruby-maven-plugin: The API of the mojo scanner is not compatible with this plugin version. Please check the plugin dependencies configured in the POM and ensure the versions match. org.apache.maven.tools.plugin.scanner.MojoScanner.populatePluginDescriptor(Lorg/apache/maven/tools/plugin/PluginToolsRequest;)V -> [Help 1]
Here's the output of tree to show that the directory structure is OK:
firstruby-maven-plugin
├── pom.xml
└── src
└── main
└── scripts
└── echo.rb
3 directories, 2 files
Any idea why it's not working?
The Codehaus jruby-maven-plugin seems to have been abandoned 5 years ago. If you're using Maven3, you can try using Maven2 but I'd look at https://github.com/torquebox/jruby-maven-plugins