Maven, SonarQube & Exclusions not working as expected - sonarqube

Referring to https://stackoverflow.com/a/18585688/3639934 I've put the following in my POM to exclude generated source files, which exist in target/generated-sources/x/y/z.
</properties>
<sonar.exclusions>file:**/generated-sources/**</sonar.exclusions>
</properties>
In my logfile I see no files excluded, however there are files in my project's directory under target/generated-sources/x/y/z.
[10:14:39.346] Source paths: pom.xml, src/main/java
[10:14:39.346] Test paths: src/test/java
[10:14:39.347] Binary dirs: target/classes
[10:14:39.348] Source encoding: UTF-8, default locale: de_DE
[10:14:39.348] Index files
[10:14:39.363] Excluded sources:
[10:14:39.363] file:**/generated-sources/**
[10:14:39.484] 47 files indexed
[10:14:39.487] 0 files ignored because of inclusion/exclusion patterns
I'm using SQ 5.1.1, Maven 3.2.3, Java 8.
Can anyone enlighten me please?

Ok, the log fooled me. The generated-sources are no part of the analysis right from the start, so it did not show up any excluded files.
When I change the exclusion pattern to match a file from say "src/main/java" it logs the expected exclusion (and does not analyze the file.)

Related

vscode-java: picj pom-k3d.xml instead of pom.xml

Currently, I've two "pom.xml" files into my project:
pom-k3d.xml
pom.xml
pom-k3d.xml contains some additional plugins and dependencies.
vscode is getting by default pom.xml.
Is there any way to set vscode pick pom-k3d.xml file instead?
Is there any way to split a "pom.xml file into two modular ones?
Any ideas?

Checkstyle and Maven's Standard Directory Layout

I'm following Maven's Standard Directory Layout for my project.
Is there a preferred directory to put my checkstyle.xml file? I've seen it on at least 3 possible locations:
src/main/resources/checkstyle.xml
src/main/checkstyle/checkstyle.xml - Example: Joda-Time
src/checkstyle/checkstyle.xml - Example: Spring Boot
Since this is mostly a file for developers, the first option gives me doubts. Would it make sense to include checkstyle.xml into the JAR file?
Thanks,
Fede
Putting checkstyle.xml in src directory doesn't really make sense, as it is not part of source code.
The most common convention I've observed in my projects is putting it into config/checkstyle/checkstyle.xml. Thousands of projects use it (filename:checkstyle.xml path:config/checkstyle) and Gradle uses this location by default.

sass-maven-plugin does not compile scss files

I try to integrate scss conversion into our existing maven project.
I tried to add
<plugin>
<groupId>nl.geodienstencentrum.maven</groupId>
<artifactId>sass-maven-plugin</artifactId>
<version>${maven.sass.plugin.version}</version>
<executions>
<execution>
<id>generate-css</id>
<phase>generate-resources</phase>
<goals>
<goal>update-stylesheets</goal>
</goals>
<configuration>
<sassOptions>
<always_update>true</always_update>
</sassOptions>
<includes>
<include>${scssSourceInclude}</include>
</includes>
<sassSourceDirectory>${project.basedir}/src/main/webapp/scss</sassSourceDirectory>
<destination>${project.build.directory}/${project.build.finalName}/css</destination>
<!-- <destination>${project.basedir}/src/main/webapp/css</destination> -->
</configuration>
</execution>
</executions>
</plugin>
When I run mvn clean install command, it compiles correctly for destination which is set to "target" directory:
[INFO] --- sass-maven-plugin:2.12:update-stylesheets (generate-css) # ui.web ---
[INFO] Checked 1 files for d:\work\git\repository\project\src\main\webapp\scss
[INFO] Checked 0 files for d:\work\git\repository\project\target\ui.web\css
[INFO] Compiling Sass templates
[INFO] No resource element was specified, using short configuration.
[INFO] Queueing Sass template for compile: d:\work\git\repository\project/src/main/webapp/scss => d:\work\git\repository\project/target/ui.web/css
...
In case of using second (commented) destination it skips scss compilation:
[INFO] --- sass-maven-plugin:2.12:update-stylesheets (generate-css) # ui.web ---
[INFO] Checked 1 files for d:\work\git\repository\project\src\main\webapp\scss
[INFO] Checked 959 files for d:\work\git\repository\project\src\main\webapp\css
[INFO] Skip compiling Sass templates, no changes.
The only difference is that I need generated css files in src directories due to further processing (minification, concatenation, adding licence header, etc.)
Dir structure:
src
|-main
|-webapp
|-scss
|- common.scss
|- app.scss
|- admin
|- admin.scss
|-css
Can somebody explain to me why the files are not processed into src directories? Is there some missing configuration which I should apply?
I think your problem would be solved by upgrading version 2.22 or later. Here's why:
The console output of the build with <destination>${project.basedir}/src/main/webapp/css</destination> shows that you already have 959 files in the destination directory. Given that you have only a single file in src/main/webapp/scss I'm assuming that most of the 959 files in src/main/webapp/css are normal, non-generated CSS files and that you intend to add those generated by the plugin.
Looking at the source of the plugin, in versions prior to 2.22, the criteria for building given that files exist in both source and destination directories is that the source directory tree has a file younger than any of the files in the destination directory. See UpdateStylesheetsMojo#buildRequired() for the 2.12 implementation. Note that the plugin does not compare source and destination files by name, it only compares "extreme" timestamps. I suspect that the build that skips compilation finds no file in src/main/webapp/scss that are younger than all files in src/main/webapp/css. You could test my hypothesis by modifying one of the .scss files before building. Issue #136: Wrong "build required" analysis when source and destination are the same describes a very similar problem. This is solved by pull request #137:
add css filter at destination when checking buildrequired which is part of version 2.22. This should make your build actually compile .scss files even in the face of newer .css files in the destination directory.
Stepping back a bit, I would advise against modifying src/ during the build. As stated in Introduction to the Standard Directory Layout
The src directory contains all of the source material for building the project [...]
whereas
The target directory is used to house all output of the build.
Behavior is built around this convention. For example mvn clean removes target/ (or more generally ${project.build.directory}). It makes no attempt at removing files generated within src/. Looking at a developer reply on the issue mentioned above it's pretty clear that the developers of the plugin share this sentiment:
Why would anyone want to mix source code and compiled code at all?
Considering this convention, I would recommend you move the .scss files to src/main/sass (the default value of sassSourceDirectory) and let the plugin output .css files to e.g. ${project.build.directory}/generated-css-resources. Then let the plugins handling minification, concatenation and license header operate on this directory as well as src/main/webapp/css. If they don't support this with configuration, you can still likely achieve this with two executions of the plugins, one for each directory. An alternative is to aggregate the generated and non-generated .css files in a directory ${project.build.directory}/aggregated-css with the help of Maven Resources Plugin's copy-resources goal, making sure the generated come first, to avoid the initial problem, and use this directory as input to further processing. I'm sure there are other ways to solve this problem as well.
I am not sure about the solution, but I tried the workaround mentioned in this post https://github.com/jruby/jruby/issues/2498 and it worked for me.
Basically downgrade the sass-maven-plugin version to 2.0 and it should work.

SonarQube Exclude a directory

I am trying to exclude a directory from being analyzed by Sonar. I have the following properties defined in my sonar-project.properties file:
sonar.sources=src/java
sonar.exclusions=src/java/test/****/*.java
The directory structure I have is:
src/java/dig
src/java/test/dig
When I run the sonar-runner I get the following info:
INFO - Excluded sources:
INFO - src/java/test/**/*.java
INFO - Excluded tests:
INFO - **/package-info.java
But when I check the result of the analysis all the packages inside the test directory are still there.
I just need to tell Sonar to not analyze the test directory and any packages inside it.
Try something like this:
sonar.exclusions=src/java/test/**
I'm able to exclude multiple directories using the below config (comma separated folder paths):
sonar.exclusions=system/**, test/**, application/third_party/**, application/logs/**
And while running the sonar runner I got the following in the log:
Excluded sources:
system/**
test/**
application/third_party/**
application/logs/**
This will work for your case:
sonar.exclusions=**/src/java/dig/ ** , **/src/java/test/dig/ **
Another configuration option is adding a maven properties sonar.exclusions. Below is a sample pom file with exclusions of static jquery directory and static pdf viewer directory.
<project >
<modelVersion>4.0.0</modelVersion>
<artifactId>my Artifact</artifactId>
<!-- Enviroment variables can be referenced as such: ${env.PATH} -->
<packaging>war</packaging>
<url>http://maven.apache.org</url>
<properties>
<junit.version>4.9</junit.version>
<mockito.version>1.9.5</mockito.version>
<jackson.version>1.9.7</jackson.version>
<powermock.version>1.5</powermock.version>
<!--Exclude the files Here-->
<sonar.exclusions>src/main/webapp/static/jquery_ui/*,src/main/webapp/static/pdf-viewer/*,src/main/webapp/static/pdf-viewer/**,src/main/webapp/static/pdf-viewer/**/*</sonar.exclusions>
</properties>
If we want to skip the entire folder following can be used:
sonar.exclusions=folderName/**/*
And if we have only one particular file just give the complete path.
All the folder which needs to be exclude and be appended here.
Easiest way is to go to the server URL after starting the server(localhost:8080) then login as admin,Go to settings>Exclusions> Source File Exclusions- Add your packages here.
Restart the server.
If you're an Azure DevOps user looking for both where and how to exclude files and folders, here ya go:
Edit your pipeline
Make sure you have the "Prepare analysis on SonarQube" task added. You'll need to look elsewhere if you need help configuring this. Suggestion: Use the UI pipeline editor vs the yaml editor if you are missing the manage link. At present, there is no way to convert to UI from yaml. Just recreate the pipeline. If using git, you can delete the yaml from the root of your repo.
Under the 'Advanced' section of the "Prepare analysis on SonarQube" task, you can add exclusions. See advice given by others for specific exclusion formats.
Example:
# Additional properties that will be passed to the scanner,
# Put one key=value per line, example:
# sonar.exclusions=**/*.bin
sonar.exclusions=MyProjectName/MyWebContentFolder/**
Note: If you're not sure on the path, you can go into sonarqube, view your project, look at all or new 'Code Smells' and the path you need is listed above each grouping of issues. You can grab the full path to a file or use wilds like these examples:
MyProjectName/MyCodeFile.cs
MyProjectName/**
If you don't have the 'Run Code Analysis' task added, do that and place it somewhere after the 'Build solution **/*.sln' task.
Save and Queue and then check out your sonarqube server to see if the exclusions worked.
what version of sonar are you using?
There is one option called "sonar.skippedModules=yourmodulename".
This will skip the whole module. So be aware of it.
You can do the same with build.gradle
sonarqube {
properties {
property "sonar.exclusions", "**/src/java/test/**/*.java"
}
}
And if you want to exclude more files/directories then:
sonarqube {
properties {
property "sonar.exclusions", "**/src/java/test/**/*.java, **/src/java/main/**/*.java"
}
}
Add comma separated folder paths sonar.exclusions=**/abc/**,**/def/**
This worked in an angular project
I typed case sensitive and used "" and it worked. Analyze time decreased to 3 minutes from 10.
# Additional properties that will be passed to the scanner,
# Put one key=value per line, example:
sonar.exclusions=**\Scripts\**\*,**\Content\**\*
Just to mention that once you excluded the files from Sonar, do the same for Jacoco plugin:
<configuration>
<excludes>
<exclude>com/acme/model/persistence/entity/TransactionEntity*</exclude>
<exclude>com/acme/model/persistence/ModelConstants.class</exclude>
</excludes>
</configuration>
add this line to your sonar-project.properties file
ex: sonar.exclusions=src/*.java
be careful if you want to exclude a folder and inside the folder there is a file you must first exclude the files or add the files one by one
for example imagine there is a folder like below:
src/app.java
src/controllers/home.java
src/services/test.java
you have to do this:
sonar.exclusions=src/app.java,src/controllers/*.java,src/services/*.java
It worked for me
You can skip library like this
project(":libABC") {
apply plugin: 'org.sonarqube'
sonarqube {
skipProject = true
}
}
This worked for me:
sonar.exclusions=src/**/wwwroot/**/*.js,src/**/wwwroot/**/*.css
It excludes any .js and .css files under any of the sub directories of a folder "wwwroot" appearing as one of the sub directories of the "src" folder (project root).

Where to place properties files when running tests via maven + grizzly + jersey test framework?

The my.properties file from my source (src/main/resources) folder keepa getting picked up and used when I try to run my JerseyTest ... whereas I would like the properties files in the test folder (src/test/resources) to be utilized.
ClassLoader classLoader = Thread.currentThread().getContextClassLoader();
try {
myProperties.load(classLoader.getResourceAsStream("my.properties"));
}
How can I configure this in Maven?
I'm using:
maven-compiler-plugin version 2.1
jersey-test-framework-grizzly2 version 1.13
UPDATE (Resolved based on accepted answer):
I noticed the text skip non existing resourceDirectory:
[INFO] --- maven-resources-plugin:2.4.3:testResources (default-testResources) # xxx ---
[WARNING] Using platform encoding (MacRoman actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /xxx/xxx/xxx/xxx/src/test/resources
Turns out that I had misspelled resources, after fixing it, everything works as outlined in the accepted answer.
While I was wasting my time looking for workarounds, I did find some interesting links for configuring properties files based on profiles:
How can I change a .properties file in maven depending on my profile?
http://maven.apache.org/guides/mini/guide-building-for-different-environments.html
When running something from the src/test/java folder, the default behavior is:
It will pick up the files in src/main/resources;
Except when there is a file with the same name in src/test/resources.
So basically it will "overwrite" the content of src/main/resources with the content of src/test/resources:
If you got a file in both folders, the one in src/test/resources will prevail.
If you got a file only in src/main/resources, it will be used.
If you got a file only in src/test/resources, it will be used.

Resources