My co-workers and I would like to incorporate SonarQube into our existing projects. Our normal development process for Java projects involves running Checkstyle on code changes to ensure they follow our style rules, committing the project to our code repository and having Jenkins build and package the latest version. We’d like to add SonarQube to this final step (through the Jenkins plugin) but we don’t want to duplicate all of our Checkstyle rules in SonarQube, since this would require us to maintain two separate sets of rules and make things more complicated if we need to make changes to the rules. We don’t want to completely switch to SonarQube since we’d like to still run Checkstyle before we commit code to our repository. We’d also prefer to maintain our own Checkstyle files as the main set of style rules as opposed to maintaining the style rules on SonarQube and downloading the generated XML files for our local development.
So is there any way to “upload” (so to speak) our existing set of Checkstyle XML files to SonarQube for it to use in its evaluation?
Thanks for the help.
AFAIK, it is not possible to have Sonar use the Checkstyle definitions from the repository. Is is, however, possible to have Sonar use the current Checkstyle suppression filter from the repository.
As for the rules definitions, I think you will have to maintain them in Sonar, and when anything is changed, also change the copy in the repository which is used by eclipse-cs. This is redundant, but at least it affects only one person - the rest of the team can reap the benefits. This approach also enables intentional differences, e.g. when some Eclipse-specific issue is checked (say, something concerning source folders, which don't exist in Sonar).
The path to the suppressions filter file can be configured in Sonar to refer to the location where your stuff is checked out for Sonar analysis. So that part can be maintained in the repository without any redundancy.
From my point of view, you should make a choice: use SonarQube or Checkstyle but not both.
Checkstyle on code changes to ensure they follow our style rules => Can't you replace this step with SonarQube. See for example: http://docs.codehaus.org/display/SONAR/Using+SonarQube+in+Eclipse or http://docs.codehaus.org/display/SONAR/Issues+Report+Plugin
Related
We have a directory structure like so
java
build/build.gradle (This does NOT exist yet, but we want this)
servers
server1/build.gradle
server2/build.gradle
libraries
lib1/build.gradle
lib2/build.gradle
We have 11 servers and 14 libraries with varying uses of dependencies. EACH server is a composite build ONLY depending on libraries (we don’t allow servers to depend on each other). In this way, as our mono-repo grows, opening up server1 does NOT get slower and slower as more and more gradle code is added(ie. gradle only loads server1 and all it’s libraries and none of the other libraries OR servers are loaded keeping things FAST).
Ok, so one problem we are running into is duplication now which is why we need build/build.gradle file AND we want EVERY module in our mono repo to include that somehow for a few goals(each goal may need a different solution)
GOAL 1: To have an ext { … } section containing a Map of Strings to gradle dependencies much like so
deps = [
'web-webserver': "org.webpieces:http-webserver:${webpiecesVersion}",
'web-webserver-test': "org.webpieces:http-webserver-test:${webpiecesVersion}",
'web-devrouter': "org.webpieces:http-router-dev:${webpiecesVersion}"
]
In this way, we want ALL our projects to them import dependencies like so
compile deps['web-webserver']
GOAL 2: We want to 'include' a standard list of plugins so we are versioning all gradle plugins the same across the repo. While the above configures all jars to avoid jar hell in a mono-repo, we would like to do the same with just this section
plugins {
id 'com.github.sherter.google-java-format' version '0.9'
}
Of course, it each project may also want to add a few more plugins OR even not depend on this section(in case of an emergency and trying to just get the job done).
GOAL 3: We want checkstyle configuration (or any plugin config) to be defined the SAME for all projects (eventually!!!). We would like the checkstyle gradle to live in a common area but have all libraries somehow pull it in. Again, it would be nice for it to be optional in that, I can pull the gradle section into my build.gradle OR can create a new one in case of emergencies so I don't have to fix all projects in the monorepo right away.
IDEALLY, perhaps I kind of want configuration injection where when I run server1/build.gradle, it actually runs java/build/build.grade as it’s parent somehow but with overrides (IF I declare 'extends xxx.gradle' maybe) then all libraries it uses also use java/build/build.gradle as their parent. I am not sure this is possible or feasible. I am pretty sure 'extends xxx' doesn't exist in gradle.
Are any of these GOALS possible?
thanks,
Dean
I have been working on a monorepo with the exact same requirement as you, using gradle composite builds as well. The way we have solved this problem is by using pre compiled plugins
You need to do a new gradle project with only the code you want to share. This will create a plugin, that you can just add as a composite build and apply to the other projects.
I'm a bit confused by why you don't just use a "standard" gradle top level build file and compose the others as subprojects.
This solves all 3 of your goals
If you are concerned by build speed, you can target each server individually simply by running
./gradlew :server1:build
But if you are not able to do this for some reason you can use the apply from: syntax as described here
This is not related to update the pom dependencies using a maven goal. I already have that sorted out.
So I am responsible for creating, packaging and maintaining common utilities. These common utilities are in turn used by all the teams in the org. Problem is that the teams using these utilities do not update the dependencies unless it is the last resort. We would like them to use the latest release version of our common utilities barring very few.
Now I have come across this Maven Versions Plugin by mojohaus which I think serves my need by using two goals - versions:update-properties and versions:use-latest-releases. It serves my purpose except two things:
I do not see a way to exclude certain groupid:artifactid from the update dependency/property
We really want this to be a compulsory thing (maybe part of the Test execution - this is for test automation utilities mainly) rather than a maven goal. Because if it is a maven goal, it needs to be invoked separately and hence becomes optional for teams.
We know that forcefully updating to latest version might cause some issues with defect re-produciblity, but we are willing to take that risk. Our utilities are really test products.
Any direction/help on this is appreciated.
Edit: We run our tests using maven goals clean install already. So they use existing pom. We want dependencies update to happen before the tests run. Also it is desirable to commit the changes to source control (bitbucket) if possible.
We have our tests setup using Jenkins but teams also run multiple test on local machines.
Edit: Found the answer to #1. The plugin provides to exlude regex for group and artifact id. using tags excludes and excludesList
Sorry for insisting. From the Maven point of view I see two main solutions:
Your projects and your utility jar are tightly coupled and every project always needs to use the latest version. Then you can bundle all projects and your utility in one multi-module project. This makes sure that everything is up to date all the time, but it requires that all projects and your utilities are always build together (not separately).
You distribute your utilities to different projects which build and release at different times. Then it is up to the projects to decide when to update. There is unfortunately no standard way to deprecate jars.
If I understand you correctly, you want something "in the middle". This may be hard to achieve.
We have a maven based Java EE project controlled by the customer. For internal reasons, we cannot execute one of the build steps, but the rest works fine and produces the jar we want.
Since editing the pom file would require taking care when committing to customer's SVN and copying the pom file would require taking care to sync changes comming from there, we are looking for a way to skip this specific step in the build section during the maven call itself, so to say mvn clean install but-leave-out-this-build-plugin-step, is there any?
Edit:
The plugin in question is the rpm-maven-plugin, which prevents the build from running on Windows. We found information how to make it work which won't really fit in our current setup. And since we cannot modify the customer's pom, I was looking for a way to trigger the skipping externally. But maybe there are other ways to just ignore/skip/fake this step?
It depends on what plugin you want to skip. Many plugins have ability to be skipped via system property (-Dblabla).
For deploy plugin it is -Dmaven.deploy.skip=true, for surefire -DskipTests=true.
Read plugin documentation, maybe you can find skip property
The rpm plugin hase a property disabled, unfortunately it is not accessible by a property. So, if setting this property in the customer's pom (or asking for editing it) with a default value of false is an option, this may be the solution.
I have a multi-module maven project and want to generate code coverage reports for subsets of the project and track the results using sonar.
The project has modules M1, M2 and M3 and I want to generate coverage reports for package foo/** and for package bar/**. The packages may exist in multiple modules.
I'm using cobertura to generate code coverage files in my maven project.
My question is: what is best practice for generating multiple code coverage reports for a single project?
One option is to configure my maven build with a profiles to generate coverage data for specific packages, e.g. coverage-foo would configure the cobertura maven plugin with <include>foo/**</include>, and coverage-bar would configure the cobertura plugin with <include>bar/**</include>. Then I would configure sonar with two branches of the project: one for foo and one for bar, with no configuration on the sonar side.
But another option is to generate code coverage data for the entire projet, and configure each sonar project with appropriate inclusions. So the sonar 'foo' branch would have settings that only the foo/** files are included, and the sonar bar branch would have similar settings for the bar package.
Both seem equivalent to me (in terms of sonar output), and I favour the first option, but I'd like to know if this is the 'best practice' method of generating multiple code coverage 'views' of the same multi-module project. Or if one option has disadvantages that I have not considered.
(Note: there is a commercial 'views' plugin for sonar but I am restricted to free plugins for now)
Given our message exchange and the information you provided, I would rather go for a mix of the 2 options in your context, i.e.:
generate code coverage data only for foo (or bar)
have sonarqube analyse your code in a way where a branch is created for foo (or bar) where only foo code (or bar code) is included
Indeed this would allow you to have the information you need about coverage, but will also allow you to only have issues related to foo / bar code.
Nevertheless, you'll have two issues then :
code that is not part of foo nor bar won't be analysed unless you create another branch where you would exclude foo AND bar code
you won't be able to have a global picture of your code quality, to do it you would have to use the portfolio plugin (but as you wrote, you can only use opensource plugins
Note that the 2. point is not that problematic as you can rebuild the information you need using SonarQube's web service (this would allow you to build an external report that would aggregate data you want to be aggregated)
Michel
The code base I am working with has a lot of generated code. In addition, there are also some deprecated files that I would want to exclude from SonarQube analysis. I've read up the documentation and looked at some answers on here about that, but it does not help in my case.
I have a multi-module maven project. So I have multiple projects in my workspace that are all part of a large application. Say I want to exclude this file:
/home/username/workspace/com.mst.rtra.importing.message/bin/com/mst/rtra/importing/message/idl/parse/idlparser.java
I don't really know how to write this in the exclusions settings on SonarQube because of how long the filepath is. Also, what if I want to exclude another file, but from a different module, say :
/home/username/workspace/com.mst.rtra.interpreter.create/
I am confused about I should write this in the exclusions box in project settings. Should I write the absolute file path due to the multi-module nature of this project? Or is there some other convention used?
In addition, if I want to exclude generated files from analysis, I would need to put file:/generated-sources/ as I saw in another answer. However, after analysis, I can still view the analysis results of those files when I open up the project in SonarQube dashboard.
We use ant rather than maven, and an older version of the Sonar ant task at that. But what works for us is setting a sonar.exclusions property in our build.xml, which accepts wildcards for filenames. For example:
<property name="sonar.exclusions" value="**/com/ex/wsdl/asvc/*.java,**/com/ex/wsdl/bsvc/*.java"/>
That skips analyzing all the code generated from a wsdl file for two services. You ought to be able to do something similar for maven.