A few basic questions on ci/cd pipelines.
When we build java code, do we create jar file before going for sonarqube analysis or does both happen simultaneously. My understanding is sonarqube analysis needs to be performed before maven build. Build should happen only if codequality crosses our quality checks.
Does sonar scanner and maven are used individually or sonar scanner is integrated with maven. I know both are possible but what is the best way that we need artifacts to be created only if code passes quality checks.
How does the sonarqube tell CI system (be it azuredevops or any other system) whether to go for next steps or break if the quality check is failed.
Usually you run your full build (which contains building the jar file or in general artifacts) and the sonar analysis will be done afterwards (unit tests coverage, static code analysis etc.) and no it is not done before it's done afterwards otherwise it would not be possible to integrate results like code coverage of the unit/integration test into the sonarqube analysis.
Technically the sonar scanner can be triggered via the Maven build (it is done via a maven plugin) and often called like this: mvn verify sonar:sonar(assumed that it is configured correctly).
SonarQube has a webhook which will be called/triggered if the quality is not as expected. Most of the time the CI/CD system have a stage which will shows the result of that and makes the final result of the build "red". Also many source code hosting solutions (GitHub, GitLab, Gitea or alike) having indicators which shows that (usually) within a pull request...
Update:
If you run sonar analysis on a project without compiling the code you will get this:
$ mvn clean sonar:sonar
[INFO] JavaClasspath initialization
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 2.952 s
[INFO] Finished at: 2022-12-04T21:41:34+01:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.sonarsource.scanner.maven:sonar-maven-plugin:3.9.1.2184:sonar (default-cli) on project kata-fraction:
Your project contains .java files, please provide compiled classes with sonar.java.binaries property, or exclude them from the analysis with sonar.exclusions property. -> [Help 1]
[ERROR]
Related
Hi I am automating code coverage on SonarCloud using Travis CI for a Maven application.
Now, running the sonar:sonar command locally submits the report on SoundCloud and I can see it as shown below with Branch master
Now I have also configured .travis.yml file to auto push the report to SoundCloud on each build as below
But when build is triggered by Travis CI it fails with following error
[INFO] Load project branches
[INFO] Load project branches (done) | time=114ms
[INFO] Load project pull requests
[INFO] Load project pull requests (done) | time=116ms
[INFO] Load branch configuration
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 27.884 s
[INFO] Finished at: 2019-05-19T16:47:23Z
[INFO] Final Memory: 93M/496M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.sonarsource.scanner.maven:sonar-maven-plugin:3.6.0.1398:sonar (default-cli) on project safenest-java-server:
Project was never analyzed. A regular analysis is required before a branch analysis`
I am new to SonarCloud and couldn't find much documentation to get help with this. Can somebody explain what exactly A regular analysis is required mean? And how to fix this?
Thanks in advance.
You should check following parameters:
Project key generated during maven build and generated on SonarCloud (or make sure that you set sonar.projectKey property that was generated on SonarCloud).
Check if provided Token value matching the one on SonarCloud (you can provide your own value on project setup page.
The error message means:
Please analyze main branch, before you will analyze other branches
You have to build main repository branch (usually master) with this configuration on Travis. After that you should be able to analyze all other branches. I hit the same problem when I was trying to analyze a feature branch which integrates my project with SonarCloud for the first time. I just merged my feature to master, pushed, and my project has been analyzed successfully. My configuration is similar to yours:
language: java
jdk: openjdk8
env: MVN_VERSION='3.6.0'
addons:
sonarcloud:
organization: $SONAR_ORGANIZATION
token:
secure: $SONAR_TOKEN
before_install:
- wget https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/$MVN_VERSION/apache-maven-$MVN_VERSION-bin.zip
- unzip -qq apache-maven-$MVN_VERSION-bin.zip
- export M2_HOME=$PWD/apache-maven-$MVN_VERSION
- export PATH=$M2_HOME/bin:$PATH
script:
- mvn -B -e verify site
- if [ -n "$SONAR_TOKEN" ]; then
mvn -B -e sonar:sonar -Dsonar.sources=pom.xml,src/main;
fi
I am trying to gather code coverage metrics on both unit and integration tests run by Maven following directions linked to by the SonarQube Maven plugin guide. I believe I have successfully generated coverage metrics with jacoco: I have files jacoco.exec and jacoco-it.exec, and though I don't know how to look inside them to find useful information, they do have different sizes in accordance with different numbers of unit and integration tests, FWIW.
From there I can run e.g. mvn sonar:sonar to upload unit test coverage to my local server, but I have not been able to upload integration test coverage. In my pom file I have a property <sonar.junit.reportPaths>target/failsafe-reports,target/failsafe-reports</sonar.junit.reportPaths> which seems to do nothing. I have tried both the latest official and codehaus versions of sonar-maven-plugin (version 3.3.0.603). I am using the latest official SonarQube Docker image (v. 6.4) which evidently contains SonarQube Version 6.3.1 according to the SQ UI (why the discrepancy there?).
I noticed in the console output when running mvn sonar:sonar, it says
[INFO] Sensor SurefireSensor [java]
[INFO] parsing /home/jason/.../target/surefire-reports
[INFO] Sensor SurefireSensor [java] (done) | time=57ms
[INFO] Sensor JaCoCoSensor [java]
[INFO] Analysing /home/jason/.../target/jacoco-it.exec
[INFO] Analysing /home/jason/.../target/jacoco.exec
[INFO] Analysing /home/jason/.../target/sonar/jacoco-merged.exec
so it's picking up both jacoco coverage reports, yet showing only unit tests in the UI. Since that output mentions target/surefire-reports but never mentions failsafe-reports, as an experiment, I tried copying everything from failsafe-reports to surefire-reports, and the IT information was present in the SQ UI, but that is a hack, and I found a similar SO question advising configuring failsafe to output to surefire-reports to trick SQ in this way.
So my ultimate question is how do I get SQ to upload IT information? Simpler questions are, how do I tell it to parse failsafe-reports, and what difference does sonar.junit.reportPaths make?
I think this link should have you covered: https://theholyjava.wordpress.com/2012/02/05/separating-integration-and-unit-tests-with-maven-sonar-failsafe-and-jacoco/
It is likely showing a combined number rather than a breakdown. I am having the same issue with gradle. The other answer is referring to an older version of sonarqube. sonar.phase property is dropped since v4.2.
Here's the link to how it is done in more recent version:
https://github.com/SonarSource/sonar-examples
I hope this helps.
We have a multi-module project built with apache-maven 3.0.5.
Here is the setup:
Running Sonar plugin through Jenkins.
SonarQube server along with MySQL resides on another computer.
Use Sonar plugin on Jenkins to run analysis which then pushes results to SonarQube server.
Here are the warnings I am getting when running a Sonar job.
**These warnings does not seem to have an effect on server results. SonarQube server displays rule violations and no classes seem to be missing there.
Before analysis :
[WARNING] The following dependencies could not be resolved at this point of the build but seem to be part of the reactor:
15:04:52 [WARNING] o com.company.product.plugins:com.company.product.external.libraries:jar:1.0.0-SNAPSHOT (provided)
15:04:52 [WARNING] Try running the build up to the lifecycle phase "package"
15:04:52 [WARNING] The following dependencies could not be resolved at this point of the build but seem to be part of the reactor:
15:04:52 [WARNING] o com.company.product.plugins:com.company.product.capturetool.core:jar:1.0.0-SNAPSHOT (provided)
During analysis :
Class 'com/company/product/core/tools/ocraction/MyClass' is not accessible through the ClassLoader.
[WARN] [15:05:25.731] Class 'com/company/product/core/tools/ocraction/MyClass' is not accessible through the ClassLoader.
After analysis :
[INFO] com.company.product.core.feature ................ SKIPPED
[INFO] com.company.product.common.feature ... SKIPPED
[INFO] My Product ............................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
I am confused as to why this is happening. I have scoured the internet without answers. I've got responses such as .m2 directory corruption to running "mvn clean package" instead of "mvn clean install" before Sonar analysis.
I managed to fix this issue. I figured out that Sonar Jenkins plugin does not actually run the command "mvn clean install" by itself. Instead, it merely runs "mvn sonar:sonar" along with any additional arguments.
I executed "mvn clean install" before Sonar's post-build step. This helped to resolve my issue.
I have a little bit of strange Problem with Jenkins, Maven and Tycho and it is hard to find out who is the culprit.
All SCM Triggered Builds fail but all manually triggered builds succeed.
Jenkins Version : Jenkins ver. 1.527
Maven Version : 3.05
I have a modularized tycho build:
<modules>
<module>../main.plugin.test</module>
<module>../main.plugin.internationalization.at</module>
<module>../crud.plugin</module>
<module>../rest.plugin</module>
<module>../main.plugin</module>
<module>../main.feature</module>
<module>../product</module>
<module>../target-definition</module>
<module>../rest.plugin.test</module>
</modules>
Jenkins is configured as a simple maven build with modules.
It just executes:
-X clean deploy
When an SCM-Build is triggered some modules are not build.
When I build it manually everything is fine:
This behaviour is consistent. I already tried
switching to a different Maven version (3.05 / 3.04)
deleting the whole workspace prior to building
clean checkout of all sources
running of -X deploy (without clean)
Any amount of manual invocations succeeds. And any amount of scm triggers fails.
Both Maven logs look exactly the same until (working):
[INFO] Reactor Build Order:
[INFO]
[INFO] client-master
[INFO] crud-plugin
[INFO] main-plugin
[INFO] ------------.main.plugin.test
[INFO] ------------.main.plugin.internationalization.at
[INFO] rest-plugin
[INFO] main-plugin
[INFO] ------------.product
[INFO] target-definition
[INFO] ------------.rest.plugin.test
and (failing):
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] main-plugin
[INFO] ------------.plugin.test
[INFO] ------------.main.plugin.internationalization.at
[INFO] rest-plugin
[INFO] main-plugin
[INFO] ------------.product
[INFO] ------------.rest.plugin.test
The final error message therefore is:
Caused by: java.lang.IllegalStateException: ------------..client:------------..crud.plugin:eclipse-plugin:1.0.0-SNAPSHOT does not provide an artifact with classifier 'null'
Customer specific module names are replaced with ------------. in this question.
I have heard repeatedly of vague problems with Tycho builds on Jenkins. The reason for these problems seems to be that some Jenkins plugin triggering these builds hooks into the Maven lifecycle and this somehow collides with what Tycho does in the Maven internals.
For the problem that you are describing, it seems that the Jenkins plugin that you are using is changing the module build order. This may be okay for a normal Maven build (where all dependencies are declared in the POMs), but may fail for a Maven/Tycho build, where dependencies are computed by Tycho during the build.
To avoid this problem, you should trigger the Maven build in a way that is closer to a normal command line build. I found that the Invoke top-level Maven targets build step from the Maven Integration plugin works without problems.
I have a Java application and I am able to compile it using maven on CLI, I have a couple of build scripts to deploy the Java code to a remote server and compile the code using Maven but unfortunately the build fails while compiling it using my build script for Jenkins. When I try to compile the code on the remote server the build is successful. The error log is as follows:
[WARNING] The POM for org.im4java:im4java:jar:1.3.2 is missing, no dependency information available
[WARNING] The POM for com.aliyun:aliyun-openservices:jar:1.0.0-20120705 is missing, no dependency information available
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] GuestPath Project ................................. SUCCESS [5.183s]
[INFO] guestchat biz module .............................. FAILURE [4.001s]
[INFO] GuestChat Web Common .............................. SKIPPED
[INFO] GuestChat Portal Webapp ........................... SKIPPED
[INFO] guestchat openapi Webapp .......................... SKIPPED
[INFO] guestchat chat server ............................. SKIPPED
[INFO] guestchat service dashboard Webapp ................ SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 10.518s
[INFO] Finished at: Wed Sep 19 10:49:39 CST 2012
[INFO] Final Memory: 9M/23M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project guestchat-biz:
Could not resolve dependencies for project com.guestops.guestchat:guestchat-biz:jar:1.0.0-SNAPSHOT:
The following artifacts could not be resolved:
org.im4java:im4java:jar:1.3.2, com.aliyun:aliyun-openservices:jar:1.0.0-20120705:
Failure to find org.im4java:im4java:jar:1.3.2 in http://repo.maven.apache.org/maven2
was cached in the local repository, resolution will not be reattempted until
the update interval of central has elapsed or updates are forced -> [Help 1]
I hope I am being clear enough, any help is highly appreciated. I can provide with more details if needed..Thanks in advance!
I think you should try to do the following:
Try to compile the project locally but make sure you're downloading the artifacts from the maven repository. Radically you can do it by renaming your ~user/.m2/repository and running ''mvn clean install''
If it compiles - your local maven is configured properly and its not a pom.xml issue. Also the remote repositories are configured right in maven. In this case its a jenkins issue or rather the issue of maven installed on the same machine with jenkins or maybe some network/security related issue (like a connection to the remote repository is firewalled and not accessible from the jenkins machine and so on).
I would suggest to check out the repository configuration on that maven, or if you want checkout the source code of your project on the jenkins machine and manually run the first step described above from that machine. You should see that the code can't be compiled and get the same error you're getting now.
Now if in during the step 1 the project can't be compiled - its just because you have had a dependency on the lacking module and they were somehow presenting in your local repository (I assume previously it could be compiled locally) and since we're kind of running a clean installation the local repository 'gets purged'. In this case you should find out where do your dependencies come from.
Things like
mvn dependency:tree on the failing module
Can be helpful here.
Hope this will help somehow
2 possible quick solutions:
If it compiles locally, then go to the build configuration on Jenkins and check the "Poll SCM" option to do polling on every build.
Run from the command line on Jenkins server mvn clean install and make sure that the artifacts are available in maven repository after the build. If they are not, download the jars of the relevant versions and put them in the local maven repository (on Jenkins server).
Good luck!
This is what the error message tells you:
The following artifacts could not be resolved:
org.im4java:im4java:jar:1.3.2,
com.aliyun:aliyun-openservices:jar:1.0.0-20120705: Failure to find
org.im4java:im4java:jar:1.3.2 in http://repo.maven.apache.org/maven2
So you have references in your pom for im4java 1.3.2 and aliyun-openservices 1.0.0-20120705, but these cannot be found in the general maven repo. If you do a search for im4java in mvnrepository.com, you'll see that it only has version 1.2, so it is reasonable that it would fail. aliyun-openservices package doesn't exist there at all.
So, where should those be found? Maybe you have installed them locally only and not to Jenkins machine?
What you can do then, is either
(recommended) set up an intranet maven repo where you would deploy the needed libraries and reference that or
install them locally on the machine running Jenkins
Both of these should fix the issue.
A quick look on maven repository, only version 1.2.0 is available and not version 1.3.2 (which you are using).
Does your code compile with a maven build?
I suggest you try to review the repository setting on the jenkins instance