i am using maven2 , hudson and sonar
while doing sonar analysis - i would like some way to append the Hudson build# to the maven version of the project
The project version changes every 2 weeks - so take an example in the first 2 weeks :
<version>abc-SNAPSHOT</version>
after two weeks the next version could be something like :
<version>xyz-SNAPSHOT</version>
what I want is to append the build# to the version already present in pom - which is being picked up and passed to sonar
NOTE:
-Dsonar.projectVersion=xyz-SNAPSHOT-${BUILD_NUMBER}
Here - I am hardcoding the version and dynamically passing the build#
what I want is to be able to dynamically pick up the version from maven ( without changing it ) and simply appending the build# dynamically
any ideas of how this can be achieved ?
Thanks,
satish
You can use Groovy script to read the version and put on environment variable:
Getting Maven Version in Jenkins
I used this script on time to parse the version:
def project = new XmlParser().parse("/pom.xml")
def artifactId = project.artifactId.text().trim()
def version = project.version.text().trim()
println "ArtifactId: [" + artifactId + "]"
println "Version: [" + version + "]"
After set "Additional properties" field with:
-Dsonar.projectVersion=${MAVEN_VERSION}-${BUILD_NUMBER}
If Jenkins don't get the variable, try install:
https://wiki.jenkins-ci.org/display/JENKINS/EnvInject+Plugin
You can use this Jenkins feature: https://github.com/jenkinsci/jenkins/pull/933/files
Basically you can access env variables at build, mapped by Jenkins from Maven GAV info
POM_DISPLAYNAME
POM_DISPLAYNAME
POM_VERSION
POM_GROUPID
POM_ARTIFACTID
POM_PACKAGING
POM_RELATIVEPATH
This is not really a Sonar issue. It's Maven project versioning.
Snaphots are designed to be dynamic. You'll discover under the hood that Maven is saving time-stamped versions in your Maven repository (I don't know how these time-stamps could be passed onto Sonar, which perhaps is what you'd like). Snapshot builds are ideal for CI jobs. software that is never used by non-developers.
What's I'd recommend is either stop using snapshots (build a "release" each time), or just accept the fact that there are two kinds of build in Maven. My rule is that if the code is being used by others then it's no longer a snapshot and should be treated as a release. This creates challenges for emerging methodologies like continuous deployment..... If every build goes to production (or production copy) then snapshots become irrelevant.
Building a release everytime is not that bad. First of all naming convention, I'd recommend:
<major>.<minor>.<patch>.<hudson build num>
The Maven's release plugin will automate most of the pain in managing the details of the release (updating the POM, tagging your SCM system). Finally there is also a useful M2_Release plugin that enables you to trigger a release from Hudson GUI.
Thanks To Andre for pointing me to the other link
I did try it but was running into some issues as noted in the comments
Assuming what I have done is just a workaround and there is a better solution ?
( I am using Hudson )
So I defined a new Job in Hudson ( Build a Maven 2/3 project (Legacy) )
Here I defined a "Groovy PostBuild"
Copied the code in the link that Andre pointed me to :
import hudson.model.*;
import hudson.util.*;
def thr = Thread.currentThread();
def currentBuild = thr?.executable;
def mavenVer = currentBuild.getParent().getModules().toArray()[0].getVersion();
def newParamAction = new hudson.model.ParametersAction(new
hudson.model.StringParameterValue("MAVEN_VERSION", mavenVer));
currentBuild.addAction(newParamAction);
I then "Trigger parameterized build on other projects " and mentioned the job in which I wanted the maven version as mentioend in the pom
Defined a parameter - ${MAVEN_VERSION}
I was then able to get the value of this parameter in the "other" job
So first thanks to Andre - he provided me a solution
I am curious to know if this is a good approach
another question I have is ( which maybe I will start a new thread ) is - how does a Hudson "Free style" job differ from a "Maven 2/3 legacy project"
The reason I ask is the same Groovy script failed in a "free style" while it worked in a "Maven legacy"
Thanks,
satish
Had the same need and solved as suggested with Groovy parsing the pom.
import jenkins.util.*;
import jenkins.model.*;
def thr = Thread.currentThread();
def currentBuild = thr?.executable;
def workspace = currentBuild.getModuleRoot().absolutize().toString();
def project = new XmlSlurper().parse(new File("$workspace/pom.xml"))
def param = new hudson.model.StringParameterValue("project.version", project.version.toString())
currentBuild.addAction(new hudson.model.ParametersAction(param));
Add this script as a post step of type "Execute system Groovy script" (so it's not needed to install Groovy) and paste the code in the "Groovy command".
Related
This article describes an interesting feature of Gradle 4.10+ called a source dependency:
https://blog.gradle.org/introducing-source-dependencies
It allows to use a Git (for example a GitHub) source code repository to build a dependency from it. However it seems like it supports only Gradle projects as source dependencies. Is it possible to use a Maven project as well and if it's possible, please show an example.
When I try to use this feature with Maven project Gradle tries to find the build.gradle file there anyway (I see it when run Gradle with the --info option) and fails with an error message like:
Git repository at https://github.com/something/something.git did not contain a project publishing the specified dependency.
The short answer
... is: "no".
Under the hood, source dependencies are composite builds. These needs to be Gradle projects as the external projects are sort of merged with the main project.
The long answer
... is: "yes but it is hard".
It is actually mentioned in the same blog post you linked to (emphasis mine):
Source dependencies make these use cases simpler to implement. Gradle takes care of automatically checking out the correct versions of dependencies, making sure the binaries are built when required. It does this everywhere that the build is run. The checked out project doesn’t even need to have an existing Gradle build. This example shows a Gradle build consuming two source dependencies that have no build system by injecting a Gradle build via plugins. The injected configuration could do anything a regular Gradle plugin can do, such as wrapping an existing CMake or Maven build.
Because it sounded like it wasn't the biggest thing in the world to create bridge between a Maven and a Gradle project in source dependencies, I gave it a shot. And I have it working except for transitive dependencies. You will basically need to do what is shown in the examples linked to above, but instead of building native libraries, you make a call-out to Maven (e.g. using a Maven plugin for Gradle).
However, the scripts I ended up with are complex enough that I would suggest you instead build the Maven project yourself, deploy it to a local Maven repository and then add that repository to the Gradle project.
<edit>
The loooooooong answer
Alright, so here is how to actually do it. The feature is poorly documented, and appears to be mostly targeted towards native projects (like C++ or Swift).
Root project setup
Take a normal Gradle project with the Java plugin applied. I did a "gradle init" in an empty folder. Assume that in this project, you are depending on a library called `` that you later want to include as a source dependency:
// [root]/build.gradle
dependencies {
implementation 'org.example:my-maven-project:1.1'
}
Note that the version number defined here must match a Git tag in the repository. This is the code revision that will be checkout out.
Then in the settings file, we define a source dependency mapping for it:
// [root]/settings.gradle
rootProject.name = 'my-project'
includeBuild('plugins') // [1]
sourceControl {
gitRepository("https://github.com/jitpack/maven-simple") { // [2]
producesModule("org.example:my-maven-project") // [3]
plugins {
id "external-maven-build" // [4]
}
}
}
[1]: This includes a Gradle project called plugins that will be explained later.
[2]: This is just an arbitrary Maven project that I found, which was relatively simple. Substitute with the actual repository you have.
[3]: This is the name of the Maven module (the same as in the dependency block) that we are defining a source build for
[4]: This defines a custom settings plugin called external-maven-build that is defined in the plugins project, which will be explained later.
Plugins project structure
Inside the root project, we define a new Gradle project. Again, you can use gradle init to initialize it as a Groovy (or whatever you like) project. Delete all generated sources and tests.
// [root]/plugins/settings.gradle
// Empty, but used to mark this as a stand-alone project (and not part of a multi-build)
// [root]/plugins/build.gradle
plugins {
id 'groovy'
id 'java-gradle-plugin' // [1]
}
repositories {
gradlePluginPortal() // [2]
}
dependencies {
implementation "gradle.plugin.com.github.dkorotych.gradle.maven.exec:gradle-maven-exec-plugin:2.2.1" // [3]
}
gradlePlugin {
plugins {
"external-maven-build" { // [4]
id = "external-maven-build"
implementationClass = "org.example.ExternalMavenBuilder"
}
}
}
[1]: In this project, we are defining a new Gradle plugin. This is a standard way to do that.
[2]: To invoke Maven, I am using another 3rd party plugin, so we need to add the Gradle plugin portal as a repository.
[3]: This is the plugin used to invoke Maven. I am not too familiar with it, and I don't know how production ready it is. One thing I noticed is that it does not model inputs and outputs, so there are no built-in support for up-to-date checking. But this can be added retrospectively.
[4]: This defines the custom plugin. Notice that it has the same ID as used in the settings file in the root project.
Plugin implementation class
Now comes the fun stuff. I chose to do it in Groovy, but it can be done in any supported JVM languages of cause.
The plugin structure is just like any other Gradle plugin. One thing to note is that it is a Settings plugin, whereas you normally do Project plugins. This is needed as it we are basically defining a Gradle project at run-time, which needs to be done as part of the initialization phase.
// [root]/plugins/src/main/groovy/org/example/ExternalMavenBuilder.groovy
package org.example
import com.github.dkorotych.gradle.maven.exec.MavenExec
import org.gradle.api.Plugin
import org.gradle.api.artifacts.ConfigurablePublishArtifact
import org.gradle.api.initialization.Settings
class ExternalMavenBuilder implements Plugin<Settings> {
void apply(Settings settings) {
settings.with {
rootProject.name = 'my-maven-project' // [1]
gradle.rootProject {
group = "org.example" //[2]
pluginManager.apply("base") // [3]
pluginManager.apply("com.github.dkorotych.gradle-maven-exec") // [4]
def mavenBuild = tasks.register("mavenBuild", MavenExec) {
goals('clean', 'package') // [5]
}
artifacts.add("default", file("$projectDir/target/maven-simple-0.2-SNAPSHOT.jar")) { ConfigurablePublishArtifact a ->
a.builtBy(mavenBuild) // [6]
}
}
}
}
}
[1]: Must match the Maven module name
[2]: Must match the Maven module group
[3]: Defines tasks like "build" and "clean"
[4]: The 3rd party plugin that makes it more easy to invoke Maven
[5]: For options, see https://github.com/dkorotych/gradle-maven-exec-plugin
[6]: Adds the Maven output as an artifact in the "default" configuration
Be aware that it does not model transitive dependencies, and it is never up-to-date due to missing inputs and outputs.
This is as far as I got with a few hours of playing around with it. I think it can be generalized into a generic plugin published to the Gradle portal. But I think I have too much on my plate as it is already. If anyone would like to continue on from here, you have my blessing :)
I have Jenkins CI pipeline which releases applications. This process works in a way that person triggers release job for application. This job checks all project dependencies via gradle dependencies command. For all dependencies that are snapshot release jobs are triggered automatically.
Release job simply upgrade lib/application version and deploys it in artifactory.
How can I automaticaly upgrade SNAPSHOT dependency version to release version in gradle?
My build.gradle file looks like this:
Properties versions = new Properties()
versions.load(new FileInputStream(rootProject.projectDir.path + "/version.properties"))
dependencies {
compile("projectA:${versions.projectAVersion}")
compile("projectB:${versions.projectBVersion}")
}
and version.properties file
projectAVersion=1.1.0-SNAPSHOT
projectBVersion=1.1.0-SNAPSHOT
In fact I am looking something similar to maven versions plugin.
Is it possible to automatically upgrade version numer inside build.gradle? How?
And harder version - is it to possible to upgrade version number when version is in external version.properties file?
EDIT
In fact I just need Maven versions-plugin (versions:use-releases and versions:use-next-releases) functionality in Gradle.
I am not that clear with your question. What I understood is that, you need to dynamically update the version value after each build.
What you can do is, get the properties value. Remove -SNAPSHOT. Update 1.1.0 with increment after each build. Like 1.1.1, 1.1.2 etc.
This can be done by
task testing {
Properties props = new Properties()
//getting and loading the property file
//Give proper path to file
File propsFile = new File('version.properties')
props.load(propsFile.newDataInputStream())
//Now strip of -SNAPSHOT and get the last digit and increment it by 1
Integer rand = props.getProperty('lastDigit').toInteger()+1
String variable=rand.toString()
//Append -SNAPSHOT with 'variable and set the property
props.setProperty('version',variable)
props.store(propsFile.newWriter(), null)
}
This will work, if my understading of your problem is correct.
I have a Gradle project with subprojects that I can issue separate build commands if I don't want to build all the subprojects at once. For example,
parent
subprojectA
subprojectB
subprojectC
I can then issue commands like ./gradlew :subprojectA:assemble :subprojectC:assemble. What I like to do is construct a meaning git tag from each subproject group and version values for those subprojects that are being built, i.e., group and version for only subprojectA and subprojectC in this case. I am thinking of writing a standalone plugin to do this but unsure where / how to get this information at build time. Any suggestions would be much appreciated.
Both group and version (as well as any other project property) is available globally in your build script.
task myTask << {
println group + "-" + version
}
If you are writing a binary plugin you can also access properties off the Project object itself via the property() method.
def group = project.property('group')
Edit
If you want to determine if a particular project is being built you can inspect the TaskExecutionGraph.
gradle.taskGraph.whenReady { graph ->
if (graph.hasTask(':subprojectA:assemble')) {
println 'Will build subprojectA'
}
}
I have a simple use case of building an OSGi bundle using Gradle build tool. The build is successful if there are java files present in the build path, but it fails otherwise.
I am using 'osgi' plugin inside the gradle script and trying to build without any java files. The build always fails with following error:
Could not copy MANIFEST.MF to
I am sure there must be some way to do it in Gradle but not able to fine. Any idea what can be done to resolve this depending on your experience.
I ran into this today as well, and #Peter's fix didn't work for me (I hadn't applied the java plugin in the first place...). However, after hours of Googling I did find this thread, which helped me find the problem.
Basically, it seems that the error occurs (as Peter stated) when no class files are found in the jar - my guess is because the plugin then cannot scan the classes for package names on which to base all the Import and Export information.
My solution was to add the following to the manifest specification:
classesDir = theSourceSet.output.classesDir
classpath = theSourceSet.runtimeClasspath
In my actual build code, I loop over all source sets to create jar tasks for them, so then it looks like this:
sourceSets.each { ss ->
assemble.dependsOn task("jar${ss.name.capitalize()}", type: Jar, dependsOn: ss.getCompileTaskName('Java')) {
from ss.output
into 'classes'
manifest = osgiManifest {
classesDir = ss.output.classesDir
classpath = ss.runtimeClasspath
// Other properties, like name and symbolicName, also set based on
// the name of the source set
}
baseName = ss.name
}
}
Running with --stacktrace indicates that the osgi plugin doesn't deal correctly with the case where both the osgi and the java plugins are applied, but no Java code is present. Removing the java plugin should solve the problem.
I had the same issue also when java code was present.
Adding these two lines to the osgiManifest closure fixed the problem:
classesDir = sourceSets.main.output.classesDir
classpath = sourceSets.main.runtimeClasspath
-- erik
Is there any way a Jenkins build can be aware of the Maven version number of a project after processing the POM?
I've got some projects where versioning is controlled by Maven, and in a post-build job we'd like to create a Debian package and call some shell scripts. What I need is for the version number that Maven used to be available as a Jenkins environment variable so I can pass it to post-build actions.
To be clear, I'm not needing to know how to get Jenkins to pass a version number to Maven; instead I want Maven to pass a version number to Jenkins!
You can use the ${POM_VERSION} variable, which was introduced with https://issues.jenkins-ci.org/browse/JENKINS-18272
After a lot of digging around (I never realised how poorly-documented Jenkins is!) I found a quite trivial solution.
Install the Groovy plugin
Add a Post Step to your Maven build of type Execute **system** Groovy script
Paste in the following snippet of Groovy:
Script:
import hudson.model.*;
import hudson.util.*;
def thr = Thread.currentThread();
def currentBuild = thr?.executable;
def mavenVer = currentBuild.getParent().getModules().toArray()[0].getVersion();
def newParamAction = new hudson.model.ParametersAction(new hudson.model.StringParameterValue("MAVEN_VERSION", mavenVer));
currentBuild.addAction(newParamAction);
The build environment variable called MAVEN_VERSION will now be available for substitution into other post-build steps in the usual manner (${MAVEN_VERSION}). I'm using it for Git tagging amongst other things.
As other answers already pointed out, if you are using the Maven project type, you have access to the $POM_VERSION variable. But if you are not, you can use this sequence of steps (ugly but reliable). Doing it this way relies on the same version of maven to determine the pom version (while handling complex parent/child pom inheritance where <version> may not even be present for the child).
Maven step with this goal:
org.apache.maven.plugins:maven-help-plugin:2.1.1:evaluate -Dexpression=project.version -l version.log
Shell step: (You may need to adjust the path to version.log depending on your hierarchy)
echo "POM_VERSION=$(grep -v '\[' version.log)" > props.properties
Inject Environment Variables step (Environment Injector Plugin):
Properties File Path: props.properties
Now you can use $POM_VERSION as if this were a Maven project.
What this does: Uses maven to print out the version together with a mess of output, then greps out the mess of output leaving just the version, writes it to a file using properties file format and then injects it into the build environment. The reason this is better than a one-liner like mvn ..... | grep -v '\[' is that using a Maven step does not make assumptions about the installed maven versions and will be handled by the same auto-installation as any other maven steps.
I used Pipeline Utility Steps plugin in a declarative pipeline job to get Maven version. In the example below I use script variable instead of environment variable, because that can be modified and passed between stages.
def TAG_SELECTOR = "UNINTIALIZED"
pipeline {
agent any
stages {
stage('Build') {
steps {
sh "mvn --batch-mode -U deploy"
script {
TAG_SELECTOR = readMavenPom().getVersion()
}
echo("TAG_SELECTOR=${TAG_SELECTOR}")
}
}
}
}
Note: You must approve the getVersion() method after creating the job in Manage jenkins > In-process Script Approval.
See also:
readMavenPom documentation
We used the Groovy Postbuild Plugin.
String regex = '.*\\[INFO\\] Building .+ (.+)';
def matcher = manager.getLogMatcher(regex);
if (matcher == null) {
version = null;
} else {
version = matcher.group(1);
}
Adding this to Jenkins for use later is a bit tricky. Give this a shot, although I remember this causing us some headaches. (Sorry, we did this a long time ago)
def addBuildParameter(String key, String value) {
manager.build.addAction(new hudson.model.ParametersAction(new hudson.model.StringParameterValue(key,value)));
}
Had the same need and solved as suggested with Groovy parsing the pom.
import jenkins.util.*;
import jenkins.model.*;
def thr = Thread.currentThread();
def currentBuild = thr?.executable;
def workspace = currentBuild.getModuleRoot().absolutize().toString();
def project = new XmlSlurper().parse(new File("$workspace/pom.xml"))
def param = new hudson.model.StringParameterValue("project.version", project.version.toString())
currentBuild.addAction(new hudson.model.ParametersAction(param));
Add this script as a post step of type "Execute system Groovy script" (so it's not needed to install Groovy) and paste the code in the "Groovy command".
Execute the Maven Plugin "exec-maven-plugin" in "Execute Shell" as a "Conditional step" worked for me:
mvn -q -Dexec.executable="echo" -Dexec.args='${projects.version}' --non-recursive org.codehaus.mojo:exec-maven-plugin:1.3.1:exec
Integrate in Jenkins:
-> "Add post-build step"
-> "Conditional steps (single or multiple)"
-> "Execute Shell:"
export MY_POM_VERSION=`mvn -q -Dexec.executable="echo"
-Dexec.args='${projects.version}' --non-recursive org.codehaus.mojo:exec-maven-plugin:1.3.1:exec` && [[
"${MY_POM_VERSION}" == "THE_VERSION_TO_BE_MATCHED" ]] && echo
"CONDITION_IS_MET"
-> "Steps to run if condition is met"
-> Add any build step you need
Notes:
THE_VERSION_TO_BE_MATCHED has to exchanged with your version
'&& echo "CONDITION_IS_MET"' is only for debugging purposes. For the same purpose you can add a '&& echo "MY_POM_VERSION=${MY_POM_VERSION}"' after the mvn command in order to understand what's going on.
This approach is more reliable than a "grep" and it could be an alternative if the Jenkins Ruby Plugin is not installed.
You could also do :
MAVEN_VERSION=`grep A -2 -B 2 "<your_project_name>" pom.xml | grep version | cut -d\> -f 2 | cut -d\< -f 1`-commit-"`echo $GIT_COMMIT`"
Explanation: assuming that you have your project name within a line or two above/below version like a normal pom:
<groupId>org.apache.bigtop</groupId>
<artifactId>bigpetstore</artifactId>
<version>1.0-SNAPSHOT</version>
Then you can easily grep for the artifactId, use the "before/after" grep actions to suck in the version with it, and then grep the version out and use the simple unix "cut" command to splice out the content between "version" tags.
I like the Jenkins-groovy integration, but this is alot easier and will work even on a build server which you dont have control over (i.e. because bash is universal).
Solution:
POM_VERSION=$( \
xmlstarlet sel \
-N x='http://maven.apache.org/POM/4.0.0' \
-t \
-v '//x:project/x:version/text()' \
pom.xml \
)
Explanation:
You can do this in a one-liner using a command-line XPath tool, such as those mentioned at "How to execute XPath one-liners from shell?". I chose XMLStarlet, but they all have similar syntax.
When parsing a POM, you have to account for namespaces. The docs here helped me figure this out.
In order to get the text for an element in XPath, you use the text() function as explained at XPath: select text node.
My POM looks like this:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.foo.bar</groupId>
<artifactId>foobar</artifactId>
<version>1.0.6-SNAPSHOT</version>
<packaging>jar</packaging>
The downside here is that if the namespace changes, you have to change the command.
Using 'Execute System Groovy Script' as follows:
import jenkins.util.*;
import jenkins.model.*;
def thr = Thread.currentThread();
def currentBuild = thr?.executable;
def projectManager = build.getProject()
def file = projectManager.getWorkspace().child("pom.xml");
def project = new XmlSlurper().parseText(file.readToString())
def param = new hudson.model.StringParameterValue("currentVersion", project.version.toString())
currentBuild.addAction(new hudson.model.ParametersAction(param));
By using Execute System Groovy script you have direct access to the build, from which you can get the project and thus the "child" file in this case pom.xml.
You won't have to create a new file and as you can see it offers very powerful access to every file within the workspace.
Based on #Akom`s answer the pre steps to have POM_VERSION are:
"Inject environment variables" with property
file your_property_file. Note if you select "Inject environment variables to the build process" the file needs to exist in the jenkins workspace.
run in a pre step execute shell the follwing
bash script.
Script
mvn org.apache.maven.plugins:maven-help-plugin:evaluate -Dexpression=project.version -l project_version
# grep for the version pattern rather than not mentioning '\['
echo "POM_VERSION=$(grep -E '^[0-9.]+(-SNAPSHOT)?$' project_version)" > your_property_file