My project consists of numerous (~100) components, most of them use ~10 core components. When I update a version of a core component I would like to have up-to-date version of it in every component that is using it.
Trouble is that components depend on many other components - and I want only one of these dependencies updated at a time (the one that has just been updated).
I am using versions-maven-plugin and the "update-properties" mojo - trouble is that it requires executing maven for each and every component in order to check if it needs update and doing the actual property update. I'm using Jenkins as a CI solution, tried that writing a custom pipeline and it sure is doable but it's still iterating over a whole project...
I was wondering if there maybe is a more efficient way to do this?
Would creating/maintaining dependency map be feasible here - do you know of any software that could be integrated into the pipeline to do that? (a little clarification here: all of my components are under the same maven groupid and the same project in gitlab).
Related
This is not related to update the pom dependencies using a maven goal. I already have that sorted out.
So I am responsible for creating, packaging and maintaining common utilities. These common utilities are in turn used by all the teams in the org. Problem is that the teams using these utilities do not update the dependencies unless it is the last resort. We would like them to use the latest release version of our common utilities barring very few.
Now I have come across this Maven Versions Plugin by mojohaus which I think serves my need by using two goals - versions:update-properties and versions:use-latest-releases. It serves my purpose except two things:
I do not see a way to exclude certain groupid:artifactid from the update dependency/property
We really want this to be a compulsory thing (maybe part of the Test execution - this is for test automation utilities mainly) rather than a maven goal. Because if it is a maven goal, it needs to be invoked separately and hence becomes optional for teams.
We know that forcefully updating to latest version might cause some issues with defect re-produciblity, but we are willing to take that risk. Our utilities are really test products.
Any direction/help on this is appreciated.
Edit: We run our tests using maven goals clean install already. So they use existing pom. We want dependencies update to happen before the tests run. Also it is desirable to commit the changes to source control (bitbucket) if possible.
We have our tests setup using Jenkins but teams also run multiple test on local machines.
Edit: Found the answer to #1. The plugin provides to exlude regex for group and artifact id. using tags excludes and excludesList
Sorry for insisting. From the Maven point of view I see two main solutions:
Your projects and your utility jar are tightly coupled and every project always needs to use the latest version. Then you can bundle all projects and your utility in one multi-module project. This makes sure that everything is up to date all the time, but it requires that all projects and your utilities are always build together (not separately).
You distribute your utilities to different projects which build and release at different times. Then it is up to the projects to decide when to update. There is unfortunately no standard way to deprecate jars.
If I understand you correctly, you want something "in the middle". This may be hard to achieve.
I am in the processing of integrating Maven into my my projects. While maven has plenty of pros i'm finding it difficult to figure out how to maintain my current development process, which is as follows:
For creating SDKs I will create a sample app, which will depend on and directly reference the SDK source code, all from within the same code project. This means that I can make easily change/debug the SDK code with one click run/debugging.
I fear this won't really be possible with Maven. Can I create some type of Hybrid approach, where I continue my normal development approach and then push builds to Maven when it is appropriate.
Update - For Clarity
My problem is that when everything is done through maven, the dependencies are built and published to Maven. Then, the dependent project pulls down compiled references and uses them. My issues is that I don't want to go through this whole process every time I make a small change to a dependency.Thanks.
You should try creating parent level pom.xml with two modules - your library and simple app to test it. In simple app's pom.xml provide a dependency on library module.
Then open in your IDE parent pom as maven project. This should be sufficient for normal debug.
Other possible approach - install you library artifact into maven repo with sources. In this case you will be able to debug it, but test app still have to load use jars from repo.
We are using the latest version of Jenkins CI and we have a large number of projects, which have Maven dependencies on other projects. We also are using Jenkins views to group associated projects together.
I would like to be able to generate a graphical representation of the project hierarchy within a view. I am aware that if I select a project that I can see the upstream projects, but going through approximately 40 projects, writing this down and compiling it into a tree would be tedious, time-consuming and error-prone.
Does anyone know of a technique or plugin for Jenkins that could achieve this? Ideally it would work against all the projects within a view.
I would prefer an automated technique rather than performing it manually, since this process would need to be run periodically (say once a month) for a management report.
Update
Having investigated this question, I am not averse to writing a script to query the Jenkins API to get the JSON or XML for the projects within a view and then asking each for its upstream projects. But I'd rather save myself some work and using someone else's tool :)
You can use Maven to generate the dependencies for each project (http://maven.apache.org/plugins/maven-dependency-plugin/tree-mojo.html).
It won't give you a dependency tree for all your Jenkins projects though. Maybe you could pull from all maven outputs and create your own? Or maybe (not really) create a super project in which all modules are your existing projects (again, not really).
There is a Downstream Buildview plugin. It's per job, and it displays job names, but if you job names are named after maven modules, it shouldn't be an issue.
What are the best practices for software versioning and multimodules projects with Maven?
I mean, when I create a multimodules project with Maven, what is the best approach for the versioning? To use a single version for all the modules (defined in the top project)? To use a version for each module (defined in the POM of each module)? Is there another approach that I'm missing? What are the pros and cons of each approach?
In general, are the different modules released together (possibly sharing the same version number)?
Thanks
Honestly it depends on what you would like to do. Multimodule projects are created for multiple reasons, one of them being you only need to deploy what has changed instead of all modules.
Think about it this way: if you had a non-multi-module project and you only had to change one line in the services layer, you have to rebuild the entire project and deploy all of the code again...even though only your services layer will change.
With multi-module projects, you can regenerate your project and deploy only what changed...your services. This reduces risk and you're assured that only your services module changed.
You also have a multitude of benefits to using multi-module projects that I'm not listing here but there is certainly a huge benefit to NOT keeping your version numbers of your modules in sync.
When you build your project, consider deploying it to a repository that will hold all compatible jars together for builds (each build creates a new folder with the parent-most pom version number). That way, you don't need to keep documentation about which jars are compatible...they're all just deployed together with a build number.
I was looking for a solution for this exact problem myself and versions-maven-plugin was exactly what I needed. I don't like the release plugin communicating with the SCM system. The versions plugin does just what we need: it sets a new version number in all poms of the project:
mvn versions:set -DnewVersion=2.0.0
Then I can proceed with commits, tags and an official build server build...
EDIT:
The versions plugin depends on how a maven multi-module project has been organised: as a result, it often does not update all POM files in a complex multi-module project.
I've found that sed and find do the job much more reliably:
sed -i 's/1.0.0-SNAPSHOT/1.0.0-RC1/g' `find . -name 'pom.xml'`
Typically you create a multi-module project because you have deemed that the various modules are parts of a single whole. Maybe the client-piece, the controller-piece and the services-piece. Or maybe the UI with services.
In any case, it makes sense to have the version numbers for the various modules to move in lock-step. However Maven does not enforce that as a rule.
As to your question
are the different modules released together (possibly sharing the same
version number)
I would think so. That is one of the reasons for having it a multi-module project. Otherwise you could have the modules as independent projects.
Of course this is the kind of stuff that is rife with edge cases and exceptions ;-)
I had the same problem with a project I`m working on. I also decided to use separate versions and even the dependency to the parent pom only has to be updated if some of the managed dependencies change. (so mostly as #vinnybad describes it)
Two additions
exists-maven-plugin
With the usage of the "org.honton.chas.exists-maven-plugin" only the modules will be deployed to the repository that have actually changed, which is really great, because also the according docker-images will only be published if something has changed on one of the service. This avoids "polluting" the image repository with different but unchanged versions.
versioning
One main downside of the "separated versions" approach are the questions regarding versioning:
What's the current version of my project?
Which module versions work with each other? (even thought they don't directly depend on each other, one does rely on what another does, e.g. they share the database schema)
To solve that I put all module versions into the dependency management part of the parent pom, even if no other module depends on them. A "integration-test" module could solve that by depending on all of the modules - and of course testing them together.
This way I would be "forced" to update the parent pom with every change, since it's referring the released module versions. This way the parent pom would have the "leading" version and at the dependency-management block state the versions of all modules that are compatible with each other (which will be ensured by the integration test).
I have a huge multi-module project , which is being built using maven.
Most of them are core modules , which are used in other modules for various purposes.
But , 9 of 10 times, there is no change in the core modules and there is just some change in the depending modules.
Sometimes, there might be a small change in 1 core module of the many.
The issue is, I want maven to know, if the current build in target folder of the core modules is the latest accroding to the code, i.e no changes were made to the core for that.If thats the case, then I want maven to skip building that module during the maven phases.except the assembly plugin phase, which takes care of assembling all my modules at one place.
Is there a neat way to do this.I.e some maven plugin already taking care of stuff like this??
Some light on this will be great help.
Thanks,
Neeraj
May be you can use the maven incremental build plugin or use Hudson/Jenkins to do an incremtal build.
Just use Hudson - it will scan your source repository for changes and only run a build once that has occurred. It works well with Maven out of the box too.