This is not related to update the pom dependencies using a maven goal. I already have that sorted out.
So I am responsible for creating, packaging and maintaining common utilities. These common utilities are in turn used by all the teams in the org. Problem is that the teams using these utilities do not update the dependencies unless it is the last resort. We would like them to use the latest release version of our common utilities barring very few.
Now I have come across this Maven Versions Plugin by mojohaus which I think serves my need by using two goals - versions:update-properties and versions:use-latest-releases. It serves my purpose except two things:
I do not see a way to exclude certain groupid:artifactid from the update dependency/property
We really want this to be a compulsory thing (maybe part of the Test execution - this is for test automation utilities mainly) rather than a maven goal. Because if it is a maven goal, it needs to be invoked separately and hence becomes optional for teams.
We know that forcefully updating to latest version might cause some issues with defect re-produciblity, but we are willing to take that risk. Our utilities are really test products.
Any direction/help on this is appreciated.
Edit: We run our tests using maven goals clean install already. So they use existing pom. We want dependencies update to happen before the tests run. Also it is desirable to commit the changes to source control (bitbucket) if possible.
We have our tests setup using Jenkins but teams also run multiple test on local machines.
Edit: Found the answer to #1. The plugin provides to exlude regex for group and artifact id. using tags excludes and excludesList
Sorry for insisting. From the Maven point of view I see two main solutions:
Your projects and your utility jar are tightly coupled and every project always needs to use the latest version. Then you can bundle all projects and your utility in one multi-module project. This makes sure that everything is up to date all the time, but it requires that all projects and your utilities are always build together (not separately).
You distribute your utilities to different projects which build and release at different times. Then it is up to the projects to decide when to update. There is unfortunately no standard way to deprecate jars.
If I understand you correctly, you want something "in the middle". This may be hard to achieve.
Related
We are using the latest version of Jenkins CI and we have a large number of projects, which have Maven dependencies on other projects. We also are using Jenkins views to group associated projects together.
I would like to be able to generate a graphical representation of the project hierarchy within a view. I am aware that if I select a project that I can see the upstream projects, but going through approximately 40 projects, writing this down and compiling it into a tree would be tedious, time-consuming and error-prone.
Does anyone know of a technique or plugin for Jenkins that could achieve this? Ideally it would work against all the projects within a view.
I would prefer an automated technique rather than performing it manually, since this process would need to be run periodically (say once a month) for a management report.
Update
Having investigated this question, I am not averse to writing a script to query the Jenkins API to get the JSON or XML for the projects within a view and then asking each for its upstream projects. But I'd rather save myself some work and using someone else's tool :)
You can use Maven to generate the dependencies for each project (http://maven.apache.org/plugins/maven-dependency-plugin/tree-mojo.html).
It won't give you a dependency tree for all your Jenkins projects though. Maybe you could pull from all maven outputs and create your own? Or maybe (not really) create a super project in which all modules are your existing projects (again, not really).
There is a Downstream Buildview plugin. It's per job, and it displays job names, but if you job names are named after maven modules, it shouldn't be an issue.
We create releases and upgrade the versions of our multi module project with the maven release plugin, which is usually triggered with a Jenkins release build. The problem is that we have a couple of modules whose versions should be updated as well, but which should not be built on the server - only the version upgrade should be performed. (These are just testing stuff only used occasionally by developers.) Any good ideas how to do that?
I believe the accepted policy is that every module should be built that forms part of a project. That includes those modules intended to test others.
If internally you have a situation in which a module is intended to test another but should definitely not form part of the build you may wish to extract that module into another project and adjust the module to depend on the artefacts of the older project's module.
You may still find that your CI server (such as Jenkins) recognises this dependency and builds your test project because it detects a changed dependency. In that case disable the automatic building of this test project.
You may find it less "smelly" to correct the testing procedure to ensure your test modules are designed to be run upon each build. Having a set of tests that should not be automatically run seems contrary to modern software engineering approaches.
We check all of our source code's dependent third-party JARs into source control along with our source code. When needed, we manually download updates to third party JARs and replace those JARs that are in source control with the newer versions. We haven't felt the need to use Maven yet as this process seems simple enough for us. But are we missing something of great value by not using Maven? Or does our scenario not warrant using Maven?
"JARs dont change much", I hear this all the time.....
Storing jars in the SCM is simple in the beginning of the project. Over time the number of jars gets larger and larger.... Wait 2 or 3 years and nobody remembers where the jars came from, what their licensing terms were and most commonly what versions are being used (important to know when analysing security vulnerabilities).....
The best article I've read recently making the case for a repository manager is:
http://www.sonatype.com/people/2012/07/wait-you-dont-have-a-repository-manager/
A little irreverant, but does make a valid point about the kind of technical inertia one encounters all the time.
Switching a project team from ANT to Maven can be scary.... Maven works quite differently, so I find it is best deployed with greenfield or adventurous project teams. For the old-school ANT users, I recommend using the Apache ivy plugin. Ivy allows such teams to outsource the management of their dependencies but keep the build technology they're comfortable with.
Ultimately the biggest benefit of using Maven are not dependency management. It's the standized build process. I've seen several failed attempts to create a "standard" ANT build process. Problem every build engineer has his opinion on what the standard should be.... Maven's approach of forcing users to write build plugins may appear restrictive in the beginning, but just like the iPhone eventually developers discover "there's a Maven plugin for that" :-)
When it comes to dependency management Maven really can be quite valuable. As Mark O'Connor suggests, running a local repository manager would likely be better than checking the artifacts into source control.
There are many tools (like m2e in eclipse) that can help with dependency management and provide valuable feedback on which modules or dependencies require which other dependencies. Maven will also make sure to get the appropriate version of a dependency even if different modules depend on different versions of a given library. That will help prevent duplicate versions of the same jar showing up in your deployed project as long as they have the same group and artifact id.
Even for a very simple project I don't think I would resort to checking dependencies into the source control system.
It's not only about 3rd Party Libraries. Mostly if you have multiple repositories. In our case, we had four repositories with lots of inter- and intra-dependencies.
Actually I started this answer and then I had to go for 15 minutes to talk to some colleague about a problem happened after someone forgot to update the .jar of one project in the other's lib directory.
And it looks more professional :)
For a project with a large number of Maven artifacts (both internally generated as well as external ones), how does one go about automating the releasing of the internally controlled artifacts as part of an overall product release.
Things to be aware of about this question, we use Jenkins and the Maven release plugin. So the operation of releasing a single artifact is automated (albeit the operation to kick-start the process is manual). However the process of releasing all the changed artefacts over the course of a release is not automated (i.e. one has to manually kick-start the release of each artifact). Part of the problem is that almost nothing is released until the end of the release, prior to that everything remains in SHAPSHOT. We have a huge number of components as well as numerous applications/services (over 30) which rely on the plethora of components. So it is not just the case of picking a component and releasing, there are release dependency hierarchies that must be followed (i.e. start at the bottom releasing components that do not use other components and then work your way up until all the applications/services are released).
It is also worth noting that we use two common parent poms which, for the most part, control the versions of the external artifact dependencies and the internal component dependencies. Some pom files for components and applications may override this, but this is (or should be) an exception and should be for a good, but temporary, reason. So when an internal artifact is released, the version in the corresponding parent dependency pom should also be updated.
The product has a release number (of course), however the various pom files technically do not share this version number. While this is not strictly true, the idea as that when parts of the software are set to end-of-life, they will not be updated in the future, thus while a limited number of artifact versions match the product's version at present, this will eventually not be the case.
Any thoughts on ways to get this process automated would be greatly appreciated. Also if you feel what I have described seems to be a crazy way to manage the software, then please provide a comment. Thank you.
You might be able to make use of the Maven Versions plugin which can help formalise versions for projects.
For example, the use-next-releases goal may allow you to release the lowest level of project and then more rapidly bring those released versions into their dependencies.
There may also be scope to use the use-next-versions goal if you fancy releasing components as necessary and simply bring your projects to the "latest" version thats been formally released.
What are the best practices for software versioning and multimodules projects with Maven?
I mean, when I create a multimodules project with Maven, what is the best approach for the versioning? To use a single version for all the modules (defined in the top project)? To use a version for each module (defined in the POM of each module)? Is there another approach that I'm missing? What are the pros and cons of each approach?
In general, are the different modules released together (possibly sharing the same version number)?
Thanks
Honestly it depends on what you would like to do. Multimodule projects are created for multiple reasons, one of them being you only need to deploy what has changed instead of all modules.
Think about it this way: if you had a non-multi-module project and you only had to change one line in the services layer, you have to rebuild the entire project and deploy all of the code again...even though only your services layer will change.
With multi-module projects, you can regenerate your project and deploy only what changed...your services. This reduces risk and you're assured that only your services module changed.
You also have a multitude of benefits to using multi-module projects that I'm not listing here but there is certainly a huge benefit to NOT keeping your version numbers of your modules in sync.
When you build your project, consider deploying it to a repository that will hold all compatible jars together for builds (each build creates a new folder with the parent-most pom version number). That way, you don't need to keep documentation about which jars are compatible...they're all just deployed together with a build number.
I was looking for a solution for this exact problem myself and versions-maven-plugin was exactly what I needed. I don't like the release plugin communicating with the SCM system. The versions plugin does just what we need: it sets a new version number in all poms of the project:
mvn versions:set -DnewVersion=2.0.0
Then I can proceed with commits, tags and an official build server build...
EDIT:
The versions plugin depends on how a maven multi-module project has been organised: as a result, it often does not update all POM files in a complex multi-module project.
I've found that sed and find do the job much more reliably:
sed -i 's/1.0.0-SNAPSHOT/1.0.0-RC1/g' `find . -name 'pom.xml'`
Typically you create a multi-module project because you have deemed that the various modules are parts of a single whole. Maybe the client-piece, the controller-piece and the services-piece. Or maybe the UI with services.
In any case, it makes sense to have the version numbers for the various modules to move in lock-step. However Maven does not enforce that as a rule.
As to your question
are the different modules released together (possibly sharing the same
version number)
I would think so. That is one of the reasons for having it a multi-module project. Otherwise you could have the modules as independent projects.
Of course this is the kind of stuff that is rife with edge cases and exceptions ;-)
I had the same problem with a project I`m working on. I also decided to use separate versions and even the dependency to the parent pom only has to be updated if some of the managed dependencies change. (so mostly as #vinnybad describes it)
Two additions
exists-maven-plugin
With the usage of the "org.honton.chas.exists-maven-plugin" only the modules will be deployed to the repository that have actually changed, which is really great, because also the according docker-images will only be published if something has changed on one of the service. This avoids "polluting" the image repository with different but unchanged versions.
versioning
One main downside of the "separated versions" approach are the questions regarding versioning:
What's the current version of my project?
Which module versions work with each other? (even thought they don't directly depend on each other, one does rely on what another does, e.g. they share the database schema)
To solve that I put all module versions into the dependency management part of the parent pom, even if no other module depends on them. A "integration-test" module could solve that by depending on all of the modules - and of course testing them together.
This way I would be "forced" to update the parent pom with every change, since it's referring the released module versions. This way the parent pom would have the "leading" version and at the dependency-management block state the versions of all modules that are compatible with each other (which will be ensured by the integration test).