I have 100s of microservices, we are trying to introduce the BOM in our project. 1 issue I can clearly see is, if I update the BOM, we need to make changes to all the microservices. Is there a better way to do this?
Example:
grandparent:1.0.0
`-- parent:1.0.0
|-- ms1:1.0.0
|--
`-- ms100:1.0.0
Now say if we make change to grandparent, we have to update that version in parent and then in all the microservices. Going and managing these many microservices is maintenance nightmate
The structure is making them coupled, shouldn't be focus on making independent, even build process.
I think first thing to resolve is to break this interdependency on build process.
If the structure is to just ease the build, there are multiple tools available to automate the build process, for example: jenkins. Jenkins would be simplest solution and popular among build tools.
Related
I have a multi-module project inside buildSrc/.
I would like to use one of these sub-projects as a dependency of a "normal" (not build-related) project.
To make things clearer, the project layout is more or less like this:
buildSrc/
p1/
p2/
main/
main1/
main2/
build.gradle
Now, what I want is to make :main:main1, for example, depend on buildSrc/p1.
I have been able to achieve this by including buildSrc/p1 in both /settings.gradle and in buildSrc/settings.gradle, but that's not very good (it's basically duplicating the project in 2 separate project trees) and causes IntelliJ, for example, to fail to import the project properly (which is my main issue, actually).
Is there a good way to solve this problem without moving buildSrc/p1 to a separate Gradle project or repository (it's important that we keep being able to evolve the code in both projects together with minimal friction)?
I was trying to build Maven pom in something similar to the following hierarchical form:
root
+-- A-POM
+-- B-POM
+-- C-POM
+---D-POM
I was hoping that this could take care of my changed module problem. That is, if C is changed, then A must be rebuilt, etc.
But I ran into the issue that it seems the packaging at root is "pom," and after that I can't have A as packaging "war" then continue to drill in to have A include B, C as its modules. It seems to me that any POM which does not have "pom" in the then it can't have child modules. Is my understanding correct? Is there a way to do what I wanted to do?
In addition, I don't seem about to chain the "changed" mechanism in Maven (must due to my lack of knowledge). I like to have Maven detect a dependent project has changed and rebuild all the affected projects.
Thanks so much!
the reactor project (the root of the multimodule project) must have pom packaging. So your nested structure is invalid since A is not of type pom and I'm pretty sure you won't get it to work this way.
Second point is that Maven is a modularized build system and uses repository mechanisms to locate pre-built artifacts instead of checking out all modules from version control and building them in a monolithic way like in the old days ;) This means that Maven cannot know what to rebuild when you change something at your module since it simple does not have all the other module there at this time.
I think this is more a CI task than that should be handled by the build system itself. I know that your can achieve such a behavior with an appropriate build/CI Server like Jenkins that supports upstream and downstream projects. This means it is able to detect dependencies between the projects and trigger other builds as soon as a dependency has been built. This comes close to the behavior you are trying to achieve.
Btw. rebuilding other projects is only required for SNAPSHOT dependencies. Jenkins with the maven plugin supports this behavior but, depending on the number of SNAPSHOT dependencies of your project, this can cause long chains of project builds on the server. Some folks are of the opinion that in general SNAPSHOT versions are hell for CI tasks since these artifacts can change over time and are not reproducible. You could think over completely omitting SNAPSHOT versions and building final versions each time. This would also obviate your requirement to rebuild other modules as soon as a module changes. There are simply no changes until you upgrade dependency versions.
We are using the latest version of Jenkins CI and we have a large number of projects, which have Maven dependencies on other projects. We also are using Jenkins views to group associated projects together.
I would like to be able to generate a graphical representation of the project hierarchy within a view. I am aware that if I select a project that I can see the upstream projects, but going through approximately 40 projects, writing this down and compiling it into a tree would be tedious, time-consuming and error-prone.
Does anyone know of a technique or plugin for Jenkins that could achieve this? Ideally it would work against all the projects within a view.
I would prefer an automated technique rather than performing it manually, since this process would need to be run periodically (say once a month) for a management report.
Update
Having investigated this question, I am not averse to writing a script to query the Jenkins API to get the JSON or XML for the projects within a view and then asking each for its upstream projects. But I'd rather save myself some work and using someone else's tool :)
You can use Maven to generate the dependencies for each project (http://maven.apache.org/plugins/maven-dependency-plugin/tree-mojo.html).
It won't give you a dependency tree for all your Jenkins projects though. Maybe you could pull from all maven outputs and create your own? Or maybe (not really) create a super project in which all modules are your existing projects (again, not really).
There is a Downstream Buildview plugin. It's per job, and it displays job names, but if you job names are named after maven modules, it shouldn't be an issue.
I don't understand the use of an Xcode workspace to organize projects with dependencies on one another. For instance I see a lot of developers create workspace structures that look like this:
Workspace
|-- App
|-- A Common Library
|-- Another Common Library
What benefit does this provide? If anyone opens the "App" project directly won't they be unable to actually build the app? They'd have to realize that a workspace exists with the necessary dependencies.
It seems to me like the better approach is to use nested projects like this:
App
|-- Libraries
| |-- A Common Library
| |-- Another Common Library
Then no project exists that cannot be built. It also seems more in line with Git's idea of submodules.
The only use I see for a workspace is to group common projects with no dependencies on one another. I'd like to hear other people's thoughts on this because I may be missing something.
I use workspaces when I want to combine projects while also maintaining project independence.
An example where I use workspaces is series of tutorial projects that progress from very simple to more complex. Each project can function as a standalone project, but grouping them together in a workspace helps my organization of the overall project.
In another instance I have an app developed for a client. The app works as both a standalone app and a module in the overall project. The independent project can build the standalone app. The other app uses a workspace that includes two projects. The module version of the app is built from a special scheme, and this combined app doesn't build without using the workspace.
One twist with the two above situations is where the build folder is stored. I have to change the Xcode preference to put the build products into unique folders for the group of tutorial projects, use a common build folder for the module within the other app setup.
In other circumstances I have plenty of projects with embedded projects. In these situations the library projects are stable. I don't attempt further development of the library projects so they are just another resource for the project. I find it easier to work where my file system organization of project resources somewhat reflects the organization of my Xcode project. So these library projects are copied into the main project's file hierarchy. It would make sense to use workspaces if I was developing the libraries and using them in multiple projects. For expedience I frequently don't bother.
Sometimes I even combine workspaces with projects containing embedded projects.
So my opinion is that both organizational tools, embedded projects and workspaces, have their merits and problems. I choose to use one or the other (or a combination) depending upon the particular circumstances.
We added nested projects into the Main project's Frameworks, so we could "include" them into the .framework product.
Main
|-- Main
|-- MainTests
|-- Frameworks
| |-- CommonLibrary.xcodeproj
| |-- AnotherCommonLibrary.xcodeproj
| |-- UIKit.framework
| |-- Foundation.framework
| |-- CoreFoundation.framework
|-- Products
See this Great Tutorial by Jeff Verkoeyen for adding Universal Frameworks to a project. It not easy, at first, but keep working on it and you'll get the hang of it.
What are the best practices for software versioning and multimodules projects with Maven?
I mean, when I create a multimodules project with Maven, what is the best approach for the versioning? To use a single version for all the modules (defined in the top project)? To use a version for each module (defined in the POM of each module)? Is there another approach that I'm missing? What are the pros and cons of each approach?
In general, are the different modules released together (possibly sharing the same version number)?
Thanks
Honestly it depends on what you would like to do. Multimodule projects are created for multiple reasons, one of them being you only need to deploy what has changed instead of all modules.
Think about it this way: if you had a non-multi-module project and you only had to change one line in the services layer, you have to rebuild the entire project and deploy all of the code again...even though only your services layer will change.
With multi-module projects, you can regenerate your project and deploy only what changed...your services. This reduces risk and you're assured that only your services module changed.
You also have a multitude of benefits to using multi-module projects that I'm not listing here but there is certainly a huge benefit to NOT keeping your version numbers of your modules in sync.
When you build your project, consider deploying it to a repository that will hold all compatible jars together for builds (each build creates a new folder with the parent-most pom version number). That way, you don't need to keep documentation about which jars are compatible...they're all just deployed together with a build number.
I was looking for a solution for this exact problem myself and versions-maven-plugin was exactly what I needed. I don't like the release plugin communicating with the SCM system. The versions plugin does just what we need: it sets a new version number in all poms of the project:
mvn versions:set -DnewVersion=2.0.0
Then I can proceed with commits, tags and an official build server build...
EDIT:
The versions plugin depends on how a maven multi-module project has been organised: as a result, it often does not update all POM files in a complex multi-module project.
I've found that sed and find do the job much more reliably:
sed -i 's/1.0.0-SNAPSHOT/1.0.0-RC1/g' `find . -name 'pom.xml'`
Typically you create a multi-module project because you have deemed that the various modules are parts of a single whole. Maybe the client-piece, the controller-piece and the services-piece. Or maybe the UI with services.
In any case, it makes sense to have the version numbers for the various modules to move in lock-step. However Maven does not enforce that as a rule.
As to your question
are the different modules released together (possibly sharing the same
version number)
I would think so. That is one of the reasons for having it a multi-module project. Otherwise you could have the modules as independent projects.
Of course this is the kind of stuff that is rife with edge cases and exceptions ;-)
I had the same problem with a project I`m working on. I also decided to use separate versions and even the dependency to the parent pom only has to be updated if some of the managed dependencies change. (so mostly as #vinnybad describes it)
Two additions
exists-maven-plugin
With the usage of the "org.honton.chas.exists-maven-plugin" only the modules will be deployed to the repository that have actually changed, which is really great, because also the according docker-images will only be published if something has changed on one of the service. This avoids "polluting" the image repository with different but unchanged versions.
versioning
One main downside of the "separated versions" approach are the questions regarding versioning:
What's the current version of my project?
Which module versions work with each other? (even thought they don't directly depend on each other, one does rely on what another does, e.g. they share the database schema)
To solve that I put all module versions into the dependency management part of the parent pom, even if no other module depends on them. A "integration-test" module could solve that by depending on all of the modules - and of course testing them together.
This way I would be "forced" to update the parent pom with every change, since it's referring the released module versions. This way the parent pom would have the "leading" version and at the dependency-management block state the versions of all modules that are compatible with each other (which will be ensured by the integration test).