We have a GIT Repository which contains multiple microservices (around 120 microservices). I would like to know is there a process available to check all the poms in the repository to see if the pom contains a specific dependency jar . For example there is a microservice A and I wanted to know if there is a dependency for jar(woodstox-core) available in the pom without manually opening the POM and checking it.
Since there are around 120 microservices analysing each pom manually for the dependency would be very time consuming . Also the same excercise needs to be repeated for other git repositories as well.
Any help on this is much appreciated
If the repo has a root parent pom file then this problem is easy. You can just run "mvn dependency:tree" from the root and this will list off all the dependencies in the children services.
If this is not the case, and each service is built as a stand alone service (in the eyes of maven). You will need to write a small script which will iterate over all the children services/folders in the root git repo and run the "mvn dependency:tree". If you are doing this in a bash script you can pipe all the output of the tree command to append to a central file and the just search one central location once you are done.
Related
Currently we have a number of maven projects (specifically Apache Camel) that reside in isolation. We also have one Jenkins job per project because we need to specify the pom.xml file in the maven build and since the projects are independent then we require one job per project.
However, we also know most of the projects share a lot of dependencies and we want to turn them into a maven multi-module project with a parent pom file where the dependencies and versions are stated. We also want to have fewer Jenkins jobs to maintain and allow more projects to be added without having to create new Jenkins job.
My question is, in the Jenkins job for the maven build I still need to specify a single pom file. Does this mean that I need to point to the parent pom file and then add theparent directory as the directory to for Jenkins to receive the git trigger? In a sense, whenever a code is committed to any of the child projects, the job gets triggered and it uses the parent pom file to build only that project where the code was committed?
Situation
I have a multi-module Maven project. In it, I have several JAR artefacts, it then gets assembled as a WAR file. Thus, the WAR artefact depends on all kinds of JAR artefacts (it also has a WAR overlay), most of them with scope "compile".
Build and deployment to a repository are fine. But when I try to retrieve the WAR artefact, I have issues. previously, I used a simple wget to retrieve it from the Nexus API, but I wanted to try the Jenkins Repository Connector - not the least reason being that it actually shows a list of available versions.
I configure a repository in
Manage Jenkins -> Configure System -> Artifact Resolver
with the URL for our repo:
http://$NEXUS/nexus/content/repositories/releases/
then in the job, i add a parameter:
Maven Repository Artifact
and use the repository configured above, then i add
Artifact Resolver
as a build step and set it up.
Problem
I am not even sure on which side this should be solved: When I run the job to try to get the WAR file from the nexus, it also starts trying to retrieve all kinds of transitive dependencies (some of which are unaccessible to this user) and fails. What I need is just the WAR file. No transitive dependencies (since they're already packaged in the WAR).
The Repository Connector plugin doesn't seem to have a switch for this, and the Maven side it's probably perfectly OK to include those dependencies in the output POM.
Question
What can I do to either stop the repository connector from retrieving transitive dependencies or retrieve the WAR artefact in a different way? Also interesting for me (but a bit broad as a question) would be general ideas about doing this kind of workflow. E.g., does anyone use other ways of deploying the WAR into their Nexus?
i submitted a patch to the repository connector plugin.
my fork:
https://github.com/rmalchow/repository-connector-plugin
working on getting it to be merged:
https://github.com/jenkinsci/repository-connector-plugin/pull/10
We have a Nexus install that has a collection of commonly used java dependencies (commons-codec, jodatime etc.)
The directory structure is similar to the below
commonjavalibraries
|
|_ commons-codec
| |_commons-codec
| |_1.2
|_ commons-compress
|_commons-compress
|_1.3
I am in the process of adding the latest version of these dependencies to the repository.
To do this I have created a new pom and setup dependency management to deploy to Nexus.
However I now know that the groupId:artifactId:version attributes of the pom dictate how the project is deployed to nexus. Which I think would mean I'd need a separate pom for each group element.
How is the central repository pom managed? I realise it's possible to proxy the central repository, unfortunately our network doesn't allow downloading of jar's hence the need for a hosted commonjavalibraries repository in the first place.
Is there a way of doing what I'm trying to achieve without creating a pom for each group element?
After a bit more digging I noticed that when dependencies were downloaded from central they were stored in .m2/repositories along with their pom files.
I wrote a Python script to copy these into another location and then walk that directory tree and run mvn deploy:deploy-file for each jar/pom pair I came across.
We do not have our own repository at the moment. So, when we build with maven it creates .m2 repository in the home directory of the current user.
Now there are two third party jars which are not found in the Maven Central. Suppose one of them is hasp-srm-api.jar. Today the process is this:
a. The pom.xml of the project depending on hasp-srm-api.jar contain these lines:
<dependency>
<groupId>com.safenet</groupId>
<artifactId>hasp</artifactId>
<version>1</version>
</dependency>
b. Before doing the first build we execute the following command:
mvn install:install-file -Dfile=hasp-srm-api.jar -DgroupId=com.safenet -DartifactId=hasp -Dversion=1 -Dpackaging=jar
My question is this - is it possible to automate this step? I would like to be able to tell maven to check whether the hasp artifact exists and if not - install it manually using the aforementioned command line. How can I do it?
NO. It is not possible to have maven automatically deploy an artifact into a repository in the fashion you suggest. This goes for both local and remote repositories. If the artifact exists in a some repository somewhere, you can add that repository to your build's list of known remote repos, but other than that you have to add it yourself.
You can add it to your local .m2 repository, but that will then only be good for that individual environment. Other dev's will have to repeat the process. This is one of the main attractions of running your own repository server( like Nexus ); you can add the artifact to that repository and then everyone in your organization can use it forever. There is still no way to automate the deployment of the artifact, but it's easy to do and is permanent.
Note, setting up a repository manager is very easy to do. It's highly recommended. It makes the whole Maven thing make a whole lot more sense.
The best solution for such problems is using a repository manager which results in installing such kind of dependencies only once into the repository manager and the whole company can use it a usual dependency. That's it.
Other option you have is to write your own maven plugin. May be below link will be right place for you start
MOJO FAQ
I've refactored all repository configuration out of my various projects into a parent pom that I've created for the specific purpose of having a single point of configuration for stuff like repo's and distribution management. This is supposed to be best practice as I understand it. In deployed my parent pom to the nexus server, and then tried to run my child projects. They can't find my parent pom . . . this kind of makes sense to me since, wihtout the parent pom they don't know about the nexus repo . . . seems like a chicken and egg kind of thing? Am I missing something obvious?
It's true that your project needs to know where to find the repositories to download its dependencies. But before going to external repositories, Maven will check your local repository to see if the artifacts it needs are in there. Does your local repository contain the parent pom? If not, you can add it by running
mvn install
on the parent pom. The problem may have been caused by deploying the parent pom directly to Nexus, bypassing your local one. You can avoid this in future by deploying using
mvn deploy
This will first install the artifact locally, and then deploy it to the external repository (Nexus, in your case). More details here: http://maven.apache.org/guides/introduction/introduction-to-the-lifecycle.html
If you're in a situation whereby your parent pom is built and deployed by others, this won't help. You can either specify the repositories in your project's pom, or in your local settings.xml. A common approach is to expect all developers to include a repository definition in their local settings.xml which points to your Nexus repository, using it as a mirror for all other repositories. You can then configure each external repository you need in Nexus, and let it retrieve any dependencies you need for you. I'm not familiar with Nexus, but more details on mirroring can be found here: http://maven.apache.org/guides/mini/guide-mirror-settings.html
You must maintain the repositories definitions for each child (maven module), this is main best practice in a build process: "Make the build portable".
Any developer/system should be able to build any module without source dependencies to the parent or other modules, only with the repository reference.