Jenkins and maven multi module Projects missing artifacts - maven

This is a simplified example of an ear project, the parent pom aggregates the EAR, the EJBs, and the jars.
I have this structure in a Maven project, stored in SVN:
parent/
|- pom.xml
|- modulA/
| |- pom.xml
|- modulB/
| |- pom.xml
modulB has a Dependency of modulA
The pom.xml have the modules section
<modules>
<module>modulA</module>
<module>modulB</module>
</modules>
And a Dependency Management section
<dependencyManagement>
<dependencies>
<dependency>
<groupId>group</groupId>
<artifactId>modulA</artifactId>
<version>0.0.2-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>group</groupId>
<artifactId>modulB</artifactId>
<version>0.0.2-SNAPSHOT</version>
</dependency>
</dependencies>
</dependencyManagement>
The sub-modules reference the parent
<parent>
<groupId>group</groupId>
<artifactId>parent</artifactId>
<version>0.0.2-SNAPSHOT</version>
<relativePath>..</relativePath>
</parent>
in my PC when I compile for the first time with maven 2.2.1 (windows)
mvn clean compile
I don't have any problems
but.... when Jenkins try to compile for first time (Maven 2.2.1 Linux RedHat)
Missing:
----------
1) modulA:jar:0.0.2-SNAPSHOT
Try downloading the file manually from the project website.
Then, install it using the command:
mvn install:install-file -DgroupId=group -DartifactId=modulA -Dversion=0.0.2- SNAPSHOT -Dpackaging=jar -Dfile=/path/to/file
Alternatively, if you host your own repository you can deploy the file there:
mvn deploy:deploy-file -DgroupId=group -DartifactId=modulA -Dversion=0.0.2-SNAPSHOT -Dpackaging=jar -Dfile=/path/to/file -Durl=[url] -DrepositoryId=[id]
Path to dependency:
1) modulB:ejb:0.0.2-SNAPSHOT
2) modulA:jar:0.0.2-SNAPSHOT
----------
1 required artifacts are missing.
Why????????
After that if I deploy the project from my pc to Artifactory, Jenkins doesn't have problems, because Jenkins downloads the artifact from the repository... but why does Jenkins depend on the artifacts in the repository?
:(
Thanks in Advance
EDIT:
I thought the dependencyManagement section only "defines" the dependencies, but if a submodule doesn't use the dependency, the dependency isn't added to the submodule.
I drop the dependencyManagement section and the problem in Jenkins still occurs.
It works on my PC without problems.

I hope above dependency management section is inside the parent pom. According to your requirement modulB has a Dependency of modulA. So I suggest you to include dependency in moduleB instead of having it in the parent pom. I think when it runs in first time maven is looking for both dependencies since you have mentioned in in the parent pom.Look at your project build order. First it builds module A and then B. In your case I hope you have include all other dependencies in moduleA's pom file and once it built it will deploy a jar file in to m2 repository. And then moduleB start to build and since your dependency is already in the m2 repository it wont shout and project will build successfully.

The first time you build parent project, your Jenkins user's maven repository won't have modulA installed. clean compile is then run successfully in modulA, but nothing is installed. When it is run in modulB, the dependency on modulA can't be resolved.
If your Jenkins job's goal was clean install instead of clean compile, then modulA's artifacts would be installed to the Jenkins user's repository before the modulB build begins, and all would work.
Presumably this worked on your own machine because either you had run mvn install at least once in modulA, or because your IDE's classpath resolved the problem for you.

Related

How to compile specific dependencies using maven

I have a situation where in i need to clean and install couple of dependencies of my maven project. While I am working on this project i am making changes in these dependencies and have to manually clean and install for every small change i am making. I am trying to find a maven command which will make my life easy.
project-bpm-process <-- parent project
project-odata-service - < dependency >
project-core-service - < dependency >
I cannot put them as sub modules as they are not really modules of my this project, they are simply dependencies. So, literally group-id does not match in complete sense (there is a partial match but does not help in any way).
Update 1:
Tried the option 2 suggested by Mark. I see below error which indicates that the sub modules (aggregated projects) are not found under the parent project's folder.
[ERROR] [ERROR] Some problems were encountered while processing the
POMs: [ERROR] Child module
E:\STS-Workspaces\default-workspace\project-bpm-process-artificial\project-core-service
of
E:\STS-Workspaces\default-workspace\project-bpm-process-artificial\pom.xml
does not exist #
[ERROR] Child module
E:\STS-Workspaces\default-workspace\project-bpm-process-artificial\project-odata-service
of
E:\STS-Workspaces\default-workspace\project-bpm-process-artificial\pom.xml
does not exist #
[ERROR] Child module
E:\STS-Workspaces\default-workspace\project-bpm-process-artificial\project-bpm-process
of
E:\STS-Workspaces\default-workspace\project-bpm-process-artificial\pom.xml
does not exist #
I just created a new maven project with packaging "pom" type and added other projects as modules. Now, "project-bpm-process-artificial" has become artificial parent of all the three projects I was talking about.
From maven documentation, i see that the path is relative.
Update 2:
Location of actual pom is located at: *E:\STS-Workspaces\default-workspace\project-bpm-process-artificial*
But other referenced projects are in *C:\Users\ramgo\git* and *E:\git-repos*. These projects are imported into eclipse for development.
The pom.xml is here:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.mycompany</groupId>
<artifactId>project-bpm-process-artificial</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>pom</packaging>
<name>project-bpm-process-artificial</name>
<url>http://maven.apache.org</url>
<modules>
<module>project-core-service</module>
<module>project-odata-service</module>
<module>project-bpm-process</module>
</modules>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
</dependencies>
</project>
Solution: For all practical reasons I found option 1 is easy to implement (option 1 provided by Marks). It hardly took 5 minutes to write a batch script. Here is the one for handy reference.
set core=<directory_path_of_core_project>
set module_one=<<directory_path_of_module_one>>
set module_two=<<directory_path_of_module_two>>
cd %core%
call mvn clean install
cd %module_one%
call mvn clean install
cd %module_two%
call mvn clean install
Option 2 seems interesting but not feasible in my case. Links don't work and no way to refer absolute path for sub modules.
.....
The notion of Maven dependency assumes that the artifact is build externally (not even necessarily with maven) and available to your project as a third-party jar.
So, the terminology in the question is misleading to me.
If you have some third-parties (I assume having their own pom.xml) but are external to your project, then obviously in your project you can't manage them. Maven can't build external stuff.
So, based on these assumptions, the choices are:
Option 1
Create a script that will:
enter the dependency directory
run mvn install on that directory
enter your project's directory
run mvn whatever on your project
Option 2
Create an "artificial" pom.xml that will have packaging type "pom" and will list both your project and the dependencies as submodules (your project and dependencies will be peers):
|__some_folder
|__pom.xml
|__dependency1
| |__pom.xml
|__dependency2
| |__pom.xml
|__your-project
|__pom.xml
In this case you will be able to operate with both your project sources like one project and you'll be able to use the following:
cd some_folder
mvn clean install --projects <your-project> --also-make
So that if your project has dependencies in other modules, they'll also be built
I would probably go with the second option, but its your choice really
Update 1
Based on the information you've provided:
Don't really count on eclipse, its not really relevant at this point. You should try to get to the point where running maven from command line should work. The eclipse will follow your poms once you've done everything right.
If you place the "artificial" pom into E:\STS-Workspaces\default-workspace\project-bpm-process-artificial then all the modules should be in sub-folders:
E:\
|_ STS-Workspaces
|_ default-workspace
|_ project-bpm-processes-artificial
|_ project-core-service
| |_pom.xml of that module
| |_.git // it can be a root of git repo
| |_src
|_ project-odata-service
| ...
|_ project-bpm-process
After that you can do the following to check yourself:
cd E:\STS-Workspaces\default-workspace\project-bpm-process-artificial
mvn clean install
It should compile all the libraries and your project
Then if you want to build your project (assuming its called project-bpm-process) then you can do from the same folder:
mvn clean install --projects project-bpm-process --also-make
If it has a dependency on project-core-service but, say, not on project-data-service then only the project-core-service will be rebuilt
Now when the maven if sorted out, you can add eclipse workspace in any other folder. I can't comment much on eclipse since I'm an IntelliJ user. In intelliJ you can just import this artificial pom and it will automatically recognize all the projects. In eclipse I think it should work similarly

packaging maven project with external jar

I've been trying to make a runnable jar from my project (in Intellij IDEA) which has a dependency to an oracle (driver -> ojdbc6) jar. When I package the project with all of the dependencies, the only one what will be excluded is the jar. Which means my db queries are going to fail when I run it.
I've found several similar questions*, but I've failed the execution of them, because I don't know the groupid and artifact id of the oracle's jar.
*like this one: build maven project with propriatery libraries included
p.s.: the jar wad added through the IDEA's feature (project structure -> modules), and with this solution the project could run without failure. The problem starts with the packaging.
Short Solution: Try using the below:
<dependency>
<groupId>LIB_NAME</groupId>
<artifactId>LIB_NAME</artifactId>
<version>1.0.0</version>
<scope>system</scope>
<systemPath>${basedir}/WebContent/WEB-INF/lib/YOUR_LIB.jar</systemPath> // give the path where your jar is present
</dependency>
Make sure that the groupId, artifactID and the version number are unique.
Long Solution:
Download the jar file to your machine.
Navigate using the prompt to the folder where you downloaded the jar.
Run the following command to install the jar to your local repository.
mvn install:install-file -DgroupId=com.oracle -DartifactId=ojdbc6 -Dversion=11.2.0.3 -Dpackaging=jar -Dfile=ojdbc6.jar -DgeneratePom=true
Finally, add the dependency to the pom.xml.
<dependency>
<groupId>com.oracle</groupId>
<artifactId>ojdbc6</artifactId>
<version>11.2.0.3</version>
</dependency>
Also, don't forget to use -U option while running the project.

Jenkins fails to build multi-module Maven project

I have a multi-module Maven project where I have multiple micro services as modules so I have modules listed in my parent pom.xml like below:
<modules>
<module>core</module>
<module>model-base</module>
<module>module1</module>
<module>module2</module>
...
<module>module5</module>
<module>module7</module>
<module>module6</module>
</modules>
Here the module7 is dependent on module5, 6 so I have dependencies listed like below in my module7 pom.xml:
<parent>
<artifactId>pojectA</artifactId>
<groupId>com.domain</groupId>
<version>1.0-SNAPSHOT</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>module7</artifactId>
<dependencies>
<dependency>
<groupId>com.domain</groupId>
<artifactId>core</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>com.domain</groupId>
<artifactId>module5</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>com.domain</groupId>
<artifactId>module6</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
</dependencies>
When I run mvn clean package in my local the module5, 6 called before the module7 as expected but in Jenkins it is trying to build module 5 then module7 making build fail saying:
[ERROR] Failed to execute goal on project module7: Could not resolve dependencies for project module7:jar:1.0-SNAPSHOT: Could not find artifact module6:jar:1.0-SNAPSHOT -> [Help 1]
Do I need to run any other jobs or re-order the modules in my pom.xml, how is it differ from local to Jenkins? Appreciate any help on this.
The order of modules is not relevant. Maven recognizes which project depends on which other project(s) and sets the build order in the reactor accordingly. See POM Reference, Aggregation (or Multi-Module):
You do not need to consider the inter-module dependencies yourself when listing the modules, i.e. the ordering of the modules given by the POM is not important. Maven will topologically sort the modules such that dependencies are always build before dependent modules.
Add Pre-Step as per below attached screenshot. This will compile all your top modules.
Then we can execute which ever module we want.
As is probably quite well understood, the issue is that the dependencies between the child modules fail because they aren't installed in the local repository yet (because they are yet to be built). The goal that causes this (for me anyway) is mvn test, which is invoked by mvn package. Your local build probably works because at some point you've done a mvn install and this has bootstrapped your system.
In Jenkins the only way I've found to make these builds work is to use the Pre-build step invoking a Maven target of install, and then build the main step as usual.

Skipping Maven Test Dependency

I am working on a project that use maven for building. What I am trying to do is to skip the test dependency. Basically running the maven build without the presence of artifact in my maven repository.
eg
<artifactId>example</artifactId>
<scope>test</scope>
This is from the pom file of my project and I have to maven build my project without having artifact example.
I have searched for solution such as use "-DskipTests=true" or "-Dmaven.test.skip=true". In my case they did skip the running of the tests but it still complains missing dependency file.
Does anyone know a way to run maven build without having to have test artifact in the maven repository?
Thanks.
See https://issues.apache.org/jira/browse/MNG-4192. I think the only way around it is to move the test-scoped dependency into a Maven profile.
Here is an example:
<profile>
<id>test-with-extra-dependency</id>
<dependencies>
<dependency>
<groupId>org.example.groupid</groupId>
<artifactId>artifact-id</artifactId>
<version>0.0.1-SNAPSHOT</version>
<scope>test</scope>
</dependency>
</dependencies>
</profile>
You should now be able to run a build without tests:
mvn clean install -Dmaven.test.skip=true -DskipTests=true
After that you can run the tests with the profile enabled:
mvn test --activate-profiles test-with-extra-dependency
I wouldn't advise it, but this can be useful in case of a circular dependency between the current module and the test dependency, i.e. you can build the module, then build the test dependency using the build result, and finally test the module with the test dependency. If you find yourself wanting to do that, please try to restructure your project to eliminate the circular dependency (eg by moving the tests to a third module).

How can I deploy only the pom file to my snapshot repository in Maven?

I would like to be able to deploy only the POM artifact (file) without the main artifact (JAR, WAR, etc), when running mvn deploy and version is a SNAPSHOT version.
Why?
We several developers working on multiple Maven projects. We have a Hudson server with a job per Maven project and version (e.g. foo-1.2, foo-1.3). Each job builds the project and deploys it to a Nexus server (upon success). Maven projects under development are marked as such by using -SNAPSHOT postfix in the version. For example: 1.2-SNAPSHOT, 1.3-SNAPSHOT.
Here's a sample scenario how a developer work is damaged due to this architecture.
Assume two Maven projects: foo-core and foo-webapp, both at version 1.2-SNAPSHOT.
Developer A is working on foo-core, made several changes and compiled it.
Developer A continues to work, but on foo-webapp.
Developer B started working and changing foo-core. It commits his work and pushes it to the SCM.
Hudson is triggered by SCM; Builds foo-core and deploys it to the snapshot repository in Nexus.
Developer A is running mvn install on foo-webapp. Maven is checking with Nexus, and finds that there is a newer version of foo-core in Nexus. It downloads it (filled with developer B changes) and than it fails compilation, since the changes made by developer A are not in the jar located in the local repository. The download overrides the file installed there by developer A.
Existing Solutions
I looked into maven-deploy-plugin, but this plugin deploys all artifacts attached to the project. If they had a way to configure which artifacts to deploy, it would have been great.
Question: Is there any way to solve this without resorting to writing my own deploy plugin, based on maven-deploy-plugin?
Basically to the -Dfile parameter, instead of the artifact, pass the pom.xml. Run the command and yay! mvn deploy won't give you any issues now. Here's a sample deploy command :
$ mvn deploy:deploy-file -DpomFile=pom.xml -Dfile=./pom.xml -DgroupId=my.group.id -DartifactId=artifact-id -DrepositoryId=bigdata-upload-snapshots -Durl=http://maven.mymaven.com/content/repositories/snapshots/
A prerequisite for this is that the repository be added in your settings.xml
[Edit]: I have supplied the parameters -DgroupId and -DartifactId of the project in the sample deploy command but they're not required (refer to Zac's comment below)
I never heard of such a possibility and also would be very astonished if that would be possible. As the pom and the resulting artifact are some kind of unit it would make no scence (to me) to deploy only parts of them.
Nevertheless you should consider to make a separate pom project which specified dependencies and plugins you might want to use on your JAR/WAR projects like this:
<groupId>foo.bar</groupId>
<artifactId>my-pom</artifactId>
<version>1.0.0</version>
<packaging>pom</packaging>
and then inherit that pom project by your JAR/WAR projects like this:
<parent>
<groupId>foo.bar</groupId>
<artifactId>my-pom</artifactId>
<version>1.0.0</version>
</parent>
This is called project inheritance. You can change and deploy your pom project independent of the "child" artifacts.
EDIT after reading motivation:
As I understand you want to prevent maven to resolve SNAPSHOT artifacts from a repository (so that local version won't be overwritten). Have you ever tried to use the mvn -nsu option (see mvn -help)?
-nsu,--no-snapshot-updates Suppress SNAPSHOT updates
I never tried it but found this reported issue. Nevertheless I would give it a try (as the issue is not commented yet).
This works for me for deploying a pom file only (e.g next to an existing jar):
(Note: you need to specify packaging also, otherwise it will be uploaded as an .xml file which is not what you want.)
mvn deploy:deploy-file \
-Dfile=pom.xml \
-Dpackaging=pom \
-DgroupId=com.mycompany.package \
-DartifactId=my-artifact \
-Dversion=2.0.1 \
-DrepositoryId=serverIdFromSettingsXMLForCredentials \
-Durl=http://repositoryserver/myrepo/
Not exactly the answer these folks were asking for. My situation was I wanted to deploy only the parent pom. I'm using the spring-boot-thin-layout in a child module. This requires the parent module be deployed into artifactory. I added the following into my project. It enables skipping of install and/or deploy phase.
In my parent pom:
<properties>
<disable.install>true</disable.install>
<disable.deploy>true</disable.deploy>
<enable.deployAtEnd>true</enable.deployAtEnd>
</properties>
<profiles>
<profile>
<id>deploy-parent</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<properties>
<disable.install>true</disable.install>
<disable.deploy>true</disable.deploy>
<deployAtEnd>${enable.deployAtEnd}</deployAtEnd>
</properties>
<build>
<finalName>${project.version}</finalName>
</build>
</profile>
</profiles>
And the in my child pom(s) or any module you don't want deployed with parent:
<properties>
<maven.install.skip>${disable.install}</maven.install.skip>
<maven.deploy.skip>${disable.deploy}</maven.deploy.skip>
<deployAtEnd>${enable.deployAtEnd}</deployAtEnd>
</properties>
So effectively when I run mvn deploy on the parent pom, it will compile all the modules, not run install on anything, and then at the end deploy any module not having `

Resources