Multi Project Maven and AWS CodeBuild - maven

I am new to Maven, and i am working on a project that have one or more dependencies on my other projects.
Example: I have 3 maven projects.
MyProject
MyProjectOther
MyProjectCommon
Both MyProject and MyProjectOther are independent of each other and do not have much to do with each other, but they both have a dependency on MyProjectCommon. (MyProjectOther dose not matter much to this question, just included it to show that MyProjectCommon is used in other projects)
Locally i can make this work, i have a workspace where i have both packages. I understand this is because Maven stores MyProjectCommon in a local repository.
Both projects (MyProject and MyProjectCommon) have their own GitHub repository, and my AWS CodeBuild is directed to build the MyProject repository. This fails as MyProjectCommon dose not exists for AWS CodeBuild.
How can i get this build to work with AWS CodeBuild, getting CodeBuild to download and build MyProjectCommon before it builds MyProject?
MyProject pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<properties>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
</properties>
<modelVersion>4.0.0</modelVersion>
<groupId>com.myproject</groupId>
<artifactId>myproject</artifactId>
<packaging>jar</packaging>
<version>1.0</version>
<name>myproject</name>
<url>http://maven.apache.org</url>
<dependencies>
<dependency>
<groupId>com.myproject</groupId>
<artifactId>myproject.common</artifactId>
<version>1.0</version>
</dependency>
</dependencies>
</project>
MyProjectCommon pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<properties>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
</properties>
<modelVersion>4.0.0</modelVersion>
<groupId>com.myproject</groupId>
<artifactId>myproject.common</artifactId>
<packaging>jar</packaging>
<version>1.0</version>
<name>myproject.common</name>
<url>http://maven.apache.org</url>
</project>

Unfortunately there is no easy way of doing this on any CI system that does not understand how underlying build dependencies are resolved and/or have 'always consistent' cached artifacts.
You do have couple of options here though. First option is to publish your artifact from the library build. You'll in this case have to setup multiple projects in CodeBuild, one for MyProject and other for MyProjectCommon. Builds for MyProjectCommon would publish artifacts to S3 (codebuild can do this) and then builds for MyProject can download the published artifact from S3 (you can also replace S3 with maven repositories but you'll have to publish the artifacts).
Other more saner option is to use maven modules to solve this issue (maven reactor) and create single repository for MyProjectCommon and MyProject. Essentially your projects end up under same repository and are built in the dependency order as specified in the pom of individual projects. Here's more documentation on the maven modules https://maven.apache.org/guides/mini/guide-multiple-modules.html
In the example you have, you'll still keep the pom file for MyProject and MyProjectCommon, but create another parent pom that defines both MyProject and MyProjectCommon as modules. You can then create CodeBuild project for this repository and run for example 'mvn install' which will compile both MyProjectCommon and MyProject in dependency order. You'll have an option to publish multiple artifacts from your build in CodeBuild or single zip with artifacts for both MyProject and MyProjectCommon.
I hope this helps.

Related

Inherit child-dependencies from maven dependency

I have two separate maven projects.
Project A contains utility classes and similar stuff. It also uses jetbrains annotations in some interfaces to mark parameters as Nullable and NotNullable.
<dependency>
<groupId>org.jetbrains</groupId>
<artifactId>annotations</artifactId>
<version>20.1.0</version>
</dependency>
Project B uses some of the Utilities of Project A. It includes it as a dependency from my repository.
<dependency>
<groupId>org.test.group</groupId>
<artifactId>Utilities</artifactId>
<version>1.0</version>
</dependency>
I can access the classes from my utilities dependency just fine. But i do not see any of the annotations on the parameters. I also can't access the jetbrains annotations in project B in any of my classes there. I'd have to add the jetbrains dependency in project B as well to do so.
Is there any way to inherit the dependencies of another dependency?
I looked at other questions and found this similar one. Tho his solution was to set the optional-parameter to false which i am not even using. Perhaps also something that needs to be configured in the maven build? Currently im running my build with goals clean package deploy without any special additional configuration.
I know gradle builds allow for implementation and api dependencies, where one of them forwards the dependency to other projects that include it and the other doesn't.
Edit: Here the full configuration of my projects
Local nexus running under localhost:8081 containing my artifacts. Local TeamCity running under localhost:8080 used for the builds and deployment to the repository.
Project A pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.test.group</groupId>
<artifactId>Utilities</artifactId>
<version>1.0</version>
<name>Utilities</name>
<dependencies>
<dependency>
<groupId>org.jetbrains</groupId>
<artifactId>annotations</artifactId>
<version>20.1.0</version>
</dependency>
</dependencies>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
</properties>
<distributionManagement>
<repository>
<id>local_nexus</id>
<name>Deployment</name>
<url>http://localhost:8081/repository/org.test.group/</url>
</repository>
</distributionManagement>
</project>
Project B pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.test.group</groupId>
<artifactId>TestProject</artifactId>
<version>1.0</version>
<name>TestProject</name>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven.compiler.source>1.8</maven.compiler.source>
<maven.compiler.target>1.8</maven.compiler.target>
</properties>
<dependencies>
<dependency>
<groupId>org.test.group</groupId>
<artifactId>Utilities</artifactId>
<version>1.0</version>
</dependency>
</dependencies>
<repositories>
<repository>
<id>local_nexus</id>
<name>Deployment</name>
<url>http://localhost:8081/repository/org.test.group/</url>
</repository>
</repositories>
<distributionManagement>
<repository>
<id>local_nexus</id>
<name>Deployment</name>
<url>http://localhost:8081/repository/org.test.group/</url>
</repository>
</distributionManagement>
</project>
Edit 2:
I've made some progress. Since i didn't change Project A's version after adding the annotations my local repository didn't fetch the new version. After a purge using mvn dependency:purge-local-repository the updated state of version 1.0 was available.
mvn dependency:tree prints now
[INFO] org.test.TestProject:jar:1.0
[INFO] \- org.test:Utilities:jar:1.0:compile
[INFO] \- org.jetbrains:annotations:jar:20.1.0:compile
Tho my IDE (Intellij) still doesn't recognize the annotation classes inside TestProject. Any idea why this is failing now?
You inherit annotations automatically, no configuration needed.
You don't inherit it, if it is of scope provided. Look at mvn dependency:tree to find out about the place and scope of the annotations library.
BTW: mvn clean package deploy is a waste of time, just use mvn clean deploy.
The issue was caused by my local maven repository. Since i didn't change the version of Project A's artifact (always 1.0) my local repository kept providing me with the old state of version 1.0.
Using mvn dependency:purge-local-repository i was able to cleanse the old 1.0 and load the new 1.0. Tho still my IDE refused to imports annotations dependency. At this point it was listed by mvn dependency:tree tho.
After tempering with the IDE for a while i decided to update my version to 1.1. Once Project A's 1.1 was built and Project B's dependency was updated to 1.1 it worked.
So basically it resulted from poor versioning of my projects, which interfered with my local maven repository.

POM for my own repo is missing, no dependency information available

I am currently learning software development and working on an assignment based on using a release from one project in another using jitpack.
The first project, which I want to add as a dependency in my second, simply has three classes and a JUnit test for each. I changed several things so I am at V5.0 release now.
https://github.com/MohamedMoustafaNUIG/assignment1/releases/tag/V5.0
My second project is not yet pushed, but this is the pom file:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.nuig</groupId>
<artifactId>assignment1_driver</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven.compiler.source>8</maven.compiler.source>
<maven.compiler.target>8</maven.compiler.target>
</properties>
<dependencies>
<dependency>
<groupId>com.github.MohamedMoustafaNUIG</groupId>
<artifactId>assignment1</artifactId>
<version>v5.0</version>
</dependency>
</dependencies>
<repositories>
<repository>
<id>jitpack.io</id>
<url>https://jitpack.io</url>
</repository>
</repositories>
</project>
When I try to build with dependencies (in netbeans), I get this error:
Failed to execute goal on project assignment1_driver:
Could not resolve dependencies for project com.nuig:assignment1_driver:jar:1.0-SNAPSHOT:
Failure to find com.github.MohamedMoustafaNUIG:assignment1:jar:v5.0 in
https://jitpack.io was cached in the local repository,
resolution will not be reattempted until the update interval of jitpack.io has
elapsed or updates are forced -> [Help 1]
I also get this warning:
The POM for com.github.MohamedMoustafaNUIG:assignment1:jar:v5.0 is missing, no dependency information available
Even though the zip file (from linked release) has the pom file.
Any help appreciated as I have no idea how to resolve this the issue
You either need to:
build assignment1 on the same machine with clean install to make it available in the local repository.
deploy assignment1 to a Maven repository with clean deploy and then use this as a <repository>.

TeamCity, Maven Build not found child module

I've Maven project with one subproject, when I run my install task on parent project from IDEA (IntelliJ IDEA) all works fine and maven resolve child module.
My projects are versioned on subversion, and this is the filesystem structure:
project
|--pom.xml
|--subproject
|
|-- branches
|-- tags
|-- trunk (here there is my subproject source, also pom.xml file)
I've create project with its subproject, from svn URL, on teamcity server.
When I run Build on parent project it fail and return me the following error:
[Step 1/1] Error reading Maven project: Some problems were encountered while processing the POMs:
[ERROR] Child module /opt/buildAgent/work/ee114e0c77ee2c44/subproject of /opt/buildAgent/work/ee114e0c77ee2c44/pom.xml does not exist #
How can I say to parent-project-build where it find the child module?
Is there something else wrong?
Parent POM:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>it.company.project</groupId>
<artifactId>MyProject</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>pom</packaging>
<name>MyProject</name>
<url>http://maven.apache.org</url>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
</dependencies>
<modules>
<module>Foo</module>
</modules>
</project>
Child POM:
<?xml version="1.0"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>it.company.project</groupId>
<artifactId>MyProject</artifactId>
<version>1.0-SNAPSHOT</version>
</parent>
<groupId>it.company.subproject</groupId>
<artifactId>subproject</artifactId>
<packaging>jar</packaging>
<version>1.0-SNAPSHOT</version>
<name>subproject</name>
<url>http://maven.apache.org</url>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
<build>
<resources>
<resource>
<directory>src/main/java</directory>
<includes>
<include>**/*.java</include>
</includes>
</resource>
</resources>
</build>
</project>
EDIT:
I added new parameter (pathSubproject) to my parent pom.xml, so when I run the parent build it skipped the previous error, but now it crash when trying to resolve parent dependency on the subproject. So I added a new parameter also at subproject (parentPath) and I passed it to relativePath inside parent tag.
Non-resolvable parent POM: Could not find artifact it.company.project:MyProject:pom:1.0-SNAPSHOT and 'parent.relativePath'
I think that my subproject POM not resolve the properties that I put inside <relativePath tag.
Is possible pass a properties to relativePath tag?
Thanks
In the <modules/> section of your parent project's pom, each module listed is a relative path to the directory containing the module.
So if you don't want to change the directory structure, you should be able to refer to the trunk of your subproject using <module>subproject/trunk</module>.
This does seem a bit clumsy though. If you are using the aggregator / modules pattern, I would recommend that project and subproject are both in the same SVN respository. If that isn't appropriate, then your subproject might not really be a module, and should be a dependency or have project as its parent artifact.
I solve my problem by creating 2 profile in my parent pom, the first one builds the parent application with all submodules, while the latter build only the the parent application.
In TeamCiTY build configuration settings I specified (inside Additional Maven command line parameters) the profile that build only parent module, and after I built the parent Application.
After that I built the parent application I built all submodules, and then I was able to build parent application with all modules.
I'm not sure that this is the right way, but in my case it worked well.

Release maven project removing automatically SNAPSHOT dependencies

I want to make my first release of a multi-module maven project using the maven release plugin. I´m facing a problem in the release:prepare step because I have SNAPSHOT dependencies.
<project ...>
<parent>
<artifactId>project</artifactId>
<groupId>ccc.aaa.bbb</groupId>
<version>0.0.1-SNAPSHOT</version>
</parent>
<artifactId>common</artifactId>
<packaging>pom</packaging>
<name>common</name>
<modules>
<module>daoModule</module>
<module>configModule</module>
<module>exceptionModule</module>
</modules>
<scm>
...
</scm>
<distributionManagement>
<repository>...</repository>
<snapshotRepository>...</snapshotRepository>
</distributionManagement>
</project>
And the modules extend this as its parent:
<project ...>
<parent>
<artifactId>common</artifactId>
<groupId>ccc.aaa.bbb</groupId>
<version>0.0.1-SNAPSHOT</version>
</parent>
<artifactId>exceptionModule</artifactId>
<name>exceptionModule</name>
<build>
...
The plugin changes the the version of the artifact 0.0.1-SNAPSHOT to a release version 0.0.1 , and creates the new snapshot version id 0.0.2-SNAPSHOT.
¿Is there any way to change automatically the version of the parents from 0.0.1-SNAPSHOT to 0.0.1 - the release that is going to be created-?
Thanks.
It should work when doing release:prepare from the aggregator project level that contains all the modules you're interested in. So your common, project artifacts and all the modules should be released at once. Also look at autoVersionSubmodules switch to do a little less typing.

Three-level pom (company - project - module) - problems with module project(s)

I am using Eclipse Indigo with m2e (which connects to an external binary of Maven 3.0.3).
Right now, the intended structure of my application is as follows:
Company-parent
--Project-parent
----Module1
----Module2
----ModuleN
I set up my poms such that Company-parent is the project of Project-parent, and Project-parent is the parent of all of the modules. The company-parent and the project-parent seem to be ok. Both are of packaging type POM.
The third level is where I start to encounter problems. I get a variety of maven errors, and all kinds of weird behavior, even with the simplest of module projects defined.
Company parent:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.scoresecret</groupId>
<artifactId>scs-global-parent</artifactId>
<version>1</version>
<packaging>pom</packaging>
</project>
Project parent:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<artifactId>scs-global-parent</artifactId>
<groupId>com.scoresecret</groupId>
<version>1</version>
</parent>
<artifactId>scs-model-parent</artifactId>
<groupId>com.scoresecret.model</groupId>
<version>1.0-SNAPSHOT</version>
<packaging>pom</packaging>
<modules>
<module>scs-model-core</module>
</modules>
</project>
Module pom:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<artifactId>scs-model-parent</artifactId>
<groupId>com.scoresecret.model</groupId>
<version>1.0-SNAPSHOT</version>
</parent>
<artifactId>scs-model-core</artifactId>
<version>1.0.0-SNAPSHOT</version>
<packaging>jar</packaging>
I can't do "Maven -> Update Project Configuration", because when I do, I get this error:
Failure to find org.apache.maven.plugins:maven-resources-plugin:pom:2.4.3 in https://my.archiva.location/archiva/repository/internal was cached in the local repository, resolution will not be reattempted until the update interval of scs.internal has elapsed or updates are forced pom.xml /scs-model-core line 1 Maven Configuration Problem
Some other errors I'm receiving:
Could not calculate build plan: Plugin org.apache.maven.plugins:maven-resources-plugin:2.4.3 or one of its dependencies could not be resolved: Failed to read artifact descriptor for org.apache.maven.plugins:maven-resources-plugin:jar:2.4.3 pom.xml /scs-model-core line 1 Maven Build Problem
Also receiving an error about m2e ignoring one of my plugin goals (from the company level pom) within the module. Am I missing something obvious here? Help!
Thanks :)
you shouldn't have the artifacts located like you described in a directory structure.
+-- Company-parent (pom.xml)
+-- Project-parent (pom.xml)
+----Module1 (pom.xml)
+----Module2 (pom.xml)
+----ModuleN (pom.xml)
To make a real good use of you company-parent (pom.xml) it should be helt into a separated area in version control and released separately in your repository manager (nexus, artifactory, archiva...). This will result in the following structure:
+-- Company-parent (pom.xml) (Separate Project)
The company-parent should be released as often as needed via the release-plugin. Lets assume we have released version 1.0 of the company-parent. The reason to have it separated is, cause the company-parent is used by many projects and not only by a single project.
and the real projects should be put into a separate folder (also in version control):
+-- Project-parent (pom.xml)
+----Module1 (pom.xml)
+----Module2 (pom.xml)
+----ModuleN (pom.xml)
So to use the company-parent in your project the project-parent has to look like:
<project....>
<modelVersion>4.0...</modelVersion>
<parent>
<artifactId>company-parent</artifactId>
<groupId>com.company.base</groupId>
<version>1.0</version>
</parent>
<packaging>pom</packaging>
<groupId>com.company.project1</groupId>
<artifactId>project-parent</artifactId>
<version>1.0.0-SNAPSHOT</version>
...
<dependencyManagement>
<!-- Project specific dependencies -->
</dependencyManagement>
<build>
<pluginManagement>
<plugins>
<!-- Project specific plugins. Better use them of the company pom -->
</plugins>
</pluginManagement>
</build>
<modules>
<module>module1</module>
<module>module2</module>
<module>moduleN</module>
</modules>
..
Now let us take a look into a module which should look like this:
<project....>
<modelVersion>4.0...</modelVersion>
<parent>
<artifactId>project-parent</artifactId>
<groupId>com.company.project1</groupId>
<version>1.0.0-SNAPSHOT</version>
</parent>
<packaging>jar</packaging>
<artifactId>module1</artifactId>
<dependencies>...</dependencies>
<build>..</build>
You should reference the parent pom with explicit path:
<parent>
<artifactId>scs-model-parent</artifactId>
<groupId>com.scoresecret.model</groupId>
<version>1.0-SNAPSHOT</version>
<relativePath>../pom.xml</relateivePath>
</parent>
Otherwise, you need to reinstall the parent pom into the repository every time you change it.
Sounds like maven isn't able to download a plugin (org.apache.maven.plugins:maven-resources-plugin:pom:2.4.3) from the repository https://my.archiva.location/archiva/repository/internal
Failure to find org.apache.maven.plugins:maven-resources-plugin:pom:2.4.3 in https://my.archiva.location/archiva/repository/internal was cached in the local repository, resolution will not be reattempted until the update interval of scs.internal has elapsed or updates are forced pom.xml /scs-model-core line 1 Maven Configuration Problem
If you browse to this URL in a browser, does it exist? I see it's using https - do you need a client cert or some form of basic auth credentials to access this url?
What are your current settings in your .m2/settings.xml show for repositories? Are you behind a company firewall or not connected to the internet which is forcing you to connect to a non-standard maven central repository?
It turns out, that my local repo contained files which ended with the extention ".lastUpdated" ... this was because I was having problems with authentication when I tried encrypting my maven passwords, so it wasn't downloading the dependencies. I deleted those files and simply ran "Maven > Update Dependencies" and it worked no problem, downloading the missing resources from my artifact repository.

Resources