Setting up Maven to allow deployment to different repositories easily - maven

I'm currently working in an environment where I define the rules for a lot of people. We currently use Hudson and Artifactory, and I want to evaluate if the switch to Jenkins and Nexus are worth the migration cost (but this is not the question).
To eval it, I have setup Maven, Jenkins, and Nexus locally, and I try to find a setup to use as much of the previous setup, so that I can compare the solutions. The problem here is:
When I use an existing POM and build and deploy it through Jenkins, it is automatically deployed to our old environment.
I have then tried to define the deploymentManagement section in my .settings file in Maven, but this is not allowed (see Configuring Maven, there
Note: the installation and user configuration cannot be used to add shared project information - for example, setting or company-wide.
I could of course copy the whole, and change the distributionManagement inside each POM, but I would like to use the same (not a copied) example in different installations.
Our current root POM contains the following section:
<distributionManagement>
<repository>
<uniqueVersion>false</uniqueVersion>
<id>company-central</id>
<name>Company central maven respository</name>
<url>https://company.com/artifactory/libs-releases</url>
</repository>
<snapshotRepository>
<id>company-snaps</id>
<name>company-snapshots</name>
<url>https://company.com/artifactory/libs-snapshots</url>
</snapshotRepository>
<downloadUrl>https://company.com/artifactory/libs-releases</downloadUrl>
</distributionManagement>
What is the easiest way to use the same POM, but to deploy it to different repository managers?
PS: I have read set up maven pom file for different deployment scenarios and don't like it (in my context), Deploying Maven artifact to multiple repositories with different settings is really a different question. Multiple deployments in maven is an interesting approach, but I would have to modify a lot of POMs only for this purpose.

I'm on a similar team, providing tools to others. We have our parent POM (not the settings.xml) set up like the below. The prod version is active unless one of my team members adds -DuseTestRepo=true to the mvn command line. You could experiment with changing the activation to look for a particular file that exists only on the Jenkins server (for example). I've also wondered if Maven interpolates properties in repo URLs. If it does, you could do <url>${my.remote.repo}/releases</url> and define my.remote.repo in settings.xml. Haven't tried that one.
<profiles>
<profile>
<id>test-repository</id>
<activation>
<property>
<name>useTestRepo</name>
<value>true</value>
</property>
</activation>
<properties>
</properties>
<distributionManagement>
<repository>
<!-- ... -->
</repository>
<snapshotRepository>
<!-- ... -->
</snapshotRepository>
</distributionManagement>
</profile>
<profile>
<id>prod-repository</id>
<activation>
<property>
<name>!useTestRepo</name>
</property>
</activation>
<properties>
</properties>
<distributionManagement>
<repository>
<!-- ... -->
</repository>
<snapshotRepository>
<!-- ... -->
</snapshotRepository>
</distributionManagement>
</profile>
As to delivering (deploying) your experimental parent POM - you would be deploying it to your test Nexus repo, right? And none of your developers are accessing that, only you. So Artifactory will contain the real com.company:corporate-parent:1.0 POM, and Nexus the com.company:corporate-parent:1.0 that contains the changes you require for your testing.
I would also consider changing the local repo in your settings.xml so you don't mix artifacts from the two remote repos.

Related

GitLab Package Registry as Maven Artifactory

I am trying to setup a GitLab project as the remote maven repository for all my other projects using GitLab Maven Package Registry. I have uploaded all dependency jar files to the package registry. All the jar files have been uploaded using maven's deploy feature. I am trying now to setup this project's package registry as the remote maven repository for all my other projects. The necessary repository settings is provided in the package registry itself, in each individual artifact page, as follows:
<repositories>
<repository>
<id>gitlab-maven</id>
<url>https://<gitlab_instance>/api/v4/projects/<project_id>/packages/maven</url>
</repository>
</repositories>
<distributionManagement>
<repository>
<id>gitlab-maven</id>
<url>https://<gitlab_instance>/api/v4/projects/<project_id>/packages/maven</url>
</repository>
<snapshotRepository>
<id>gitlab-maven</id>
<url>https://<gitlab_instance>/api/v4/projects/<project_id>/packages/maven</url>
</snapshotRepository>
</distributionManagement>
However, even after adding this in the pom.xml of other projects, their builds are failing citing dependency unavailability. It seems that this project's package registry is unreachable from other projects. What am I doing wrong here? Are there any additional steps I need to undertake?
Did you amend the settings.xml which you are using when calling a maven command?
There something like this:
<servers>
<server>
<id>gitlab-maven</id>
<configuration>
<httpHeaders>
<property>
<name>Job-Token</name>
<value>${env.CI_JOB_TOKEN}</value>
</property>
</httpHeaders>
</configuration>
</server>
</servers>
should be present. To use the settings.xml with your specific maven command, you can specify it by typing:
mvn -s settings.xml package
You need to add an authenticate header(personal token, CI job token, or deploy token).
This is the doc: Adding the GitLab Package Registry as a Maven remote

How to see what Maven is sending to a server during deploy?

I'm trying to use Github's new Actions CI server to deploy packages to Github's new packages feature. It's not going well.
I think it's all set up correctly, but I get this error:
Failed to execute goal org.apache.maven.plugins:maven-deploy-plugin:2.7:deploy
(default-deploy) on project myproject: Failed to deploy artifacts: Could not
find artifact com.mycompany:myproject:pom:1.5 in github
(https://maven.pkg.github.com/mycompany/mycompany_repository) -> [Help 1]
This happens after it appears to upload that same pom successfully:
Uploading to github: https://maven.pkg.github.com/mycompany/mycompany_repository
/com/mycompany/myproject/1.5/myproject-1.5.pom
Progress (1): myproject-1.5.pom (4.1/6.1 kB)
Progress (1): myproject-1.5.pom (6.1 kB)
So, it looks to me like it is successfully uploading the pom, but then it fails to download the same pom a few seconds later.
I'm running the deploy with debug switches on: mvn -X -e deploy, but I can't see the exact http commands that Maven is sending to the server.
How do I debug this? Is there some Maven/Aether transport or something that will log what is going on under the covers?
In case anyone else lands here looking for a solution to OPs issue publishing to github, I had a similar issue and found that the URLs needed in settings.xml and pom.xml are inconsistent. In your settings.xml, the repo URL needs to be of the form https://maven.pkg.github.com/myuser/com/mycompany/mypackage, whereas in your project's pom file, it needs to be of the form https://maven.pkg.github.com/myuser/mypackage. So, for example, your settings.xml file in ~/.m2 would look something like this:
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0
http://maven.apache.org/xsd/settings-1.0.0.xsd">
<activeProfiles>
<activeProfile>github</activeProfile>
</activeProfiles>
<profiles>
<profile>
<id>github</id>
<repositories>
<repository>
<id>central</id>
<url>https://repo1.maven.org/maven2</url>
<releases><enabled>true</enabled></releases>
<snapshots><enabled>false</enabled></snapshots>
</repository>
<repository>
<id>github</id>
<name>GitHub Apache Maven Packages</name>
<url>https://maven.pkg.github.com/myuser/com/mycompany/mypackage</url>
<releases><enabled>true</enabled></releases>
<snapshots><enabled>false</enabled></snapshots>
</repository>
</repositories>
</profile>
</profiles>
<servers>
<server>
<id>github</id>
<username>myuser</username>
<password>mypersonalaccesstoken</password>
</server>
</servers>
</settings>
Whereas the pom.xml file in the root of your project would need to look like this:
<project>
...
<groupId>org.mycompany</groupId>
<artifactId>mypackage</artifactId>
<version>1.0.0</version>
...
<distributionManagement>
<repository>
<id>github</id>
<name>GitHub Apache Maven Packages</name>
<url>https://maven.pkg.github.com/myuser/mypackage</url>
</repository>
</distributionManagement>
...
</project>
Other than this minor (but crucial) detail, my steps were the same as those outlined here. This allowed me to publish my Maven package to github package registry.
You can enable debug logging in the workflows.
Just add the secret:
ACTIONS_RUNNER_DEBUG
And set to true
See a similar answer here
I just spend 3 hours debugging why the guide on the page did not work for me. If you are following the guide posted here 1.
OWNER is your github username, and REPOSITORY is - you guessed it, the repo name.
Just remember to use lowercase in both OWNER and REPOSITORY.
When generating the personal access token, make sure the scopes for the token are the repo:* scopes as well as the more obvious write:packages and read:packages scopes (do not disable the repo scopes)
Otherwise it does just that
The following solution works for me:
Create a repository for packages e.g. maven-packages
Add <server></server> settings under <servers> in settings.xml: (do this per id used below)
<server>
<id>github</id>
<username>YOUR GITHUB USERNAME</username>
<password>A GITHUB TOKEN YOU CREATE FOR PUBLISHING PACKAGES</password>
</server>
Do NOT add <activeProfiles>, <profile> or <repositories> to settings.xml (only add <server> elements) as this is redundant for publishing and I am adding them to consuming projects' maven.xml so no need for duplication.
Add repository/ies to distributionManagement in pom.xml as follows:
<distributionManagement>
<snapshotRepository>
<id>github-snapshot</id>
<name>GitHub snapshot</name>
<url>https://maven.pkg.github.com/OWNER/maven-packages/</url>
<uniqueVersion>true</uniqueVersion>
</snapshotRepository>
<repository>
<id>github-release</id>
<name>GitHub release</name>
<url>https://maven.pkg.github.com/OWNER/maven-packages/</url>
<uniqueVersion>false</uniqueVersion>
</repository>
</distributionManagement>
Where OWNER is the GitHub account your project is / projects are under and maven-packages is the repositories you want to publish you project(s) to.
This enables using a dedicated repository for listing packages instead of publishing each project's package to a different (its own) repository, making consumption of multiple packages from your GitHub account easier, as you only need to configure a single repository for these packages:
<repositories>
<repository>
<id>github</id>
<name>GitHub</name>
<url>https://maven.pkg.github.com/OWNER/maven-packages/</url>
</repository>
</repositories>
Note: in the <servers> section of your settings.xml define a <server> per id used in repositories and distributionManagement e.g. github-snapshot, github-release, github in the above examples.

Maven add a repository for a specific OS

I am new to maven and I am trying to create a single pom file for both linux and windows. For dependency resolution, I have 2 p2 repositories. 1 for windows and the other is for linux.
I also have a target definition (.target) in which I am adding all external repositories for dependecy resolution.
Is there a way that I can add/activate a repository location per OS? Below is my trial.
Using maven pom profiles:
In this trial, I added all my repositories (common for windows and linux) to the target definition. I then created 2 new profiles and added the OS-specific p2 repo like so
<profile>
<id>Linux</id>
<activation>
<activeByDefault>false</activeByDefault>
<os><name>linux</name></os>
</activation>
<repositories>
<repository>
<id>p2_lin</id>
<url>http://example.com/data/LINUX_P2</url>
<layout>p2</layout>
</repository>
</repositories>
</profile>
<profile>
<id>Windows</id>
<activation>
<activeByDefault>false</activeByDefault>
<os><name>win32</name></os>
</activation>
<repositories>
<repository>
<id>trace_p2_win</id>
<url>http://example.com/data/WINDOWS_P2</url>
<layout>p2</layout>
</repository>
</repositories>
</profile>
While calling maven, I add this profile to the parameters mvn clean install -PWindows,Test
Unfortunately, the above configuration gives an unsatisfied dependency.
But when I remove this section from the parent pom file and add the correct repository (example: >http://example.com/data/WINDOWS_P2) to the target definition without the other one, everything seems to be fine.

Using multiple Maven mirrors to pull different jar version into a project

I am experiencing issues pulling in a new version of a library which lives in a different repository. I believe I have to update my settings.xml and my parent level pom.xml which specifies the new version.
Currently my settings.xml reads like the following:
<settings>
<mirrors>
<mirror>
<id>nexus</id>
<url>http://serenity.gm.edu/nexus/content/groups/public/</url>
<mirrorOf>central</mirrorOf>
</mirror>
<mirror>
<id>mods</id>
<url>http://7.169.72.8:8081/nexus/content/repositories/releases/</url>
<mirrorOf>central</mirrorOf>
</mirror>
</mirrors>
<profiles>
<profile>
<id>default</id>
<repositories>
<repository>
<id>mod-thirdparty</id>
<url>http://7.169.72.8:8081/nexus/content/repositories/thridparty/</url>
</repository>
<repository>
<id>mod-snapshots</id>
<url>http://7.169.72.8:8081/nexus/content/repositories/snapshots/</url>
</repository>
</repositories>
</profile>
</profiles>
<servers>
<server>
<id>Tomcat</id>
<username>admin</username>
<password>xxx</password>
</server>
</servers>
The default mirror where everything gets pulled in from is serenity. I am trying to pull in a custom library mod3.2 from 7.169.72.8 but version 2.3 keeps getting pulled in from serenity. I have tried a number of <mirrorOf> settings such as *,!central with no success.
My parent pom.xml is fairly straightforward defining a list of modules, proerties, and global dependencies. Here are the items of interest (it is too long to show).
<project ...>
...
<distributionManagement>
<repository>
<id>releases</id>
<name>mod2-releases</name>
<url>http://7.169.72.8:8081/nexus/content/repositories/releases</url>
</repository>
<snapshotRepository>
<id>snapshots</id>
<name>mod2-snapshots</name>
<url>http://7.169.72.8:8081/nexus/content/repositories/snapshots</url>
</snapshotRepository>
</distributionManagement>
<properties>
...
<mod2.version>3.2</mod2.version>
...
</properties>
...
<dependencyManagement>
<dependencies>
<dependency>
<groupId>mod2.commons</groupId>
<artifactId>mod2.commons.lang</groupId>
<version>${mod2.version}</version>
</dependency>
<dependency>
<groupId>mod2.commons</groupId>
<artifactId>mod2.commons.audit</groupId>
<version>${mod2.version}</version>
</dependency>
</dependencies>
</dependencyManagement>
</project>
To recap - I'm trying to get version mod2 version 3.2 from 7.169.72.8 but all I can pull in mod2 version 2.3 from the serenity repo. Thanks for any help.
In your configuration, you are declaring multiple mirrors for the Central repository. However, Maven does not support that:
Note that there can be at most one mirror for a given repository. In other words, you cannot map a single repository to a group of mirrors that all define the same value. Maven will not aggregate the mirrors but simply picks the first match. If you want to provide a combined view of several repositories, use a repository manager instead.
In your case, Maven is picking the first mirror of Central, which is of id nexus, and ignoring the second one, which is of id mods. Therefore, when fetching for your artifact, it only searches nexus and fails to find it.
There are a couple of solutions:
If you really want to have multiple mirrors for the same repository, you will need to use a repository manager and configure this on the repo manager itself.
It's not clear why you need to mirror Central and point to 7.169.72.8:8081 if this repository doesn't have all the artifacts needed to make your build work. If the rest are hosted on serenity.gm.edu, you could host all of them on serenity.gm.edu.
You could also not use the mods mirror at all. Let serenity.gm.edu be the mirror for Central and just declare a new repository that isn't Central to fetch your artifacts. You already declared the mod-thirdparty and mod-snapshots repositories, so this is a matter of activating the profile they are under with:
<activeProfiles>
<activeProfile>default</activeProfile>
</activeProfiles>
in the settings.xml. With this, Maven will also look for your artifacts at the URL configured for those repo (and those are not mirrored by nexus).

In Maven how do I copy files using the wagon plugin?

Summary: How do I copy some generated files into a webserver (eg IIS or Apache) directory using Maven?
Details:
I have a working application that builds in Maven. I've managed to get it building using the webstart-maven-plugin which produces all the needed files (.jar and .jnlp) in a directory target/jnlp. It also creates a zip file with them all in at target/foo-1.0.zip.
At the moment the webstart plugin does not have a deploy goal - the request for it has ended up on the FAQ (question 3). It may be implemented in future, but the suggestion for the time being is to use wagon-maven-plugin.
I've never used Wagon. To start with I'd like to just copy the files to a local directory served up by a webserver. Later I'd like to copy them remotely, probably using ftp. Can someone give an example to what I need to add to the pom.xml to get the local copy working (and hopefully an ftp example too?). I can't find it in the documentation. From reading I think I might also need the Wagon Maven File Provider but as this seems to have almost no documentation I'm not sure.
Wagon providers are only there to provide additional network protocol supports (such as FTP).
If you want to copy file to a webserver (local or distant) you can use Maven upload plugin :
...
<plugin>
<groupId>com.atlassian.maven.plugins</groupId>
<artifactId>maven-upload-plugin</artifactId>
</plugin>
...
In parent pom :
<plugin>
<groupId>com.atlassian.maven.plugins</groupId>
<artifactId>maven-upload-plugin</artifactId>
<version>1.1</version>
<configuration>
<resourceSrc>
${project.build.directory}/${project.build.finalName}.${project.packaging}
</resourceSrc>
<resourceDest>${jboss.deployDir}</resourceDest>
<serverId>${jboss.host}</serverId>
<url>${jboss.deployUrl}</url>
</configuration>
</plugin>
And to configure parameters in a smart way, I use maven profiles (in parent pom) :
<profiles>
<!-- local deployment -->
<profile>
<id>developpement</id>
<properties>
<jboss.host>localhost</jboss.host>
<jboss.deployDir>appli/jboss-4.0.4.GA/server/default/deploy/</jboss.deployDir>
<jboss.deployUrl>file://C:/</jboss.deployUrl>
</properties>
</profile>
<!-- distant deployment -->
<profile>
<id>validation</id>
<properties>
<jboss.host>ENV_val</jboss.host>
<jboss.deployDir>/home/envval/jboss/server/default/deploy/</jboss.deployDir>
<jboss.deployUrl>scp://PROJECT_LAN_HOST</jboss.deployUrl>
</properties>
</profile>
</profiles>
I've created an "ant launcher", to use it by clicking under Eclipse ant view :
<target name="copy war to JBoss local" description="Copy war to local JBoss">
<maven goal="upload:upload" options="-Pdeveloppement" />
</target>
But you can simply run it on a command line :
mvn upload:upload -Pdeveloppement
EDIT : By the way, for distant deployment, you may need a login password for scp to work. You have to add them to you Maven settings.xml file :
<settings>
...
<servers>
<server>
<id>ENV_val</id>
<username>login</username>
<password>password</password>
</server>
</servers>
...
</settings>
EDIT: You'll need to add the Atlassian repository:
<pluginRepositories>
<pluginRepository>
<id>Atlassian</id>
<url>https://maven.atlassian.com/repository/public</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</pluginRepository>
</pluginRepositories>
EDIT: depending upong the remote protocol you'll have to add wagon extensions, see Uploading a directory using sftp with Maven
In the end I didn't use the Maven upload plugin - it seemed a bit limited and not part of the main maven distribution. I used the maven wagon plugin as suggested. Here is the simplest possible pom that I could make that worked. Hopefully others will find it useful, as I couldn't find anything similar easily.
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>wagon-maven-plugin</artifactId>
<version>1.0-beta-3</version>
<configuration>
<fromDir>${project.build.directory}/jnlp</fromDir>
<includes>*</includes>
<url>file://c:/inetpub/wwwroot</url>
<toDir>jnlp</toDir>
</configuration>
</plugin>
For remote distributions, you just change the URL type, and possibly add wagon extensions as necessary.

Resources