I am trying to release this Scala library to Maven using Nexus OSS repository manager (sonatype).
My library has a dependency on the latest version of the Play Framework (com.typesafe.play:play_2.13:2.8.18), which in turn depends on an old version of com.fasterxml.jackson (2.11.4). In my build configuration I have overriden the dependency using:
dependencyOverrides += "com.fasterxml.jackson.core" % "jackson-databind" % "2.13.4"
I have verified that the build uses the correct version of jackson-databind using the sbt dependency tree plugin, and also by inspecting the cache files.
Despite this, when I release to maven by using the sbt-sonatype plugin (version 3.9.14), my release is rejected by the Sonatype Lift vulnerability scanner because of vulnerabilities in pkg:maven/com.fasterxml.jackson.core/jackson-databind#2.11.4, i.e. an older version than the library that is actually used in my build, as specified in dependencyOverrides.
When I examine the build artifacts that are automatically submitted to the Nexus repo manager, the only dependencies that are mentioned are top-level dependencies are specified in the POM file (see below). So it seems that Sonatype Lift looks at these top-level dependencies, and walks the dependency graph looking for vulnerabilities in the implied dependencies, ignoring the fact that I have explicitly overriden the version of jackson-databind in my build.
Is there any way for the dependency override to be propagated to the POM?
<?xml version='1.0' encoding='UTF-8'?>
<project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://maven.apache.org/POM/4.0.0">
<modelVersion>4.0.0</modelVersion>
<groupId>com.mesonomics</groupId>
<artifactId>play-hmac-signatures_2.13</artifactId>
<packaging>jar</packaging>
<description>play-hmac-signatures</description>
<url>https://github.com/phelps-sg/play-hmac-signatures</url>
<version>0.2.2</version>
<licenses>
<license>
<name>Apache-2.0</name>
<url>http://www.apache.org/licenses/LICENSE-2.0</url>
<distribution>repo</distribution>
</license>
</licenses>
<name>play-hmac-signatures</name>
<organization>
<name>com.mesonomics</name>
<url>https://github.com/phelps-sg/play-hmac-signatures</url>
</organization>
<scm>
<url>https://github.com/phelps-sg/play-hmac-signatures</url>
<connection>git#github.com:phelps-sg/play-hmac-signatures.git</connection>
</scm>
<developers>
<developer>
<id>phelps-sg</id>
<name>Steve Phelps</name>
<url>https://github.com/usernamehttps://github.com/phelps-sg</url>
<email>sphelps#sphelps.net</email>
</developer>
</developers>
<dependencies>
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<version>2.13.10</version>
</dependency>
<dependency>
<groupId>com.typesafe.play</groupId>
<artifactId>play_2.13</artifactId>
<version>2.8.18</version>
</dependency>
<dependency>
<groupId>com.google.inject</groupId>
<artifactId>guice</artifactId>
<version>5.1.0</version>
</dependency>
<dependency>
<groupId>org.scalactic</groupId>
<artifactId>scalactic_2.13</artifactId>
<version>3.2.14</version>
</dependency>
<dependency>
<groupId>org.scalatest</groupId>
<artifactId>scalatest_2.13</artifactId>
<version>3.2.14</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.scalatestplus.play</groupId>
<artifactId>scalatestplus-play_2.13</artifactId>
<version>5.1.0</version>
<scope>test</scope>
</dependency>
</dependencies>
</project>
It seems that the artifact is in fact eventually published on Maven, despite the reported vulnerabilities, and it was simply a case of waiting.
Related
With Apache Maven 3.3.9, We are using maven ranges for some dependencies version resolution as there are a lot of maven modules in my application an keeping the specific dependency version and changing to each release is bit pain.
To ilusstrate the structire of current application its like
pom.xml (parent)
<groupId>com.myapp</groupId>
<artifactId>myapp</artifactId>
<version>0.0.5-SNAPSHOT</version>
<packaging>pom</packaging>
<name>parentpom</name>
<modules>
<module>../myapp-service</module>
</modules>
<dependencies>
<dependency>
<groupId>com.someotherapp</groupId>
<artifactId>someotherappA</artifactId>
<version>[1.1.0, 1.2.0)</version>
</dependency>
<dependency>
<groupId>com.someotherapp</groupId>
<artifactId>someotherappB</artifactId>
<version>[1.1.0, 1.2.0)</version>
</dependency>
</dependencies>
pom.xml (child module)
<parent>
<groupId>com.myapp</groupId>
<artifactId>myapp</artifactId>
<version>[0.0.1,2.0.0)</version>
<relativePath>../myapp/pom.xml</relativePath>
</parent>
<groupId>com.myapp.service</groupId>
<artifactId>myapp-service</artifactId>
<version>1.0.9-SNAPSHOT</version>
<packaging>jar</packaging>
<name>myapp-service</name>
<dependencies>
<dependency>
<groupId>com.someotherapp</groupId>
<artifactId>ssomeotherappA</artifactId>
</dependency>
<dependency>
<groupId>com.someotherapp</groupId>
<artifactId>someotherappB</artifactId>
</dependency>
</dependencies>
EXPECT : I expect the child module should pick the last released dependency version(no snapshot) like someotherappA-1.2.0 and not the latest version ´someotherappA-1.2.1-SNAPSHOT´
PROBELM : If available dependency version in artifactory are like ´someotherappA-1.2.1-SNAPSHOT´(latest snapshot) and ´someotherappA-1.2.0´(stable release), then maven artifact ´myapp-service´ pick SNAPSHOT dependencies rather release version,even tough i am specifying -DallowSnapshots=false which means snapshot versions are not allowed in maven version ranges.
Here is what i am using to build my child module
mvn versions:resolve-ranges -f "myapp-service/pom.xml" -DgenerateBackupPoms=true -DallowSnapshots=false -DprocessDependencyManagement=false -DprocessParent=true
mvn clean install
I found similar problem here how do I stop maven version ranges from using snapshots and seems there is an open bug in maven https://issues.apache.org/jira/browse/MNG-3092 . However, I am looking possible workaround or solution if someone bypassed this similar problem.
I'm trying to construct a codebase where subsystems can be developed as maven modules, without the importing POM needing to concern itself with the internal structure of the maven module.
The "importing" pom
<project>
<artifactId>application</artifactId>
<packaging>jar</packaging>
<dependencies>
<dependency>
<artifactId>submodule-1</artifactId>
</dependency>
</dependencies>
</project>
The "imported" pom
<project>
<artifactId>submodule-1</artifactId>
<packaging>pom</packaging>
<modules>
<module>api</module>
<module>implementation</module>
</modules>
<dependencies>
<dependency>
<artifactId>api</artifactId>
</dependency>
<dependency>
<artifactId>implementation</artifactId>
</dependency>
</dependencies>
</project>
This does seem to work, at least partially; the generated JARs appear to be on the classpath during mvn package. IntelliJ shows the application has a dependency on submodule-1 and transitively on api and implementation. However, mvn dependency:tree fails while building submodule-1 saying
Could not resolve dependencies for project submodule-1:pom:1.0.0-SNAPSHOT: Could not find artifact api:jar:1.0.0-SNAPSHOT
I'm trying to determine if this is a valid pattern, that of having packaging of pom, with defined dependencies which are also defined modules in the POM.
Have I stumbled upon a working-but-not-supported edge case, or is the dependency plugin broken in some way, or I'm breaking it in some way, or something else?
We use a Maven BOM to manage dependencies of a suite of libraries. The dependencyManagement section of the BOM generally uses version ranges to specify versions of these libraries, e.g., [2.0,2.1) Child pom.xml's using the BOM do not specify versions for these managed dependencies. (Edit for clarification: we use specific versions for third party dependencies, the ranges are used for internal libraries that are undergoing development, where the versions can change rapidly. We define version ranges to ensure broad compatibility between these libraries, i.e. all within the same major version.)
(Note that this is not a multi-module project. Libraries and service projects using the BOM mechanism just declare it as a parent and pull it from a Nexus repository. they are not built together.)
We also have some build system scripts that use versions:resolve-ranges to pin the versions of dependencies appearing in our library and service pom.xml's (not the BOM's pom.xml). These pom.xml's with resolved ranges are checked in to source control and tagged, so that if we need to roll back a deployment to an earlier version, we can use that tagged pom.xml to make a build that uses the same dependency versions as the original build, even if a newer version of a dependency is now available (and thus resolve-ranges would come up with the newer version if we reran it).
I just noticed that these two mechanisms are not working well together. Running versions:resolve-ranges on the library or service pom.xml only resolves the ranges in that pom.xml. Versions under dependency management are still not specified, so if we made a new build using this pom.xml, we'd get the latest dependency version in range at build time. Not what we want!
Is there a way to use versions:resolve-ranges (or any other plugin or technique) to resolve the managed versions and stick them into the child pom.xml?
Here is a contrived example.
The BOM:
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.maventest</groupId>
<artifactId>myproject</artifactId>
<packaging>pom</packaging>
<version>1.0-SNAPSHOT</version>
<name>myproject</name>
<url>http://maven.apache.org</url>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>commons-lang</groupId>
<artifactId>commons-lang</artifactId>
<version>[2.0, 2.3]</version>
</dependency>
</dependencies>
</dependencyManagement>
</project>
Child project using the BOM (one managed dependency, one unmanaged):
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<parent>
<groupId>com.maventest</groupId>
<artifactId>myproject</artifactId>
<version>1.0-SNAPSHOT</version>
<relativePath>../myproject/pom.xml</relativePath>
</parent>
<modelVersion>4.0.0</modelVersion>
<groupId>com.maventest</groupId>
<artifactId>mytest</artifactId>
<packaging>jar</packaging>
<version>1.0-SNAPSHOT</version>
<name>mytest</name>
<url>http://maven.apache.org</url>
<dependencies>
<dependency>
<groupId>commons-lang</groupId>
<artifactId>commons-lang</artifactId>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>[3.8, 3.9)</version>
<scope>test</scope>
</dependency>
</dependencies>
</project>
Snippet from mvn dependency:tree showing effective versions of dependencies:
[INFO] com.maventest:mytest:jar:1.0-SNAPSHOT
[INFO] +- commons-lang:commons-lang:jar:2.3:compile
[INFO] \- junit:junit:jar:3.8.2-brew:test
Dependency section from mytest pom.xml after mvn versions:resolve-ranges:
<dependencies>
<dependency>
<groupId>commons-lang</groupId>
<artifactId>commons-lang</artifactId>
<scope>compile</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.2-brew</version>
<scope>test</scope>
</dependency>
</dependencies>
So the unmanaged dependency is resolved, as expected. But the managed one is not. How can I get it to be resolved too?
Forgot about this question! In the end I could never find a way to pin the managed versions which were based on ranges. So I did stop defining the versions in the BOM and just specified them with ranges in each child pom. More boilerplate in the child poms, but not that bad.
We were still able to define properties that specified the ranges in the BOM which the children could use, making it a bit easier to bump all the ranges all when necessary.
Problem description
We have a Maven aggregator pom with some child poms (modules) all having the same version:
pom.xml (parent zoo, version 2.0.0)
|-- pom.xml (child module cat, version 2.0.0)
|-- pom.xml (child module dog, version 2.0.0)
|-- ...
Within the dependency management section all children are declared with the project version to ease declaration of dependencies.
The parent pom looks like
<groupId>com.acme</groupId>
<artifactId>zoo</artifactId>
<version>2.0.0</version>
<packaging>pom</packaging>
<modules>
<module>cat</module>
<module>dog</module>
</modules>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.acme</groupId>
<artifactId>cat</artifactId>
<version>${project.version}</version>
</dependency>
<!-- other child modules go here -->
</dependencies>
</dependencyManagement>
The child poms are defined as
<parent>
<groupId>com.acme</groupId>
<artifactId>zoo</artifactId>
<version>2.0.0</version>
</parent>
<groupId>com.acme</groupId>
<artifactId>cat</artifactId>
<dependencies>
<dependency>
<groupId>com.acme</groupId>
<artifactId>dog</artifactId>
</dependency>
</dependencies>
There is another pom which declares the parent pom as its parent too (inheritance) but is not listed as sub module in this parent (no aggregation). This pom has a different version.
<parent>
<groupId>com.acme</groupId>
<artifactId>zoo</artifactId>
<version>2.0.0</version>
</parent>
<groupId>com.acme</groupId>
<artifactId>boo</artifactId>
<version>1.0.0</version>
<dependencies>
<dependency>
<groupId>com.acme</groupId>
<artifactId>dog</artifactId>
</dependency>
</dependencies>
Actually we have expected that the version of the dependency com.acme.dog is pulled from the dependency management section of the parent pom com.acme.zoo and is equal to 2.0.0. However the Maven documentation on project interpolation and variables says
One factor to note is that these variables are processed after inheritance as outlined above. This means that if a parent project uses a variable, then its definition in the child, not the parent, will be the one eventually used.
That is: in the reactor build the variable ${project.version} used in the dependency management section of the parent pom com.acme.zoo is evaluated with respect to com.acme.bar and equal to 1.0.0 what is not as intended.
Note
There is a workaround with using a variable in the parent pom which has to be kept in sync with the parent pom versions. However, this solution is incompatible with the Maven Release Plugin.
Question
How can we achieve the desired behaviour
aggregator pom with children having the same version
declaration of children in the dependency management section to ensure that all dependencies have the same version
use of inheritance together with different versions
compatibility with maven-release-plugin
without the pitfalls of project interpolation of variables?
The maven release plugin is able to change the versions of the dependencies managed in the parent pom.
So if you define your maven parent like this:
<groupId>com.acme</groupId>
<artifactId>zoo</artifactId>
<version>2.0.0-SNAPSHOT</version>
<packaging>pom</packaging>
<modules>
<module>cat</module>
<module>dog</module>
</modules>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.acme</groupId>
<artifactId>cat</artifactId>
<version>2.0.0-SNAPSHOT</version>
</dependency>
<!-- other child modules go here -->
</dependencies>
</dependencyManagement>
As you see the versions of the parent and the managed dependency are the same. I set them to a SNAPSHOT version because the release plugin will create the final versions on release:perform
Your child poms can stay as you had them.
Because in your setup, your parent project is also the reactor you can then call
mvn release:perform -DautoVersionSubmodules=true
which will update the version of the parent in all submodules when you run this command. That option is essentially the same as if you run
mvn versions:update-child-modules
meaning it will change the child poms.
After you run the mvn release:perform command your parent pom will look like this:
<groupId>com.acme</groupId>
<artifactId>zoo</artifactId>
<version>2.0.1-SNAPSHOT</version>
<packaging>pom</packaging>
<modules>
<module>cat</module>
<module>dog</module>
</modules>
<dependencyManagement>
<dependencies>
<dependency>
<groupId>com.acme</groupId>
<artifactId>cat</artifactId>
<version>2.0.1-SNAPSHOT</version>
</dependency>
<!-- other child modules go here -->
</dependencies>
</dependencyManagement>
and your child poms like this
<parent>
<groupId>com.acme</groupId>
<artifactId>zoo</artifactId>
<version>2.0.1-SNAPSHOT</version>
</parent>
<groupId>com.acme</groupId>
<artifactId>cat</artifactId>
<dependencies>
<dependency>
<groupId>com.acme</groupId>
<artifactId>dog</artifactId>
</dependency>
</dependencies>
The final versions will only exist in the tag created by the release:prepare command.
PS: You may define other versions for the final and the next development version when they are prompted after running the release:prepare command.
The simplest solution is modify pom of zoo and replace <version>${project.version}</version> with <version>2.0.0</version>
Please note:
when you change version to next number, for example 2.0.1, with
versions-maven-plugin, dependency management section will be also
updated.
Spring use simplest solution, see
http://central.maven.org/maven2/org/springframework/spring-framework-bom/4.2.7.RELEASE/spring-framework-bom-4.2.7.RELEASE.pom
Summary: using <version>${project.version}</version> in dependency management is wrong idea.
From Maven Introduction to the pom : http://maven.apache.org/guides/introduction/introduction-to-the-pom.html
Project Inheritance > Example 1 > The Solution
Alternatively, if we want the groupId and / or the version of your
modules to be the same as their parents, you can remove the groupId
and / or the version identity of your module in its POM.
<project>
<parent>
<groupId>com.mycompany.app</groupId>
<artifactId>my-app</artifactId>
<version>1</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>my-module</artifactId>
</project>
My approach to that is to track it in the child POM. It's a bit less typing overall, close to where the actual dependency lives and is low maintenance for most projects. YMMV
<dependencies>
...
<dependency>
<groupId>${project.groupId}</groupId>
<artifactId>foo-sibling</artifactId>
<version>${project.version}</version>
</dependency>
...
</dependencies>
I was trying to set up a Maven project that will contain user defined functions (UDFs) that I'd like to use in my Hive queries. I started with a Maven project containing no source files, and the following POM:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>exp</groupId>
<artifactId>HiveUdfTestProject</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven.compiler.source>1.7</maven.compiler.source>
<maven.compiler.target>1.7</maven.compiler.target>
</properties>
<dependencies>
<dependency>
<groupId>org.apache.hive</groupId>
<artifactId>hive-exec</artifactId>
<version>0.14.0</version>
</dependency>
</dependencies>
</project>
When I tried to build the project, I get the following error:
Failed to execute goal on project HiveUdfTestProject: Could not
resolve dependencies for project
exp:HiveUdfTestProject:jar:1.0-SNAPSHOT: The following artifacts could
not be resolved:
org.apache.calcite:calcite-core:jar:0.9.2-incubating-SNAPSHOT,
org.apache.calcite:calcite-avatica:jar:0.9.2-incubating-SNAPSHOT:
Could not find artifact
org.apache.calcite:calcite-core:jar:0.9.2-incubating-SNAPSHOT -> [Help
1]
I found the calcite-core-incubating jar in the maven central repository (but not the incubating-snapshot version) required by the hive-exec 0.14.0 dependency.
Adding the calcite-core from maven central got rid of the original error, and introduced a new missing dependency "pentaho-aggdesigner-algorithm" which I found on ConJars.
Adding the conjars repo and the pentaho dependency made a new missing dependency appear "org.apache.calcite:calcite-avatica:jar:0.9.2-incubating-SNAPSHOT" whose incubating (but not snapshot) dependency was available in the maven central repo.
Adding the calcite-avatica dependency to the POM made the empty project build successfully at last.
Here is the final POM needed to make a project intended for Hive UDFs build:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>exp</groupId>
<artifactId>HiveUdfTestProject</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<maven.compiler.source>1.7</maven.compiler.source>
<maven.compiler.target>1.7</maven.compiler.target>
</properties>
<repositories>
<repository>
<id>conjars.org</id>
<url>http://conjars.org/repo</url>
</repository>
</repositories>
<dependencies>
<!-- From Maven Central -->
<dependency>
<groupId>org.apache.hive</groupId>
<artifactId>hive-exec</artifactId>
<version>0.14.0</version>
</dependency>
<dependency>
<groupId>org.apache.calcite</groupId>
<artifactId>calcite-core</artifactId>
<version>0.9.2-incubating</version>
</dependency>
<dependency>
<groupId>org.apache.calcite</groupId>
<artifactId>calcite-avatica</artifactId>
<version>0.9.2-incubating</version>
</dependency>
<!-- From conjars -->
<dependency>
<groupId>org.pentaho</groupId>
<artifactId>pentaho-aggdesigner-algorithm</artifactId>
<version>5.1.3-jhyde</version>
</dependency>
</dependencies>
</project>
Once the empty project built, I tried integrating the POM settings into a larger existing Maven project and saw errors about calcite-core specifically looking for the snapshot version. To get past this, I changed the hive-exec dependency to look like this:
<dependency>
<groupId>org.apache.hive</groupId>
<artifactId>hive-exec</artifactId>
<version>0.14.0</version>
<exclusions>
<exclusion>
<groupId>org.apache.calcite</groupId>
<artifactId>*</artifactId>
</exclusion>
</exclusions>
</dependency>
I explicitly included the calcite-core and calcite-avatica projects as dependencies, and my project (which also includes the Hive 14 dependencies), no longer failed with the 'artifacts could not be resolved error'
<dependency>
<groupId>org.apache.calcite</groupId>
<artifactId>calcite-core</artifactId>
<version>1.0.0-incubating</version>
</dependency>
<dependency>
<groupId>org.apache.calcite</groupId>
<artifactId>calcite-avatica</artifactId>
<version>1.0.0-incubating</version>
</dependency>
From what I can tell, this is an open issue with Hive 14. See https://issues.apache.org/jira/browse/HIVE-8906 for more info.
I've resolved the same issue by adding below dependencies to /ql/pom.xml
org.pentaho
pentaho-aggdesigner-algorithm
5.1.3-jhyde
<dependency>
<groupId>eigenbase</groupId>
<artifactId>eigenbase-properties</artifactId>
<version>1.1.4</version>
</dependency>
<dependency>
<groupId>net.hydromatic</groupId>
<artifactId>linq4j</artifactId>
<version>0.4</version>
</dependency>
and below repository to /pom.xml under positories
<repository>
<id>conjars</id>
<name>Concurrent Conjars repository</name>
<url>http://conjars.org/repo</url>
<layout>default</layout>
</repository>