I am trying to resolve dependency version conflicts while using the below dependencies.
The worst one I am facing is zucchini project supports the apache commons-io versions from 1.4 to latest one. It does not support versions
below 1.4
and at the same time pagerduty-client supports commons-io versions below 1.4 version.
So It is not possible to specify a common version of this dependency (dependency management)
which supports in both zucchini and pager-duty client (both are third party libraries).
In this particular situation I couldn't find a possible way to resolve this issue. Any help will be appreciated.
<dependency>
<groupId>com.comcast.zucchini</groupId>
<artifactId>zucchini</artifactId>
<version>[2.2.5,)</version>
</dependency>
<dependency>
<groupId>com.github.dikhan</groupId>
<artifactId>pagerduty-client</artifactId>
<version>3.0.2</version>
</dependency>
Possibility 1
If the old and new commons-io package/class names are a close enough match, excluding the old dependency from pagerduty-client could possibly work.
https://maven.apache.org/guides/introduction/introduction-to-optional-and-excludes-dependencies.html
<dependency>
<groupId>com.github.dikhan</groupId>
<artifactId>pagerduty-client</artifactId>
<version>3.0.2</version>
<exclusions>
<exclusion>
<groupId>org.apache.commons</groupId>
<artifactId>commons-io</artifactId>
</exclusion>
</exclusions>
</dependency>
This relies on the binary API of commons-io between versions 1.3.2 and 2.x being similar enough.
There does seem to be lots of overlap, looking at the code of each version:
https://github.com/apache/commons-io/tree/commons-io-1.3.2/src/java/org/apache/commons/io
https://github.com/apache/commons-io/tree/commons-io-2.5/src/main/java/org/apache/commons/io
Possibility 2
Split up your application so that the commons-io dependency is not shared and does not conflict.
It could be that the pagerduty-client and zucchini parts of your application do not need to be 'bundled' together, so split them up.
If they do need work together then you could still have two apps/processes and send messages between them.
Note
I cloned the pagerduty-client repo and changed the commons-io dependency from org.apache.commons:commons-io:1.3.2 to commons-io:commons-io:2.5 and the tests worked, so maybe you can suggest to the project owner that they upgrade commons-io.
And looking at the code it seems commons-io is hardly used (one place, HttpApiServiceImpl.java):
\pagerduty-client>findstr /s /c:"commons" *.java
src\main\java\com\github\dikhan\pagerduty\client\events\domain\AcknowledgeIncident.java:import org.apache.commons.lang3.StringUtils;
src\main\java\com\github\dikhan\pagerduty\client\events\domain\Incident.java:import org.apache.commons.lang3.StringUtils;
src\main\java\com\github\dikhan\pagerduty\client\events\domain\Incident.java:import org.apache.commons.lang3.builder.Builder;
src\main\java\com\github\dikhan\pagerduty\client\events\domain\Payload.java:import org.apache.commons.lang3.StringUtils;
src\main\java\com\github\dikhan\pagerduty\client\events\domain\ResolveIncident.java:import org.apache.commons.lang3.StringUtils;
src\main\java\com\github\dikhan\pagerduty\client\events\HttpApiServiceImpl.java:import org.apache.commons.io.IOUtils;
src\main\java\com\github\dikhan\pagerduty\client\events\PagerDutyEventsClient.java:import org.apache.commons.lang3.StringUtils;
src\main\java\com\github\dikhan\pagerduty\client\events\utils\FakePagerDutyEventsClient.java:import org.apache.commons.lang3.StringUtils;
As your commons-io is the problem you'll have to look further up the line. That means either upgrade pagerduty-client to a version that uses a newer version of commons-io that Cucumber likes, or downgrade zucchini to require a version of Cucumber that works with pagerduty-client as well.
This is a common problem with some jakarta commons packages, they at some point decided to massively change the public interface without changing the package name, causing conflicts like this for users.
You may be in luck, I once worked on a project where we had to rewrite thousands of lines of code just so we could link to a library we desperately needed that depended on a newer version of commons-io than the one we'd been using.
Related
Background
commons-beanutils-core version 1.8.0 has some security issues that I am trying to avoid.
So, I am using commons-beanutils 1.9.4
Problem
I cannot seem to prevent other libraries from importing commons-beanutils-core version 1.8.0. Nowhere in my pom file do I include it. I am assuming some other dependency implicitly includes it. And, since commons-beanutils-core no longer is supported and they have moved to just using commons-beanutils for all newer versions, when I explicitly write:
<dependency>
<groupId>commons-beanutils</groupId>
<artifactId>commons-beanutils</artifactId>
<version>1.9.4</version>
</dependency>
It does not remove the implicit references to commons-beanutils-core.
Question
How do I make the other packages not download commons-beanutils-core version 1.8.0?
Notes:
There have been other packages that were downloaded as dependencies that I did not explicitly include in my pom.xml. A scan show that some of these packages were security risks. So, my solution was to just explicitly include a higher version of the package that was previously included implicitly. And that removed the old version of the package. But that solution does not work here since commons-beanutils-core is no longer used in the newest version.
Update 1
I have learned that including the following code will make the pom.xml think that 1.8.0 will be provided, and in a sense it will since I will include 1.9.4. But I am unsure if the code will use the 1.9.4 if it thinks it should look for 1.8.0. This code enables me to remove 1.8.0 but I don't know if my deception here will cause problems down the line.
<dependency>
<groupId>commons-beanutils</groupId>
<artifactId>commons-beanutils-core</artifactId>
<version>1.8.0</version>
<scope>provided</scope>
</dependency>
Using provided scope can have side effects on runtime. Provided scope dependencies are used in compilation but not packaged so if the commons-beanutils-core has a method which is not present in commons-beanutils you will get NoSuchMethodException.
Better solution will be use mvn dependency:tree to find which dependencies had dependency on commons-beanutils-core and
Either update the dependency to a version which uses 1.9.4 version of commons-beanutils.
Or use exclusion to exclude commons-beanutils-core from the dependencies using commons-beanutils-core as a dependency.
If you can use method 1 it's better if you cannot find a dependency which uses the latest version, then use method 2.
I am trying to run Java code from a Maven project that uses both beam-runners-google-cloud-dataflow-java and bigtable-client-core, and I cannot get it to properly reconcile dependencies amongst these two. When I run and attempt to create a BigtableDataClient, I get the following error:
java.lang.NoSuchFieldError: TE_HEADER
at io.grpc.netty.shaded.io.grpc.netty.Utils.<clinit> (Utils.java:74)
at io.grpc.netty.shaded.io.grpc.netty.NettyChannelBuilder.<clinit> (NettyChannelBuilder.java:72)
at io.grpc.netty.shaded.io.grpc.netty.NettyChannelProvider.builderForAddress (NettyChannelProvider.java:37)
at io.grpc.netty.shaded.io.grpc.netty.NettyChannelProvider.builderForAddress (NettyChannelProvider.java:23)
at io.grpc.ManagedChannelBuilder.forAddress (ManagedChannelBuilder.java:39)
at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createSingleChannel (InstantiatingGrpcChannelProvider.java:242)
at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createChannel (InstantiatingGrpcChannelProvider.java:198)
at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.getTransportChannel (InstantiatingGrpcChannelProvider.java:185)
at com.google.api.gax.rpc.ClientContext.create (ClientContext.java:160)
at com.google.cloud.bigtable.data.v2.stub.EnhancedBigtableStub.create (EnhancedBigtableStub.java:151)
at com.google.cloud.bigtable.data.v2.BigtableDataClient.create (BigtableDataClient.java:138)
at com.google.cloud.bigtable.data.v2.BigtableDataClient.create (BigtableDataClient.java:130)
...
I can only conclude this is due to an issue with version conflict on the relevant libraries (either grpc-netty or grpc-netty-shaded); I'm using 1.17 for grpc-netty and 1.23 for grpc-netty-shaded. I've tried using dependencyManagement to force the use of version 1.23.0 for both grpc-netty and grpc-netty-shaded, and then tried 1.17 for both, but this doesn't help. I've also tried using earlier versions of both the Beam runners and bigtable-client-core, and this doesn't help either.
The relevant Maven dependencies are:
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-runners-google-cloud-dataflow-java</artifactId>
<version>2.15.0</version>
</dependency>
<dependency>
<groupId>com.google.cloud.bigtable</groupId>
<artifactId>bigtable-client-core</artifactId>
<version>1.12.1</version>
</dependency>
I look at the code for Utils.java (https://github.com/grpc/grpc-java/blame/master/netty/src/main/java/io/grpc/netty/Utils.java), and I don't see any evidence that I'd be using any earlier version that might not have this constant (it's been there since version 1.7).
I'm completely baffled what the issue is here. How do I identify the dependency conflict? Is there another way I can find what version of the class Maven is actually looking at here?
When I look at my local Maven cache (~/.m2/repository/) I see 10-20 versions of certain artifacts, some of which are being used for only a single specific build (project). I would like to get rid of this duplication (true, they are different versions, but I'd still think a depending project would be able to tolerate a micro or minor version difference) and to somehow ask Maven to resort to the closest available version during dependency resolution, if a specific artifact version is missing in the local repository.
For example, if I have versions 1.0.0, 1.1.2, 1.4.0 and 2.0.0 of a foo:bar artifact in my local cache, I would like Maven to:
use 1.1.2 for a build requiring 1.1.0
use 1.4.0 for a build requiring 1.4.10
use 2.0.0 for a build requiring 2.5.0
without having to manually change the pom of the specific build(s).
I am fairly aware of the risks associated with switching dependency versions without proper analysis, and I'm only asking for a mechanism to be utilized for non-critical builds (such as a tool/library that I just cloned off a VCS, and would like to run and try out), preferably activated only when a particular flag is provided.
Is there something out there, like a Maven extension or plugin (that can be applied on a system-wide scale, and activated on demand with a flag), that can help me achieve my goal?
P.S.: Since the definition of "closest" could be ambiguous (given the fact that Maven may not know which of 1.4.0 and 2.0.0 is closer to 1.5.0 depending on the actual release versions lying between them), it would even be sufficient if I can specify the version on the build command (e.g. mvn package -Dfoo:bar=1.4.0), still without making any manual pom changes. (While this may already be possible for versions that have been specified as <properties> entries, I would like a generic solution where even hard-coded versions in transitive dependencies could be overridden.)
P.P.S.: Please note tht the project(s) that would be built would not have been authored/composed by me, so I don't really have control/authority over their actual pom files. What I'm looking for is a way to override dependency versions in their pom files without doing any manual modifications at source level.
In order to change a transitive dependency, you need to exclude the transitive dependency in your direct dependency, then add a direct dependency in your pom.
For example, if you have a dependency on foo.jar (which depends on xyz.jar version 1.3) and on bar.jar (which depends on xyz.jar version 1.4), you can have these two sections in your pom:
<!-- Define the version(s) that you allow your dependencies to depend on. -->
<dependencyManagement>
<dependency>
<groupId>projectXyz</groupId>
<artifactId>xyz</artifactId>
<version>[1.0,2.0)</version>
</dependency>
<dependency>
<groupId>projectFoo</groupId>
<artifactId>foo</artifactId>
<version>1</version>
<exclusions>
<exclusion>
<groupId>projectXyz</groupId>
<artifactId>xyz</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>projectBar</groupId>
<artifactId>bar</artifactId>
<version>5</version>
<exclusions>
<exclusion>
<groupId>projectXyz</groupId>
<artifactId>xyz</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencyManagement>
...
<!-- Declare your dependencies but don't allow them to suck in their transitive dependencies. -->
<dependencies>
<dependency>
<groupId>projectXyz</groupId>
<artifactId>xyz</artifactId>
</dependency>
<dependency>
<groupId>projectFoo</groupId>
<artifactId>foo</artifactId>
</dependency>
<dependency>
<groupId>projectBar</groupId>
<artifactId>bar</artifactId>
</dependency>
</dependencies>
...
This will pick up the most recent version of xyz.jar that it can, and that will be the only version used. When foo and bar use xyz, they'll have the version that you've allowed into your project.
The best thing to do is to use a parent pom (or bom: bill of materials) with a well-defined and well-maintained dependencyManagement section. Stick with a single version of everything, just share that "everything" between all projects. You can override versions in projects if you need to.
If you'd rather define versions in each project, then version ranges will work. For the three examples you gave, you would use things like:
[1.1.0, 1.2)
[1.4.0, 1.5)
[2.0.0,)
(Open to correction here.. I haven't used version ranges in almost 10 years.)
Finally, to get it to use versions that are already available, rather than download the best ones, all you can do is to use a local artifact repository as your central maven repository, and turn off access to maven central and bintray.
Closest thing I found so far: https://github.com/jboss/maven-dependency-management-extension/blob/master/README.md
It can be dropped into ${MAVEN_HOME}/lib/ext and utilized for overriding versions of specific dependencies, e.g. mvn install -Dversion:junit:junit=4.10. While it doesn't offer the suggested "intelligent version derivation" approach, it's a good-enough solution.
We have recently upgraded from Drools 5 to Drools 6 and have run into disturbing conflict issues.
We have kie-ci imported into out project. kie-ci brings in sisu-guava. sisu-guava changes the accessibility of some of the classes from google's guava. Unfortunately, it uses the same package name as google's guava.
Since we're working with google's guava in our project, we are running into conflicts of classes.
An attempt to remove sisu-guava from the project (using a maven exclusion) results in accessibility exceptions, as the kie-ci code attempt to access classes which are public in sisu-guava but are private in google's guava.
Any idea how to get round this.
This may not be correct solution for all situation, but I was able to resolve this issue by excluding the susi-guava jar in my pom:
<dependency>
<groupId>org.jbpm</groupId>
<artifactId>jbpm-kie-services</artifactId>
<version>${jbpm.version}</version>
<exclusions>
<exclusion>
<groupId>org.sonatype.sisu</groupId>
<artifactId>sisu-guava</artifactId>
</exclusion>
</exclusions>
</dependency>
I seem to have the same problem using drools 6.2. Drools is dependent on guava 10.0.1, where as my project had a dependency on guava 16 and maven was picking the version 16 (correctly).
On inspecting the dependency tree, I find that the drools dependency on guava is dictated by "org.eclipse.sisu:org.eclipse.sisu.plexus:jar:0.0.0.M5:runtime".
There is a newer version of org.eclipse.sisu.plexus, so I added the following to my project's pom to pick up the latest version, which is:
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.eclipse.sisu</groupId>
<artifactId>org.eclipse.sisu.plexus</artifactId>
<version>0.3.1</version>
</dependency>
</dependencies>
</dependencyManagement>
Now, there does not seem to be a dependency on guava, for drools and the problem is solved and my project can use version 16 of guava.
I have created a maven web project using the below site.
http://www.mkyong.com/maven/how-to-create-a-web-application-project-with-maven/
I have performed all the steps given there and executed a simple hello world program. Now, I have to include spring dependencies into my eclipse web project.
So
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring</artifactId>
<version>3.1.2</version>
</dependency>
In the dependencies tag, I added the above configuration. Now, it is saying as below:
unable to find jars from the repositories (local and as well as remote)
It gave suggestion to execute the command:
mvn install -artifactid=springframework (something like this)
But when I mentioned version as 2.5.6 it's correctly taken. Is it the problem with the version 3.1.2 being unavailable at maven repository? How do I get the latest versions if maven is not working properly for latest versions?
It also gave me the suggestion to go for manual download and put in local repository.
The Maven coordinates changed over time.
Try:
<dependency>
<groupId>org.springframework</groupId>
<artifactId>org.springframework.core</artifactId>
<version>3.1.2.RELEASE</version>
</dependency>
OR Try:
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-core</artifactId>
<version>3.1.2.RELEASE</version>
</dependency>
I'll just find if there is an all-in-one POM or dependency. But "spring-full" looks 1.2.x only and "spring" 2.5.x. CHECKED: Can't find one I've been using separate modules in all projects for sometime (this is better anyway, fine grained dependencies).
The location you can search is at http://ebr.springsource.com/repository/app/
for 3.1.2 see http://ebr.springsource.com/repository/app/library/version/detail?name=org.springframework.spring&version=3.1.2.RELEASE&searchType=librariesByName&searchQuery=spring
Spring have changed their repository URL and online locations at least 3 times to my knowledge over the past 4 years. So I'd look for current information on their website about setting up a Maven <repositories> config to obtain their JARs. Beware of articles with out of date information :(
Also notice the artifactId is different in the 2 example this is another gotcha issue with spring. The "org.springframework.core" are their EBR and OSGi compliant versions of their software. The "spring-core" is the older pre-OSGi co-ordinates. Find what works for you and don't mix them in the same project. For example I am using "spring-core" because I use 3.2.0.M2 which are Milestone releases. But the production release EBR co-ordinates are the best to use.
Sorry for so many edits... but it has been a minefield even if you understand the heritage of getting Spring Source software.