Background
commons-beanutils-core version 1.8.0 has some security issues that I am trying to avoid.
So, I am using commons-beanutils 1.9.4
Problem
I cannot seem to prevent other libraries from importing commons-beanutils-core version 1.8.0. Nowhere in my pom file do I include it. I am assuming some other dependency implicitly includes it. And, since commons-beanutils-core no longer is supported and they have moved to just using commons-beanutils for all newer versions, when I explicitly write:
<dependency>
<groupId>commons-beanutils</groupId>
<artifactId>commons-beanutils</artifactId>
<version>1.9.4</version>
</dependency>
It does not remove the implicit references to commons-beanutils-core.
Question
How do I make the other packages not download commons-beanutils-core version 1.8.0?
Notes:
There have been other packages that were downloaded as dependencies that I did not explicitly include in my pom.xml. A scan show that some of these packages were security risks. So, my solution was to just explicitly include a higher version of the package that was previously included implicitly. And that removed the old version of the package. But that solution does not work here since commons-beanutils-core is no longer used in the newest version.
Update 1
I have learned that including the following code will make the pom.xml think that 1.8.0 will be provided, and in a sense it will since I will include 1.9.4. But I am unsure if the code will use the 1.9.4 if it thinks it should look for 1.8.0. This code enables me to remove 1.8.0 but I don't know if my deception here will cause problems down the line.
<dependency>
<groupId>commons-beanutils</groupId>
<artifactId>commons-beanutils-core</artifactId>
<version>1.8.0</version>
<scope>provided</scope>
</dependency>
Using provided scope can have side effects on runtime. Provided scope dependencies are used in compilation but not packaged so if the commons-beanutils-core has a method which is not present in commons-beanutils you will get NoSuchMethodException.
Better solution will be use mvn dependency:tree to find which dependencies had dependency on commons-beanutils-core and
Either update the dependency to a version which uses 1.9.4 version of commons-beanutils.
Or use exclusion to exclude commons-beanutils-core from the dependencies using commons-beanutils-core as a dependency.
If you can use method 1 it's better if you cannot find a dependency which uses the latest version, then use method 2.
Related
I am trying to run Java code from a Maven project that uses both beam-runners-google-cloud-dataflow-java and bigtable-client-core, and I cannot get it to properly reconcile dependencies amongst these two. When I run and attempt to create a BigtableDataClient, I get the following error:
java.lang.NoSuchFieldError: TE_HEADER
at io.grpc.netty.shaded.io.grpc.netty.Utils.<clinit> (Utils.java:74)
at io.grpc.netty.shaded.io.grpc.netty.NettyChannelBuilder.<clinit> (NettyChannelBuilder.java:72)
at io.grpc.netty.shaded.io.grpc.netty.NettyChannelProvider.builderForAddress (NettyChannelProvider.java:37)
at io.grpc.netty.shaded.io.grpc.netty.NettyChannelProvider.builderForAddress (NettyChannelProvider.java:23)
at io.grpc.ManagedChannelBuilder.forAddress (ManagedChannelBuilder.java:39)
at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createSingleChannel (InstantiatingGrpcChannelProvider.java:242)
at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.createChannel (InstantiatingGrpcChannelProvider.java:198)
at com.google.api.gax.grpc.InstantiatingGrpcChannelProvider.getTransportChannel (InstantiatingGrpcChannelProvider.java:185)
at com.google.api.gax.rpc.ClientContext.create (ClientContext.java:160)
at com.google.cloud.bigtable.data.v2.stub.EnhancedBigtableStub.create (EnhancedBigtableStub.java:151)
at com.google.cloud.bigtable.data.v2.BigtableDataClient.create (BigtableDataClient.java:138)
at com.google.cloud.bigtable.data.v2.BigtableDataClient.create (BigtableDataClient.java:130)
...
I can only conclude this is due to an issue with version conflict on the relevant libraries (either grpc-netty or grpc-netty-shaded); I'm using 1.17 for grpc-netty and 1.23 for grpc-netty-shaded. I've tried using dependencyManagement to force the use of version 1.23.0 for both grpc-netty and grpc-netty-shaded, and then tried 1.17 for both, but this doesn't help. I've also tried using earlier versions of both the Beam runners and bigtable-client-core, and this doesn't help either.
The relevant Maven dependencies are:
<dependency>
<groupId>org.apache.beam</groupId>
<artifactId>beam-runners-google-cloud-dataflow-java</artifactId>
<version>2.15.0</version>
</dependency>
<dependency>
<groupId>com.google.cloud.bigtable</groupId>
<artifactId>bigtable-client-core</artifactId>
<version>1.12.1</version>
</dependency>
I look at the code for Utils.java (https://github.com/grpc/grpc-java/blame/master/netty/src/main/java/io/grpc/netty/Utils.java), and I don't see any evidence that I'd be using any earlier version that might not have this constant (it's been there since version 1.7).
I'm completely baffled what the issue is here. How do I identify the dependency conflict? Is there another way I can find what version of the class Maven is actually looking at here?
I have a persistent problem with maven dependencies version changes in IntelliJ. Whenever I try to use a previous version of a library and change the dependency version in my pom.xml nothing happens. Maven continues to use the newer version of the library.
For example I want to use:
<dependency>
<groupId>org.springframework.security.oauth.boot</groupId>
<artifactId>spring-security-oauth2-autoconfigure</artifactId>
<version>2.0.1.RELEASE</version>
</dependency>
But Maven repo has version 2.0.2 saved :
<dependency>
<groupId>org.springframework.security.oauth.boot</groupId>
<artifactId>spring-security-oauth2-autoconfigure</artifactId>
<version>2.0.2.RELEASE</version>
</dependency>
So for my projects version 2.0.2.RELEASE is used.
I tried reimporting the project first. Then I tried "reimpor all maven projects". Then I checked Settings > Maven > Always update snapshots. I also tried opening the project settings and deleting the dependency from there, but on reimport the 2.0.2 version will be imported in the project. For now the only thing that works is deleting manually the folder from the ".m2" folder.
Shouldn't library versions be strictly followed and shouldn't version 2.0.1 v be used for my project?
The moment you change the version of the artifacts, maven will use the same version. It will never use neither new version nor the older version. Since you are using intellij, you can check which are the jar files along with their version used. See below the screenshot.
You can expand the External libraries as shown below and you can check the dependencies used in pom.xml.
Besides, you can also check in command prompt. Go to command prompt and point to the project directory and type the following command.
mvn install dependency:copy-dependencies
You can see all the required dependencies along with version information in target folder.
I suggest you not to delete the .m2 directory as you may have to download all the dependencies once again.
If you want to enforce the use of a particular dependency version you can use:
<dependencyManagement>
<dependencies>
<dependency>
<groupId>org.springframework.security.oauth.boot</groupId>
<artifactId>spring-security-oauth2-autoconfigure</artifactId>
<version>2.0.1.RELEASE</version>
</dependency>
</dependencies>
</dependencyManagement>
What this will do is exclude the dependency unless it actually gets used, and then if it does gets used it only uses the version you have specified.
Not clear what is the issue.
Repo can contain everything, no matter if dependency is present locally.
Also, Idea does not resolve dependency itself, we use maven api to resolve them.
By default, maven takes dependency which is nearest to root (see https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html)
Specifiying explicit dependency in root pom should force using this version.
Could you please provide mvn dependency:tree output and corresponding IDEA maven dependency diagram (if you have IU)?
If Idea resolve another dependency version than maven, please fill an issue at https://youtrack.jetbrains.com/issues
I am trying to resolve dependency version conflicts while using the below dependencies.
The worst one I am facing is zucchini project supports the apache commons-io versions from 1.4 to latest one. It does not support versions
below 1.4
and at the same time pagerduty-client supports commons-io versions below 1.4 version.
So It is not possible to specify a common version of this dependency (dependency management)
which supports in both zucchini and pager-duty client (both are third party libraries).
In this particular situation I couldn't find a possible way to resolve this issue. Any help will be appreciated.
<dependency>
<groupId>com.comcast.zucchini</groupId>
<artifactId>zucchini</artifactId>
<version>[2.2.5,)</version>
</dependency>
<dependency>
<groupId>com.github.dikhan</groupId>
<artifactId>pagerduty-client</artifactId>
<version>3.0.2</version>
</dependency>
Possibility 1
If the old and new commons-io package/class names are a close enough match, excluding the old dependency from pagerduty-client could possibly work.
https://maven.apache.org/guides/introduction/introduction-to-optional-and-excludes-dependencies.html
<dependency>
<groupId>com.github.dikhan</groupId>
<artifactId>pagerduty-client</artifactId>
<version>3.0.2</version>
<exclusions>
<exclusion>
<groupId>org.apache.commons</groupId>
<artifactId>commons-io</artifactId>
</exclusion>
</exclusions>
</dependency>
This relies on the binary API of commons-io between versions 1.3.2 and 2.x being similar enough.
There does seem to be lots of overlap, looking at the code of each version:
https://github.com/apache/commons-io/tree/commons-io-1.3.2/src/java/org/apache/commons/io
https://github.com/apache/commons-io/tree/commons-io-2.5/src/main/java/org/apache/commons/io
Possibility 2
Split up your application so that the commons-io dependency is not shared and does not conflict.
It could be that the pagerduty-client and zucchini parts of your application do not need to be 'bundled' together, so split them up.
If they do need work together then you could still have two apps/processes and send messages between them.
Note
I cloned the pagerduty-client repo and changed the commons-io dependency from org.apache.commons:commons-io:1.3.2 to commons-io:commons-io:2.5 and the tests worked, so maybe you can suggest to the project owner that they upgrade commons-io.
And looking at the code it seems commons-io is hardly used (one place, HttpApiServiceImpl.java):
\pagerduty-client>findstr /s /c:"commons" *.java
src\main\java\com\github\dikhan\pagerduty\client\events\domain\AcknowledgeIncident.java:import org.apache.commons.lang3.StringUtils;
src\main\java\com\github\dikhan\pagerduty\client\events\domain\Incident.java:import org.apache.commons.lang3.StringUtils;
src\main\java\com\github\dikhan\pagerduty\client\events\domain\Incident.java:import org.apache.commons.lang3.builder.Builder;
src\main\java\com\github\dikhan\pagerduty\client\events\domain\Payload.java:import org.apache.commons.lang3.StringUtils;
src\main\java\com\github\dikhan\pagerduty\client\events\domain\ResolveIncident.java:import org.apache.commons.lang3.StringUtils;
src\main\java\com\github\dikhan\pagerduty\client\events\HttpApiServiceImpl.java:import org.apache.commons.io.IOUtils;
src\main\java\com\github\dikhan\pagerduty\client\events\PagerDutyEventsClient.java:import org.apache.commons.lang3.StringUtils;
src\main\java\com\github\dikhan\pagerduty\client\events\utils\FakePagerDutyEventsClient.java:import org.apache.commons.lang3.StringUtils;
As your commons-io is the problem you'll have to look further up the line. That means either upgrade pagerduty-client to a version that uses a newer version of commons-io that Cucumber likes, or downgrade zucchini to require a version of Cucumber that works with pagerduty-client as well.
This is a common problem with some jakarta commons packages, they at some point decided to massively change the public interface without changing the package name, causing conflicts like this for users.
You may be in luck, I once worked on a project where we had to rewrite thousands of lines of code just so we could link to a library we desperately needed that depended on a newer version of commons-io than the one we'd been using.
When I look at my local Maven cache (~/.m2/repository/) I see 10-20 versions of certain artifacts, some of which are being used for only a single specific build (project). I would like to get rid of this duplication (true, they are different versions, but I'd still think a depending project would be able to tolerate a micro or minor version difference) and to somehow ask Maven to resort to the closest available version during dependency resolution, if a specific artifact version is missing in the local repository.
For example, if I have versions 1.0.0, 1.1.2, 1.4.0 and 2.0.0 of a foo:bar artifact in my local cache, I would like Maven to:
use 1.1.2 for a build requiring 1.1.0
use 1.4.0 for a build requiring 1.4.10
use 2.0.0 for a build requiring 2.5.0
without having to manually change the pom of the specific build(s).
I am fairly aware of the risks associated with switching dependency versions without proper analysis, and I'm only asking for a mechanism to be utilized for non-critical builds (such as a tool/library that I just cloned off a VCS, and would like to run and try out), preferably activated only when a particular flag is provided.
Is there something out there, like a Maven extension or plugin (that can be applied on a system-wide scale, and activated on demand with a flag), that can help me achieve my goal?
P.S.: Since the definition of "closest" could be ambiguous (given the fact that Maven may not know which of 1.4.0 and 2.0.0 is closer to 1.5.0 depending on the actual release versions lying between them), it would even be sufficient if I can specify the version on the build command (e.g. mvn package -Dfoo:bar=1.4.0), still without making any manual pom changes. (While this may already be possible for versions that have been specified as <properties> entries, I would like a generic solution where even hard-coded versions in transitive dependencies could be overridden.)
P.P.S.: Please note tht the project(s) that would be built would not have been authored/composed by me, so I don't really have control/authority over their actual pom files. What I'm looking for is a way to override dependency versions in their pom files without doing any manual modifications at source level.
In order to change a transitive dependency, you need to exclude the transitive dependency in your direct dependency, then add a direct dependency in your pom.
For example, if you have a dependency on foo.jar (which depends on xyz.jar version 1.3) and on bar.jar (which depends on xyz.jar version 1.4), you can have these two sections in your pom:
<!-- Define the version(s) that you allow your dependencies to depend on. -->
<dependencyManagement>
<dependency>
<groupId>projectXyz</groupId>
<artifactId>xyz</artifactId>
<version>[1.0,2.0)</version>
</dependency>
<dependency>
<groupId>projectFoo</groupId>
<artifactId>foo</artifactId>
<version>1</version>
<exclusions>
<exclusion>
<groupId>projectXyz</groupId>
<artifactId>xyz</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>projectBar</groupId>
<artifactId>bar</artifactId>
<version>5</version>
<exclusions>
<exclusion>
<groupId>projectXyz</groupId>
<artifactId>xyz</artifactId>
</exclusion>
</exclusions>
</dependency>
</dependencyManagement>
...
<!-- Declare your dependencies but don't allow them to suck in their transitive dependencies. -->
<dependencies>
<dependency>
<groupId>projectXyz</groupId>
<artifactId>xyz</artifactId>
</dependency>
<dependency>
<groupId>projectFoo</groupId>
<artifactId>foo</artifactId>
</dependency>
<dependency>
<groupId>projectBar</groupId>
<artifactId>bar</artifactId>
</dependency>
</dependencies>
...
This will pick up the most recent version of xyz.jar that it can, and that will be the only version used. When foo and bar use xyz, they'll have the version that you've allowed into your project.
The best thing to do is to use a parent pom (or bom: bill of materials) with a well-defined and well-maintained dependencyManagement section. Stick with a single version of everything, just share that "everything" between all projects. You can override versions in projects if you need to.
If you'd rather define versions in each project, then version ranges will work. For the three examples you gave, you would use things like:
[1.1.0, 1.2)
[1.4.0, 1.5)
[2.0.0,)
(Open to correction here.. I haven't used version ranges in almost 10 years.)
Finally, to get it to use versions that are already available, rather than download the best ones, all you can do is to use a local artifact repository as your central maven repository, and turn off access to maven central and bintray.
Closest thing I found so far: https://github.com/jboss/maven-dependency-management-extension/blob/master/README.md
It can be dropped into ${MAVEN_HOME}/lib/ext and utilized for overriding versions of specific dependencies, e.g. mvn install -Dversion:junit:junit=4.10. While it doesn't offer the suggested "intelligent version derivation" approach, it's a good-enough solution.
In my case, the pom entry is like this:
<dependency>
<groupId>org.activiti</groupId>
<artifactId>activiti-engine</artifactId>
</dependency>
in my repository, there are 4 different versions of this jar varying from 5.8, 5.9, 5.14 and 5.15. But my system downloads 5.9 rather than 5.15.
can anyone explain this how and why? Is the dependency set in any other parent poms?
If you want to use the jar in version "5.15" you can specify the version in your maven dependency like that :
<dependency>
<groupId>org.activiti</groupId>
<artifactId>activiti-engine</artifactId>
<version>5.15</version>
</dependency>
In fact you should always put a version tag :
When you depend on a plugin or a dependency, you can use the a version value of LATEST or RELEASE. LATEST refers to the latest released or snapshot version of a particular artifact, the most recently deployed artifact in a particular repository. RELEASE refers to the last non-snapshot release in the repository. In general, it is not a best practice to design software which depends on a non-specific version of an artifact. If you are developing software, you might want to use RELEASE or LATEST as a convenience so that you don't have to update version numbers when a new release of a third-party library is released. When you release software, you should always make sure that your project depends on specific versions to reduce the chances of your build or your project being affected by a software release not under your control. Use LATEST and RELEASE with caution, if at all.