Today, I ran into a runtime NoSuchMethodError caused by a guava dependency conflict:
One of my third-party dependencies, let's call it A, invokes HostAndPort.getHost(), which is defined in some guava version between (18.0, 23.6-jre]. A depends on guava:23.6-jre, so it's compiled without any problem. But I have another dependency, let's call it B, depend on guava:18.0, and unfortunately according to the rule of "nearest wins", maven choose guava:18.0.
Everything works fine while compiling and packaging, but the devil comes out in runtime, causing my program to crash, which is bad.
So, I wonder is there any way I can detect this kind of problem using tools like maven, so that I can fix them before launching my program?
Related
I'm trying to parse WSDL files using the EasyWSDL library on my (atlassian) maven plugin project. I keep getting this error when I try to parse the file and it fails when the project tries to reach the TransformerFactoryImpl class from said "xalan" package. The package is located inside the JDK, so it's supposed to be available to runtime, yet for some reason it can't reach it.
I've tried importing the package through
<import-package>com.sun.org.apache.xalan.internal.*,</import-package>
section of the pom.xml, but that gives me the following error:
Unresolved requirements: [[my.plugin.package [304](R 304.0)] osgi.wiring.package; (osgi.wiring.package=com.sun.org.apache.xalan.internal.xsltc.trax)]
Which makes it seem like the other packages of xalan are found, except for the xsltc.trax one.
I've tried creating a bundle extension for OSGi that exports this one package and exposes it to the classpath but due to lack of experience and pretty poor documentation on that matter I can't get that working either. Adding Xalan as a dependency and using <scope>provided</scope> does nothing as well.
I need to find a way to put this specific package on the classpath when the program is running. When I run unit-tests on my implementations they work just fine, so it seems to purely be a runtime classpath problem.
Is there anyone with experience on this matter? I've little experience with OSGi, and my experience with maven isn't huge either. So I'm hoping it's something I did wrong myself instead of this being an impossible problem to solve.
I was studying Maven's build system and it adds a lot of transitive dependencies because of its transitive dependency system (maven itself does not add dependencies but transitive dependency system does). I see issues with it like major version conflicts and unknown dependencies coming in.
I was thinking why is the system designed this way and why not take direct dependencies. My library does not need to depend on something which my dependency is using but not my library (I mean I understand why it needs to be included in the build list, my dependencies need to build using those, but why does it needs to cause major version conflict?). Am I missing something fundamental here? One thing that I can think of is that my library's build dependency list can grow to be very big because of all the direct dependencies I will need to take, but that does seem to be as big of a problem as problems with transitive dependency system.
I am new to build systems, so please don't be too harsh. I also tried to google this question but didn't find useful answers but please feel free to comment anything that I might have missed.
Thanks
If you need library A to run, and library A needs library B to run, and this needs C to run, it is very tedious to figure this out and add all the relevant dependencies to your project.
Before Maven and Gradle, many people worked that way and found out, that it is much easier to let a build tool figure out the transitive dependencies.
My library does not need to depend on something which my dependency is using but not my library [...]
This is your major misconception. There are two possibilities:
The direct dependency of your library exposes types from the transitive dependency in its public API. To use this public API you need to access these types, so you need the transitive dependencies during compile time.
The direct dependency of your library only uses its own dependency internally, but not in its public API. In this case, your library does not need to depend on the transitive dependency during compile time. But as soon as your library code runs (even in a test), it may use some functionality of its direct dependency that internally uses functionality of the transitive dependency, causing your library code to fail.
[...] I mean I understand why it needs to be included in the build list, my dependencies need to build using those [...]
There is no actual build list (or order) for external dependencies, because they are used when they are already built (the downloaded .jar files contain compiled .class files). But as I mentioned above, you will need the transitive dependencies either during compile time or during runtime (e.g. tests), so your build system (Maven or Gradle) will fetch them for you.
[...] but why does it needs to cause major version conflict?
#khmarbaise already explained in his comment, why and how version conflict between transitive dependencies may occur:
You are using two libs X and Y. Both of them using another lib (A) So X is using A in version 1.0.0 but Y is using A in version 2.0.0. In the end you can't have both on the classpath there must be done a decision for one version. So depending on how X,Y are implemented either X can fail while using A in V1.0.0 or Y can fail in using A in V1.0.0 or with V2.0.0 the same... This can happen if X or Y are being updated. This is also true for different version combinations like A in 1.0.0 and 1.1.0 (if compatibility is not 100%)
I'm working in a big project with lots of modules (they are portlets), and dependency management is becoming harder and harder. We have problems like that some jars are used by all portlets and still not provided for the server, so we are including them in every WEB-INF/lib, or dependencies are declared but not used at all, and stuff like that.
I'm trying to clean up that mess, but I'm afraid of making a mistake and not detecting it until it is too late.
Lets say I mark a dependency as "provided", and it is needed in a strange business case I forgot to test, and the server does not provide such a dependency.
Organization is bureaucratic, and I can not access the server to check the actual presence of a given jar.
Is there any way I can make the a check for dependencies once the war is installed, and see if everything is there and accessible?
War files contain a copy of pom.xml in META-INF, so at least part of the information is there.
I would love to see something like
INFO: Checking provided dependency org.drools-drools-core-4.0.7........done
ERROR: Checking provided dependency org.drools-drools-compiler-4.0.7........FAIL
in the logs...
There is no such tool since all things you're talking about happen in run-time. You'll have to trace all ClassNotFoundException logs and find artifacts that cointain given classes, e.g. using GrepCode. That's the best I figured out some time ago having similar case. Far more easy is to check if you have some declared and unused dependencies or undeclared and used (as transitives) by doing mvn dependency:analyze. It usually works pretty well, but be aware it uses Maven 2 dependency resolution, so it can lie sometimes when using Maven 3. From my experience, it doesn't happen very often, but sometimes it does and causes some problems though.
I have a module X that is dependent on a Third party library which in turn depends on apache-commons-collections 2.1.
In module X, I want to use the latest apache-commons-collections 3.0 which has some additional methods than 2.1. If I add a dependency to 3.0, I'm guessing this will create a problem since the class loader just picks up the first class it sees in the classpath. Is there a good way to get around this problem?
Thanks,
S
IMHO there is no really good way without additional solution for modularity (like Java EE's EAR or OSGi). I guess however that you're asking about just simple web (or not) module that directly use this 3rd party lib. I'm afraid you have to resolve this conflict manually. If fact, Maven won't provide 2 versions of commons-collections and depend on classloader's resolution, but rather resolve dependencies graph and pick the version it guess it's better with your POMs' declarations in mind. That means, if you declare in module X dependency on commons-collections version 3.0, that version will be used since this declaration is more important than some 3rd party lib's dependencies.
That's a serious problem of Java Platform itself, cousing such problems like the famous JAR hell. Unfortunately, it is your problem to choose and declare commons-collections version that satisfy both you and your 3rd party lib.
Just add the dependency to 3.0 to your project and it will prefer it over the older version of 2.1. To be explicit you can add an exclusion. In any case use the dependency plugin and the analyze and tree goals to see what is happening.
Long story short... this happens all the time and will be fine and in any case you can control what happens.
Trying to build Spring-based application one needs to figure out all necessary dependencies the application will have.
For example, I was using HibernateTemplate, and each time I run the application the ClassNotFound exception comes out. So I “google” for jars that contains this particular class, after search mvnrepository to find appropriate artifact. Always confuse about which version to use.
And it’s happened again and again, and only after few hours and few dozens of dependencies added the application become runnable.
But even after that, I tried to use my app. on different computer with slightly different parameters, and slf4j class not found error appeared, even after testing extensively on the developer machine, still some dependencies missing.
Now it works fine, but I want to distribute my application, and not sure if on another system there will no dependency missing.
So, what is the best practice to determine all necessary dependencies not only at design time but in runtime too? Is there any tool for that?
How one can manage versions confusion, when there are dozens of dependencies each with its own version?
They both resolve dependencies, so you keep a dependency file and it does all the heavy lifting of making sure everything is included in your builds. Use the full spring dependency list. I had problems with SLF4J too.
http://mvnrepository.com/artifact/org.springframework/spring-full/1.2.8