I am working on migrating multi module java project into maven. Now for most of them i migrated to maven.
Finally i am aware my project have lot of unnecessary jars included, and i want to clean them up.
I know maven has plugin command, mvn dependency:analyze. Which works very well.
dependency:analyze analyzes the dependencies of this project and determines which are: used and declared; used and undeclared; unused and declared. based on static code analysis.
Now my question is that, how can i remove reported unused and declared dependency for cleanup purpose. It could be possible those jars were getting used at runtime and my code will compile perfectly fine after removing but blow up at runtime.
An example: mycode compile with one of opensource library antisamy.jar but it require batik.jar at runtime. And mvn dependency:analyze reports me to remove batik.jar.
IS my understanding correct or i need expert inputs here.
Your understanding seems to be correct.
But I'm not sure why you'd think that there is a tool that could cover all the bases.
Yes, if you use stuff by reflection, no tool can reliably detect the fact that you depend on this class or the other.
For example consider this snippet:
String myClassName = "com." + "example." + "SomeClass";
Class.forName(myClassName);
I don't think you can build a tool that can crawl through the code and extract all such references.
I'd use a try-and-fail approach instead, which would consist of:
remove all dependencies that dependency:analyze says are superfluous
whenever you find one that was actually used, you just add it back
This could work well because I expect that the number of dependencies that are actually used by reflection to be extremely small.
Related
I'm trying to refactor a big project, and there is also a big pom. I want to refactor it and remove unused dependencies. I used
mvn dependency:analyze
to look at some unused dependencies. The fact is that some libraries used at runtime are considered unused, like this answer said: https://stackoverflow.com/a/42967645/18089908.
In addition, in my pom all the dependencies are missing the scope tag.
Is there a way to see which dependencies are required at runtime?
There is no general way to do this.
Read the docs and run your integration tests.
I know you can access different modules (included using include) in a project via org.gradle.api.Project#getSubprojects(), and I know you can get the name and directories of separate builds that have been included (using includeBuild) via org.gradle.api.invocation.Gradle#getIncludedBuilds().
But how can my plugin get information such as the locations of Java source files and class files for projects included using includeBuild?
My goal here is to determine which files have changed in the current git branch (which I can do), and then collect their corresponding class files into a jar file that's used for our patching mechanism that inserts the patch jars at the front of the classpath rather than redeploying the whole application.
I don’t think it is a goal of Gradle to provide including builds with detailed information on included builds. Currently, the Gradle docs basically only state two goals for such composite builds:
combine builds that are usually developed independently, […]
decompose a large multi-project build into smaller, more isolated chunks […]
Actually, isolation between the involved builds seems to be an important theme in general:
Included builds do not share any configuration with the composite build, or the other included builds. Each included build is configured and executed in isolation.
For that reason, it also doesn’t seem to be possible or even desired to let an including build consume any build configurations (like task outputs) of an included build. That would only couple the builds and hence thwart the isolation goal.
Included builds interact with other builds only via dependency substitution:
If any build in the composite has a dependency that can be satisfied by the included build, then that dependency will be replaced by a project dependency on the included build.
So, if you’d like to consume specific parts of an included build from the including build, then you have to do multiple things:
Have a configuration in the included build which produces these “specific parts” as an artifact.
Have a configuration in the including build which consumes the artifact as a dependency.
Make sure that both configurations are compatible wrt. their capabilities so that dependency substitution works.
Let some task in the including build use the dependency artifact in whatever way you need.
Those things happen kind of automatically when you have a simple dependency between two Gradle projects, like a Java application depending on a Java library. But you can define your own kinds of dependencies, too.
The question is: would that really be worth the effort? Can’t you maybe solve your goal more easily or at least without relying on programmatically retrieved information on included builds? For example: if you know that your included build produces class files under build/classes/java/main, then maybe just take the classes of interest from there via org.gradle.api.initialization.IncludedBuild#getProjectDir().
I know, this may not be the answer you had hoped to get. I still hope it’s useful.
What maven Excluded dependencies was invented for?
Should I care to exclude any dependencies other then for fixing libraries conflicts?
What would good maven project architecture look like:
tend to exclude as much as possible
or as minimum as needed?
I don't know the reasons for the design but I've seen it used in the following cases:
I had a library which had junit as a compile time dependency so JUnit code leaked into my production code.
I had a library which uses log4j. Since I'm using slf4j, I used dependency exclusion to get rid of the hardwired logging framework and used a slf4j-log4j bridge instead so I could ultimately log to logback.
In another case, I was using only some features of a framework and didn't need all the dependencies. Since they weren't optional in the first place, I used exclusions to keep my classpath lean and clean.
General rules:
Use it to get rid of things that break your build
Get rid of things that you're replacing with something else
Get rid of things that you know you don't need (optional)
If none of the rules apply, leave the dependency alone; chances are that the immediate dependency might change over time and suddenly, it will need some dependency that seemed superfluous before and you code will unexpectedly break.
In addition to Aaron's answer:
An exclusion is usually needed when the provider of the dependency did something wrong (i.e. did not make a dependency optional where he should have, included an actual logging backend - as opposed to api - or used the wrong scope).
The one exception is logging frameworks. See Aaron's answer for that.
So no, do only exclude dependencies if you have a specific reason to exclude them.
I'm setting up a (java) maven project that depends on a library (Jettison, among others) that is in the Maven repo. Jettison, in turn, depends on stax. I need to run a tool (Jar Jar Links) on stax (to change the namespace). How do I alter the rules for a transitive dependency in a maven project? My transitive dependencies are being included in my target folder using the copy-dependencies goal (I assume this is how things are usually done). I assume that this is the point where the plugin would be run on the transitively-generated artifact.
Extra question: I don't need this at this point but how would I go about altering the source in the transitive dependency? I can get the jar of the source with mvn dependency:sources but, from there, I'm not sure what the right approach is.
Victory!
Seems at least two people are even more clueless about Maven than me so let me explain what I'm doing before I report the fix at the bottom of this post (spoiler alert: it looks to be a bug in JarJar).
Android uses Java but its missing a lot of the java core (specifically, javax classes). The Android DEX compiler (which converts .jars to Android .dex files) won't even allow you to compile things in the java.* or javax.* namespace because it'll (usually) break stuff. However, in some (many) cases, there are routines that you might want to include -- specifically because they are used by existing libraries. The most legendary is StAX, which is why Google posted an example of how to include it here in the Dalvik repo's wiki. The example uses JarJar... with ant. Transitive dependencies are not really an issue when you aren't using a repo so they are not addressed in the wiki.
I was able to get JarJar to run on my source with Maven but without changing the namespaces in the dependencies (and transitive dependencies), that's worthless. Hence my question.
I thought that the copy-dependencies plugin might be useful for... copying the dependencies and running a transforming plugin in the process. Copying dependencies is mentioned as a step in the official "Maven in 5 minutes" doc so it seemed like a good start but maybe the the people who wrote the official docs don't know how to use it :-) . Either way, it it didn't help -- there is no simple way I could see to transform the jars as it copies.
Using the verbose spew from Maven, I was able to see that Jar Jar was in fact processing my jars properly... and then throwing out the result. It would have packaged the converted classes from the transitive dependencies in my artifact with the rest of my code but, instead, it "Excluded" them. Jar Jar parameters are basically undocumented and most of the tags aren't even listed in the docs but all of the examples I could find use a section with wild-cards that tell it what classes to hold onto. At least I thought (think?) that's what the section is for. Instead, it seems to randomly throw out stuff. Basically, the section is busted. For example, I had:
<keep>
<pattern>com.example.**</pattern>
</keep>
...thinking that this would keep classes that began with com.example. Wrong. It keeps whatever the hell it wants. I tried a million things in that spot until one worked:
<keep>
<pattern>*.**</pattern>
</keep>
This only keeps the classes I wanted -- the classes it updated and the originals of the ones that it didnt touch. Note that ** doesn't even work. This is version 1.8 of the JarJar plugin (the version most poms Ive found use).
Back to work.
I'm working in a big project with lots of modules (they are portlets), and dependency management is becoming harder and harder. We have problems like that some jars are used by all portlets and still not provided for the server, so we are including them in every WEB-INF/lib, or dependencies are declared but not used at all, and stuff like that.
I'm trying to clean up that mess, but I'm afraid of making a mistake and not detecting it until it is too late.
Lets say I mark a dependency as "provided", and it is needed in a strange business case I forgot to test, and the server does not provide such a dependency.
Organization is bureaucratic, and I can not access the server to check the actual presence of a given jar.
Is there any way I can make the a check for dependencies once the war is installed, and see if everything is there and accessible?
War files contain a copy of pom.xml in META-INF, so at least part of the information is there.
I would love to see something like
INFO: Checking provided dependency org.drools-drools-core-4.0.7........done
ERROR: Checking provided dependency org.drools-drools-compiler-4.0.7........FAIL
in the logs...
There is no such tool since all things you're talking about happen in run-time. You'll have to trace all ClassNotFoundException logs and find artifacts that cointain given classes, e.g. using GrepCode. That's the best I figured out some time ago having similar case. Far more easy is to check if you have some declared and unused dependencies or undeclared and used (as transitives) by doing mvn dependency:analyze. It usually works pretty well, but be aware it uses Maven 2 dependency resolution, so it can lie sometimes when using Maven 3. From my experience, it doesn't happen very often, but sometimes it does and causes some problems though.