I am cleaning up dependencies from the build.gradle file for a big-sized Java project. How do I identify the usage of a certain dependency in the code-base ?
The solution that comes to my mind right away would be to remove the dependency and run compile to see if it was needed. You can of course skip checking the ones you are sure that are needed. If there's a lot of suspects you can comment out several at a time and if build fails and you don't know which one is a valid dependency, you can use binary search ;) to speed up the process.
Related
I am working on migrating multi module java project into maven. Now for most of them i migrated to maven.
Finally i am aware my project have lot of unnecessary jars included, and i want to clean them up.
I know maven has plugin command, mvn dependency:analyze. Which works very well.
dependency:analyze analyzes the dependencies of this project and determines which are: used and declared; used and undeclared; unused and declared. based on static code analysis.
Now my question is that, how can i remove reported unused and declared dependency for cleanup purpose. It could be possible those jars were getting used at runtime and my code will compile perfectly fine after removing but blow up at runtime.
An example: mycode compile with one of opensource library antisamy.jar but it require batik.jar at runtime. And mvn dependency:analyze reports me to remove batik.jar.
IS my understanding correct or i need expert inputs here.
Your understanding seems to be correct.
But I'm not sure why you'd think that there is a tool that could cover all the bases.
Yes, if you use stuff by reflection, no tool can reliably detect the fact that you depend on this class or the other.
For example consider this snippet:
String myClassName = "com." + "example." + "SomeClass";
Class.forName(myClassName);
I don't think you can build a tool that can crawl through the code and extract all such references.
I'd use a try-and-fail approach instead, which would consist of:
remove all dependencies that dependency:analyze says are superfluous
whenever you find one that was actually used, you just add it back
This could work well because I expect that the number of dependencies that are actually used by reflection to be extremely small.
The project I'm working on has dependencies on a few well known and big libraries. Things are working well, the transitive dependencies are playing nice with each other, for now. But unfortunately the total bundle size is around 100 megs.
I'm not sure if this is too large or not, but is there a way in maven to effectively remove dependencies, without making pom.xml very verbose and long?
And pointers to help me in the right direction would be awesome!
You can remove dependencies by declaring exclusions. The question I would post to first though is .. why do you want to remove them?
Unless you know that the dependencies are not needed, it might not make sense to try to remove them. Especially not if there are no issues e.g. in terms of application performance or startup times.
The Maven Dependency Plugin as well as the Eclipse integration have tools that allow you to understand the dependencies better with tools like the dependency:tree goal or the Dependnecy Hierarchy view of the POM.
Don't try to fix something that is not broken.
BUT ..
if you really know what is needed at runtime and use the tooling from Maven and M2e you can potentially remove a lot of bulk of your final artifact. However you will have to configure it in the pom using dependency exclusions. Newer Maven versions even allow patterns being used.
Another thing you can do is use a tool like proguard that removes all unused classes from the final artifact. This can be considerably complex but also VERY effective.
It will really be up to you to find the right balance between effort and benefit of outcome.
What maven Excluded dependencies was invented for?
Should I care to exclude any dependencies other then for fixing libraries conflicts?
What would good maven project architecture look like:
tend to exclude as much as possible
or as minimum as needed?
I don't know the reasons for the design but I've seen it used in the following cases:
I had a library which had junit as a compile time dependency so JUnit code leaked into my production code.
I had a library which uses log4j. Since I'm using slf4j, I used dependency exclusion to get rid of the hardwired logging framework and used a slf4j-log4j bridge instead so I could ultimately log to logback.
In another case, I was using only some features of a framework and didn't need all the dependencies. Since they weren't optional in the first place, I used exclusions to keep my classpath lean and clean.
General rules:
Use it to get rid of things that break your build
Get rid of things that you're replacing with something else
Get rid of things that you know you don't need (optional)
If none of the rules apply, leave the dependency alone; chances are that the immediate dependency might change over time and suddenly, it will need some dependency that seemed superfluous before and you code will unexpectedly break.
In addition to Aaron's answer:
An exclusion is usually needed when the provider of the dependency did something wrong (i.e. did not make a dependency optional where he should have, included an actual logging backend - as opposed to api - or used the wrong scope).
The one exception is logging frameworks. See Aaron's answer for that.
So no, do only exclude dependencies if you have a specific reason to exclude them.
I'm setting up a (java) maven project that depends on a library (Jettison, among others) that is in the Maven repo. Jettison, in turn, depends on stax. I need to run a tool (Jar Jar Links) on stax (to change the namespace). How do I alter the rules for a transitive dependency in a maven project? My transitive dependencies are being included in my target folder using the copy-dependencies goal (I assume this is how things are usually done). I assume that this is the point where the plugin would be run on the transitively-generated artifact.
Extra question: I don't need this at this point but how would I go about altering the source in the transitive dependency? I can get the jar of the source with mvn dependency:sources but, from there, I'm not sure what the right approach is.
Victory!
Seems at least two people are even more clueless about Maven than me so let me explain what I'm doing before I report the fix at the bottom of this post (spoiler alert: it looks to be a bug in JarJar).
Android uses Java but its missing a lot of the java core (specifically, javax classes). The Android DEX compiler (which converts .jars to Android .dex files) won't even allow you to compile things in the java.* or javax.* namespace because it'll (usually) break stuff. However, in some (many) cases, there are routines that you might want to include -- specifically because they are used by existing libraries. The most legendary is StAX, which is why Google posted an example of how to include it here in the Dalvik repo's wiki. The example uses JarJar... with ant. Transitive dependencies are not really an issue when you aren't using a repo so they are not addressed in the wiki.
I was able to get JarJar to run on my source with Maven but without changing the namespaces in the dependencies (and transitive dependencies), that's worthless. Hence my question.
I thought that the copy-dependencies plugin might be useful for... copying the dependencies and running a transforming plugin in the process. Copying dependencies is mentioned as a step in the official "Maven in 5 minutes" doc so it seemed like a good start but maybe the the people who wrote the official docs don't know how to use it :-) . Either way, it it didn't help -- there is no simple way I could see to transform the jars as it copies.
Using the verbose spew from Maven, I was able to see that Jar Jar was in fact processing my jars properly... and then throwing out the result. It would have packaged the converted classes from the transitive dependencies in my artifact with the rest of my code but, instead, it "Excluded" them. Jar Jar parameters are basically undocumented and most of the tags aren't even listed in the docs but all of the examples I could find use a section with wild-cards that tell it what classes to hold onto. At least I thought (think?) that's what the section is for. Instead, it seems to randomly throw out stuff. Basically, the section is busted. For example, I had:
<keep>
<pattern>com.example.**</pattern>
</keep>
...thinking that this would keep classes that began with com.example. Wrong. It keeps whatever the hell it wants. I tried a million things in that spot until one worked:
<keep>
<pattern>*.**</pattern>
</keep>
This only keeps the classes I wanted -- the classes it updated and the originals of the ones that it didnt touch. Note that ** doesn't even work. This is version 1.8 of the JarJar plugin (the version most poms Ive found use).
Back to work.
I'm working in a big project with lots of modules (they are portlets), and dependency management is becoming harder and harder. We have problems like that some jars are used by all portlets and still not provided for the server, so we are including them in every WEB-INF/lib, or dependencies are declared but not used at all, and stuff like that.
I'm trying to clean up that mess, but I'm afraid of making a mistake and not detecting it until it is too late.
Lets say I mark a dependency as "provided", and it is needed in a strange business case I forgot to test, and the server does not provide such a dependency.
Organization is bureaucratic, and I can not access the server to check the actual presence of a given jar.
Is there any way I can make the a check for dependencies once the war is installed, and see if everything is there and accessible?
War files contain a copy of pom.xml in META-INF, so at least part of the information is there.
I would love to see something like
INFO: Checking provided dependency org.drools-drools-core-4.0.7........done
ERROR: Checking provided dependency org.drools-drools-compiler-4.0.7........FAIL
in the logs...
There is no such tool since all things you're talking about happen in run-time. You'll have to trace all ClassNotFoundException logs and find artifacts that cointain given classes, e.g. using GrepCode. That's the best I figured out some time ago having similar case. Far more easy is to check if you have some declared and unused dependencies or undeclared and used (as transitives) by doing mvn dependency:analyze. It usually works pretty well, but be aware it uses Maven 2 dependency resolution, so it can lie sometimes when using Maven 3. From my experience, it doesn't happen very often, but sometimes it does and causes some problems though.