I have a springboot project that we got going quickly by using the various appropriate spring-boot-starter jars to bring in the appropriate transitive dependencies that we needed. Now, as we get closer to production launch, we are adding code-quality and code-inspection tools to tighten up loose ends. Some of these are tagging the starter jars as problematic, but I am finding inconsistent advice as to how to handle the problem.
Spring.io documentation fails to weigh in on this. Starters are described here: Spring Starters, but it really just says they are used to "get going quickly" but does not indicate that they are intended for use in production. The starters are not listed in the 'production ready' features (which focuses on Actuator). The Packaging for Production makes no mention of starters. From Spring documentation alone, you are encougaged to use the starters (to get going quickly), but are not told to, or told not to, use them in production.
If the spring project adds either gradle-lint plugin, or uses maven-dependency-plugin, use of the starters flags problems. The starters include no code themselves, but they pull in useful transitive dependencies (by design), but that's contra-indicated by the lint plugins. Related incident here
From gradle-lint Generally, applications only use a subset of the libraries included in such families. The unnecessary dependencies included with the family both increase the footprint of the application itself. If the 'application' is actually itself a library, these unnecessary dependencies leak downstream to its users, increasing their footprint and potentially introducing breaking version conflict resolution problems.
Is the "right" answer to remove starter jars before going live? Should starters be used in production? I am sure there are plenty of projects that have gone to production with starters, and probably they have done so successfully. But it looks to me like the authorities differ on what to do, so looking to reconcile them.
I think this is likely to be closed as it's opinion based. Speaking as a member of the Spring Boot team, the starters are absolutely intended for use in production. There's no point in something that gets you going quickly if it then creates more work later on. Some people dislike relying on transitive dependencies indirectly and believe that all dependencies that you require should be declared directly. I think the linting tools you've mentioned are enforcing this opinion. You can either configure the tool to quieten it or manually declare the starters' dependencies. I would do the former.
Related
Gradle is working on new features for sharing dependency versions between projects that will provide central locations (settings.gradle, libs.versions.toml) for declaring common dependencies.
It's already possible (and easy) to share a dependencies block through a plugin, so what is the downside of the plugin approach to sharing dependencies, versus the new version catalogs and dependency bundles? What do these new features improve upon?
This is not a thorough answer. However, let me share what I think makes a difference. We need to keep in mind that Gradle is built around developer productivity and making builds as fast as possible.
Centralizing common dependency declaration makes sense to be supported out of the box. Currently, there is a good chance that when you look at different Gradle projects, each one of them may implement a different approach to this. Cédric Champeau iterates over some existing pattern in this blog. Having a standard solution makes it easier to get started as developer. Cédric further states
Long story short: the presence of a catalog makes discoverability and maintenance easier, but it doesn’t remove any of the flexibility that Gradle offers. We’re thinking about ways to enforce that all direct dependencies are declared via a catalog in the future.
Declaring dependencies in libs.versions.toml allows Gradle to skip build script compilation when dependency versions are changed. This is significantly faster than changing the same in a script plugin. As a side-effect of declaring dependencies in libs.versions.toml, we may see third-party tooling that update dependencies automatically in the future.
What maven Excluded dependencies was invented for?
Should I care to exclude any dependencies other then for fixing libraries conflicts?
What would good maven project architecture look like:
tend to exclude as much as possible
or as minimum as needed?
I don't know the reasons for the design but I've seen it used in the following cases:
I had a library which had junit as a compile time dependency so JUnit code leaked into my production code.
I had a library which uses log4j. Since I'm using slf4j, I used dependency exclusion to get rid of the hardwired logging framework and used a slf4j-log4j bridge instead so I could ultimately log to logback.
In another case, I was using only some features of a framework and didn't need all the dependencies. Since they weren't optional in the first place, I used exclusions to keep my classpath lean and clean.
General rules:
Use it to get rid of things that break your build
Get rid of things that you're replacing with something else
Get rid of things that you know you don't need (optional)
If none of the rules apply, leave the dependency alone; chances are that the immediate dependency might change over time and suddenly, it will need some dependency that seemed superfluous before and you code will unexpectedly break.
In addition to Aaron's answer:
An exclusion is usually needed when the provider of the dependency did something wrong (i.e. did not make a dependency optional where he should have, included an actual logging backend - as opposed to api - or used the wrong scope).
The one exception is logging frameworks. See Aaron's answer for that.
So no, do only exclude dependencies if you have a specific reason to exclude them.
What is the difference between spring source dm server specific Import-Bundle and OSGi's Require-Bundle?
I am confused whether to use Import-Bundle or Require-Bundle in my project.
Import-Bundle is similar to Require-Bundle, it creates a complete dependency on the other bundle, including that bundle's dependencies. This transitivity is bad because you have no idea what you depend, creating the infamous "big ball of mud" problem we're so familiar with in Object oriented programming.
In OO, we've found a solution to this entanglement by using interfaces, they separate implementation from specification. OSGi is built around a similar albeit of an higher order concept of service contracts. These contracts (interfaces, permissions, helper classes) are stored in a package. In contract based programming you depend on the contracts, not the implementations. Ergo, an OSGi bundle should depend on packages since they represent the contracts.
Import-Package <=> interface
Import-Bundle/Require-Bundle <=> implementation class
Import-Bundle is NOT OSGi, it is a proprietary Spring extension. It is a cleaner form for Require-Bundle; the uncleanliness was necessary to support some Eclipse use cases. The OSGi decided not to adopt this header since the Require-Bundle/Import-Bundle is fundamentally broken if you want to build systems from components.
Ideally you should try to rather use Import-Package instead. It makes you bundles less dependent on each other. It also allows to show that you only depend on a part of a bundle. This is also important for managing versions. In OSGi you can define the versions of exported packages independent of the bundle version. So you can make sure you only change versions of an API if it really changes. This can make your app much more manageable.
It's explained here at SpringSource
So summarizing: Import-Bundle will import all exported packages of a certain bundle, it will resolve that when deploying, while Require-Bundle really requires a bundle with that type, and that relationship stays that way during runtime.
Normally they would behave pretty much the same. For example it can be different when:
You have 'split packages': packages that exist in multiple bundles, you might 'lose' dependencies with Import-Package / Import-Bundle that you can only express with Require-Bundle (Note that you really should avoid split packages if you can)
I think the Bundle->Package resolution is when you deploy that bundle. If you redeploy the bundle with the exported bundles to a version with different export, I don't think the bundle will notice. To be honest I'm not exactly sure about this one.
All in all, I'd say stick with the OSGi standard: Import-Package or Require-Bundle if you really need it. You'll have a bit more headers but you'll so many more options in the long run.
I'm working in a big project with lots of modules (they are portlets), and dependency management is becoming harder and harder. We have problems like that some jars are used by all portlets and still not provided for the server, so we are including them in every WEB-INF/lib, or dependencies are declared but not used at all, and stuff like that.
I'm trying to clean up that mess, but I'm afraid of making a mistake and not detecting it until it is too late.
Lets say I mark a dependency as "provided", and it is needed in a strange business case I forgot to test, and the server does not provide such a dependency.
Organization is bureaucratic, and I can not access the server to check the actual presence of a given jar.
Is there any way I can make the a check for dependencies once the war is installed, and see if everything is there and accessible?
War files contain a copy of pom.xml in META-INF, so at least part of the information is there.
I would love to see something like
INFO: Checking provided dependency org.drools-drools-core-4.0.7........done
ERROR: Checking provided dependency org.drools-drools-compiler-4.0.7........FAIL
in the logs...
There is no such tool since all things you're talking about happen in run-time. You'll have to trace all ClassNotFoundException logs and find artifacts that cointain given classes, e.g. using GrepCode. That's the best I figured out some time ago having similar case. Far more easy is to check if you have some declared and unused dependencies or undeclared and used (as transitives) by doing mvn dependency:analyze. It usually works pretty well, but be aware it uses Maven 2 dependency resolution, so it can lie sometimes when using Maven 3. From my experience, it doesn't happen very often, but sometimes it does and causes some problems though.
There are two main approaches when developing an OSGi application with Maven: POM-first and MANIFEST first.
I'm looking for an answer that is in a form of a table that shows pros and cons of each method.
To be more specific, I would also like to know how it relates to:
Maturity of toolset
Vendor independence
Development ease (which includes finding people who can do the development on the tooling)
Compatibility
Avoiding ClassNotFound
Avoiding manual work
At present this is what I can come up with
POM-First Pros (using maven-bundle-plugin)
Leverages existing Maven skills, repositories and tooling.
Likely easier to find people who know how to manage pom.xml rather than MANIFEST.MF along with pom.xml
Most of the information in MANIFEST.MF can be obtained from the pom.xml itself.
Can work with other IDEs not just Eclipse based ones.
Less invasive, just add the single plugin and change the packaging type to "bundle"
POM-First Cons
ClassNotFoundException more likely to occur at runtime. However, this can be mitigated using pax-exam (although it is very complicated to set up).
Still need to understand how the MANIFEST is setup to make sure the instructions configuration element is set correctly.
MANIFEST-first Pros (using tycho-maven-plugin)
Seems to be the recommended approach, or at least talked about as the recommended approach, but I can't really see why it has significant benefit. (Hence why this question was asked).
Good for developing Eclipse plugins and integrates well with PDE
Provides tooling for testing thus allowing ClassNotFoundException to appear during JUnit testing rather than runtime.
MANIFEST-first Cons
Seems to only work well on Eclipse based IDEs. You don't have to use Eclipse, but without the PDE would you want to?
Violates DRY principles since I have to do put keep the names and versions from the POM and MANIFEST.MF in sync.
Need to name things in a specific fashion
You cannot mix, meaning existing Maven multi-project installations cannot just tack on OSGi support
A lot more configuration compared to maven-bundle-plugin is needed to get less warnings: http://wiki.eclipse.org/Tycho/Reference_Card#Examplary_parent_POM
Have to make test cases a separate project. It won't run when built in src/test/java.
Seems that it will only test classes that are exposed, in other words those in ".internal." is not testable.
If I were asked for a recommendation for an enterprise that is using Maven already and want to move to OSGi then it would be POM first
If I were asked for a recommendation for someone who is doing Eclipse plugin development, then it is Manifest first -- with tycho
I think you should choose by use case. For server side OSGi projects I favour the pom first style. It nicely matches the maven builds and is much less error prone than Manifest first.
In fact bnd which is behind the maven bundle plugin gets the Manifest right for most cases without any additional config. The trick is to use some naming rules. For example if you name internal package impl or internal the will not be exported. Using this style you can not use the Eclipse plugin perspective (at least without bndtools which I do not like) but I did not yet miss this perspective. I am a developer in the Apache Karaf, CXF and Camel projects where we use this style and it works great. Especially for CXF and Camel it is great that we can support OSGi and non OSGi deployments with the same build and tools.
For Eclipse RCP applications Manifest first is the way to go as you need the plugin perspective and the Eclipse IDE tools. If you want to combine that with maven then tycho is probably the way to go.
MANIFEST first does not lock you to Eclipse (although I'd be surprised if more than a tiny minority would use anything else). The MANIFEST is the file that counts, and needs to be added to a jar, regardless how you do that.
On the other hand, POM first completely locks you to Maven, you lose the advantage that an OSGi bundle is a regular jar you can make any way you want.
I've tried both, I really prefer MANIFEST first. The MANIFEST file is a really important file, I prefer to craft that file over crafting a file that produces that file. If something weird happens, (and it will at some point) the MANIFEST file is the first to check, it's just easier if it's your own file. Besides, you will have to be familiar with it anyway.
So, if Maven is your alpha and omega, POM first will suit you best, but you'll still need to have in-depth understanding of the MANIFEST file.