Jetty AnnotationParser scanned from multiple locations warning - gradle

I have a project build with Gradle, it's actually a Vaadin project, with a servlet where I use Jetty.
At the startup (i.e gradle run) I have a lot of different warning message from AnnotationParser about duplication of classes. I copy only one because the log is quite verbose:
[INFO ] 11:22:50.375 org.eclipse.jetty.server.Server.doStart() - jetty-9.4.31.v20200723; built: 2020-07-23T17:57:36.812Z; git: 450ba27947e13e66baa8cd1ce7e85a4461cacc1d; jvm 13.0.2+8
[WARN ] 11:22:50.777 org.eclipse.jetty.annotations.AnnotationParser.addParsedClass() - javax.websocket.ClientEndpoint scanned from multiple locations: jar:file:///Users/fox/.gradle/caches/modules-2/files-2.1/javax.websocket/javax.websocket-api/1.0/fc843b649d4a1dcb0497669d262befa3918c7ba8/javax.websocket-api-1.0.jar!/javax/websocket/ClientEndpoint.class, jar:file:///Users/fox/.gradle/caches/modules-2/files-2.1/javax.websocket/javax.websocket-client-api/1.0/afcf19e889d8725576811f8d47ab6c65d9dcbd78/javax.websocket-client-api-1.0.jar!/javax/websocket/ClientEndpoint.class
In this line AnnotationParser warns me that the ClientEndpoint.class is present in two jar, the javax.websocket-api-1.0.jar and the javax.websocket-client-api-1.0.jar.
With the command gradle dependencies I could see:
...
+--- org.eclipse.jetty.websocket:javax-websocket-client-impl:9.4.31.v20200723
| +--- org.eclipse.jetty.websocket:websocket-client:9.4.31.v20200723 (*)
| \--- javax.websocket:javax.websocket-client-api:1.0
+--- org.eclipse.jetty.websocket:websocket-server:9.4.31.v20200723 (*)
\--- javax.websocket:javax.websocket-api:1.0
In my gradle.build I only have:
dependencies {
// Vaadin
implementation enforcedPlatform('com.vaadin:vaadin-bom:18.0.6')
implementation("com.vaadin:vaadin-core")
implementation group: 'com.github.appreciated', name: 'vaadin-css-grid', version: '2.0.0'
// Logging
implementation group: 'org.apache.logging.log4j', name: 'log4j-core', version: '2.13.1'
implementation group: 'org.apache.logging.log4j', name: 'log4j-api', version: '2.13.1'
implementation group: 'org.apache.logging.log4j', name: 'log4j-slf4j-impl', version: '2.13.1'
// Testing
testImplementation group: 'junit', name: 'junit', version: '4.12'
testImplementation group: 'org.skyscreamer', name: 'jsonassert', version: '1.5.0'
// Jetty
implementation group: 'org.eclipse.jetty', name: 'jetty-server', version: '9.4.31.v20200723'
implementation group: 'org.eclipse.jetty', name: 'jetty-webapp', version: '9.4.31.v20200723'
implementation group: 'org.eclipse.jetty.websocket', name: 'websocket-server', version: '9.4.31.v20200723'
implementation group: 'org.eclipse.jetty.websocket', name: 'javax-websocket-server-impl', version: '9.4.31.v20200723'
}
All these logs are really annoying (they slow down the start up really much) and I can't understand if they are dangerous or not.
Which is the right way to avoid duplications ?
Is there a technique to instruct AnnotationParser to scan only some dependencies?
I know this topic is present in different questions but I didn't find a solution for gradle or a common solution strategy.
Thanks,
Stefano

Having the same class name in multiple locations on your classpath is a bad idea.
This is the most common form of unstable operation on Java there is!
The classloaders in Java have no guarantee of load order behavior if the same class is present in multiple locations within the classloader.
In one run you might accidentally load the classes in the order you intend, and have it run properly, then at a future date you run the same program and the load order is different, now you are running with a different class version and you have unexpected behavior.
The only way to fix this is to clean up your classloader and ensure that you only have 1 version of the class you intend to use.
This is what Jetty is telling you.
As for this specific one, javax.websocket-api and javax.websocket-client-api, you want to exclude javax.websocket-client-api at the gradle level, as all of the websocket client classes are also present in the javax.websocket-api.
The javax.websocket-client-api jar is only intended for projects that only use the javax.websocket Client, without a javax.websocket Server.
Following the suggestion of joakim-erdfelt
I have modified my build.gradle and this prevent the problem:
implementation ('org.eclipse.jetty.websocket:javax-websocket-server-impl:9.4.31.v20200723') {
exclude module: 'javax.websocket-client-api'
}
The Gradle documentation is here: Excluding transitive dependencies

Related

Multiple Build-Tasks with changing sets of dependencies in Gradle

I would like to run tests of an library against different backend implementations.
The implementations are injected in my project by the magic of Spring.
Currently I do this, by changing the line
testImplementation group: 'org.mycorp.exmpl' name: 'redis-backend' version: '0.0.1'
to
testImplementation group: 'org.mycorp.exmpl' name: 'mongo-backend' version: '0.0.1'
to
testImplementation group: 'org.mycorp.exmpl' name: '<whatever-backend>' version: '<whatever.version>'
then running
gradlew test
(resp. clicking on 'test' in my IntelliJ IDE)
This is cumbersome. And I can't do that with my build pipeline (I guess).
In my imagination, I would be able to create
a task testRedis which depends on the task buildRedisTestClasses
a task testMongo which depends on the task buildMongoTestClasses
...
so I can just select the correct task in my IDE or run all of them in my build pipeline.
I found some hints on the web, that you can use something like this:
dependencies {
testImplementation group: 'org.mycorp.exmpl', name: 'redis-backend', version: '0.0.1', configuration: 'redis'
testImplementation group: 'org.mycorp.exmpl', name: 'mongo-backend', version: '0.0.1', configuration: 'mongo'
}
But thats about where I stopped finding/understanding.
(As you might notice, I am a gradle novice)
Try something alike:
./gradlew test -Dbackend=redis
dependencies {
if (System.getProperty("backend") == "redis") {
implementation "org.mycorp.exmpl:redis-backend:0.0.1"
} else {
implementation "org.mycorp.exmpl:mongo-backend:0.0.1"
}
}
Or even:
./gradlew test -Dbackend=redis-backend:0.0.1
implementation "org.mycorp.exmpl:${System.getProperty("backend")}"
Making it a project property -Pbackend with a default value suggested.
See Environment Options.

What are Gradle dependency configurations?

Could somebody please explain in more details (preferably with some real-life examples) what dependency configurations are in Gradle?
For example, let's assume I have:
compile group: 'org.foo', name: 'commons-lang', version: '1.2.3', configuration: 'runtime'
testCompile group: 'org.foo', name: 'commons-io', version: '2.4.1', configuration: 'testing'
I was under the impression that compile and testCompile are the configurations. If so, then what's the configuration: 'testing' part for?
These are build configuration names of artifacts... they act just alike when passing a build configuration name to a referenced project, except that these packages always carry the one configuration name they were built with - while projects may have several, resulting in several artifacts.

How to use Spring's reactive WebClient in a Spring Web Application which runs on Tomcat

Unable to use Spring's reactive WebClient in a Spring boot based Web Application which runs on Tomcat. Application startup itself is failing due to unresolved dependencies.
I looked at https://github.com/spring-projects/spring-boot/issues/9690 but unfortunately thats not an option for me as I cannot set the WebApplication type to None
My gradle file:
dependencies {
compile(group: 'org.springframework.boot', name: 'spring-boot-starter-data-jpa') {
exclude(module: 'tomcat-juli')
}
compile group: 'org.springframework.boot', name: 'spring-boot-starter-data-redis'
compile group: 'org.springframework.boot', name: 'spring-boot-starter-web'
compile group: 'org.springframework.session', name: 'spring-session'
compile group: 'org.hibernate', name: 'hibernate-java8'
compile group: 'org.hibernate', name: 'hibernate-validator', version: '5.2.4.Final'
compile group: 'com.fasterxml.jackson.datatype', name: 'jackson-datatype-guava'
compile group: 'com.fasterxml.jackson.datatype', name: 'jackson-datatype-hibernate5'
compile group: 'com.fasterxml.jackson.datatype', name: 'jackson-datatype-jsr310'
compile group: 'com.google.guava', name: 'guava'
compile group: 'org.apache.commons', name: 'commons-lang3', version: '3.3.2'
compile group: 'org.postgresql', name: 'postgresql'
compile group: 'io.jsonwebtoken', name: 'jjwt', version:'0.9.0'
compile group: 'com.nimbusds', name: 'nimbus-jose-jwt', version:'5.12'
// Spring managed dependency is not working, so resorted to using specific version
// compile('org.springframework.boot:spring-boot-starter-webflux')
compile group: 'org.springframework.boot', name: 'spring-boot-starter-webflux', version: '2.1.4.RELEASE'
/*
compile group: 'org.springframework', name: 'spring-web-reactive', version: '5.0.0.M1'
compile group: 'org.springframework', name: 'spring-web', version: '5.1.6.RELEASE'
*/
implementation 'com.auth0:jwks-rsa:0.7.0
}
bootRepackage {
enabled = false
}
Error:
Caused by: java.lang.NoClassDefFoundError: org/springframework/http/client/reactive/ClientHttpConnector
at org.springframework.web.reactive.function.client.WebClient.create(WebClient.java:144)
I then tried adding,
compile group: 'org.springframework', name: 'spring-web-reactive', version: '5.0.0.M1'
Which then complained about something else, which made me add
compile group: 'org.springframework', name: 'spring-web', version: '5.1.6.RELEASE'
But then another error shows up while starting my application, which makes me think there is some other problem.
To be precise,I simply want to call a MicroService using Reactive WebClient from within another MicroService (which is Spring Boot based Web Application).
Using RestTemplate works fine but I just wanted to try the reactive WebClient and stuck with error.
Due to the way we host our dependencies, managed dependencies are not working. Our dependency server is unable to resolve them. That's the reason for having specific version of spring boot dependencies
org.springframework.http.client.reactive.ClientHttpConnector is part of the spring-web jar so it should be available if use use spring-boot-starter-webflux or spring-boot-starter-web. If you're writing a traditional servlet based application then you should be using spring-boot-starter-web.
Check the output of gradle dependencies to ensure that you have a spring-web jar. If you do, the chances are it's somehow been corrupted and you should clear your gradle cache.
I'd also recommend that you use Spring Boot's managed dependencies rather than specifify version numbers on each dependency. This will ensure that you get versions that are known to work well together.

"Unresolved requirement: Import-Package" for a module not in my build.gradle

I want to use Elasticsearch's Client Java class within a Liferay 7 SP4 FP30 module, so I wrote this build.gradle:
dependencies {
compileOnly group: "com.liferay", name: "com.liferay.portal.search.elasticsearch", version: "2.1.14"
compileOnly group: "com.liferay", name: "org.elasticsearch", version: "2.2.0.LIFERAY-PATCHED-1"
compileOnly group: "biz.aQute.bnd", name: "biz.aQute.bndlib", version: "3.1.0"
compileOnly group: "com.liferay", name: "com.liferay.osgi.util", version: "3.0.0"
compileOnly group: "com.liferay", name: "com.liferay.portal.spring.extender", version: "2.0.0"
compileOnly group: "com.liferay.portal", name: "com.liferay.portal.kernel", version: "2.0.0"
compileOnly group: "com.liferay", name: "com.liferay.portal.security.audit.api", version: "2.0.0"
compileOnly group: "com.liferay", name: "com.liferay.portal.configuration.metatype", version: "2.0.0"
compileOnly group: "org.osgi", name: "org.osgi.compendium", version: "5.0.0"
}
... and a Java class containing code such as import com.liferay.portal.search.elasticsearch.connection.ElasticsearchConnectionManager; and Client client = elasticsearchConnectionManager.getClient();
It builds fine.
But when I try to start the module, this error happens:
org.osgi.framework.BundleException: Could not resolve module: mymodule [548]
Unresolved requirement: Import-Package: com.liferay.portal.search.elasticsearch.connection
Why is this happening? My build.gradle does not mention this module ending in .connection, and Maven does not seem to have any such module.
#gjoranv is correct, just because you in is on your gradle.build it does not mean it will be in your environment.
First things first, the error is due to the lack of a used package, in Java's conventional sense. So you will need a module, as represented by a jar file, that makes this package public.
As liferay is pretty version dependent when it comes to Elastic Search, and relies on accident versions, you might get away with using not exposed packages, and forcing the exposure, normally through a Uber module.
If you are feeling lucky, you can also use compileInclude, instead of compileOnly. Including the library this way will possibly make a mess, as it will embed the jar inside your jar and expose all packages.
Another possibility, which normally is way less aggressive is to embed the jar, and set the classpath inside your bundle. To do this you just need to declare your dependency as compile, and add the classpath in your bnd.bnd file. (it sounds harder than it is, it should be a trivial process)
Another issue to have in mind is the alignment with your ElasticSearch and you liferay deployment:2.2-2.4.x but this is just because you might fall into class conversion exceptions and API mismatch if your objects are used by other bundles or when interfacing with an old ES.
Embedding example:
gradle.build
compile "org.apache.httpcomponents:httpclient"
compile "org.apache.httpcomponents:httpcore"
bnd.bnd
-includeresource: lib/httpclient.jar=httpclient-4.5.3.jar,\
lib/httpcore.jar=httpcore-4.4.6.jar
Bundle-ClassPath: ., lib/httpclient.jar, lib/httpcore.jar
I'm not familiar with Liferay and gradle, but I've been working with OSGi (apache felix) and maven for a long time. The error message indicates that your bundle uses the package com.liferay.portal.search.elasticsearch.connection, but the runtime environment does not have a bundle that exports that package. The package in question is contained in the first dependency mentioned in your build.gradle, but it's not exported. If you like, you can open the bundle jar and peek into its manifest.mf by downloading it from the maven central repo.
Since the package is not exported (only com.liferay.portal.search.elasticsearch.settings is), I assume it's a signal that it's not intended for external use. So maybe you should check if there's another way of doing what you want.
From looking at the Liferay docs for using 3rd party libraries, it seems you are trying to expand the library into your module. Maybe you could try the embedding strategy instead, if you still need to use the .connection package.

How to exclude dependency with classifier (platform version) in Gradle?

In my project I have dependency on 'org.nd4j:nd4j-native-platform:0.6.0' which brings me transitive dependencies:
Gradle: org.nd4j:nd4j-native:linux-ppc64le:0.6.0
Gradle: org.nd4j:nd4j-native:macosx-x86_64:0.6.0
Gradle: org.nd4j:nd4j-native:windows-x86_64:0.6.0
Gradle: org.nd4j:nd4j-native:linux-x86_64:0.6.0
I want to exclude nd4j-native:linux-ppc64le and nd4j-native:macosx-x86_64 since my application does not support these platforms. I write in my Gradle file:
configurations {
all.collect { configuration ->
configuration.exclude(group: 'org.nd4j', module: 'nd4j-native', classifier: 'linux-ppc64le')
}
}
Gradle says:
Error:(44, 0) Could not set unknown property 'classifier' for object of type org.gradle.api.internal.artifacts.DefaultExcludeRule.
It seems that gradle does not support exclusion by classifier.
How to exclude such a transitive dependencies?
Update: Gradle allows us to exclude dependencies, but what if we have several dependencies with the same id and group but different classifiers?
I have faced the same issued. I have used a deeplearning4j library with Gradle dependency.
compile group: 'org.nd4j', name: 'nd4j-native-platform', version: '1.0.0-beta'
compile group: 'org.deeplearning4j', name: 'deeplearning4j-core', version: '1.0.0-beta'
When I use this it is also downloading other platform classifiers and its size is almost 500MB. but my use case is specific to the Windows platform so i don't need other classifiers for Linux and Android and other platforms.If I exclude the group it is also excluding the classifier for the windows also . And in Gradle as of my knowledge, we can not exclude specific classifiers.
So the question was how to remove the specific classifier. What I found strange is when I made jar file of the project and extracted jar it shows me the org.nd4j:nd4j-native:linux-ppc64le:0.6.0 and other jars but when I generate dependency tree it is not showing me the specific jar in the tree.
So in order to find out in which specific module and project the jar is coming from I made a separate maven project and with this
<dependency>
<groupId>org.nd4j</groupId>
<artifactId>nd4j-native-platform</artifactId>
<version>1.0.0-beta</version>
</dependency>
<dependency>
<groupId>org.deeplearning4j</groupId>
<artifactId>deeplearning4j-core</artifactId>
<version>1.0.0-beta</version>
</dependency>
and then I have generated a dependency tree. It showed me the jars in the dependency tree.
What I did is I have removed the whole module and I have added the required classifier in a particular module with a specific version and it worked for me.
compile (group: 'org.deeplearning4j', name: 'deeplearning4j-core', version: '1.0.0-beta')
{
exclude group: 'org.bytedeco.javacpp-presets', module: 'opencv-platform'
exclude group: 'org.bytedeco.javacpp-presets', module: 'leptonica-platform'
exclude group: 'org.bytedeco.javacpp-presets', module: 'hdf5-platform'
}
compile (group: 'org.nd4j', name: 'nd4j-native-platform', version: '1.0.0-beta')
{
exclude group: 'org.bytedeco.javacpp-presets', module: 'openblas-platform'
}
compile group: 'org.nd4j', name: 'nd4j-native', version: '1.0.0-beta', classifier: "windows-x86_64"
compile group: 'org.bytedeco.javacpp-presets', name: 'openblas', version: '0.2.20-1.4.1'
compile group: 'org.bytedeco.javacpp-presets', name: 'openblas', version: '0.2.20-1.4.1', classifier: "windows-x86"
compile group: 'org.bytedeco.javacpp-presets', name: 'openblas', version: '0.2.20-1.4.1', classifier: "windows-x86_64"
compile group: 'org.bytedeco.javacpp-presets', name: 'opencv', version: '3.4.1-1.4.1'
compile group: 'org.bytedeco.javacpp-presets', name: 'opencv', version: '3.4.1-1.4.1',classifier: "windows-x86"
compile group: 'org.bytedeco.javacpp-presets', name: 'opencv', version: '3.4.1-1.4.1',classifier: "windows-x86_64"
compile group: 'org.bytedeco.javacpp-presets', name: 'leptonica', version: '1.75.3-1.4.1'
compile group: 'org.bytedeco.javacpp-presets', name: 'leptonica', version: '1.75.3-1.4.1',classifier: "windows-x86"
compile group: 'org.bytedeco.javacpp-presets', name: 'leptonica', version: '1.75.3-1.4.1',classifier: "windows-x86_64"
Doing this reduced my jar size to almost 250MB
How to exclude such a transitive dependencies?
I think, the only way is to exclude all transitive dependencies by it's module or group and manually provide dependencies on libraries for platforms your application supports. Because classifiers are supported in dependency declaration.
And the same way you can handle the case, when you have a number of dependencies with the same module and grooup, but with different classifiers. Just add such dependencies manually with it's classifier property.

Resources