I am trying to integrate Spark application with spring boot but since spark core also has jetty server and servlet packages, they are conflicting with spring boot web starter servlet packages.
I already followed the post below to exclude starter-logging
https://www.linkedin.com/pulse/integrating-spark-spring-boot-neeraj-malhotra
I tried excluding servlet from spark core package, but it wouldn't start up or build using gradle build.
My goal is to run this app as a Rest API using spring boot web starter.
Thanks
I found the solution a long time ago, but couldn't post this.
I had to exclude a bunch of packages from spark-core that contain servlet packages
compile(group: 'org.apache.spark', name: 'spark-core_2.11', version: '1.6.0') {
exclude group: 'org.slf4j', module: 'slf4j-simple'
exclude group: 'org.eclipse.jetty', module: 'jetty-server'
exclude group: 'com.sun.jersey', module: 'jersey-server'
exclude group: 'com.sun.jersey', module: 'jetty-core'
exclude group: 'org.eclipse.jetty.orbit', module: 'javax.servlet'
//exclude group: 'org.eclipse.jetty', module: 'jetty-server'
}
Related
I have a project build with Gradle, it's actually a Vaadin project, with a servlet where I use Jetty.
At the startup (i.e gradle run) I have a lot of different warning message from AnnotationParser about duplication of classes. I copy only one because the log is quite verbose:
[INFO ] 11:22:50.375 org.eclipse.jetty.server.Server.doStart() - jetty-9.4.31.v20200723; built: 2020-07-23T17:57:36.812Z; git: 450ba27947e13e66baa8cd1ce7e85a4461cacc1d; jvm 13.0.2+8
[WARN ] 11:22:50.777 org.eclipse.jetty.annotations.AnnotationParser.addParsedClass() - javax.websocket.ClientEndpoint scanned from multiple locations: jar:file:///Users/fox/.gradle/caches/modules-2/files-2.1/javax.websocket/javax.websocket-api/1.0/fc843b649d4a1dcb0497669d262befa3918c7ba8/javax.websocket-api-1.0.jar!/javax/websocket/ClientEndpoint.class, jar:file:///Users/fox/.gradle/caches/modules-2/files-2.1/javax.websocket/javax.websocket-client-api/1.0/afcf19e889d8725576811f8d47ab6c65d9dcbd78/javax.websocket-client-api-1.0.jar!/javax/websocket/ClientEndpoint.class
In this line AnnotationParser warns me that the ClientEndpoint.class is present in two jar, the javax.websocket-api-1.0.jar and the javax.websocket-client-api-1.0.jar.
With the command gradle dependencies I could see:
...
+--- org.eclipse.jetty.websocket:javax-websocket-client-impl:9.4.31.v20200723
| +--- org.eclipse.jetty.websocket:websocket-client:9.4.31.v20200723 (*)
| \--- javax.websocket:javax.websocket-client-api:1.0
+--- org.eclipse.jetty.websocket:websocket-server:9.4.31.v20200723 (*)
\--- javax.websocket:javax.websocket-api:1.0
In my gradle.build I only have:
dependencies {
// Vaadin
implementation enforcedPlatform('com.vaadin:vaadin-bom:18.0.6')
implementation("com.vaadin:vaadin-core")
implementation group: 'com.github.appreciated', name: 'vaadin-css-grid', version: '2.0.0'
// Logging
implementation group: 'org.apache.logging.log4j', name: 'log4j-core', version: '2.13.1'
implementation group: 'org.apache.logging.log4j', name: 'log4j-api', version: '2.13.1'
implementation group: 'org.apache.logging.log4j', name: 'log4j-slf4j-impl', version: '2.13.1'
// Testing
testImplementation group: 'junit', name: 'junit', version: '4.12'
testImplementation group: 'org.skyscreamer', name: 'jsonassert', version: '1.5.0'
// Jetty
implementation group: 'org.eclipse.jetty', name: 'jetty-server', version: '9.4.31.v20200723'
implementation group: 'org.eclipse.jetty', name: 'jetty-webapp', version: '9.4.31.v20200723'
implementation group: 'org.eclipse.jetty.websocket', name: 'websocket-server', version: '9.4.31.v20200723'
implementation group: 'org.eclipse.jetty.websocket', name: 'javax-websocket-server-impl', version: '9.4.31.v20200723'
}
All these logs are really annoying (they slow down the start up really much) and I can't understand if they are dangerous or not.
Which is the right way to avoid duplications ?
Is there a technique to instruct AnnotationParser to scan only some dependencies?
I know this topic is present in different questions but I didn't find a solution for gradle or a common solution strategy.
Thanks,
Stefano
Having the same class name in multiple locations on your classpath is a bad idea.
This is the most common form of unstable operation on Java there is!
The classloaders in Java have no guarantee of load order behavior if the same class is present in multiple locations within the classloader.
In one run you might accidentally load the classes in the order you intend, and have it run properly, then at a future date you run the same program and the load order is different, now you are running with a different class version and you have unexpected behavior.
The only way to fix this is to clean up your classloader and ensure that you only have 1 version of the class you intend to use.
This is what Jetty is telling you.
As for this specific one, javax.websocket-api and javax.websocket-client-api, you want to exclude javax.websocket-client-api at the gradle level, as all of the websocket client classes are also present in the javax.websocket-api.
The javax.websocket-client-api jar is only intended for projects that only use the javax.websocket Client, without a javax.websocket Server.
Following the suggestion of joakim-erdfelt
I have modified my build.gradle and this prevent the problem:
implementation ('org.eclipse.jetty.websocket:javax-websocket-server-impl:9.4.31.v20200723') {
exclude module: 'javax.websocket-client-api'
}
The Gradle documentation is here: Excluding transitive dependencies
I'm trying to use Springdoc on a multi-module gradle project, and I'm not able to make it work.
Has someone already done something similar ?
My Gradle project structure looks like this:
My project
Application
module1
api
domain
repository
module2
sames as module1
the Application module has dependencies on each module project
and
implementation group: 'org.springframework.boot', name: 'spring-boot-starter', version: springBootVersion
on API submodule there is:
implementation group: 'org.springframework.boot', name: 'spring-boot-starter-web', version: springStarterWeb
implementation group: 'org.springdoc', name: 'springdoc-openapi-ui', version: '1.4.2'
This is the link to a sample application with multi-module project using gradle.
It contains, different kinds of configuration using gradle multi-module projects of the springdoc-openapi-ui and the springdoc-openapi-mvc-core.
https://github.com/springdoc/springdoc-openapi-demos/tree/master/sample-springdoc-openapi-microservices
You can see how configuration on gradle is done.
Unable to use Spring's reactive WebClient in a Spring boot based Web Application which runs on Tomcat. Application startup itself is failing due to unresolved dependencies.
I looked at https://github.com/spring-projects/spring-boot/issues/9690 but unfortunately thats not an option for me as I cannot set the WebApplication type to None
My gradle file:
dependencies {
compile(group: 'org.springframework.boot', name: 'spring-boot-starter-data-jpa') {
exclude(module: 'tomcat-juli')
}
compile group: 'org.springframework.boot', name: 'spring-boot-starter-data-redis'
compile group: 'org.springframework.boot', name: 'spring-boot-starter-web'
compile group: 'org.springframework.session', name: 'spring-session'
compile group: 'org.hibernate', name: 'hibernate-java8'
compile group: 'org.hibernate', name: 'hibernate-validator', version: '5.2.4.Final'
compile group: 'com.fasterxml.jackson.datatype', name: 'jackson-datatype-guava'
compile group: 'com.fasterxml.jackson.datatype', name: 'jackson-datatype-hibernate5'
compile group: 'com.fasterxml.jackson.datatype', name: 'jackson-datatype-jsr310'
compile group: 'com.google.guava', name: 'guava'
compile group: 'org.apache.commons', name: 'commons-lang3', version: '3.3.2'
compile group: 'org.postgresql', name: 'postgresql'
compile group: 'io.jsonwebtoken', name: 'jjwt', version:'0.9.0'
compile group: 'com.nimbusds', name: 'nimbus-jose-jwt', version:'5.12'
// Spring managed dependency is not working, so resorted to using specific version
// compile('org.springframework.boot:spring-boot-starter-webflux')
compile group: 'org.springframework.boot', name: 'spring-boot-starter-webflux', version: '2.1.4.RELEASE'
/*
compile group: 'org.springframework', name: 'spring-web-reactive', version: '5.0.0.M1'
compile group: 'org.springframework', name: 'spring-web', version: '5.1.6.RELEASE'
*/
implementation 'com.auth0:jwks-rsa:0.7.0
}
bootRepackage {
enabled = false
}
Error:
Caused by: java.lang.NoClassDefFoundError: org/springframework/http/client/reactive/ClientHttpConnector
at org.springframework.web.reactive.function.client.WebClient.create(WebClient.java:144)
I then tried adding,
compile group: 'org.springframework', name: 'spring-web-reactive', version: '5.0.0.M1'
Which then complained about something else, which made me add
compile group: 'org.springframework', name: 'spring-web', version: '5.1.6.RELEASE'
But then another error shows up while starting my application, which makes me think there is some other problem.
To be precise,I simply want to call a MicroService using Reactive WebClient from within another MicroService (which is Spring Boot based Web Application).
Using RestTemplate works fine but I just wanted to try the reactive WebClient and stuck with error.
Due to the way we host our dependencies, managed dependencies are not working. Our dependency server is unable to resolve them. That's the reason for having specific version of spring boot dependencies
org.springframework.http.client.reactive.ClientHttpConnector is part of the spring-web jar so it should be available if use use spring-boot-starter-webflux or spring-boot-starter-web. If you're writing a traditional servlet based application then you should be using spring-boot-starter-web.
Check the output of gradle dependencies to ensure that you have a spring-web jar. If you do, the chances are it's somehow been corrupted and you should clear your gradle cache.
I'd also recommend that you use Spring Boot's managed dependencies rather than specifify version numbers on each dependency. This will ensure that you get versions that are known to work well together.
I use gradle and latest.release everywhere to be promiscuous, however, recently my Spring boot application stopped running from command line. When I run gradle clean bootRun I get...
Exception in thread "main" java.lang.IllegalArgumentException: LoggerFactory is not a Logback LoggerContext but Logback is on the classpath. Either remove Logback or the competing implementation (class org.slf4j.impl.Log4jLoggerFactory loaded from file:/.../.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-log4j12/1.7.25/110cefe2df103412849d72ef7a67e4e91e4266b4/slf4j-log4j12-1.7.25.jar). If you are using WebLogic you will need to add 'org.slf4j' to prefer-application-packages in WEB-INF/weblogic.xml: org.slf4j.impl.Log4jLoggerFactory
Per this post I tried...
configurations.all {
exclude group: 'org.springframework.boot', module: 'spring-boot-starter-tomcat'
exclude group: 'org.springframework.boot', module: 'spring-boot-starter-logging'
exclude group: 'org.springframework.boot', module: 'logback-classic'
exclude group: "org.slf4j"
}
and that silences the Logback error but spring won't start and fails with no error.
Caused by: org.gradle.process.internal.ExecException: Process 'command '/Library/Java/JavaVirtualMachines/jdk-10.0.1.jdk/Contents/Home/bin/java'' finished with non-zero exit value 1
What am I missing?
I had this and fixed it by excluding spring-boot-starter-logging and logback-classic and adding spring log4j2 implementation in gradle.build
configurations {
all {
exclude group: 'org.springframework.boot', module: 'spring-boot-starter-logging'
exclude group: 'ch.qos.logback', module: 'logback-classic'
}
}
dependencies {
implementation 'org.springframework.boot:spring-boot-starter-log4j2:2.2.6.RELEASE'
Needed to remove.. compile 'org.apache.spark:spark-sql_2.11:latest.release'
Haven't figured out through why.
Spring Boot automatically configures the Log Back library because of the web starter. You should exclude the spring-boot-starter-logging to fix the issue.
To exclude the default LogBack configuration on the gradle project,
configurations {
all*.exclude module : 'spring-boot-starter-logging'
all*.exclude module : 'logback-classic'
}
You can also try module replacement to replace the LogBack to Log4j or Log4j2
I am trying to run a spring boot embedded web container application that needs to talk to hdfs. I have included the spring data hadoop in my pom.xml but when I run the application on the cluster i get this error
Caused by: java.io.FileNotFoundException: JAR entry lib/hadoop-core-1.2.1.jar!/core-default.xml not found in hdfspub-0.0.1-SNAPSHOT.jar
at .jar.JarURLConnec...ction.java:122)
at JarURLConnec...ction.java:132)
at com.sun.org.apache.xerces.internal.impl.XMLEntityM anager.setupCurrentEntity(XMLEntityManager.java:65 1)
at com.sun.org.apache.xerces.internal.impl.XMLVersion Detector.determineDocVersion(XMLVersionDetector.ja va:186)
at com.sun.org.apache.xerces.internal.parsers.XML11Co nfiguration.parse(XML11Configuration.java:772)
at com.sun.org.apache.xerces.internal.parsers.XML11Co nfiguration.parse(XML11Configuration.java:737)
at com.sun.org.apache.xerces.internal.parsers.XMLPars er.parse(XMLParser.java:119)
at com.sun.org.apache.xerces.internal.parsers.DOMPars er.parse(DOMParser.java:232)
at com.sun.org.apache.xerces.internal.jaxp.DocumentBu ilderImpl.parse(DocumentBuilderImpl.java:284)
at javax.xml.parsers.DocumentBuilder.parse(DocumentBu ilder.java:180)
at org.apache.hadoop.conf.Configuration.loadResource( Configuration.java:1156)
Even thought if I look at my jar I can see that hadoop-core-1.2.1.jar is there
$ unzip -l hdfspub-0.0.1-SNAPSHOT.jar | grep hadoop-core-1.2.1.jar
4203713 10-25-2013 21:04 lib/hadoop-core-1.2.1.jar
Any help would be appreciated
I had same problem, solution is to add hadoop.core to your pom/build.gradle but you have to exclude from spring.data.hadoop and apache.hadoop groups:
tomcat
org.mortbay.jetty
javax.servlet
my build.gradle:
compile('org.apache.hadoop:hadoop-core:1.2.1'){
exclude group: 'org.mortbay.jetty'
exclude group: 'javax.servlet'
exclude group: 'tomcat'
}
compile('org.springframework.data:spring-data-hadoop:1.0.2.RELEASE') {
exclude group: 'org.mortbay.jetty'
exclude group: 'javax.servlet'
exclude group: 'tomcat'
}