Dears,
I have a jersey - spring api deployed on apache tomcat 9.0.46. (Jersey to handle restful services JAX-RS and Spring to handle all my beans{controllers, DAO, SessionFactory, JPA etc...}).
Everything works fine on tomcat 9 on windows...
When deploying the exact same war in ubuntu tomcat 9.0.46, the ContextLoader is getting triggered 3 times and I have all my singletons instantiated 3 times. I'm deploying the api on tomcat ports 80 and 443 (https - godady certificate).
once I start tomcat the war is deployed and ports 80 and 443 get started (netstat -tulnp | grep java) and I see in log all singletons instantiated. (pool-2) Applicationcontext class my custom spring #Configuration class and it is getting triggered and DB is accessed without any issues
2021-06-09 14:41:52,128 1104 [main] INFO o.s.web.context.ContextLoader - Root WebApplicationContext initialized in 905 ms
2021-06-09 14:41:53,124 2100 [pool-2-thread-1] INFO skd.app.core.ApplicationContext - TASK::cleanExpiredStatuses
then the server takes few minutes (around 10 minutes 14:41 above then 14:51 below) and when port 8005 is started I see again the ContextLoader is triggered again 2 times.
09-Jun-2021 14:51:36.196 WARNING [main] org.apache.catalina.util.SessionIdGeneratorBase.createSecureRandom Creation of SecureRandom instance for session ID generation using [SHA1PRNG] took [584,064] milliseconds.
09-Jun-2021 14:51:36.592 INFO [main] org.glassfish.jersey.server.ApplicationHandler.initialize Initiating Jersey application, version Jersey: 2.6 2014-02-18 21:52:53...
09-Jun-2021 14:51:37.042 INFO [main] org.apache.catalina.startup.HostConfig.deployWAR Deployment of web application archive [/opt/apache-tomcat-9.0.46/webapps/skd-service.war] has finished in [588,185] ms
2021-06-09 14:51:39,388 696 [main] INFO o.s.web.context.ContextLoader - Root WebApplicationContext initialized in 581 ms
09-Jun-2021 14:51:39.632 INFO [main] org.glassfish.jersey.server.ApplicationHandler.initialize Initiating Jersey application, version Jersey: 2.6 2014-02-18 21:52:53...
09-Jun-2021 14:51:40.013 INFO [main] org.apache.catalina.startup.HostConfig.deployWAR Deploying web application archive [/opt/apache-tomcat-9.0.46/webapps/skd-service.war]
and again for the 3rd time:
2021-06-09 14:51:41,989 744 [main] INFO o.s.web.context.ContextLoader - Root WebApplicationContext initialized in 605 ms
09-Jun-2021 14:51:42.232 INFO [main] org.glassfish.jersey.server.ApplicationHandler.initialize Initiating Jersey application, version Jersey: 2.6 2014-02-18 21:52:53...
09-Jun-2021 14:51:42.602 INFO [main] org.apache.catalina.startup.HostConfig.deployWAR Deployment of web application archive [/opt/apache-tomcat-9.0.46/webapps/skd-service.war] has finished in [2,590] ms
Everything is working fine in windows, only when deploying to ubuntu tomcat, I'm getting this.
Does anyone have a clue why this difference in tomcat behaviour between windows and ubuntu for the same exact WAR file?
I have managed to figure out the problem. The issue was related to tomcat configuration in /conf/server.xml. Multiple Hosts will trigger the context loader to be triggered for each. I was keeping the default appBase to webapps for all host thus triggering the ContextLoader of each my war for each host. Another reason the ContextLoader will triggered multiple times as well is defining the option inside unless you need to load something external to your war.
I recommend reading specs:
https://tomcat.apache.org/tomcat-4.1-doc/config/host.html
Related
I just migrated to a new spring boot version lately, namely v2.6.2 and now I find the following exception of some of the services logs.
23:27:45.148 [main] INFO o.s.b.w.e.tomcat.TomcatWebServer - Tomcat started on port(s): 8080 (https) with context path ''
23:27:47.046 [main] INFO o.a.c.i.engine.AbstractCamelContext - Routes startup (total:1 started:1)
23:27:47.048 [main] INFO o.a.c.i.engine.AbstractCamelContext - Started route1 (jms://queue:inp.contentextractor.entry)
23:27:47.049 [main] INFO o.a.c.i.engine.AbstractCamelContext - Apache Camel 3.14.0 (camel-1) started in 2s228ms (build:622ms init:1s32ms start:574ms)
23:27:47.061 [main] INFO eu.hermes.esb.cloud.Application - Started Application in 25.305 seconds (JVM running for 27.737)
23:28:20.356 [https-jsse-nio-8080-exec-4] INFO o.a.c.c.C.[Tomcat].[localhost].[/] - Initializing Spring DispatcherServlet 'dispatcherServlet'
23:28:20.357 [https-jsse-nio-8080-exec-4] INFO o.s.web.servlet.DispatcherServlet - Initializing Servlet 'dispatcherServlet'
23:28:20.359 [https-jsse-nio-8080-exec-4] INFO o.s.web.servlet.DispatcherServlet - Completed initialization in 2 ms
00:05:19.986 [https-jsse-nio-8080-Acceptor] ERROR org.apache.tomcat.util.net.Acceptor - Socket accept failed
java.io.IOException: Duplicate accept detected. This is a known OS bug. Please consider reporting that you are affected: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1924298
at org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:545)
at org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:78)
at org.apache.tomcat.util.net.Acceptor.run(Acceptor.java:129)
at java.base/java.lang.Thread.run(Thread.java:834)
05:20:49.981 [https-jsse-nio-8080-Acceptor] ERROR org.apache.tomcat.util.net.Acceptor - Socket accept failed
java.io.IOException: Duplicate accept detected. This is a known OS bug. Please consider reporting that you are affected: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1924298
at org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:545)
at org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:78)
at org.apache.tomcat.util.net.Acceptor.run(Acceptor.java:129)
at java.base/java.lang.Thread.run(Thread.java:834)
What does it mean and how harmful is it ?
Following the link provided in the bug (https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1924298), and looking into comments made by community members, it seems that there is something in tomcat usage that uncovered a hidden issue in linux kernal 5.10-rc4 and was most likely fixed in 5.10-rc6.
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1924298/comments/15
This issue was reported across centOS, amazon linux, Arch linux and even Window 2019 - when using a lower linux kernel than 5.10.
The conclusion from this discussion seems to be the suggestion to update the OS itself to a stable version that uses linux kernel >= 5.10
On my local machine Windows 10 (64 bit) I start docker Toolbox. Then I pull Tomcat image and run it like this:
docker run -it tomcat
It's success run.
31-Dec-2019 17:54:27.598 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deploying web application directory [/usr/local/tomcat/webapps/ROOT]
31-Dec-2019 17:54:28.849 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory [/usr/local/tomcat/webapps/ROOT] has finished in [1,246] ms
31-Dec-2019 17:54:28.851 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deploying web application directory [/usr/local/tomcat/webapps/host-manager]
31-Dec-2019 17:54:29.029 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory [/usr/local/tomcat/webapps/host-manager] has finished in [178] ms
31-Dec-2019 17:54:29.031 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deploying web application directory [/usr/local/tomcat/webapps/docs]
31-Dec-2019 17:54:29.128 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory [/usr/local/tomcat/webapps/docs] has finished in [90] ms
31-Dec-2019 17:54:29.129 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deploying web application directory [/usr/local/tomcat/webapps/manager]
31-Dec-2019 17:54:29.251 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory [/usr/local/tomcat/webapps/manager] has finished in [122] ms
31-Dec-2019 17:54:29.254 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deploying web application directory [/usr/local/tomcat/webapps/examples]
31-Dec-2019 17:54:30.480 INFO [localhost-startStop-1] org.apache.catalina.startup.HostConfig.deployDirectory Deployment of web application directory [/usr/local/tomcat/webapps/examples] has finished in [1,226] ms
31-Dec-2019 17:54:30.499 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["http-nio-8080"]
31-Dec-2019 17:54:30.592 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["ajp-nio-8009"]
31-Dec-2019 17:54:30.621 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in 3185 ms
But when I open my browser and input
http://127.0.0.1:8080/
But page not found
Why?
#DavidMaze is right: there are simultaneously several issues (at least two, maybe more, see below) in the situation you describe, and each issue individually prevents a browser on the Windows host from accessing the web application deployed in the Docker container:
The web application needs to listen to the 0.0.0.0 special IP address (not localhost).
The -p (--publish) option of docker run is required. Let's say the web-app listens for port 8080, then to publish this exposed port to the 80 port on the host: docker run -p 80:8080 […] (but the two ports can be the same)
Finally as you use Docker Toolbox on Windows, you need to identify the Docker Machine IP where the web-app will be available (not localhost): if this information is not directly displayed by Docker Toolbox at startup, you may want to run: docker-machine ip dev.
For more details
Regarding items 1. 2., see also How to convert a Spring-Boot web service into a Docker image?
Regarding item 3., see e.g. Can't access Spring Boot with my docker image
(albeit these threads deal with Spring-Boot, the details there are quite generic)
My Grails 4 application, deployed to GCP appears to be trying to start up after being deployed but never comes up properly. Application requests return a 500 response. There are no errors or clues with DEBUG log level output at the root level.
The same application runs fine locally in development mode.
The production configuration is as per the Grails 3 deployment (to GCP) guide except for the adjustments that were necessary to make it work for Grails 4/Java 11.
Most of the bootstrapping appears to carry out as expected;
Spring Security configures successfully
Spring Security REST configures successfully
Spring beans are registered
Connects to Cloud SQL instance
Database schema is created (by Liquibase)
Plugins are loaded successfully
Then it gets to the following familiar lines of output logging;
INFO --- [main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat initialized with port(s): 8080 (http)
INFO --- [main] o.a.coyote.http11.Http11NioProtocol : Initializing ProtocolHandler ["http-nio-8080"]
INFO --- [main] o.apache.catalina.core.StandardService : Starting service [Tomcat]
INFO --- [main] org.apache.catalina.core.StandardEngine : Starting Servlet engine: [Apache Tomcat/9.0.17]
and restarts..
while normally, the next phase of bootstrapping (happens locally) would be;
[restartedMain] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring embedded WebApplicationContext
[restartedMain] o.s.web.context.ContextLoader : Root WebApplicationContext: initialization completed in 42018 ms
Probably a long shot question but any level of clues or suggestions would be much appreciated.
I've run out of doors to open. :-(
You could try to update Tomcat version just in case.
But looking at Grails documentation:
https://docs.grails.org/latest/guide/upgrading.html
There's some configuration made when you upgrade to Grails 4 from
Grails 3.3.x to prevents the server from restarting when views or
message bundles are changed.
Either modifying a gsp in Grails 4 M2 cause the application to restart.
https://github.com/grails/grails-core/issues/11284
I am trying to get a Spring Boot WAR to deploy to a private Tomcat 7.0.68 instance hosted by DailyRazor. I cannot figure out why the Spring Boot application will not work. I do not seem to get any errors. Locally, I have gotten this to work with STS 3.8.4, Tomcat 7.0.78, Tomcat 8.5.11, and a couple other versions of Tomcat.
I never see the Spring Boot banner display on the DailyRazor instance nor much of the other logging indicating that my Spring Boot application is starting and loading all of the beans. This is the only logging I see:
INFO main org.apache.catalina.core.StandardService - Stopping service Catalina
INFO main org.apache.catalina.core.AprLifecycleListener - The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /usr/local/tomcat/users/user_id/jdk/jre/lib/amd64/server:/usr/local/tomcat/users/user_id/jdk/jre/lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
INFO main org.apache.catalina.startup.Catalina - Initialization processed in 2299 ms
INFO main org.apache.catalina.core.StandardService - Starting service Catalina
INFO main org.apache.catalina.core.StandardEngine - Starting Servlet Engine: Apache Tomcat/7.0.68
INFO foobar.com-startStop-1 org.apache.catalina.startup.TldConfig - At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.
INFO foobar.com-startStop-1 org.apache.catalina.util.SessionIdGeneratorBase - Creation of SecureRandom instance for session ID generation using [SHA1PRNG] took [271] milliseconds.
INFO foobar.com-startStop-1 org.apache.catalina.startup.HostConfig - Deploying web application archive /home/user_id/tomcat/webapps/foobar.com/ROOT.war
INFO foobar.com-startStop-1 org.apache.catalina.startup.TldConfig - At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.
INFO foobar.com-startStop-1 org.apache.catalina.startup.HostConfig - Deployment of web application archive /home/user_id/tomcat/webapps/foobar.com/ROOT.war has finished in 1,603 ms
INFO foobar.com-startStop-1 org.apache.catalina.startup.HostConfig - Deploying web application archive /home/user_id/tomcat/webapps/foobar.com/plant-service-0.0.1-SNAPSHOT.war
INFO foobar.com-startStop-1 org.apache.catalina.startup.TldConfig - At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.
INFO foobar.com-startStop-1 org.apache.catalina.core.ContainerBase.[Catalina].[foobar.com].[/plant-service-0.0.1-SNAPSHOT] - 3 Spring WebApplicationInitializers detected on classpath
INFO foobar.com-startStop-1 org.apache.catalina.core.ContainerBase.[Catalina].[foobar.com].[/plant-service-0.0.1-SNAPSHOT] - Initializing Spring embedded WebApplicationContext
INFO foobar.com-startStop-1 org.apache.catalina.startup.HostConfig - Deployment of web application archive /home/user_id/tomcat/webapps/foobar.com/plant-service-0.0.1-SNAPSHOT.war has finished in 21,769 ms
INFO main org.apache.catalina.startup.Catalina - Server startup in 26534 ms
INFO ajp-bio-127.0.0.1-9592-exec-1 org.apache.catalina.core.ContainerBase.[Catalina].[foobar.com].[/plant-service-0.0.1-SNAPSHOT] - Initializing Spring FrameworkServlet 'dispatcherServlet'
I have following the instructions for packaging as a WAR, extending SpringBootServletInitializer, overriding configure(...), etc. Again, this all works locally on multiple versions of Tomcat. I have tried to adjust the logging, but I do not see anything useful.
Any hints? Suggestions?
EDIT
Not sure if it is related, but when shutting down Tomcat, I see the following:
INFO main org.apache.catalina.core.StandardService - Stopping service Catalina
ERROR foobar.com-startStop-2 org.apache.catalina.loader.WebappClassLoaderBase - The web application [] appears to have started a thread named [Abandoned connection cleanup thread] but has failed to stop it. This is very likely to create a memory leak.
INFO foobar.com-startStop-2 org.apache.catalina.core.ContainerBase.[Catalina].[foobar.com].[/plant-service-0.0.1-SNAPSHOT] - Destroying Spring FrameworkServlet 'dispatcherServlet'
INFO foobar.com-startStop-2 org.apache.catalina.core.ContainerBase.[Catalina].[foobar.com].[/plant-service-0.0.1-SNAPSHOT] - Closing Spring root WebApplicationContext
ERROR foobar.com-startStop-2 org.apache.catalina.loader.WebappClassLoaderBase - The web application [/plant-service-0.0.1-SNAPSHOT] registered the JDBC driver [com.mysql.jdbc.Driver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.
ERROR foobar.com-startStop-2 org.apache.catalina.loader.WebappClassLoaderBase - The web application [/plant-service-0.0.1-SNAPSHOT] appears to have started a thread named [Abandoned connection cleanup thread] but has failed to stop it. This is very likely to create a memory leak.
Could you please ensure you did all of this correctly ?
From Create a deployable war file (Spring boot documentation) :
Create a deployable war file
The first step in producing a deployable war file is to provide a
SpringBootServletInitializer subclass and override its configure
method. This makes use of Spring Framework’s Servlet 3.0 support and
allows you to configure your application when it’s launched by the
servlet container. Typically, you update your application’s main class
to extend SpringBootServletInitializer:
#SpringBootApplication
public class Application extends SpringBootServletInitializer {
#Override
protected SpringApplicationBuilder configure(SpringApplicationBuilder application) {
return application.sources(Application.class);
}
public static void main(String[] args) throws Exception {
SpringApplication.run(Application.class, args);
}
}
The next step is to update your build configuration so that your
project produces a war file rather than a jar file. If you’re using
Maven and using spring-boot-starter-parent (which configures Maven’s
war plugin for you) all you need to do is to modify pom.xml to
change the packaging to war:
<packaging>war</packaging>
EDIT
Do you have spring-boot-starter-web and spring-boot-starter-tomcat dependencies ?
To build a war file that is both executable and deployable into an
external container you need to mark the embedded container
dependencies as “provided”, e.g:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<!-- ... -->
<packaging>war</packaging>
<!-- ... -->
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-tomcat</artifactId>
<scope>provided</scope>
</dependency>
<!-- ... -->
</dependencies>
</project>
recently I'm trying to call a REST-service, published over Apache-CXF. Everything is embedded in an Equinox 3.11.0 with Jetty 9.4.1.
Container starts Jetty, configured for HTTP/2:
[main] INFO org.eclipse.jetty.server.Server - jetty-9.4.1.v20170120
[main] INFO org.eclipse.jetty.server.handler.ContextHandler - Started o.e.j.s.ServletContextHandler#6f89f665{/,jar:file:///C:/workspaces/com.ebase.eox/cnf/generated/fw/org.eclipse.osgi/12/0/bundleFile!/,AVAILABLE}
[main] INFO org.eclipse.jetty.server.AbstractConnector - Started ServerConnector#187eb9a8{HTTP/1.1,[http/1.1, h2c]}{0.0.0.0:8080}
[main] INFO org.eclipse.jetty.util.ssl.SslContextFactory - x509=X509#1517f633(jetty,h=[],w=[]) for SslContextFactory#4fe01803(bundleentry://12.fwk1276261147/jettyhome/etc/keystore,bundleentry://12.fwk1276261147/jettyhome/etc/keystore)
[main] INFO org.eclipse.jetty.server.AbstractConnector - Started ServerConnector#3015db78{SSL,[ssl, alpn, h2, http/1.1]}{0.0.0.0:8443}
Wenn i call a REST-Service over https/ssl i get an error:
Caused by: java.lang.ArrayIndexOutOfBoundsException: 1
at org.eclipse.jetty.http.PreEncodedHttpField.putTo(PreEncodedHttpField.java:119)
at org.eclipse.jetty.http2.hpack.HpackEncoder.encode(HpackEncoder.java:289)
at org.eclipse.jetty.http2.hpack.HpackEncoder.encode(HpackEncoder.java:179)
at org.eclipse.jetty.http2.generator.HeadersGenerator.generateHeaders(HeadersGenerator.java:72)
at org.eclipse.jetty.http2.generator.HeadersGenerator.generate(HeadersGenerator.java:56)
at org.eclipse.jetty.http2.generator.Generator.control(Generator.java:80)
at org.eclipse.jetty.http2.HTTP2Session$ControlEntry.generate(HTTP2Session.java:1143)
at org.eclipse.jetty.http2.HTTP2Flusher.process(HTTP2Flusher.java:168)
at org.eclipse.jetty.util.IteratingCallback.processing(IteratingCallback.java:241)
at org.eclipse.jetty.util.IteratingCallback.iterate(IteratingCallback.java:224)
at org.eclipse.jetty.http2.HTTP2Session.frame(HTTP2Session.java:676)
at org.eclipse.jetty.http2.HTTP2Session.frames(HTTP2Session.java:648)
at org.eclipse.jetty.http2.HTTP2Stream.headers(HTTP2Stream.java:91)
at org.eclipse.jetty.http2.server.HttpTransportOverHTTP2.commit(HttpTransportOverHTTP2.java:185)
at org.eclipse.jetty.http2.server.HttpTransportOverHTTP2.send(HttpTransportOverHTTP2.java:111)
at org.eclipse.jetty.server.HttpChannel.sendResponse(HttpChannel.java:687)
at org.eclipse.jetty.server.HttpChannel.write(HttpChannel.java:743)
at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:234)
at org.eclipse.jetty.server.HttpOutput.write(HttpOutput.java:218)
... 61 more
It seems (i have proven it), that PreEncodedHttpField cannot load all Implementations of HttpFieldPreEncoder over ServiceLoader.
Iterator<HttpFieldPreEncoder> iter = ServiceLoader.load(HttpFieldPreEncoder.class,PreEncodedHttpField.class.getClassLoader()).iterator();
ServiceLoader sees only a Http1FieldPreEncoder, which is in the same Bundle as PreEncodedHttpField, so observable for a BundleClassLoader. This Implementation is registerd under Index "0".
In my case, it should be used an HTTP/2 Implementation HpackFieldPreEncoder, which bundle in http2-hpack, but not visible for ServiceLoader
Is there any workaround to make this second implementation visible for PreEncodedHttpField to avoid my Exception?
Thank you