My goal
I want to access Namenode UI through KnoxSSO and browse hdfs files without Ambari.
What have I already done
I've already set up Security cluster and HDFS HA with QJM.
Apache Knox uses Pac4J provider to authenticate users with Openid Connect.
There are 3 instances of Apache Knox beyond Nginx reverse proxy.
I've also have several services working fine with KnoxSSO:
Resourcemanager UI
Jobhistory UI
WebHDFS
Apache Zeppelin
Spark History Server
The problem
Despite of all other services are accessable by their UI through knox, I can't achieve it with Namenode IU (HDFSUI).
I folowed instructions by official documentation but when I try to follow hdfsui link it's cause probably infinite loop redirecting and show: HTTP ERROR 500 java.io.IOException: Service connectivity error.
In the gateway.log file I have numerous exceptions, like these:
2022-03-19 18:15:35,509 WARN knox.gateway (IdentityAsserterHttpServletRequestWrapper.java:scrubOfExistingPrincipalParams(202)) - Possible identity spoofing attempt - impersonation parameter removed: doAs
2022-03-19 18:15:35,521 WARN knox.gateway (DefaultDispatch.java:executeOutboundRequest(183)) - Connection exception dispatching request: https://my.hadoop.domain:443/gateway/default/hdfs?doAs=username java.net.SocketTimeoutException: Read timed out
java.net.SocketTimeoutException: Read timed out
at java.base/java.net.SocketInputStream.socketRead0(Native Method)
at java.base/java.net.SocketInputStream.socketRead(SocketInputStream.java:115)
at java.base/java.net.SocketInputStream.read(SocketInputStream.java:168)
at java.base/java.net.SocketInputStream.read(SocketInputStream.java:140)
at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:478)
at java.base/sun.security.ssl.SSLSocketInputRecord.readHeader(SSLSocketInputRecord.java:472)
at java.base/sun.security.ssl.SSLSocketInputRecord.bytesInCompletePacket(SSLSocketInputRecord.java:70)
at java.base/sun.security.ssl.SSLSocketImpl.readApplicationRecord(SSLSocketImpl.java:1374)
at java.base/sun.security.ssl.SSLSocketImpl$AppInputStream.read(SSLSocketImpl.java:985)
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at org.apache.knox.gateway.dispatch.DefaultDispatch.executeOutboundRequest(DefaultDispatch.java:166)
at org.apache.knox.gateway.dispatch.DefaultDispatch.executeRequest(DefaultDispatch.java:152)
at org.apache.knox.gateway.dispatch.DefaultDispatch.executeRequestWrapper(DefaultDispatch.java:135)
at org.apache.knox.gateway.dispatch.DefaultDispatch.doGet(DefaultDispatch.java:300)
at org.apache.knox.gateway.dispatch.GatewayDispatchFilter$GetAdapter.doMethod(GatewayDispatchFilter.java:174)
at org.apache.knox.gateway.dispatch.GatewayDispatchFilter.doFilter(GatewayDispatchFilter.java:125)
at org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:58)
at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:348)
at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:262)
at org.apache.knox.gateway.identityasserter.common.filter.AbstractIdentityAssertionFilter.doFilterInternal(AbstractIdentityAssertionFilter.java:193)
at org.apache.knox.gateway.identityasserter.common.filter.AbstractIdentityAssertionFilter.continueChainAsPrincipal(AbstractIdentityAssertionFilter.java:149)
at org.apache.knox.gateway.identityasserter.common.filter.CommonIdentityAssertionFilter.doFilter(CommonIdentityAssertionFilter.java:94)
at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:348)
at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:262)
at org.apache.knox.gateway.filter.rewrite.api.UrlRewriteServletFilter.doFilter(UrlRewriteServletFilter.java:57)
at org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:58)
at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:348)
at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:262)
at org.apache.knox.gateway.provider.federation.jwt.filter.AbstractJWTFilter$1.run(AbstractJWTFilter.java:249)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/javax.security.auth.Subject.doAs(Subject.java:423)
at org.apache.knox.gateway.provider.federation.jwt.filter.AbstractJWTFilter.continueWithEstablishedSecurityContext(AbstractJWTFilter.java:244)
at org.apache.knox.gateway.provider.federation.jwt.filter.SSOCookieFederationFilter.doFilter(SSOCookieFederationFilter.java:164)
at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:348)
at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:262)
at org.apache.knox.gateway.filter.XForwardedHeaderFilter.doFilter(XForwardedHeaderFilter.java:50)
at org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:58)
at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:348)
at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:262)
at org.apache.knox.gateway.GatewayFilter.doFilter(GatewayFilter.java:166)
at org.apache.knox.gateway.GatewayFilter.doFilter(GatewayFilter.java:93)
at org.apache.knox.gateway.GatewayServlet.service(GatewayServlet.java:135)
at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1443)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:791)
at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1626)
at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:228)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1612)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1582)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:234)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.apache.knox.gateway.trace.TraceHandler.handle(TraceHandler.java:51)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.apache.knox.gateway.filter.CorrelationHandler.handle(CorrelationHandler.java:41)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.apache.knox.gateway.filter.PortMappingHelperHandler.handle(PortMappingHelperHandler.java:106)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.websocket.server.WebSocketHandler.handle(WebSocketHandler.java:115)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:516)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540)
at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395)
at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:773)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:905)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-03-19 18:15:35,522 ERROR knox.gateway (AbstractGatewayFilter.java:doFilter(60)) - Failed to execute filter: java.io.IOException: Service connectivity error.
2022-03-19 18:15:35,522 ERROR knox.gateway (AbstractGatewayFilter.java:doFilter(60)) - Failed to execute filter: java.io.IOException: Service connectivity error.
2022-03-19 18:15:35,522 ERROR knox.gateway (AbstractGatewayFilter.java:doFilter(60)) - Failed to execute filter: java.io.IOException: Service connectivity error.
2022-03-19 18:15:35,523 ERROR knox.gateway (GatewayFilter.java:doFilter(168)) - Gateway processing failed: java.io.IOException: Service connectivity error.
java.io.IOException: Service connectivity error.
at org.apache.knox.gateway.dispatch.DefaultDispatch.executeOutboundRequest(DefaultDispatch.java:184)
at org.apache.knox.gateway.dispatch.DefaultDispatch.executeRequest(DefaultDispatch.java:152)
at org.apache.knox.gateway.dispatch.DefaultDispatch.executeRequestWrapper(DefaultDispatch.java:135)
at org.apache.knox.gateway.dispatch.DefaultDispatch.doGet(DefaultDispatch.java:300)
at org.apache.knox.gateway.dispatch.GatewayDispatchFilter$GetAdapter.doMethod(GatewayDispatchFilter.java:174)
at org.apache.knox.gateway.dispatch.GatewayDispatchFilter.doFilter(GatewayDispatchFilter.java:125)
at org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:58)
at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:348)
at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:262)
at org.apache.knox.gateway.identityasserter.common.filter.AbstractIdentityAssertionFilter.doFilterInternal(AbstractIdentityAssertionFilter.java:193)
at org.apache.knox.gateway.identityasserter.common.filter.AbstractIdentityAssertionFilter.continueChainAsPrincipal(AbstractIdentityAssertionFilter.java:149)
at org.apache.knox.gateway.identityasserter.common.filter.CommonIdentityAssertionFilter.doFilter(CommonIdentityAssertionFilter.java:94)
at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:348)
at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:262)
at org.apache.knox.gateway.filter.rewrite.api.UrlRewriteServletFilter.doFilter(UrlRewriteServletFilter.java:57)
at org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:58)
at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:348)
at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:348)
at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:262)
at org.apache.knox.gateway.filter.XForwardedHeaderFilter.doFilter(XForwardedHeaderFilter.java:50)
at org.apache.knox.gateway.filter.AbstractGatewayFilter.doFilter(AbstractGatewayFilter.java:58)
at org.apache.knox.gateway.GatewayFilter$Holder.doFilter(GatewayFilter.java:348)
at org.apache.knox.gateway.GatewayFilter$Chain.doFilter(GatewayFilter.java:262)
at org.apache.knox.gateway.GatewayFilter.doFilter(GatewayFilter.java:166)
at org.apache.knox.gateway.GatewayFilter.doFilter(GatewayFilter.java:93)
at org.apache.knox.gateway.GatewayServlet.service(GatewayServlet.java:135)
at org.eclipse.jetty.servlet.ServletHolder$NotAsync.service(ServletHolder.java:1443)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:791)
at org.eclipse.jetty.servlet.ServletHandler$ChainEnd.doFilter(ServletHandler.java:1626)
at org.eclipse.jetty.websocket.server.WebSocketUpgradeFilter.doFilter(WebSocketUpgradeFilter.java:228)
at org.eclipse.jetty.servlet.FilterHolder.doFilter(FilterHolder.java:193)
at org.eclipse.jetty.servlet.ServletHandler$Chain.doFilter(ServletHandler.java:1601)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:548)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:602)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:235)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1612)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1434)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:501)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1582)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1349)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:234)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.apache.knox.gateway.trace.TraceHandler.handle(TraceHandler.java:51)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.apache.knox.gateway.filter.CorrelationHandler.handle(CorrelationHandler.java:41)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.apache.knox.gateway.filter.PortMappingHelperHandler.handle(PortMappingHelperHandler.java:106)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.websocket.server.WebSocketHandler.handle(WebSocketHandler.java:115)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:146)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:516)
at org.eclipse.jetty.server.HttpChannel.lambda$handle$1(HttpChannel.java:383)
at org.eclipse.jetty.server.HttpChannel.dispatch(HttpChannel.java:556)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:375)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:273)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ssl.SslConnection$DecryptedEndPoint.onFillable(SslConnection.java:540)
at org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:395)
at org.eclipse.jetty.io.ssl.SslConnection$2.succeeded(SslConnection.java:161)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:105)
at org.eclipse.jetty.io.ChannelEndPoint$1.run(ChannelEndPoint.java:104)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:336)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:313)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171)
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129)
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:375)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:773)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:905)
at java.base/java.lang.Thread.run(Thread.java:829)
In debug logs I saw some strange headers like that
X-Forwarded-Context: /gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default/gateway/default
Could anyone know, what's going wrong?
Try to add the following line for HDFSUI role in your topology file:
<version>2.7.0</version>
It should be look like this:
<service>
<role>HDFSUI</role>
<version>2.7.0</version>
<url>http://NAMENODE:50070</url>
</service>
https://issues.apache.org/jira/browse/KNOX-1644
Related
I'm trying to setup a Debezium source connector in docker which uses ContentBasedRouting and I'm following the official doc.
I have all the following dependencies in /kafka/connect/debezium-connector-mysql:
antlr4-runtime-4.7.2.jar groovy-3.0.9.jar
debezium-connector-mysql-1.0.3.Final.jar groovy-json-3.0.9.jar
debezium-core-1.0.3.Final.jar groovy-jsr223-3.0.9.jar
debezium-ddl-parser-1.0.3.Final.jar mysql-binlog-connector-java-0.19.1.jar
debezium-scripting-1.7.2.Final.jar mysql-connector-java-8.0.16.jar
At the moment I'm trying to add the connector with the following transforms:
"transforms": "route",
"transforms.route.type": "io.debezium.transforms.ContentBasedRouter",
"transforms.route.language": "jsr223.groovy",
"transforms.route.topic.expression": "value.after.name == 'x' ? 'topic1' : null",
"transforms.route.null.handling.mode": "drop"
I get http status 500 and the log:
javax.servlet.ServletException: org.glassfish.jersey.server.ContainerException: java.lang.NoClassDefFoundError: io/debezium/DebeziumException
at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:408)
at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:346)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:365)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:318)
at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:205)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:852)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:544)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1581)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:233)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1307)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:188)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:482)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1549)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1204)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:221)
at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:173)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:127)
at org.eclipse.jetty.server.Server.handle(Server.java:494)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:374)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:268)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:311)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:782)
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:918)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.glassfish.jersey.server.ContainerException: java.lang.NoClassDefFoundError: io/debezium/DebeziumException
at org.glassfish.jersey.servlet.internal.ResponseWriter.rethrow(ResponseWriter.java:254)
at org.glassfish.jersey.servlet.internal.ResponseWriter.failure(ResponseWriter.java:236)
at org.glassfish.jersey.server.ServerRuntime$Responder.process(ServerRuntime.java:436)
at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:261)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248)
at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244)
at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
at org.glassfish.jersey.internal.Errors.process(Errors.java:244)
at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265)
at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:232)
at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:679)
at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:392)
... 28 more
Caused by: java.lang.NoClassDefFoundError: io/debezium/DebeziumException
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:398)
at org.apache.kafka.common.config.ConfigDef.parseType(ConfigDef.java:719)
at org.apache.kafka.connect.runtime.ConnectorConfig.enrich(ConnectorConfig.java:308)
at org.apache.kafka.connect.runtime.AbstractHerder.validateConnectorConfig(AbstractHerder.java:302)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder$6.call(DistributedHerder.java:745)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder$6.call(DistributedHerder.java:742)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.tick(DistributedHerder.java:342)
at org.apache.kafka.connect.runtime.distributed.DistributedHerder.run(DistributedHerder.java:282)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
... 1 more
Caused by: java.lang.ClassNotFoundException: io.debezium.DebeziumException
at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:588)
at org.apache.kafka.connect.runtime.isolation.PluginClassLoader.loadClass(PluginClassLoader.java:104)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
... 14 more
Tried adding a connector without the transform (everything else unchanged) and it worked.
What am I doing wrong?
Ok so I solved it by starting from clean environment because I've probably played with a mix of too many versions. Made sure I'm using Debezium 1.8 for everything and maybe the weirdest thing is that I had to put all the Debezium jars on the connect container in /kafka/libs because it didn't work in /kafka/connect/debezium-connector-mysql for some reason...
I submitted the spark program to run on YARN.
The program runs normally and the yarnUI page is also displayed normally but some error logs showed in the resource manager log.
2020-07-01 10:05:43,497 ERROR webapp.Dispatcher (Dispatcher.java:service(171)) - error handling URI: /cluster
org.eclipse.jetty.io.RuntimeIOException: org.eclipse.jetty.io.EofException
at org.eclipse.jetty.server.ResponseWriter.isOpen(ResponseWriter.java:133)
at org.eclipse.jetty.server.ResponseWriter.println(ResponseWriter.java:314)
at org.apache.hadoop.yarn.webapp.hamlet2.HamletImpl$EImp._p(HamletImpl.java:110)
at org.apache.hadoop.yarn.webapp.hamlet2.Hamlet$SCRIPT.__(Hamlet.java:457)
at org.apache.hadoop.yarn.server.resourcemanager.webapp.RMAppsBlock.renderData(RMAppsBlock.java:194)
at org.apache.hadoop.yarn.server.webapp.AppsBlock.render(AppsBlock.java:145)
at org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:69)
at org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:79)
at org.apache.hadoop.yarn.webapp.View.render(View.java:243)
at org.apache.hadoop.yarn.webapp.view.HtmlBlock$Block.subView(HtmlBlock.java:43)
at org.apache.hadoop.yarn.webapp.hamlet2.Hamlet.__(Hamlet.java:30354)
at org.apache.hadoop.yarn.server.resourcemanager.webapp.AppsBlockWithMetrics.render(AppsBlockWithMetrics.java:29)
at org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:69)
at org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:79)
at org.apache.hadoop.yarn.webapp.View.render(View.java:243)
at org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:49)
at org.apache.hadoop.yarn.webapp.hamlet2.HamletImpl$EImp._v(HamletImpl.java:117)
at org.apache.hadoop.yarn.webapp.hamlet2.Hamlet$TD.__(Hamlet.java:848)
at org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:71)
at org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:82)
at org.apache.hadoop.yarn.webapp.Dispatcher.render(Dispatcher.java:206)
at org.apache.hadoop.yarn.webapp.Dispatcher.service(Dispatcher.java:165)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:287)
at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:277)
at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:182)
at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85)
at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:941)
at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:875)
at org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebAppFilter.doFilter(RMWebAppFilter.java:178)
at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:829)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:119)
at com.google.inject.servlet.GuiceFilter$1.call(GuiceFilter.java:133)
at com.google.inject.servlet.GuiceFilter$1.call(GuiceFilter.java:130)
at com.google.inject.servlet.GuiceFilter$Context.call(GuiceFilter.java:203)
at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:130)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.apache.hadoop.security.http.XFrameOptionsFilter.doFilter(XFrameOptionsFilter.java:57)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:644)
at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:592)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.apache.hadoop.security.http.CrossOriginFilter.doFilter(CrossOriginFilter.java:98)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1604)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:539)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.eclipse.jetty.io.EofException
at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:199)
at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:420)
at org.eclipse.jetty.io.WriteFlusher.completeWrite(WriteFlusher.java:375)
at org.eclipse.jetty.io.SelectChannelEndPoint$3.run(SelectChannelEndPoint.java:107)
at org.eclipse.jetty.io.SelectChannelEndPoint.onSelected(SelectChannelEndPoint.java:193)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.processSelected(ManagedSelector.java:283)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:181)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
... 5 more
Caused by: java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
at sun.nio.ch.IOUtil.write(IOUtil.java:51)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:177)
... 12 more
2020-07-01 10:05:43,497 ERROR webapp.Dispatcher (Dispatcher.java:service(171)) - error handling URI: /cluster
org.eclipse.jetty.io.RuntimeIOException: org.eclipse.jetty.io.EofException
at org.eclipse.jetty.server.ResponseWriter.isOpen(ResponseWriter.java:133)
at org.eclipse.jetty.server.ResponseWriter.println(ResponseWriter.java:314)
at org.apache.hadoop.yarn.webapp.hamlet2.HamletImpl$EImp._p(HamletImpl.java:110)
at org.apache.hadoop.yarn.webapp.hamlet2.Hamlet$SCRIPT.__(Hamlet.java:457)
at org.apache.hadoop.yarn.server.resourcemanager.webapp.RMAppsBlock.renderData(RMAppsBlock.java:194)
at org.apache.hadoop.yarn.server.webapp.AppsBlock.render(AppsBlock.java:145)
at org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:69)
at org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:79)
at org.apache.hadoop.yarn.webapp.View.render(View.java:243)
at org.apache.hadoop.yarn.webapp.view.HtmlBlock$Block.subView(HtmlBlock.java:43)
at org.apache.hadoop.yarn.webapp.hamlet2.Hamlet.__(Hamlet.java:30354)
at org.apache.hadoop.yarn.server.resourcemanager.webapp.AppsBlockWithMetrics.render(AppsBlockWithMetrics.java:29)
at org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:69)
at org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:79)
at org.apache.hadoop.yarn.webapp.View.render(View.java:243)
at org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:49)
at org.apache.hadoop.yarn.webapp.hamlet2.HamletImpl$EImp._v(HamletImpl.java:117)
at org.apache.hadoop.yarn.webapp.hamlet2.Hamlet$TD.__(Hamlet.java:848)
at org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:71)
at org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:82)
at org.apache.hadoop.yarn.webapp.Dispatcher.render(Dispatcher.java:206)
at org.apache.hadoop.yarn.webapp.Dispatcher.service(Dispatcher.java:165)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:287)
at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:277)
at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:182)
at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85)
at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:941)
at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:875)
at org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebAppFilter.doFilter(RMWebAppFilter.java:178)
at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:829)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:119)
at com.google.inject.servlet.GuiceFilter$1.call(GuiceFilter.java:133)
at com.google.inject.servlet.GuiceFilter$1.call(GuiceFilter.java:130)
at com.google.inject.servlet.GuiceFilter$Context.call(GuiceFilter.java:203)
at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:130)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.apache.hadoop.security.http.XFrameOptionsFilter.doFilter(XFrameOptionsFilter.java:57)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:644)
at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:592)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.apache.hadoop.security.http.CrossOriginFilter.doFilter(CrossOriginFilter.java:98)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1604)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:539)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.eclipse.jetty.io.EofException
at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:199)
at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:420)
at org.eclipse.jetty.io.WriteFlusher.completeWrite(WriteFlusher.java:375)
at org.eclipse.jetty.io.SelectChannelEndPoint$3.run(SelectChannelEndPoint.java:107)
at org.eclipse.jetty.io.SelectChannelEndPoint.onSelected(SelectChannelEndPoint.java:193)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.processSelected(ManagedSelector.java:283)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:181)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
... 5 more
Caused by: java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
at sun.nio.ch.IOUtil.write(IOUtil.java:51)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:177)
... 12 more
2020-07-01 10:05:43,497 ERROR webapp.Dispatcher (Dispatcher.java:service(171)) - error handling URI: /cluster
org.eclipse.jetty.io.RuntimeIOException: org.eclipse.jetty.io.EofException
at org.eclipse.jetty.server.ResponseWriter.isOpen(ResponseWriter.java:133)
at org.eclipse.jetty.server.ResponseWriter.println(ResponseWriter.java:314)
at org.apache.hadoop.yarn.webapp.hamlet2.HamletImpl$EImp._p(HamletImpl.java:110)
at org.apache.hadoop.yarn.webapp.hamlet2.Hamlet$SCRIPT.__(Hamlet.java:457)
at org.apache.hadoop.yarn.server.resourcemanager.webapp.RMAppsBlock.renderData(RMAppsBlock.java:194)
at org.apache.hadoop.yarn.server.webapp.AppsBlock.render(AppsBlock.java:145)
at org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:69)
at org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:79)
at org.apache.hadoop.yarn.webapp.View.render(View.java:243)
at org.apache.hadoop.yarn.webapp.view.HtmlBlock$Block.subView(HtmlBlock.java:43)
at org.apache.hadoop.yarn.webapp.hamlet2.Hamlet.__(Hamlet.java:30354)
at org.apache.hadoop.yarn.server.resourcemanager.webapp.AppsBlockWithMetrics.render(AppsBlockWithMetrics.java:29)
at org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:69)
at org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:79)
at org.apache.hadoop.yarn.webapp.View.render(View.java:243)
at org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:49)
at org.apache.hadoop.yarn.webapp.hamlet2.HamletImpl$EImp._v(HamletImpl.java:117)
at org.apache.hadoop.yarn.webapp.hamlet2.Hamlet$TD.__(Hamlet.java:848)
at org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:71)
at org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:82)
at org.apache.hadoop.yarn.webapp.Dispatcher.render(Dispatcher.java:206)
at org.apache.hadoop.yarn.webapp.Dispatcher.service(Dispatcher.java:165)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:287)
at com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:277)
at com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:182)
at com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85)
at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:941)
at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:875)
at org.apache.hadoop.yarn.server.resourcemanager.webapp.RMWebAppFilter.doFilter(RMWebAppFilter.java:178)
at com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:829)
at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:119)
at com.google.inject.servlet.GuiceFilter$1.call(GuiceFilter.java:133)
at com.google.inject.servlet.GuiceFilter$1.call(GuiceFilter.java:130)
at com.google.inject.servlet.GuiceFilter$Context.call(GuiceFilter.java:203)
at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:130)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.apache.hadoop.security.http.XFrameOptionsFilter.doFilter(XFrameOptionsFilter.java:57)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:644)
at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:592)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.apache.hadoop.security.http.CrossOriginFilter.doFilter(CrossOriginFilter.java:98)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.apache.hadoop.http.HttpServer2$QuotingInputFilter.doFilter(HttpServer2.java:1604)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
at org.eclipse.jetty.server.Server.handle(Server.java:539)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:333)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.eclipse.jetty.io.EofException
at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:199)
at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:420)
at org.eclipse.jetty.io.WriteFlusher.completeWrite(WriteFlusher.java:375)
at org.eclipse.jetty.io.SelectChannelEndPoint$3.run(SelectChannelEndPoint.java:107)
at org.eclipse.jetty.io.SelectChannelEndPoint.onSelected(SelectChannelEndPoint.java:193)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.processSelected(ManagedSelector.java:283)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(ManagedSelector.java:181)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:249)
... 5 more
Caused by: java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
at sun.nio.ch.IOUtil.write(IOUtil.java:51)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:177)
... 12 more
The version of Hadoop is 3.0.3.
Linux info is below:
LSB Version: :core-4.1-amd64:core-4.1-noarch:cxx-4.1-amd64:cxx-4.1-noarch:desktop-4.1-amd64:desktop-4.1-noarch:languages-4.1-amd64:languages-4.1-noarch:printing-4.1-amd64:printing-4.1-noarch
Distributor ID: RedHatEnterpriseServer
Description: Red Hat Enterprise Linux Server release 7.5 (Maipo)
Release: 7.5
Codename: Maipo .
I have tested calling the api test/cluster path, and the result can be returned normally.
Welcome to Stackoverflow.
Caused by: org.eclipse.jetty.io.EofException
at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:199)
at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:420)
at org.eclipse.jetty.io.WriteFlusher.completeWrite(WriteFlusher.java:375)
...
Caused by: java.io.IOException: Connection reset by peer
This means that the server attempted to flush out the remaining bytes to the client, but the client closed the connection before all of those bytes were sent.
This happens, fairly often with web browsers when the user navigates off the page, or closes the tab before the entire contents of the response have been send.
This happens even more often with clients that have flaky internet connections, such as mobile clients (such as when they switch towers, or go from mobile to wifi, or wifi to mobile, etc)
This also happens, with some regularity, when the client simply doesn't read the entire response and closes the connection. This often is a result of a client that isn't following the HTTP spec properly.
Example: If the client knows it is going to close the connection when it is done, it should have added the Connection: close header on the request.
The client reads the response up until it has what it wants, then closes the connection. (such as when reading JSON or XML it will read up until the JSON or XML is complete, but not any other data after it, even whitespace). This is a fact that surprises many developers using 3rd party parsing libraries. Good libraries have specific configuration options to keep reading until Soft EOF (eg: the end of the HTTP response on a persistent connection)
I am trying to build a Firestore connection from my Google App Engine web application. I have followed the tutorial on google's website (https://cloud.google.com/firestore/docs/quickstart-servers) and setup the necessary service key as well as set the environment variable GOOGLE_APPLICATION_CREDENTIALS to the path of the json file with the key. However, I keep getting a NullPointerException on the line when I call the build() ater setting my project id. Has anyone encountered this same problem and have fixed it before? I can't find any similar problems online. There is not many lines of code so it seems like the problem should be relatively simple.
This is the error itself:
Caused by:
java.lang.NullPointerException
at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:770)
at com.google.cloud.ServiceOptions$Builder.setCredentials(ServiceOptions.java:203)
at com.google.cloud.firestore.FirestoreOptions$Builder.build(FirestoreOptions.java:213)
at com.tellafirma.mappers.PierMapper.(PierMapper.java:63)
at com.tellafirma.controller.PierController.displayLogin(PierController.java:60)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:219)
at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:132)
at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:104)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandleMethod(RequestMappingHandlerAdapter.java:745)
at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:686)
at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:80)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:925)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:856)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:936)
at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:827)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:812)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:867)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1623)
at com.google.appengine.tools.development.ResponseRewriterFilter.doFilter(ResponseRewriterFilter.java:134)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
at com.google.appengine.tools.development.HeaderVerificationFilter.doFilter(HeaderVerificationFilter.java:34)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
at com.google.appengine.api.blobstore.dev.ServeBlobFilter.doFilter(ServeBlobFilter.java:63)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:48)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
at com.google.appengine.tools.development.jetty9.StaticFileFilter.doFilter(StaticFileFilter.java:123)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
at com.google.appengine.tools.development.DevAppServerModulesFilter.doDirectRequest(DevAppServerModulesFilter.java:366)
at com.google.appengine.tools.development.DevAppServerModulesFilter.doDirectModuleRequest(DevAppServerModulesFilter.java:349)
at com.google.appengine.tools.development.DevAppServerModulesFilter.doFilter(DevAppServerModulesFilter.java:116)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
at com.google.appengine.tools.development.DevAppServerRequestLogFilter.doFilter(DevAppServerRequestLogFilter.java:44)
at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1602)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1588)
at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1557)
at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
at com.google.appengine.tools.development.jetty9.DevAppEngineWebAppContext.doScope(DevAppEngineWebAppContext.java:94)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at com.google.appengine.tools.development.jetty9.JettyContainerService$ApiProxyHandler.handle(JettyContainerService.java:595)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
at org.eclipse.jetty.server.Server.handle(Server.java:502)
at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:364)
at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
at java.lang.Thread.run(Thread.java:748)
I have tried redoing the service key and setting the GOOGLE_APPLICATION_CREDENTIALS again through the command line. I have printed out the FirestoreOptions object and the FirestoreOptions.Builder object and those have memory addresses that are non-null. I separated each metthod call to try and see specificalyl where the error was happening and it is on the build() line. I also updated my cloud components to see if anything was out of date.
public PierMapper(String id)
{
projectId = id;
firestoreOptions = FirestoreOptions.getDefaultInstance();
System.out.println(firestoreOptions.toString());
firestoreBuilder = firestoreOptions.toBuilder();
System.out.println(firestoreBuilder.toString());
firestoreBuilder = firestoreBuilder.setProjectId(projectId);
System.out.println(firestoreBuilder.toString());
System.out.println(firestoreBuilder.build());
firestoreOptions = firestoreBuilder.build();
database = firestoreOptions.getService();
}
I need to get this connection working to Firestore so I can begin to read and write data from my webapp.
Try setting GOOGLE_APPLICATION_CREDENTIALS to the full path to your json file. I suspect your app isn't finding your key file.
You can also try using gcloud auth application-default login, as described here, instead of the env variable.
When I try to display the system info for my SonarQube 5.6 server I get an exception in the logs on the page. Any idea what could be causing this?
This question (Getting an error as "Could not initialize class sun.net.www.protocol.http.HttpURLConnection" in ZK Framework) implies that the jar is missing from the jdk. Is that possible? I'm using java-1.8.0-openjdk-1.8.0.151-1.b12.el6_9.x86_64. Has the HttpUrlConnection implementation been changed?
2018.04.23 12:12:15 ERROR web[rails]
Java::JavaLang::NoClassDefFoundError (Could not initialize class sun.net.www.protocol.http.HttpURLConnection):
sun.net.www.protocol.http.Handler.openConnection(Handler.java:62)
sun.net.www.protocol.http.Handler.openConnection(Handler.java:57)
java.net.URL.openConnection(URL.java:979)
org.apache.commons.io.IOUtils.toByteArray(IOUtils.java:633)
org.apache.commons.io.IOUtils.toByteArray(IOUtils.java:617)
org.sonar.server.platform.monitoring.ProcessSystemInfoClient.connect(ProcessSystemInfoClient.java:53)
org.sonar.server.platform.ws.InfoAction.writeJson(InfoAction.java:81)
org.sonar.server.platform.ws.InfoAction.handle(InfoAction.java:66)
org.sonar.server.ws.WebServiceEngine.execute(WebServiceEngine.java:107)
sun.reflect.GeneratedMethodAccessor136.invoke(Unknown Source)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:425)
org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:292)
org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:44)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:70)
rubyjit.Api::JavaWsController$$index_E9F115BCB78D2E226E3015CDA52C1912485D30D01792011778.__file__(app/controllers/api/java_ws_controller.rb:34)
rubyjit.Api::JavaWsController$$index_E9F115BCB78D2E226E3015CDA52C1912485D30D01792011778.__file__(app/controllers/api/java_ws_controller.rb)
org.jruby.internal.runtime.methods.JittedMethod.call(JittedMethod.java:161)
org.jruby.RubyClass.finvoke(RubyClass.java:527)
org.jruby.RubyBasicObject.send(RubyBasicObject.java:2590)
org.jruby.RubyKernel.send(RubyKernel.java:2223)
org.jruby.RubyKernel$INVOKER$s$send.call(RubyKernel$INVOKER$s$send.gen)
org.jruby.internal.runtime.methods.JavaMethod$JavaMethodOneOrNBlock.call(JavaMethod.java:350)
org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:326)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:170)
rubyjit.ActionController::Base$$perform_action_DA0FC0A901DD1257EC969DB0C482DD8514B8F04B1792011778.__file__(gems/gems/actionpack-2.3.15/lib/action_controller/base.rb:1333)
rubyjit.ActionController::Base$$perform_action_DA0FC0A901DD1257EC969DB0C482DD8514B8F04B1792011778.__file__(gems/gems/actionpack-2.3.15/lib/action_controller/base.rb)
org.jruby.internal.runtime.methods.JittedMethod.call(JittedMethod.java:141)
org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:185)
org.jruby.internal.runtime.methods.AliasMethod.call(AliasMethod.java:56)
org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:306)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:136)
rubyjit.ActionController::Filters::InstanceMethods$$call_filters_A7E1D3FFBF56E3CB7C0EBA850688505249A05AE91792011778.__file__(gems/gems/actionpack-2.3.15/lib/action_controller/filters.rb:617)
rubyjit.ActionController::Filters::InstanceMethods$$call_filters_A7E1D3FFBF56E3CB7C0EBA850688505249A05AE91792011778.__file__(gems/gems/actionpack-2.3.15/lib/action_controller/filters.rb)
org.jruby.internal.runtime.methods.JittedMethod.call(JittedMethod.java:261)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:236)
rubyjit.ActionController::Filters::InstanceMethods$$perform_action_with_filters_4C7C67B4A5564FF5DC76344AF2372660F608839C1792011778.__file__(gems/gems/actionpack-2.3.15/lib/action_controller/filters.rb:610)
rubyjit.ActionController::Filters::InstanceMethods$$perform_action_with_filters_4C7C67B4A5564FF5DC76344AF2372660F608839C1792011778.__file__(gems/gems/actionpack-2.3.15/lib/action_controller/filters.rb)
org.jruby.internal.runtime.methods.JittedMethod.call(JittedMethod.java:141)
org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:185)
org.jruby.internal.runtime.methods.AliasMethod.call(AliasMethod.java:56)
org.jruby.internal.runtime.methods.AliasMethod.call(AliasMethod.java:56)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:134)
rubyjit.ActionController::Benchmarking$$perform_action_with_benchmark_303676EADF39D1B66CBA6FF2131B0D6C74E043451792011778.block_0$RUBY$__file__(gems/gems/actionpack-2.3.15/lib/action_controller/benchmarking.rb:68)
rubyjit$ActionController::Benchmarking$$perform_action_with_benchmark_303676EADF39D1B66CBA6FF2131B0D6C74E043451792011778$block_0$RUBY$__file__.call(rubyjit$ActionController::Benchmarking$$perform_action_with_benchmark_303676EADF39D1B66CBA6FF2131B0D6C74E043451792011778$block_0$RUBY$__file__)
org.jruby.runtime.CompiledBlock.yield(CompiledBlock.java:112)
org.jruby.runtime.CompiledBlock.yield(CompiledBlock.java:95)
org.jruby.runtime.CompiledBlock.yieldSpecific(CompiledBlock.java:75)
org.jruby.runtime.Block.yieldSpecific(Block.java:111)
When I unfortunately need to restart my Oracle database (after a failure, or for any reason). My persistence unit cannot reconnect to the database (after it is up again).
Here is the error log :
[#|2012-03-28T16:59:01.384+0200|WARNING|glassfish3.1.1|org.eclipse.persistence.session.file:/home/distriapps/glassfish3/glassfish/domains/distriapps/applications/SiteApplicationUpdater-Central/SAU-CommonFacade-ejb_jar/_SiteApplicationUpdater_PU|_ThreadID=40;_ThreadName=Thread-2;|
Local Exception Stack:
Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.3.0.v20110604-r9504): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: java.sql.SQLRecoverableException: Closed Connection
Error Code: 17008
Call: SELECT t0.ID, t0.ADDRESS, t0.CITY, t0.COUNTRY, t0.DESCRIPTION, t0.EXHIBITORNAME, t0.GLASSFISHVERSION, t0.LASTSYNCTIMESTAMP, t0.LASTUPDATETIMESTAMP, t0.MACHINENAME, t0.OSARCH, t0.OSNAME, t0.OSVERSION, t0.SITEAPPLICATIONMACHINEID, t0.ZIPCODE FROM SAU_REPORTS t0 WHERE ((SELECT COUNT(t1.ID) FROM SAU_ISLINKS t1 WHERE (t1.SiteApplicationReport_id = t0.ID)) = ?)
bind => [1 parameter bound]
Query: ReadAllQuery(referenceClass=SiteApplicationReport sql="SELECT t0.ID, t0.ADDRESS, t0.CITY, t0.COUNTRY, t0.DESCRIPTION, t0.EXHIBITORNAME, t0.GLASSFISHVERSION, t0.LASTSYNCTIMESTAMP, t0.LASTUPDATETIMESTAMP, t0.MACHINENAME, t0.OSARCH, t0.OSNAME, t0.OSVERSION, t0.SITEAPPLICATIONMACHINEID, t0.ZIPCODE FROM SAU_REPORTS t0 WHERE ((SELECT COUNT(t1.ID) FROM SAU_ISLINKS t1 WHERE (t1.SiteApplicationReport_id = t0.ID)) = ?)")
at org.eclipse.persistence.exceptions.DatabaseException.sqlException(DatabaseException.java:333)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.processExceptionForCommError(DatabaseAccessor.java:1494)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.basicExecuteCall(DatabaseAccessor.java:636)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeCall(DatabaseAccessor.java:535)
at org.eclipse.persistence.internal.sessions.AbstractSession.basicExecuteCall(AbstractSession.java:1702)
at org.eclipse.persistence.sessions.server.ServerSession.executeCall(ServerSession.java:566)
at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:207)
at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:193)
at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeSelectCall(DatasourceCallQueryMechanism.java:264)
at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.selectAllRows(DatasourceCallQueryMechanism.java:646)
at org.eclipse.persistence.internal.queries.ExpressionQueryMechanism.selectAllRowsFromTable(ExpressionQueryMechanism.java:2592)
at org.eclipse.persistence.internal.queries.ExpressionQueryMechanism.selectAllRows(ExpressionQueryMechanism.java:2551)
at org.eclipse.persistence.queries.ReadAllQuery.executeObjectLevelReadQuery(ReadAllQuery.java:418)
at org.eclipse.persistence.queries.ObjectLevelReadQuery.executeDatabaseQuery(ObjectLevelReadQuery.java:1097)
at org.eclipse.persistence.queries.DatabaseQuery.execute(DatabaseQuery.java:829)
at org.eclipse.persistence.queries.ObjectLevelReadQuery.execute(ObjectLevelReadQuery.java:1056)
at org.eclipse.persistence.queries.ReadAllQuery.execute(ReadAllQuery.java:390)
at org.eclipse.persistence.queries.ObjectLevelReadQuery.executeInUnitOfWork(ObjectLevelReadQuery.java:1144)
at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.internalExecuteQuery(UnitOfWorkImpl.java:2863)
at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1501)
at org.eclipse.persistence.internal.sessions.AbstractSession.retryQuery(AbstractSession.java:1571)
at org.eclipse.persistence.sessions.server.ClientSession.retryQuery(ClientSession.java:593)
at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.retryQuery(UnitOfWorkImpl.java:5453)
at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1537)
at org.eclipse.persistence.internal.sessions.AbstractSession.retryQuery(AbstractSession.java:1571)
at org.eclipse.persistence.sessions.server.ClientSession.retryQuery(ClientSession.java:593)
at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.retryQuery(UnitOfWorkImpl.java:5453)
at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1537)
at org.eclipse.persistence.internal.sessions.AbstractSession.retryQuery(AbstractSession.java:1571)
at org.eclipse.persistence.sessions.server.ClientSession.retryQuery(ClientSession.java:593)
at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.retryQuery(UnitOfWorkImpl.java:5453)
at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1537)
at org.eclipse.persistence.internal.sessions.AbstractSession.retryQuery(AbstractSession.java:1571)
at org.eclipse.persistence.sessions.server.ClientSession.retryQuery(ClientSession.java:593)
at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.retryQuery(UnitOfWorkImpl.java:5453)
at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1537)
at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1483)
at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1457)
at org.eclipse.persistence.internal.jpa.EJBQueryImpl.executeReadQuery(EJBQueryImpl.java:485)
at org.eclipse.persistence.internal.jpa.EJBQueryImpl.getResultList(EJBQueryImpl.java:742)
at com.xdcinema.siteapplicationupdater.commonfacade.SiteApplicationReportFacade.findUnlinkedSite(SiteApplicationReportFacade.java:275)
at sun.reflect.GeneratedMethodAccessor2440.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.glassfish.ejb.security.application.EJBSecurityManager.runMethod(EJBSecurityManager.java:1052)
at org.glassfish.ejb.security.application.EJBSecurityManager.invoke(EJBSecurityManager.java:1124)
at com.sun.ejb.containers.BaseContainer.invokeBeanMethod(BaseContainer.java:5366)
at com.sun.ejb.EjbInvocation.invokeBeanMethod(EjbInvocation.java:619)
at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:800)
at com.sun.ejb.EjbInvocation.proceed(EjbInvocation.java:571)
at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.doAround(SystemInterceptorProxy.java:162)
at com.sun.ejb.containers.interceptors.SystemInterceptorProxy.aroundInvoke(SystemInterceptorProxy.java:144)
at sun.reflect.GeneratedMethodAccessor164.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.sun.ejb.containers.interceptors.AroundInvokeInterceptor.intercept(InterceptorManager.java:861)
at com.sun.ejb.containers.interceptors.AroundInvokeChainImpl.invokeNext(InterceptorManager.java:800)
at com.sun.ejb.containers.interceptors.InterceptorManager.intercept(InterceptorManager.java:370)
at com.sun.ejb.containers.BaseContainer.__intercept(BaseContainer.java:5338)
at com.sun.ejb.containers.BaseContainer.intercept(BaseContainer.java:5326)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandler.invoke(EJBLocalObjectInvocationHandler.java:214)
at com.sun.ejb.containers.EJBLocalObjectInvocationHandlerDelegate.invoke(EJBLocalObjectInvocationHandlerDelegate.java:88)
at $Proxy376.findUnlinkedSite(Unknown Source)
at com.xdcinema.siteapplicationupdater.centralserver.jsf.SiteApplicationReportController.getItems(SiteApplicationReportController.java:598)
at sun.reflect.GeneratedMethodAccessor366.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at javax.el.BeanELResolver.getValue(BeanELResolver.java:302)
at com.sun.faces.el.DemuxCompositeELResolver._getValue(DemuxCompositeELResolver.java:176)
at com.sun.faces.el.DemuxCompositeELResolver.getValue(DemuxCompositeELResolver.java:203)
at com.sun.el.parser.AstValue.getValue(AstValue.java:116)
at com.sun.el.parser.AstValue.getValue(AstValue.java:163)
at com.sun.el.ValueExpressionImpl.getValue(ValueExpressionImpl.java:219)
at com.sun.faces.facelets.el.TagValueExpression.getValue(TagValueExpression.java:109)
at javax.faces.component.ComponentStateHelper.eval(ComponentStateHelper.java:194)
at javax.faces.component.ComponentStateHelper.eval(ComponentStateHelper.java:182)
at javax.faces.component.UIData.getValue(UIData.java:731)
at org.primefaces.component.datatable.DataTable.getValue(DataTable.java:740)
at javax.faces.component.UIData.getDataModel(UIData.java:1798)
at javax.faces.component.UIData.getRowCount(UIData.java:356)
at org.primefaces.component.datatable.DataTableRenderer.encodeTbody(DataTableRenderer.java:600)
at org.primefaces.component.datatable.DataTableRenderer.encodeEnd(DataTableRenderer.java:91)
at javax.faces.component.UIComponentBase.encodeEnd(UIComponentBase.java:875)
at javax.faces.component.UIComponent.encodeAll(UIComponent.java:1763)
at com.sun.faces.context.PartialViewContextImpl$PhaseAwareVisitCallback.visit(PartialViewContextImpl.java:518)
at com.sun.faces.component.visit.PartialVisitContext.invokeVisitCallback(PartialVisitContext.java:183)
at javax.faces.component.UIData.visitTree(UIData.java:1411)
at javax.faces.component.UIComponent.visitTree(UIComponent.java:1600)
at javax.faces.component.UIComponent.visitTree(UIComponent.java:1600)
at javax.faces.component.UIForm.visitTree(UIForm.java:344)
at javax.faces.component.UIComponent.visitTree(UIComponent.java:1600)
at javax.faces.component.UIComponent.visitTree(UIComponent.java:1600)
at com.sun.faces.context.PartialViewContextImpl.processComponents(PartialViewContextImpl.java:376)
at com.sun.faces.context.PartialViewContextImpl.processPartial(PartialViewContextImpl.java:297)
at javax.faces.context.PartialViewContextWrapper.processPartial(PartialViewContextWrapper.java:183)
at javax.faces.component.UIViewRoot.encodeChildren(UIViewRoot.java:981)
at javax.faces.component.UIComponent.encodeAll(UIComponent.java:1756)
at com.sun.faces.application.view.FaceletViewHandlingStrategy.renderView(FaceletViewHandlingStrategy.java:390)
at com.sun.faces.application.view.MultiViewHandler.renderView(MultiViewHandler.java:131)
at com.sun.faces.lifecycle.RenderResponsePhase.execute(RenderResponsePhase.java:121)
at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:101)
at com.sun.faces.lifecycle.LifecycleImpl.render(LifecycleImpl.java:139)
at javax.faces.webapp.FacesServlet.service(FacesServlet.java:594)
at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1539)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:281)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:175)
at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:655)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:595)
at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:98)
at com.sun.enterprise.web.PESessionLockingStandardPipeline.invoke(PESessionLockingStandardPipeline.java:91)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:162)
at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:330)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:231)
at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:232)
at com.sun.grizzly.http.ProcessorTask.invokeAdapter(ProcessorTask.java:828)
at com.sun.grizzly.http.ProcessorTask.doProcess(ProcessorTask.java:725)
at com.sun.grizzly.http.ProcessorTask.process(ProcessorTask.java:1019)
at com.sun.grizzly.http.DefaultProtocolFilter.execute(DefaultProtocolFilter.java:225)
at com.sun.grizzly.DefaultProtocolChain.executeProtocolFilter(DefaultProtocolChain.java:137)
at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:104)
at com.sun.grizzly.DefaultProtocolChain.execute(DefaultProtocolChain.java:90)
at com.sun.grizzly.http.HttpProtocolChain.execute(HttpProtocolChain.java:79)
at com.sun.grizzly.ProtocolChainContextTask.doCall(ProtocolChainContextTask.java:54)
at com.sun.grizzly.SelectionKeyContextTask.call(SelectionKeyContextTask.java:59)
at com.sun.grizzly.ContextTask.run(ContextTask.java:71)
at com.sun.grizzly.util.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:532)
at com.sun.grizzly.util.AbstractThreadPool$Worker.run(AbstractThreadPool.java:513)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.sql.SQLRecoverableException: Closed Connection
at oracle.jdbc.driver.PhysicalConnection.prepareStatement(PhysicalConnection.java:3560)
at com.sun.gjc.spi.base.ConnectionHolder.prepareStatement(ConnectionHolder.java:562)
at com.sun.gjc.spi.jdbc40.ConnectionWrapper40.prepareCachedStatement(ConnectionWrapper40.java:255)
at com.sun.gjc.spi.jdbc40.ConnectionWrapper40.prepareCachedStatement(ConnectionWrapper40.java:52)
at com.sun.gjc.spi.ManagedConnection.prepareCachedStatement(ManagedConnection.java:964)
at com.sun.gjc.spi.jdbc40.ConnectionWrapper40.prepareStatement(ConnectionWrapper40.java:173)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.prepareStatement(DatabaseAccessor.java:1474)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.prepareStatement(DatabaseAccessor.java:1423)
at org.eclipse.persistence.internal.databaseaccess.DatabaseCall.prepareStatement(DatabaseCall.java:697)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.basicExecuteCall(DatabaseAccessor.java:585)
... 125 more|#]
EclipseLink is used in a JDBC connection Pool with Glassfish 3.1.1 .
Do you know a convenient method for the pool to reconnect automatically after losing the connection to the database ?
I think that the problem is not caused by EclipseLink but by the glassfish connection pool which you use.
The pool seems to be not aware that its connections are closed/invalid and still returning them to your application/eclipselink.
I never configured this pool, but please consult the documentation how to enable the keep alive/ verification feature within the connection pool.
This will ensure that the pool will test idle connections periodically and remove/renew them in case they are detected as invalid/closed.