Problem description
Application implements Spring webflux websocket Netty server that forwards ffmpeg stream output to websocket protocol. This application allows multiple viewers to see the stream from their web browser by using jsmpeg javascript library.
This solution works as expected in Firefox web browser.
https://i.stack.imgur.com/X4Fm4.png
But in Chrome web browser the stream fails no load. Chrome browser cannot get data from the websocket connection.
Follows an screenshot of the problem seen in Chrome DevTools.
https://i.stack.imgur.com/xWvll.png
Question
What should be changed in this solution so that stream can be opened in Chrome?
How to reproduce the problem
Pull following git repository: https://github.com/haraldsegliens/spring-webflux-websocket-jsmpeg-chrome-problem/tree/master
Install FFMpeg: https://ffmpeg.org/download.html
Update ffmpeg-command variable in src/main/resources/application.yaml so it references the ffmpeg program in your system
Run Spring application. The application by default creates a following endpoint: ws://localhost:8080/stream
Open view_stream.html in Chrome browser
My research of the problem
Turned on TRACE logs and opened the stream in Chrome browser. I see following error in application logs:
2022-09-02 15:26:36.952 - DEBUG [reactor-http-nio-3] r.n.h.s.HttpServerOperations --- [08cf17fd, L:/[0:0:0:0:0:0:0:1]:8080 - R:/[0:0:0:0:0:0:0:1]:54815] New http connection, requesting read
2022-09-02 15:26:36.952 - DEBUG [reactor-http-nio-3] r.n.t.TransportConfig --- [08cf17fd, L:/[0:0:0:0:0:0:0:1]:8080 - R:/[0:0:0:0:0:0:0:1]:54815] Initialized pipeline DefaultChannelPipeline{(reactor.left.httpCodec = io.netty.handler.codec.http.HttpServerCodec), (reactor.left.httpTrafficHandler = reactor.netty.http.server.HttpTrafficHandler), (reactor.right.reactiveBridge = reactor.netty.channel.ChannelOperationsHandler)}
2022-09-02 15:26:36.955 - DEBUG [reactor-http-nio-3] r.n.h.s.HttpServerOperations --- [08cf17fd, L:/[0:0:0:0:0:0:0:1]:8080 - R:/[0:0:0:0:0:0:0:1]:54815] Increasing pending responses, now 1
2022-09-02 15:26:36.955 - DEBUG [reactor-http-nio-3] r.n.h.s.HttpServer --- [08cf17fd-1, L:/[0:0:0:0:0:0:0:1]:8080 - R:/[0:0:0:0:0:0:0:1]:54815] Handler is being applied: org.springframework.http.server.reactive.ReactorHttpHandlerAdapter#6c90595a
2022-09-02 15:26:36.956 - TRACE [reactor-http-nio-3] o.s.w.s.a.HttpWebHandlerAdapter --- [08cf17fd-2] HTTP GET "/stream", headers={masked}
2022-09-02 15:26:36.962 - DEBUG [reactor-http-nio-3] o.s.w.r.h.SimpleUrlHandlerMapping --- [08cf17fd-2] Mapped to com.edi.pacs.video_stream.FfmpegStreamSocketHandler#7ead1d80
2022-09-02 15:26:36.963 - DEBUG [reactor-http-nio-3] r.n.ReactorNetty --- [08cf17fd-1, L:/[0:0:0:0:0:0:0:1]:8080 - R:/[0:0:0:0:0:0:0:1]:54815] Removed handler: reactor.left.httpTrafficHandler, pipeline: DefaultChannelPipeline{(reactor.left.httpCodec = io.netty.handler.codec.http.HttpServerCodec), (reactor.right.reactiveBridge = reactor.netty.channel.ChannelOperationsHandler)}
2022-09-02 15:26:36.964 - DEBUG [reactor-http-nio-3] r.n.ReactorNetty --- [08cf17fd-1, L:/[0:0:0:0:0:0:0:1]:8080 - R:/[0:0:0:0:0:0:0:1]:54815] Non Removed handler: reactor.left.accessLogHandler, context: null, pipeline: DefaultChannelPipeline{(reactor.left.httpCodec = io.netty.handler.codec.http.HttpServerCodec), (reactor.right.reactiveBridge = reactor.netty.channel.ChannelOperationsHandler)}
2022-09-02 15:26:36.964 - DEBUG [reactor-http-nio-3] r.n.ReactorNetty --- [08cf17fd-1, L:/[0:0:0:0:0:0:0:1]:8080 - R:/[0:0:0:0:0:0:0:1]:54815] Non Removed handler: reactor.left.httpMetricsHandler, context: null, pipeline: DefaultChannelPipeline{(reactor.left.httpCodec = io.netty.handler.codec.http.HttpServerCodec), (reactor.right.reactiveBridge = reactor.netty.channel.ChannelOperationsHandler)}
2022-09-02 15:26:36.964 - DEBUG [reactor-http-nio-3] i.n.h.c.h.w.WebSocketServerHandshaker --- [id: 0x08cf17fd, L:/[0:0:0:0:0:0:0:1]:8080 - R:/[0:0:0:0:0:0:0:1]:54815] WebSocket version V13 server handshake
2022-09-02 15:26:36.964 - DEBUG [reactor-http-nio-3] i.n.h.c.h.w.WebSocketServerHandshaker --- WebSocket version 13 server handshake key: BCbwUez1uBWHEcnoIKUeBw==, response: 2L7i3HHoJx/Q9QKFH1IWl7ZYzYg=
2022-09-02 15:26:36.964 - DEBUG [reactor-http-nio-3] i.n.h.c.h.w.WebSocketServerHandshaker --- Requested subprotocol(s) not supported: null
2022-09-02 15:26:36.965 - DEBUG [reactor-http-nio-3] o.s.w.r.s.a.ReactorNettyWebSocketSession --- [08cf17fd-2] Session id "b528e63" for http://localhost:8080/stream
2022-09-02 15:26:36.966 - TRACE [reactor-http-nio-3] o.s.w.r.s.a.ReactorNettyWebSocketSession --- [08cf17fd-2] Sending WebSocket BINARY message (8 bytes)
2022-09-02 15:26:36.966 - TRACE [reactor-http-nio-3] i.n.h.c.h.w.WebSocket08FrameEncoder --- Encoding WebSocket Frame opCode=2 length=8
2022-09-02 15:26:36.966 - TRACE [reactor-http-nio-3] o.s.w.s.a.HttpWebHandlerAdapter --- [08cf17fd-2] Completed 200 OK, headers={}
2022-09-02 15:26:36.967 - TRACE [reactor-http-nio-3] o.s.h.s.r.ReactorHttpHandlerAdapter --- [08cf17fd-1, L:/[0:0:0:0:0:0:0:1]:8080 - R:/[0:0:0:0:0:0:0:1]:54815] Handling completed
2022-09-02 15:26:36.967 - INFO [reactor-http-nio-3] r.F.S.2 --- onSubscribe(SinkManyBestEffort.DirectInner)
2022-09-02 15:26:36.967 - INFO [reactor-http-nio-3] r.F.S.2 --- request(128)
2022-09-02 15:26:36.967 - ERROR [reactor-http-nio-3] r.n.t.ServerTransport --- [08cf17fd-1, L:/[0:0:0:0:0:0:0:1]:8080 - R:/[0:0:0:0:0:0:0:1]:54815] onUncaughtException(ws{uri=/stream, connection=SimpleConnection{channel=[id: 0x08cf17fd, L:/[0:0:0:0:0:0:0:1]:8080 - R:/[0:0:0:0:0:0:0:1]:54815]}})
java.io.IOException: An established connection was aborted by the software in your host machine
at java.base/sun.nio.ch.SocketDispatcher.read0(Native Method)
at java.base/sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:46)
at java.base/sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:276)
at java.base/sun.nio.ch.IOUtil.read(IOUtil.java:233)
at java.base/sun.nio.ch.IOUtil.read(IOUtil.java:223)
at java.base/sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:389)
at io.netty.buffer.PooledByteBuf.setBytes(PooledByteBuf.java:258)
at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1132)
at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:357)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:151)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:722)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:658)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:584)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:832)
2022-09-02 15:26:36.968 - TRACE [reactor-http-nio-3] r.n.c.ChannelOperations --- [08cf17fd-1, L:/[0:0:0:0:0:0:0:1]:8080 - R:/[0:0:0:0:0:0:0:1]:54815] Disposing ChannelOperation from a channel
java.lang.Exception: ChannelOperation dispose stack
at reactor.netty.channel.ChannelOperations.dispose(ChannelOperations.java:196)
at reactor.netty.transport.ServerTransport$ChildObserver.onUncaughtException(ServerTransport.java:467)
at reactor.netty.channel.FluxReceive.drainReceiver(FluxReceive.java:232)
at reactor.netty.channel.FluxReceive.onInboundError(FluxReceive.java:453)
at reactor.netty.channel.ChannelOperations.onInboundError(ChannelOperations.java:488)
at reactor.netty.channel.ChannelOperationsHandler.exceptionCaught(ChannelOperationsHandler.java:126)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281)
at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273)
at io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1377)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281)
at io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.handleReadException(AbstractNioByteChannel.java:125)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:177)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:722)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:658)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:584)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:832)
2022-09-02 15:26:36.968 - INFO [reactor-http-nio-3] r.F.S.2 --- cancel()
My assumption is that Chrome doesn't "like" something about the given response from the Spring application and then Google closed the connection.
Next what I'll try is to use Jetty or Undertow as server but if it solves my problem, it won't solve my question - why doesn't it work with Netty.
Fixed the problem. Problem was that some web browsers are "allergic" to subprotocol header being empty.
chrome empty subprotocol header network log
There are two possible solutions for this problem.
First solution, fill the subprotocol value. Done the fix in https://github.com/haraldsegliens/spring-webflux-websocket-jsmpeg-chrome-problem/tree/solution/use-protocol and Chrome now handles the stream correctly.
chrome now works by using subprotocol
Second solution, fix jsmpeg so that it doesn't write subprotocol header with empty value instead it should skip subprotocol header. Done fix in https://github.com/haraldsegliens/spring-webflux-websocket-jsmpeg-chrome-problem/tree/solution/fix-in-jsmpeg
Following pull request that fixes the problem in jsmpeg repository: https://github.com/phoboslab/jsmpeg/pull/143
chrome now works by fixing jsmpeg
Related
There was an TimeoutException as shown below while using JDK 11 for WebClient:
Caused by: java.util.concurrent.TimeoutException: Did not observe any item or terminal signal within 60000ms in 'source(MonoDefer)' (and no fallback has been configured)
at reactor.core.publisher.FluxTimeout$TimeoutMainSubscriber.handleTimeout(FluxTimeout.java:295) ~[reactor-core-3.4.24.jar!/:3.4.24]
However, if JDK 8 were used then it works fine and no above TimeoutException with the same test case.
The application is built by using Spring Boot V2.7.5 with "spring-boot-starter-webflux". Here is the version of "reactor-netty-http" in the application:
<dependency>
<groupId>io.projectreactor.netty</groupId>
<artifactId>reactor-netty-http</artifactId>
<version>1.0.24</version>
</dependency>
Here are the observations:
If idle enough time ( aobut 30 seconds ) after the call of a WebClient then below connnection was closed:
2022-11-14 11:17:47.773 DEBUG 7600 --- [ctor-http-nio-3] reactor.netty.http.client.HttpClient : [3d257d24, L:/xx.xx.xx.xx:64172 - R:nnctj2prxc4t.apps.internal.xxxx.net/xx.xx.xx.xx:443] READ COMPLETE
2022-11-14 11:18:17.264 DEBUG 7600 --- [ctor-http-nio-3] reactor.netty.http.client.HttpClient : [3d257d24, L:/xx.xx.xx.xx:64172 - R:nnctj2prxc4t.apps.internal.xxxx.net/xx.xx.xx.xx:443] READ COMPLETE
2022-11-14 11:18:17.265 DEBUG 7600 --- [ctor-http-nio-3] reactor.netty.http.client.HttpClient : [3d257d24, L:/xx.xx.xx.xx:64172 ! R:nnctj2prxc4t.apps.internal.xxxx.net/xx.xx.xx.xx:443] USER_EVENT: SslCloseCompletionEvent(java.nio.channels.ClosedChannelException)
2022-11-14 11:18:17.265 DEBUG 7600 --- [ctor-http-nio-3] reactor.netty.http.client.HttpClient : [3d257d24, L:/xx.xx.xx.xx:64172 ! R:nnctj2prxc4t.apps.internal.xxxx.net/xx.xx.xx.xx:443] INACTIVE
2022-11-14 11:18:17.265 DEBUG 7600 --- [ctor-http-nio-3] reactor.netty.http.client.HttpClient : [3d257d24, L:/xx.xx.xx.xx:64172 ! R:nnctj2prxc4t.apps.internal.xxxx.net/xx.xx.xx.xx:443] UNREGISTERED
As you can see, the HttpClient on the thread 'ctor-http-nio-3' was closed and unregistered after about 30 seconds with a SslCloseCompletionEvent(java.nio.channels.ClosedChannelException).
Then try to use the webclient again by sending a request. Here is an output which was successfully:
2022-11-14 11:18:59.542 DEBUG 7600 --- [ctor-http-nio-4] reactor.netty.http.client.HttpClient : [476de49d] REGISTERED
2022-11-14 11:18:59.630 DEBUG 7600 --- [ctor-http-nio-4] reactor.netty.http.client.HttpClient : [476de49d] CONNECT: nnctj2prxc4t.apps.internal.xxxxx.net/xx.xx.xx.xx:443
2022-11-14 11:18:59.743 DEBUG 7600 --- [ctor-http-nio-4] reactor.netty.http.client.HttpClient : [476de49d, L:/xx.xx.xx.xx:64197 - R:nnctj2prxc4t.apps.internal.xxxxx.net/xx.xx.xx.xx:443] ACTIVE
2022-11-14 11:18:59.849 DEBUG 7600 --- [ctor-http-nio-4] reactor.netty.http.client.HttpClient : [476de49d, L:/xx.xx.xx.xx:64197 - R:nnctj2prxc4t.apps.internal.xxxxx.net/xx.xx.xx.xx:443] READ COMPLETE
2022-11-14 11:18:59.858 DEBUG 7600 --- [ctor-http-nio-4] reactor.netty.http.client.HttpClient : [476de49d, L:/xx.xx.xx.xx:64197 - R:nnctj2prxc4t.apps.internal.xxxxx.net/xx.xx.xx.xx:443] USER_EVENT: SslHandshakeCompletionEvent(SUCCESS)
2022-11-14 11:18:59.860 DEBUG 7600 --- [ctor-http-nio-4] reactor.netty.http.client.HttpClient : [476de49d-1, L:/xx.xx.xx.xx:64197 - R:nnctj2prxc4t.apps.internal.xxxxx.net/xx.xx.xx.xx:443] WRITE: 2023B
.
.
.
As you can see a new connection was registered on a new thread 'ctor-http-nio-4'. The call was successfully.
However if the connection was closed successfully - SslCloseCompletionEvent(SUCCESS) and unregistered. Here is an output:
2022-11-14 09:10:41.295 DEBUG 21332 --- [ctor-http-nio-4] reactor.netty.http.client.HttpClient : [9a6b8e52, L:/xx.xx.xx.xx:56897 - R:nnctj2prxc4t.apps.internal.xxxxx.net/xx.xx.xx.xx:443] CLOSE
2022-11-14 09:10:41.303 DEBUG 21332 --- [ctor-http-nio-4] reactor.netty.http.client.HttpClient : [9a6b8e52, L:/xx.xx.xx.xx:56897 ! R:nnctj2prxc4t.apps.internal.xxxxx.net/xx.xx.xx.xx:443] USER_EVENT: SslCloseCompletionEvent(SUCCESS)
2022-11-14 09:10:41.303 DEBUG 21332 --- [ctor-http-nio-4] reactor.netty.http.client.HttpClient : [9a6b8e52, L:/xx.xx.xx.xx:56897 ! R:nnctj2prxc4t.apps.internal.xxxxx.net/xx.xx.xx.xx:443] READ COMPLETE
2022-11-14 09:10:41.303 DEBUG 21332 --- [ctor-http-nio-4] reactor.netty.http.client.HttpClient : [9a6b8e52, L:/xx.xx.xx.xx:56897 ! R:nnctj2prxc4t.apps.internal.xxxxx.net/xx.xx.xx.xx:443] INACTIVE
2022-11-14 09:10:41.303 DEBUG 21332 --- [ctor-http-nio-4] reactor.netty.http.client.HttpClient : [9a6b8e52, L:/xx.xx.xx.xx:56897 ! R:nnctj2prxc4t.apps.internal.xxxxx.net/xx.xx.xx.xx:443] UNREGISTERED
Then try to use the webclient again by sending a request which is registered an new connection. Here is an output:
2022-11-14 09:11:32.362 DEBUG 21332 --- [ctor-http-nio-5] reactor.netty.http.client.HttpClient : [ad2bc21d] REGISTERED
2022-11-14 09:11:32.424 DEBUG 21332 --- [ctor-http-nio-5] reactor.netty.http.client.HttpClient : [ad2bc21d] CONNECT: nnctj2prxc4t.apps.internal.xxxxx.net/xx.xx.xx.xx:443
2022-11-14 09:11:32.528 DEBUG 21332 --- [ctor-http-nio-5] reactor.netty.http.client.HttpClient : [ad2bc21d, L:/xx.xx.xx.xx:56922 - R:nnctj2prxc4t.apps.internal.xxxxx.net/xx.xx.xx.xx:443] ACTIVE
2022-11-14 09:11:32.970 DEBUG 21332 --- [ctor-http-nio-5] reactor.netty.http.client.HttpClient : [ad2bc21d, L:/xx.xx.xx.xx:56922 - R:nnctj2prxc4t.apps.internal.xxxxxx.net/xx.xx.xx.xx:443] READ COMPLETE
2022-11-14 09:11:33.063 DEBUG 21332 --- [ctor-http-nio-5] reactor.netty.http.client.HttpClient : [ad2bc21d, L:/xx.xx.xx.xx:56922 - R:nnctj2prxc4t.apps.internal.xxxxxx.net/xx.xx.xx.xx:443] READ COMPLETE
2022-11-14 09:11:33.063 DEBUG 21332 --- [ctor-http-nio-5] reactor.netty.http.client.HttpClient : [ad2bc21d, L:/xx.xx.xx.xx:56922 - R:nnctj2prxc4t.apps.internal.xxxxx.net/xx.xx.xx.xx:443] READ COMPLETE
2022-11-14 09:11:33.165 DEBUG 21332 --- [ctor-http-nio-5] reactor.netty.http.client.HttpClient : [ad2bc21d, L:/xx.xx.xx.xx:56922 - R:nnctj2prxc4t.apps.internal.xxxxx.net/xx.xx.xx.xx:443] USER_EVENT: SslHandshakeCompletionEvent(SUCCESS)
.
.
.
Caused by: java.util.concurrent.TimeoutException: Did not observe any item or terminal signal within 60000ms in 'source(MonoDefer)' (and no fallback has been configured)
at reactor.core.publisher.FluxTimeout$TimeoutMainSubscriber.handleTimeout(FluxTimeout.java:295) ~[reactor-core-3.4.24.jar!/:3.4.24]
As you can see, after the connection was closed successfully and unregistered then later call will result TimeoutException with a new registered connection.
Above issue occurred while using JDK 11 (zulu-jdk11.0.14) only. If use JDK 8 (zulu8.61.0.18-sa-jdk8.0.331) then everything is working fine.
I tried to use the same appliation with different JDK 8 and JDK 11. Using JDK 8, there is no issue for WebClient to connect to the REST API and get the data back successfully.
However, I tried to use JDK 11 then I got ReadTimeoutException.
The Application is built by using Spring Boot 2.7.5 which is the latest version. It is using WebClient to connect the REST API server. The built target is Java 8.
Even I tried to build the applicaiton with target Java 11 and running it in Java 11 but still getting the same error.
How can I solve this?
We have two Spring boot applications using the same Elasticsearch database. The queries work fine. But the console of the second spring boot application is always flooded with these warning messages as soon as it boots up. Is there any configuration we are missing, either in the Spring boot app or at Elasticsearch?
2021-02-16 16:15:03.757 INFO 19828 --- [main] o.s.d.e.c.TransportClientFactoryBean: Adding transport node : 127.0.0.1:9300
2021-02-16 16:15:04.760 WARN 19828 --- [ient_boss][T#1]] o.e.transport.TransportService: Transport response handler not found of id [1]
2021-02-16 16:15:04.932 WARN 19828 --- [ient_boss][T#2]] o.e.transport.TransportService: Transport response handler not found of id [3]
2021-02-16 16:15:08.104 WARN 19828 --- [ient_boss][T#7]] o.e.transport.TransportService: Transport response handler not found of id [9]
2021-02-16 16:15:13.232 WARN 19828 --- [ient_boss][T#8]] o.e.transport.TransportService: Transport response handler not found of id [59]
Getting below issue when i fired request to spring boot (internally using Netty server)
2019-02-21 07:28:21.647 ERROR 9 --- [or-http-epoll-2] reactor.netty.tcp.TcpServer : [id: 0x460eebde, L:/10.0.125.18:80 - R:/10.0.125.55:56958] onUncaughtException(SimpleConnection{channel=[id: 0x460eebde, L:/10.0.125.18:80 - R:/10.0.125.55:56958]})
io.netty.channel.unix.Errors$NativeIoException: syscall:read(..) failed: Connection reset by peer
at io.netty.channel.unix.FileDescriptor.readAddress(..)(Unknown Source) ~[netty-transport-native-unix-common-4.1.31.Final.jar!/:4.1.31.Final]enter code here
Not able to send the files to SFTP server. I didn't find the exact issue from the logs see below. used Spring integration SFTP configuration to send my files to client instance. I have referred this link https://blog.pavelsklenar.com/spring-integration-sftp-upload-example/ for implementation. what i m doing wrong here. Any help would be appreciated thank you.
below is the code snippet
DefaultSftpSessionFactory factory = new DefaultSftpSessionFactory(true);
factory.setHost(sftpHost);
factory.setPort(sftpPort);
factory.setUser(sftpUser);
factory.setPassword(sftpPasword);
factory.setAllowUnknownKeys(true);
see the logs below
10-12-2018 03:03:22.261 [task-scheduler-1] INFO org.springframework.integration.sftp.session.DefaultSftpSessionFactory.promptYesNo - The authenticity of host 'example.com' can't be established.
RSA key fingerprint is d1:e9:4e:64:a8:e5:19:46:7a:0e:79:2d:bf:27:cb:4c.
Are you sure you want to continue connecting?
10-12-2018 03:03:22.264 [task-scheduler-1] DEBUG org.springframework.integration.sftp.session.DefaultSftpSessionFactory.promptYesNo - No UserInfo provided - The authenticity of host 'example.com' can't be established.
RSA key fingerprint is d1:e9:4e:64:a8:e5:19:46:7a:0e:79:2d:bf:27:cb:4c.
Are you sure you want to continue connecting?, returning:true
10-12-2018 03:03:22.265 [task-scheduler-1] WARN com.jcraft.jsch.log - Permanently added 'example.com' (RSA) to the list of known hosts.
10-12-2018 03:03:22.269 [task-scheduler-1] INFO com.jcraft.jsch.log - SSH_MSG_NEWKEYS sent
10-12-2018 03:03:22.270 [task-scheduler-1] INFO com.jcraft.jsch.log - SSH_MSG_NEWKEYS received
10-12-2018 03:03:22.280 [task-scheduler-1] INFO com.jcraft.jsch.log - SSH_MSG_SERVICE_REQUEST sent
10-12-2018 03:03:22.306 [task-scheduler-1] INFO com.jcraft.jsch.log - SSH_MSG_SERVICE_ACCEPT received
10-12-2018 03:03:22.308 [task-scheduler-1] DEBUG org.springframework.integration.sftp.session.DefaultSftpSessionFactory.showMessage - SSH Server supporting SFTP and SCP
10-12-2018 03:03:22.337 [task-scheduler-1] INFO com.jcraft.jsch.log - Authentications that can continue: publickey,keyboard-interactive,password
10-12-2018 03:03:22.338 [task-scheduler-1] INFO com.jcraft.jsch.log - Next authentication method: publickey
10-12-2018 03:03:22.342 [task-scheduler-1] INFO com.jcraft.jsch.log - Authentications that can continue: keyboard-interactive,password
10-12-2018 03:03:22.349 [task-scheduler-1] INFO com.jcraft.jsch.log - Next authentication method: keyboard-interactive
10-12-2018 03:03:22.425 [task-scheduler-1] INFO com.jcraft.jsch.log - Authentication succeeded (keyboard-interactive).
10-12-2018 03:03:22.859 [task-scheduler-1] DEBUG org.springframework.integration.sftp.session.SftpSession.rename - File: CONVERGE_20181013.zip.writing was successfully renamed to CONVERGE_20181013.zip
10-12-2018 03:03:22.861 [task-scheduler-1] INFO com.jcraft.jsch.log - Disconnecting from example.com port 22
10-12-2018 03:03:22.866 [Connect thread example.com session] INFO com.jcraft.jsch.log - Caught an exception, leaving main loop due to Socket closed
I am unable to understand the reason behind intermittent HikariCP Connection is not available.
From the logs, It doesn't look like a connection leak issue. A bigger problem is I am unable to predictably reproduce the error. Following is a sample log trace where the error starts while this gist contains it till the end.
2017-12-12T19:31:55.958Z DEBUG <> [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Pool stats (total=10, active=1, idle=9, waiting=0)
2017-12-12T19:31:57.052Z WARN <> [main] c.zaxxer.hikari.pool.ProxyConnection - HikariPool-1 - Connection org.postgresql.jdbc.PgConnection#1de5f0ef marked as broken because of SQLSTATE(08P01), ErrorCode(0)
org.postgresql.util.PSQLException: Expected command status BEGIN, got EMPTY.
at org.postgresql.core.v3.QueryExecutorImpl$1.handleCommandStatus(QueryExecutorImpl.java:515)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2180)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:288)
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:430)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:356)
at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:168)
at org.postgresql.jdbc.PgPreparedStatement.executeQuery(PgPreparedStatement.java:116)
at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeQuery(ProxyPreparedStatement.java:52)
at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeQuery(HikariProxyPreparedStatement.java)
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.extract(ResultSetReturnImpl.java:70)
at org.hibernate.id.SequenceGenerator.generateHolder(SequenceGenerator.java:116)
at org.hibernate.id.SequenceHiLoGenerator.generate(SequenceHiLoGenerator.java:62)
at org.hibernate.event.internal.AbstractSaveEventListener.saveWithGeneratedId(AbstractSaveEventListener.java:101)
at org.hibernate.jpa.event.internal.core.JpaPersistEventListener.saveWithGeneratedId(JpaPersistEventListener.java:67)
... MORE DETAILS IN [GIST][1]
2017-12-12T19:31:57.067Z ERROR <> [main] c.o.r.s.t.RAUTwitterUserService - populateFromPayload: id=128, twitter_handle=non_local, exception=org.springframework.transaction.TransactionSystemException: Could not roll back JPA transaction; nested exception is javax.persistence.PersistenceException: unexpected error when rollbacking
2017-12-12T19:31:57.067Z INFO <> [main] c.o.r.s.t.TwitterCollectorService - RAU from RAU service: RAU(id=129, twitter=non_global)
2017-12-12T19:31:57.089Z DEBUG <> [HikariPool-1 connection adder] com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Added connection org.postgresql.jdbc.PgConnection#419dcf40