NativeIoException:io.netty.channel.unix.Errors$NativeIoException recvAddress(..) failed: Connection reset by peer - spring-boot

I am trying to find the solution for below error which occurs in our Appdynamics logs when we perform load testing with Jmeter for 5tps. I am using spring cloud gateway 2.7.8 and netty version - 4.1.87.Final for routing purpose. The same error we do not see in our kubernetes logs or in Kibana logs. I am not able to trace from where the logs is coming up.
NativeIoException:io.netty.channel.unix.Errors$NativeIoException recvAddress(..) failed: Connection reset by peer Error capture limit has been reached, this stack trace is truncated.
Can someone help me understand why this error is occurring ?

Related

Why does Apache Nifi throws Connection Load Balance Error?

I'm using nifi in version 1.14.0 and facing a very weird situation.
I have a loadbalanced connection and after disconnection in the cluster (due to out of memory errors)
I'm getting the following error:
Error load balance server -
standartLoadBalanceProtocol attempted to receive flowfile from peer for connection with id xxx but no connection exists with that ID
Did anyone experience anything familiar? Thanks for the help!
I think it relates to this ticket https://issues.apache.org/jira/plugins/servlet/mobile#issue/NIFI-9695. Please upgrade to NiFi version 1.16 and see if this error happens again.

ORA-12570: Network Session: Unexpected packet read error

we are getting ORA-12570: Network Session: Unexpected packet read error from our webapi written in .Net core 2.2. The API is hosted in Alpine Docker OS 3.11 in GCP using kubernetes. We are using Oracle.ManagedDataAccess.Core version 2.19.60.
The intrenal error message we get is
Oracle.ManagedDataAccess.Client.OracleException (0x80004005): ORA-12570: Network Session: Unexpected packet read error ---> OracleInternal.Network.NetworkException (0x80004005): ORA-12570: Network Session: Unexpected packet read error ---> System.Net.Sockets.SocketException (110): Operation timed out.
Per the website http://www.dba-oracle.com/t_ora_12570_tns_packet_reader_failure.htm, ORA-12570 occur due to listener configuration. IS that true? Also Let us know how if tracing works in linux for ODP.Net core.
Thanks
This is a generic error; it is not necessarily related to the Oracle Listener. The key here is "System.Net.Sockets.SocketException (110): Operation timed out." This could be a lot of things; you really need to do Oracle Net tracing to determine what's going on. Could be that your client can't see the network at all, or that network latency or packet routing are not what they should be, or several other things.

Excessive ERROR logs failed hearbeats with Spring SockJS and non-websocket transports

We're using Spring's SockJS with Stomp to send push messages to the browser.
It works fine so far, but when a client (browser) disconnects, we get an ERROR log everytime because the scheduled hearbeat fails:
11:11:18.241 [MessageBroker-1] ERROR [org.springframework.scheduling.support.TaskUtils$LoggingErrorHandler] - Unexpected error occurred in scheduled task.
org.springframework.web.socket.sockjs.SockJsTransportFailureException: Failed to write SockJsFrame content='h'; nested exception is org.apache.catalina.connector.ClientAbortException: java.io.IOException: Broken pipe
at org.springframework.web.socket.sockjs.transport.session.AbstractSockJsSession.writeFrame(AbstractSockJsSession.java:339) ~[spring-websocket-4.3.4.RELEASE.jar:4.3.4.RELEASE]
at org.springframework.web.socket.sockjs.transport.session.PollingSockJsSession.writeFrame(PollingSockJsSession.java:82) ~[spring-websocket-4.3.4.RELEASE.jar:4.3.4.RELEASE]
at org.springframework.web.socket.sockjs.transport.session.AbstractSockJsSession.sendHeartbeat(AbstractSockJsSession.java:255) ~[spring-websocket-4.3.4.RELEASE.jar:4.3.4.RELEASE]
at org.springframework.web.socket.sockjs.transport.session.AbstractSockJsSession$HeartbeatTask.run(AbstractSockJsSession.java:451) ~[spring-websocket-4.3.4.RELEASE.jar:4.3.4.RELEASE]
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54) ~[spring-context-4.3.4.RELEASE.jar:4.3.4.RELEASE]
But only when the client uses XHR-Polling fallback, websocket doesn't cause any errors. We use nginx as reverse proxy in front of tomcat 8.0.39. Nginx logs 499 (client closed connection) a few seconds before the SockJsTransportFailureException is risen
Is there a way to get rid of the ERROR logs when a heartbeat fails? I don't want to disable all logging from org.springframework.scheduling.support.TaskUtils as we might miss important errors.

Oracle service bus Invocation Error

Whenever I try to invoke a proxy service in OSB it gives me error. Here's the error I see:
com.bea.wli.sb.transports.TransportException:
The invocation resulted in an error:
Unknown error while processing message for service ProxyService {Proxy service path}
A HTTP 500 is returned.
In the message flow I can see a request was returned as it is saying "No service has been invoked." In the logs I can see "Error to place router information".
Any ideas for how to overcome this?
HTTP 500 is essentially "Internal Server Error". Check OSB server logs for full stack trace. The problem is around the area where Proxy Service has been deployed.

Socket Exception : recv failed with oracle thin driver

I am facing an issue where my test suite randomly fails with an socket exception
oracle.jdbc.driver.T4CStatement 1267 - Throwing SQLException: java.net.SocketException: Software caused connection abort: recv failed
The test suite fails with this exception when a given set of test cases are executed in a particular order. I got the above error log after enabling the oracle jdbc driver logs. The query which leads to this error is always a "DROP SEQUENCE query". There is nothing special about this query since it is fired 'n' number of times during the execution flow.
One of the blog link points out that the above error is because the server side sockets gets closed before the client expects. To troubleshoot more on this point I tried analyzing the Oracle TNSListener logs - listener.log file but was not able to gather much information since the log file only contained information about the socket CONNECT function call.
What could be the possible causes of the above error in addition to the one the blog link mentions?
How can I configure the Oracle TNSListener to provide more detailed information about the socket communication? For e.g. Trace information when the server socket close event is fired.
I would appreciated if anyone could point out to a possible cause of this error or provide more information which could help me to troubleshoot this issue further based on the above two points
You can set the trace level if you have access to the lsnrctl utility:
LSNRCTL> show trc_level
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=xxx)(PORT=1521)))
LISTENER parameter "trc_level" set to off
The command completed successfully
LSNRCTL> set trc_level admin
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=xxx)(PORT=1521)))
LISTENER parameter "trc_level" set to admin
The command completed successfully
LSNRCTL> show trc_level
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=xxx)(PORT=1521)))
LISTENER parameter "trc_level" set to admin
The command completed successfully
LSNRCTL>
From the docs, trc_level is one of:
Specify one of the following trace levels:
off for no trace output
user for user trace information
admin for administration trace information
support for Oracle Support Services trace information

Resources