I'm having a problem with ActiveMQ 5.6.0 on Windows 7.
WARN | Transport Connection to: tcp://127.0.0.1:55793 failed: java.net.SocketException: Software caused connection abort: recv failed
WARN | Transport Connection to: tcp://127.0.0.1:55791 failed: java.net.SocketException: Software caused connection abort: recv failed
WARN | Transport Connection to: tcp://127.0.0.1:55792 failed: java.net.SocketException: Software caused connection abort: recv failed
Anyone know what might cause this? Here's a screenshot:
Can you run your app/JRE with the following command line arguments?
-Djava.net.preferIPv4Stack=true
See here for more info.
Related
we have some issue with ambari-metrics-collector service , ( we have HDP cluster version - 2.6.4 with 8 nodes )
ambari metrics collector service can’t start or start of few second then failed
the details about metrics collector version
rpm -qa | grep metrics
ambari-metrics-grafana-2.6.1.0-143.x86_64
ambari-metrics-monitor-2.6.1.0-143.x86_64
ambari-metrics-collector-2.5.0.3-7.x86_64
ambari-metrics-hadoop-sink-2.6.1.0-143.x86_64
all machines are rhel 7.2
we performed the following steps in order to resolve the problem
1.restart metrics-collector service
su - ams -c '/usr/sbin/ambari-metrics-collector --config /etc/ambari-metrics-collector/conf/ stop'
su - ams -c '/usr/sbin/ambari-metrics-collector --config /etc/ambari-metrics-collector/conf/ start'
or
ambari-metrics-collector stop
ambari-metrics-collector start
2.restart ambari-metrics-monitor on all nodes
ambari-metrics-monitor stop
ambari-metrics-monitor start
3.clean the folder /var/lib/ambari-metrics-collector/hbase-tmp/zookeeper/
mv /var/lib/ambari-metrics-collector/hbase-tmp/zookeeper/zookeeper_0 /tmp/bck/zookeeper/
Then restart metrics-collector service
4.Tuning the metrics-collector parameters according - https://docs.cloudera.com/HDPDocuments/Ambari-2.2.1.0/bk_ambari_reference_guide/content/_ams_general_guidelines.html
we update the follwing parameters in ambari
metrics_collector_heap_size=1024
hbase_regionserver_heapsize=1024
hbase_master_heapsize=512
hbase_master_xmn_size=128
status for now: - steps 1-4 doesn’t help
From the logs we can see the following:
log file - ambari-metrics-collector.log
2020-06-25 09:06:14,474 WARN org.apache.zookeeper.ClientCnxn: Session 0x172eab71f310002 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)
2020-06-25 09:06:14,575 WARN org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper, quorum=master02.sys671.com:61181, exception=org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /ams-hbase-unsecure/meta-region-server
log file - hbase-ams-master-master02.sys671.com.log
2020-06-25 09:38:18,799 WARN [RS:0;master02:51842-SendThread(master02.sys671.com:61181)] zookeeper.ClientCnxn: Session 0x172ead5d73a0004 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1125)
2020-06-25 09:38:20,437 INFO [main-SendThread(master02.sys671.com:61181)] zookeeper.ClientCnxn: Opening socket connection to server master02.sys671.com/23.2.35.171:61181. Will not attempt to authenticate using SASL (unknown error)
2020-06-25 09:38:20,438 WARN [main-SendThread(master02.sys671.com:61181)] zookeeper.ClientCnxn: Session 0x172ead5d73a0002 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
we also not see that port is listening ( timeline.metrics.service.webapp.address )
netstat -tulpn | grep 6188
any advice how to continue from this point ?
we'll appreciate to get any help about this problem
I installed the MQ Light server on Windows 7 and followed the steps at https://developer.ibm.com/messaging/docs/mq-light-api/tutorials/security-tutorial/ to enable SSL using a self signed certificate generated by OpenSSL 1.1.0g.
At that point, once I started the MQ light server, my browser timed out trying to do the TLS handshake to the server and any attempt to connect to the server immediately using the Java MQ libraries aborted because the server closed the connection.
I couldn't figure out where the MQ server logs are to check them. Disabling SSL results in the connection working again.
I'm using Oracle Java 8u161 (64-bit).
Any idea what's going on here and how to get SSL working?
Caused by: com.ibm.mq.jmqi.JmqiException: CC=2;RC=2397;AMQ9204: Connection to host 'localhost(5671)' rejected. [1=com.ibm.mq.jmqi.JmqiException[CC=2;RC=2397;AMQ9771: SSL handshake failed. [1=javax.net.ssl.SSLHandshakeException[Remote host closed connection during handshake],3=localhost/127.0.0.1:5671 (localhost),4=SSLSocket.startHandshake,5=default]],3=localhost(5671),5=RemoteTCPConnection.protocolConnect]
at com.ibm.mq.jmqi.remote.api.RemoteFAP.jmqiConnect(RemoteFAP.java:2282)
at com.ibm.mq.jmqi.remote.api.RemoteFAP.jmqiConnect(RemoteFAP.java:1294)
at com.ibm.mq.ese.jmqi.InterceptedJmqiImpl.jmqiConnect(InterceptedJmqiImpl.java:376)
at com.ibm.mq.ese.jmqi.ESEJMQI.jmqiConnect(ESEJMQI.java:560)
at com.ibm.mq.MQSESSION.MQCONNX_j(MQSESSION.java:916)
at com.ibm.mq.MQManagedConnectionJ11.<init>(MQManagedConnectionJ11.java:235)
... 16 more
Caused by: com.ibm.mq.jmqi.JmqiException: CC=2;RC=2397;AMQ9771: SSL handshake failed. [1=javax.net.ssl.SSLHandshakeException[Remote host closed connection during handshake],3=localhost/127.0.0.1:5671 (localhost),4=SSLSocket.startHandshake,5=default]
at com.ibm.mq.jmqi.remote.impl.RemoteTCPConnection.protocolConnect(RemoteTCPConnection.java:1329)
at com.ibm.mq.jmqi.remote.impl.RemoteConnection.connect(RemoteConnection.java:863)
at com.ibm.mq.jmqi.remote.impl.RemoteConnectionSpecification.getSessionFromNewConnection(RemoteConnectionSpecification.java:409)
at com.ibm.mq.jmqi.remote.impl.RemoteConnectionSpecification.getSession(RemoteConnectionSpecification.java:305)
at com.ibm.mq.jmqi.remote.impl.RemoteConnectionPool.getSession(RemoteConnectionPool.java:146)
at com.ibm.mq.jmqi.remote.api.RemoteFAP.jmqiConnect(RemoteFAP.java:1730)
... 21 more
Caused by: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1002)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1385)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1413)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1397)
at com.ibm.mq.jmqi.remote.impl.RemoteTCPConnection$6.run(RemoteTCPConnection.java:1298)
at com.ibm.mq.jmqi.remote.impl.RemoteTCPConnection$6.run(RemoteTCPConnection.java:1290)
at java.security.AccessController.doPrivileged(Native Method)
at com.ibm.mq.jmqi.remote.impl.RemoteTCPConnection.protocolConnect(RemoteTCPConnection.java:1290)
... 26 more
Caused by: java.io.EOFException: SSL peer shut down incorrectly
at sun.security.ssl.InputRecord.read(InputRecord.java:505)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:983)
... 33 more
After some JMS connectivity problems I've noticed in logs:
o.a.a.b.T.Transport:? - Transport Connection to: tcp://100.100.100.100:61141 failed: java.io.EOFException
o.a.a.b.T.Transport:? - Transport Connection to: tcp://100.100.100.100:61156 failed: java.io.EOFException
o.a.a.b.T.Transport:? - Transport Connection to: tcp://100.100.100.100:61148 failed: java.io.EOFException
o.a.a.b.T.Transport:? - Transport Connection to: tcp://100.100.100.100:61161 failed: java.io.EOFException
o.a.a.b.T.Transport:? - Transport Connection to: tcp://100.100.100.100:61192 failed: java.io.EOFException
o.a.a.b.T.Transport:? - Transport Connection to: tcp://100.100.100.100:61197 failed: java.io.EOFException
o.a.a.b.T.Transport:? - Transport Connection to: tcp://100.100.100.100:61226 failed: java.io.EOFException
o.a.a.b.T.Transport:? - Transport Connection to: tcp://100.100.100.100:61273 failed: java.io.EOFException
o.a.a.b.T.Transport:? - Transport Connection to: tcp://100.100.100.100:61241 failed: java.io.EOFException
Why JMS is retrying in such way? Does ActiveMQ client/broker technology have any port discovery/negotation protocol?
ActiveMQ has discovery using IP Multicast and failover mechanisms. Not clear why it reconnects to the same host, failover config may have the same host. Client configuration would be helpful to understand.
I run nutch in my hadoop cluster.
When the job gets to the step fetch data I get java.net.SocketException: Connection reset. Here's the full stacktrace:
2013-10-09 00:34:05,922 INFO org.apache.nutch.fetcher.Fetcher: fetch of Url error : xxxxxxx failed with: java.net.SocketException: Connection reset
2013-10-09 00:34:05,923 ERROR org.apache.nutch.protocol.httpclient.Http: Failed to get protocol output
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:189)
at java.net.SocketInputStream.read(SocketInputStream.java:121)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
at org.apache.commons.httpclient.HttpParser.readRawLine(HttpParser.java:77)
at org.apache.commons.httpclient.HttpParser.readLine(HttpParser.java:105)
at org.apache.commons.httpclient.HttpConnection.readLine(HttpConnection.java:1115)
at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.readLine(MultiThreadedHttpConnectionManager.java:1373)
at org.apache.commons.httpclient.HttpMethodBase.readStatusLine(HttpMethodBase.java:1832)
at org.apache.commons.httpclient.HttpMethodBase.readResponse(HttpMethodBase.java:1590)
at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:995)
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:397)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:170)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:324)
at org.apache.nutch.protocol.httpclient.HttpResponse.<init>(HttpResponse.java:94)
at org.apache.nutch.protocol.httpclient.Http.getResponse(Http.java:154)
at org.apache.nutch.protocol.http.api.HttpBase.getProtocolOutput(HttpBase.java:140)
at org.apache.nutch.fetcher.Fetcher$FetcherThread.run(Fetcher.java:703)
You have to indicate url's protocol on your seedlist! for example:
http://stackoverflow.com/
https://google.com
ftp://foo.bar
I'd be very pleased getting help from yours so many thanks in advance.
Can't manage to connect to a WAS server using VisualVM.
WAS properties setup:
Servers->Server types->WebSphere application servers->server1->Process
definition->Java Virtual Machine -> Properties JVM
-Djavax.management.builder.initial=
-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false
-Dcom.sun.management.jmxremote.port=9999
WAS console output:
INFO [com.sun.tools.visualvm.jmx.impl.JmxModelImpl]:
connect(service:jmx:rmi:///jndi/rmi://:9999/jmxrmi)
java.io.EOFException: SSL peer shut down incorrectly
at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:333)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:798)
Caused: javax.net.ssl.SSLHandshakeException: Remote host closed
connection during handshake
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:817)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1138)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:632)
at com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:59)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
at java.io.DataOutputStream.flush(DataOutputStream.java:106)
at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:211)
Caused: java.rmi.ConnectIOException: error during JRMP connection
establishment; nested exception is:
javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake
at sun.rmi.transport.tcp.TCPChannel.createConnection(TCPChannel.java:286)
at sun.rmi.transport.tcp.TCPChannel.newConnection(TCPChannel.java:184)
at sun.rmi.server.UnicastRef.newCall(UnicastRef.java:322)
at sun.rmi.registry.RegistryImpl_Stub.lookup(Unknown Source)
at com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:97)
Caused: javax.naming.CommunicationException [Root exception is
java.rmi.ConnectIOException: error during JRMP connection
establishment; nested exception is:
javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake]
at com.sun.jndi.rmi.registry.RegistryContext.lookup(RegistryContext.java:101)
at com.sun.jndi.toolkit.url.GenericURLContext.lookup(GenericURLContext.java:185)
at javax.naming.InitialContext.lookup(InitialContext.java:392)
at javax.management.remote.rmi.RMIConnector.findRMIServerJNDI(RMIConnector.java:1886)
at javax.management.remote.rmi.RMIConnector.findRMIServer(RMIConnector.java:1856)
at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:257)
Caused: java.io.IOException: Failed to retrieve RMIServer stub
at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:338)
at com.sun.tools.visualvm.jmx.impl.JmxModelImpl$ProxyClient.tryConnect(JmxModelImpl.java:496)
at com.sun.tools.visualvm.jmx.impl.JmxModelImpl$ProxyClient.connect(JmxModelImpl.java:440)
at com.sun.tools.visualvm.jmx.impl.JmxModelImpl.connect(JmxModelImpl.java:220)
at com.sun.tools.visualvm.jmx.impl.JmxModelImpl.(JmxModelImpl.java:209)
at com.sun.tools.visualvm.jmx.impl.JmxModelProvider.createModelFor(JmxModelProvider.java:61)
at com.sun.tools.visualvm.jmx.impl.JmxModelProvider.createModelFor(JmxModelProvider.java:42)
at com.sun.tools.visualvm.core.model.ModelFactory.getModel(ModelFactory.java:111)
at com.sun.tools.visualvm.tools.jmx.JmxModelFactory.getJmxModelFor(JmxModelFactory.java:69)
at com.sun.tools.visualvm.jmx.impl.JmxApplicationProvider.addJmxApplication(JmxApplicationProvider.java:267)
at com.sun.tools.visualvm.jmx.impl.JmxApplicationProvider.createJmxApplication(JmxApplicationProvider.java:185)
at com.sun.tools.visualvm.jmx.JmxApplicationsSupport.createJmxApplicationImpl(JmxApplicationsSupport.java:286)
at com.sun.tools.visualvm.jmx.JmxApplicationsSupport.createJmxApplicationInteractive(JmxApplicationsSupport.java:263)
[catch] at
com.sun.tools.visualvm.jmx.impl.AddJMXConnectionAction$1.run(AddJMXConnectionAction.java:80)
at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:1418)
at org.openide.util.RequestProcessor$Processor.run(RequestProcessor.java:1957)
Thank you a lot again.
Regards.
Good morning.
Well,we've discovered the solution,kinda easy indeed.
The propertie -Djava.rmi.server.hostname=hostname was pointing to localhost instead of ip.
I Hope this info be usefull to someone more.
Thank you very much.
Regards.