In Windows 10, I'm not able to connect to MongoDB server with the following errors:
>mongo "mongodb+srv://xxx-dsvlb.mongodb.net/test" --username xxx --verbose
2018-11-01T11:34:19.273+0700 D - [main] User Assertion: DNSHostNotFound: Failed to look up service "":This operation returned because the timeout period expired. C:\data\mci\6411135b04f345f6d01072b56250cba6\src\src\mongo/util/dns_query_windows-impl.h 254
MongoDB shell version v4.0.3
2018-11-01T11:34:30.535+0700 D - [main] User Assertion: DNSHostNotFound: Failed to look up service "":This operation returned because the timeout period expired. C:\data\mci\6411135b04f345f6d01072b56250cba6\src\src\mongo/util/dns_query_windows-impl.h 254
Enter password:
connecting to: mongodb+srv://xxx-dsvlb.mongodb.net/test
2018-11-01T11:35:16.589+0700 D - [js] User Assertion: DNSHostNotFound: Failed to look up service "":This operation returned because the timeout period expired. C:\data\mci\6411135b04f345f6d01072b56250cba6\src\src\mongo/util/dns_query_windows-impl.h 254
2018-11-01T11:35:16.590+0700 D NETWORK [js] creating new connection to:xxx-shard-00-02-dsvlb.mongodb.net.:27017
2018-11-01T11:35:17.356+0700 D - [js] User Assertion: SSLHandshakeFailed: QueryContextAttributes for connection info failed with-2146893055 C:\data\mci\6411135b04f345f6d01072b56250cba6\src\src\mongo/transport/session_asio.h 240
2018-11-01T11:35:17.357+0700 D NETWORK [js] creating new connection to:xxx-shard-00-01-dsvlb.mongodb.net.:27017
2018-11-01T11:35:18.197+0700 D - [js] User Assertion: SSLHandshakeFailed: QueryContextAttributes for connection info failed with-2146893055 C:\data\mci\6411135b04f345f6d01072b56250cba6\src\src\mongo/transport/session_asio.h 240
2018-11-01T11:35:18.198+0700 D NETWORK [js] creating new connection to:xx-shard-00-00-dsvlb.mongodb.net.:27017
2018-11-01T11:35:19.017+0700 D - [js] User Assertion: SSLHandshakeFailed: QueryContextAttributes for connection info failed with-2146893055 C:\data\mci\6411135b04f345f6d01072b56250cba6\src\src\mongo/transport/session_asio.h 240
2018-11-01T11:35:19.018+0700 D - [js] User Assertion: InternalError: couldn't connect to server lakon-shard-00-00-dsvlb.mongodb.net.:27017, connection attempt failed: SSLHandshakeFailed: QueryContextAttributes for connection info failed with-2146893055 src\mongo\scripting\mozjs\mongo.cpp 756
2018-11-01T11:35:19.021+0700 E QUERY [js] Error: couldn't connect to server lakon-shard-00-00-dsvlb.mongodb.net.:27017, connection attempt failed: SSLHandshakeFailed: QueryContextAttributes for connection info failed with-2146893055 :
connect#src/mongo/shell/mongo.js:257:13
#(connect):1:6
2018-11-01T11:35:19.024+0700 D - [js] User Assertion: Location12513: connect failed src\mongo\shell\shell_utils.cpp 343
2018-11-01T11:35:19.024+0700 I QUERY [js] MozJS GC prologue heap stats - total: 4056565 limit: 0
2018-11-01T11:35:19.027+0700 I QUERY [js] MozJS GC epilogue heap stats - total: 421536 limit: 0
2018-11-01T11:35:19.027+0700 I QUERY [js] MozJS GC prologue heap stats - total: 313504 limit: 0
2018-11-01T11:35:19.028+0700 I QUERY [js] MozJS GC epilogue heap stats - total: 131244 limit: 0
2018-11-01T11:35:19.029+0700 D - [main] User Assertion: Location12513: connect failed src\mongo\scripting\mozjs\proxyscope.cpp 300
exception: connect failed
Using MongoDB shell 3.6.2 on Windows 10, I still cannot connect but with a different error (confusing, isn't it?):
>mongo "mongodb+srv://xxx-dsvlb.mongodb.net/test" --username xxx --password xxx
MongoDB shell version v3.6.2
connecting to: mongodb+srv://xxx-dsvlb.mongodb.net/test
MongoDB server version: 3.6.8
2018-11-01T11:01:52.923+0700 E QUERY [thread1] Error: Authentication failed. :
DB.prototype._authOrThrow#src/mongo/shell/db.js:1608:20
#(auth):6:1
#(auth):1:2
exception: login failed
However, with Ubuntu 16.04 I can connect just fine to the exact same server:
⟫ mongo "mongodb+srv://xxx-dsvlb.mongodb.net/test" --username xxx --password xxx
MongoDB shell version v4.0.3
connecting to: mongodb+srv://xxx-dsvlb.mongodb.net/test
2018-11-01T04:27:02.536+0000 I NETWORK [js] Starting new replica set monitor for lakon-shard-0/xxx-shard-00-02-dsvlb.mongodb.net.:27017,xxx-shard-00-00-dsvlb.mongodb.net.:27017,xxx-shard-00-01-dsvlb.mongodb.net.:27017
2018-11-01T04:27:02.561+0000 I NETWORK [ReplicaSetMonitor-TaskExecutor] Successfully connected to xxx-shard-00-02-dsvlb.mongodb.net.:27017 (1 connections now open to xxx-shard-00-02-dsvlb.mongodb.net.:27017 with a 5 second timeout)
2018-11-01T04:27:02.562+0000 I NETWORK [js] Successfully connected to xxx-shard-00-00-dsvlb.mongodb.net.:27017 (1 connections now open to xxx-shard-00-00-dsvlb.mongodb.net.:27017 with a 5 second timeout)
2018-11-01T04:27:02.563+0000 I NETWORK [js] changing hosts to xxx-shard-0/xxx-shard-00-00-dsvlb.mongodb.net:27017,xxx-shard-00-01-dsvlb.mongodb.net:27017,lakon-shard-00-02-dsvlb.mongodb.net:27017 from xxx-shard-0/xxx-shard-00-00-dsvlb.mongodb.net.:27017,xxx-shard-00-01-dsvlb.mongodb.net.:27017,xxx-shard-00-02-dsvlb.mongodb.net.:27017
2018-11-01T04:27:02.570+0000 I NETWORK [ReplicaSetMonitor-TaskExecutor] Successfully connected to xxx-shard-00-00-dsvlb.mongodb.net:27017 (1 connections now open to xxx-shard-00-00-dsvlb.mongodb.net:27017 with a 5 second timeout)
2018-11-01T04:27:02.573+0000 I NETWORK [js] Successfully connected to xxx-shard-00-02-dsvlb.mongodb.net:27017 (1 connections now open to xxx-shard-00-02-dsvlb.mongodb.net:27017 with a 5 second timeout)
Implicit session: session { "id" : UUID("4a6488c7-7a22-44d4-977e-07eb09ef37f6") }
MongoDB server version: 3.6.8
WARNING: shell and server versions do not match
2018-11-01T04:27:02.588+0000 I NETWORK [ReplicaSetMonitor-TaskExecutor] Successfully connected to xxx-shard-00-01-dsvlb.mongodb.net:27017 (1 connections now open to xxx-shard-00-01-dsvlb.mongodb.net:27017 with a 5 second timeout)
Welcome to the MongoDB shell.
For interactive help, type "help".
For more comprehensive documentation, see
http://docs.mongodb.org/
Questions? Try the support group
http://groups.google.com/group/mongodb-user
MongoDB Enterprise xxx-shard-0:PRIMARY>
A MongoDB Atlas support staff told me this is due to network connection on my part, but I'm sure that is not the root issue, because I can connect to the server when using other client such as Robo 3T using the same Windows 10 computer.
This issue happens ONLY when using MongoDB shell (both 3.6.2 and 4.0.3) in Windows 10.
It's probably a bug with MongoDB Shell and Windows 10 implementation?
it's a bit late but here, I had this problem when my shell's version is 4.0.5 then I install 4.2.11, it solve this problem. I tried many things with different connection string syntax and it does not solve the problem and still stuck at SSLHandshake error, so I guess if version were same it might solve the problem(mine 4.0.5 and remote was 4.2.11) and go ahead with new version installation(though I still think it's not version problem but I don't know what is). This problem only happen while I was in shell, connecting from the client like NoSqlBooster or Spring seem to work fine. My Robo3T have a problem connecting but randomly, sometimes once sometimes multiple re-trying.
Related
We run neo4j (3.5.21) in an EC2 instance. Today, after I restarted the server, noticed this error:
Expected hostname at index 7: bolt://:7687". Starting Neo4j failed: Component 'org.neo4j.server.AbstractNeoServer$ServerComponentsLifecycleAdapter#75401424' was successfully initialized, but failed to start
Service start logs:
Active database: graph.db
Directories in use:
home: /var/lib/neo4j
config: /etc/neo4j
logs: /var/log/neo4j
plugins: /var/lib/neo4j/plugins
import: /var/lib/neo4j/import
data: /var/lib/neo4j/data
certificates: /var/lib/neo4j/certificates
run: /var/run/neo4j
Starting Neo4j.
WARNING: Max 1024 open files allowed, minimum of 40000 recommended. See the Neo4j manual.
Started neo4j (pid 22577). It is available at http://0.0.0.0:7474/
There may be a short delay until the server is ready.
See /var/log/neo4j/neo4j.log for current status.
This is what I see in neo4j.log:
2022-12-03 20:29:49.886+0000 INFO Bolt enabled on 0.0.0.0:7687.
2022-12-03 20:29:51.968+0000 INFO Started.
2022-12-03 20:29:52.121+0000 INFO Stopping...
2022-12-03 20:29:52.231+0000 INFO Stopped.
2022-12-03 20:29:52.233+0000 ERROR Failed to start Neo4j: Starting Neo4j failed: Component 'org.neo4j.server.AbstractNeoServer$ServerComponentsLifecycleAdapter#75401424' was successfully initialized, but failed to start. Please see the attached cause exception "Expected hostname at index 7: bolt://:7687". Starting Neo4j failed: Component 'org.neo4j.server.AbstractNeoServer$ServerComponentsLifecycleAdapter#75401424' was successfully initialized, but failed to start. Please see the attached cause exception "Expected hostname at index 7: bolt://:7687".
org.neo4j.server.ServerStartupException: Starting Neo4j failed: Component 'org.neo4j.server.AbstractNeoServer$ServerComponentsLifecycleAdapter#75401424' was successfully initialized, but failed to start. Please see the attached cause exception "Expected hostname at index 7: bolt://:7687".
at org.neo4j.server.exception.ServerStartupErrors.translateToServerStartupError(ServerStartupErrors.java:45)
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:187)
at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:124)
at org.neo4j.server.ServerBootstrapper.start(ServerBootstrapper.java:91)
at org.neo4j.server.CommunityEntryPoint.main(CommunityEntryPoint.java:32)
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.server.AbstractNeoServer$ServerComponentsLifecycleAdapter#75401424' was successfully initialized, but failed to start. Please see the attached cause exception "Expected hostname at index 7: bolt://:7687".
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:473)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:111)
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:180)
... 3 more
Caused by: org.neo4j.graphdb.config.InvalidSettingException: Unable to construct bolt discoverable URI using '' as hostname: Expected hostname at index 7: bolt://:7687
at org.neo4j.server.rest.discovery.DiscoverableURIs$Builder.add(DiscoverableURIs.java:133)
at org.neo4j.server.rest.discovery.DiscoverableURIs$Builder.lambda$addBoltConnectorFromConfig$1(DiscoverableURIs.java:155)
at java.util.Optional.ifPresent(Optional.java:159)
at org.neo4j.server.rest.discovery.DiscoverableURIs$Builder.addBoltConnectorFromConfig(DiscoverableURIs.java:145)
at org.neo4j.server.rest.discovery.CommunityDiscoverableURIs.communityDiscoverableURIs(CommunityDiscoverableURIs.java:38)
at org.neo4j.server.CommunityNeoServer.lambda$createDBMSModule$0(CommunityNeoServer.java:99)
at org.neo4j.server.modules.DBMSModule.start(DBMSModule.java:59)
at org.neo4j.server.AbstractNeoServer.startModules(AbstractNeoServer.java:249)
at org.neo4j.server.AbstractNeoServer.access$700(AbstractNeoServer.java:102)
at org.neo4j.server.AbstractNeoServer$ServerComponentsLifecycleAdapter.start(AbstractNeoServer.java:541)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:452)
... 5 more
Caused by: java.net.URISyntaxException: Expected hostname at index 7: bolt://:7687
at java.net.URI$Parser.fail(URI.java:2847)
at java.net.URI$Parser.failExpecting(URI.java:2853)
at java.net.URI$Parser.parseHostname(URI.java:3389)
at java.net.URI$Parser.parseServer(URI.java:3235)
at java.net.URI$Parser.parseAuthority(URI.java:3154)
at java.net.URI$Parser.parseHierarchical(URI.java:3096)
at java.net.URI$Parser.parse(URI.java:3052)
at java.net.URI.<init>(URI.java:673)
at org.neo4j.server.rest.discovery.DiscoverableURIs$Builder.add(DiscoverableURIs.java:128)
... 15 more
2022-12-03 20:29:52.243+0000 INFO Neo4j Server shutdown initiated by request
EC2: t3.large
OS: Ubuntu 18.04.5 LTS (GNU/Linux 5.4.0-1092-aws x86_64)
I have already tried restarting the server, restarting the service multiple times without any success. We have not changed anything on the networking (vpc, subnet, security groups, network interface, etc)
Curious if there's a config I am missing. Any help will be much appreciated.
We have a sprinboot application which stops retrying to connect with solace queues after 3 connection attempts. We get below information logged and then application just does not respond and we have to restart the application:
2021-09-15 16:49:08.021 INFO 4444 --- [recovery-thread] bitronix.tm.recovery.Recoverer : recoverer is already running, abandoning this recovery request
2021-09-15 16:50:04.862 INFO 4444 --- [connect_service] c.s.j.protocol.impl.TcpClientChannel : Connection attempt failed to host '<<hostname>>' ReconnectException com.solacesystems.jcsmp.JCSMPSecurityException: Error performing login to LoginContext (*****) cause: javax.security.auth.login.LoginException: *****
2021-09-15 16:50:07.865 INFO 4444 --- [connect_service] c.s.j.protocol.impl.TcpClientChannel : Connecting to host 'orig=tcp://<<hostname>>:55555, scheme=tcp://, host=<<hostname>>, port=55555' (host 1 of 1, smfclient 2, attempt 3 of 3, this_host_attempt: 1 of 1)
2021-09-15 16:50:07.877 INFO 4444 --- [connect_service] c.s.j.protocol.impl.TcpClientChannel : Connection attempt failed to host '<<hostname>>' ReconnectException com.solacesystems.jcsmp.JCSMPSecurityException: Error performing login to LoginContext (*****) cause: javax.security.auth.login.LoginException: *****
2021-09-15 16:50:10.878 INFO 4444 --- [connect_service] c.s.j.protocol.impl.TcpClientChannel : Stale reconnect task, aborting reconnect.
Below is our configuration for connecting to solace queues:
spring.jta.bitronix.connectionfactory.className=com.solacesystems.jms.SolXAConnectionFactoryImpl
spring.jta.bitronix.connectionfactory.driverProperties.host=smf://<<hostname>>:55555
spring.jta.bitronix.connectionfactory.driverProperties.VPN=<<vpn>>
spring.jta.bitronix.connectionfactory.driverProperties.authenticationScheme=AUTHENTICATION_SCHEME_GSS_KRB
spring.jta.bitronix.connectionfactory.driverProperties.KRBServiceName=HOST
In our service class we are just autowiring the object of jmsTemplate and publishing messages on the queue.
I went through few documentations and tried adding below configuration:
spring.jta.bitronix.connectionfactory.ignore-recovery-failures=true
But still I am facing the same issue. Any suggestions
====Edit
I face this issue only when I put my laptop in airplane mode and reconnect. If I just disconnect from VPN and connect back solace connection is getting reestablished
The SolXAConnectionFactory interface allows for you to tune the connect and reconnect parameters. Docs here.
You'll want to checkout these and maybe a few others. I suggest searching the javadoc for "retry" and "retries":
connectRetries
connectRetriesPerHost
connectTimeoutInMillies
reconnectRetries
I did more research and found the following helpful, would try it in my application : https://solace.community/discussion/917/why-won-t-my-solace-enterprise-application-reconnect-after-an-ha-failover To set it at JNDI, I think this should also be configured at SolAdmin -> JMS Administration -> connection factory -> Transport Properties.
After going through the various documentations and doing some hit and trials, below properties turn out too be useful. Hope it can help somebody:
spring.jta.bitronix.connectionfactory.driverProperties.reconnectRetries = -1
spring.jta.bitronix.connectionfactory.driverProperties.connectRetries = -1
I have a Storm cluster with 1 Nimbus, 4 Supervisors and 2 Zookeeper nodes. My Storm.yaml is as following:
storm.zookeeper.servers:
- "storage14"
- "storage15"
nimbus.seeds: ["storage01"]
#storm.local.hostname: "storage05"
supervisor.supervisors:
- "storage02"
- "storage03"
- "storage04"
- "storage05"
storm.local.dir: "/tmp/storm"
worker.childopts: "-Xmx%HEAP-MEM%m -XX:+PrintGCDetails -Xloggc:artifacts/gc.log -XX:+PrintGCDateStamps -XX:+PrintGCTimeStamps -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=artifacts/heapdump"
This storm.yaml file is used by both Nimbus and Supervisors. When Nimbus is started I have the storm.local.hostname commented out as is shown above.
However, when starting Supervisors on respective nodes, I uncomment the storm.local.hostname and set it to the hostname of the node on which the supervisor is being launched. For instance if I was launching the supervisor on storage05, the storm.yaml file would have the following additional config param:
storm.local.hostname: "storage05"
The problem is even though Nimubs is launched successfully and I can see it on the Storm UI, some supervisors do not seem to be able to connect to Nimbus. For instance of the 4 nodes I start supervisors on, Storm UI often shows only 2 of them connected. However, if I ssh in to these nodes and run jps, I can see that the supervisor process is running on ALL of these nodes.
The Supervisors at the nodes which do end up connecting are not the same always, so it is definitely not a problem with those specific nodes.
Another thing to notice is if I try to execute a topology on whatever nodes that got connected, it does not get registered by the cluster and I can not see that topology on the UI either.
What do you think might be causing this erratic behavior?
UPDATE:
Tail end of nimbus.log has the following lines
2017-01-25 00:04:25.216 o.a.s.s.o.a.z.ClientCnxn [WARN] Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
at org.apache.storm.shade.org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at org.apache.storm.shade.org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
2017-01-25 00:04:25.317 o.a.s.s.o.a.z.ClientCnxn [INFO] Opening socket connection to server storage15/192.168.140.195:2181. Will not attempt to authenticate using SASL (unknown error)
2017-01-25 00:04:25.317 o.a.s.s.o.a.z.ClientCnxn [WARN] Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
at org.apache.storm.shade.org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at org.apache.storm.shade.org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
2017-01-25 00:04:25.686 o.a.s.s.o.a.z.ClientCnxn [INFO] Opening socket connection to server storage15/192.168.140.195:2181. Will not attempt to authenticate using SASL (unknown error)
2017-01-25 00:04:25.686 o.a.s.s.o.a.z.ClientCnxn [WARN] Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
at org.apache.storm.shade.org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at org.apache.storm.shade.org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
2017-01-25 00:04:25.787 o.a.s.s.o.a.z.ClientCnxn [INFO] Opening socket connection to server storage14/192.168.140.194:2181. Will not attempt to authenticate using SASL (unknown error)
2017-01-25 00:04:25.787 o.a.s.s.o.a.z.ClientCnxn [WARN] Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:701)
at org.apache.storm.shade.org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at org.apache.storm.shade.org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
Your UPDATE (nimbus log) indicates that your Nimbus cannot connect Zookeeper cluster. Please check that Zookeeper cluster (storage14/storage15) is accessible from storage01 (not only node is accessible, but also do telnet to Zookeeper server via "telnet storage14 (and/or storage15) 2181").
When ZK connectivity issue is gone please try starting supervisor again.
I have installed an instance of Oracle Enterprise Manager Database Control 10g Release 10.2.0.1.0 in Centos 6.5
Attempts to launch the Enterprise Manager agent with the command:
emctl start dbconsole
After a few minutes, I said that failure.
Logs throw me:
emagent.log
07/01/2016 11:16:22 Thread-3086911712 Agent 10.1.0.4.1 Starting from
/u01/app/oracle/product/10.2.0 (00701) 07/01/2016 11:16:22
Thread-3086911712 EMAgent started successfully (00702)
emagent.trc
07/01/2016 11:16:22 Thread-3086911712 WARN http: snmehl_connect:
failed to connect (myserver: 3600): Connection refused (error = 111)
Thread-3086911712 07.01.2016 11:16:22 pingManager ERROR:
nmepm_pingReposURL: Can not connect to http: // myserver: 3600 / em /
upload /: retStatus = -32 07/01/2016 11:16:22 WARN command
Thread-3086911712: Subsystem Job Timeout Set at 600 seconds 07/01/2016
11:16:22 Thread-3086911712 WARN upload: Upload manager has not Failure
script: disabled 07/01/2016 11:16:22 Thread-3086911712 WARN upload:
Recovering left over xml files in upload directory 07/01/2016 11:16:22
Thread-3086911712 WARN upload: Recovered 109 left over xml files in
upload directory 07/01/2016 11:16:22 Thread-WARN 125 881 248 http:
snmehl_connect: failed to connect (myserver: 3600): Connection refused
(error = 111)
07.01.2016 11:16:22 125 881 248 Thread-ERROR command: nmejcn: http connection failed to http: // myserver: 3600 / em / upload /:
retStatus = -32 07/01/2016 11:16:22 Thread-3086911712 WARN metadata:
Metric Disk_Path does not Have any data columns 07/01/2016 11:16:22
Thread-3086911712 WARN metadata: Metric osm_diskGroupPolicies does not
Have any data columns
07.01.2016 11:16:22 132 508 576 Thread-upload ERROR: Error in uploadXMLFiles. Trying again in 60.00 seconds. 07/01/2016 11:16:22
Thread-27175840 WARN http: snmehl_connect: failed to connect
(myserver: 3600): Connection refused (error = 111) 07/01/2016 11:16:22
Thread-pingManager 27,175,840 ERROR: nmepm_pingReposURL: Can not
connect to http: // myserver: 3600 / em / upload /: retStatus = -32
emdctl.trc
07/01/2016 11:21:53 Thread-3086911712 WARN http: snmehl_connect:
failed to connect (myserver: 3873): Connection refused (error = 111)
I could already solve, I did the following:
set ORACLE_SID and ORACLE_HOME
emca -repos recreate
emca -config dbcontrol db
emctl start dbconsole
I was facing an issue with Bonecp 0.7.1 on a Playframework app using postgresql 9.2.4 on Heroku. It seems this version had a DB connection leak causing after several DB accesses the folllwing error :
[error] c.j.b.h.AbstractConnectionHook - Failed to acquire connection Sleeping for 1000ms and trying again. Attempts left: 1. Exception: null.Message:FATAL: too many connections for role "eonqhnjenuislk" Database warning
[error] c.j.b.PoolWatchThread - Error in trying to obtain a connection. Retrying in 1000ms
org.postgresql.util.PSQLException: FATAL: too many connections for role "eonqhnjenuislk"
at org.postgresql.core.v3.ConnectionFactoryImpl.readStartupMessages(ConnectionFactoryImpl.java:469) ~[postgresql-9.1-901.jdbc4.jar:na]
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:112) ~[postgresql-9.1-901.jdbc4.jar:na]
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66) ~[postgresql-9.1-901.jdbc4.jar:na]
at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125) ~[postgresql-9.1-901.jdbc4.jar:na]
at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30) ~[postgresql-9.1-901.jdbc4.jar:na]
at org.postgresql.jdbc3g.AbstractJdbc3gConnection.<init>(AbstractJdbc3gConnection.java:22) ~[postgresql-9.1-901.jdbc4.jar:na]
As every threads of the connection pool was acquired and retained, the application was not reachable anymore until I restarted it.
Then I heard that this issue was corrected in Bonecp 0.8.0, so I upgraded the lib. But the issue seems not to be completely fixed. In fact, now connection threads are not retained anymore what make the application reachable at anytime but sometimes a DB connection close suddenly... The app throws the following error causing an 500 error to the end users :
javax.persistence.PersistenceException: org.postgresql.util.PSQLException: This connection has been closed.
at com.avaje.ebeaninternal.server.transaction.TransactionManager.createTransaction(TransactionManager.java:331)
at com.avaje.ebeaninternal.server.core.DefaultServer.createServerTransaction(DefaultServer.java:2056)
at com.avaje.ebeaninternal.server.core.BeanRequest.createImplicitTransIfRequired(BeanRequest.java:58)
at com.avaje.ebeaninternal.server.core.PersistRequest.initTransIfRequired(PersistRequest.java:81)
at com.avaje.ebeaninternal.server.persist.DefaultPersister.executeSqlUpdate(DefaultPersister.java:146)
at com.avaje.ebeaninternal.server.core.DefaultServer.execute(DefaultServer.java:1928)
at com.avaje.ebeaninternal.server.core.DefaultServer.execute(DefaultServer.java:1935)
at com.avaje.ebeaninternal.server.core.DefaultSqlUpdate.execute(DefaultSqlUpdate.java:148)
at actor.PublicParkingPlacesActor$1.apply(PublicParkingPlacesActor.java:41)
at actor.PublicParkingPlacesActor$1.apply(PublicParkingPlacesActor.java:26)
at play.libs.F$Promise$PromiseActor.onReceive(F.java:425)
at akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:159)
at akka.actor.ActorCell.receiveMessage(ActorCell.scala:425)
at akka.actor.ActorCell.invoke(ActorCell.scala:386)
at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:230)
at akka.dispatch.Mailbox.run(Mailbox.scala:212)
at akka.dispatch.ForkJoinExecutorConfigurator$MailboxExecutionTask.exec(AbstractDispatcher.scala:502)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:262)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:975)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1478)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)
Caused by: org.postgresql.util.PSQLException: This connection has been closed.
at org.postgresql.jdbc2.AbstractJdbc2Connection.checkClosed(AbstractJdbc2Connection.java:714)
at org.postgresql.jdbc2.AbstractJdbc2Connection.setAutoCommit(AbstractJdbc2Connection.java:661)
at com.jolbox.bonecp.ConnectionHandle.setAutoCommit(ConnectionHandle.java:1292)
at play.api.db.BoneCPApi$$anon$1.onCheckOut(DB.scala:328)
at com.jolbox.bonecp.AbstractConnectionStrategy.postConnection(AbstractConnectionStrategy.java:75)
at com.jolbox.bonecp.AbstractConnectionStrategy.getConnection(AbstractConnectionStrategy.java:92)
at com.jolbox.bonecp.BoneCP.getConnection(BoneCP.java:553)
at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:131)
at play.db.ebean.EbeanPlugin$WrappingDatasource.getConnection(EbeanPlugin.java:146)
at com.avaje.ebeaninternal.server.transaction.TransactionManager.createTransaction(TransactionManager.java:297)
... 20 more
Thanks a lot for your help!
EDIT :
DB configuration :
db.default.isolation=READ_COMMITTED
db.default.partitionCount=2
db.default.maxConnectionsPerPartition=10
db.default.minConnectionsPerPartition=5
db.default.acquireIncrement=1
db.default.acquireRetryAttempts=2
db.default.acquireRetryDelay=5 seconds
db.default.connectionTimeout=10 second
db.default.idleMaxAge=10 minute
db.default.idleConnectionTestPeriod=5 minutes
db.default.initSQL="SELECT 1"
db.default.maxConnectionAge=1 hour
EDIT 2:
Here is the DB config I set according to this post Heroku/Play/BoneCp connection issues
These changes reduce the number of "This connection has been closed" issues, but I still get 1 or 2 of them perdays what makes some HTTP requests to fail. So the issue is still not fixed:
db.default.isolation=READ_COMMITTED
db.default.partitionCount=2
db.default.maxConnectionsPerPartition=10
db.default.minConnectionsPerPartition=5
db.default.acquireIncrement=1
db.default.acquireRetryAttempts=2
db.default.acquireRetryDelay=5 seconds
db.default.connectionTimeout=10 seconds
db.default.idleMaxAge=10 minutes
db.default.idleConnectionTestPeriod=30 seconds
db.default.initSQL="SELECT 1"
db.default.maxConnectionAge=30 minutes