Springboot Hikari Oracle RAC jdbc connection error - oracle

I'm getting error when deployingg project
INFO com.zaxxer.hikari.HikariDataSource - HikariPool-1 - Starting...
WARN o.s.b.a.j.DataSourceHealthIndicator - DataSource health check failed
java.lang.RuntimeException: Driver oracle.jdbc.OracleDriver claims to not accept jdbcUrl, jdbc-url:...
Connection String
jdbc:oracle:thin:
#(DESCRIPTION=(CONNECT_TIMEOUT=5)(TRANSPORT_CONNECT_TIMEOUT=3)(RETRY_COUNT=1)(ADDRESS=(PROTOCOL=TCP)(HOST=my2.org)(PORT=123))(ADDRESS=(PROTOCOL=TCP)(HOST=my1.org)(PORT=123))(CONNECT_DATA=(SERVICE_NAME=DG)))

Related

getting error Connection oracle.jdbc.driver.T4CConnection#1f193686 marked as broken because of SQLSTATE(08006), ErrorCode(17002)

**> I am getting exception after upgrading oracle11g to oracle19C, using hikari as connection pool and hibernate and spring boot. Please
help.
When using IN query with lower number of value list its working but with large numbers its giving error.
Error Recieved:-**
WARN 2022-05-20 11:56:03,250 [http-nio-8080-exec-3] HikariPool-1 - Connection oracle.jdbc.driver.T4CConnection#40896e8e marked as broken because of SQLSTATE(08006), ErrorCode(17002)java.sql.SQLRecoverableException: IO Error: Connection reset
Caused by: java.net.SocketException: Connection reset
at java.base/java.net.SocketInputStream.read(SocketInputStream.java:186)
at java.base/java.net.SocketInputStream.read(SocketInputStream.java:140)
at oracle.net.ns.Packet.receive(Packet.java:316)
at oracle.net.ns.DataPacket.receive(DataPacket.java:128)
at oracle.net.ns.NetInputStream.getNextPacket(NetInputStream.java:315)
at oracle.net.ns.NetInputStream.read(NetInputStream.java:257)
at oracle.net.ns.NetInputStream.read(NetInputStream.java:177)
at oracle.net.ns.NetInputStream.read(NetInputStream.java:95)
at oracle.jdbc.driver.T4CSocketInputStreamWrapper.readNextPacket(T4CSocketInputStreamWrapper.java:132)
at oracle.jdbc.driver.T4CSocketInputStreamWrapper.read(T4CSocketInputStreamWrapper.java:88)
at oracle.jdbc.driver.T4CMAREngineStream.unmarshalUB1(T4CMAREngineStream.java:395)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:410)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:269)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:655)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:270)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:91)
at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:807)
... 148 common frames omitted
WARN 2022-05-20 11:56:03,251 [http-nio-8080-exec-3] SQL Error: 17002, SQLState: 08006
ERROR 2022-05-20 11:56:03,251 [http-nio-8080-exec-3] IO Error: Connection reset
DEBUG 2022-05-20 11:56:03,251 [HikariPool-1 connection closer] HikariPool-1 - Closing connection oracle.jdbc.driver.T4CConnection#40896e8e: (connection is broken)
ERROR 2022-05-20 11:56:03,252 [http-nio-8080-exec-3] Application exception overridden by rollback exception
org.springframework.dao.DataAccessResourceFailureException: could not extract ResultSet; nested exception is org.hibernate.exception.JDBCConnectionException: could not extract ResultSet

Disabled ZooKeeper in bootstrap.yml, it still runs during testing

I am running a SpringBoot App. I have bootstrap-test.yml (located under src/test/resources/config), which looks like:
spring.cloud.config.enabled: false
spring.cloud.service-registry.auto-registration.enabled: false
spring.cloud.zookeeper.discovery.enabled: false
spring.cloud.zookeeper.discovery.register: false
pom:
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-zookeeper-discovery</artifactId>
<version>2.2.2.RELEASE</version>
</dependency>
However, when I try to run an integration test:
#ActiveProfiles("test")
#DirtiesContext
#SpringBootTest
public class TestClass......
ZooKeeper still attempts to connect "live":
16:02:14.752 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.memory.free=493MB
16:02:14.752 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.memory.max=8116MB
16:02:14.752 [main] INFO org.apache.zookeeper.ZooKeeper - Client environment:os.memory.total=610MB
16:02:14.778 [main] INFO o.a.c.f.imps.CuratorFrameworkImpl - Starting
16:02:14.783 [main] INFO org.apache.zookeeper.ZooKeeper - Initiating client connection, connectString=localhost:2181 sessionTimeout=60000 watcher=org.apache.curator.ConnectionState#76d828ff
16:02:14.788 [main] INFO o.apache.zookeeper.ClientCnxnSocket - jute.maxbuffer value is 4194304 Bytes
16:02:14.798 [main-SendThread(localhost:2181)] INFO org.apache.zookeeper.ClientCnxn - Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181. Will not attempt to authenticate using SASL (unknown error)
16:02:14.805 [main] INFO o.a.c.f.imps.CuratorFrameworkImpl - Default schema
16:02:16.805 [main-SendThread(localhost:2181)] WARN org.apache.zookeeper.ClientCnxn - Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused: no further information
at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:779)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:357)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1214)
I want to disable all zookeeper bootstrap actions for integration test. What gives, I'm frustrated and have no idea what is wrong: why is Zookeeper still trying to run when I disabled it? Thanks in advance.
this should work !
spring.cloud.zookeeper.enabled=false
for yaml,
spring:
cloud:
zookeeper:
enabled: false

Spring boot crashes randomly without errors

I have a Spring boot application in version 2.1.2.RELEASE. Once a month or few weeks application crashes without any specific information in logs.
Server where app is deploy is CentOS ver 7, app is launched by a systemd service.
Tried analysing heap dump, no luck, checked logs of postgres, tomcat etc. nothing there.
Stacktrace:
INFO ;read-5;523575402;2019-08-07 17:27:24,714;o.s.i.e.EventDrivenConsumer; Removing {logging-channel-adapter:_org.springframework.integration.errorLogger} as a subscriber to the 'errorChannel' channel
INFO ;read-5;523575418;2019-08-07 17:27:24,730;o.s.i.c.PublishSubscribeChannel; Channel 'application.errorChannel' has 0 subscriber(s).
INFO ;read-5;523575418;2019-08-07 17:27:24,730;o.s.i.e.EventDrivenConsumer; stopped _org.springframework.integration.errorLogger
INFO ;read-5;523575502;2019-08-07 17:27:24,814;o.s.s.c.ThreadPoolTaskScheduler; Shutting down ExecutorService 'taskScheduler'
INFO ;read-5;523575541;2019-08-07 17:27:24,853;o.s.s.c.ThreadPoolTaskExecutor; Shutting down ExecutorService 'applicationTaskExecutor'
INFO ;read-5;523575572;2019-08-07 17:27:24,884;o.s.o.j.LocalContainerEntityManagerFactoryBean; Closing JPA EntityManagerFactory for persistence unit 'default'
INFO ;read-5;523575596;2019-08-07 17:27:24,908;c.z.h.HikariDataSource; HikariPool-1 - Shutdown initiated...
INFO ;read-5;523575631;2019-08-07 17:27:24,943;c.z.h.HikariDataSource; HikariPool-1 - Shutdown completed.
WARN ;exec-8;523575668;2019-08-07 17:27:24,980;c.z.h.p.ProxyConnection; HikariPool-1 - Connection org.postgresql.jdbc.PgConnection#6ff70ea8 marked as broken because of SQLSTATE(08006), ErrorCode(0)
org.postgresql.util.PSQLException: An I/O error occurred while sending to the backend.
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:335)
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:143)
at org.postgresql.jdbc.PgPreparedStatement.executeQuery(PgPreparedStatement.java:106)
at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeQuery(ProxyPreparedStatement.java:52)
at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeQuery(HikariProxyPreparedStatement.java)
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.extract(ResultSetReturnImpl.java:60)
at org.hibernate.loader.Loader.getResultSet(Loader.java:2173)
at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1936)
at org.hibernate.loader.Loader.executeQueryStatement(Loader.java:1898)
WARN ;exec-6;523575677;2019-08-07 17:27:24,989;c.z.h.p.ProxyConnection; HikariPool-1 - Connection org.postgresql.jdbc.PgConnection#71c6647 marked as broken because of SQLSTATE(08006), ErrorCode(0)
org.postgresql.util.PSQLException: An I/O error occurred while sending to the backend.
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:335)
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365)
at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:143)
at org.postgresql.jdbc.PgPreparedStatement.executeQuery(PgPreparedStatement.java:106)
at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeQuery(ProxyPreparedStatement.java:52)
at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeQuery(HikariProxyPreparedStatement.java)
WARN ;exec-8;523575721;2019-08-07 17:27:25,033;o.h.e.j.s.SqlExceptionHelper; SQL Error: 0, SQLState: 08006
WARN ;exec-6;523575721;2019-08-07 17:27:25,033;o.h.e.j.s.SqlExceptionHelper; SQL Error: 0, SQLState: 08006
ERROR;exec-8;523575721;2019-08-07 17:27:25,033;o.h.e.j.s.SqlExceptionHelper; An I/O error occurred while sending to the backend.
ERROR;exec-6;523575721;2019-08-07 17:27:25,033;o.h.e.j.s.SqlExceptionHelper; An I/O error occurred while sending to the backend.
INFO ;exec-6;523575885;2019-08-07 17:27:25,197;o.h.e.i.DefaultLoadEventListener; HHH000327: Error performing load command : org.hibernate.exception.JDBCConnectionException: could not extract ResultSet
org.hibernate.exception.JDBCConnectionException: could not extract ResultSet
at org.hibernate.exception.internal.SQLStateConversionDelegate.convert(SQLStateConversionDelegate.java:115)
at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:42)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:113)
Has anyone got idea how can I what might be going on? Last crash was 3, 4 weeks before that.
That type of error happens when the database connection has been lost, which in turn can happen for a whole number of reasons, the most simple one being that it's being restarted after system updates or as part of a scheduled operation.
HikariCP should be able to recover according to the following link (provided that your HikariCP version is recent enough):
https://github.com/brettwooldridge/HikariCP/wiki/Rapid-Recovery
You may need to configure some properties in your pool. The full list of properties is available here:
https://github.com/brettwooldridge/HikariCP
I recommend you configure the following in HikariCP:
connectionTestQuery: will test the connection periodically, dropping dead connections. Use it in conjunction with socketTimeout in your PostgreSQL connection string
leakDetectionThreshold: will help with applications having connection leaks

PostgreSQL hibernate and spring

I'm trying to use Hibernate with Spring and PostgreSQL, but I have lots of errors like:
org.postgresql.Driver : Connection error
ERROR 3424 --- [ main] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Exception during pool initialization.
I have put postgreSQL.Driver in my libs but nothing to do error say there.
Edit:
2018-03-19 15:54:53.660 INFO 7640 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Starting...
2018-03-19 15:54:54.664 WARN 7640 --- [ main] unknown.jul.logger : ConnectException occurred while connecting to localhost:5432`
`at com.ttmik.back.MainKt.main(main.kt:19) ~[classes/:na]
2018-03-19 15:54:54.675 ERROR 7640 --- [ main] org.postgresql.Driver : Connection error:
org.postgresql.util.PSQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
Check if your PostgreSQL can be reached by using telnet. If you get something like:
telnet localhost 5432
Connecting To localhost...
Could not open connection to the host, on port 5432: Connection failed
then most likely need to change listen_addresses to * in postgresql.conf file as per this answer on serverfault.

HikariCP connection broken/unavailable woes

I am unable to understand the reason behind intermittent HikariCP Connection is not available.
From the logs, It doesn't look like a connection leak issue. A bigger problem is I am unable to predictably reproduce the error. Following is a sample log trace where the error starts while this gist contains it till the end.
2017-12-12T19:31:55.958Z DEBUG <> [HikariPool-1 housekeeper] com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Pool stats (total=10, active=1, idle=9, waiting=0)
2017-12-12T19:31:57.052Z WARN <> [main] c.zaxxer.hikari.pool.ProxyConnection - HikariPool-1 - Connection org.postgresql.jdbc.PgConnection#1de5f0ef marked as broken because of SQLSTATE(08P01), ErrorCode(0)
org.postgresql.util.PSQLException: Expected command status BEGIN, got EMPTY.
at org.postgresql.core.v3.QueryExecutorImpl$1.handleCommandStatus(QueryExecutorImpl.java:515)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2180)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:288)
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:430)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:356)
at org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:168)
at org.postgresql.jdbc.PgPreparedStatement.executeQuery(PgPreparedStatement.java:116)
at com.zaxxer.hikari.pool.ProxyPreparedStatement.executeQuery(ProxyPreparedStatement.java:52)
at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.executeQuery(HikariProxyPreparedStatement.java)
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.extract(ResultSetReturnImpl.java:70)
at org.hibernate.id.SequenceGenerator.generateHolder(SequenceGenerator.java:116)
at org.hibernate.id.SequenceHiLoGenerator.generate(SequenceHiLoGenerator.java:62)
at org.hibernate.event.internal.AbstractSaveEventListener.saveWithGeneratedId(AbstractSaveEventListener.java:101)
at org.hibernate.jpa.event.internal.core.JpaPersistEventListener.saveWithGeneratedId(JpaPersistEventListener.java:67)
... MORE DETAILS IN [GIST][1]
2017-12-12T19:31:57.067Z ERROR <> [main] c.o.r.s.t.RAUTwitterUserService - populateFromPayload: id=128, twitter_handle=non_local, exception=org.springframework.transaction.TransactionSystemException: Could not roll back JPA transaction; nested exception is javax.persistence.PersistenceException: unexpected error when rollbacking
2017-12-12T19:31:57.067Z INFO <> [main] c.o.r.s.t.TwitterCollectorService - RAU from RAU service: RAU(id=129, twitter=non_global)
2017-12-12T19:31:57.089Z DEBUG <> [HikariPool-1 connection adder] com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Added connection org.postgresql.jdbc.PgConnection#419dcf40

Resources