I'm trying to set up a simple test in Apache Nifi to connect to an existing PostgreSQL instance. I'm able to connect outside of nifi using other tools like dBeaver, and am fairly sure my connection string is proper. I have tried putting the postgresql jdbc driver in all sorts of places, but still keep seeing the "No suitable driver" error. I'll include some screenshots of my DBCPConnectionPool controller as well as my stack traces.
I have seen other posts like this, but none of them seem to lead to any solutions for me. Any help is appreciated.
Stack Trace
19-11-05 23:50:09,933 ERROR [Timer-Driven Process Thread-2] o.a.nifi.processors.standard.ExecuteSQL ExecuteSQL[id=3d68fb42-016e-1000-0ea4-abcc7dcc2e48] Unable to execute SQL select query select * from records; due to org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException: Cannot create JDBC driver of class 'org.postgresql.Driver' for connect URL 'jdbc:postgres://salt.db.elephantsql.com:5432/oickotoy'. No FlowFile to route to failure: org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException: Cannot create JDBC driver of class 'org.postgresql.Driver' for connect URL 'jdbc:postgres://salt.db.elephantsql.com:5432/oickotoy'
org.apache.nifi.processor.exception.ProcessException: java.sql.SQLException: Cannot create JDBC driver of class 'org.postgresql.Driver' for connect URL 'jdbc:postgres://salt.db.elephantsql.com:5432/oickotoy'
at org.apache.nifi.dbcp.DBCPConnectionPool.getConnection(DBCPConnectionPool.java:442)
at org.apache.nifi.dbcp.DBCPService.getConnection(DBCPService.java:49)
at sun.reflect.GeneratedMethodAccessor609.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:87)
at com.sun.proxy.$Proxy91.getConnection(Unknown Source)
at org.apache.nifi.processors.standard.AbstractExecuteSQL.onTrigger(AbstractExecuteSQL.java:223)
at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1176)
at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:213)
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.sql.SQLException: Cannot create JDBC driver of class 'org.postgresql.Driver' for connect URL 'jdbc:postgres://salt.db.elephantsql.com:5432/oickotoy'
at org.apache.commons.dbcp2.DriverFactory.createDriver(DriverFactory.java:75)
at org.apache.commons.dbcp2.BasicDataSource.createConnectionFactory(BasicDataSource.java:472)
at org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:538)
at org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:753)
at org.apache.nifi.dbcp.DBCPConnectionPool.getConnection(DBCPConnectionPool.java:438)
... 19 common frames omitted
Caused by: java.sql.SQLException: No suitable driver
at org.apache.commons.dbcp2.DriverFactory.createDriver(DriverFactory.java:68)
... 23 common frames omitted
What solved this issue for me was oddly enough deleting the Database connection URL (jdbc:postgresql://.....), applying the empty connection string to the controller service, then re-typing the connection string and then applying the now re-typed connection string to the controller service.
Seems like some kind of special character has caused this hiccup
What ended up solving this in the end was a version problem. I was apparently working on a dev release of Nifi that this was broken in. After I upgraded to the latest stable release, my problem went away.
Related
In development environment, spring boot with hikari, jdbc connection is unstable, if idle for some time, then call api again, it will fail(guess the network unstable cause it, because in production environment is ok)
2022-03-08 12:13:35.571 [http-nio-9090-exec-6] WARN com.zaxxer.hikari.pool.PoolBase - HikariPool-1 - Failed to validate connection com.mysql.cj.jdbc.ConnectionImpl#72415749 (No operations allowed after connection closed.). Possibly consider using a shorter maxLifetime value.
### The error occurred while executing a query
### Cause: org.springframework.jdbc.CannotGetJdbcConnectionException: Failed to obtain JDBC Connection; nested exception is java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not available, request timed out after 30038ms.] with root cause
com.mysql.cj.exceptions.ConnectionIsClosedException: No operations allowed after connection closed.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:61)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:105)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:151)
at com.mysql.cj.NativeSession.checkClosed(NativeSession.java:1209)
at com.mysql.cj.jdbc.ConnectionImpl.checkClosed(ConnectionImpl.java:567)
at com.mysql.cj.jdbc.ConnectionImpl.setNetworkTimeout(ConnectionImpl.java:2484)
at com.zaxxer.hikari.pool.PoolBase.setNetworkTimeout(PoolBase.java:550)
at com.zaxxer.hikari.pool.PoolBase.isConnectionAlive(PoolBase.java:165)
at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:179)
at com.zaxxer.hikari.pool.HikariPool.getConnection(HikariPool.java:155)
at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:128)
at org.springframework.jdbc.datasource.DataSourceUtils.fetchConnection(DataSourceUtils.java:157)
but next time it will ok.
Because it is development environment, no performance requirements, so I want to disable connection pool, that is every time use connection just create a new connection.
SO how to config spring.datasource.hikari.XXX to disable connection pool and create new jdbc connection every time when use?
You can use different datasource as SimpleDriverDataSource
as of Spring Boot 2.3.x, the following works out of the box with no need to exclude anything:
spring.datasource.type=org.springframework.jdbc.datasource.SimpleDriverDataSource
Also you can check MySQL recommended settings
As a database we use PostgreSQL. Application uses spring libraries, and transaction is managed by spring. Transactional annotation is used correctly, where it is needed as read-only, and where it is used with write abilities. But in logs we occasinally see exception
2020-12-30 11:39:13.513 [jmsContainer-8] ERROR o.h.e.jdbc.spi.SqlExceptionHelper - ERROR: cannot execute nextval() in a read-only transaction
2020-12-30 11:39:13.515 [jmsContainer-8] ERROR o.s.t.s.TransactionSynchronizationUtils - TransactionSynchronization.afterCompletion threw exception
javax.persistence.PersistenceException: org.hibernate.exception.GenericJDBCException: could not extract ResultSet
at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:147)
at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:155)
at org.hibernate.internal.ExceptionConverterImpl.convert(ExceptionConverterImpl.java:162)
at org.hibernate.internal.SessionImpl.firePersist(SessionImpl.java:780)
at org.hibernate.internal.SessionImpl.persist(SessionImpl.java:758)
at jdk.internal.reflect.GeneratedMethodAccessor2794.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.springframework.orm.jpa.SharedEntityManagerCreator$SharedEntityManagerInvocationHandler.invoke(SharedEntityManagerCreator.java:301)
at com.sun.proxy.$Proxy226.persist(Unknown Source)
Connections to PostgreSQL is provided through pg_bouncer, and pool_mode=sesssion, the server_reset_query=DISCARD ALL.
We tried restarting the pg_bouncer, though maybe some connection by default set read-only and being kept alive, but it did not help.
Any thoughts?
I am getting the Unable to detect database type for Sybase Datasource configured in my spring boot application.
Spring boot version: 1.5.3
application.properties
spring.datasource.url=jdbc:jtds:sybase://db-server-name:5555/DBNAME
spring.datasource.username=db_user
spring.datasource.password=db_password
spring.datasource.driver-class-name=net.sourceforge.jtds.jdbc.Driver
spring.jpa.database=sybase
spring.jpa.database-platform=org.hibernate.dialect.SybaseDialect
Debug Information:
// line number 75 in AbstractDatabaseInitializer class
String productName = JdbcUtils.commonDatabaseName(JdbcUtils
.extractDatabaseMetaData(this.dataSource, "getDatabaseProductName")
.toString());
// productName is returned as Sybase from JdbcUtils.extractDatabaseMetaData
DatabaseDriver databaseDriver = DatabaseDriver.fromProductName(productName);
// but there is no databaseDriver found and is returned as UNKNOWN.
// There is no Sybase database configured in DatabaseDriver enum
There is no Sybase datasource configured in DatabaseDriver enum. Can you quickly help to fix this issue ?
Error Stack Trace for reference:
Caused by: java.lang.IllegalStateException: Unable to detect database type
at org.springframework.boot.autoconfigure.AbstractDatabaseInitializer.getDatabaseName(AbstractDatabaseInitializer.java:80)
at org.springframework.boot.autoconfigure.batch.BatchDatabaseInitializer.getDatabaseName(BatchDatabaseInitializer.java:54)
at org.springframework.boot.autoconfigure.AbstractDatabaseInitializer.initialize(AbstractDatabaseInitializer.java:61)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleElement.invoke(InitDestroyAnnotationBeanPostProcessor.java:366)
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor$LifecycleMetadata.invokeInitMethods(InitDestroyAnnotationBeanPostProcessor.java:311)
at org.springframework.beans.factory.annotation.InitDestroyAnnotationBeanPostProcessor.postProcessBeforeInitialization(InitDestroyAnnotationBeanPostProcessor.java:134)
... 17 common frames omitted
It works for me. Just put this property in application.properties file. I was facing issue with mysql so I passed mysql here if you are using some other database just pass that(for example: schema-sybase.sql)
spring.batch.schema=classpath:org/springframework/batch/core/schema-mysql.sql
It looks like our detection algorithm is broken with MySQL so I've created this issue to double check it.
In the meantime, please specify the path to your initalization script, this will prevent Spring Boot to attempt to auto-detect it.
I am trying to configure WSO2 API Manager 2.0.0 with Enterprise-db Advance Server (postgres) 9.5.
I have configured Postgres JDBC Driver (postgresql-9.4.1212.jre7.jar) with it and configured all required data-sources required for WSO2-AM.
I am getting following error when start wso2-am server, please advise what is wrong here.
Caused by: org.wso2.carbon.user.core.UserStoreException: DB error occurred while checking is existing domain : PRIMARY & tenant id : -1234
Caused by: java.sql.SQLException: Uncaught underlying exception.
Caused by: java.lang.NullPointerException: tuples must be non-null
at org.postgresql.jdbc.PgResultSet.<init>(PgResultSet.java:147)
at org.postgresql.jdbc.PgStatement.createResultSet(PgStatement.java:161)
at org.postgresql.jdbc.PgStatement$StatementResultHandler.handleResultRows(PgStatement.java:213)
at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2037)
at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:291)
at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:432)
at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:358)
at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:305)
at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:291)
at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:269)
at org.postgresql.jdbc.PgConnection.execSQLUpdate(PgConnection.java:480)
at org.postgresql.jdbc.PgConnection.getTransactionIsolation(PgConnection.java:850)
Please Note: When i try to configure EDB-JDBC driver (edb-jdbc17.jar) with it gives different error. Caused by: java.lang.Exception: Unsupported database: EnterpriseDB. Database will not be created automatically by the WSO2 Registry. Please create the database using appropriate database scripts for the database.
Creating db with scripts did not help.
Below Exception is coming in specific server, issue is not consistent.
org.springframework.jdbc.support.MetaDataAccessException: Error while extracting DatabaseMetaData; nested exception is java.sql.SQLException:
Connection is not associated with a managed connection.org.jboss.resource.adapter.jdbc.jdk6.WrappedConnectionJDK6#49828f4c
at org.springframework.jdbc.support.JdbcUtils.extractDatabaseMetaData(JdbcUtils.java:296)
at org.springframework.jdbc.support.JdbcUtils.extractDatabaseMetaData(JdbcUtils.java:320)
at org.springframework.jdbc.support.SQLErrorCodesFactory.getErrorCodes(SQLErrorCodesFactory.java:213)
at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.setDataSource(SQLErrorCodeSQLExceptionTranslator.java:141)
at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.<init>(SQLErrorCodeSQLExceptionTranslator.java:104)
at org.springframework.jdbc.support.JdbcAccessor.getExceptionTranslator(JdbcAccessor.java:99)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:969)
at org.springframework.jdbc.core.JdbcTemplate.call(JdbcTemplate.java:1003)
at org.springframework.jdbc.object.StoredProcedure.execute(StoredProcedure.java:144)
at com.watsons.tfo.sp.dao.SPSaveTransactionDetail.execute(SupplierPromotionDaoImpl.java:3354)
at com.watsons.tfo.sp.dao.SupplierPromotionDaoImpl.savePromoTransaction(SupplierPromotionDaoImpl.java:422)
at com.watsons.tfo.sp.service.SupplierPromotionServiceImpl.savePromoTransaction(SupplierPromotionServiceImpl.java:158)
at sun.reflect.GeneratedMethodAccessor1010.invoke(Unknown Source)
--
org.springframework.dao.RecoverableDataAccessException: CallableStatementCallback; SQL [{call PKG_TFO_SUPPLIER_PROMOTION.PR_INS_SAVE_TRANSACTION_DET(?, ?)}]; No more data to read from socket; nested exception is java.sql.SQLRecoverableException: No more data to read from socket
at com.watsons.tfo.sp.service.SupplierPromotionServiceImpl.savePromoTransaction(SupplierPromotionServiceImpl.java:160)
at sun.reflect.GeneratedMethodAccessor1010.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:110)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at $Proxy206.savePromoTransaction(Unknown Source)
at com.watsons.tfo.sp.controller.SupplierPromoTransactionController.saveSupplierPromoTransaction(SupplierPromoTransactionController.java:1077)
at com.watsons.tfo.sp.controller.SupplierPromoTransactionController$$FastClassByCGLIB$$8994f9f9.invoke(<generated>)
First i thought the is because of stale connection, So I have added stale connection checker in the oracle-ds.xml. But again the issue is came up
Please help me to resolve this.
Disabling the CachedConnection Manager
<Valve className="org.jboss.web.tomcat.tc5.jca.CachedConnectionValve"
cachedConnectionManagerObjectName="jboss.jca:service=CachedConnectionManager"
transactionManagerObjectName="jboss:service=TransactionManager" />
The above tag is mentioned in server.xml. this cachedconnectionvalve should be turned off in production server.
There are two main configuration parameters that have a direct effect on performance and scalability: cached connection manager and HTTP session replication.
The CachedConnectionManager is configured to be in the servlet container in debug mode. It's also configured in the production configuration but with debug mode off.
I commented the CachedConnectionValve at end of the server.xml file.
Also commented the CachedConnectionManager in META-INF/jboss-service.xml.
The "Connection is not associated with a managed connection" is gone and application is running fine.
Update:
Found one more log for the same error in the oracle log files...
Oracle version: Oracle Database 11.2.0.2
i.e XMLSEQUENC/EEXTRACT FAILS WITH ORA-7445[__INTEL_NEW_MEMCPY()+5395]
This is the bug with the XML Processing in oracle DB version, Then
After instructed by oracle people, Updated the patch (11666959) for oracle .
Same is mentioned in oracle web site.
11666959 is required for the release utility; XMLSEQUENC/EEXTRACT FAILS WITH ORA-7445[__INTEL_NEW_MEMCPY()+5395] IN 11.2.0.2