I am working on the spring batch application which inserts the new entry into BATCH_JOB_INSTANCE internally. Version of spring is 2.2.1 and database is Azure.
While running the job I am getting the below error. As suggested in one of the site, I enabled the SET IDENTITY_INSERT to ON even though I have not used any INSERT statements to BATCH_JOB_INSTANCE. But it is of no use.
Caused by: org.springframework.dao.DataIntegrityViolationException: PreparedStatementCallback; SQL [INSERT into BATCH_JOB_INSTANCE(JOB_INSTANCE_ID, JOB_NAME, JOB_KEY, VERSION) values (?, ?, ?, ?)]; Cannot insert explicit value for identity column in table 'BATCH_JOB_INSTANCE' when IDENTITY_INSERT is set to OFF.; nested exception is com.microsoft.sqlserver.jdbc.SQLServerException: Cannot insert explicit value for identity column in table 'BATCH_JOB_INSTANCE' when IDENTITY_INSERT is set to OFF.
at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:247) ~[spring-jdbc-5.2.1.RELEASE.jar:5.2.1.RELEASE]
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) ~[spring-jdbc-5.2.1.RELEASE.jar:5.2.1.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.translateException(JdbcTemplate.java:1443) ~[spring-jdbc-5.2.1.RELEASE.jar:5.2.1.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:633) ~[spring-jdbc-5.2.1.RELEASE.jar:5.2.1.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:862) ~[spring-jdbc-5.2.1.RELEASE.jar:5.2.1.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:917) ~[spring-jdbc-5.2.1.RELEASE.jar:5.2.1.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:922) ~[spring-jdbc-5.2.1.RELEASE.jar:5.2.1.RELEASE]
at org.springframework.batch.core.repository.dao.JdbcJobInstanceDao.createJobInstance(JdbcJobInstanceDao.java:120) ~[spring-batch-core-4.2.0.RELEASE.jar:4.2.0.RELEASE]
at org.springframework.batch.core.repository.support.SimpleJobRepository.createJobExecution(SimpleJobRepository.java:140) ~[spring-batch-core-4.2.0.RELEASE.jar:4.2.0.RELEASE]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_144]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_144]
It would be better if you post your Job configuration...
DataIntegrityViolationException clearly states that an attempt to insert or update data results in violation of an integrity constraint.
DataIntegrityViolationException Docs
While job configuration jobBuilderFactory.get(JOB_NAME).incrementer(new RunIdIncrementer())
to increment the run id and every run gets the unique id
Related
I have configured a spring state machine with JPA config to an H2 Database.
I see the following error at run time
Caused by: org.h2.jdbc.JdbcSQLIntegrityConstraintViolationException: NULL not allowed for column "STATEACTIONS_ID"; SQL statement:
insert into state_action (JpaRepositoryState_id, exitActions_id) values (?, ?) [23502-214]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:508)
at org.h2.message.DbException.getJdbcSQLException(DbException.java:477)
In spring-data-jdbc 2.3.2 with 2.6.4 data-jdbc starter. I am seeing the following output. Not clear if this is my error in a model layer or an issue with the framework.
This happens when the root aggregate tries to get updated because of a one-to-many reference modification.
As far as I can tell the SQL spec expects a SET statement here. This is the exception I am getting :
Caused by: org.springframework.jdbc.BadSqlGrammarException: PreparedStatementCallback; bad SQL grammar [UPDATE "tb_entity" WHERE "tb_entity"."field_id" = ?]; nested exception is org.postgresql.util.PSQLException: ERROR: syntax error at or near "WHERE"
Position: 21
Any suggestion is welcome.
Take a look at the foreign table primary key. Even though not strictly necessary to be in consistent state. Anyways, one of the statement of 3NF is that all records in a table must be uniquely identified not matter if they don't represent an entity perse. Data-jdbc uses this key for the relations with List, Set, Map.
https://docs.spring.io/spring-data/jdbc/docs/current/reference/html/#jdbc.entity-persistence.types
We are using spring-boot with spring-boot-starter-jooq. For some complex checks of constraints triggers are used which raise error with specific message. The exception is caught on persistence layer and converted into business exception.
The behavior has been changed in last spring boot versions 2.4.x. For example hsqldb with scherma:
CREATE TABLE tab1 (
K INT PRIMARY KEY
)^;
CREATE TRIGGER trig1 BEFORE INSERT ON tab1
BEGIN ATOMIC
SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT = 'some error';
END^;
and the code for insert into table:
#Transactional
public void withTriggerException() {
create.insertInto(DSL.table("tab1"))
.set(DSL.field("k"), 1)
.execute();
}
The type and the message of thrown exception differs from spring-boot version.
2.3.6, 2.3.9
org.springframework.jdbc.UncategorizedSQLException: jOOQ; uncategorized SQLException for SQL [insert into tab1 (k) values (cast(? as int))]; SQL state [45000]; error code [5800]; some error; nested exception is java.sql.SQLException: some error
2.4.0, 2.4.1, 2.4.2
java.lang.NullPointerException: null
2.4.3
org.jooq.exception.DataAccessException: SQL [insert into tab1 (k) values (?)]; Unspecified RuntimeException
at org.jooq_3.14.7.HSQLDB.debug(Unknown Source) ~[na:na]
at org.jooq.impl.Tools.translate(Tools.java:2892) ~[jooq-3.14.7.jar:na]
at org.jooq.impl.DefaultExecuteContext.exception(DefaultExecuteContext.java:730) ~[jooq-3.14.7.jar:na]
at org.springframework.boot.autoconfigure.jooq.JooqExceptionTranslator.handle(JooqExceptionTranslator.java:83) ~[spring-boot-autoconfigure-2.4.3.jar:2.4.3]
at org.springframework.boot.autoconfigure.jooq.JooqExceptionTranslator.exception(JooqExceptionTranslator.java:55) ~[spring-boot-autoconfigure-2.4.3.jar:2.4.3]
at org.jooq.impl.ExecuteListeners.exception(ExecuteListeners.java:274) ~[jooq-3.14.7.jar:na]
at org.jooq.impl.AbstractQuery.execute(AbstractQuery.java:390) ~[jooq-3.14.7.jar:na]
at org.jooq.impl.AbstractDelegatingQuery.execute(AbstractDelegatingQuery.java:119) ~[jooq-3.14.7.jar:na]
at com.example.demo.FooDAO.withTriggerException(DemoApplication.java:38) ~[classes/:na]
...
The behavior has been reproduced on hsqldb but is same on PostgreSQL. Another errors (duplicate key, foreign keys, etc.) are working as expected.
It is possible to obtain error message ("some error" in example) from trigger with spring-boot 2.4.x?
You probably ran into this issue:
https://github.com/jOOQ/jOOQ/issues/11304, which has been fixed in jOOQ 3.14.6 and 3.15.0.
Some background on the incompatible change in Spring that produced this NPE regression in jOOQ can be seen here:
https://github.com/spring-projects/spring-framework/issues/24064
The fix should be available in Spring Boot 2.5.0 and 2.4.3:
https://github.com/spring-projects/spring-boot/issues/25214
https://github.com/spring-projects/spring-boot/issues/25240
You should be able to access the JDBC SQLException from jOOQ's DataAccessException using DataAccessException.getCause(Class), e.g.
exception.getCause(SQLException.class);
But there's probably a better way to have the Spring exception translator provide you with the actual cause directly.
I have a Jhipster Spring boot in production and after a while it gives this error:
SQL: delete from jhi_persistent_audit_event where event_id=?
.
HHH000315: Exception executing batch [org.hibernate.StaleStateException: Batch update returned unexpected row count from update [0]; actual row count: 0; expected: 1], SQL: delete from jhi_persistent_audit_event where event_id=?
2020-03-01 12:00:00.132 ERROR 14354 --- [ms-scheduling-1] o.h.i.ExceptionMapperStandardImpl : HHH000346: Error during managed flush [Batch update returned unexpected row count from update [0]; actual row count: 0; expected: 1]
2020-03-01 12:00:00.137 ERROR 14354 --- [ms-scheduling-1] o.s.s.s.TaskUtils$LoggingErrorHandler : Unexpected error occurred in scheduled task.
Unexpected error occurred in scheduled task.
org.springframework.orm.ObjectOptimisticLockingFailureException: Batch update returned unexpected row count from update [0]; actual row count: 0; expected: 1; nested exception is org.hibernate.StaleStateException: Batch update returned unexpected row count from update [0]; actual row count: 0; expected: 1
at org.springframework.orm.jpa.vendor.HibernateJpaDialect.convertHibernateAccessException(HibernateJpaDialect.java:339)
at org.springframework.orm.jpa.vendor.HibernateJpaDialect.translateExceptionIfPossible(HibernateJpaDialect.java:254)
at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:537)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:746)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:714)
at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:534)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:305)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:689)
at com.gotop.nms.service.AuditEventService$$EnhancerBySpringCGLIB$$3c01613a.removeOldAuditEvents(<generated>)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:84)
at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
at org.springframework.scheduling.concurrent.ReschedulingRunnable.run(ReschedulingRunnable.java:93)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.hibernate.StaleStateException: Batch update returned unexpected row count from update [0]; actual row count: 0; expected: 1
at org.hibernate.jdbc.Expectations$BasicExpectation.checkBatched(Expectations.java:67)
at org.hibernate.jdbc.Expectations$BasicExpectation.verifyOutcome(Expectations.java:54)
at org.hibernate.engine.jdbc.batch.internal.BatchingBatch.checkRowCounts(BatchingBatch.java:149)
at org.hibernate.engine.jdbc.batch.internal.BatchingBatch.performExecution(BatchingBatch.java:124)
at org.hibernate.engine.jdbc.batch.internal.BatchingBatch.addToBatch(BatchingBatch.java:89)
at org.hibernate.persister.entity.AbstractEntityPersister.delete(AbstractEntityPersister.java:3498)
at org.hibernate.persister.entity.AbstractEntityPersister.delete(AbstractEntityPersister.java:3755)
at org.hibernate.action.internal.EntityDeleteAction.execute(EntityDeleteAction.java:99)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:604)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:478)
at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:356)
at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:39)
at org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1483)
at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:512)
at org.hibernate.internal.SessionImpl.flushBeforeTransactionCompletion(SessionImpl.java:3321)
at org.hibernate.internal.SessionImpl.beforeTransactionCompletion(SessionImpl.java:2517)
at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.beforeTransactionCompletion(JdbcCoordinatorImpl.java:447)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.beforeCompletionCallback(JdbcResourceLocalTransactionCoordinatorImpl.java:178)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.access$300(JdbcResourceLocalTransactionCoordinatorImpl.java:39)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransactionCoordinatorImpl.java:271)
at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:104)
at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:533)
... 22 common frames omitted
The database is Mysql
This exception maybe can happen while deleting a record by Id that does not exists at all. So how do I fix this is JHipster.
Where do you think this SQL is called ?
This occurs in removeOldAuditEvents method of AuditEventService which is annotated with #Transactionnal at class level.
This method is #Scheduled annotated and you have multiple instances of your app running. So, each day at same hour all your instances compete to purge events older than 30 days.
This is a classical case of batch jobs in multiple instances apps.
So, you have several alternatives:
select an instance responsible for purging events maybe with a spring profile
externalize the scheduling by exposing your purge method as an API endpoint (see AuditResource) correctly secured that you will call from a cron or any external scheduler and using an API gateway to route to only one instance
catch ObjectOptimisticLockingFailureException and ignore it in this method; in general it's not recommended but in this case I guess it is acceptable because one instance will succeed and this is what you want. Maybe configuring pessimistic locking would make sense.
implement a distributed lock either in database or in Hazelcast that you might already use for distributed caching
I am facing an issue with persisting records for an Entity(YieldCurveArchive) with a field (reason) having a length equivalent to 2048 characters. Following are the scenarios and their results:
Upload new Entity with smaller reason fields: Works fine.
Upload Entity with reason fields
changed with larger (2048 characters) data set: Works fine. Again as
above. This is an update of the existing records.
Upload new Entity
with larger (2048 characters) reason fields: Fails.
I have also tried flushing the hibernate query buffer using entityManager.flush() but the above test results do not change.
I suspect there could be an issue with the buffering that Hibernate performs before actually doing a final insert into database:
Selection of the available records in database to compare with
existing dataset.
Insert being fired but only kept in the buffer of
hibernate to fire a bulk update.
Another select being fired for a
different client. Upon seeing another select, hibernate decides to
fire the insert into the database and then fails.
An excerpt of the trace from the failed logs:
2013-07-02 12:46:00,792 WARN [pool-1-thread-4]: org.hibernate.util.JDBCExceptionReporter - SQL Error: 1400, SQLState: 23000
2013-07-02 12:46:00,792 ERROR [pool-1-thread-4]: org.hibernate.util.JDBCExceptionReporter - ORA-01400: cannot insert NULL into ("CORE_TOTEM"."YC_MONTHLY_ARCHIVE"."PKEY")
2013-07-02 12:46:00,792 WARN [pool-1-thread-4]: org.hibernate.util.JDBCExceptionReporter - SQL Error: 1400, SQLState: 23000
2013-07-02 12:46:00,792 ERROR [pool-1-thread-4]: org.hibernate.util.JDBCExceptionReporter - ORA-01400: cannot insert NULL into ("CORE_TOTEM"."YC_MONTHLY_ARCHIVE"."PKEY")
2013-07-02 12:46:00,985 ERROR [pool-1-thread-4]: org.hibernate.event.def.AbstractFlushingEventListener - Could not synchronize database state with session
org.hibernate.exception.ConstraintViolationException: Could not execute JDBC batch update
at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:94)
at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:66)
at org.hibernate.jdbc.AbstractBatcher.executeBatch(AbstractBatcher.java:275)
at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:266)
at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:167)
at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:321)
at org.hibernate.event.def.DefaultAutoFlushEventListener.onAutoFlush(DefaultAutoFlushEventListener.java:64)
at org.hibernate.impl.SessionImpl.autoFlushIfRequired(SessionImpl.java:996)
at org.hibernate.impl.SessionImpl.list(SessionImpl.java:1141)
at org.hibernate.impl.QueryImpl.list(QueryImpl.java:102)
at org.hibernate.ejb.QueryImpl.getResultList(QueryImpl.java:65)
at markit.totem.dao.DaoHelper.list(DaoHelper.java:19)
at com.markit.totem.rates.yieldcurve.dao.YieldCurveArchiveDao.getArchivesMonthly(YieldCurveArchiveDao.java:172)
at com.markit.totem.rates.yieldcurve.dao.YieldCurveArchiveDao$$FastClassByCGLIB$$55ef1569.invoke(<generated>)
at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:149)
at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:628)
at com.markit.totem.rates.yieldcurve.dao.YieldCurveMonthlyArchiveDao$$EnhancerByCGLIB$$6aa04fe7.getArchivesMonthly(<generated>)
at com.markit.totem.rates.yieldcurve.results.upload.monthly.YieldCurveMonthlyArchivePersister.getExisting(YieldCurveMonthlyArchivePersister.java:38)
at com.markit.totem.rates.yieldcurve.results.upload.YieldCurveArchivePersister.persist(YieldCurveArchivePersister.java:68)
at com.markit.totem.rates.yieldcurve.results.upload.YieldCurveArchivePersister$$FastClassByCGLIB$$ac0db3fb.invoke(<generated>)
at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:149)
at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:628)
at com.markit.totem.rates.yieldcurve.results.upload.monthly.YieldCurveMonthlyArchivePersister$$EnhancerByCGLIB$$4b00cfa0.persist(<generated>)
at com.markit.totem.rates.yieldcurve.results.upload.YieldCurveResultsUploader.upload(YieldCurveResultsUploader.java:158)
at com.markit.totem.rates.yieldcurve.results.upload.YieldCurveResultsUploadTask.run(YieldCurveResultsUploadTask.java:53)
at com.markit.totem.workflow.WorkflowExecutor.executeWorkflowTask(WorkflowExecutor.java:258)
at com.markit.totem.workflow.WorkflowExecutor.executeSubWorkflow(WorkflowExecutor.java:227)
at com.markit.totem.workflow.WorkflowExecutor.access$000(WorkflowExecutor.java:17)
at com.markit.totem.workflow.WorkflowExecutor$1.run(WorkflowExecutor.java:72)
at markit.totem.dao.Transactionator.execute(Transactionator.java:19)
at markit.totem.dao.Transactionator$$FastClassByCGLIB$$c9204755.invoke(<generated>)
at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:149)
at org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:700)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:106)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:635)
at markit.totem.dao.Transactionator$$EnhancerByCGLIB$$4eec78e2.execute(<generated>)
at com.markit.totem.workflow.WorkflowExecutor.execute(WorkflowExecutor.java:83)
at com.markit.totem.workflow.WorkflowExecutor.execute(WorkflowExecutor.java:213)
at com.markit.totem.workflow.WorkflowManager$OldWorkflow.execute(WorkflowManager.java:218)
at com.markit.totem.workflow.WorkflowManager$1.run(WorkflowManager.java:119)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.sql.BatchUpdateException: ORA-01400: cannot insert NULL into ("CORE_TOTEM"."YC_MONTHLY_ARCHIVE"."PKEY")
at oracle.jdbc.driver.DatabaseError.throwBatchUpdateException(DatabaseError.java:367)
at oracle.jdbc.driver.OraclePreparedStatement.executeBatch(OraclePreparedStatement.java:9055)
at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeBatch(NewProxyPreparedStatement.java:1723)
at org.hibernate.jdbc.BatchingBatcher.doExecuteBatch(BatchingBatcher.java:70)
at org.hibernate.jdbc.AbstractBatcher.executeBatch(AbstractBatcher.java:268)
My initial thoughts were that it fails due to the encoding difference in database and default encoding from Hibernate.
Like underlying database uses Oracle and the NLS_CHARACTERSET is defined as:
NLS_CHARACTERSET AL32UTF8
Hence it stores, 1 character as 4 bytes and any special character as 6 bytes. My insert into the Oracle fails for 1001 but passes till i insert 1000 characters. However, if I do an update with more characters thats not the case, which makes it more confusing.
Any pointers would be of great help?