I am facing an issue with persisting records for an Entity(YieldCurveArchive) with a field (reason) having a length equivalent to 2048 characters. Following are the scenarios and their results:
Upload new Entity with smaller reason fields: Works fine.
Upload Entity with reason fields
changed with larger (2048 characters) data set: Works fine. Again as
above. This is an update of the existing records.
Upload new Entity
with larger (2048 characters) reason fields: Fails.
I have also tried flushing the hibernate query buffer using entityManager.flush() but the above test results do not change.
I suspect there could be an issue with the buffering that Hibernate performs before actually doing a final insert into database:
Selection of the available records in database to compare with
existing dataset.
Insert being fired but only kept in the buffer of
hibernate to fire a bulk update.
Another select being fired for a
different client. Upon seeing another select, hibernate decides to
fire the insert into the database and then fails.
An excerpt of the trace from the failed logs:
2013-07-02 12:46:00,792 WARN [pool-1-thread-4]: org.hibernate.util.JDBCExceptionReporter - SQL Error: 1400, SQLState: 23000
2013-07-02 12:46:00,792 ERROR [pool-1-thread-4]: org.hibernate.util.JDBCExceptionReporter - ORA-01400: cannot insert NULL into ("CORE_TOTEM"."YC_MONTHLY_ARCHIVE"."PKEY")
2013-07-02 12:46:00,792 WARN [pool-1-thread-4]: org.hibernate.util.JDBCExceptionReporter - SQL Error: 1400, SQLState: 23000
2013-07-02 12:46:00,792 ERROR [pool-1-thread-4]: org.hibernate.util.JDBCExceptionReporter - ORA-01400: cannot insert NULL into ("CORE_TOTEM"."YC_MONTHLY_ARCHIVE"."PKEY")
2013-07-02 12:46:00,985 ERROR [pool-1-thread-4]: org.hibernate.event.def.AbstractFlushingEventListener - Could not synchronize database state with session
org.hibernate.exception.ConstraintViolationException: Could not execute JDBC batch update
at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:94)
at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:66)
at org.hibernate.jdbc.AbstractBatcher.executeBatch(AbstractBatcher.java:275)
at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:266)
at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:167)
at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:321)
at org.hibernate.event.def.DefaultAutoFlushEventListener.onAutoFlush(DefaultAutoFlushEventListener.java:64)
at org.hibernate.impl.SessionImpl.autoFlushIfRequired(SessionImpl.java:996)
at org.hibernate.impl.SessionImpl.list(SessionImpl.java:1141)
at org.hibernate.impl.QueryImpl.list(QueryImpl.java:102)
at org.hibernate.ejb.QueryImpl.getResultList(QueryImpl.java:65)
at markit.totem.dao.DaoHelper.list(DaoHelper.java:19)
at com.markit.totem.rates.yieldcurve.dao.YieldCurveArchiveDao.getArchivesMonthly(YieldCurveArchiveDao.java:172)
at com.markit.totem.rates.yieldcurve.dao.YieldCurveArchiveDao$$FastClassByCGLIB$$55ef1569.invoke(<generated>)
at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:149)
at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:628)
at com.markit.totem.rates.yieldcurve.dao.YieldCurveMonthlyArchiveDao$$EnhancerByCGLIB$$6aa04fe7.getArchivesMonthly(<generated>)
at com.markit.totem.rates.yieldcurve.results.upload.monthly.YieldCurveMonthlyArchivePersister.getExisting(YieldCurveMonthlyArchivePersister.java:38)
at com.markit.totem.rates.yieldcurve.results.upload.YieldCurveArchivePersister.persist(YieldCurveArchivePersister.java:68)
at com.markit.totem.rates.yieldcurve.results.upload.YieldCurveArchivePersister$$FastClassByCGLIB$$ac0db3fb.invoke(<generated>)
at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:149)
at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:628)
at com.markit.totem.rates.yieldcurve.results.upload.monthly.YieldCurveMonthlyArchivePersister$$EnhancerByCGLIB$$4b00cfa0.persist(<generated>)
at com.markit.totem.rates.yieldcurve.results.upload.YieldCurveResultsUploader.upload(YieldCurveResultsUploader.java:158)
at com.markit.totem.rates.yieldcurve.results.upload.YieldCurveResultsUploadTask.run(YieldCurveResultsUploadTask.java:53)
at com.markit.totem.workflow.WorkflowExecutor.executeWorkflowTask(WorkflowExecutor.java:258)
at com.markit.totem.workflow.WorkflowExecutor.executeSubWorkflow(WorkflowExecutor.java:227)
at com.markit.totem.workflow.WorkflowExecutor.access$000(WorkflowExecutor.java:17)
at com.markit.totem.workflow.WorkflowExecutor$1.run(WorkflowExecutor.java:72)
at markit.totem.dao.Transactionator.execute(Transactionator.java:19)
at markit.totem.dao.Transactionator$$FastClassByCGLIB$$c9204755.invoke(<generated>)
at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:149)
at org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:700)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:106)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:635)
at markit.totem.dao.Transactionator$$EnhancerByCGLIB$$4eec78e2.execute(<generated>)
at com.markit.totem.workflow.WorkflowExecutor.execute(WorkflowExecutor.java:83)
at com.markit.totem.workflow.WorkflowExecutor.execute(WorkflowExecutor.java:213)
at com.markit.totem.workflow.WorkflowManager$OldWorkflow.execute(WorkflowManager.java:218)
at com.markit.totem.workflow.WorkflowManager$1.run(WorkflowManager.java:119)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.sql.BatchUpdateException: ORA-01400: cannot insert NULL into ("CORE_TOTEM"."YC_MONTHLY_ARCHIVE"."PKEY")
at oracle.jdbc.driver.DatabaseError.throwBatchUpdateException(DatabaseError.java:367)
at oracle.jdbc.driver.OraclePreparedStatement.executeBatch(OraclePreparedStatement.java:9055)
at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeBatch(NewProxyPreparedStatement.java:1723)
at org.hibernate.jdbc.BatchingBatcher.doExecuteBatch(BatchingBatcher.java:70)
at org.hibernate.jdbc.AbstractBatcher.executeBatch(AbstractBatcher.java:268)
My initial thoughts were that it fails due to the encoding difference in database and default encoding from Hibernate.
Like underlying database uses Oracle and the NLS_CHARACTERSET is defined as:
NLS_CHARACTERSET AL32UTF8
Hence it stores, 1 character as 4 bytes and any special character as 6 bytes. My insert into the Oracle fails for 1001 but passes till i insert 1000 characters. However, if I do an update with more characters thats not the case, which makes it more confusing.
Any pointers would be of great help?
Related
I have configured a spring state machine with JPA config to an H2 Database.
I see the following error at run time
Caused by: org.h2.jdbc.JdbcSQLIntegrityConstraintViolationException: NULL not allowed for column "STATEACTIONS_ID"; SQL statement:
insert into state_action (JpaRepositoryState_id, exitActions_id) values (?, ?) [23502-214]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:508)
at org.h2.message.DbException.getJdbcSQLException(DbException.java:477)
I am very new in iseries/DB2.
We use V7R3. We have table that is generated every day by RPG program as physical file. For accessing the table data from java we use jt400.jar jdbc driver.
Most of the table queries work fine but some complex queries that are using "DENSE_RANK() OVER(ORDER BY" and "ROW_NUMBER() OVER(PARTITION BY" time to time hanging and causing CPU 100%. only killing the job on AS400 side is resolving the issue.
in the AS400 log I see:
Job 969954/QUSER/QZDASOINIT started on 02/21/19 at 09:36:46 in subsystem
QUSRWRK in QSYS. Job entered system on 02/21/19 at 09:36:46.
User USERXX from client X.X.X.X connected to server.
Use of function TIMESTAMP_FORMAT in QSYS2 not valid.
Use of function TIMESTAMP_FORMAT in QSYS2 not valid.
Data mapping error on member TABLE_NAME.
Data mapping error on member TABLE_NAME.
Data mapping error on member TABLE_NAME.
Data mapping error on member TABLE_NAME.
Value in date, time, or timestamp string not valid.
It looks similar as problem described in the Why am I getting a "[SQL0802] Data conversion of data mapping error" exception?
and probably the problem is related to invalid data stored to DATE type columns.
Looking to the DATE columns I see that some records displayed as <null> in SQuirrel SQL Client. Interesting that here are 2 different <null>'s returned by distinct query.
If I run
select distinct VARCHAR_FORMAT(DATE_COLUMN, 'YYYY/MM/DD') from TABLE_NAME
I get
0001/01/01
and
9999/12/31
for these <null>'s rows.
if I run Select * from TABLE where DATE_COLUMN is null I don't get any results. So I am not sure what kind of <null>'s is that.
Not sure if these records can cause an issue.
UPD: when I run
Select * from TABLE
I see errors in JDBC client log:
Warning: [SQL0181] Value in date, time, or timestamp string not valid.
SQLState: 01534
ErrorCode: 181
Warning: [SQL0181] Value in date, time, or timestamp string not valid.
SQLState: 01534
ErrorCode: 181Warning: [SQL0181] Value in date, time, or timestamp string not valid.
SQLState: 01534
ErrorCode: 181
Warning: [SQL0181] Value in date, time, or timestamp string not valid.
SQLState: 01534
ErrorCode: 181
Query 1 of 1, Rows read: 100, Elapsed time (seconds) - Total: 0.252, SQL query: 0.005, Reading results: 0.247
based on https://www.consolut.com/en/s/sap-ides-access/d/s/p/40/doc/XH-SQL0181/ it should be incorrect date somewhere in the table
The question is there any way to find and filter records that have "invalid" data (causing above exception in the log) from SQL side?
We are using entitymanager merge method to either update (if existing) or insert (if not existing) records in a certain table. What happens is if it is an update, there are no errors, but if it is insert, we see the following error from our logs:
Caused by: java.sql.SQLException: ORA-28001: the password has expired\n DSRA0010E: SQL State = 99999, Error Code = 28,001\n\tat oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:450)\n\tat oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:392)\n\tat oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:385)\n\tat oracle.jdbc.driver.T4CTTIfun.processError(T4CTTIfun.java:938)\n\tat oracle.jdbc.driver.T4CTTIoauthenticate.processError(T4CTTIoauthenticate.java:480)\n\tat oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:655
When we checked the datasource if connection successful - we get no password expired error.
We also tried manual insert to the table via oracle sql developer tool - no error
Can anyone advise why this is happening?
I'm runnig a Spring Boot batch with two different connections. An Oracle DB and a DB2 DB both using SimpleJDBCTemplate.
The problem happens when I make a query for the second time about 1 hour after the first one happens because I have 300,000 inserts before runs the second time.
The Oracle connection keeps alive for so many time that I believe it throws the IOException.
I'm thinking about closing the Oracle connection before those 300,000 inserts...
Here is the stacktrace:
Io exception: EDC8120I Connection ended abnormally.; nested exception
is java.sql.SQLException: Io exception: EDC8120I Connection ended
abnormally.
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:84)
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81)
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81)
at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:660)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:695)
at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:727)
Use batch insert by 1000 items in each iteration.
DBMS - oracle 11gr2
we recently upgraded the IDE and OS of a java console application that runs 10 times a month.
now None of our queries work properly since hibernate converts the letter i to İ which is not I. All insert statements and id fields affected negatively.
now in our queries
id -> İD not ID
insert -> İNSERT not INSERT
neither select statements nor inserts do not execute.
for example:
in the previous version Select id from table a is now sent to the db as
SELECT İD from table a ,
and we dont have such a column called İD at all.
stack trace is as follows
Internal Exception: java.sql.SQLSyntaxErrorException: ORA-00904: "İD": geçersiz belirleyici (invalid identifier)
Query: ReadObjectQuery(name="readObject" referenceClass=OrtakEntity sql="SELECT İD, ADİ, ALACAKLARİNDAHACİZVARMİ, ANNEADİ, FROM A WHERE (İD = ?)")
********************sistem toplam sure***********************0.0
********************Time unit toplam sure***********************0
Local Exception Stack:
Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.2.0.v20110202-r8913): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: java.sql.SQLSyntaxErrorException: ORA-00904: "İD": geçersiz belirleyici
Error Code: 904
at org.eclipse.persistence.exceptions.DatabaseException.sqlException(DatabaseException.java:333)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.basicExecuteCall(DatabaseAccessor.java:684)
at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeCall(DatabaseAccessor.java:526)
at org.eclipse.persistence.internal.session
I resolved the issue by changing the settings of the computer to English and United States as location. (still not the best approach but now the program runs)