JdbcSQLSyntaxErrorException: Sequence "SYSTEM_SEQUENCE_*" not found - spring

I am trying to recover an H2 database that seems corrupted by using H2 recovery script (H2 v.1.4.199, and also .197 and .200), like this:
java -cp h2-1.4.200.jar org.h2.tools.RunScript -url "jdbc:h2:file:./testdb_recovered" -user test -password test -script testdb.h2.sql -showResults
Following error occurrs:
CREATE CACHED TABLE PUBLIC.VECTORS_DATA(
ID BIGINT DEFAULT (NEXT VALUE FOR PUBLIC.SYSTEM_SEQUENCE_08DF5748_291A_46FD_9B83_818A3975DAF2) NOT NULL NULL_TO_DEFAULT SEQUENCE PUBLIC.SYSTEM_SEQUENCE_08DF5748_291A_46FD_9B83_818A3975DAF2 SELECTIVITY 100,
NAME VARCHAR(256) NOT NULL SELECTIVITY 1,
WIDTH INT SELECTIVITY 1,
HEIGHT INT SELECTIVITY 1,
DATA_SIZE INT SELECTIVITY 4
);Exception in thread "main" org.h2.jdbc.JdbcSQLSyntaxErrorException: Sequence "SYSTEM_SEQUENCE_08DF5748_291A_46FD_9B83_818A3975DAF2" not found; SQL statement:
CREATE CACHED TABLE PUBLIC.VECTORS_DATA(
ID BIGINT DEFAULT (NEXT VALUE FOR PUBLIC.SYSTEM_SEQUENCE_08DF5748_291A_46FD_9B83_818A3975DAF2) NOT NULL NULL_TO_DEFAULT SEQUENCE PUBLIC.SYSTEM_SEQUENCE_08DF5748_291A_46FD_9B83_818A3975DAF2 SELECTIVITY 100,
NAME VARCHAR(256) NOT NULL SELECTIVITY 1,
WIDTH INT SELECTIVITY 1,
HEIGHT INT SELECTIVITY 1,
DATA_SIZE INT SELECTIVITY 4
) [90036-199]
The "VECTORS_DATA" table corresponds to a Java entity used in a Spring project:
#Entity
#Table(name = "vectors_data")
public class VectorData {
/**
* Database id.
*/
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
/**
* Vector's data byte array.
*/
#javax.persistence.Transient
private byte[] data;
/**
* Vector's data byte array size. Used for persistency.
*/
#Column(name = "dataSize")
private int dataSize;
/**
* Vector's width in internal units.
*/
#Column(name = "width")
private int widthInternalUnit;
/**
* Vector's height in internal units.
*/
#Column(name = "height")
private int heightInternalUnit;
/**
* Vector's name.
*/
#Column(name = "name")
private String name;
H2 code statuses suggest that a sequence is trying to be accessed which does not exist. I am not sure how to proceed, although I've tried recovering it by using multiple versions, no help. Is there a way to debug recovery process more efficiently? Because I see no problem with the entity itself. Thanks!

After a bit of struggle, I found a solution. Since H2 complained there was no sequence to be found, I opened the generated SQL file and found the code block that created the table in question (VectorData). Added a line above it that creates a sequence which was causing the error, like this:
CREATE SEQUENCE <sequence_name>;
Ran the command to create a db with:
java -cp h2-1.4.200.jar org.h2.tools.RunScript -url "jdbc:h2:file:./testdb_recovered" -user test -password test -script testdb.h2.sql -showResults
Started the config via IntelliJ and voila, data was there.

Related

Sorting in Java | How to get value from other field of entity if the current field is null while sorting entity list in java?

Context :
Consider following entity in spring boot project,
#Entity
public class Transaction{
#Id
private Integer tId;
private Integer amount;
private LocalDate invoiceDate;
private LocalDate tDate;
}
A method which creates object of transaction, doesn't set tDate.
There is an independent method which sets tDate.
So, in the database some entries don't have 'tDate'.
Problem :
Now, I want to show all the entries in database sorted using tDate but, for those entries which does not contain tDate it should consider invoiceDate.
Sample Database entries
t_id
amount
invoice_date
t_date
1
1200
2/3/2022
4/3/2022
2
2434
5/3/2022
6/3/2022
3
234
3/3/2022
NULL
Sample expected Output
[[tId = 3, amount = 234, invoiceDate = 3/3/2022, tDate = NULL]
[tId = 1, amount = 1200, invoiceDate = 2/3/2022, tDate = 4/3/2022]
[tId = 2, amount = 2434, invoiceDate = 5/3/2022, tDate = 6/3/2022]]
Note : Highlighted dates above are the dates considered for sorting.
What I tried
I tried to use the combination of nullsLast(Comparator.naturalOrder()) and thenComparing() methods of Comparator.comparing() but it doesn't give the expected output. It sorts the entries withtDate and entries without tDate separately.
Thank you in advance for any help!!
Also, I'm using MongoRepository in repository layer, so I can use the Sort object as well.
Thank you!

Spring Boot + Hibernate - Insert query getting slow down

I am working on one spring boot application. Here I have 100,000 records that are inserted into db by different process. and its inserting one by one. I can't do batch insert.
So in starting some of the task performing well and not taking too much time ,but application process some and database is growing slowly - 2, insert time is increasing.
How can I speed up the process or avoid to get it slow?
The quickest way for inserts would be to use a prepared Statement.
Inject the jdbcTemplate and use its batchUpdate method and set the batch size. It's lightning fast.
If you think you cannot use the batch insert, which is hard for me to understand, then set the batch size to 1.
However, the most optimal batch size is certainly larger than that and depends on the insert statement. You have to experiment a bit with it.
Here an example for you with a class called LogEntry. Substitute class, table, columns and attributes by your class, table, columns and attributes and place it into your repository implementation.
Also make sure you set the application properties as mentioned here https://stackoverflow.com/a/62414315/12918872
Regarding the id Generator either set a sequence id generator (also shown in that link) or like in my case, just generate it on your own by asking for the maxId of your table at the beginning and then counting up.
#Autowired
private JdbcTemplate jdbcTemplate;
public void saveAllPreparedStatement2(List<LogEntry> logEntries) {
int batchSize = 2000;
int loops = logEntries.size() / batchSize;
for (int j = 0; j <= loops; j++) {
final int x = j;
jdbcTemplate.batchUpdate("INSERT INTO public.logentries(\r\n"
+ " id, col1, col2, col3, col4, col5, col6)\r\n"
+ " VALUES (?, ?, ?, ?, ?, ?, ?);\r\n", new BatchPreparedStatementSetter() {
public void setValues(PreparedStatement ps, int i) throws SQLException {
int counter = x * batchSize + i;
if (counter < logEntries.size()) {
LogEntry logEntry = logEntries.get(counter);
ps.setLong(1, (long) logEntry.getId());
ps.setString(2, (String) logEntry.getAttr1());
ps.setInt(3, (int) logEntry.getAttr2());
ps.setObject(4, logEntry.getAttr3(), Types.INTEGER);
ps.setLong(5, (long) logEntry.getAttr4());
ps.setString(6, (String) logEntry.getAttr5());
ps.setObject(7, logEntry.getAttr6(), Types.VARCHAR);
}
}
public int getBatchSize() {
if (x * batchSize == (logEntries.size() / batchSize) * batchSize) {
return logEntries.size() - (x * batchSize);
}
return batchSize;
}
});
}
}
Some advices for you :
It is not normal if you say the inserting time is getting increasing if more records are inserted. From my experience , most probably it is due to some logic bugs in your program such that you are processing more unnecessary data when you are inserting more records. So please revise your inserting logic first.
Hibernate cannot batch insert entities if the entity are using IDENTITY to generate is ID . You have to change it to use SEQUENCE to generate the ID with the pooled or pooled-lo algorithm.
Make sure you enable the JDBC batching feature in the hibernate configuration
If you are using PostgreSQL , you can add reWriteBatchedInserts=true in the JDBC connection string which will provide 2-3x performance gain.
Make sure each transaction will insert a batch of entities and then commit but not each transaction only insert one entity.
For more details about points (2), (3) and (4) , you can refer to my previous answers at this.

Hibernate Envers revision number order

I used Envers to audit my entities. The DB used is Oracle 11g. For the revision number of the revision entity, I used a oracle sequence. But since the oracle sequence does not guarantee monotonic increasing of each call to nextval, In my revision table, I got something like this,
We can see that the rev number is not monotonically increasing. In the audit table of my entity, I have this:
By chaining the records together with the rev number to form a timeline, I got :
24 --> 1302 --> 1303 --> 1355 --> 1304 --> 1356 --> 1357 --> 1305 -->1358 --> null
In fact, I have a cluster of 2 servers. They both can persist data into the DB. I suspect that this has relation to the problem of order in revnumber. Because of this problem. Query like MyEntity anEntity = auditReader.find(MyEntity.class, id, revNum) doesn't work because of
org.hibernate.envers.query.impl.AbstractAuditQuery.getSingleResult(AbstractAuditQuery.java:117) . I checked the SQL generated by Hibernate, in the where clause
where
myentity.rev<=?
and myentity.revtype<>?
and myentity.id=?
and (
myentity.revend>?
or myentity.revend is null
)
so for the rev number 1356, several audit records are retrieved, for example
1356 --> 1357 and 1305 -->1358 because rev num 1356 is just between the two ends.
How can I solve this problem? I mean to make the rev number monotonically increasing one transaction after another.
UPDATE
My revision entity
#Entity
#RevisionEntity(CustomRevisionListener.class)
#Table(name = "REV_TABLE")
#SequenceGenerator(name = "GENERIC_GENERATOR", sequenceName = "SQ_REVISION_ID")
public class Revision {
#Id
#GeneratedValue(strategy = GenerationType.SEQUENCE, generator ="GENERIC_GENERATOR")
#Column(nullable = false, name = REV)
#RevisionNumber
private long rev;
}
You probably want to take a look at how the revision number sequence is defined inside the SequenceIdRevisionEntity class. We essentially define it as follows:
#Id
#GeneratedValue(generator = "RevisionNumberSequenceGenerator")
#GenericGenerator(
name = "RevisionNumberSequenceGenerator",
strategy = "org.hibernate.envers.enhanced.OrderedSequenceGenerator",
parameters = {
#Parameter(name = "table_name", value = "REVISION_GENERATOR"),
#Parameter(name = "sequence_name", value = "REVISION_GENERATOR"),
#Parameter(name = "initial_value", value = "1"),
#Parameter(name = "increment_size", value = "1")
}
)
#RevisionNumber
private int id;
The key is specifying that the initial_value and increment_size are defined as 1 to avoid the hi-lo gaps you're noticing with your existing sequence definition.

Tests with DBunit and Oracle 10g autogenerated primary key identifiers not synchronized (JPA/Hibernate)

I'm testing a JPA/Hibernate application with DBunit and Oracle 10g. When I start my test I load to the database 25 rows with an identifier.
That's the xml where I have my data, that I insert with DBUnit
<entity entityId="1" ....
<entity entityId="2" ....
<entity entityId="3" ....
<entity entityId="4" ....
That's my entity class with JPA annotations (not hibernate specific)
#Entity
#Table(name = "entity")
public class Entity{
#Id
#GeneratedValue(strategy=GenerationType.Auto)
private Integer entityId;
...}
Those are the parameter values of the database connection with Oracle10g
jdbc.driverClassName=oracle.jdbc.OracleDriver
jdbc.url=jdbc:oracle:thin:#192.168.208.131:1521:database
jdbc.username=hr
jdbc.password=root
hibernate.dialect=org.hibernate.dialect.Oracle10gDialect
dbunit.dataTypeFactoryName=org.dbunit.ext.oracle.Oracle10DataTypeFactory
After insert this data in Oracle I run a test where I make Entity entity = new Entity() (I don't have to set manually the identifier because it's autogenerated)
#Test
public void testInsert(){
Entity entity = new Entity();
//other stuff
entityTransaction.begin();
database.insertEntity(entity);//DAO call
entityTransaction.commit();
}
and when the test makes the commit of the transaction I get the following error
javax.persistence.RollbackException: Error while commiting the transaction
at org.hibernate.ejb.TransactionImpl.commit(TransactionImpl.java:71)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
...
Caused by: org.hibernate.exception.ConstraintViolationException: Could not execute JDBC batch update
at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:94)
at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:66)
at org.hibernate.jdbc.AbstractBatcher.executeBatch(AbstractBatcher.java:275)
at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:266)
at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:167)
at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:321)
at org.hibernate.event.def.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:50)
at org.hibernate.impl.SessionImpl.flush(SessionImpl.java:1027)
at org.hibernate.impl.SessionImpl.managedFlush(SessionImpl.java:365)
at org.hibernate.transaction.JDBCTransaction.commit(JDBCTransaction.java:137)
at org.hibernate.ejb.TransactionImpl.commit(TransactionImpl.java:54)
... 26 more
Caused by: java.sql.BatchUpdateException: ORA-00001: restricción única (HR.SYS_C0058306) violada
at oracle.jdbc.driver.DatabaseError.throwBatchUpdateException(DatabaseError.java:345)
at oracle.jdbc.driver.OraclePreparedStatement.executeBatch(OraclePreparedStatement.java:10844)
at org.hibernate.jdbc.BatchingBatcher.doExecuteBatch(BatchingBatcher.java:70)
at org.hibernate.jdbc.AbstractBatcher.executeBatch(AbstractBatcher.java:268)
... 34 more
I have debugged it and the problem is that the entityId of the new object is 1, and already exists a entity with that Id. So, I don't know who is the responsable DBunit ? Oracle ? Why are not synchronized the identifiers of Oracle database and the identifier that JPA/hibernate gives to my entity in my testing code?
Thanks for your time
I think the AUTO generation type, in Oracle, is in fact a sequence generator. If you don't specify which sequence it must use, Hibernate is probably creating one for you and using it, and its default start value is 1.
Using AUTO is useful for quick prototyping. For a real application, use a concrete generation type (SEQUENCE, for Oracle), and create your sequences yourself with an appropriate start value to avoid duplicate keys.
You could use ids < 0 in your test data sets. Not only will your sequences never come in conflict with test records, but also you'll easily distinguish records that were inserted by the tests.
The AUTO sequencing strategy usually defaults to the TABLE sequencing strategy, but in the case of Oracle, the sequencing strategy uses an Oracle sequence named hibernate_sequence (which is the default, unless you specify a sequence name in the strategy). The starting value of the sequence happens to be 1, which conflicts with the existing entity that is loaded using DbUnit, hence resulting in the ConstraintViolationException exception being thrown.
For the purpose of unit tests, you could perform either of the following:
Issue a ALTER SEQUENCE... command to set the next value of the sequence, after loading data into the database. This will ensure that the JPA provider will use a sequence value that does not conflict with the existing Ids of the entities populated from your current XML file, by DbUnit.
Specify the name of the sequence in XML file loaded as the IDataSet eventually used by DbUnit. The actual sequence values will have to be replaced in the IDataSet using a SELECT <sequence_name>.nextval FROM DUAL. The following section is reproduced as is and is credited to this site:
I spend a couple of hours reading the dbUnit docs/facs/wikis and
source code trying to figure out how to use Oracle sequences, but
unless I overlooked something, I think this is not possible with the
current implementation.
So I took some extra time to find a workaround to insert Oracle
sequence generated IDs into dbUnit's datasets, muchlike what
ReplacementDataSet does. I subclassed DatabaseTestCase already earlier
in a abstract class (AbstractDatabaseTestCase) to be able to use a
common connection in case of insertion of my testcases in a testsuite.
But I added the following code just now. It looks up the first row of
each table in the dataset to determine which columns need sequence
replacement. The replacement is done on the "${…}" expression value.
This code is "quick and dirty" and surely needs some cleanup and
tuning.
Anyways, this is just a first try. I'll post further improvements as I
go, if this can be of any help to anyone.
Stephane Vandenbussche
private void replaceSequence(IDataSet ds) throws Exception {
ITableIterator iter = ds.iterator();
// iterate all tables
while (iter.next()) {
ITable table = iter.getTable();
Column[] cols = table.getTableMetaData().getColumns();
ArrayList al = new ArrayList(cols.length);
// filter columns containing expression "${...}"
for (int i = 0; i < cols.length; i++) {
Object o = table.getValue(0, cols[i].getColumnName());
if (o != null) {
String val = o.toString();
if ((val.indexOf("${") == 0) && (val.indexOf("}") == val.length() - 1)) {
// associate column name and sequence name
al.add(new String[]{cols[i].getColumnName(), val.substring(2, val.length()-1)});
}
}
}
cols = null;
int maxi = table.getRowCount();
int maxj = al.size();
if ((maxi > 0) && (maxj > 0)) {
// replace each value "${xxxxx}" by the next sequence value
// for each row
for (int i = 0; i < maxi; i++) {
// for each selected column
for (int j = 0; j < maxj; j++) {
String[] field = (String[])al.get(j);
Integer nextVal = getSequenceNextVal(field[1]);
((DefaultTable) table).setValue(i, field[0], nextVal);
}
}
}
}
}
private Integer getSequenceNextVal(String sequenceName) throws SQLException, Exception {
Statement st = this.getConnection().getConnection().createStatement();
ResultSet rs = st.executeQuery("SELECT " + sequenceName + ".nextval FROM dual");
rs.next();
st = null;
return new Integer(rs.getInt(1));
}
My AbstractDatabaseTestCase class has a boolean flag
"useOracleSequence" which tells the getDataSet callback method to call
replaceSequence.
I can now write my xml dataset as follows :
<dataset>
<MYTABLE FOO="Hello" ID="${MYTABLE_SEQ}"/>
<MYTABLE FOO="World" ID="${MYTABLE_SEQ}"/>
<OTHERTABLE BAR="Hello" ID="${OTHERTABLE_SEQ}"/>
<OTHERTABLE BAR="World" ID="${OTHERTABLE_SEQ}"/>
</dataset>
where MYTABLE_SEQ is the name of Oracle sequence to be used.

What does Hibernate map a boolean datatype to when using an Oracle database by default?

By default if I create a field in an entity like:
#NotNull
boolean myBoolean;
And I let Hibernate auto-create my tables. What Oracle data type will this map to?
As #Arthur said it maps to Number(1) which would be the standard sql bit where 0 == false and 1 == true. As an alternative you can map char(1) to 'T' or 'F' like this
#org.hibernate.annotations.Type(type="true_false")
#NotNull
boolean myBoolean;
or map it to 'Y' or 'N'
#org.hibernate.annotations.Type(type="yes_no")
#NotNull
boolean myBoolean;
Simply Number(1)
If you want, use SchemaExport to generate a script to your target database. Something like
AnnotationConfiguration configuration = new AnnotationConfiguration();
configuration
.addAnnotatedClass(<TYPE_YOUR_CLASS>.class)
.setProperty(Environment.USER, <TYPE_YOUR_USER>)
.setProperty(Environment.PASS, <TYPE_YOUR_PASSWORD>)
.setProperty(Environment.URL, <TYPE_YOUR_URL>)
.setProperty(Environment.DIALECT, <TYPE_YOUR_DIALECT>)
.setProperty(Environment.DRIVER, <TYPE_YOUR_DRIVER>);
SchemaExport schema = new SchemaExport(configuration);
schema.setOutputFile("schema.sql");
schema.create(<DO_YOU_WANT_TO_PRINT_TO_THE_CONSOLE>, <DO_YOU_WANT_TO_EXPORT_THE_SCRIPT_TO_THE_DATABASE>);
This is what you really need
Java POJO:
//#Type(type="true_false") //not working for '1' and '0' in NUMERIC(1) field
#Type(type= "org.hibernate.type.NumericBooleanType")
#NotNull(message="NOT_NULL")
#Column(name = "IS_DELEGATION", nullable = false)
private Boolean isDelegation;
Oracle DDL
alter table agent add (is_delegation number(1) default 0 not null);
As stated in
Hibernate docu

Resources