I use the following tech stack:
spring-boot-starter-data-jpa
HikariCP for connection pooling
Oracle DB
My actual code looks similar to this.
/// My trigger looks like this
CREATE OR REPLACE TRIGGER FILE_BRI
BEFORE INSERT
ON FILE
FOR EACH ROW
BEGIN
SELECT FILE_SEQ.NEXTVAL INTO :NEW.ID FROM DUAL;
END;
///
public class FILE implements Serializable {
#Id
#SequenceGenerator(
name = "FILE_SEQ",
sequenceName = "FILE_SEQ",
allocationSize = 1)
#GeneratedValue(
strategy = GenerationType.SEQUENCE,
generator = "FILE_SEQ"
)
private long id;
}
public class ServiceA () {
#Transactional(propagation = REQUIRES_NEW, isolation = READ_COMMITTED)
public File insertFile() {
// Below line returns the inserted File object with ID as '58496'
return fileRepository.save(file)
}
#Transactional(propagation = REQUIRES_NEW, isolation = READ_COMMITTED)
public AccessControl insertAccessControl() {
// Below line results in 'SQLIntegrityConstraintViolationException' (full error at the bottom of this post)
return accessControlRepository.save(accessControlFile)
}
}
Public class FileProcessor() {
ServiceA serviceA;
public void someMethod() {
// insert the file and get the inserted record
File insertedFile = serviceA.insertFile(file);
// get the ID from the inserted file and make another insert into another table
serviceA.insertAccessControl(insertedFile.getId()); // inserted file ID is '58496'
}
}
This is my investigation:
When I verified the ID of the inserted record in the table "FILE" is '58497', however repository.save() returned a different value.
When I make the second insert on table "ACCESS_CONTROL_FILE" with FILE_ID as '58496' it results in the error below because the FILE with ID as '58496' does not exist.
Caused by: java.sql.SQLIntegrityConstraintViolationException: ORA-01400: cannot insert NULL into ("DB_OWNER"."ACCESS_CONTROL_FILE"."FILE_ID")
I'm puzzled as to why would repository.save() return a different ID(i.e. ID=58496) than what is actually inserted(ID=58497) in the database!
I've investigated all options that I could find on the internet related to 'Propagation and Isolation'.
As mentioned in comments, Looks like a database trigger is causing the issue. Disable the trigger to let JPA to manage the ID generation.
Related
New to spring data JPA. I have sequence created in oracle table. I am using JPARepository saveall() method to save the data. Inserts work without any problem but when i try to update any existing rows in the table it always tries to insert throwing unique constraint error since i have unique index created for that table.
Entity Class
#Id
#GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "generator1")
#SequenceGenerator(sequenceName = "emp_seq", name = "generator1",allocationSize = 1)
#Column(name = "USER_ID")
private Long id;
Save method invocation
public void persistEmployees(List<Employee> employees) {
employeeRepo.savaAll(employees);
}
Repository class
#Repository
public interface EmployeeRepository extends JpaRepository<Employee, Long> { }
How can i tell JPA to look for existing row before inserting? Any help is much appreciated!!!
In My Humble Opinion;
Using "Exists" condition-check as "sub-query" for all constrained columns, before Update will solve this.
Im inserting a CSV file in a database table using Spring Batch and Spring Data (and Hibernate).
each time I insert the CSV I have to delete the previous data in the table using the data-jpa deleteAll() method. the problem is that the ids of the table are incremented automatically and continuously (#GeneratedValue(strategy = GenerationType.IDENTITY)) after each delete/insert statement.
I want that after each delete the ids start on 1. the only way that I found to do that is by altering the index (i know its not the best way, so your suggestions are welcomed)
the Question is :
is there any method to run this SQL request
ALTER TABLE res AUTO_INCREMENT=1;
in Java object using Spring Data or Hibernate?
Thanks
Is it possible to generate id on java side and do not use embedded db autoincrement feature?
So the best way will be to generate id explicitly and set it to entity.
Other cases are:
Truncate Table
TRUNCATE TABLE table_name;
This will reset the auto increment on the table as well as deleting all records from that table.
Drop and Recreate
Table DROP TABLE table_name;
CREATE TABLE table_name { ... };
So I think, second is what are you looking for
Instead of Altering the table, I have customized the way that Hibernate Generates the Ids.
instead of using :
#GeneratedValue(strategy = GenerationType.IDENTITY)
I have implemented a custom id generator :
#GenericGenerator(name = "sequence_id", strategy =
"com.xyz.utils.CustomIdGenerator",
parameters = {
#org.hibernate.annotations.Parameter(
name = "table_name", value = "myTable")
})
#GeneratedValue(generator = "sequence_id")
the CustomIdGenerator class :
public class CustomIdGenerator implements IdentifierGenerator,Configurable{
private String table_name ;
#Override
public Serializable generate(SharedSessionContractImplementor session, Object object)
throws HibernateException {
Connection connection = session.connection();
try {
Statement statement=connection.createStatement();
ResultSet rs=statement.executeQuery("select count(id) as Id from "
+table_name );
if(rs.next())
{
int id=rs.getInt(1)+1;
Integer generatedId = new Integer(id);
System.out.println("Generated Id: " + generatedId);
return generatedId;
}
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return null;
}
#Override
public void configure(Type type, Properties params, ServiceRegistry serviceRegistry)
throws MappingException {
setTable_name(params.getProperty("table_name")); }
//getters and setters
}
the problem of this solution is the execution of select for each id so it seems generating load on the DBMS and its slow.
and the rs is looping twice for the first id ()
any suggestion for optimization is welcomed
I'm trying to write a default method in a CrudRepository dao that:
Locks a table such that inserts are prevented (but reads and updates would be okay)
Looks for an existing entry using another method in the dao.
If found, returns the found entry.
If not found, inserts the provided entry, and returns it.
Unlocks the table.
I've looked into using #Lock(LockModeType.PESSIMISTIC_WRITE) on the method, but since it doesn't have an associated #Query annotation, I don't think it does anything.
I also tried creating lock and unlock methods in the dao:
#Query(value = "LOCK TABLES mail WRITE", nativeQuery = true)
void lockTableForWriting();
#Query(value = "UNLOCK TABLES", nativeQuery = true)
void unlockTable();
But those threw a SQLGrammerException on LOCK and UNLOCK.
I can't get a row lock because the row doesn't exist yet. Or if it does exist, I'm not going to update anything, I'm just going to not insert anything and move on to other things.
There are other cases where I want multiple records saved with the same transaction id, so I can't just make the column unique and try/catch the save.
In my service layer, I do some lookup attempts and short-circuit when possible, but there's still a chance that multiple near-simultaneous calls might result in two attempts to insert the same data.
It needs to be handled at the db level since there will be multiple instances of the service running. So two competing calls might be on different machines that talk to the same database.
Here's my repository:
#Repository
public interface MailDao extends CrudRepository<Mail, Long> {
default Mail safeSave(Mail mail) {
return Optional.ofNullable(findByTransactionId(mail.getTransactionId()))
.orElseGet(() -> save(mail));
}
default Mail findByTransactionId(String transactionId) {
final List<Mail> mails = findAllByTransactionId(transactionId);
// Snipped code that selects a single entry to return
// If one can't be found, null is returned.
}
#Query(value = "SELECT m " +
" FROM Mail m " +
" WHERE m.transactionId = :transactionId ")
List<Mail> findAllByTransactionId(#Param("transactionId") String transactionId);
}
And here's a bit of what the Mail model looks like:
#Entity
#Table(name = "mail")
public class Mail implements Serializable {
private static final long serialVersionUID = 1L;
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column(name = "mail_id", unique = true, nullable = false)
private Long mailId;
#Column(name = "transaction_id", nullable = false)
private String transactionId;
// Snipped code for other parameters,
// constructors, getters and setters.
}
Here's the general idea of the service method that would be calling safeSave.
#Service
public class MailServiceImpl implements MailService {
#Inject private final MailDao mailDao;
// Snipped injected other stuff
#Override
public void saveMailInfo(final String transactionId) {
Objects.requireNonNull(transactionId, "null transactionId passed to saveMailInfo");
if (mailDao.findByTransactionId(transactionId) != null) {
return;
}
// Use one of the injected things to do some lookup stuff
// using external services
if (mailDao.findByTransactionId(transactionId) != null) {
return;
}
// Use another one of the injected things to do more lookup
if (/* we've got everything we need */) {
final Mail mail = new Mail();
mail.setTransactionId(transactionId);
// Snipped code to set the rest of the stuff
mailDao.safeSave(mail);
}
}
}
What I'm trying to prevent is two nearly-simultaneous calls to saveMailInfo causing duplicate records in the database.
The underlying database is MySQL, but we also use an H2 in-memory database for unit tests, and will be changing to PostgreSQL soon, so it'd be nice to have something db-agnostic.
Update 1:
I tried creating a custom impl:
public interface MailDaoCustom {
Mail safeSave(Mail mail);
}
Updated MailDao to implement it:
public interface MailDao extends CrudRepository<Mail, Long>, MailDaoCustom
Then the impl:
public class MailDaoImpl implements MailDaoCustom {
#Autowired private MailDao mailDao;
#Autowired private EntityManager em;
public Mail safeSave(Mail mail) {
Query lockTable = em.createNativeQuery("LOCK TABLES mail WRITE");
Query unlockTable = em.createNativeQuery("UNLOCK TABLES");
try {
lockTable.executeUpdate();
return Optional.ofNullable(mailDao.findByTransactionId(mail.getTransactionId()))
.orElseGet(() -> mailDao.save(mail));
} finally {
unlockTable.executeUpdate();
}
}
}
This is the error I got when testing the above. It's the same as when I tried making the methods in the dao with the queries to lock and unlock:
SQL Error: 42000, SQLState: 42000
Syntax error in SQL statement "LOCK[*] TABLES MAIL WRITE "; SQL statement:
LOCK TABLES mail WRITE [42000-168]
My testing has been done using unit tests that use an H2 in-memory database, though, and not MySQL. It looks like H2 might not really have table locking, though.
Is there maybe another solution to my issue that doesn't involve table locking?
Update 2: I went with an INSERT ... WHERE NOT EXISTS query in a custom repo similar to my first update:
public interface MailDaoCustom {
Mail safeSave(Mail mail);
}
Updated MailDao to implement it:
public interface MailDao extends CrudRepository<Mail, Long>, MailDaoCustom
And then the impl looks like this:
public class MailDaoImpl implements MailDaoCustom {
#Autowired private MailDao dao;
#Autowired private EntityManager em;
public Mail safeSave(Mail mail) {
// Store a new mail record only if one doesn't already exist.
Query uniqueInsert = em.createNativeQuery(
"INSERT INTO mail " +
" (transaction_id, ...) " +
"SELECT :transactionId, ... " +
" WHERE NOT EXISTS (SELECT 1 FROM mail " +
" WHERE transaction_id = :transactionId) ");
uniqueInsert.setParameter("transactionId", mail.getTransactionId());
// Snipped setting of the rest of the parameters in the query
uniqueInsert.executeUpdate();
// Now go get the record
Mail entry = dao.findByTransactionId(mail.getTransactionId());
// Detatch the entry so that we can attach the provided mail object later.
em.detach(entry);
// Copy all the data from the db entry into the one provided to this method
mail.setMailId(entry.getMailId());
mail.setTransactionId(entry.getTransactionId());
// Snipped setting of the rest of the parameters in the provided mail object
// Attach the provided object to the entity manager just like the save() method would.
em.merge(mail);
return mail;
}
}
It's not quite as clean as I was hoping, and I fear I may have made some mistake about something that's kind of hidden and I won't find out until things blow up. But we'll see.
I went with a INSERT INTO ... WHERE NOT EXISTS query, and a custom repo. This is listed above under update 2, but I'm putting it here too so that it's easier to find.
public interface MailDaoCustom {
Mail safeSave(Mail mail);
}
Updated MailDao to implement it:
public interface MailDao extends CrudRepository<Mail, Long>, MailDaoCustom
And then the impl looks like this:
public class MailDaoImpl implements MailDaoCustom {
#Autowired private MailDao dao;
#Autowired private EntityManager em;
public Mail safeSave(Mail mail) {
// Store a new mail record only if one doesn't already exist.
Query uniqueInsert = em.createNativeQuery(
"INSERT INTO mail " +
" (transaction_id, ...) " +
"SELECT :transactionId, ... " +
" WHERE NOT EXISTS (SELECT 1 FROM mail " +
" WHERE transaction_id = :transactionId) ");
uniqueInsert.setParameter("transactionId", mail.getTransactionId());
// Snipped setting of the rest of the parameters in the query
uniqueInsert.executeUpdate();
// Now go get the record
Mail entry = dao.findByTransactionId(mail.getTransactionId());
// Detach the entry so that we can attach the provided mail object later.
em.detach(entry);
// Copy all the data from the db entry into the one provided to this method
mail.setMailId(entry.getMailId());
mail.setTransactionId(entry.getTransactionId());
// Snipped setting of the rest of the parameters in the provided mail object
// Attach the provided object to the entity manager just like the save() method would.
em.merge(mail);
return mail;
}
}
I recently replaced PostgreSql unit test database with an in memory H2 Database. I couldn't figure out why couple tests are failing, it was working fine with Postgre. There are approx. 280+ unit tests in this application. The failing dao tests are selecting from a view and the entity has a EmbeddedId, those the columns of that view.
See below code (Note: I changed names, code and the sql to hide the real names when writing this email, but this was a working unit test with Postgre db)
<pre>
#Table(name = "item_view") // <- item_view is a database view
public class ItemV implements Serializable
{
.....
#EmbeddedId // <- entity has an embedded id
private ItemVId id;
.....
#Embeddable
public static class ItemVId implements Serializable //<- This is the embeddedId
{
#Column(name = "item_id", updatable=false, insertable=false)
private Long ItemId; //<- col no.1 of view
#Column(name = "item_type_id", updatable=false, insertable=false)
private Integer ItemTypeId; //<- col no.2 of view
.....
ItemType is an enum
And the view is
CREATE OR REPLACE VIEW item_view AS
( ( ( SELECT pt.id as item_id, cit.id as item_type_id
FROM xyz pt, item_type cit
WHERE pt.name::text = 'xyz'::text
UNION
SELECT z.id as item_id, cit.id as item_type_id
FROM zzz z, item_type cit
WHERE z.name::text = 'zzz'::text)
..............
and the dao method is
public ItemView find(Long itemId, ItemType itemType)
{
String hql = " from ItemV iv where iv.id.itemId = :itemId and iv.id.itemTypeId = :itemTypeId ");
List<ItemView> result = (List<ItemView>)getEntityManager()
.createQuery(hql)
.setParameter("itemId", itemId)
.setParameter("itemTypeId", itemType.getId())
.setMaxResults(1)
.getResultList();
return result.isEmpty()
? null : result.get(0);
}
This dao method always returns empty results, never finding existing rows in the view??? I know those rows exist because when I do getAll() on the same dao I see results and I see a matching row for the criteria.
Is there anything special about selecting rows from a view in H2 database?
Thanks
Ok fixed, I had to use a smaller number for LOCK_TIMEOUT value, so now I can connect to database and see values. Selecting from view problem also fixed.
I have to say H2 is really neat and elegant. I'm glad, I switched the unit test db to H2.
i get this error could not get next sequence value when I try to save this Entity with Hibernate:
package beans;
import javax.persistence.*;
#Entity
#Table(schema = "EVGENY")
public class Article {
#SequenceGenerator(name="ArticleGen", sequenceName="ARTICLESEC")
#Id
#GeneratedValue(generator= "ArticleGen")
private int id;
#Column(name="title")
private String title;
#Column(name="text")
private String text;
#Column(name="postat")
private String postat;
#ManyToOne
#JoinColumn(name = "USER_ID")
private UserAcc user;
public Article(){
}
Get Set...
}
insert into article (title) values('asdfaf');
in Oracle SQL Developer this insert into article (title) values('asdfaf'); works well.
if i set id variable explicitly ( Article a = new Article();a.setId(3); )
everything is OK. I double checked the name of the sequence.
Check user permissions on the sequence. Most of the time it is grant issue
I know a lot of reasons to get this exception. Maybe you can check the questions and give me some more details about your problem:
check if the 'hibernate.dialect' is set to Oracle?
permission on sequence ok (schemata, select etc.)?
is there some trigger behind the table and throwing plsql error?
some other plsql that could break the insert?
is the transaction broken?
was there an exception (sometimes silent) somwhere before the call of create (like stale state or out of bounce) so that transaction is marked for rollback?
is there a connection is erroneous error before exception?
is the entity maybe already attached (check with contains)?
do you use spring or another framework where you can add exception resolvers or translators?
which version of oracle database do you use? 10 or 11? and are the used drivers correct?
is there a findSingle call before that does not return any value?