According to the Quarkus docs, including the line below in the application.properties should result in delete statements to be batched.
quarkus.hibernate-orm.jdbc.statement-batch-size=1000
However, I can't get this to work. Regardless of this property all delete statements are sent to the database individually instead of in batches.
Is there anything else I need to do?
To reproduce, use a simple entity like this:
#Entity
#Table(name = "book")
public class Book {
#GeneratedValue(strategy = IDENTITY)
#Id
private Long id;
private String title;
public Book() {
}
public Long getId() {
return id;
}
}
insert records into the database like this (on PostgreSQL):
INSERT INTO book (id, title)
VALUES(generate_series(1, 200), 'a title');
and a simple integration test like this:
#QuarkusTest
class BookDeleteIT {
#Inject EntityManager em;
#Test
void deletes_records_in_batches() {
List<Book> books = getBooks();
deleteBooks(books);
}
#Transactional
List<Book> getBooks() {
return em.createQuery("SELECT b FROM Book b").getResultList();
}
#Transactional
void deleteBooks(List<Book> books) {
books.forEach(book -> delete(book));
}
private int delete(Book book) {
return em.createQuery("DELETE FROM Book b WHERE b.id = :id")
.setParameter("id", book.getId())
.executeUpdate();
}
}
When I run this test, the deletes are sent to the database individually instead of in batches.
I suppose the error is the way you're deleting a book: use em.remove(book) instead of the query and Hibernate will accumulate deletions.
There are possibilities that deleting a managed entity using a query instead of EntityManager prevents your JPA provider (Hibernate) to manage entity lifecycle and do some optimizations (like batch deletion).
Related
I am using Bidirectional #OneToOne from Hibernate documentation. I have created an identical model for the test.
I can't get Phone via PhoneDetails. I get an error - Message Request processing failed; nested exception is org.hibernate.LazyInitializationException: could not initialize proxy [com.example.model.Phone#1] - no Session.
I've tried many options and it doesn't work.
Please tell me how to get the Phone correctly? I sit all day trying to do this. I did not find any options on the Internet, so I ask here.
Phone.java
#Entity(name = "Phone")
public class Phone {
#Id
#GeneratedValue
private Long id;
#Column(name = "`number`")
private String number;
#OneToOne(mappedBy = "phone",
cascade = CascadeType.ALL,
orphanRemoval = true,
fetch = FetchType.LAZY)
private PhoneDetails details;
public Phone() {
}
public Phone(String number) {
this.number = number;
}
// Getters and setters are omitted for brevity
public void addDetails(PhoneDetails details) {
details.setPhone( this );
this.details = details;
}
public void removeDetails() {
if ( details != null ) {
details.setPhone( null );
this.details = null;
}
}
}
PhoneDetails.java
#Entity(name = "PhoneDetails")
public class PhoneDetails {
#Id
#GeneratedValue
private Long id;
private String provider;
private String technology;
#OneToOne(fetch = FetchType.LAZY)
#JoinColumn(name = "phone_id")
private Phone phone;
public PhoneDetails() {
}
public PhoneDetails(String provider, String technology) {
this.provider = provider;
this.technology = technology;
}
// Getters and setters are omitted for brevity
}
LifecycleController.java
#Controller
public class LifecycleController {
#Autowired
ServiceJpa serviceJpa;
#GetMapping(value = "/savePhoneAndPhoneDetails")
public String savePersonAddress () {
Phone phone = new Phone( "123-456-7890" );
PhoneDetails details = new PhoneDetails( "T-Mobile", "GSM" );
phone.addDetails( details );
serviceJpa.savPhone( phone );
return "/savePhoneAndPhoneDetails";
}
#GetMapping(value = "/getPhone")
public String addPersonAddress () {
PhoneDetails address = serviceJpa.findPhoneDetailsById(2L).orElseThrow();
Phone phone = address.getPhone();
/*
An error appears here -
could not initialize proxy
[com.example.model.Phone#1] - no Session
*/
System.out.println(phone.getNumber());
return "/getPhone";
}
}
ServiceJpa.java
#Service
#Transactional
public class ServiceJpa {
#Autowired
PhoneJpa phoneJpa;
#Autowired
PhoneDetailsJpa phoneDetailsJpa;
#Transactional
public void savPhone(Phone phone) {
phoneJpa.save(phone);
}
#Transactional
public Optional<PhoneDetails> findPhoneDetailsById(Long id) {
return phoneDetailsJpa.findById(id);
}
}
interface PhoneJpa.java
#Repository
public interface PhoneJpa extends JpaRepository<Phone, Long> {
}
interface PhoneDetailsJpa.java
#Repository
public interface PhoneDetailsJpa extends JpaRepository<PhoneDetails, Long> {
}
I agree with Andriy's comment with a slight addition of "You should not access [lazily loaded] entity details outside transaction bounds". But, for starters, is there some reason you want the OneToOne to be FetchType.LAZY to begin with? If you changed it to EAGER, your "lazy" problem would be resolved by virtue of it no longer being a lazy reference but being a real hydrated object.
If that is not the exact route you want to take, there are a dozen ways to EAGERLY fetch things in general and frankly too many to present a single solution here as best/ideal. As your code exists, since all the dereferencing (for now) is happening inside your Controller, then Andriy's suggestion to add #Transaction to the Controller may suffice in that it will be lazily fetched when you need it.
But in the future, if you have Lazy elements in a POJO that get returned to the stack higher than the controller, say, just before they are serialized to JSON for example, then even the CONTROLLER's #Transactional wouldn't be "high" enough in the stack and you'll end up with the same Lazy init problem..
Also, by having it be Lazy and then dereferencing it elsewhere, you're guaranteeing two trips to the Database. With proper FETCH/JOIN eager loads, you would limit that to one, which can be another performance benefit.
So either way, you're back to the real problem at hand.. looking for ways to ensure your operations occur ENTIRELY inside a Transaction boundary OR having to completely hydrate the object so no "Lazy" danglers get dereferenced outside of that.. i.e. by making them eager or by force-initializing any potential Lazy proxies/collections.
I have a repository
public interface PersonRepository extends JpaRepository<Person, Long> {}
and the Entity looks like this:
#Data
#Entity
#NoArgsConstructor
#AllArgsConstructor
public class Person {
#Id
private Long id;
#NotBlank
private String name;
}
I want to have a method which checks if all "persons" exist in database table by id, this what I have so far:
void checkIfAllPersonsExist(List<Long> personIds) {
var persons = personRepository.findAllById(personIds);
if (personIds.size() != persons.size()) {
personIds.removeAll(persons.stream().map(Persons::getId).collect(toList()));
throw new NotFoundException("Persons with id's [id:%s] does not exist", personIds);
}
}
I wonder if Spring JPA Repository can provide anything more elegant? Like specific named query which returns id's which does not exist?
If you want to just know that there are some ids that not exist you can count them
#Query("select COUNT(p.id) from Person p where p.id in :ids")
Long countIds(List<Long> ids);
Or equivalent based on
long countByIdIn(Collection<Long> ids);
Or return list of ids that exists
#Query("select p.id from Person p where p.id in :ids")
List<Long> getExistenIds(List<Long> ids);
And then filter out what you need.
personIds.removeAll(personRepository.getExistenIds(personIds));
if (!personIds.isEmpty()) {
throw new NotFoundException("Persons with id's [id:%s] does not exist", personIds);
}
First of all, your repository should extend JpaRepository<Person, Long> instead of JpaRepository<Person, String >, because your entity's id type is Long.
In and NotIn keywords can help you to achive your goal. Please check them out in this document: Query Creation - Spring Data JPA - Reference Documentation
I modified your code a little bit and it works for me.
Repository class:
public interface PersonRepository extends JpaRepository<Person, Long> {
List<Person> findByIdIn(Collection<Long> ids);
}
And sample snippet:
#Component
public class Bootstrap implements CommandLineRunner {
#Autowired
private PersonRepository repository;
#Override
public void run(String... args) throws Exception {
savePersons();
testFindMethod();
}
private void savePersons() {
Person person1 = Person.builder().id(1L).name("Name 1").build();
Person person2 = Person.builder().id(2L).name("Name 2").build();
Person person3 = Person.builder().id(3L).name("Name 3").build();
Person person4 = Person.builder().id(4L).name("Name 4").build();
repository.save(person1);
repository.save(person2);
repository.save(person3);
repository.save(person4);
}
private void testFindMethod() {
List<Long> toFind = new ArrayList<>();
toFind.add(1L);
toFind.add(2L);
toFind.add(3L);
checkIfAllPersonsExist(toFind);
toFind.add(7L);
checkIfAllPersonsExist(toFind);
}
void checkIfAllPersonsExist(List<Long> personIds) {
List<Person> persons = repository.findByIdIn(personIds);
if (personIds.size() != persons.size()) {
System.out.println("Sizes are different");
} else {
System.out.println("Sizes are same!");
}
}
}
And this is console output:
Sizes are same!
Sizes are different
I hope this will help you.
With this JPA repository method you can get the elements which ids doesn't exists:
List<Person> findByIdNotIn(List<Long> personIds);
If you want to remove them like in your example, you can use this one:
List<Person> deleteByIdNotIn(List<Long> personIds);
I hope it helps!
I have an issue when updating entity with OneToOne relationship, it creates record instead of updating the existing one. Below are the sample entities.
#Entity
#Table(schema = "crm", name = "persons")
public class Person {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
public Long getId() {
return id;
}
#OneToOne(mappedBy = "person", fetch = FetchType.LAZY)
public Employee getEmployee() {
return employee;
}
}
#Entity
#Table(schema = "crm", name = "employees")
public class Employee {
#Id
public Long getId() {
return id;
}
#OneToOne
#MapsId //Use the person PK id value as Employee PK id
#JoinColumn(name = "id")
public Person getPerson() {
return person;
}
}
I am using the PagingAndSortingRepository of Spring Data. Below is the service layer to update the entity.
#Override
#Transactional
public EmployeeResponse updateEmployee(Employee aEmployee) {
EmployeeResponse response = new EmployeeResponse();
try {
Optional<Employee> probableEmployee = employeeRepository.findById(aEmployee.getId());
if (!probableEmployee.isPresent()) {
throw new RecordNotFoundException(String.format(MessageConstants.EMPLOYEE_ID_NOT_FOUND, aEmployee.getId()));
}
Employee existingEmployeeToUpdate = probableEmployee.get();
EmployeeEntityHelper.updateExistingEntity(aEmployee, existingEmployeeToUpdate);
existingEmployeeToUpdate = employeeRepository.save(existingEmployeeToUpdate);
response.setSuccessfulResponse(existingEmployeeToUpdate);
} catch (Exception ex) {
LOGGER.error(ex.getLocalizedMessage(), ex);
response.setErrorAttributes(false, ReturnCode.FAILED.getCode(), ex.getLocalizedMessage());
}
return response;
}
The EmployeeEntityHelper.updateExistingEntity(source, target) will simply copy all properties of entities from source to target.
The save() method will generate an insert for Person even if I am explicitly passing the id existing in DB. But for employee it will generate an update which is expected.
Below is the updateExistingEntity() method:
public static void updateExistingEntity(Employee source, Employee target) {
copyProperties(source, target, Arrays,asList("person", "employeeNumber", "hiredDate", "birthDate"));
}
private static void copyProperties(Object aSource, Object aTarget, Iterable<String> aProperties) {
BeanWrapper sourceWrapper = PropertyAccessorFactory.forBeanPropertyAccess(aSource);
BeanWrapper targetWrapper = PropertyAccessorFactory.forBeanPropertyAccess(aTarget);
aProperties.forEach(p ->
targetWrapper.setPropertyValue(p, sourceWrapper.getPropertyValue(p))
);
}
Generally, a new INSERT instead of UPDATEs could be due to:
Either entity passed to save has no ID set, thus persist is called under the hood
Or entity with given ID is not present in the database thus merge fails (and maybe it is persisted as a fallback, dunno)
Check if entities have set ID field.
In your model , override equal/hash-code and just chek the id parameter
maybe it will help you :)
The issue is already fixed, somehow the copyproperties logic is not right in the sense that I am updating/setting the person also.
I have a JPA entity with Lazy loaded collection on it. I do not need the collection every time.
#Entity(name = "Foo")
#Access(AccessType.FIELD)
#Table(name = "TEST", schema = "TEST")
public class Foo implements Serializable {
private static final long serialVersionUID = 1L;
#OneToMany(mappedBy="foo", targetEntity=Bar.class, fetch=FetchType.LAZY, cascade=CascadeType.ALL)
private List<Bar> bars;
}
#Entity(name = "Bar")
#Access(AccessType.FIELD)
#Table(name = "TEST", schema = "TEST")
public class Bar implements Serializable {
private static final long serialVersionUID = 1L;
#ManyToOne(targetEntity = Foo.class)
#JoinColumn(name = "FOO_ID", referencedColumnName = "ID")
private Foo foo;
}
I have a few methods on a service class that perform a lot of database interactions and at the end save a Foo entity to the database. I need this to happen for about a 100 items in a collection.
#Service
public class FooService {
#Autowired
private FooRepository fooRepository;
public void processAllFoos() {
fooRepository.findAll().forEach(foo -> {
processFoo(foo);
});
}
private void processFoo(Foo foo) {
foo.getBars().forEach(bar -> {
// Do a lot of time consuming stuff here that involves
// entities of other types and modify each bar object
});
fooRepository.save(foo);
}
}
processAllFoos gets called from a #RESTController whenever it gets a request.
However, I do not want processAllFoos to be wrapped in a single database transaction, because that locks up the entire Foo table till the business logic is executed for all Foos.
If I make the processFoo method #Transactional I get the LazyInitializationException which complains that the Hibernate session is non-existent. To make this work I need to make all methods in the call stack #Transactional so that the nested methods can join onto the calling method's transaction. But this locks the entire Foo table as mentioned above.
Adding a OpenSessionInViewFilter for the dispatcher servlet solves my problem but I've read that there are issues with performance and entity detaching/reattaching (which I do in other parts of the application) with this approach.
Is there a way I can do what I want to without using the OpenSessionInView approach? What other vulnerabilities am I adding by using this approach?
Spring/Hibernate 4.x
Based on the answer below, I was able to do the following:
#Service
public class FooService {
#Autowired
private FooRepository fooRepository;
#Autowired
private TransactionTemplate transactionTemplate;
public void processAllFoos() {
fooRepository.findAll().forEach(foo -> {
transactionTemplate.execute(new TransactionCallback<Object>() {
public Object doInTransaction(TransactionStatus status) {
try {
processFoo(foo);
status.flush();
} catch(Exception e) {
status.setRollbackOnly();
}
return null;
}
});
});
}
private void processBar(Foo foo) {
foo.getBars().foreEach(bar -> {
// Do a lot of time consuming stuff here that involves
// entities of other types and modify each bar object
});
fooRepository.save(foo);
}
}
OpenSessionInViewFilter commonly used to solve LazyInitialization problem in View layer (UI components or page templates), because View layer can't and must not manage transactions directly.
In your case another way to get all the Bar objects can be applied.
First You get all the Foo object ids instead to get fully objects.
Second Use Foo ids collection to iterate thru related Bar objects.
Third If you don't want one BIG transaction then you can use Spring Transaction template to manage transactions explicitly.
Your code example may look like this:
#Service
public class FooService {
#Autowired
private FooRepository fooRepository;
#Autowired
private BarRepository barRepository;
#Autowired
private TransactionTemplate transactionTemplate;
public void processAllFoos() {
final List < Long > fooIdList = transactionTemplate.execute(new TransactionCallback() {
public Object doInTransaction(TransactionStatus status) {
return fooRepository.findIdList();
}
});
transactionTemplate.execute(new TransactionCallback() {
public Object doInTransaction(TransactionStatus status) {
barRepository.findByFooIdList(fooIdList).forEach(bar - > {
processBar(bar);
});
return null;
}
});
}
private void processBar(Bar bar) {
// Do a lot of time consuming stuff here that involves
// entities of other types and modify each bar object
barRepository.save(bar);
}
}
Example below shows how to solve your task without some performance overheads. But you should understand that if Foo and Bar tables linked with foreign key constraint, then related record in Foo table may be blocked by RDBMS each time you update row in Bar table.
Model structure:
#MappedSuperclass
public class BaseModel<K extends Comparable> implements Serializable, Comparable<Object> {
private static final long serialVersionUID = 1L;
#Id
private K id;
#Version
private Integer version;
// getter/setter
}
#Entity
public class MyEntity extends BaseModel<String> {
// some fields and it's getter/setter
}
Record in my database for my_entity:
id: 1
version: 1
...
Below is my update method:
void update(String id, Integer currentVersion, ....) {
MyEntity myEntity = myRepository.findOne(id);
myEntity.setVersion(currentVersion);
// other assignments
myRepository.save(myEntity);
}
Below is the query being fired when this method is invoked.
update my_entity set version=?, x=?, y=?, ...
where id=? and version=?
I am expecting OptimisticLockException when currentVersion passed in above method is other than 1.
Can any body help me why I am not getting OptimisticLockException?
I am using spring-boot for my webmvc project.
Section 11.1.54 of the JPA specification notes that:
In general, fields or properties that are specified with the Version
annotation should not be updated by the application.
From experience, I can advise that some JPA providers (OpenJPA being one) actually throw an exception should you try to manually update the version field.
While not strictly an answer to your question, you can re-factor as below to ensure both portability between JPA providers and strict compliance with the JPA specification:
public void update(String id, Integer currentVersion) throws MyWrappedException {
MyEntity myEntity = myRepository.findOne(id);
if(currentVersion != myEntity.getVersion()){
throw new MyWrappedException();
}
myRepository.save(myEntity);
//still an issue here however: see below
}
Assuming your update(...) method is running in a transaction however you still have an issue with the above as section 3.4.5 of the JPA specification notes:
3.4.5 OptimisticLockException Provider implementations may defer writing to the database until the end of the transaction, when
consistent with the lock mode and flush mode settings in effect. In
this case, an optimistic lock check may not occur until commit time,
and the OptimisticLockException may be thrown in the "before
completion" phase of the commit. If the OptimisticLockException must
be caught or handled by the application, the flush method should be
used by the application to force the database writes to occur. This
will allow the application to catch and handle optimistic lock
exceptions.
Essentially then, 2 users can submit concurrent modifications for the same Entity. Both threads can pass the initial check however one will fail when the updates are flushed to the database which may be on transaction commit i.e. after your method has completed.
In order that you can catch and handle the OptimisticLock exception, your code should then look something like the below:
public void update(String id, Integer currentVersion) throws MyWrappedException {
MyEntity myEntity = myRepository.findOne(id);
if(currentVersion != myEntity.getVersion()){
throw new MyWrappedException();
}
myRepository.save(myEntity);
try{
myRepository.flush()
}
catch(OptimisticLockingFailureException ex){
throw new MyWrappedException();
}
}
Use EVICT before updating when using JPA. I did not get the #Version to work either. The property was increased but no exception was thrown when updating an object that had the wrong version-property.
The only thing I have got to work is to first EVICT the object and then save it. Then the HibernateOptimisticLockingException is thrown if the Version properties does not match.
Set the hibernates ShowSQL to 'true' to verify that the actual update sql ends with "where id=? and version=?". If the object is not evicted first, the update statement only has "where id=?", and that will (for obvious reasons) not work.
Optimistic hibernation lock works out of the box (You don't must put a version for Entity):
#Entity
#Table(name = "product")
public class Product {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
private Long quantity;
private Long likes;
#Version
private Long version;
public Product() {
}
//setter and getter
//equals and hashcode
repository
public interface ProductRepository extends JpaRepository<Product, Long> {}
service
#Service
public class ProductOptimisticLockingService {
private final ProductRepository productRepository;
public ProductOptimisticLockingService(ProductRepository productRepository) {
this.productRepository = productRepository;
}
#Transactional(readOnly = true)
public Product findById(Long id, String nameThread){
Product product =
productRepository
.findById(id)
.get();
System.out.printf(
"\n Select (%s) .... " +
"(id:) %d | (likes:) %d | (quantity:) %d | (version:) %d \n",
nameThread,
product.getId(),
product.getLikes(),
product.getQuantity(),
product.getVersion()
);
return product;
}
#Transactional(isolation = Isolation.READ_COMMITTED)
public void updateWithOptimisticLocking(Product product, String nameThread) {
try {
productRepository.save(product);
} catch (ObjectOptimisticLockingFailureException ex) {
System.out.printf(
"\n (%s) Another transaction is already working with a string with an ID: %d \n",
nameThread,
product.getId()
);
}
System.out.printf("\n--- Update has been performed (%s)---\n", nameThread);
}
}
test
#SpringBootTest
class ProductOptimisticLockingServiceTest {
#Autowired
private ProductOptimisticLockingService productService;
#Autowired
private ProductRepository productRepository;
#Test
void saveWithOptimisticLocking() {
/*ID may be - 1 or another. You must put the ID to pass in your methods. You must think how to write right your tests*/
Product product = new Product();
product.setLikes(7L);
product.setQuantity(5L);
productRepository.save(product);
ExecutorService executor = Executors.newFixedThreadPool(2);
Lock lockService = new ReentrantLock();
Runnable taskForAlice = makeTaskForAlice(lockService);
Runnable taskForBob = makeTaskForBob(lockService);
executor.submit(taskForAlice);
executor.submit(taskForBob);
executorServiceMethod(executor);
}
/*------ Alice-----*/
private Runnable makeTaskForAlice(Lock lockService){
return () -> {
System.out.println("Thread-1 - Alice");
Product product;
lockService.lock();
try{
product = productService
.findById(1L, "Thread-1 - Alice");
}finally {
lockService.unlock();
}
setPause(1000L); /*a pause is needed in order for the 2nd transaction to attempt
read the line from which the 1st transaction started working*/
lockService.lock();
try{
product.setQuantity(6L);
product.setLikes(7L);
update(product,"Thread-1 - Alice");
}finally {
lockService.unlock();
}
System.out.println("Thread-1 - Alice - end");
};
}
/*------ Bob-----*/
private Runnable makeTaskForBob(Lock lockService){
return () -> {
/*the pause makes it possible to start the transaction first
from Alice*/
setPause(50L);
System.out.println("Thread-2 - Bob");
Product product;
lockService.lock();
try{
product = findProduct("Thread-2 - Bob");
}finally {
lockService.unlock();
}
setPause(3000L); /*a pause is needed in order for the 1st transaction to update
the string that the 2nd transaction is trying to work with*/
lockService.lock();
try{
product.setQuantity(5L);
product.setLikes(10L);
update(product,"Thread-2 - Bob");
}finally {
lockService.unlock();
}
System.out.println("Thread-2 - Bob - end");
};
}
private void update(Product product, String nameThread){
productService
.updateWithOptimisticLocking(product, nameThread);
}
private Product findProduct(String nameThread){
return productService
.findById(1L, nameThread);
}
private void setPause(long timeOut){
try {
TimeUnit.MILLISECONDS.sleep(timeOut);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
private void executorServiceMethod(ExecutorService executor){
try {
executor.awaitTermination(10L, TimeUnit.SECONDS);
} catch (InterruptedException e) {
e.printStackTrace();
}
executor.shutdown();
}
}