I'm using Spring Boot, JPA, Oracle 12C and a Typed Query below to select 'NEW' items to process. Once I've selected a 'NEW' item, I update its status so it's no longer eligible for selection but I'm seeing a concurrency issue with the same items getting picked up.
I read here that i needed to set a 'LockModeType.PESSIMISTIC_WRITE' on the query to prevent other Threads from selecting the same row but it doesn't appear to be working.
Have I missed something below or do i need another configuration to prevent concurrent threads from retrieving the same rows from my Table? Is the issue to do with the lock level or the Entity Manager not getting updated/refreshed?
My #Transactional Service:
#Override
#Transactional(isolation = Isolation.READ_COMMITTED, rollbackFor=RuntimeException.class)
public MyObject retrieveItemByStatus(StatusEnum status) {
return myRepository.retrieveItemByStatus(status);
}
The Query in my repository layer:
#Override
public MyObject retrieveItemByStatus(StatusEnum status) {
String sql = "SELECT t FROM myTable t WHERE status = :status ORDER BY id ASC";
try {
TypedQuery<MyObject> query = em.createQuery(sql, MyObject.class).setParameter("status", status);
query.setLockMode(LockModeType.PESSIMISTIC_WRITE);
query.setFirstResult(0);
query.setMaxResults(1);
MyObject myObject = (MyObject) query.getSingleResult();
if (myObject != null) {
myObject.setStatus(StatusEnum.IN_PROGRESS);
MyObject myUpdatedObject = em.merge(myObject);
return myUpdatedObject;
}
} catch (IllegalArgumentException iae) {
//some logging
} catch(NoResultException nrf) {
//some logging
} catch(Exception ex) {
//some logging
}
return null;
}
I can confirm this observation. I have several Lock-Modes tested with a H2-Database, and all worked as expected. Neither of the pessimistic Lock-Modes worked correctly in combination with an Oracle database. Therefore, the question: what is wrong with this code?
With Oracle two of these concurrent code executions yield the same data although the first should block the second one:
// Every Thread gets its own Hibernate session:
final Session session = HibernateSessionHolder.get();
session.getTransaction().begin();
final List<EventDeliveryDataDB> eddList =
session.createCriteria(EventDeliveryDataDB.class)
.setLockMode(LockMode.PESSIMISTIC_WRITE) // with READ the same
.add(eq("progress", NEW))
.list();
eddList.stream().forEach(eddElem -> eddElem.setProgress(IN_PROGRESS));
session.getTransaction().commit();
Hibernate Log:
Hibernate: select this_.DD_ID as DD_ID1_2_0_, this_.CHANNEL_NAME as CHANNEL_NAME2_2_0_, this_.created as created3_2_0_, this_.DELIVERY_TIME as DELIVERY_TIME4_2_0_, this_.ERROR_CODE as ERROR_CODE5_2_0_, this_.ERROR_MESSAGE as ERROR_MESSAGE6_2_0_, this_.EVENT_ID as EVENT_ID7_2_0_, this_.MAX_RETRIES as MAX_RETRIES8_2_0_, this_.PROGRESS as PROGRESS9_2_0_, this_.PROGRESS_ID as PROGRESS_ID10_2_0_, this_.RECIPIENT_CRID as RECIPIENT_CRID11_2_0_, this_.RETRY_COUNTER as RETRY_COUNTER12_2_0_, this_.RETRY_TIME as RETRY_TIME13_2_0_, this_.updated as updated14_2_0_ from HR.NOS_DELIVERY_DATA this_ where this_.PROGRESS=?
Hibernate: select this_.DD_ID as DD_ID1_2_0_, this_.CHANNEL_NAME as CHANNEL_NAME2_2_0_, this_.created as created3_2_0_, this_.DELIVERY_TIME as DELIVERY_TIME4_2_0_, this_.ERROR_CODE as ERROR_CODE5_2_0_, this_.ERROR_MESSAGE as ERROR_MESSAGE6_2_0_, this_.EVENT_ID as EVENT_ID7_2_0_, this_.MAX_RETRIES as MAX_RETRIES8_2_0_, this_.PROGRESS as PROGRESS9_2_0_, this_.PROGRESS_ID as PROGRESS_ID10_2_0_, this_.RECIPIENT_CRID as RECIPIENT_CRID11_2_0_, this_.RETRY_COUNTER as RETRY_COUNTER12_2_0_, this_.RETRY_TIME as RETRY_TIME13_2_0_, this_.updated as updated14_2_0_ from HR.NOS_DELIVERY_DATA this_ where this_.PROGRESS=?
Hibernate: select DD_ID from HR.NOS_DELIVERY_DATA where DD_ID =? for update
Hibernate: select DD_ID from HR.NOS_DELIVERY_DATA where DD_ID =? for update
Hibernate: update HR.NOS_DELIVERY_DATA set CHANNEL_NAME=?, created=?, DELIVERY_TIME=?, ERROR_CODE=?, ERROR_MESSAGE=?, EVENT_ID=?, MAX_RETRIES=?, PROGRESS=?, PROGRESS_ID=?, RECIPIENT_CRID=?, RETRY_COUNTER=?, RETRY_TIME=?, updated=? where DD_ID=?
Hibernate: update HR.NOS_DELIVERY_DATA set CHANNEL_NAME=?, created=?, DELIVERY_TIME=?, ERROR_CODE=?, ERROR_MESSAGE=?, EVENT_ID=?, MAX_RETRIES=?, PROGRESS=?, PROGRESS_ID=?, RECIPIENT_CRID=?, RETRY_COUNTER=?, RETRY_TIME=?, updated=? where DD_ID=?
AFAIK you can't block "reading" in oracle... pessimistic lock corresponds select for update which doesn't block other select statements ... It only forces it to read the old version of the data (before the select for update ran) ... It will block only other select for update statements (thus other queries having pessimistic lock)
Related
To give you the context about the issue I am facing, this is a customer table in a Postgres database and its status is update by EventHandler which picks up events from a single SQS queue. This error comes up
ObjectOptimisticLockingFailureException
Batch update returned unexpected row count from update [0]; actual row count: 0; expected: 1; statement executed: update customer set created_by=?, lock_id=?, modifiedat=?, modified_by=?, app_id=?, client=?, comments=?, customer_id=?, decision=?, source=? where id=? and lock_id=?; nested exception is org.hibernate.StaleStateException: Batch update returned unexpected row count from update [0]; actual row count: 0; expected: 1; statement executed: update customer set created_by=?, lock_id=?, modifiedat=?, modified_by=?, application_id=?, client_app=?, comments=?, customer_id=?, decision=?, source=? where id=? and lock_id=?
Now this error shows that batch update is happening but nowhere in this function I am doing batch updates. The code where this insert happens is
public Customer updateOrCreateCustomer(int customerId, String applicationId, String status) {
Customer customer = customerRepository.findByCustomerId(customerId);
if(customer == null) {
customer = new Customer();
customer.setCustomerId(customerId);
customer.setApplicationId(applicationId);
customer.status(status);
log.info("Creating Customer with Customer Id - {} Application Id - {}", customerId, applicationId);
} else {
customer.setStatus(status);
log.info("Updating Customer with Customer Id - {} Application Id - {}", customerId, applicationId);
}
return customerRepository.save(customer);
}
Also, in my application.yml, I have set the batch_size property of JPA to 50 but this is being used in a different API where I need to do batch inserts
jpa:
hibernate:
ddl-auto: none
open-in-view: false
properties:
generate_statistics: false
hibernate:
order_inserts: true
jdbc:
batch_size: 50
I don't know why JPA is doing batch updates in updateOrCreateCustomer function. What I guess is that many requests are coming at the same time so JPA sees that batch_size is set, so it automatically combines all these queries into one to optimize inserts/updates and does that. Please help
I think if your object is modified in another thread, and then you try to commit that same object from another thread at the same time, you maybe get ObjectOptimisticLockingFailureException
The solution I know is to raise the separation level so that the items are commited one by one and there is no synchronization error.
#Transactional(isolation = Isolation.ISOLATION_REPEATABLE_READ)
I am creating a batch program which will pick records having particular status say 'x' from Oracle Database and do some processing and update status to 'y' once processing is successful.
However, there would be multiple instances of same batch running and I have to make sure that no two batches pick same records in 'x' status. I am trying to achieve this by using 'Select ... for Update' in Spring data.
While testing same I can see that the lock is not restricting read in other sessions. I cannot post original code but Pseudo code for same is present below. Any reference to Source using #Lock & #Transactional will be helpful.
++ I am able to see correct query (Select For update) being written in trace logs, but still lock is not acquired.
---Starting Pseudo code---
#Controller
class {
#PutMapping
#Transactional
methodCall(){
City city = emMethod(Long id)
//My print statement that data has been selected
sleep thread for 15 sec
//Print statement after sleep
update Method call here
}
}
---End of Controller class
#Repository
Class Repos{
#Autowired
EntityManager em;
public City emMethod(Long id){
Query q = em.CreateQuery("Select C from City C where id =:id");
q.setParameter("id",id);
**q.setLockMode(LockModeType.PESSIMISTIC_WRITE)**;
List list = q.getResultList();
return (City) list.get(0);
}
#Modifying
myUpdateMethod(City city){//update body here};
}
}
When i want to modify the database via JPQL i have to mark the query as Transactional and Modiyfing. If i do so, the return type of the method representing the query has to be either void or int(representing the number of edited rows i think). Why are only the two return types allowed? If i do a HTTP-PUT request and update the object with an own JPQL query, i would like to return the updated object again. Whats the best way to do it if the return type of the query has to be void or int? Do i have to do a seperate query/request again which selects the object after it was updated?
EDIT:
Thats how i call the query:
if (inactivityListDTO.getProjectIds().size() > 0) {
projectRepository.updateProjectsIsArchivedByProjectIds(inactivityListDTO.getProjectIds(), inactivityListDTO.getIsArchived());
}
Thats the query:
#Transactional
#Modifying
#Query("UPDATE Project project SET project.isArchived = :isArchived,
project.archivedDate = current_date " +
"WHERE project.id IN :ids")
void updateProjectsIsArchivedByProjectIds(#Param("ids") List<Long> ids, #Param("isArchived") boolean isArchived);
Because it finally boils down to execute a standard UPDATE SQL in the DB , and the UPDATE in standard SQL only returns the number of records being updated and does not return a result set.
And yes , if you need get a record 's value after update , you have to query it again. Alternatively , you should consider using a JPA way to update a record , which first query the object , then update it by changing its state . Something like below (Assume you are using spring #Transactional to manage the transactional boundary):
#Transactional
public void changeEmployeeSalary(Integer employeeId , Integer salary){
Employee employee = entityManager.find(Employee.class , employeeId);
employee.setSalary(salary);
}
In this way , you do not need to query the record again after it is updated and you also do not need to manually write a UPDATE SQL.
I have the following bean definition:
#Bean
JdbcPollingChannelAdapter jdbcPollingChannelAdapter() {
// Get all the pending jobs
JdbcPollingChannelAdapter adapter = new JdbcPollingChannelAdapter(jdbcTemplate, "select id from poller_jobs where status = 'PENDING'");
// Immediately mark them as running so the next jdbc poll doesn't re-process them
adapter.setUpdateSql("update poller_jobs set status='RUNNING' where id in (:id)");
adapter.setMaxRowsPerPoll(100);
adapter.setRowMapper((r, i) -> r.getLong("id"));
return adapter;
}
And this fails because the row mapper maps to just a long id and so the adapter doesn't know how to grab the ID that I need in the update. Anyone know how to do that without needing to select * and map to a full object? That seems like more overhead than I really need.
This works for me:
<inbound-channel-adapter data-source="dataSource"
channel="target"
query="select id from item where status=2"
update="update item set status=10 where id in (:#root)"
update-per-row="true"
row-mapper="columnRowMapper"/>
<beans:bean id="columnRowMapper" class="org.springframework.jdbc.core.SingleColumnRowMapper"/>
So, the first is (:#root) as param placeholder just because the default setUpdateSqlParameterSourceFactory() is ExpressionEvaluatingSqlParameterSourceFactory, where the root object of evaluation context is the result of SELECT or, as it is stand by the update-per-row="true", each row in the ResultSet:
if (payload != null && this.updateSql != null) {
if (this.updatePerRow) {
for (Object row : payload) {
executeUpdateQuery(row);
}
}
else {
executeUpdateQuery(payload);
}
}
Therefore what you need in your configuration is this two lines of code:
adapter.setUpdateSql("update poller_jobs set status='RUNNING' where id in (:#root)");
adapter.setUpdatePerRow(true);
The SingleColumnRowMapper does the trick for really single column in the ResultSet, BTW.
I'm facing a singular problem...
I need to update an entity, but i don't know when it is really updated
My method is
#Override
#Transactional(isolation = Isolation.SERIALIZABLE)
public void lightOn(int idInterruttore) {
Interruttore interruttore = dao.findById(idInterruttore);
String inputPin = interruttore.getInputPin();
String pinName = interruttore.getRelePin();
GpioController gpio = interruttore.getGpio();
GpioPinDigitalOutput rele = gpio.provisionDigitalOutputPin(RaspiPin.getPinByName(pinName));
try {
DateTime date = new DateTime();
Date now = date.toDate();
int i = 1;
while (getInput(inputPin, gpio) != 1) {
if(i > 1){
logger.debug(String.format("Try n %s", i));
}
pushButton(rele);
Thread.sleep(1000);
i++;
}
dao.updateInterruttore(idInterruttore, now, true);
} catch (GpioPinExistsException | InterruptedException gpe) {
logger.error("GPIO già esistente", gpe);
} finally {
gpio.unprovisionPin(rele);
}
logger.debug(String.format("After the update status should be true and it's %s",
interruttore.isStato()));
}
updateInterruttore is (i used this form to be sure to call the commit after the update... I have the lock Option because multiple call can be done to this method but only the first must update
#Override
public void updateInterruttore(int idInterruttore, Date dateTime, boolean stato) {
Session session = getSession();
Transaction tx = session.beginTransaction();
String update = "update Interruttore i set i.dateTime = :dateTime, i.stato = :stato where idInterruttore = :idInterruttore";
session.createQuery(update).setTimestamp("dateTime", dateTime).setBoolean("stato", stato)
.setInteger("idInterruttore", idInterruttore).setLockOptions(LockOptions.UPGRADE).executeUpdate();
tx.commit();
}
}
Well... when I update the log says me:
After the update status should be true and it's false
This happens only the first time I call the method, the second time interruttore.isStato is correctly true.
Why this happens?
This happens because you're updating the database directly with the update statement. Hibernate does not update automatically an already loaded entity in this case. If you reload the entity after the call to dao.updateInterruttore you should get the updated data.
Two notes:
1) You are using a query to apply the update. In that case, Hibernate will no update the entity that is in the session. Unless you update the entity itself and call session.save(interruttore), then the entity will not be updated. (But the update shows up in the DB.) Furthermore, I don't understand why you just don't update the entity and save it via session.save().
2) You are annotating the service method with #Transactional. (Assuming that's Spring annotation) If you use JTA, your tx.commit() will have no effect. But once the method completes, your transaction is committed. (or rolled back if the method throws an exception) If you are not using JTA, then get rid of #Transactional and manage transaction in your DAO method, as you are doing. But that's considered bad practice.