I am creating a batch program which will pick records having particular status say 'x' from Oracle Database and do some processing and update status to 'y' once processing is successful.
However, there would be multiple instances of same batch running and I have to make sure that no two batches pick same records in 'x' status. I am trying to achieve this by using 'Select ... for Update' in Spring data.
While testing same I can see that the lock is not restricting read in other sessions. I cannot post original code but Pseudo code for same is present below. Any reference to Source using #Lock & #Transactional will be helpful.
++ I am able to see correct query (Select For update) being written in trace logs, but still lock is not acquired.
---Starting Pseudo code---
#Controller
class {
#PutMapping
#Transactional
methodCall(){
City city = emMethod(Long id)
//My print statement that data has been selected
sleep thread for 15 sec
//Print statement after sleep
update Method call here
}
}
---End of Controller class
#Repository
Class Repos{
#Autowired
EntityManager em;
public City emMethod(Long id){
Query q = em.CreateQuery("Select C from City C where id =:id");
q.setParameter("id",id);
**q.setLockMode(LockModeType.PESSIMISTIC_WRITE)**;
List list = q.getResultList();
return (City) list.get(0);
}
#Modifying
myUpdateMethod(City city){//update body here};
}
}
When i want to modify the database via JPQL i have to mark the query as Transactional and Modiyfing. If i do so, the return type of the method representing the query has to be either void or int(representing the number of edited rows i think). Why are only the two return types allowed? If i do a HTTP-PUT request and update the object with an own JPQL query, i would like to return the updated object again. Whats the best way to do it if the return type of the query has to be void or int? Do i have to do a seperate query/request again which selects the object after it was updated?
EDIT:
Thats how i call the query:
if (inactivityListDTO.getProjectIds().size() > 0) {
projectRepository.updateProjectsIsArchivedByProjectIds(inactivityListDTO.getProjectIds(), inactivityListDTO.getIsArchived());
}
Thats the query:
#Transactional
#Modifying
#Query("UPDATE Project project SET project.isArchived = :isArchived,
project.archivedDate = current_date " +
"WHERE project.id IN :ids")
void updateProjectsIsArchivedByProjectIds(#Param("ids") List<Long> ids, #Param("isArchived") boolean isArchived);
Because it finally boils down to execute a standard UPDATE SQL in the DB , and the UPDATE in standard SQL only returns the number of records being updated and does not return a result set.
And yes , if you need get a record 's value after update , you have to query it again. Alternatively , you should consider using a JPA way to update a record , which first query the object , then update it by changing its state . Something like below (Assume you are using spring #Transactional to manage the transactional boundary):
#Transactional
public void changeEmployeeSalary(Integer employeeId , Integer salary){
Employee employee = entityManager.find(Employee.class , employeeId);
employee.setSalary(salary);
}
In this way , you do not need to query the record again after it is updated and you also do not need to manually write a UPDATE SQL.
I'm using Spring Boot, JPA, Oracle 12C and a Typed Query below to select 'NEW' items to process. Once I've selected a 'NEW' item, I update its status so it's no longer eligible for selection but I'm seeing a concurrency issue with the same items getting picked up.
I read here that i needed to set a 'LockModeType.PESSIMISTIC_WRITE' on the query to prevent other Threads from selecting the same row but it doesn't appear to be working.
Have I missed something below or do i need another configuration to prevent concurrent threads from retrieving the same rows from my Table? Is the issue to do with the lock level or the Entity Manager not getting updated/refreshed?
My #Transactional Service:
#Override
#Transactional(isolation = Isolation.READ_COMMITTED, rollbackFor=RuntimeException.class)
public MyObject retrieveItemByStatus(StatusEnum status) {
return myRepository.retrieveItemByStatus(status);
}
The Query in my repository layer:
#Override
public MyObject retrieveItemByStatus(StatusEnum status) {
String sql = "SELECT t FROM myTable t WHERE status = :status ORDER BY id ASC";
try {
TypedQuery<MyObject> query = em.createQuery(sql, MyObject.class).setParameter("status", status);
query.setLockMode(LockModeType.PESSIMISTIC_WRITE);
query.setFirstResult(0);
query.setMaxResults(1);
MyObject myObject = (MyObject) query.getSingleResult();
if (myObject != null) {
myObject.setStatus(StatusEnum.IN_PROGRESS);
MyObject myUpdatedObject = em.merge(myObject);
return myUpdatedObject;
}
} catch (IllegalArgumentException iae) {
//some logging
} catch(NoResultException nrf) {
//some logging
} catch(Exception ex) {
//some logging
}
return null;
}
I can confirm this observation. I have several Lock-Modes tested with a H2-Database, and all worked as expected. Neither of the pessimistic Lock-Modes worked correctly in combination with an Oracle database. Therefore, the question: what is wrong with this code?
With Oracle two of these concurrent code executions yield the same data although the first should block the second one:
// Every Thread gets its own Hibernate session:
final Session session = HibernateSessionHolder.get();
session.getTransaction().begin();
final List<EventDeliveryDataDB> eddList =
session.createCriteria(EventDeliveryDataDB.class)
.setLockMode(LockMode.PESSIMISTIC_WRITE) // with READ the same
.add(eq("progress", NEW))
.list();
eddList.stream().forEach(eddElem -> eddElem.setProgress(IN_PROGRESS));
session.getTransaction().commit();
Hibernate Log:
Hibernate: select this_.DD_ID as DD_ID1_2_0_, this_.CHANNEL_NAME as CHANNEL_NAME2_2_0_, this_.created as created3_2_0_, this_.DELIVERY_TIME as DELIVERY_TIME4_2_0_, this_.ERROR_CODE as ERROR_CODE5_2_0_, this_.ERROR_MESSAGE as ERROR_MESSAGE6_2_0_, this_.EVENT_ID as EVENT_ID7_2_0_, this_.MAX_RETRIES as MAX_RETRIES8_2_0_, this_.PROGRESS as PROGRESS9_2_0_, this_.PROGRESS_ID as PROGRESS_ID10_2_0_, this_.RECIPIENT_CRID as RECIPIENT_CRID11_2_0_, this_.RETRY_COUNTER as RETRY_COUNTER12_2_0_, this_.RETRY_TIME as RETRY_TIME13_2_0_, this_.updated as updated14_2_0_ from HR.NOS_DELIVERY_DATA this_ where this_.PROGRESS=?
Hibernate: select this_.DD_ID as DD_ID1_2_0_, this_.CHANNEL_NAME as CHANNEL_NAME2_2_0_, this_.created as created3_2_0_, this_.DELIVERY_TIME as DELIVERY_TIME4_2_0_, this_.ERROR_CODE as ERROR_CODE5_2_0_, this_.ERROR_MESSAGE as ERROR_MESSAGE6_2_0_, this_.EVENT_ID as EVENT_ID7_2_0_, this_.MAX_RETRIES as MAX_RETRIES8_2_0_, this_.PROGRESS as PROGRESS9_2_0_, this_.PROGRESS_ID as PROGRESS_ID10_2_0_, this_.RECIPIENT_CRID as RECIPIENT_CRID11_2_0_, this_.RETRY_COUNTER as RETRY_COUNTER12_2_0_, this_.RETRY_TIME as RETRY_TIME13_2_0_, this_.updated as updated14_2_0_ from HR.NOS_DELIVERY_DATA this_ where this_.PROGRESS=?
Hibernate: select DD_ID from HR.NOS_DELIVERY_DATA where DD_ID =? for update
Hibernate: select DD_ID from HR.NOS_DELIVERY_DATA where DD_ID =? for update
Hibernate: update HR.NOS_DELIVERY_DATA set CHANNEL_NAME=?, created=?, DELIVERY_TIME=?, ERROR_CODE=?, ERROR_MESSAGE=?, EVENT_ID=?, MAX_RETRIES=?, PROGRESS=?, PROGRESS_ID=?, RECIPIENT_CRID=?, RETRY_COUNTER=?, RETRY_TIME=?, updated=? where DD_ID=?
Hibernate: update HR.NOS_DELIVERY_DATA set CHANNEL_NAME=?, created=?, DELIVERY_TIME=?, ERROR_CODE=?, ERROR_MESSAGE=?, EVENT_ID=?, MAX_RETRIES=?, PROGRESS=?, PROGRESS_ID=?, RECIPIENT_CRID=?, RETRY_COUNTER=?, RETRY_TIME=?, updated=? where DD_ID=?
AFAIK you can't block "reading" in oracle... pessimistic lock corresponds select for update which doesn't block other select statements ... It only forces it to read the old version of the data (before the select for update ran) ... It will block only other select for update statements (thus other queries having pessimistic lock)
I have the following bean definition:
#Bean
JdbcPollingChannelAdapter jdbcPollingChannelAdapter() {
// Get all the pending jobs
JdbcPollingChannelAdapter adapter = new JdbcPollingChannelAdapter(jdbcTemplate, "select id from poller_jobs where status = 'PENDING'");
// Immediately mark them as running so the next jdbc poll doesn't re-process them
adapter.setUpdateSql("update poller_jobs set status='RUNNING' where id in (:id)");
adapter.setMaxRowsPerPoll(100);
adapter.setRowMapper((r, i) -> r.getLong("id"));
return adapter;
}
And this fails because the row mapper maps to just a long id and so the adapter doesn't know how to grab the ID that I need in the update. Anyone know how to do that without needing to select * and map to a full object? That seems like more overhead than I really need.
This works for me:
<inbound-channel-adapter data-source="dataSource"
channel="target"
query="select id from item where status=2"
update="update item set status=10 where id in (:#root)"
update-per-row="true"
row-mapper="columnRowMapper"/>
<beans:bean id="columnRowMapper" class="org.springframework.jdbc.core.SingleColumnRowMapper"/>
So, the first is (:#root) as param placeholder just because the default setUpdateSqlParameterSourceFactory() is ExpressionEvaluatingSqlParameterSourceFactory, where the root object of evaluation context is the result of SELECT or, as it is stand by the update-per-row="true", each row in the ResultSet:
if (payload != null && this.updateSql != null) {
if (this.updatePerRow) {
for (Object row : payload) {
executeUpdateQuery(row);
}
}
else {
executeUpdateQuery(payload);
}
}
Therefore what you need in your configuration is this two lines of code:
adapter.setUpdateSql("update poller_jobs set status='RUNNING' where id in (:#root)");
adapter.setUpdatePerRow(true);
The SingleColumnRowMapper does the trick for really single column in the ResultSet, BTW.
I am trying to update a Company object in a new transaction and expecting the same object to be retrieved having updated parameters. But they are not :( Name does not change. 'after' and 'before' are the same. Database was updated but the outer transaction doesn't know about this. Do you know any workarounds for such a case?
#Transactional(propagation = Propagation.REQUIRES_NEW, readOnly = false)
public Status newTransactionTest() {
logger.info("newTransactionTest() INNER");
Company company = companyDAO.findOne(10000013);
company.setName(company.getName() + "X");
return Status.OK;
}
#Transactional(propagation = Propagation.REQUIRED, readOnly = false)
public Status test() {
logger.info("test() BEFORE");
Company company1 = companyDAO.findOne(10000013);
String before = company1.getName();
// run in a new transaction
applicationContext.getBean(beanName, AdminService.class).newTransactionTest();
logger.info("test() AFTER");
Company company2 = companyDAO.findOne(10000013);
String after = company2.getName();
logger.info("COMPANY NAME BEFORE: " + before);
logger.info("COMPANY NAME AFTER: " + after);
return Status.OK;
}
and the logs are:
test() BEFORE
connection: 122 select company0_.name as name7_4_0_ ... where company0_.id=5000062
newTransactionTest() INNER
connection: 123 select company0_.name as name7_4_0_ ... where company0_.id=5000062
connection: 123 update Company set name='TestorexX' where id=5000062
connection: 123 commit
test() AFTER
connection: 122 select company0_.name as name7_4_0_ ... where company0_.id=5000062
COMPANY NAME BEFORE: Testorex
COMPANY NAME AFTER: Testorex
Ok, finally I have fixed this bug. The problem was with a default isolation level which was on my local mySQL instance and Cloud SQL set to REPEATABLE-READ. To check these settings I used:
SHOW VARIABLES WHERE Variable_name ='tx_isolation'
so my repeated queries returned the same result because of cashing done on the db level not as I expected on hibernate/spring
To change REPEATABLE-READ to READ COMMITTED I added this to my persistence.xml
<property name="hibernate.connection.isolation">2</property>
Where
1: READ UNCOMMITTED
2: READ COMMITTED
4: REPEATABLE READ
8: SERIALIZABLE
Now everything works fine as it is expected. In the beginning of every new transaction hibernate does
SET SESSION TRANSACTION ISOLATION LEVEL READ COMMITTED
The outer transaction sees changes from the inner!