I am using Quartz in clustered mode
I have some row lock contention on DB level caused by excessive call to :
org.quartz.jobStore.selectWithLockSQL
"SELECT * FROM QRTZ_LOCKS WHERE SCHED_NAME = :"SYS_B_0" AND LOCK_NAME = :1 FOR UPDATE"
I read quartz docs and is still not very clear to me why is above query is executed.
What is the purpose of having this row lock ?
Regards
The locks table is used by quartz for coordinating multiple schedulers when deployed in cluster mode. In a cluster only one node should fire the trigger, so a lock is used to avoid multiple nodes acquiring the same trigger.
From the clustering section of the documentation (http://quartz-scheduler.org/generated/2.2.1/html/qs-all/#page/Quartz_Scheduler_Documentation_Set%2Fre-cls_cluster_configuration.html%23):
Clustering currently only works with the JDBC-Jobstore (JobStoreTX or
JobStoreCMT), and essentially works by having each node of the cluster
share the same database. Load-balancing occurs automatically, with
each node of the cluster firing jobs as quickly as it can. When a
trigger's firing time occurs, the first node to acquire it (by placing
a lock on it) is the node that will fire it.
In my case, I was experiencing a similar issue. I was using quartz fir running jobs whose logic involved fetching data from a foreign db. Whenever the connection between the application db and foreign db stopped due to some reason and the connection came back up the issue of locks surfaced and we used to get messages like this in the database logs
2021-01-14 12:06:17.935 KST [46836] STATEMENT:
SELECT * FROM HVACQRTZ_LOCKS WHERE SCHED_NAME = 'schedulerFactoryBean' AND LOCK_NAME = $1 FOR UPDATE
2021-01-14 12:06:18.937 KST [46836] ERROR: current transaction is aborted, commands ignored until end of transaction block
To solve this issue I used this property of quartz and once after using this property the issue went away. By default, the foe update part will be there at the end of the query but since the default query is replaced by the query which I wrote in the property file the for update portion is gone and no locks appear now and everything seems to be working smoothly.
selectWithLockSQL: SELECT * FROM {0}LOCKS WHERE LOCK_NAME = ?
Related
I'm working on a app where I have some entities in the database that have a column representing the date until that particular entity is available for some actions. When it expires I need to change it's state, meaning updating a column representing it's state.
What I'm doing so far, whenever I ask the database for those entities to do something with them, I first check if they are not expired and if they are, I update them. I don't particularly like this approach, since that means I will have a bunch of records in the database that would be in the wrong state just because I haven't queried them. Another approach would be to have a periodic task that runs over those records and updates them as necessary. That I also don't like since again, I would have records in a inconsistent state and in this case, the first approach seems more reasonable.
Is there another way of doing this, am I missing something? I need to mention, I use spring-boot + hibernate for my application. The underlying db is Postgresql. Is there any technology specific trick I can use to obtain what I want?
in database there it no triger type expired. if you have somethind that expired and you should do somethig with that there is two solutions (you have wrote about then) : do some extra with expired before you use data , and some cron/task (it might be on db level or on server side).
I recomend you use cron approach. Here is explanation :
do something with expired before you get data :
updated before select
+: you update expired data before you need it , and here are questions - update only that you requested or all that expired... update all might be time consumed in case if from all records you need just 2 records and updated 2000 records that are not related you you working dataset.
-: long time to update all record ; if database is shared - access to db not only throth you application , logic related to expired is not executed(if you have this case); you need controll entry point where you should do something with expired and where you shouldn't ; if time expired in min , sec - then even after you execure logic for expired , in next sec new records might be expired too;also if you need update workflow logic for expired data handling you need keep it in one plase - in cron , in case with update before you do select you should update changed logic too.
CRON/TASK
-: you should spend time to configure it just once 30-60 mins max:) ;
+: it's executed in the background ; if your db is used not only by your application , expired data logic also be available; you don't have to check(and don't rememebr about it , and explaine about for new employee....) is there any staled data in your java code before select something; you do split logic between cares about staled data , and normal queries do db .
You can execute 'select for update' in cron and even if you do select during update time from server side query you will wait will staled data logic complets and you get in select up to date data
for spring :
spring scheduling documentation , simple example spring-quartz-schedule
for db level postgresql job scheduler
scheduler/cron it's best practices for such things
I'm looking for a way to save and immediately lock an entity on a DB in order to avoid that other thread access the entity before the thread creator ends.
I'm using Hibernate 4.3.11 and Spring 4.2.5.
Thanks in advance.
Although there is lock mode - LockMode.WRITE - but as the documentation states
A WRITE lock is obtained when an object is updated or inserted. This
lock mode is for internal use only and is not a valid mode for load()
or lock() (both of which throw exceptions if WRITE is specified)..
If it's just that you are only inserting rows then you cannot specifically lock the database rows using hibernate as the rows are not yet committed.
The moment your code (hibernate or without) inserts rows in database and not yet commits - there are transactional locks held which gets released on transaction commit. The nature of locks and the manner in which this internally happens is database specific. However if you are interested in locking some rows (already existing) , then you
can query the data using
session.get(TestEntity.class, 1, LockMode.PESSIMISTIC_WRITE);
This will hold a pessimistic lock (typically by issuing SELECT .... FOR UPDATE) for the duration of transaction and no other thread/transaction can modify the data on which lock has been taken.
A possible way should be increase transaction level to serializable.
This level ensure data is locked until is not used in other transaction.
Hibernate offer's two types of locks Optimistic and Pessimistic. Its straight forward.
1)Optimistic uses versioning where in it will have a version column in the database and check it before it updates or else throw the exception
2)Pessimistic is some thing like a database handles the locking on that row and it will get released after the operation is completed, there are few options are there which is similarly like how you imagine like read lock, write lock
https://docs.jboss.org/hibernate/orm/4.0/devguide/en-US/html/ch05.html
If you are using PostgreSQL I think the below example works:
#Query(value = """with ins_artist as (
insert into artist
values (301, 'Whoever1')
returning *
) select artist_id
from ins_artist
for update""", nativeQuery = true)
#Transactional(propagation = Propagation.REQUIRED)
Long insertArtist(); // returns artist ID
PS: I ran this query on https://postgres.devmountain.com/ . But it would need testing on a Java app.
In Quartz version 2.2 how can we find if a job is finished and when?! I have the job key .
First I tried to get the job trigger and see the trigger status.
List<? extends Trigger> triggers = sched.getTriggersOfJob(jobKey);
for (Trigger trigger : triggers) {
....
}
But I find that quartz deletes the job trigger from database when the trigger finish successfully.
I googled an find http://forums.terracotta.org/forums/posts/list/6791.page
Quartz cleans up all of its own unused data so that an administrator
doesn't have to delete records filling up the database (many users
have millions of triggers firing repeatedly. it is impractical and
performance-hindering to keep all that data around).
If you want a history of when triggers have fire, implement a
TriggerListener and record the info yourself, much as the
LoggingTriggerHistoryPlugin does.
Quartz 2.2 is likely to add a history feature with new api for
retrieving the data.
On the other hand I review the quartz code (http://www.quartz-scheduler.org/api/2.2.1/org/quartz/Trigger.CompletedExecutionInstruction.html) and find that the trigger state can be set to NOOP, RE_EXECUTE_JOB, SET_TRIGGER_COMPLETE, DELETE_TRIGGER, SET_ALL_JOB_TRIGGERS_COMPLETE, SET_TRIGGER_ERROR, SET_ALL_JOB_TRIGGERS_ERROR
I think this enumeration is used for failed trigger, But I wonder if there is a way to make Trigger persist.
My question is quite similar to this one but I need more guidance. I also read the Oracle context doc.
The current (test) trigger is :
CREATE OR REPLACE TRIGGER CHASSIS_DT_EVNT_AIUR_TRG_OLD AFTER DELETE OR INSERT OR UPDATE
OF ETA
ON CHASSITRANSPORTS
REFERENCING NEW AS New OLD AS Old
FOR EACH ROW
DECLARE
BEGIN
INSERT INTO TS_CHASSIS_DATE_EVENTS (CHASSISNUMBER,DATETYPE,TRANSPORTLEGSORTORDER,OLDDATE,CREATEDBY,CREATEDDATE,UPDATEDBY,UPDATEDDATE) VALUES (:old.chassino,'ETA',:old.sortorder,:old.eta,'xyz',sysdate,'xyz',sysdate);
EXCEPTION
WHEN OTHERS THEN
NULL;
END TS_CHASSIS_DT_EVNT_AIUR_TRG;
Now the 'CREATEDBY', 'UPDATEDBY' will be the web application users who have logged in and made the changes which caused the trigger execution, hence, these values need to be passed from the application.
The web application :
Is deployed in Websphere Application Server where the datasources are configured
As expected, is using db connection pooling
My question is which approach mentioned in the thread and the doc. should I take to avoid the 'concurrency' issues i.e the updates by the app. users in multiple sessions at the application level as well the db level should not interfere with each other.
I don't think any one of the approaches in that link would apply to you, primarily due to multi-user environment and connection pooling.
Connection pooling by nature allows different connections to share the same session. Setting a context (either sys_context or any other application context) is valid throughout the lifetime of the session. So two different connections can overwrite each other's values and read other's values. (concurrency issues)
I'd actually argue against doing an insert like this inside a trigger at all. It seems to me the insert you are doing is to write to a log table all updates that happened on the main table. If that is the case, why not insert to the log table at the time of making any updates to this table.
So the procedure that does UPDATE CHASSITRANSPORTS ... would also have another INSERT statement inside it that writes to the other table. If there is no procedure and it is a direct update statement from the application, then write a procedure for this.
You could say that there are multiple places where the same update happens and I'll suggest that in that scenario create an API for the base table CHASSITRANSPORTS that handles updates and so behind a black box also writes to the log table. Any place where you need to update that table column you'd use that API.
(I'm ignoring the fact that you are suppressing all errors in the trigger with WHEN OTHERS THEN NULL with the hope that this is probably just a small example)
When the BJ is currently RUNNING and if server is STOPPED, the BATCH JOB status in Spring Batch Admin still shows its RUNNING even after the server is stopped, it needs to be FAILED
Please help on this? Do we need to handle it manually or we can achieve it out of the box.
Need help on this?
the job repository (database implementation) reflects the last 'known' state. so, in the case where the job was running and the JVM crashed, then it will never be updated in the database.
in the event of the database being 'out-of-sync' with the JVM, then the process needs to be a manual one, there doesn't appear to be an out of the box solution for it. the simplest solution would be to execute a script on startup that checked the batch tables for any RUNNING jobs and then 'failed' them.
update batch_job_execution set STATUS = 'FAILED', EXIT_CODE = 'FAILED', EXIT_MESSAGE = 'FORCED UPDATE' where job_execution_id in (select job_execution_id from batch_job_execution where status = 'RUNNING');
one thing you will want to consider in this situation is if the JobRepository tables, and the jobs associated with them, are shared with another JVM. in this case, you may wish to do a pass that also evaluates if the job is still running beyond the maximum runtime of any history it has. (a subselect with max() end_time - create_time for the same job_name)