I have an application where users can schedule to run reports at a certain time. The data for the reports is populated by an overnight batch process which might get delayed (occasionally) due to upstream issues. When that happens, the reports need to be re-tried every few minutes until we have data. I have a db table that records the status of the batch, there will be a row in that table when the data becomes available for this report.
I am thinking along these lines: When the data is not available, the task will cancel itself, but before it does that, it will dynamically schedule another copy of itself to execute every 5 mins. Once the data becomes available, the repeating task will run once and cancel all future runs.
Is this possible? Am I on the right track?
Any help is much appreciated.
Thanks.
Related
I have a large set of users in my project like 50m.
I should create a playlist for each user every day, for doing this, I'm currently using this method:
I have a column in my users' table that holds the latest time of creating a playlist for that user, and I name it last_playlist_created_at.
I run a query on the users' table and get the top 1000s, that selects the list of users which their last_playlist_created_at is past one day and sort the result in ascending order by last_playlist_created_at
After that, I run a foreach on the result and publish a message for each in my message-broker.
Behind the message-broker, I start around 64 workers to process the messages (create a playlist for the user) and update last_playlist_created_at in the users' table.
If my message-broker messages list was empty, I will repeat these steps (While - Do-While)
I think the processing method is good enough and can be scalable as well,
but the method we use to create the message for each user is not scalable!
How should I do to dispatch a large set of messages for each of my users?
Ok, so my answer is completely based on your comment where you mentioned that you use while(true) to check if the playlist needs to be updated which does not seem so trivial.
Although this is a design question and there are multiple solutions, here's how I would solve it.
First up, think of updating the playlist for a user as a job.
Now, in your case this is a scheduled Job. ie. once a day.
So, use a scheduler to schedule the next job time.
Write a Scheduled Job Handler to push this to a Message Queue. This part is just to handle multiple jobs at the same time where you could control the flow.
Generate the playlist for the user based on the job. Create a Schedule event for the next day.
You could persist Scheduled Job data just to avoid race conditions.
Env: Oracle 12c R2
Trying to understand what the best approach would be to set up an Oracle DBMS_SCHEDULER job that would be used to monitor a DBMS_ALERT trigger that checks when a specific column value changes within a table.
The thing is, this table column value change will sometimes occur on a frequent basis and sometimes it may only occur twice a day but I will need to monitor this column change via the DBMS_ALERT.
The trigger I have is as follows and I have a procedure called check_signal that checks for the signal that I wish to use within the DBMS_SCHEDULER job.
The goal that I am trying to achieve is that I am going to have the situation where I will need to run say, three jobs:
Job1
Job2
Job3
The thing is, the payload returned from Job1 is required and and passed as parameters into Job2 and again, the payload returned from Job2 is required and passed as parameters into Job3.
It is this wait/alert that I am trying to achieve through the use of DBMS_ALERTS.
create or replace trigger my_tab_upd after update of status on my_tab for each row
begin
dbms_alert.signal('mystatusalert', 'changed from '||:old.status||' to '||:new.status||'.');
end;
/
This will be used via a web-based application which is used by multiple users.
Just unsure on how to setup this scheduled job that will continuously check for the alert and then be used within the web app.
If there is a better means than DBMS_ALERT, then please let me know.
The general answer is simple, while polling for events every N seconds you get an average delay N/2 seconds and maximal delay of N seconds.
In context of DBMS_ALERT you should re-think this approach, as this will implement polling with wait on the event.
The periodically executed jobs make basically tho thinks:
DBMS_ALERT.REGISTER on an event name
wait with DBMS_ALERT.WAITONE
Assume that the DBMS_SCHEDULER jobs runs every 10 seconds and it is started in the phase with frequent signalling. So the first execution returns quickly after receiving an event.
The second execution falls in the quite period, so the job will wait hours to get an event.
I think this is not what you expect as
1) the waiting job will have an open session - what you want to avoid as follows from you other question
You may use timeout = 0 in the DBMS_ALERT.WAITONE, but this will return close to no events, except those fired accidentally between the REGISTER and WAITONE
2) if in the first 10 seconds two events are signalled, the second one will be lost as at the signaling time the subscribing job is not active and no registration exists.
I see that any updates to scheduled script execution is not captured in the update set.
What is the criteria to have changes captured?
Can we manually configure the list of items to be and not to be captured in updates set.?
Tables with the attribute update_synch set to true are captured in update sets. This is the attribute set on the collection entry in sys_dictionary.
Scheduled script execution definitions (sysauto_script) should actually be captured in update sets, but the actual sys_trigger record which actually causes the scheduled script to be executed per the schedule is NOT update_synch'd, and that's by design. The sys_trigger table is modified heavily by the actual scheduler service (e.g. resetting next action on every execution, run once jobs created and destroyed for things like workflow timers)
Technically, you could add the update_synch attribute to a sys_dictionary collection entry to cause it to be captured by update sets, but that is highly ill-advised, unless you really know what you're doing.
You can manually add non-update-synch'd records to your update set ad-hoc by way of a script described on the servicenowguru website.
I am using Spring batch JDBCCursorItemReader to read set of data from a table. Once data is read spring batch will process each row in a chunk(reader, processor, writer). Now I want to update/delete those records which my reader fetched to avoid reprocessing by another instance of same job. Can someone please tell me how can I do this in reader?
Thanks
Like it has been pointed out this might be a bad design idea. However if your sure this is what you want to do,
create a two step job,
step a, with commit interval as 1
Read the record
Write the updated record with the current job execution id
step b
Read the record where job execution id is current job execution id
process and update as needed
Notes
I do not recommend this approach for reasons stated in the comments
A commit interval of 1 would kill you performance wise so this approach if ever used should be for a low volume job only.
Hi I am new to Oracle Scheduler. My question is - Can we give both repeat interval and event condition in the Schedule object for a single job?
I have this requirement in job scheduling - A job should run at a scheduled time, but only if a certain event has occured.
For eg.
Job1 should run
- at 10 am every day
- but only if same job from yesterday is not running anymore. (This I gonna figure out based on the table entry.) So the event gonna be a cell entry say 'ENDED' in the table job_statuses.
Would be easier if I can give both info in the same job. Else another approach I gonna try is - Schedule the job based on time. If the earlier instance is still running , reschedule the job based on event. But this looks clumsy.
Thanks in advance.
Mayank
I'd encode the condition in the PL/SQL of the procedure itself. i.e. it runs at 10am every day, but the first thing it does is check if the previous job had finished successfully.
What you could do is create 3 jobs
EVENT_JOB
REPEAT_JOB
ACTUAL_WORK_JOB
EVENT_JOB and REPEAT_JOB just start ACTUAL_WORK_JOB. If that is already - or still - running, you get an error on which you can react accordingly.