Background Job A
Picks a record and update the status as picked
It will process sequence of steps
Once everything is done, status is changed to complete
When there is an error encountered during the steps, status is changed to error
Another Job B does the following
Picks a record where the status is picked or error status
Process the sequence of steps
Job B - Reprocessing Job which handles the exception and system crash use case
Example,
Job A - When the application crashes some of the records will be left with status Picked
so inorder to handle the crash scenario and error scenario
Job B - looks for both picked and error status record
Issue:
Job A should take only the fresh records
Job A should handle the exception case and error case. Job B should not fetch the fresh records.
How to handle this situation?
You can use the time stamp here or check the date
Like
Job B should be only take yesterday or like previous hour timestamp record
So the fresh record are taken by job A and the previous hours which would be termed as old job would be taken by Job B
Related
I have a case where I want the job to run from Day 1 to Day 5 of a
month and when it's predecessor is success.
But what's happening
is the job runs once a day after start time even when predecessor
run multiple times through the day( I want my job to also refresh as
many times as predecessor do)
Job A
date_conditions: 1
run_calendar: D1_D5
start_times: "00:00"
condition: s(Job B)
I tried removing start_time but with calendar it takes a default start time of "00:00" anyway and gets scheduled for next day after first run.
Thanks in advance.
If I understand you correctly, you want Job A to run only on Days 1-5, but every time Job B runs on those days.
You could update your script logic for Job A so that it only truly runs on Day 1 to Day 5 (the rest of the month would be some kind of dummy script), then change the dependency so it's just dependent on Job B.
If you want to use the scheduler, recommend you clone Job B and schedule that to run on Days 1-5. Then change Job A to be dependent on the Job B clone.
In my application I need to log when each job is dispatched, retried and executed. The reason why I need this log is to detect whether on a failure the job is re-dispatched or not.
Also I need to keep 1 month long the job dispatch.
In another words I need to log:
Time dispatched,
Time redispatched,
Time finished
error that caused re-dispatch
where the job has been redispatched.
Is there some sort of trigger in laravel 5.7 that allows you to do custom logs during job dispatch?
So far I have seen that there are the jobs and failed_jobs tables that contains only ephemeral information only during the job execution. After job excecution records are removed.
If you want to build it yourself, then Job events
If you want to have a dashboard and logs for everything you mentioned, then Laravel Horizon
I want to use queue job to update data by time create to database
Example . i have table job_history
I will insert multiple record into job_history table by status (status = 0 not finish)
I want to handle all record with status = 0 by queue (time create ascending)
(It mean after processing the record 1 completed (update record with status = 1 finished or timeout by 60s )
then automatically next to the record 2,3... until the end (update status=1)
And when i create new record into job_history table then queue always listen to continue handle with status =0
=> I can handle by cronjob (but cron job only configured at least once a minute => That interrupts the work) ,
I want to handle if record 1 finished will continue 2 ( not wating 1 minute by cronjob)
I don't know laravel queue can do this job not ?
and how to setup it ?
In production in order tu consume queue jobs you can use supervisor.
You can read this doc : https://laravel.com/docs/7.x/queues#supervisor-configuration
queue:work handles a job at a time, sequentially, as long as there are jobs to do in the queue. You don't have to do anything special to achieve what you're asking for.
Also, you don't need (strictly) a supervisor to run queues, those can be handled with a cron and a relatively simple script to check if the worker is running and launching or restarting it as needed.
Yes, you can handle this situation by excuting this command:
queue:work --stop-when-empty
With the cron, it will check every minute if the queue is not empty. Then it will do all existing jobs in the queue. Then terminate.
I am working on a use case where I have a cron job scheduled (via quartz) which reads certain entries from db and process them.
Now in each schedule, I can get thousands of records which need to be processed. Processing each record takes time (in seconds/minutes). Currently all those records are getting processed on single node (node elected by quartz). Now my challenge is to parallelize these records processing. Please help me in solving below concerns :
How I can distribute these records/tasks to a cluster of machines
If any machine fails after processing few records then remaining records should be processed by healthy nodes in cluster
Get a signal that all record processing is finished.
Create cron jobs to run separately on each host at the desired frequency. You will need some form of lock on each record or some form of range lock on the record set to ensure that servers process mutually exclusive set of records.
e.g. : You can add following new field to all records:
Locked By Server:
Locked for Duration (or lock expiration time):
On each run, each cron picks a set of records that have expired or empty locks and then it aquires the lock on a small set of records by putting these two entries. Then it proceeds to process them. If it crashes or gets stuck the lock expires, otherwise it is released on completion.
I want to create a Scheduled Job in Oracle11g Express.
I am new to Job Scheduling and my search so far points to chains but as I want to create a Job out of a function that runs irregulary at a yet unknown date I believe that Chains aren't working for my case.
The Job will be created when a Procedure finishes, which determines its Scheduled Date X.
The Job will perform some critical changes which is why I don't want it to start while other regular scheduled jobs are running.
I want it to wait until the other jobs finish.
I only want to have it run once and than drop the job.
Is there some good practice for this case or some Option I have missed?