I have a DAG that triggered by airflow scheduler and runs at 1 min passed midnight every day. I want to change the start time every day to be dynamic, one day to start at 1:00 am , the next day at 00:16 and etc. Is there any configuration in airflow to do this? If not what can I do?
As far as I know airflow doesnt support this. But you can do a workaround. You can schedule a dag to run midnight every day, and this dag change the start date of the other dag.
Related
I wanna remove just a time period without a date, from another time period.
for example, assume, I have a time period like 08:00 to 16:00, and I wanna remove 10:00 to 12:00 period from that period time.
actually, the first period of time has been converted to two periods of time. 08:00 to 10:00 and 12:00 to 16:00.
What is the best way to implement it? it should be implemented just with if else? or it has another best way to implement it.
I have a dbms_scheduler job which has to run every 4 hours, in timezone 'Europe/Berlin'
My question is: what happens on days with a change in daylight saving time?
Let's say, last runtime was 01:00 AM. Not time either is set from 02:00AM to 03:00AM or from 03:00AM to 02:00AM.
Is the next runtime of my job after 3 hours in the first and after 5 hours in the second case or will it be just after 4 hours, regardless of the change in daylight saving time?
In the Heroku documentation it's said that addons, such as Scheduler, add up to your montly hours, meaning you would have to pay if it adds up more than 30 hours (I have only one web dyno, so I'm using 720 montly hours, and the maximum free is 750 hours). So, how to calculate how much Scheduler takes from your montly hours? For example, if I have a daily task, how much would it take from my montly hours?
It means the wall clock time of the time taken for the jobs that are executed by scheduler to run.
So, for instance, if you're running a 1 minute job every hour, you'd be looking at 720 minutes of time every 30 days.
I believe this last answer is incorrect - the scheduler costs the same hourly as a regular dyno - right?
I have programmed a punch clock system. I need to modify it to comply with California overtime rules such that if someone works more than 8 hours in 24, they receive overtime. I am stumped on how to go about doing this that is not computationally intensive.
Our punches are rounded to 15 minute intervals, meaning people will punch in at 8:00 AM, 8:15 AM, 8:30 AM and so on.
So if someone starts at 8am on Monday, works a total of 8 hours, and starts at 7am on Tuesday, they get an hour of overtime?
Assume you have a list of start/stop date time pairs for a given employee. This list has to include start/stop date time pairs from the previous time period.
Get the first start/stop date time pair from the current pay period.
Get the previous start/stop date time pair.
Determine the interval in hours and fractions of hours between the previous start and the current start.
If the interval is greater than or equal to 24, get the next current pay period start/stop date time pair, and go to 2. Exit if no more start/stop date time pairs.
Else, if the interval is less than 24, calculate the overtime in the current start/stop date time pair. The lessor of (24 - interval) and the amount of hours worked in the current start/stop date time pair.
Get the next current pay period start/stop date time pair. Exit if no more start/stop date time pairs.
Hold the previous start/stop date time pair.
Go to 3.
I want to run my ruby script x times a day (the number might change) on my linux box. What would be the best way to do so if I do not want it to happen at the same time? I want the time (hour and minute) to be random
I was thinking of using at command. The script would be called by at in x hours/minutes or so and then the script would set up another call by at. Not sure if there is any better way or only ruby way.
I'd consider using the at program to run the programs (instead of using cron directly, because cron really only works on a fixed schedule). I'd also create a program (I'd use Perl; you'll use Ruby) to schedule a random delay until the next time the job is executed.
You'll need to consider whether it is crucial that the job is executed 'x' times in 24 hours, and how the randomness should work. What is the range of variation in times. For example, you might have a cron job run at midnight plus 7 minutes, say, which then schedules 'x' at jobs spaced evenly through the day, with a random deviation in the schedule of ±30 minutes. Or you might prefer an alternative that schedules a the jobs with an average gap of 24/x hours and a random deviation of some amount. The difference is that the first mechanism guarantees that you get x events in the day (unless you make things too extreme); the second might sometimes only get x-1 events, or x+1 events, in 24 hours.
I think scheduler solutions are bit limiting, to get most flexible random action, turn your script to daemon and code the loop / wait yourself.
For Ruby there seems to be this: http://raa.ruby-lang.org/project/daemons/
I guess you can setup a cronjob that calls on a bash script which delays execution by a random time but I don't know if you can do it somehow inside the cronjob.
You can find some information on how to do that on this site and if you don't know about crontab and cronjobs you can find more information about that here.
If you want to run X times a day, set your crontab entry to:
0 */X * * * command_to_run
where X is the hourly interval you want to fire your job on to get the desired number of executions/day. For instance, use 2 to fire off every two hours for a total of 12 executions/day.
In your code use this at the top to force it to sleep a random time up to that cron interval:
# How long the program takes to run, in seconds. Be liberal unless having
# two instances running is OK.
EXECUTION_TIME = 10
INTERVAL = 2 * 60 * 60 - EXECUTION_TIME
sleep(rand(INTERVAL))
The idea is that cron will start your program at a regular interval, but then it will sleep some random number of seconds within that interval before continuing.
Change the value for EXECUTION_TIME to however long you think it will take for the code to run, to give it a chance to finish before the next interval occurs. Change the "2" in the INTERVAL to whatever your cron interval is.
I haven't tested this but it should work, or at least get you on the right path.