I've got a strange issue.
I use a custom command to share, via FB api, some url.
Now this command is scheduled to a fixed cron configuration but I would like this to be randomic, maybe between time ranges.
I tried using a random sleep function inside command, but I got strange behavior.
Did anyone already have to face this problem?
One solution is to put the code you want to run into a Queued Job, and then delay that job a random amount. For example, this code will run at a random time every hour (the same can easily be extrapolated to random time every day or on some other interval):
$schedule
->call(function () {
$job = new SomeQueuedJob();
// Do it at some random time in the next hour.
$job->delay(rand(0, 60 * 59));
dispatch($job);
})
->hourly();
You can generate random time at beginning.
protected function schedule(Schedule $schedule)
{
$expression = \Cache::get('schedule:expressions:inspire');
if (!$expression) {
$randomMinute = random_int(0, 59);
// hour between 8:00 and 19:00
$hourRange = range(8, 19);
shuffle($hourRange);
// execute twice a day
$randomHours = array_slice($hourRange, 0, 2);
$expression = $randomMinute . ' ' . implode($randomHours, ',') . ' * * * *';
\Cache::put('schedule:expressions:inspire', $expression, Carbon::now()->endOfDay());
}
$schedule->command('inspire')->cron($expression);
}
Related
I tried delaying each notification (or job) using Laravel queues. Still, instead of the notifications being delayed one after the other, the delay is applied to all the notifications at once. However, the delay is only applied to the first notification, and the rest are sent out simultaneously as the first message. Please, can anyone in this community help me?
This is part of the method that gets only the applicants whose admissions are accepted or rejected and then notify them individually. Unfortunately, the mail server we are using only allows sending a maximum of 30 emails per 1 minute; therefore, I need to use queuing to limit sending only (not more than) 1 email per 2 seconds.
foreach ($applications as $application)
{
$application_id = $application->id;
$admission = Admission::where('application_id', $application_id)->first();
if (empty($admission->admission_no) & $application->status === 5) {
$this->generateAdmissionNumber($application->id, $application->course_id);
$application->update(['done' => 1]);
$admissions = Admission::where('course_id', '=', $id)->get();
foreach ($admissions as $admission) {
$admission->update(['done' => 1]);
}
}
$application->done = 1;
$application->update();
$user = $application->users;
if ($user->notify_if_decision_is_made === 1)
{
$user->notify((new DecisionNotification($user))
->delay(Carbon::now()->addSeconds(2)));
}
}
I followed the guides but still have no solution. I looked at many solutions here but did not see any issues like this one.
I set QUEUE_CONNECTION=database in my .env file, added shouldQueue in the DecisionNotification file, and run php artisan queue:work from the terminal to execute the jobs. I appreciate any help.
The ->delay() only gives a delay to the running job. As your loop is running almost instantly, it won't delay put a delay of 2 seconds between each job.
One way to achieve what you want is:
foreach ($applications as $i => $application) {
// your code chunk
$user->notify((new DecisionNotification($user))->delay($i * 2)));
}
This results in delays as:
Job 1: 0s
Job 2: 2s
Job 3: 4s
Job 4: 6s
Job 5: 8s
...
By the way, ->delay() also accepts integer as second.
You can use sleep() function, which holds the loop for defined period, for example if you want to hold loop for 2 seconds, use sleep(2).
Your code should be:
foreach ($applications as $application) {
$application_id = $application->id;
$admission = Admission::where('application_id', $application_id)->first();
if (empty($admission->admission_no) & $application->status === 5) {
$this->generateAdmissionNumber($application->id, $application->course_id);
$application->update(['done' => 1]);
$admissions = Admission::where('course_id', '=', $id)->get();
foreach ($admissions as $admission) {
$admission->update(['done' => 1]);
}
}
$application->done = 1;
$application->update();
$user = $application->users;
if ($user->notify_if_decision_is_made === 1) {
$user->notify((new DecisionNotification($user))->delay(Carbon::now()->addSeconds(2)));
}
sleep(2);
}
Hope this will be useful.
Read More about sleep()
Is it possible to get the stage view time values programatically?
I would like to print within the console the stage times for all the stages including the stage titles.
Something like:
Build Code: 19s
Unit Tests: 7s
Integration Tests: 17s
Looking at the plugin code I can see how it's rendered through the formatter, but I can't figure out how I could access those values through a Jenkinsfile.
In the source code it looks like millisecond time is being used. I can convert that easily to better human readable time.
/**
* Calculate how long something took from start to end
* #param start Start time in milliseconds
* #param end End Time in milliseconds
* #return Duration as String from start to end
*/
def calculateDuration(start, end) {
long elapsedTime = end - start
Long second = (elapsedTime / 1000).longValue() % 60;
Long minute = (elapsedTime / (1000 * 60)).longValue() % 60;
Long hour = (elapsedTime / (1000 * 60 * 60)).longValue() % 24;
Long remainderMillis = elapsedTime % 1000
return "${hour}h ${minute}m ${second}s ${remainderMillis}ms"
}
So this isn't going to be the super easy solution you're looking for, but I didn't know how to do this either and wanted to try.
Basically I'm writing simple HTML to a file at the OS level, and at the end publishing it via the Rich Text Publisher plugin. It then displays on the build page.
I'm not sure how to properly format the duration into hh:mm:ss
def logit(logMessage) {
logFile.append(logMessage + "\n")
}
def calculateDuration(start, end) {
long elapsedTime = end - start
Long second = (elapsedTime / 1000).longValue() % 60;
Long minute = (elapsedTime / (1000 * 60)).longValue() % 60;
Long hour = (elapsedTime / (1000 * 60 * 60)).longValue() % 24;
Long remainderMillis = elapsedTime % 1000
return "${hour}h ${minute}m ${second}s ${remainderMillis}ms"
}
node () {
stage('Create logfile') {
sh "rm -f /tmp/log.html"
logFile = new File("/tmp/log.html")
logit("<html>")
logit(" <body>")
}
stage('Time this stage') {
start = System.currentTimeMillis()
logit("Start time " + start + "<br>")
sleep(3)
end = System.currentTimeMillis()
logit("End time " + end + "<br>")
dur = calculateDuration(start, end)
println "Duration: " + dur
logit("Duration: " + dur + "<br>")
}
stage('Publish') {
logit(" </body>")
logit("</html>")
rtp (nullAction: '1', stableText: '${FILE:/tmp/log.html}')
}
}
Ironically, years later I have discovered a better solution that gives me exactly what I needed, which was the time by stage.
Depending on the type of Jenkins job, Freeestyle or Multibranch you can look at the exposed Jenkins API endpoints to get the information.
Freestyle Endpoint: env.BUILD_URL/api/json?pretty=true
Multibranch endpoint: env.BUILD_URL/wfapi/describe
It's a JSON response, so it's easy to parse and store by using readJSON Jenkins DSL.
Below you can see an example response for a freestyle job. Sorry for the blackout but didn't want to expose corporate dir structures.
When is laravel executing the yearly() function of it's task scheduling? Is it from the first time it was used or is it each year on 01.01.YYYY?
I checked up the laravel docs and several pages, but I wasn't able to find out the answer to this question.
$schedule->call(function () {
DB::table("invoice_settings")
->where("key", "invoice_number")
->update(["value" => 860001]);
})->yearly();
I expect that it's running on the 01.01.YYYY.
Laravel's yearly coresponds to crontab's yearly event. Described here:
https://crontab.guru/every-year
This is the cron produced:
0 0 1 1 *
Which runs:
At 00:00 on day-of-month 1 in January.
The yearly function is defined in:
vendor/laravel/framework/src/Illuminate/Console/Scheduling/ManagesFrequencies.php
Here you can find the daily, monthly and yearly, and other frequency functions.
public function daily()
{
return $this->spliceIntoPosition(1, 0)
->spliceIntoPosition(2, 0);
}
public function monthly()
{
return $this->spliceIntoPosition(1, 0)
->spliceIntoPosition(2, 0)
->spliceIntoPosition(3, 1);
}
public function yearly()
{
return $this->spliceIntoPosition(1, 0)
->spliceIntoPosition(2, 0)
->spliceIntoPosition(3, 1)
->spliceIntoPosition(4, 1);
}
As in the official Laravel documentation is written that the daily function runs daily at midnight, since the yearly function is defined in the same way it will run at the 1/1/YYYY at midnight.
I am using Firebase realtime database and overtime there is a lot of stale data in it and I have written a script to delete the stale content.
My Node structure looks something like this:
store
- {store_name}
- products
- {product_name}
- data
- {date} e.g. 01_Sep_2017
- some_event
Scale of the data
#Stores: ~110K
#Products: ~25
Context
I want to cleanup all the data which is like 30 months old. I tried the following approach :-
For each store, traverse all the products and for each date, delete the node
I ran ~30 threads/script instances and each thread is responsible for deleting a particular date of data in that month. The whole script is running for ~12 hours to delete a month data with above structure.
I have placed a limit/cap on the number of pending calls in each script and it is evident from logging that each script reaches the limit very quickly and speed of firing the delete call is much faster than speed of deletion So here firebase becomes a bottleneck.
Pretty evident that I am running purge script at client side and to gain performance script should be executed close to the data to save network round trip time.
Questions
Q1. How to delete firebase old nodes efficiently ?
Q2. Is there a way we can set a TTL on each node so that it cleans up automatically ?
Q3. I have confirmed from multiple nodes that data has been deleted from the nodes but firebase console is not showing decrease in data. I also tried to take backup of data and it still is showing some data which is not there when I checked the nodes manually. I want to know the reason behind this inconsistency.
Does firebase make soft deletions So when we take backups, data is actually there but is not visible via firebase sdk or firebase console because they can process soft deletes but backups don't ?
Q4. For the whole duration my script is running, I have a continuous rise in bandwidth section. With below script I am only firing delete calls and I am not reading any data still I see a consistency with database read. Have a look at this screenshot ?
Is this because of callbacks of deleted nodes ?
Code
var stores = [];
var storeIndex = 0;
var products = [];
var productIndex = -1;
const month = 'Oct';
const year = 2017;
if (process.argv.length < 3) {
console.log("Usage: node purge.js $beginDate $endDate i.e. node purge 1 2 | Exiting..");
process.exit();
}
var beginDate = process.argv[2];
var endDate = process.argv[3];
var numPendingCalls = 0;
const maxPendingCalls = 500;
/**
* Url Pattern: /store/{domain}/products/{product_name}/data/{date}
* date Pattern: 01_Jan_2017
*/
function deleteNode() {
var storeName = stores[storeIndex],
productName = products[productIndex],
date = (beginDate < 10 ? '0' + beginDate : beginDate) + '_' + month + '_' + year;
numPendingCalls++;
db.ref('store')
.child(storeName)
.child('products')
.child(productName)
.child('data')
.child(date)
.remove(function() {
numPendingCalls--;
});
}
function deleteData() {
productIndex++;
// When all products for a particular store are complete, start for the new store for given date
if (productIndex === products.length) {
if (storeIndex % 1000 === 0) {
console.log('Script: ' + beginDate, 'PendingCalls: ' + numPendingCalls, 'StoreIndex: ' + storeIndex, 'Store: ' + stores[storeIndex], 'Time: ' + (new Date()).toString());
}
productIndex = 0;
storeIndex++;
}
// When all stores have been completed, start deleting for next date
if (storeIndex === stores.length) {
console.log('Script: ' + beginDate, 'Successfully deleted data for date: ' + beginDate + '_' + month + '_' + year + '. Time: ' + (new Date()).toString());
beginDate++;
storeIndex = 0;
}
// When you have reached endDate, all data has been deleted call the original callback
if (beginDate > endDate) {
console.log('Script: ' + beginDate, 'Deletion script finished successfully at: ' + (new Date()).toString());
process.exit();
return;
}
deleteNode();
}
function init() {
console.log('Script: ' + beginDate, 'Deletion script started at: ' + (new Date()).toString());
getStoreNames(function() {
getProductNames(function() {
setInterval(function() {
if (numPendingCalls < maxPendingCalls) {
deleteData();
}
}, 0);
});
});
}
PS: This is not the exact structure I have but it is very similar to what we have (I have changed the node names and tried to make the example a realistic example)
Whether the deletes can be done more efficiently depends on how you now do them. Since you didn't share the minimal code that reproduces your current behavior it's hard to say how to improve it.
There is no support for a time-to-live property on documents. Typically developers do the clean-up in a administrative program/script that runs periodically. The more frequently you run the cleanup script, the less work it has to do, and thus the faster it will be.
Also see:
Delete firebase data older than 2 hours
How to delete firebase data after "n" days
Firebase actually deletes the data from disk when you tell it to. There is no way through the API to retrieve it, since it is really gone. But if you have a backup from a previous day, the data will of course still be there.
I have been trying for a while for saving time in my database in timeformat like 10:00:00.
What I want is simply pick up time from timepicker and save data(time) in timeformat.
Here is what I have done:
I have used timepicker as it gets data in format 10:52 AM.
I have used accessor methods to save time as follows:
public function setRTimeAttribute($value)
{
$this->attributes['r_time'] = date('h:i:A', strtotime($value));
}
My post controller in case needed as follows:
public function postSessionReservation(Request $request,$profile_slug)
{
$restaurant = Restaurant::where('slug_profile', $profile_slug)->select('name','ar_name','slug_profile','id')->firstOrFail();
$this->validate($request,[
'no_of_seats'=>'required',
'r_date'=>'required',
'r_time'=>'required',
'restaurant_id'=>'required',
]);
$data = [
'no_of_seats'=>$request->no_of_seats,
'r_date'=>$request->r_date,
'r_time'=>$request->r_time,
'restaurant_id'=>$request->restaurant_id
];
$minutes = 1;
Session::put('reservation',json_encode($data),$minutes);
return redirect($restaurant->slug_profile.'/complete-reservation');
}
But it saves the time as 12:00:00 each time.
I do not know how 12:00:00 is generated whenever I save the time. I tried a lot but could not figure out.
Hope you guys will help me solve this.
Thanks in advance.
First remove spaces between time "10 : 52 AM" to "10:52 AM" and then change your line to this:
$this->attributes['r_time'] = date('H:i', strtotime( $value ));
This will surely help, I already tested it.
For example you can remove spaces through:
$a = '10 : 52 PM';
$x = preg_replace('/\s*:\s*/', ':', $a);
date("H:i", strtotime($x));