Laravel Queues - why is Queue job retrying every 3 seconds, if I specify timeout=5 and expire=>7? - laravel-5

I am trying to set up my queue worker in Laravel 5.3 and am trying to ensure that I fully understand nuances of the interrelated parameters expire, --tries, --timeout, because the results I am seeing suggest I am not understanding something.
Note: In the Laravel 5.3 documentation (https://laravel.com/docs/5.3/queues#introduction), it refers to retry_after as being a parameter for the database queue connection type, but in my installation of Laravel 5.3, that parameter seems to be expire.
In any case, I have a job, let's call it SendReminder. I purposefully added a line of code that causes an error, so I can test the functioning of the queue tries/retries and eventual labeling of the job as failed in the failed_jobs table.
Here is my database queue config in queue.php:
'database' => [
'driver' => 'database',
'table' => 'jobs',
'queue' => 'default',
'retry_after' => 7,
'expire' => 7,
],
For testing, I want to let it try 3 times before being sent to failed_jobs table.
I expect the job to take not more than 5 seconds, and so setting --timeout=5, and expire to 7 (or retry_after to 7), would seem to be in line with the Queue documentation, which says "The --timeout value should always be at least several seconds shorter than your retry_after configuration value. This will ensure that a worker processing a given job is always killed before the job is retried."
Based on above, I expect the job to be considered failed after 21 seconds.
After running php artisan queue:work --daemon --queue=email,default --timeout=5 --tries=3 and then dispatching the job via php artisan schedule:run >> /dev/null 2>&1
I count with my stopwatch 9 seconds until the queue worker displays
[2021-09-02 14:22:14] Failed: App\Jobs\PrepareSendReminderJob
and in my Laravel log I see
[2021-09-02 14:22:08] local.ERROR: ErrorException:
[2021-09-02 14:22:11] local.ERROR: ErrorException:
[2021-09-02 14:22:14] local.ERROR: ErrorException:
This doesn't seem to make sense. I would expect, based on my expire being set to 7, that the failure message would only come 21 seconds after the job is dispatched and immediately picked up by the queue. And I get same result every time - 3 seconds between each error message in the log.
Can someone explain why I am seeing this result?
Thanks!
-- Edit:
And here is the job class:
<?php
namespace App\Jobs;
use App\Jobs\Job;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
use Log;
class SendReminderJob extends Job implements ShouldQueue
{
use InteractsWithQueue, SerializesModels;
/**
* Create a new job instance.
*
* #return void
*/
//public function __construct(\App\Services\SearchService $searchService)
public function __construct()
{
$this->queue = 'email';
}
/**
* Execute the job.
*
* #return void
*/
public function handle()
{
// nothing here for now, except for this line to cause an error on purpose
Log::infoo('text');
}
}

Related

Notification Testing no record created during tests in notifications table

I am trying to test if notification is created in the database, in my local, this works as intended however the test environment has some strange behaviour.
$user->notify(new ExampleNotification());
$this->assertDatabaseCount('notifications', 1);
//returns green with QUEUE_CONNECTION=sync but red with QUEUE_CONNECTION=database
...
class ExampleNotification extends Notification implements ShouldBroadcast, ShouldQueue
{
use Queueable;
public function viaQueues(): array
{
return [
'database' => 'notifications'
];
}
/**
* Get the notification's delivery channels.
*
* #return array
*/
public function via(User $notifiable): array
{
return ['database']
}
ps: I do not use Notification::fake() anywhere in my tests
Does anybody know why the different queue connections would behave differently?
Any queue driver other than the sync one works asynchronously i.e. when you send a notification job it will be queued and then the queue worker will remove it from the queue and then send the actual notification.
This means if you send a notification during a test, even if the queue worker is running the notification will most likely not be sent before the test is done.
To avoid this you should either use the sync queue or use Notification::fake() and just check if a notification would have been sent.

Integrating stripe webhook in laravel with redis queue driver

I've been implemented stripe webhook in laravel 7 using library https://github.com/spatie/laravel-stripe-webhooks
The goal is to subscribe my users to plan created in stripe and generate an invoice on charge succeeded webhook request. Now to achieve this I created a cron script that dispatches the job to subscribe users. Also, set up a webhook endpoint in stripe. All setup is done and configuration in environment variables. It works perfectly when set the queue connection to "sync". However, when setting the queue connection to Redis it doesn't work.
Here's the code inside my config/stripe-webhooks.php
<?php
return [
/*
* Stripe will sign each webhook using a secret. You can find the used secret at the
* webhook configuration settings: https://dashboard.stripe.com/account/webhooks.
*/
'signing_secret' => env('STRIPE_WEBHOOK_SECRET'),
/*
* You can define the job that should be run when a certain webhook hits your application
* here. The key is the name of the Stripe event type with the `.` replaced by a `_`.
*
* You can find a list of Stripe webhook types here:
* https://stripe.com/docs/api#event_types.
*/
'jobs' => [
'charge_succeeded' => \App\Jobs\StripeWebhooks\ChargeSucceededJob::class,
// 'source_chargeable' => \App\Jobs\StripeWebhooks\HandleChargeableSource::class,
// 'charge_failed' => \App\Jobs\StripeWebhooks\HandleFailedCharge::class,
],
/*
* The classname of the model to be used. The class should equal or extend
* Spatie\StripeWebhooks\ProcessStripeWebhookJob.
*/
'model' => \Spatie\StripeWebhooks\ProcessStripeWebhookJob::class,
];
And inside the command that dispatches SubscribeCustomerJob:
$subscribed = 0;
$users = User::role('admin')->where(function ($q) {
$q->where('stripe_id', '!=', null)->where('verified_employees', '>=', 5);
})->get();
foreach ($users as $user) {
if ( $user->subscribed('default') && now()->format('Y-m-d') >= $user->trial_ends_at->format('Y-m-d')) {
SubscribeCustomerJob::dispatch($user)->onQueue('api');
$subscribed++;
}
}
Handling webhook requests using jobs
<?php
namespace App\Jobs\StripeWebhooks;
use Illuminate\Bus\Queueable;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
use Spatie\WebhookClient\Models\WebhookCall;
class HandleChargeableSource implements ShouldQueue
{
use InteractsWithQueue, Queueable, SerializesModels;
/** #var \Spatie\WebhookClient\Models\WebhookCall */
public $webhookCall;
public function __construct(WebhookCall $webhookCall)
{
$this->webhookCall = $webhookCall;
}
public function handle()
{
// At this point, I store data to Payments table,
// generate invoice and send email notification to subscribed user.
}
}
The output inside the logs of job:
[2021-04-14 07:53:46][Nr84GbvR3kxqnBGRrrWHtkTj34XRYsGv] Processing: App\Jobs\SubscribeCustomerJob
[2021-04-14 07:53:53][Nr84GbvR3kxqnBGRrrWHtkTj34XRYsGv] Processed: App\Jobs\SubscribeCustomerJob
Inside table in webhook_calls:
webhook_calls table click to see it
All thigs works well with queue connection set to sync. However, the problem is when I set queue connection to "redis".
I know webhook call works because there's data inside webhook_calls table but I guess fails to reach the job.
Any idea with stripe webhook and laravel using redis as queue driver, please add your comment below and thanks in advance.
I've fixed this issue it's my bad that I accidentally put an invalid queue name in my queue worker setup in supervisor.
[program:queue-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /var/www/html/artisan queue:work --queue=api,notification
autostart=true
autorestart=true
user=root
numprocs=8
redirect_stderr=true
stdout_logfile=/var/www/html/worker.log

Laravel 5.5 Queue Job doesn't respect timeout

I have a job in Laravel 5.5 that doesn't respect the value in the variable public $timeout set on the job as described in the laravel documentation.
If I set the value of $timeout to, for example, 120 seconds I would expect the job to be terminated after it has ran for 120 seconds. I'm using RabbitMQ on a Heroku dyno.
Codebase:
Laravel 5.5
Extension:
RabbitMQ Queue driver for Laravel
Platform:
Heroku dyno (Amazon).
Example code:
class ExampleJob implements ShouldQueue
{
use InteractsWithQueue, Queueable, SerializesModels;
public $timeout = 120;
public function handle()
{
DB::statement($this->buildCallStatement());
}
}
Example Procfile:
worker: php artisan queue:listen rabbitmq --queue=high,medium,low --tries=1 --memory=512 --timeout=0
On your example command you are setting a timeout of 0 which is likely overriding your member variable. Also, setting a timeout of 0 can be dangerous and may cause an infinite loop.

How to run only one job at time in same laravel queue?

I have saas service which working with API. It has limits so I need that one user account doing only one request at the same time.
For this I queuing with OnQueue($user->name);
then in handle() doing job...
I need only one job can be run in users queue. At the same time may be run only diffent queues 1 job per 1 queue.
Im using redis connection.
This my job class:
public function __construct(Accounts $acc)
{
$this->acc = $acc;
$this->ownjob = $acc->prepareJobQueue();
}
public function handle()
{
$acc = $this->acc;
$job = $this->ownjob;
$api = new Api([
'login' => $acc->login,
'password' => $acc->password,
]);
if ($api->checkLogin()) {
info("{$acc->login} OK Authorized");
foreach ($job['queue'] as $term) {
switch($term['type']) {
case 'hashtag':
info("{$acc->login} Queuing: type - {$term['type']}, value - {$term['value']}");
$hashtag = Hashtags::where('cha_name',$term['value'])->first();
$answer = $api->getUsersByHashtag($hashtag,50);
break;
case 'concurency':
info("{$acc->login} Queuing: type - {$term['type']}, value - {$term['value']}");
$soc_user = Users::where('soc_unique_id',$term['value'])->first();
$answer = $api->getUserFollowers($soc_user);
break;
default:
break;
}
}
} else {
info("{$acc->login} NOT Authorized - STOP JOB");
}
}
This is how I dispatching job:
$accounts = Accounts::select(['id', 'login', 'hashtag_filter', 'concurency_filter'])->whereNotNull('hashtag_filter')->get();
foreach ($accounts as $acc) {
doFollowing::dispatch($acc)->onQueue($acc->login);
}
Use mxl/laravel-queue-rate-limit Composer package.
It enables you to rate limit Laravel jobs on specific queue without using A third party driver such as Redis.
Install it with:
$ composer require mxl/laravel-queue-rate-limit:^1.0
This package is compatible with Laravel 5.5+ and uses [auto-discovery][1] feature to add MichaelLedin\LaravelQueueRateLimit\QueueServiceProvider::class to providers.
Add rate limit (x number of jobs per y seconds) settings to config/queue.php:
'rateLimit' => [
'mail' => [
'allows' => 1, // number of jobs
'every' => 5 // time interval in seconds
]
]
These settings allow to run 1 job every 5 seconds on mail queue.
Make sure that default queue driver (default property in config/queue.php) is set to any value except sync.
Run queue worker with --queue mail option:
$ php artisan queue:work --queue mail
You can run worker on multiple queues, but only queues referenced in rateLimit setting will be rate limited:
$ php artisan queue:work --queue mail,default
Jobs on default queue will be executed without rate limiting.
Queue some jobs to test rate limiting:
SomeJob::dispatch()->onQueue('mail');
SomeJob::dispatch()->onQueue('mail');
SomeJob::dispatch()->onQueue('mail');
SomeJob::dispatch();
You could use Laravel's builtin rate limiting for this (note that it does require Redis).
Inside your job:
Redis::funnel('process-name')->limit(1)->then(function () {
// Your job logic here
});
Note that if you don't provide a second callback to then, it will throw an exception if it cannot obtain the lock (which will cause your job to fail)
If you're using Laravel 6 or higher you could also opt to do this in a job middleware, rather than in the job itself (handy if you have multiple jobs that share the same lock)
More info on Laravel's rate limiting: https://laravel.com/docs/5.8/queues#rate-limiting
You could limit numprocs per queue in your Supervisor or Horizon setup.
If you only spawn one queue worker per user I believe you will get your desired behaviour.

How to set dynamic SMTP data in Laravel 5.4 for queued emails?

In my application each user can use his own SMTP server. Therefor the config must be provided. I'm using Laravel Notifications to send the emails. If I'm using no queue (that means sync), there is no problem.
I made a CustomNotifiable Trait:
config([
'mail.host' => $setting->smtp_host,
'mail.port' => $setting->smtp_port,
'mail.username' => $setting->smtp_username,
'mail.password' => $setting->smtp_password,
'mail.encryption' => $setting->smtp_encryption,
'mail.from.address' => $setting->smtp_from_address,
'mail.from.name' => $setting->smtp_from_name,
]);
(new \Illuminate\Mail\MailServiceProvider(app()))->register();
After that, I restore the original config:
config([
'mail' => $originalMailConfig
]);
(new \Illuminate\Mail\MailServiceProvider(app()))->register();
No problem until now.
But if it's queued, just the first config after starting the queue worker will be taken for all further emails, even if any other SMTP config is provided. The default config from config/mail.php will be overridden. But this only works the first time.
I've made in the AppServiceProvider::boot method (the SMTP config is stored at the notification):
Queue::before(function (JobProcessing $event) {
// Handle queued notifications before they get executed
if (isset($event->job->payload()['data']['command']))
{
$payload = $event->job->payload();
$command = unserialize($payload['data']['command']);
// setting dynamic SMTP data if required
if (isset($command->notification->setting))
{
config([
'mail.host' => $command->notification->setting->smtp_host,
'mail.port' => $command->notification->setting->smtp_port,
'mail.username' => $command->notification->setting->smtp_username,
'mail.password' => $command->notification->setting->smtp_password,
'mail.encryption' => $command->notification->setting->smtp_encryption,
'mail.from.address' => $command->notification->setting->smtp_from_address,
'mail.from.name' => $command->notification->setting->smtp_from_name,
]);
(new \Illuminate\Mail\MailServiceProvider(app()))->register();
}
}
});
Of course, the original config get restored:
Queue::after(function (JobProcessed $event) use ($originalMailConfig) {
$payload = $event->job->payload();
$command = unserialize($payload['data']['command']);
// restore global mail settings
if (isset($command->notification->setting))
{
config([
'mail' => $originalMailConfig
]);
(new \Illuminate\Mail\MailServiceProvider(app()))->register();
}
});
It seems, as the Swift Mailer has a cache or something like that. I registered a new MailServiceProvider, which should simply replace the old one. So if I set the config with the new SMTP data, the new registered provider should take them. Logging the config shows even in the TransportManager, that the correct SMTP data were set, right before sending the mail, but the mail was sent with the first set config.
I found this thread and tried the linked solution, but with the same result: How to set dynamic SMTP details laravel
So I need a way to override the Services / ServiceProvider / SMTP config. Even if the Supervisor restarts the queue, there is a chance that multiple emails with different configs should be send at the same time.
In Laravel 5.4+, as I see that the Mailer Class is a singleton that hold a MailTransport Class, which is responsible for the config of SMTP mail and is a singleton,too; I just have to override the config using the following approach:
First, I setup a trait so I can just turn this feature on some Mails:
trait MailSenderChangeable
{
/**
* #param array $settings
*/
public function changeMailSender($settings)
{
$mailTransport = app()->make('mailer')->getSwiftMailer()->getTransport();
if ($mailTransport instanceof \Swift_SmtpTransport) {
/** #var \Swift_SmtpTransport $mailTransport */
$mailTransport->setUsername($settings['email']);
$mailTransport->setPassword($settings['password']);
}
}
}
Then, in the build() method of your mail class, you can utilize the above trait and call:
$this->changeMailSender([
'email'=>$this->company->email,
'password'=>$this->company->email_password,
]);
Boom, let the Laravel do the rest.
After a lot of researching I stumbled upon the different queue commands. I tried queue:listen (which is not described in the Laravel 5.4 docs) instead of queue:work and the problems are gone.
Of course, this doesn't really explain the described behavior, but fortunately it doesn't matter, because I can live with this solution/workaround.
Another strange behavior is, that from time to time the queue worker throws an exception because the database was locked. No idea, when or why this happened.
This post explained a little bit, why things can happen: What is the difference between queue:work --daemon and queue:listen
In a nutshell, queue:listen solved my problem and another very strange db lock problem as well.

Resources