Inconsistent behavior on Laravel Queue Job - laravel

I have a job that looks like this:
<?php
namespace App\Jobs;
use App\User;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use Illuminate\Support\Facades\Redis;
class FakeJob implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
/** #var User */
private $user;
public $tries = 30;
/**
* Create a new job instance.
*
* #param User $user
*/
public function __construct(User $user)
{
$this->user = $user;
}
/**
* Execute the job.
*
* #return void
*/
public function handle()
{
try {
Redis::throttle('key')->allow(1)->every(60)->then(function () {
// Job logic...
throw new \Exception('fake exception');
$this->logInfo($this->user->name);
});
} catch(\Illuminate\Contracts\Redis\LimiterTimeoutException $e) {
$this->logInfo('inside LimiterTimeout catch, ' . $e->getMessage());
$this->release(1);
}
}
public function failed(\Exception $exception) {
// Send user notification of failure, etc...
$this->logInfo('inside failure, ' . $exception->getMessage());
}
public function logInfo($message) {
$path = storage_path('logs/FakeJob.log');
$logText = time() . ' ' . $message;
file_put_contents($path, $logText.PHP_EOL , FILE_APPEND | LOCK_EX);
}
}
Now you'll notice that I have a Redis throttler in the handle function, and that I immediately throw an error inside the callback. This is what I'm testing, and that error is what's going wrong.
You see, I'm trying to distinguish between LimiterTimeoutExceptions and exceptions thrown inside the Throttler callback. When I run the job through the sync connection, everything works as expected: FakeJob::dispatch($user)->onConnection('sync'); Errors that happen because I made a request within 60 seconds of a previous request go into the LimiterTimeoutException catch block, and errors that happen inside the Throttle callback go to the failed function.
But when I schedule the job through my default scheduler, which is Database, it seems as though EVERY error is going through the LimiterTimeoutException catch block, until I hit the retry limit - that error goes into the failed function, but not the 'fake exception' error.
I feel very confused about this.
Of note: even though my 'fake exception' ends up being caught by the LimiterTimeoutException catch block, the $e->getMessage() function does NOT return fake exception there for some reason. But I'm 100% certain that it is that error that is causing it to go there, because it's definitely not due to the throttler in those tests.

OK so actually what's happening is that it won't end up in the failed() function until it hits the retry limit, no matter what, regardless of any exception thrown. If I want to deal with LimiterTimeoutException errors in a different way from every other error, I'll have to wrap my Redis::throttle call in a try-catch and just check the type of error in the catch.

Related

Laravel mailable, data not being passed to Blade when implements ShouldQueue

Im trying to send an email, with ShouldQueue, and passing data into the email blade view, but it fails if i implement ShouldQueue.
See below for my code.
The code as it is below, does execute the mail and places it in jobs queue (database driver). It works and i can see the job in jobs table.
Then i run php artisan queue:work to start worker on my computer.
When worker tries to execute job, it fails. It is now removed from jobs table and new entry to failed_jobs table. The email is not sent.
In failed_jobs table, exception column, i get this error (stacktrace removed)
Next Facade\Ignition\Exceptions\ViewException: Undefined variable $data (View: /Users/xxx/xxx/xxx/resources/views/mails/ContactForm.blade.php) in /Users/xxx/xxx/xxx/resources/views/mails/ContactForm.blade.php:2
IF I remove "implements ShouldQueue" from the ContactFormMail class, everything works and email is being sent, but then, the email is not being put in the queue.
I have tried passing data both ways described in Laravel docs and i have also tried restarting worker.
Any ideas where i go wrong?
If it was not clear, I want to pass data to ContactForm.blade.php and also place email in queue.
Here is my code
Controller:
public function submit()
{
$this->validate();
$data = [
'name' => $this->name,
'email' => $this->email,
'message' => $this->message,
];
try {
Mail::to('xxx#xxx.xx')->send(new ContactFormMail($data) );
$this->name = "";
$this->email = "";
$this->message = "";
$this->status = "Message sent!";
} catch (Exception $e) {
$this->status = "Something went wrong!";
}
}
ContactFormMail class:
<?php
namespace App\Mail;
use Illuminate\Bus\Queueable;
use Illuminate\Mail\Mailable;
use Illuminate\Queue\SerializesModels;
use Illuminate\Contracts\Queue\ShouldQueue;
class ContactFormMail extends Mailable implements ShouldQueue
{
use Queueable, SerializesModels;
public $data;
/**
* Create a new message instance.
*
* #return void
*/
public function __construct($data)
{
$this->data = $data;
}
/**
* Build the message.
*
* #return $this
*/
public function build()
{
return $this->view('mails.ContactForm');
}
}
ContactForm blade file:
<div>
{{$data['name']}}<br>
{{$data['email']}}<br>
{{$data['message']}}<br>
</div>
Please try restarting the queue worker. Probably something got stuck. After restarting the queue it "should" work.
php artisan queue:restart

Pusher filling up with Laravel Broadcasts

Recently been observing this in our Laravel project that Pusher is getting clogged up with messages upto the limit. This is abnormal for usage of peak concurrent - 16 and 4 million messages a day.
Going through the Pusher dashboard, below is the extent of the damage being done.
There is a specific pattern of usage. When I turn off the supervisord worker, the graph stops and when I start the worker again, it comes back up in a predictable pattern shown in the graph below.
Below is a sample message I got when running redis-cli monitor.
As you can see above, the index [attempts] is 69507. Does that mean that this event has been broadasted 69507 times ? Why would an event be broadcasted so many times ? When does an event stop being broadcasted ? Am I doing something wrong ?
Here is the code for the AgendaParticipantUpdated event. Would be great if there is something fundamentally wrong in the way the event has been implemented.
<?php
namespace App\Events;
use App\AgendaParticipant;
use App\Agenda;
use Illuminate\Queue\SerializesModels;
use Illuminate\Broadcasting\PrivateChannel;
use Illuminate\Foundation\Events\Dispatchable;
use Illuminate\Broadcasting\InteractsWithSockets;
use Illuminate\Contracts\Broadcasting\ShouldBroadcast;
class AgendaParticipantUpdated implements ShouldBroadcast
{
use Dispatchable, InteractsWithSockets, SerializesModels;
/** #var agendaParticipant */
public $agendaParticipant;
/**
* Create a new event instance.
*
* #param AgendaParticipant $agendaParticipant
*/
public function __construct(AgendaParticipant $agendaParticipant)
{
$this->agendaParticipant = $agendaParticipant;
}
/**
* Get the channels the event should broadcast on.
*
* #return \Illuminate\Broadcasting\Channel|array
*/
public function broadcastOn()
{
return new PrivateChannel("meetings.{$this->agendaParticipant->agenda->meeting->id}");
}
/**
* Get the data to broadcast.
*
* #return array
*/
public function broadcastWith()
{
$agenda = Agenda::where('id', $this->agendaParticipant->agenda->id)->first();
$agenda->load(['participants' => function ($query) {
$query->orderBy('order')->orderBy('id');
}, 'participants.member.user']);
return [
'agendaParticipant' => $this->agendaParticipant,
'agenda' => $agenda,
];
}
}
There are lots of other broadcasts too like this one :
As you see above, this broadcast has attempt count of 873245.
Below is the code for the DiscussionCreated event :
<?php
namespace App\Events;
use App\Discussion;
use App\Member;
use Illuminate\Broadcasting\Channel;
use Illuminate\Contracts\Broadcasting\ShouldBroadcastNow;
use Illuminate\Http\Response;
use Illuminate\Queue\SerializesModels;
use Illuminate\Broadcasting\PrivateChannel;
use Illuminate\Foundation\Events\Dispatchable;
use Illuminate\Broadcasting\InteractsWithSockets;
use Illuminate\Contracts\Broadcasting\ShouldBroadcast;
class DiscussionCreated implements ShouldBroadcast
{
use Dispatchable, InteractsWithSockets, SerializesModels;
/**
* #var Discussion
*/
public $discussion;
/**
* Create a new event instance.
*
* #param Discussion $discussion
*/
public function __construct(Discussion $discussion)
{
$this->discussion = $discussion;
}
public function broadcastOn()
{
$author = Member::where('id',$this->discussion->author_id)->firstOrFail();
return new PrivateChannel("group.{$author->group_id}");
}
public function broadcastWith()
{
if(\Auth::check())
{
$group = $this->discussion->author->group;
$group->load(['meetings', 'facilitator']);
$facilitatorId = optional($group->facilitator)->id;
$members = $group->members()->with('user')->get();
$discussion = $this->discussion;
$member = auth()->user()->activeGroupMember;
$list_read_discussions=\DB::table('members')
->where('member_id', '=', $member->id)
->join('acknowledgements', 'members.id', '=', 'acknowledgements.member_id')
->join('discussions', 'discussions.id', '=', 'acknowledgements.discussion_id')
->where('discussions.target_type','=','App\Group')
->whereNotExists( function($q) {
$q->select(\DB::raw(1))
->from('replies')
->where('acknowledgements.discussion_id','=','replies.discussion_id')
->whereColumn('replies.updated_at', '>','acknowledgements.updated_at');
})
->distinct('acknowledgements.discussion_id')
->pluck('acknowledgements.discussion_id');
$group_discussions_count= $discussion->count();
$group_read_discussions=$list_read_discussions;
$unreadGroupCount=max(0,$group_discussions_count - $group_read_discussions->count());
$authedMemberId = auth()->id();
//private menu section
$authedMember = auth()->user()->activeGroupMember;
foreach($members as $target){
$discussions_to_target_count=\App\Discussion::
where('target_type','=','App\Member')
->where('target_id','=',$target->id)
->Where('author_id','=',$authedMember->id)
->count();
$discussions_from_target_count=\App\Discussion::
where('target_type','=','App\Member')
->where('target_id','=',$authedMember->id)
->Where('author_id','=',$target->id)
->count();
$read_discussions_to_target_count=\DB::table('acknowledgements')
->where('acknowledgements.member_id', '=', $authedMember->id)
->join('discussions', 'discussions.id', '=', 'acknowledgements.discussion_id')
->where('discussions.target_type','=','App\Member')
->where('discussions.target_id', '=', $target->id)
->where('discussions.author_id', '=', $authedMember->id)
->whereNotExists( function($q) {
$q->select(\DB::raw(1))
->from('replies')
->where('acknowledgements.discussion_id','=','replies.discussion_id')
->whereColumn('replies.updated_at', '>','acknowledgements.updated_at');
})
->distinct('acknowledgements.discussion_id')
->pluck('acknowledgements.discussion_id')
->count();
$read_discussions_from_target_count=\DB::table('acknowledgements')
->where('acknowledgements.member_id', '=', $authedMember->id)
->join('discussions', 'discussions.id', '=', 'acknowledgements.discussion_id')
->where('discussions.target_type','=','App\Member')
->where('discussions.target_id', '=', $authedMember->id)
->where('discussions.author_id', '=', $target->id)
->whereNotExists( function($q) {
$q->select(\DB::raw(1))
->from('replies')
->where('acknowledgements.discussion_id','=','replies.discussion_id')
->whereColumn('replies.updated_at', '>','acknowledgements.updated_at');
})
->distinct('acknowledgements.discussion_id')
->pluck('acknowledgements.discussion_id')
->count();
$target->unreadPrivateCount=max(0,$discussions_to_target_count
+$discussions_from_target_count
-$read_discussions_from_target_count
-$read_discussions_to_target_count);
}
$this->discussion->replies = [];
$this->discussion->replies_count = 0;
$this->discussion->sort = $this->discussion->created_at->toDateTimeString();
$this->discussion->has_user_subscribed = 1;
return [
'discussion' => $this->discussion ? $this->discussion->toArray() : array(),
];
}
}
}
Would appreciate any help on this since it is interfering with our daily operations.
As you can see above, the index [attempts] is 69507. Does that mean that this event has been broadasted 69507 times?
Yes. It means that your queue worker has processed this job (broadcast) almost 70K times.
Why would an event be broadcasted so many times?
I believe the reason is that your broadcast job throws an Exception (it is failing). Laravel would keep trying to execute the failed jobs until they either: execute successfully without any exceptions or until max attempts/timeout values are reached. From the attached screenshots, I believe that you haven't configured such values. (maxTries is empty in your screenshots)
When does an event stop being broadcasted ? Am I doing something wrong ?
I think you need to do two things:
Define the maximum number of attempts for your queue workers. One of the ways to do so is by specifying tries parameter for your queue workers, i.e. run php artisan queue:work --tries=3(Check Laravel Documentation for more information).
Check your error log files and find the exception being thrown from your broadcast jobs and fix it.
I hope this helps. Please let me know if you have any follow-up questions.
Update: Consider giving Laravel Horizon a try. It would give you a nice dashboard to monitor your jobs (including broadcasts), see the number of failing jobs and the causes (exceptions). It also has built-in notification settings that could help you figure out early when there is a problem.
I don't know if that can be you case but I encountered a similar problem using Laravel 5.6
Whet I found out was that the cron for the queued jobs with
* * * * * cd /path-to-your-project && php artisan schedule:run >> /dev/null 2>&1
I forgot to specify the user processing the cron, so the process was handled by the super user and subsequently locking up files / resources with the wrong permissions
so, I changed the cron command to
* * * * * apache:apache cd /path-to-your-project && php artisan schedule:run >> /dev/null 2>&1
which fixed my problem. Now I have my logs clear with no failed process attempts

Laravel: How can a queued job move itself to failed_jobs on fail?

I have a queueable Job that creates a new user...
<?php
namespace App\Jobs;
...
class CreateNewUser implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
/** #var array */
public $newUserData;
public function __construct($newUserData)
{
$this->newUserData = $newUserData;
}
/**
* Execute the job.
*
* #return void
*/
public function handle()
{
$email = $this->newUserData['email'];
if (User::whereEmail($email)->count()) {
// TODO: Make this job fail immediately
throw new UserWithEmailExistsException('User with email ' . $email . ' already exists');
}
...
}
}
It's a queued job because we batch process CSVs to load in many users at a time, each one requiring an entry in 2 tables plus multiple entries in roles and permissions tables. Too slow to do synchronously.
I have a check at the start of the handle() method to see if a user with the same email address has not already been created (because potentially several jobs could be queued to create users with the same email) and it throws a custom Exception if it does.
If that check fails, I don't ever want the queue worker to attempt this job again because I know it will continue to fail indefinitely, it's a waste of time to re-attempt even once more. How can I manually force this job to fail once and for all and move over to the failed jobs table?
P.S. I have found the SO answer about the fail() helper and the $this->markAsFailed() method but these still do not immediately move the job from jobs to failed_jobs.
I've completely rewritten my answer for you. I was in a rush last night and thought I had a quick solution to your problem, sorry for the misinformation.
You could do something like the code below, this won't put the job into the failed jobs table but it will prevent the re-attempts from happening. I hope that helps.
<?php
namespace App\Jobs;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
class TestJob extends Job implements ShouldQueue
{
use InteractsWithQueue, SerializesModels;
public function __construct()
{
parent::__construct();
}
public function handle()
{
if ($this->attempts() > 0) {
// Will not reattempt this job.
return;
}
throw new \Exception('Job will fail.');
}
}

Listen callback is not working in Pusher API Laravel 5.4

Problem
I can confirm that Pusher API is receiving the message. I saw in Debug console of Pusher website. But listen callback is not working at all.
I am following this tutorial to implement Pusher in Laravel 5.4
Below were the step by step things done.
composer require pusher/pusher-php-server
npm install --save laravel-echo pusher-js
instantiated the Echo instance in your resources/assets/js/bootstrap.js
Initialized the pusher key in env and in bootstrap.js file.
Finally, I wrote below code in blade.
<script>
window.Echo.channel('SendMessageChannel.1')
.listen('App.Events.SendMessageEvent', (e) => {
console.log(e);
});
</script>
Controller Code
broadcast(new SendMessageEvent("Hi"))->toOthers();
Event Code
class SendMessageEvent implements ShouldBroadcast
{
use Dispatchable, InteractsWithSockets, SerializesModels;
public $Message;
public function __construct($message)
{
$this->Message = $message;
}
public function broadcastOn()
{
return new PrivateChannel('SendMessageChannel.2');
}
}
Am I missing anything?
You have to listen to: SendMessageEvent without the namespace.
when you listen to a private channel, you need to to listen to private-SendmessageChannel or you use Echo.private('SendmessageChannel')
Because we fixxed the issue via teamspeak at some parts it's difficult to explain it in this answer in full detail.
One problem was also that the event was fired before the client started to listen to it. The best way is to debug with the pusher console and to fire custom events.
I have been using Laravel 5.4 events quite efficiently since 6 months with a project. I had same trouble when I was doing initial setup. Let me guide you with whatever I have done to get it working.
I can see you have controller to initiate an Event, SendMessageEvent to send message content to pusherjs. You need to check following stuff to get it going.
Check 1:
But you have not mentioned if you have an Eventhandler defined. Event handler works as a bridge between the SendMessageEvent and its actual broadcaster.
So define an Eventhandler create one folders like app / Handlers / Events /. (here Handlers and Events are folders. where Events is inside Handlers)
create one file inside this Events folder with a name e.g.
HandleMyMessage.php
And put this code in it:
<?php
namespace App\Handlers\Events;
use App\Events\SendMessageEvent; // This should be your event class..
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
class HandleMyMessage{
protected $name;
public function __construct() {
//
}
public function handle(Message $event) {
// No need to write anything here
}
}
Check 2:
There should be one provider at app / Providers / EventServiceProvider.php location, and put following code in EventServiceProvider.php
<?php
namespace App\Providers;
use Illuminate\Support\Facades\Event;
use Illuminate\Foundation\Support\Providers\EventServiceProvider as ServiceProvider;
class EventServiceProvider extends ServiceProvider
{
/**
* The event listener mappings for the application.
*
* #var array
*/
protected $listen = [
'App\Events\SendMessageEvent' => [
'App\Handlers\Events\EventServiceProvider',
],
];
/**
* Register any events for your application.
*
* #return void
*/
public function boot()
{
parent::boot();
}
}
Check 3:
And you should change syntax of sending event message in your controller like this:
$broadcast_data['first_name'] = 'John';
$broadcast_data['last_name'] = 'Doe';
$return_socket['data'] = $broadcast_data;
Event::fire(new Message($return_socket)); // Sending message to pusherjs via Laravel Event.
More on this, you also need Redis installed on your system and run it.
Hope this helps. If you need more information for any of above, just comment. I am posting a reference link for the same.
Happy coding!!! :-)
I got it working. Below is the correct code. I hope this will be useful to others for sending real time messaging.
Js Work
<script>
window.Echo.channel('private-SendMessageChannel.{!! \Auth::user()->UserID !!}')
.listen('SendMessageEvent', (e) => {
console.log(e);
});
</script>
Controller Code
broadcast(new SendMessageEvent("Hi", 1))->toOthers();
//Here 1 is recepient ID
Event Code
class SendMessageEvent implements ShouldBroadcast
{
use Dispatchable, InteractsWithSockets, SerializesModels;
public $Message;
private $UserID;
public function __construct($message, $RecepientID)
{
$this->Message = $message;
$this->UserID = $RecepientID;
}
public function broadcastOn()
{
return new PrivateChannel('SendMessageChannel.' . $UserID);
}
}
In my case, I had used:
public function broadcastAs()
{
return 'message';
}
If you've set the broadcast name, you must use that name in the listener differently.
For those who have done the same mistake as I did:
If you customize the broadcast name using the broadcastAs method, you should make sure to register your listener with a leading . character. This will instruct Echo to not prepend the application's namespace to the event:
https://laravel.com/docs/9.x/broadcasting#broadcast-name

Laravel mail Invalid view

I have a command in my code that is run daily by cron that sends emails to all new users. It used to work ok, but after I have swithched the queue driver to SQS and upgraded Laravel 5.2 to 5.3 it started throwing an error.
InvalidArgumentExceptionvendor/laravel/framework/src/Illuminate/Mail/Mailer.php:379
Invalid view.
I don't know what might cause the error, because I have not deleted the view. Also when I run the command manually it does not throw any errors.
Here is the command code:
public function handle()
{
$subscriptions = Purchase::where('created_at', '>=', Carbon::now()->subDay())
->groupBy('user_id')
->get();
$bar = $this->output->createProgressBar(count($subscriptions));
foreach ($subscriptions as $subscription) {
$user = $subscription->user;
// if ($user->is_active_customer) {
Mail::to($user)->bcc(env('BCC_RECEIPTS_EMAIL'))->send(new NeedHelp());
// }
$bar->advance();
}
$bar->finish();
$this->info("\nSuccess! " . number_format(count($subscriptions)) . ' emails were sent.');
}
Here is the NeedHelp class code (I have changed the email and sender name for this thread):
<?php
namespace App\Mail;
use App\User;
use Illuminate\Bus\Queueable;
use Illuminate\Mail\Mailable;
use Illuminate\Queue\SerializesModels;
use Illuminate\Contracts\Queue\ShouldQueue;
class NeedHelp extends Mailable
{
use Queueable, SerializesModels;
/**
* Create a new message instance.
*
*/
public function __construct(){
}
/**
* Build the message.
*
* #return $this
*/
public function build()
{
return $this->subject('Need help?')
->from('default#mail.com', 'Sender')
->view('emails.need-help');
}
}
I have found the error. The reason was that I have accidentally connected two applications to the same queue, which caused them to process jobs and emails of each other which resulted in this error.

Resources