Laravel 5.4 saving models via async queue - laravel

So I am trying to optimize my site and on every page load and exit I save a metric (time on page, ip address etc) for analytics. However these are decent sized bottlenecks on my server. When viewing the time it takes for things to run my entire function takes ~1-2ms and then saving to the DB takes ~100-200ms. So my goal is to run my function and then dispatch a new job, that will do the actual saving of the metric. This way all of the saving of my models can be offloaded to a queue. Below is a copy of my job
class SaveMetric implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
/**
* Create a new job instance.
*
* #return void
*/
public function __construct()
{
//
}
/**
* Execute the job.
*
* #return void
*/
public function handle(Metrics $metric)
{
//
$metric->save();
}
}
Then in my controller function after I grab all the values I need I run this
dispatch(new SaveMetric($newMetric));
This seems to run but does not seem to do anything. Am I missing something? (Edit) This does ~something~ it just saves a record to the DB with null in all the fields, as if I created a new metric without any values.
Is it required to pass a queue into the job dispatch?
Do I need to run a daemon or something similar to actually process the things in the queue?
I created the job using the artisan make:job command

You're pretty close.
class SaveMetric implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
protected $metric;
/**
* Create a new job instance.
*
* #param Metrics $metric
*/
public function __construct(Metrics $metric)
{
$this->metric = $metric;
}
/**
* Execute the job.
*
* #return void
*/
public function handle()
{
$this->metric->save();
}
}
According to the docs:
In this example, note that we were able to pass an Eloquent model directly into the queued job's constructor. Because of the SerializesModels trait that the job is using, Eloquent models will be gracefully serialized and unserialized when the job is processing. If your queued job accepts an Eloquent model in its constructor, only the identifier for the model will be serialized onto the queue. When the job is actually handled, the queue system will automatically re-retrieve the full model instance from the database.

Related

Why handle method does not fire during job execution?

I have always used events and listeners to add tasks to the queue. Now I'm trying to use Jobs. I do it like this:
my job.
class eventJob implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
public $message;
/**
* Create a new job instance.
*
* #return void
*/
public function __construct($message)
{
$this->message = $message;
}
/**
* Execute the job.
*
* #return void
*/
public function handle()
{
Log::alert($this->message);
}
}
My .env file: QUEUE_CONNECTION=database
In my controller, I dispatch the event like this:
eventJob::dispatch('my message');
A new record appears in the jobs table and to execute it I run php artisan queue:work
The record is removed from the jobs table, but nothing appears in the file logs
I tried in the handle method and the constructor to do throw new \Exception("Error Processing the job", 1); But nothing is written in the filed_jobs table, from which I made the assumption that the handle method and the constructor do not execute.
I also tried running my job like this:
$job = new eventJob('my test message'); dispatch($job);
But it does not change anything
I don't know why but when I changed config/queue.php file from 'default' => env('QUEUE_CONNECTION', 'sync') to 'default' => env('QUEUE_CONNECTION', 'database') everything started working as it should

SQS stops receiving messages when MySQL is locked

So, I have SQS setup and it receives messages from my Laravel 8 application by way of a Job. A request comes in and this request has data that is popped onto the queue. The queue worker then processes this message and writes it to the database. I am adding an index to the table that the data is written to, the alter table command locks the table during this operation. While the table is locked SQS does not receive any new messages despite the request processor working and dispatching jobs without error.
Any ideas why this would prevent the dispatched jobs from being queued?
class ProcessIncomingAISData implements ShouldQueue{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
protected $ais_message;
/**
* Create a new job instance.
*
* #return void
*/
public function __construct(String $ais_message)
{
$this->ais_message = $ais_message;
}
/**
* Execute the job.
*
* #return void
*/
public function handle()
{
try{
$ais = AISData::factory($this->ais_message);
$ais->save();
return true;
}catch(Exception $e){
Log::error($e);
}
}
This job is called by:
ProcessIncomingAISData::dispatch($data)->onQueue('ais-incoming-data-queue');

Notification don't work on queue but work as Direct

i have a issue with notifications on laravel, if i send a notification directly without a queue this work as well
this notifiction needs send a email and save in database
i use this to call notify as exemple
$user = \App\User::find(1);
$candidato = \App\CandidatoVaga::where('id_user','=','1')->first();
$user->notify(new \App\Notifications\ConviteVagaCandidato($candidato));
And this is \App\Notifications\ConviteVagaCandidato
<?php
namespace App\Notifications;
use Illuminate\Bus\Queueable;
use Illuminate\Notifications\Notification;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Notifications\Messages\MailMessage;
class ConviteVagaCandidato extends Notification implements ShouldQueue
{
use Queueable;
protected $CandidatoVaga;
/**
* Create a new notification instance.
*
* #return void
*/
public function __construct(\App\CandidatoVaga $CandidatoVaga)
{
$this->CandidatoVaga = $CandidatoVaga;
}
/**
* Get the notification's delivery channels.
*
* #param mixed $notifiable
* #return array
*/
public function via($notifiable)
{
return ['database','mail'];
}
/**
* Get the mail representation of the notification.
*
* #param mixed $notifiable
* #return \Illuminate\Notifications\Messages\MailMessage
*/
public function toMail($notifiable)
{
return (new MailMessage)
->greeting('Olá, '.$this->CandidatoVaga->user->DadosPessoais->nome)
->subject('Convite')
->markdown('email.convite_selecao');
}
/**
* Get the array representation of the notification.
*
* #param mixed $notifiable
* #return array
*/
public function toArray($notifiable)
{
return [
'id_vaga' => $this->CandidatoVaga->id_vaga,
'id_user' => $this->CandidatoVaga->id_user,
'mensagem' => 'Você foi pré selecionado para a vaga',
'tipo' => 'Nova Vaga',
];
}
}
This return a sql error SQLSTATE[42601]: Syntax error: 7 ERRO
but without implements ShouldQueue works as well
One of the differences between the sync queue driver and a real queue driver is how the queued job handles stored Models.
Since the sync queue driver is processing the jobs directly in the same process, there is no extra work being done. If you build your Notification with a Model, it uses that exact model instance.
However, when using a real queue, Laravel has to serialize the data stored on the notification for the queue worker to handle it. Because models cannot be serialized easily, what it actually does is just store the model key on the notification, and then when the queue worker processes that job, it re-retrieves that model from the database using the stored key.
Based on the query you mentioned in the comments, it looks to me like your \App\CandidatoVaga model does not have a primary key ($primaryKey is empty). Because of this, there is no primary key field to query ("candidatos_vaga".""), and there is no primary key value stored (is null).
I see you've already come up with a solution for yourself. If, however, you still wanted to attempt to just use the model, you can try this:
Override the getQueueableId() method on your model. By default, this returns the primary key field. But, since you don't have one defined, you would need to override this method to provide some unique data that can be used to find your record again.
Override the newQueryForRestoration() method on your model. By default, this builds a query using the primary key field. But, since you don't have one defined, you would need to override this method to generate a query using the data generated by the getQueueableId() method.
NB: this is untested. I have never done this; this is just what I see looking through the source code.
I was able to solve it in a palliative way
public $id_vaga;
public $id_user;
public $nome;
public $titulo_vaga;
/**
* Create a new notification instance.
*
* #return void
*/
public function __construct($CandidatoVaga)
{
$this->id_vaga = $CandidatoVaga->id_vaga;
$this->id_user = $CandidatoVaga->id_user;
$this->nome = $CandidatoVaga->user->DadosPessoais->nome;
$this->titulo_vaga = $CandidatoVaga->vaga->titulo_vaga;
}

Understand Laravel 5.3 Events

I create event using command php artisan event:make EventTest and that generates this class:
class EventTest
{
use InteractsWithSockets, SerializesModels;
/**
* Create a new event instance.
*
* #return void
*/
public function __construct()
{
//
}
/**
* Get the channels the event should broadcast on.
*
* #return Channel|array
*/
public function broadcastOn()
{
return new PrivateChannel('channel-name');
}
}
Official documentation for Laravel 5.3 Events however looks different https://laravel.com/docs/5.3/events#defining-events, event that it uses as example looks like this:
class OrderShipped extends Event
{
use SerializesModels;
public $order;
/**
* Create a new event instance.
*
* #param Order $order
* #return void
*/
public function __construct(Order $order)
{
$this->order = $order;
}
}
As you can see it extends Event, my Event created by artisan does not extend Event, also my class uses InteractsWithSockets while class on laravel documentation does not. Why this difference, what does it mean? Is Laravel's 5.3 documentation already out of date and not updated to reflect latest changes?
It appears you're correct. It looks like this particular part of the documentation was not updated with the 5.3 release.
To answer your question more completely, Laravel 5.3 no longer uses the abstract Event class you were seeing referenced in the 5.2 documentation.
The abstraction itself was unnecessary, and so it was removed.
As for the InteractsWithSockets trait you're seeing, that was added for a more seamless integration with Websockets. If your application isn't using Websockets (see: Socket.io), then you don't have to leverage these traits and the use statement may be removed safely.
Anyone may edit the documentation, so you may want to consider putting in a pull request with the docs here.

Laravel 5 - Queued Commands throwing spl_autoload_call() error

UPDATE - This has been narrowed down to beanstalkd, sync works
I am receiving the following error when attempting to run queued commands in my production environment:
exception 'ErrorException' with message 'unserialize(): Function spl_autoload_call() hasn't defined the class it was called for'
in /home/forge/default/vendor/laravel/framework/src/Illuminate/Queue/CallQueuedHandler.php:74
I have tried both the beanstalkd and database drivers, no change. For simplicity, I am using the following command:
<?php namespace App\Commands;
use App\Commands\Command;
use App\User;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Bus\SelfHandling;
use Illuminate\Contracts\Queue\ShouldBeQueued;
class TestQueueCommand extends Command implements SelfHandling, ShouldBeQueued {
use InteractsWithQueue, SerializesModels;
/**
* #var User
*/
private $user;
/**
* Create a new command instance.
*
* #param User $user
*/
public function __construct(User $user)
{
//
$this->user = $user;
}
/**
* Execute the command.
*
* #return void
*/
public function handle()
{
\Log::info("You gave me " . $this->user->fullName());
}
}
Dispatch code:
get('queue-test', function()
{
Bus::dispatch(new TestQueueCommand(User::first()));
});
This works in my Homestead environment, fails in production (Digital Ocean, Forge). I have several beanstalkd workers and I have tried restarting them. I have also run php artisan queue:flush.
Here is the code where the error is occurring (from source):
/**
* Handle the queued job.
*
* #param \Illuminate\Contracts\Queue\Job $job
* #param array $data
* #return void
*/
public function call(Job $job, array $data)
{
$command = $this->setJobInstanceIfNecessary(
$job, unserialize($data['command'])
);
$this->dispatcher->dispatchNow($command, function($handler) use ($job)
{
$this->setJobInstanceIfNecessary($job, $handler);
});
if ( ! $job->isDeletedOrReleased())
{
$job->delete();
}
}
In the past, I also ran into a similar issue while unserializing. The problem was the default Beanstalk job size (65,535 bytes), which might not be big enough if the class being serialized contains lots of properties that need to be kept (increasing the size of the serialized string and using more than 65K for storage).
In order to solve this, try setting the size to 131,072 or even 262,144 bytes using the -z option, on the configuration file (/etc/default/beanstalkd):
BEANSTALKD_EXTRA="-z 262144"
After that, you should restart the service.
Also note that the configuration file path might be other, depending on the distribution you're using.
And since you're using Digital Ocean, you might find their documentation useful.

Resources