So, I have SQS setup and it receives messages from my Laravel 8 application by way of a Job. A request comes in and this request has data that is popped onto the queue. The queue worker then processes this message and writes it to the database. I am adding an index to the table that the data is written to, the alter table command locks the table during this operation. While the table is locked SQS does not receive any new messages despite the request processor working and dispatching jobs without error.
Any ideas why this would prevent the dispatched jobs from being queued?
class ProcessIncomingAISData implements ShouldQueue{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
protected $ais_message;
/**
* Create a new job instance.
*
* #return void
*/
public function __construct(String $ais_message)
{
$this->ais_message = $ais_message;
}
/**
* Execute the job.
*
* #return void
*/
public function handle()
{
try{
$ais = AISData::factory($this->ais_message);
$ais->save();
return true;
}catch(Exception $e){
Log::error($e);
}
}
This job is called by:
ProcessIncomingAISData::dispatch($data)->onQueue('ais-incoming-data-queue');
Related
I have always used events and listeners to add tasks to the queue. Now I'm trying to use Jobs. I do it like this:
my job.
class eventJob implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
public $message;
/**
* Create a new job instance.
*
* #return void
*/
public function __construct($message)
{
$this->message = $message;
}
/**
* Execute the job.
*
* #return void
*/
public function handle()
{
Log::alert($this->message);
}
}
My .env file: QUEUE_CONNECTION=database
In my controller, I dispatch the event like this:
eventJob::dispatch('my message');
A new record appears in the jobs table and to execute it I run php artisan queue:work
The record is removed from the jobs table, but nothing appears in the file logs
I tried in the handle method and the constructor to do throw new \Exception("Error Processing the job", 1); But nothing is written in the filed_jobs table, from which I made the assumption that the handle method and the constructor do not execute.
I also tried running my job like this:
$job = new eventJob('my test message'); dispatch($job);
But it does not change anything
I don't know why but when I changed config/queue.php file from 'default' => env('QUEUE_CONNECTION', 'sync') to 'default' => env('QUEUE_CONNECTION', 'database') everything started working as it should
I have a Laravel application and I use files with supervisor.
When I launch a job I have the following error after 60 seconds.
I have increased this delay everywhere but the timeout remains at 60 seconds.
Do you know how to increase the timeout of a job please?
I use the database connector (I also tried with redis) but it's the same
Thank you.
class BuildAvailabilities implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
public $timeout = 600;
protected $userId;
/**
* Create a new job instance.
*
* #return void
*/
public function __construct($userId)
{
$this->onQueue('availabilities');
$this->userId = $userId;
}
/**
* Execute the job.
*
* #return void
*/
public function handle()
{
$availabilityService = app(AvailabilityBuilderService::class);
$availabilityService->updateAvailabilities($this->userId);
}
/**
* Get the middleware the job should pass through.
*
* #return array
*/
public function middleware()
{
return [(new WithoutOverlapping($this->userId))->dontRelease()];
}
}
In horizon.php under your's connection try to specify the 'timeout' property to higher value and increase also the 'tries' property of your connection. But the main problem for us is that our's jobs are just too unoptimized and runs too long.
Also you can also specify public $maxExceptions = 3; in job.
Tell me when you find a solution
And check this: https://github.com/laravel/horizon/issues/255
I'm trying to figure out what the correct way to do this is.
public $tries = 10;
/**
* Execute the job.
*
* #return void
*/
public function handle(){
$result_of_some_logic = false;
if($result_of_some_logic){
// for the purpose of this example, we're all done here.
} else{
// we need to retry 10 minutes from now. how to trigger this attempt, and this attempt only!, to fail?
}
}
I read the laravel documentation but it just isn't clear to me what the correct way to do this is. I noticed that if I create a php error (for instance throw new whateverisnotdeclaredinnamespace()) the job attempt will fail, the worker will retry until the amount of $tries have exceeded. This is pretty much the behavior I want but I obviously want a clean code solution.
So to summarise: in Laravel 5.8 what is the correct way to "mark" the attempt to have failed in the handle() function?
You could try releasing the job back to the queue with a delay:
$this->release($delayInSeconds);
Why not to use failed function to handle Your error?
https://laravel.com/docs/5.8/queues#dealing-with-failed-jobs
<?php
namespace App\Jobs;
use Exception;
use App\Podcast;
use App\AudioProcessor;
use Illuminate\Bus\Queueable;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
class ProcessPodcast implements ShouldQueue
{
use InteractsWithQueue, Queueable, SerializesModels;
protected $podcast;
/**
* Create a new job instance.
*
* #param Podcast $podcast
* #return void
*/
public function __construct(Podcast $podcast)
{
$this->podcast = $podcast;
}
/**
* Execute the job.
*
* #param AudioProcessor $processor
* #return void
*/
public function handle(AudioProcessor $processor)
{
// Process uploaded podcast...
}
/**
* The job failed to process.
*
* #param Exception $exception
* #return void
*/
public function failed(Exception $exception)
{
// Send user notification of failure, etc...
}
}
So I am trying to optimize my site and on every page load and exit I save a metric (time on page, ip address etc) for analytics. However these are decent sized bottlenecks on my server. When viewing the time it takes for things to run my entire function takes ~1-2ms and then saving to the DB takes ~100-200ms. So my goal is to run my function and then dispatch a new job, that will do the actual saving of the metric. This way all of the saving of my models can be offloaded to a queue. Below is a copy of my job
class SaveMetric implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
/**
* Create a new job instance.
*
* #return void
*/
public function __construct()
{
//
}
/**
* Execute the job.
*
* #return void
*/
public function handle(Metrics $metric)
{
//
$metric->save();
}
}
Then in my controller function after I grab all the values I need I run this
dispatch(new SaveMetric($newMetric));
This seems to run but does not seem to do anything. Am I missing something? (Edit) This does ~something~ it just saves a record to the DB with null in all the fields, as if I created a new metric without any values.
Is it required to pass a queue into the job dispatch?
Do I need to run a daemon or something similar to actually process the things in the queue?
I created the job using the artisan make:job command
You're pretty close.
class SaveMetric implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
protected $metric;
/**
* Create a new job instance.
*
* #param Metrics $metric
*/
public function __construct(Metrics $metric)
{
$this->metric = $metric;
}
/**
* Execute the job.
*
* #return void
*/
public function handle()
{
$this->metric->save();
}
}
According to the docs:
In this example, note that we were able to pass an Eloquent model directly into the queued job's constructor. Because of the SerializesModels trait that the job is using, Eloquent models will be gracefully serialized and unserialized when the job is processing. If your queued job accepts an Eloquent model in its constructor, only the identifier for the model will be serialized onto the queue. When the job is actually handled, the queue system will automatically re-retrieve the full model instance from the database.
I want to test that the job has been released back onto the queue in certain circumstances.
This is my job class:
class ChargeOrder extends Job
{
use InteractsWithQueue, SerializesModels;
/**
* The order model which is to be charged
*/
protected $order;
/**
* The token or card_id which allows us to take payment
*/
protected $source;
/**
* Create a new job instance.
*
* #param App\Order $order;
* #param string $source;
* #return array
*/
public function __construct($order, $source)
{
$this->order = $order;
$this->source = $source;
}
/**
* Execute the job.
*
* #return void
*/
public function handle(Charge $charge)
{
$result = $charge->execute($this->source, $this->order->totalInclVat());
$exception_errors = config('payment.errors.exception_errors');
// If we have an error that isn't caused by the user (anything but a card error)
// We're going to notify ourselves via slack so we can investigate.
if (array_key_exists('error', $result) && in_array($result['error']['code'], array_keys(config('payment.errors.other_error'))))
{
$client = new Client(config('services.slack.channels.payment_errors.url'), config('services.slack.channels.payment_errors.settings'));
$client->send(app()->environment() . ": " . $result['error']['code']);
}
// If the error is in the list of errors that throw an exception, then throw it.
if (array_key_exists('error', $result) && (in_array($result['error']['type'], $exception_errors) || in_array($result['error']['code'], $exception_errors)))
{
$status_code = config('payment.errors')[$result['error']['type']][$result['error']['code']]['status_code'];
$message = config('payment.errors')[$result['error']['type']][$result['error']['code']]['message'];
throw new BillingErrorException($status_code, $message);
}
// If we still have an error, then it something out of the user's control.
// This could be a network error, or an error with the payment system
// Therefore, we're going to throw this job back onto the queue so it can be processed later.
if (array_key_exists('error', $result) && in_array($result['error']['code'], array_keys(config('payment.errors.other_error'))))
{
$this->release(60);
}
}
}
I need to test that "$this->release(60)" is called in certain circumstances.
I'm trying to mock the job contract as so, in my tests:
// Set Up
$this->job = Mockery::mock('Illuminate\Contracts\Queue\Job');
$this->app->instance('Illuminate\Contracts\Queue\Job', $this->job);
And then
// During Test
$this->job->shouldReceive('release')->once();
But this isn't working.
Anybody have any ideas?
Try adding the following in you test before dispatching the job:
Queue::after(function (JobProcessed $event) {
$this->assertTrue($event->job->isReleased());
});
The code above will be triggered after the job is done and checks that the job has been released.
Make sure to remove any calls to Queue::fake()and $this->expectsJob() since these will prevent the actual job from being executed.
I solved this problem by creating an event that is only fired after the job is released back into the queue. Then in my tests I can use the Event mocks to watch for that event after I dispatch a job and know if we released it back into the queue or not.
// In your job
$this->release();
event(new MyJobReleasedEvent()); // Every time you have a release() call
// In your Unit Test
Event::fake([MyJobReleasedEvent::class]);
dispatch(new MyJob();
Event::assertDispatched(MyJobReleasedEvent::class);
If you wanted to get fancy I'm sure you could wire up your own Job class that did this automatically when release() was called, but I needed it infrequently enough to just do it inline as-needed.