I have a Laravel application and I use files with supervisor.
When I launch a job I have the following error after 60 seconds.
I have increased this delay everywhere but the timeout remains at 60 seconds.
Do you know how to increase the timeout of a job please?
I use the database connector (I also tried with redis) but it's the same
Thank you.
class BuildAvailabilities implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
public $timeout = 600;
protected $userId;
/**
* Create a new job instance.
*
* #return void
*/
public function __construct($userId)
{
$this->onQueue('availabilities');
$this->userId = $userId;
}
/**
* Execute the job.
*
* #return void
*/
public function handle()
{
$availabilityService = app(AvailabilityBuilderService::class);
$availabilityService->updateAvailabilities($this->userId);
}
/**
* Get the middleware the job should pass through.
*
* #return array
*/
public function middleware()
{
return [(new WithoutOverlapping($this->userId))->dontRelease()];
}
}
In horizon.php under your's connection try to specify the 'timeout' property to higher value and increase also the 'tries' property of your connection. But the main problem for us is that our's jobs are just too unoptimized and runs too long.
Also you can also specify public $maxExceptions = 3; in job.
Tell me when you find a solution
And check this: https://github.com/laravel/horizon/issues/255
Related
I am trying to learn about Jobs and queues in Laravel , when i try to learn something new i always take a basic example by myself and try to understand the workflow better.
Okay here is the problem
I have created a Job in Laravel as you can see in the handle method i am just trying to print a simple message on the laravel.logger , this works totally fine.
class SendEmailJob implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
protected $email;
/**
* Create a new job instance.
*
* #return void
*/
public function __construct($email)
{
$this->email = $email;
}
/**
* Execute the job.
*
* #return void
*/
public function handle()
{
Log::info($this->email . '<<<<<<<<<<<<<<<');
}
}
My problem is that i want to delay this job for 2 minutes then to dispatch it ,
i have tried it this way but without success both logs are printed immediately when index method is being called but i want $job = new SendEmailJob("This will show after 2 minutes"); to be called after 2 minutes an not printed immediately
public function index(){
$on = Carbon::now()->addMinutes(2);
Log::info('Test');
$job = new SendEmailJob("This will show after 2 minutes");
$job->delay($on);
dispatch($job);
return "none";
}
You can take a look at the documentation: https://laravel.com/docs/8.x/queues#delayed-dispatching
You can do following: (new SendEmailJob("This will show after 2 minutes"))->delay(now()->addMinutes(2)); or
SendEmailJob::dispatch("This will show after 2 minutes")->delay(now()->addMinutes(2));
I'm trying to figure out what the correct way to do this is.
public $tries = 10;
/**
* Execute the job.
*
* #return void
*/
public function handle(){
$result_of_some_logic = false;
if($result_of_some_logic){
// for the purpose of this example, we're all done here.
} else{
// we need to retry 10 minutes from now. how to trigger this attempt, and this attempt only!, to fail?
}
}
I read the laravel documentation but it just isn't clear to me what the correct way to do this is. I noticed that if I create a php error (for instance throw new whateverisnotdeclaredinnamespace()) the job attempt will fail, the worker will retry until the amount of $tries have exceeded. This is pretty much the behavior I want but I obviously want a clean code solution.
So to summarise: in Laravel 5.8 what is the correct way to "mark" the attempt to have failed in the handle() function?
You could try releasing the job back to the queue with a delay:
$this->release($delayInSeconds);
Why not to use failed function to handle Your error?
https://laravel.com/docs/5.8/queues#dealing-with-failed-jobs
<?php
namespace App\Jobs;
use Exception;
use App\Podcast;
use App\AudioProcessor;
use Illuminate\Bus\Queueable;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
class ProcessPodcast implements ShouldQueue
{
use InteractsWithQueue, Queueable, SerializesModels;
protected $podcast;
/**
* Create a new job instance.
*
* #param Podcast $podcast
* #return void
*/
public function __construct(Podcast $podcast)
{
$this->podcast = $podcast;
}
/**
* Execute the job.
*
* #param AudioProcessor $processor
* #return void
*/
public function handle(AudioProcessor $processor)
{
// Process uploaded podcast...
}
/**
* The job failed to process.
*
* #param Exception $exception
* #return void
*/
public function failed(Exception $exception)
{
// Send user notification of failure, etc...
}
}
I have an API REST built using Laravel 5.8 framework.
I have to get the data for my app from a public REST API. For this, i have to do a lot of requests. I made a seeder what do this, and it takes 2 minutes for the entire migration data process approximately (take from the public Api and insert in my application database).
I cant run the seeder by a cronjob because it does not work (it looks like need an user to execute the command to work). So, i created a job class what i call from my kernel file, the queue configuration was seted as database (QUEUE_CONNECTION=database), like a scheduled task (to execute with Supervisor: https://laravel.com/docs/5.8/queues#supervisor-configuration).
Despite this, the job fails, because it takes a long time to execute, so my data is not updated.
What i can do process my jobs by batch successfully?
This is my kernel.php
<?php
namespace App\Console;
use App\Jobs\Seed\ApiPlayerStatisticJob;
use Illuminate\Console\Scheduling\Schedule;
use Illuminate\Foundation\Console\Kernel as ConsoleKernel;
class Kernel extends ConsoleKernel
{
/**
* The Artisan commands provided by your application.
*
* #var array
*/
protected $commands = [
//
];
/**
* Define the application's command schedule.
*
* #param \Illuminate\Console\Scheduling\Schedule $schedule
* #return void
*/
protected function schedule(Schedule $schedule)
{
$schedule->job(new ApiPlayerStatisticJob)->everyFiveMinutes();
}
/**
* Register the commands for the application.
*
* #return void
*/
protected function commands()
{
$this->load(__DIR__.'/Commands');
require base_path('routes/console.php');
}
}
My ApiPlayerStatisticJob.php
<?php
namespace App\Jobs\Seed;
use App\ApiExternal;
use App\ApiPlayer;
use App\ApiTeam;
use Illuminate\Bus\Queueable;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
class ApiPlayerStatisticJob implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
/**
* Create a new job instance.
*
* #return void
*/
public function __construct()
{
//
}
/**
* Execute the job.
*
* #return void
*/
public function handle()
{
$api_teams = ApiTeam::get();
foreach ($api_teams as $api_team) {
// echo 'TEAM: '.$api_team->id;
$api_external_players = ApiExternal::getTeamPlayerStatistics($api_team->id);
foreach ($api_external_players as $api_external_player) {
// echo PHP_EOL.'PLAYER: '.$api_external_player['id'];
$api_player = ApiPlayer::find($api_external_player['id']);
if ($api_player != null) {
$api_player->update($api_external_player);
// echo PHP_EOL.'> PLAYER UPDATED ';
} else {
// echo PHP_EOL.'X PLAYER DIDNT UPDATED ';
}
}
// echo PHP_EOL;
// echo PHP_EOL;
}
}
}
And my Seeder what i used for construct my job (by replication, excluding the print expressions):
<?php
use Illuminate\Database\Seeder;
use App\ApiExternal;
use App\ApiPlayer;
use App\ApiTeam;
class ApiPlayerStatisticsSeeder extends Seeder
{
/**
* Run the database seeds.
*
* #return void
*/
public function run()
{
$api_teams = ApiTeam::get();
foreach ($api_teams as $api_team) {
echo 'TEAM: '.$api_team->id;
$api_external_players = ApiExternal::getTeamPlayerStatistics($api_team->id);
foreach ($api_external_players as $api_external_player) {
echo PHP_EOL.'PLAYER: '.$api_external_player['id'];
$api_player = ApiPlayer::find($api_external_player['id']);
if ($api_player != null) {
$api_player->update($api_external_player);
echo PHP_EOL.'> PLAYER UPDATED ';
} else {
echo PHP_EOL.'X PLAYER DIDNT UPDATED ';
}
}
echo PHP_EOL;
echo PHP_EOL;
}
}
}
Finally, the static function ApiExternal::getTeamPlayerStatistics(int id) do all requests necessaries to get th data (like 30 requests), so what i can do to process this job (or the seeder directly) in background without it fails?
Did you configure the config/queue.php file correctly?
Quote from the documentation in the Job expirations paragraph:
In your config/queue.php configuration file, each queue connection defines a retry_after option. This option specifies how many seconds the queue connection should wait before retrying a job that is being processed.
I need to set a delay between specific failed jobs on listeners.
I know if specify the onption --delay=5 its works, but I need specific delay on a listener (not on standard job). I try put the property delay on Listener, but doesn't works.
<?php
namespace Froakie\Listeners;
use Carbon\Carbon;
use Froakie\Events\ExampleEvent;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Queue\InteractsWithQueue;
/**
* Class ExampleListener
*
* #package Froakie\Listeners
* #author Miguel Borges <miguel.borges#edirectinsure.com>
*/
class ExampleListener implements ShouldQueue
{
use InteractsWithQueue;
/**
* The number of seconds the job can run before timing out.
*
* #var int
*/
public $timeout = 5;
/**
* The number of times the job may be attempted.
*
* #var int
*/
public $tries = 3;
public $delay = 5;
public $seconds;
/**
* Handle the event.
*
* #param \Froakie\Events\ExampleEvent $event
* #throws \Exception
*/
public function handle(ExampleEvent $event)
{
// $this->delay(5);
throw new \Exception('test');
}
}
You use release to delay a retry. Example:
public function handle(ExampleEvent $event)
{
if ($this->attempts() <= $this->tries) {
try {
//Try something
} catch (\Exception $e) {
//Try again later
$this->release($this->delay)
}
} else {
//Force end the job
$this->delete();
}
}
It should be noted, however, that the value entered is the delay time in seconds. So if you wanted to to delay it by 5 minutes:
$this->release(300);
UPDATE - This has been narrowed down to beanstalkd, sync works
I am receiving the following error when attempting to run queued commands in my production environment:
exception 'ErrorException' with message 'unserialize(): Function spl_autoload_call() hasn't defined the class it was called for'
in /home/forge/default/vendor/laravel/framework/src/Illuminate/Queue/CallQueuedHandler.php:74
I have tried both the beanstalkd and database drivers, no change. For simplicity, I am using the following command:
<?php namespace App\Commands;
use App\Commands\Command;
use App\User;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Bus\SelfHandling;
use Illuminate\Contracts\Queue\ShouldBeQueued;
class TestQueueCommand extends Command implements SelfHandling, ShouldBeQueued {
use InteractsWithQueue, SerializesModels;
/**
* #var User
*/
private $user;
/**
* Create a new command instance.
*
* #param User $user
*/
public function __construct(User $user)
{
//
$this->user = $user;
}
/**
* Execute the command.
*
* #return void
*/
public function handle()
{
\Log::info("You gave me " . $this->user->fullName());
}
}
Dispatch code:
get('queue-test', function()
{
Bus::dispatch(new TestQueueCommand(User::first()));
});
This works in my Homestead environment, fails in production (Digital Ocean, Forge). I have several beanstalkd workers and I have tried restarting them. I have also run php artisan queue:flush.
Here is the code where the error is occurring (from source):
/**
* Handle the queued job.
*
* #param \Illuminate\Contracts\Queue\Job $job
* #param array $data
* #return void
*/
public function call(Job $job, array $data)
{
$command = $this->setJobInstanceIfNecessary(
$job, unserialize($data['command'])
);
$this->dispatcher->dispatchNow($command, function($handler) use ($job)
{
$this->setJobInstanceIfNecessary($job, $handler);
});
if ( ! $job->isDeletedOrReleased())
{
$job->delete();
}
}
In the past, I also ran into a similar issue while unserializing. The problem was the default Beanstalk job size (65,535 bytes), which might not be big enough if the class being serialized contains lots of properties that need to be kept (increasing the size of the serialized string and using more than 65K for storage).
In order to solve this, try setting the size to 131,072 or even 262,144 bytes using the -z option, on the configuration file (/etc/default/beanstalkd):
BEANSTALKD_EXTRA="-z 262144"
After that, you should restart the service.
Also note that the configuration file path might be other, depending on the distribution you're using.
And since you're using Digital Ocean, you might find their documentation useful.