I am trying to test that some data gets populated on a page that is done by a job. In my testing environment, the queue isn't running.
Is there any way to manually run the jobs from a function in a controller? I have retrieved all Jobs from my database by doing the following:
$allJobs = Jobs::all();
foreach ($allJobs as $job) {
// $job->handle(); ????
}
What I would like is to iterate over each job and process them myself. My test suite can wait for these jobs to be processed. I can't seem to find any documentation about this. Thanks!
If the goal is to be able to write tests for your jobs, it is fairly simple:
public function testJobsEvents()
{
$job = new \App\Jobs\YourJob;
$job->handle();
// Assert the side effect of your job...
}
Related
I have a batched series of chained jobs, and inside those chains I need to be able to batch other jobs.
Say I have 3 Clients
For Each Client I need to
Sync their details with an external API
Create 0 or more new cases and sync them individually
Update 0 or more existing cases and sync them individually
And I need the wrapping batch to keep track of when this is all finished.
I currently have the following structure:
$jobs = $clients->map(fn(Client $client) => [
new SyncClientJob(...),
new CreateMultipleCasesJob(...),
new UpdateMultipleCasesJob(...)
]);
Bus::batch($jobs)->name('BatchA')->etc()
In CreateCasesJob, something along the lines of
public function handle()
{
$jobs = $collection_of_new_cases->map(fn(Case $case) => new CreateSingleCaseJob($case));
Bus::batch($jobs)->dispatch();
}
CreateCasesJob and UpdateCasesJob should both dispatch their own batch of jobs, since each case needs to be synced individually
The problem is of course that the Create/Update jobs are "complete" in the chain when they're dispatched, not when all their internal jobs are completed. So the BatchA job will be marked as completed when it hasn't yet synced any cases.
I solved this by having each batch of jobs dispatch an event in the ->finally() callback. The listener for that event would then build and start the next batch.
I have job A which downloads xml and then calls other job B which will create data on database. This job B will be called in loop and can be more than 10.000 items. First tried to use chain method but problem is that, if someone will call queue in wrong sequence it will not work. Then tried to use batch from new Laravel 8. Collecting all jobs (more than 10000) to one batch can cause out of memory exception. Other problem is calling job C at the end. This job will update some credentials. Thats why job A and B must be runned successfully. May be there is any good idea for this situation?
Laravel's job batching feature allows you to easily execute a batch of jobs and then perform some action when the batch of jobs has completed executing.
If you have an out-of-memory problem with Jobs Batching you are doing things wrong. Since the queues are executed one by one if you have it configured that way there should be no problems, even if they are more than 100k records. So, make sure you glue one Job for each item, and execute the action, you won't have problems with this.
Then, you could do something like this.
$chain = [
new ProcessPodcast(Podcast::find(1)),
new ProcessPodcast(Podcast::find(2)),
new ProcessPodcast(Podcast::find(3)),
new ProcessPodcast(Podcast::find(4)),
new ProcessPodcast(Podcast::find(5)),
...
// And so on for all your items.
// This should be generated by a foreach with all his items.
];
Bus::batch($chain)->then(function (Batch $batch) {
// All jobs completed successfully...
// Uupdate some credentials...
})->catch(function (Batch $batch, Throwable $e) {
// First batch job failure detected...
})->finally(function (Batch $batch) {
// The batch has finished executing...
})->dispatch();
I'm developing a binary multilevel marketing system in Laravel, at the registration time there we have to perform a task to entries for many types of bonus for each parent nodes of a new user. This task is time-consuming.
No one user want to see buffering and task taking more than 30 second that is not the right way.
I want to run this mechanism in the background and send a success message that your account created successfully.
You could use observers that trigger queued jobs.
After the user does an action on a model, the observers create queued jobs in the background. While the queue is being processed the user can continue working.
either implement laravel job and queues or use https://github.com/spatie/async.
you can invoke sub processes to make your task
use Spatie\Async\Pool;
$pool = Pool::create();
foreach ($things as $thing) {
$pool->add(function () use ($thing) {
// Do a thing
})->then(function ($output) {
// Handle success
})->catch(function (Throwable $exception) {
// Handle exception
});
}
$pool->wait();
In need run other function without stop.
How to use thread in laravel 5.6?
For example:
public function index()
{
$id = "123456";
$this->run_bot($id);
return view("index");
}
Funtion run_bot it takes about 10 minutes !!!!
I need run run_bot in a thread.
How to craete thread in laravel 5.6?
Look into Symfomy's Process Component.
As an example, you can start the process and then later wait for it to complete:
$process = new Process('ls -lsa');
$process->start();
// ... do other things
// this is optional, you don't need to wait if not necessary
$process->wait();
The solution you are looking for is how to run asynchronous jobs. This can be done with a queue service (like AWS SQS) and the Laravel queue worker.
It will allow you to send the job (really light work so it's really speed). And then, asynchronously, retrieve and execute the job.
Everything you will need to know is here :
https://laravel.com/docs/5.6/queues
Let me know if it helped you :)
I am working with scheduling in Laravel 5.3. Previously, I was using one server to host the laravel application. Now that I am using two servers to run the Laravel App, how do I ensure that both servers are not running the same jobs at the same time?
Recently, I saw an Event method called "withoutOverlapping()". See https://laravel.com/docs/5.3/scheduling#preventing-task-overlaps
In my case, withoutOverlapping() cannot help me as I am working in a clustered environment.
Are there any workarounds or suggestions regarding this?
First of all, define if it is critical or not to avoid running task multiple times.
For example, if your app is using a task to do some sort of cleanup, there is almost no drawback to run it on every server (who care if you try to delete messages with +10 min twice?)
If it is absolutely critical to run every task only one time, you'll need to define a "main server" that will execute tasks, and a slave server that will just answer to requests but not perform any task. This is quite trivial as you just have to give every env a different name in your .env, and test against that when you define the scheduler tasks.
This is the easiest way, seriously don't bother making a database locking mecanism or whatever so you can synchronise tasks accross servers. Even OS's struggle to manage properly synchronisation against threads on the same machine, why do you want to implement the same accross different machines?
Here's what I've done when I ran into the same problems with load balancing:
class MutexCommand extends Command {
private $hash = null;
public function cleanup() {
if (is_string($this->hash)) {
Redis::del($this->hash);
$this->hash = null;
}
}
protected abstract function generateHash();
protected abstract function handleInternal();
public final function handle() {
register_shutdown_function([$this,"cleanup"]);
try {
$this->hash = $this->generateHash();
//Set a value if it does not exist atomically. Will fail if it does exist.
//Essentially setnx is the mechanism to acquire the lock
if (!Redis::setnx($this->hash,true)) {
$this->hash = null; //Prevent it from being cleaned up
throw new Exception("Already running");
}
$this->handleInternal();
} finally {
$this->cleanup();
}
}
}
Then you can write your commands:
class ThisShouldNotOverlap extends MutexCommand {
public function generateHash() {
return "Unique key for mutex, you can just use the class name if you want by doing return static::class";
}
public function handleInternal() { /* do stuff */ }
}
Then whenever you try to run the same command on multiple instances one would successfully acquire the "lock" and the others should fail.
Of course this assumes that you are using a non-clustered redis cache.
If you are not using redis then there's probably similar locking mechanisms you can implement in other caches, if you are using a clustered redis then you may need to use the RedLock locking mechanism
Essentially no, there's no a natural way using Laravel to know if another Laravel app have the same job on the job dispatcher.
We have some options there to find a solution:
Create a intermediate app that manages the jobs from the other apps.
Allow only one app to dispatch jobs.
Use worker queues, you have some packages for this, I would recommend to use Laravel 5 with WebSockets and Queue Asynchronously.
First of all Laravel scheduler isn't designed to work in a clustered environment. It was never intended to be that way.
I would suggest you should have a dedicated cron instance which manages your Laravel scheduler jobs.