Confusions about Laravel Queues - laravel

I am using Laravel Queues and I am using IronMQ for it. But I have little bit confusion about how this process.
I have set my default connection in queue.php as 'default' => 'iron' and also set iron settings in same file.
Now I use
$this->dispatch(new createEvents($data, $user));
while createEvents class is a job class created as explained in Laravel tutorial. Now when following code is executed
$this->dispatch(new createEvents($data, $user));
It successfully creates a queue in my ironmQ account under project.
Now here is my confusion starts. I have queued some task to that queue but now how will I run that queue? How will I run the task that is queued? Do I need to create some extra code for it or Do I need to do some settings for it. Please guide

You don't need to go to your server and run this command by hand, you need to have process that will keep running, and perform those jobs.
I would recomment "supervisord"
http://supervisord.org/
This programs is for launching a script and keep it running, even if it fails, it will relaunch it(until certain amount of failures of course)
After you install it, you should probably create this supervisor task file:
[program:queue]
command=php artisan queue:listen --tries=3 --env=your_environment
directory=/path/to/laravel
stdout_logfile=/path/to/laravel/app/storage/logs/supervisord.log
redirect_stderr=true
autostart=true
autorestart=true

You can do php artisan queue:listen it will start all listed queue
or if you specify the queue name php artisan queue:listen queue_name
Don't forget to run php artisan queue:failed-table. This will make failed_jobs table in your database.
So if anything goes wrong when the queue run it will save failed queue to the database.
If you want the failed queue to get insert the database add this when run listen:
php artisan queue:listen connection-name --tries=3
to run the failed queue php artisan queue:retry all
Hope i answer your question.

Once your job is in the queue, and according to your question it is, you have two simple options:
Run one or more queue listeners on the same/different servers (using supervisor is recommended in Laravel documentation, see sample configuration)
Run queue worker manually or automatically, on regular basis (crontab)
php artisan queue:work iron
This command will fetch one job from the queue and process it. You launch it again – it fetches one more, and so on.
If you don't do extra processing and your queue driver is not 'sync' – your job will never see the day light.
My advice – launch queue workers manually on your development/test machine, and use supervisor on production server.
If your project is small and it doesn't require great scalability, you may want to simply switch to 'sync' driver (jobs will be processed immediately). There is no need to make the infrastructure more complicated, unless there is real necessity!

Related

How to make a persistent background service constantly listening to the database with laravel?

I want to make an uptime control application with Laravel.
New sites will be added to the database constantly.
Each site will be thoroughly checked. Whether the site is online or not, when was it interrupted, how long did it take to go online, etc. I will store such details in the database.
No problem so far, I can do these. I wanted to explain these parts to explain better.
However, I am stuck on how to make a background service through laravel.
For example, if I were doing this with plain php, I would create a service for linux and run it with "systemctl start foo", with the command "systemctl enable foo" I would always make it run. I would create a php file and it could run forever in a "while" loop. or I could use nohup or supervisor.
But I don't want to go this way. This is a framework and I want to move forward in that direction.
I did a lot of research but couldn't find a solution. There is a laravel queue way for some operations, but that's not exactly what I want. For this to work, we add it to the queue with the "dispatch" method and run the processes in the queue with the "php artisan queue:work" command. We use supervisor for continuous operation.
But in my case there is no "dispatch" trigger. There is a constant listening to the database and running a transaction.
I created a command with "php artisan make:command foo" and I run a command with "php artisan foo" it works ok.
But how can I run it continuously, is there a way around this? Should I go outside the framework?
If I can resolve this issue without leaving the framework, is there a way to monitor them on the frontend? (websocket etc. external)
Because I plan to run other services in the background and it would be great to be able to view their current status, uptime, network usage and output.
Or a small idea that comes to my mind, I will start the first process manually (I will manually add it to the queue via dispatch and run the queue on the supervisor side). When the first transaction is completed, it will send the next site to Jobs via dispatch. but it doesn't seem like a very healthy way.
I preferred to proceed using supervisor.
I used a configuration like this:
[program:your-progam-name]
process_name=%(program_name)s_%(process_num)02d
command=php /home/foo/bar/baz/laravel_path/artisan yourProgramCommandCode
autostart=true
autorestart=true
stopasgroup=true
killasgroup=true
user=paparazi
redirect_stderr=true
stdout_logfile=/home/foo/bar/baz/laravel_path/logs/visor.log
stopwaitsecs=3600
details: https://laravel.com/docs/9.x/queues#configuring-supervisor

how to use queue when requests are parallel in laravel

my function in controller Calling parallel and i create a job for use queue in laravel Because parallel call causing the problem
i call this job in my function :
$this->dispatch(new ProcessReferal($orderId));
and i run this command in terminal :
php artisan queue:work --tries=3
But my job is still running in parallel
And processes the process simultaneously
what's wrong?
If you are checking it on Local server. Then, You have to add QUEUE_DRIVER=database in .env file.
QUEUE_DRIVER=sync is used for parallel call
Hi there,
With queue laravel, you need config some info in your code:
See more: https://laravel.com/docs/5.8/queues#connections-vs-queues
First:
Driver: default sync, so you need change it to: database, redis...
You can change it in .env file (QUEUE_DRIVER=database...)
Connections: Very important if you setting driver is database and use mutil DB for your project.
Second:
Laravel queue have some config, but i think we need see 3 thing: retry_after, timeout, tries. When you work with large job, retry_after and timeout is very important.
Hope it can help you.

laravel queue without DB query

I want to minimize DB queries for an incoming request. It currently requires writes to 6 different tables. The processing does not need to be done before returning the response.
I therefore consider laravel queues, but I wonder if I could also get rid of the separate query that is needed to write to the queue/jobs table. Can I store jobs locally instead of writing them to the DB?
One possible hack would be to have a separate route that I send the data to that does the processing. This way i would not need to write to the DB, but could rather just forward the data to that route without waiting for a response. Would this be faster than writing a job to the DB?
If you don't want to use native queue driver (that is using DB) I can advised you to use Rabbit-MQ. here is a good driver implementation for Laravel:
RabbitMQ Queue driver for Larave
You will have to run deamon service in the CLI using:
./artisan queue:work (deamon version - after you made each change in code you will have to call ./artisan queue:restart to refresh the code for queue)
or
./artisan queue:listen (for ./artisan queue:work --once process productor)
or use Supervisor with:
./artisan queue:work --once
(my personal favourite)
After that each task you will put to queue (with dispatch() or \Queue::push()) will be delivered to RabbitMQ server and will be exectuted through the queue driver.

Laravel 5.5 listen dynamically generated queues

My application requires to have dynamically generated queues with some prefix like "process_user_1", "process_user_2", "process_user_n"
The main idea is to separate execution of some jobs depends on model ID.
To run watcher I need execute command php artisan queue:work --queue process_user_1
I didn't find possibility to put pattern like
php artisan queue:work --queue process_user_*
And only one way what I found it is to run them manually each time before job is sent. But it's so dirty...
Maybe someone know another way?
EXAMPLE:
I have 10 users. Each user has 100 jobs to process in queue.
If I put them to one queue like "process_user_job" users will wait a
lot of time when jobs will be finished.
So I want to separate queues to speed up result returning

How to fire Laravel Queues with beanstalkd

I'm pretty new to the whole Queue'd jobs thing in Laravel 4. I have some process heavy tasks I need the site to run in the background after being fired by the user doing a particular action.
When I was doing the local development for my site I was using this:
Queue::push('JobClass', array('somedata' => $dataToBeSent));
And I was using the local "sync" driver to do it. (The jobs would just automatically fire, impacting on the user experience but I assumed when going into the production phase I could switch it to beanstalkd and they would then be run in the background)
Which brings me to where I'm at now. I have beanstalkd set up with the dependencies installed with composer and the beanstalkd process listening for new jobs. I installed a beanstalk admin interface and can see my jobs going into the queue, but I have no idea how to actually get them to run!
Any help would be apprieciated, thanks!
This is actually a really badly documented feature in Laravel.
What you actually need to do is have the JobClass.php in a folder that is auto-loaded, I use app/commands, but they can also be in app/controllers or app/models if you like. And this function needs to have a fire event that takes the $job and $data argument.
To run these, simply execute php artisan queue:listen --timeout=60 in your terminal, and it will be busy emptying the queue, until it's empty, or it's been running for longer then 60 seconds. (Small note: The timeout is the time-limit to start a queue, so it may run for 69 seconds if 1 job takes 10 seconds.
If you only want to run 1 job (perfect for testing), run php artisan queue:work
There are tools like Supervisord that make sure your job handlers keep running, but I recommend to just make a Cron task that starts every X minutes based on how fast the data needs to be processed, and on how much data comes in.
Keep in mind you need to path your artisan.
php /some/path/to/artisan queue:work

Resources