I am working on a Laravel application, users can place football bets.
This is a simplified version of my tables:
users
- id
- name
bets
- id
- id_user
- cost
- profit (e.g. can be 0 if user lost this bet or any integer value if won)
- created_at (default laravel column, this should be used to group bets by month)
I need to show a chart with ROI (not looking for the formula, this can be simplified as calculateROI in your comments) of last six months from current one.
Let's assume current month is july, how can i write a query or use Eloquent to have something like:
[
[
"february" => 2%
],
[
"march" => 0%
],
[
"april" => 100%
],
[
"may" => 500%
],
[
"june" => 13%
],
[
"july" => 198%
],
]
Related
Is there any rule of thumb or any logical relation between maxProcesses, number of supervisors and the total number of queues in laravel horizon?
What if I have 15 supervisors and 40 queues (each supervisor has multiple queues based on their category)? What is the maximum number of maxProcesses I can assign to each supervisor (suppose balancing auto)?
I want to know that if there's a rule of thumb for a better performance on horizon by tuning these numbers, for example if the number of supervisor-x should not exceed the total number of queues and if the maxProcesses should not exceed a certain number based on the OS spec running the processes.
Is there any logical relation between these numbers? Is there a good document about this issue? I have seen this document on supervisor and also the Laravel Horizon docs, but have not found the answer to my questions.
I need to explain things in detail in order to understand the relation between all these things.
Supervisor exists out of some simple settings. The most important once are these:
[program:laravel-worker]
process_name=%(program_name)s_%(process_num)02d
command=php /home/forge/app.com/artisan queue:work
autostart=true
autorestart=true
numprocs=8
The most important setting here is numprocs=8, from supervisor the manual it says:
Supervisor will start as many instances of this program as named by numprocs. Note that if numprocs > 1, the process_name expression must include %(process_num)s (or any other valid Python string expression that includes process_num) within it.
This configuration of supervisor running a program called artisan queue:work will create 8 instances (processes, workers, the same thing) of artisan queue:work. This means that 8 jobs can be processed simultaneously, nothing more, nothing less.
Horizon doesn't define the numprocs, the only important setting you'll have to know is the stopwaitsecs=3600. This should always be far greater than the maximum time a job runs in your entire application. Here the absolute maximum amount would be 60 minutes.
Now Horizon comes with a balancing strategy where you can define the min and max number of processes (workers) and it's strategy using
'balance' => 'auto',
'minProcesses' => 1,
'maxProcesses' => 10,
What Horizon offers to do here is scale up or down the amount of processes (workers) according to the amount of workload present in the queue(s).
If you define a supervisor configuration like the following:
'environments' => [
'production' => [
'supervisor-1' => [
'connection' => 'redis',
'queue' => ['default', 'events', 'xls', 'whatever'],
'balance' => 'auto',
'minProcesses' => 10,
'maxProcesses' => 40,
'balanceMaxShift' => 1,
'balanceCooldown' => 3,
'tries' => 3,
],
],
],
Then all 4 queues, default, events, xls and whatever run all under the same conditions, will have a total of 40 workers available and a minimum of 10. So not each queue has 40 workers available, but all combined have 40 workers (processes) available.
The key point here of getting a good scale for each queue to work optimally, is to divide them into different categories, e.g.
short-load -> each job takes about 1 to 5 seconds.
medium-load -> each job takes about 5 to 30 seconds.
long-load -> each job takes up to 5 minutes.
extreme-load -> each job takes longer than 5 minutes, up to an hour.
If you only end up with two scenarios, like short-load and long-load, then you will have two configurations for horizon in such a way which would define how fast supervisor will respond to spawn new workers and how many times it will try to repeat a job if it has failed (where you seriously don't want to try a job that will fail each time after 59 minutes 3 times).
'environments' => [
'production' => [
'supervisor-1' => [
'connection' => 'redis',
'queue' => ['default', 'events'],
'balance' => 'auto',
'minProcesses' => 10,
'maxProcesses' => 40,
'balanceMaxShift' => 10,
'balanceCooldown' => 1,
'tries' => 3,
],
'supervisor-long-run' => [
'connection' => 'redis',
'queue' => ['xls', 'whatever'],
'balance' => 'auto',
'minProcesses' => 1,
'maxProcesses' => 10,
'balanceMaxShift' => 1,
'balanceCooldown' => 3,
'tries' => 1,
],
],
],
In one of your last comments you asked
I want to understand all those calculations you make, what's the formula for it
The formula is, 1 supervisor instance can have many queues, and all of these queues have a maximum amount of workers available. The queues are not that important, but the amount of jobs (and the kind of jobs) placed in these queues in a certain amount of time is.
Example:
4 queues producing 120 jobs each minute, need x amount of workers to be processed. If you scale up (or down) the amount of workers (processes), the amount of time it takes to process all these jobs until the queues are empty relates to the amount of workers you make available.
If you have 10 workers available, then 10 jobs will be processed simultaneously.
If you have 120 workers available, then 120 jobs will be processed simultaneously.
If 1 job takes 10 seconds to complete (as an example average) and an average of 120 jobs are put on a queue each minute. If you would like to process (clear the queue) all jobs within one minute, you need 120 jobs * 10 seconds for each job / 60 seconds in a minute = the amount of workers (processes) needed to complete all those jobs within 1 minute.
So yes, you can scale up the amount of workers to 64, 512 or 24890. It comes all back to the question how much load can your hardware handle.
Hope it made sense.
I'll clean up the text tomorrow using only workers, processes or instances .. it's such a mess ;)
Eagerloading with pagination is simple:
Model::with(['relation1', 'relation2'])->paginate();
There are 6 models M1, ..., M6 and model M1 has foreign key to models M2, ..., M6. There are at least 2,000,000 records in each model and model M1 has more than 10,000,000 records. The following statement
M1::paginate();
is fast enough but when relations are included, it takes more than 45 seconds to return the results. To improve the performance, I need to run the M1::paginate(); at the beginning, then include other relations.
My solution is to loop through the collection, gather the ids and add the relations. I would like to know does such thing have been implemented in Laravel before?
Whenever you are unsure about how the queries made, open the console (php artisan tinker) and write the following:
DB::listen(fn($q) => dump([$q->sql, $q->bindings, $q->time]))
For each query you make (in the current console session), you'll get an array containing the SQL, the bindings and the time it actually takes for the database to return the data (this does not take into account how long it takes PHP to turn these results into an Eloquent Collection).
For example, for a Model (A) that has one hasMany relation with another Model (B), look at the output below:
>>> DB::listen(fn($q) => dump([$q->sql, $q->bindings, $q->time]))
=> null
>>> App\Models\A::with('b')->get()->first()->id
array:3 [
0 => "select * from "a""
1 => []
2 => 0.0
]
array:3 [
0 => "select * from "b" where "b"."a_id" in (1, 2, 3, 4, 5,
6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26,
27)"
1 => []
2 => 0.0
]
=> 1
>>> App\Models\A::with('b')->paginate(5)->first()->id
array:3 [
0 => "select count(*) as aggregate from "a""
1 => []
2 => 0.0
]
array:3 [
0 => "select * from "a" limit 5 offset 0"
1 => []
2 => 0.0
]
array:3 [
0 => "select * from "b" where "b"."a_id" in (1, 2, 3, 4, 5)"
1 => []
2 => 0.0
]
As you can see, the pagination has an effect on the relationship queries made.
I have the following two collections:
Collection {#402 ▼
#items: array:1 [▼
4007 => "4007 - Container Deposit - 18.00 Drum - In Stock: 0.00"
]
}
Collection {#398 ▼
#items: array:3 [▼
1000 => "1000 - Acetone - 162.00 KG - In Stock: 10000.00"
1001 => "1001 - Acetone - 15.80 KG - In Stock: 0.00"
24662 => "24662 - 1L Untd Antifreeze Orange FO2272A60(Prem - 1.00 Litre - In Stock: 0.00"
]
}
Using Laravel's collection merge function:
$merged = $ref_prod_containers->merge($ref_cust_prod);
dd($merged);
I get the following:
Collection {#397 ▼
#items: array:4 [▼
0 => "4007 - Container Deposit - 18.00 Drum - In Stock: 0.00"
1 => "1000 - Acetone - 162.00 KG - In Stock: 10000.00"
2 => "1001 - Acetone - 15.80 KG - In Stock: 0.00"
3 => "24662 - 1L Untd Antifreeze Orange FO2272A60(Prem - 1.00 Litre - In Stock: 0.00"
]
}
However I wish to retain the original keys. The merge function is removing them and replacing with 0,1,2,3.
Thanks, Julian
You can use Laravel Collection's union() method. Beware that this behaves differently from merge() when dealing with duplicate keys: if the same key is present in both $array1 and $array2 and you go $merged = $array1->union($array2), then the value of $array1 will end up in the $merged collection, and the value of $array2 will be discarded (Laravel union documentation).
I would try to use string keys for the merging and merged collection.
From the laravel docs section collections, function merge()
If the given array's keys are numeric, the values will be appended to the end of the collection:
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
I am quite new in elastic search here so if anybody can help me here
Suppose I have selected
1) Category - Hollywood
2) Sub-Category - Bond Special
3) Genre - Action & Drama & Comedy ( as multiple selection will be there )
4) Language - English, Russian and Hindi ( as multiple selection will be there)
5) Release Year - 1990,1999,2000 ( as multiple selection will be there)
6) 3D Movie - True OR False (any one will be selected)
7) SortBy - “A-Z”OR “Z-A” OR “Date”
Can anyone help me in making this query for elastic-search. I will use "match_phrase" for making AND condition but the issue is matching parameters or search parameter will be multiple and comma separated (u can say).
and my index array is given below : -
[_source] => Array (
[id] => 43
[value] => GREENBERG
[imageName] => Done
[date] => (1905) USA (Bengali)
[language] => (Bengali) | 1905 | 1.47hrs
[directorName] => Alejandro González Iñárritu, Ang Lee
[castForSearch] => Ben Stiller, John Turturro
[viewDetailsUrl] => /movie/greenberg
[movieType] => Animation
[rating] => 0
[cast] => Ben Stiller, John Turturro, Olivier Martinez
[synopsis] => A man from Los Angeles, who moved to New York years ago, returns to L.A. to figure out his life while he house-sits for his brother. He soon sparks with his brother's assistant.
[code] => HLR06
[type] => Non-3D
[trailer] => https://www.youtube.com/watch?v=cwdliqOGTLw
[imdb_code] => 1234654
[tags] => Array
(
[0] => Animation
)
[genre] => Adventure
[languages] => Bengali
[categories_filter] => Array
(
[0] => Category 2,Hollywood
)
[sub_categories_filter] => Array
(
[0] => Sub-Category 1,Sub-Category 4,Sub-Category 5,Sub-Category 6,Sub-Category 7
)
)
Weekly Sunday 12 AM
everyday 12 AM
every day 12:15 AM
daily 12:01 AM
daily 12:01 AM
joinScreenCancellationScheduler - Weekly Sunday 12 AM
0 0 * * 7 curl <url>
goLiveDate - everyday 12 AM
0 0 * * * curl <url>
nearestDateDisable - every day 12:15 AM
15 0 * * * curl <url>
reminderOfEvent - daily 12:01 AM
01 0 * * * curl <url>
thresholdNotMet - daily 12:01 AM
daily 12:01 AM
To match against one of multiple possible values, use a terms query. You don't need a match_phrase query because you're not doing any sort of free-text matching.
You'll need to split the comma-separated values into arrays before indexing your data into Elasticsearch (or use a comma-separated tokenizer).
Your use case suggests that you don't care about scoring but only about filtering, in which case your query should probably just have a filter.
Sorting is not the same as filtering; for your A-Z/Z-A/Date sorting you'll need to use a sort clause outside of the query.
The final thing would probably look like this:
GET /my_index/my_type/_search
{
"query": {
"bool": {
"filter": [
"terms": { "genre": ["Action", "Drama", "Comedy"] },
"terms": { "language": ["English", "Russian", "Hindi"] },
// more terms filters
]
}
},
"sort": { "title": "asc" }
}
I created this device which sends a point to my webserver. My web server stores a Point instance which has the attributes created_at to reflect the point's creation time. My device consistently sends a request to my server at a 180s interval.
Now I want to see the periods of time my device has experienced outages in the last 7 days.
As an example, let's pretend it's August 3rt (08/03). I can query my Points table for points up to the last 3 days sorted by created_at
Points = [ point(name=p1, created_at="08/01 00:00:00"),
point(name=p2, created_at="08/01 00:03:00"),
point(name=p3, created_at="08/01 00:06:00"),
point(name=p4, created_at="08/01 00:20:00"),
point(name=p5, created_at="08/03 00:01:00"),
... ]
I would like to write an algorithm that can list out the following outages:
outages = {
"08/01": [ "00:06:00-00:20:00", "00:20:00-23:59:59" ],
"08/02": [ "00:00:00-23:59:59" ],
"08/03": [ "00:00:00-00:01:00" ],
}
Is there an elegant way to do this?