I'm trying to list the amount of games players played (and finished) in a data-table. Currently I use WithCount, but it turns out to be quite slow.
$data = Player::withCount(['games' => function (Builder $query) {
$query->where('ended', 1);
}])->get();
Is there a better way to grab the count of records with a where condition that isn't as resource intensive?
Related
I'm wondering if what I'd like to achieve is possible with Eloquent. Say you have a Post model with a Comments model relationship, and the Comment model has a votes property.
I know you can get a Post collection with the Comments count with:
$posts = Post::withCount('comments')->get();
And that this is how to apply a votes filter on the relationship:
$posts = Post::whereHas('comments', function (Builder $q, $vote_count) {
$q->where('votes', '>=', $vote_count);
});
But what I would like to do, ideally in one query, is filter (using where()) and sort (using orderBy()) the posts collection on an average vote score.
To get that average for one Post I would first get the sum with sum('votes') and divide it by the withCount('comments') to get the average.
But I'm wondering if I can make a filter for a collection, all in one query, that filters on the average vote score.
You can get the Post comments votes average passing an array with the relationship name as key and an anonymous function as value. Inside the function you can perform a selectRaw querying the votes column average. Then you will have available that "column" (an alias actually) to do the where and the orderBy:
$value = 5;
Post::withCount(['comments as comments_avg_votes' => function($query) {
$query->select(DB::raw('avg(votes)'));
}])
->where('comments_avg_votes', '>', $value)
->orderByDesc('comments_avg_votes')
->get();
I have 2 models, Transaction, TrackingNumber.
The relationships and fields are:
Transaction: hasMany TrackingNumber
Transaction fields: id | carrier
TrackingNumber fields: id | transaction_id
I am trying to query TrackingNumbers but I want to return only where the carrier is ups. In this sense:
a) If I use eager loading, it will return all the tracking numbers regardless of shipper and only return transaction relationship based on the where:
$collection = TrackingNumber::with(['transaction' => function($q) {
$q->where('carrier', 'ups')
}])->get();
So this is not what I want. What I want is that the $collection has only the TrackingNumbers with given carrier instead.
b) If I do something like this, the result is what I want, but it is very inefficient (as it makes 2 queries and stores data in array; also if the array is too big, it fails on whereIn.)
$transactionIds = Transaction::where('carrier', 'ups')->pluck('id');
$trackingNumbers = TrackingNumber::whereIn('id', $transactionIds)->get();
// The result is what I want but it is inefficient.
c) I also tried something like this but it doesn't care about the where :/
\DB::table('tracking_numbers')
->join('transactions', 'tracking_numbers.transaction_id', '=', 'transactions.id')
->where('transactions.carrier', 'ups')
->get();
d) I don't want to use whereHas() because it gets extremely slow with a lot of data (I have +1m rows in my tables)
What is the Laravel way to achieve this?
Using whereHas should get you the results you want:
TrackingNumber::whereHas('transaction', function ($query) {
$query->where('carrier', 'ups');
})->get();
Or, try restructuring the query from c like:
\DB::table('tracking_numbers')
->join('transactions', function ($join) {
$join->on('tracking_numbers.transaction_id', '=', 'transactions.id')
->where('transactions.carrier', 'ups');
})
->get();
I use the query to get the models that the user has, but only 1 model is returned for each user. How to get all? If I set $count=3 I should receive 3 or more models in group, but only first row is returned
$items->where(/*.....*/)
->groupBy('user_id')
->havingRaw("COUNT(*) >= {$count}")->get()
UPDATE
I solved it. I created a separate function for preparing the query and used it 2 times.
I think this may be an incorrect solution, but it works
$items = Items::query();
$this->prepareQuery($request, $items)
$items->whereHas('user', function ($q) use ($count, $request){
$q->whereHas('items', function ($query) use ($request){
$this->prepareQuery($request, $query);
}, '>=', $count);
})
->paginate(4);
This may work.Sometimes take can help.
$items->where(/*.....*/)
->groupBy('user_id')
->take({{$count}});
If you have relationships set up you can quite simply call:
$model->models()
$model being the model you're wanting the list for.
models() being the name of the relationship between the two items.
For example, a post may have many comments. You can call $post->comments() to get a list of the posts comments.
You may also query this from the database directly with some eloquent magic.
Post::with('comments')->where(/*....*/)->get();
EDIT:
You can check to see if a model has X number of related models like so.
Post::has('comments', '>=', 3)->where(/*...*/)->get();
An example:
$passing_students = App\Exam::whereHas('students', function ($query) {
$query->where('mark', '>=', $exam->pass_mark);
})->get();
I am interested in fetching all exams with students who passed however in the relationship query function I'm not sure how to access the parent model's properties such that I can complete the comparison. What should be in place of $exam->pass_mark?
Note that I'm looking for a solution that is done within the single query builder as I'm aware that this can be easily done in a separate foreach loop.
Probably instead of:
$query->where('mark', '>=', $exam->pass_mark);
you should use here:
$query->whereColumn('mark', '>=', 'exams.pass_mark');
Above exams.pass_mark is name of Exam model table (I assumed you use exams and column name from this table.
We are developing an API with LUMEN.
Today we had a confused problem with getting the collection of our "TimeLog"-model.
We just wanted to get all time logs with additional informationen from the board model and task model.
In one row of time log we had a board_id and a task_id. It is a 1:1 relation on both.
This was our first code for getting the whole data. This took a lot of time and sometimes we got a timeout:
BillingController.php
public function byYear() {
$timeLog = TimeLog::get();
$resp = array();
foreach($timeLog->toArray() as $key => $value) {
if(($timeLog[$key]->board_id && $timeLog[$key]->task_id) > 0 ) {
array_push($resp, array(
'board_title' => isset($timeLog[$key]->board->title) ? $timeLog[$key]->board->title : null,
'task_title' => isset($timeLog[$key]->task->title) ? $timeLog[$key]->task->title : null,
'id' => $timeLog[$key]->id
));
}
}
return response()->json($resp);
}
The TimeLog.php where the relation has been made.
public function board()
{
return $this->belongsTo('App\Board', 'board_id', 'id');
}
public function task()
{
return $this->belongsTo('App\Task', 'task_id', 'id');
}
Our new way is like this:
BillingController.php
public function byYear() {
$timeLog = TimeLog::
join('oc_boards', 'oc_boards.id', '=', 'oc_time_logs.board_id')
->join('oc_tasks', 'oc_tasks.id', '=', 'oc_time_logs.task_id')
->join('oc_users', 'oc_users.id', '=', 'oc_time_logs.user_id')
->select('oc_boards.title AS board_title', 'oc_tasks.title AS task_title','oc_time_logs.id','oc_time_logs.time_used_sec','oc_users.id AS user_id')
->getQuery()
->get();
return response()->json($timeLog);
}
We deleted the relation in TimeLog.php, cause we don't need it anymore. Now we have a load time about 1 sec, which is fine!
There are about 20k entries in the time log table.
My questions are:
Why is the first method out of range (what causes the timeout?)
What does getQuery(); exactly do?
If you need more information just ask me.
--First Question--
One of the issues you might be facing is having all those huge amount of data in memory, i.e:
$timeLog = TimeLog::get();
This is already enormous. Then when you are trying to convert the collection to array:
There is a loop through the collection.
Using the $timeLog->toArray() while initializing the loop based on my understanding is not efficient (I might not be entirely correct about this though)
Thousands of queries are made to retrieve the related models
So what I would propose are five methods (one which saves you from hundreds of query), and the last which is efficient in returning the result as customized:
Since you have many data, then chunk the result ref: Laravel chunk so you have this instead:
$timeLog = TimeLog::chunk(1000, function($logs){
foreach ($logs as $log) {
// Do the stuff here
}
});
Other way is using cursor (runs only one query where the conditions match) the internal operation of cursor as understood is using Generators.
foreach (TimeLog::where([['board_id','>',0],['task_id', '>', 0]])->cursor() as $timelog) {
//do the other stuffs here
}
This looks like the first but instead you have already narrowed your query down to what you need:
TimeLog::where([['board_id','>',0],['task_id', '>', 0]])->get()
Eager Loading would already present the relationship you need on the fly but might lead to more data in memory too. So possibly the chunk method would make things more easier to manage (even though you eagerload related models)
TimeLog::with(['board','task'], function ($query) {
$query->where([['board_id','>',0],['task_id', '>', 0]]);
}])->get();
You can simply use Transformer
With transformer, you can load related model, in elegant, clean and more controlled methods even if the size is huge, and one greater benefit is you can transform the result without having to worry about how to loop round it
You can simply refer to this answer in order to perform a simple use of it. However incase you don't need to transform your response then you can take other options.
Although this might not entirely solve the problem, but because the main issues you face is based on memory management, so the above methods should be useful.
--Second question--
Based on Laravel API here You could see that:
It simply returns the underlying query builder instance. To my observation, it is not needed based on your example.
UPDATE
For question 1, since it seems you want to simply return the result as response, truthfully, its more efficient to paginate this result. Laravel offers pagination The easiest of which is SimplePaginate which is good. The only thing is that it makes some few more queries on the database, but keeps a check on the last index; I guess it uses cursor as well but not sure. I guess finally this might be more ideal, having:
return TimeLog::paginate(1000);
I have faced a similar problem. The main issue here is that Elloquent is really slow doing massive task cause it fetch all the results at the same time so the short answer would be to fetch it row by row using PDO fetch.
Short example:
$db = DB::connection()->getPdo();
$query_sql = TimeLog::join('oc_boards', 'oc_boards.id', '=', 'oc_time_logs.board_id')
->join('oc_tasks', 'oc_tasks.id', '=', 'oc_time_logs.task_id')
->join('oc_users', 'oc_users.id', '=', 'oc_time_logs.user_id')
->select('oc_boards.title AS board_title', 'oc_tasks.title AS task_title','oc_time_logs.id','oc_time_logs.time_used_sec','oc_users.id AS user_id')
->toSql();
$query = $db->prepare($query->sql);
$query->execute();
$logs = array();
while ($log = $query->fetch()) {
$log_filled = new TimeLog();
//fill your model and push it into an array to parse it to json in future
array_push($logs,$log_filled);
}
return response()->json($logs);