I'm working in Laravel 4, and I have a Child model with multiple EducationProfiles:
class Child extends EloquentVersioned
{
public function educationProfiles()
{
return $this->hasMany('EducationProfile');
}
}
If I wanted to get all the EducationProfiles for each kid under age 10 it would be easy:
Child::where('date_of_birth','>','2004-03-27')->with('educationProfiles')->all();
But say (as I do) that I would like to use with() to grab a calculated value for the Education Profiles of each of those kids, something like:
SELECT `education_profiles`.`child_id`, GROUP_CONCAT(`education_profiles`.`district`) as `district_list`
In theory with() only works with relationships, so do I have any options for associating the district_list fields to my Child models?
EDIT: Actually, I was wondering whether with('educationProfiles') generates SQL equivalent to:
EducationProfile::whereIn('id',array(1,2,3,4))
or whether it's actually equivalent to
DB::table('education_profiles')->whereIn('id',array(1,2,3,4))
The reason I ask is that in the former I'm getting models, if it's the latter I'm getting unmodeled data, and thus I can probably mess it up as much as I want. I assume with() generates an additional set models, though. Anybody care to correct or confirm?
Ok, I think I've cracked this nut. No, it is NOT possible to eager load arbitrary queries. However, the tools have been provided by the Fluent query builder to make it relatively easy to replicate eager loading manually.
First, we leverage the original query:
$query = Child::where('date_of_birth','>','2004-03-27')->with('educationProfiles');
$children = $query->get();
$eagerIds = $query->lists('id');
Next, use the $eagerIds to filterDB::table('education_profile') in the same way that with('educationProfiles') would filter EducationProfile::...
$query2 = DB::table('education_profile')->whereIn('child_id',$eagerIds)->select('child_id', 'GROUP_CONCAT(`education_profiles`.`district`) as `district_list`')->groupBy('child_id');
$educationProfiles = $query2->lists('district_list','child_id');
Now we can iterate through $children and just look up the $educationProfiles[$children->id] values for each entry.
Ok, yes, it's an obvious construction, but I haven't seen it laid out explicitly anywhere before as a means of eager loading arbitrary calculations.
You can add a where clause to your hasMany() call like this:
public function educationProfilesUnderTen() {
$ten_years_ago = (new DateTime('10 years ago'))->format('Y-m-d');
return $this->hasMany('EducationProfile')->where('date_of_birth', '>', $ten_years_ago)
}
Related
I really tried to understand the difference between the with() method and the load() method, but couldn't really understand.
As I see it, using the with() method is "better" since I eager load the relation. It seems that if I use load() I load the relation just as if I would use the hasMany() (or any other method that relates to the relation between objects).
Do I get it wrong?
Both accomplish the same end results—eager loading a related model onto the first. In fact, they both run exactly the same two queries. The key difference is that with() eager loads the related model up front, immediately after the initial query (all(), first(), or find(x), for example); when using load(), you run the initial query first, and then eager load the relation at some later point.
"Eager" here means that we're associating all the related models for a particular result set using just one query, as opposed to having to run n queries, where n is the number of items in the initial set.
Eager loading using with()
If we eager load using with(), for example:
$users = User::with('comments')->get();
...if we have 5 users, the following two queries get run immediately:
select * from `users`
select * from `comments` where `comments`.`user_id` in (1, 2, 3, 4, 5)
...and we end up with a collection of models that have the comments attached to the user model, so we can do something like $users->comments->first()->body.
"Lazy" eager loading using load()
Or, we can separate the two queries, first by getting the initial result:
$users = User::all();
which runs:
select * from `users`
And later, if we decide that we need the related comments for all these users, we can eager load them after the fact:
$users = $users->load('comments');
which runs the 2nd query:
select * from `comments` where `comments`.`user_id` in (1, 2, 3, 4, 5)
...and we end up with the same result, just split into two steps. Again, we can call $users->comments->first()->body to get to the related model for any item.
Why use load() vs. with()? load() gives you the option of deciding later, based on some dynamic condition, whether or not you need to run the 2nd query. If, however, there's no question that you'll need to access all the related items, use with().
The alternative to either of these would be looping through the initial result set and querying a hasMany() relation for each item. This would end up running n+1 queries, or 6 in this example. Eager loading, regardless of whether it's done up-front with with() or later with load(), only runs 2 queries.
As #damiani said, Both accomplish the same end results—eager loading a related model onto the first. In fact, they both run exactly the same two queries. The key difference is that with() eager loads the related model up front, immediately after the initial query (all(), first(), or find(x), for example); when using load(), you run the initial query first, and then eager load the relation at some later point.
There is one more difference between With() & load(), you can put the conditions when using with() but you can't do the same in case of load()
For example:
ProductCategory::with('children')
->with(['products' => function ($q) use($SpecificID) {
$q->whereHas('types', function($q) use($SpecificID) {
$q->where('types.id', $SpecificID)
});
}])
->get();
#damiani Explanied difference between load() and with() as well but he said load() is not cacheable so I wanna say couple words about it.
Let assume we have a blog post and related with comments. And we're fetching together and caching it.
$post = Cache::remember("post.".$slug,720,function()use($slug){
return Post::whereSlug($slug)->with("comments")->first();
});
But if there is a new comment and we want to display it immediately, we have to clear post cache and fetch post and comments together again. And that causes unnecessary queries. Lets think there are another queries for tags, media, contributors of the post etc. it will increase amount of resource usage..
public function load($relations)
{
$query = $this->newQueryWithoutRelationships()->with(
is_string($relations) ? func_get_args() : $relations
);
$query->eagerLoadRelations([$this]);
return $this;
}
As you can see above when we use the method it loads given relation and returns model with fetched relation. So you can return it outside of a callback.
$post = Cache::remember("post.".$slug,720,function()use($slug){
return Post::whereSlug($slug)->first();
});
$post = Cache::remember("post.relation.images.".$slug,720,function()use($post){
return $post->load("images");
});
$post = Cache::remember("post.relation.comments".$slug,720,function()use($post){
return $post->load("comments");
});
So if we load them seperatly, next time when some of them updated all you need to do clear specific relation cache and fetch it again. No need to fetch post, tags, images etc. over and over.
So, in order to check the existence of a relationship on a model, we use the has function on the relationship like model1->has('relationship1').
While it is possible to supply the model1->with() function with an array of relations to eager load them all, both has and whereHas functions do not accept arrays as parameters. How to check for the existence of multiple relationships?
Right now, I am running multiple has functions on the same model (The relations are not nested):
model1->has('relationship1')
->has('relationship2')
->has('relationship3')
But that is tedious and error-prone. Solution anyone?
There unfortunately isn't a way to pass an array of relationships to has() or whereHas(), but you can use a QueryScope instead. On your Model, define the following:
public function scopeCheckRelationships($query){
return $query->has("relationship1")->has("relationship2")->has("relationship3");
}
Then, when querying your Model in a Controller, simply run:
$result = Model::checkRelationships()->get();
The function name to use a Scope is the name of the function, minus the word scope, so scopeCheckRelationships() is used as checkRelationships().
Also, it's actually possible to pass the relationships you want to query as a param:
public function scopeCheckRelationships($query, $relationships = []){
foreach($relationships AS $relationship){
$query->has($relationship);
// Might need to be `$query = $query->has(...);`, but I don't think so.
}
return $query;
}
...
$result = Model::checkRelationships(["relationship1", "relationship2", "relationship3"])->get();
In case you need this to be dynamic.
Here's the documentation for Query Scopes if you need more info: https://laravel.com/docs/5.8/eloquent#query-scopes
We are developing an API with LUMEN.
Today we had a confused problem with getting the collection of our "TimeLog"-model.
We just wanted to get all time logs with additional informationen from the board model and task model.
In one row of time log we had a board_id and a task_id. It is a 1:1 relation on both.
This was our first code for getting the whole data. This took a lot of time and sometimes we got a timeout:
BillingController.php
public function byYear() {
$timeLog = TimeLog::get();
$resp = array();
foreach($timeLog->toArray() as $key => $value) {
if(($timeLog[$key]->board_id && $timeLog[$key]->task_id) > 0 ) {
array_push($resp, array(
'board_title' => isset($timeLog[$key]->board->title) ? $timeLog[$key]->board->title : null,
'task_title' => isset($timeLog[$key]->task->title) ? $timeLog[$key]->task->title : null,
'id' => $timeLog[$key]->id
));
}
}
return response()->json($resp);
}
The TimeLog.php where the relation has been made.
public function board()
{
return $this->belongsTo('App\Board', 'board_id', 'id');
}
public function task()
{
return $this->belongsTo('App\Task', 'task_id', 'id');
}
Our new way is like this:
BillingController.php
public function byYear() {
$timeLog = TimeLog::
join('oc_boards', 'oc_boards.id', '=', 'oc_time_logs.board_id')
->join('oc_tasks', 'oc_tasks.id', '=', 'oc_time_logs.task_id')
->join('oc_users', 'oc_users.id', '=', 'oc_time_logs.user_id')
->select('oc_boards.title AS board_title', 'oc_tasks.title AS task_title','oc_time_logs.id','oc_time_logs.time_used_sec','oc_users.id AS user_id')
->getQuery()
->get();
return response()->json($timeLog);
}
We deleted the relation in TimeLog.php, cause we don't need it anymore. Now we have a load time about 1 sec, which is fine!
There are about 20k entries in the time log table.
My questions are:
Why is the first method out of range (what causes the timeout?)
What does getQuery(); exactly do?
If you need more information just ask me.
--First Question--
One of the issues you might be facing is having all those huge amount of data in memory, i.e:
$timeLog = TimeLog::get();
This is already enormous. Then when you are trying to convert the collection to array:
There is a loop through the collection.
Using the $timeLog->toArray() while initializing the loop based on my understanding is not efficient (I might not be entirely correct about this though)
Thousands of queries are made to retrieve the related models
So what I would propose are five methods (one which saves you from hundreds of query), and the last which is efficient in returning the result as customized:
Since you have many data, then chunk the result ref: Laravel chunk so you have this instead:
$timeLog = TimeLog::chunk(1000, function($logs){
foreach ($logs as $log) {
// Do the stuff here
}
});
Other way is using cursor (runs only one query where the conditions match) the internal operation of cursor as understood is using Generators.
foreach (TimeLog::where([['board_id','>',0],['task_id', '>', 0]])->cursor() as $timelog) {
//do the other stuffs here
}
This looks like the first but instead you have already narrowed your query down to what you need:
TimeLog::where([['board_id','>',0],['task_id', '>', 0]])->get()
Eager Loading would already present the relationship you need on the fly but might lead to more data in memory too. So possibly the chunk method would make things more easier to manage (even though you eagerload related models)
TimeLog::with(['board','task'], function ($query) {
$query->where([['board_id','>',0],['task_id', '>', 0]]);
}])->get();
You can simply use Transformer
With transformer, you can load related model, in elegant, clean and more controlled methods even if the size is huge, and one greater benefit is you can transform the result without having to worry about how to loop round it
You can simply refer to this answer in order to perform a simple use of it. However incase you don't need to transform your response then you can take other options.
Although this might not entirely solve the problem, but because the main issues you face is based on memory management, so the above methods should be useful.
--Second question--
Based on Laravel API here You could see that:
It simply returns the underlying query builder instance. To my observation, it is not needed based on your example.
UPDATE
For question 1, since it seems you want to simply return the result as response, truthfully, its more efficient to paginate this result. Laravel offers pagination The easiest of which is SimplePaginate which is good. The only thing is that it makes some few more queries on the database, but keeps a check on the last index; I guess it uses cursor as well but not sure. I guess finally this might be more ideal, having:
return TimeLog::paginate(1000);
I have faced a similar problem. The main issue here is that Elloquent is really slow doing massive task cause it fetch all the results at the same time so the short answer would be to fetch it row by row using PDO fetch.
Short example:
$db = DB::connection()->getPdo();
$query_sql = TimeLog::join('oc_boards', 'oc_boards.id', '=', 'oc_time_logs.board_id')
->join('oc_tasks', 'oc_tasks.id', '=', 'oc_time_logs.task_id')
->join('oc_users', 'oc_users.id', '=', 'oc_time_logs.user_id')
->select('oc_boards.title AS board_title', 'oc_tasks.title AS task_title','oc_time_logs.id','oc_time_logs.time_used_sec','oc_users.id AS user_id')
->toSql();
$query = $db->prepare($query->sql);
$query->execute();
$logs = array();
while ($log = $query->fetch()) {
$log_filled = new TimeLog();
//fill your model and push it into an array to parse it to json in future
array_push($logs,$log_filled);
}
return response()->json($logs);
In Laravel 4.2 I have a model called Product with many-to-many relationshis to other models like Country or Category. I want to filter out products that are "incomplete", which means they have no connected countries or no connected categories. I can use whereDoesntHave() method to filter out one relation. When I use it two times in one query it creates AND condition, but I need OR. I can't find orWhereDoesntHave() method in API documentation. I can't pass multiple relations as arguments because it expects first argument to be a string.
I need something like this:
$products = Product::whereDoesntHave('categories')->orWhereDoesntHave('countries')->get();
Is there any way to achive whereDoesntHave() with multiple OR conditions?
You can use doesntHave and specify the boolean operator:
$products = Product::doesntHave('categories')->doesntHave('countries', 'or')->get();
Actually you only need whereDoesntHave if you want to pass in a closure to filter the related models before checking if any of them exist. In case you want to do that you can pass the closure as third argument:
$products = Product::doesntHave('categories', 'or', function($q){
$q->where('active', false);
})->doesntHave('countries', 'or')->get();
Since Laravel 5.5 there is an orWhereDoesntHave function.
You may use it like this
Product::whereDoesntHave('categories', function($q){ //... })
->orWhereDoesntHave('countries', function($q){//...})
->get();
From you example it seems that you are not using a where clause, so you may just use
Product::doesntHave('categories')
->orDoesntHave('countries')
->get();
Use
Product::whereDoesntHave('categories')->doesntHave('countries', 'or')->get();
Laravel Source Code:
whereDoesntHave https://github.com/illuminate/database/blob/master/Eloquent/Builder.php#L654
calls
https://github.com/illuminate/database/blob/master/Eloquent/Builder.php#L628
internally.
Let’s say we have Authors and Books, with 1-n relationship – one Author can have one or many Books. Here’s how it looks in app\Author.php:
public function books()
{
return $this->hasMany(\App\Book::class, 'author_id');
}
Now, what if we want to show only those Authors that have at least one book? Simple, there’s method has():
$authors = Author::has('books')->get();
Similarly, there’s an opposite method – what if we want to query only the authors without any books? Use doesnthave():
$authors = Author::doesnthave('books')->get();
It’s not only convenient, but also super-easy to read and understand, even if you’re not a Laravel developer, right?
I really tried to understand the difference between the with() method and the load() method, but couldn't really understand.
As I see it, using the with() method is "better" since I eager load the relation. It seems that if I use load() I load the relation just as if I would use the hasMany() (or any other method that relates to the relation between objects).
Do I get it wrong?
Both accomplish the same end results—eager loading a related model onto the first. In fact, they both run exactly the same two queries. The key difference is that with() eager loads the related model up front, immediately after the initial query (all(), first(), or find(x), for example); when using load(), you run the initial query first, and then eager load the relation at some later point.
"Eager" here means that we're associating all the related models for a particular result set using just one query, as opposed to having to run n queries, where n is the number of items in the initial set.
Eager loading using with()
If we eager load using with(), for example:
$users = User::with('comments')->get();
...if we have 5 users, the following two queries get run immediately:
select * from `users`
select * from `comments` where `comments`.`user_id` in (1, 2, 3, 4, 5)
...and we end up with a collection of models that have the comments attached to the user model, so we can do something like $users->comments->first()->body.
"Lazy" eager loading using load()
Or, we can separate the two queries, first by getting the initial result:
$users = User::all();
which runs:
select * from `users`
And later, if we decide that we need the related comments for all these users, we can eager load them after the fact:
$users = $users->load('comments');
which runs the 2nd query:
select * from `comments` where `comments`.`user_id` in (1, 2, 3, 4, 5)
...and we end up with the same result, just split into two steps. Again, we can call $users->comments->first()->body to get to the related model for any item.
Why use load() vs. with()? load() gives you the option of deciding later, based on some dynamic condition, whether or not you need to run the 2nd query. If, however, there's no question that you'll need to access all the related items, use with().
The alternative to either of these would be looping through the initial result set and querying a hasMany() relation for each item. This would end up running n+1 queries, or 6 in this example. Eager loading, regardless of whether it's done up-front with with() or later with load(), only runs 2 queries.
As #damiani said, Both accomplish the same end results—eager loading a related model onto the first. In fact, they both run exactly the same two queries. The key difference is that with() eager loads the related model up front, immediately after the initial query (all(), first(), or find(x), for example); when using load(), you run the initial query first, and then eager load the relation at some later point.
There is one more difference between With() & load(), you can put the conditions when using with() but you can't do the same in case of load()
For example:
ProductCategory::with('children')
->with(['products' => function ($q) use($SpecificID) {
$q->whereHas('types', function($q) use($SpecificID) {
$q->where('types.id', $SpecificID)
});
}])
->get();
#damiani Explanied difference between load() and with() as well but he said load() is not cacheable so I wanna say couple words about it.
Let assume we have a blog post and related with comments. And we're fetching together and caching it.
$post = Cache::remember("post.".$slug,720,function()use($slug){
return Post::whereSlug($slug)->with("comments")->first();
});
But if there is a new comment and we want to display it immediately, we have to clear post cache and fetch post and comments together again. And that causes unnecessary queries. Lets think there are another queries for tags, media, contributors of the post etc. it will increase amount of resource usage..
public function load($relations)
{
$query = $this->newQueryWithoutRelationships()->with(
is_string($relations) ? func_get_args() : $relations
);
$query->eagerLoadRelations([$this]);
return $this;
}
As you can see above when we use the method it loads given relation and returns model with fetched relation. So you can return it outside of a callback.
$post = Cache::remember("post.".$slug,720,function()use($slug){
return Post::whereSlug($slug)->first();
});
$post = Cache::remember("post.relation.images.".$slug,720,function()use($post){
return $post->load("images");
});
$post = Cache::remember("post.relation.comments".$slug,720,function()use($post){
return $post->load("comments");
});
So if we load them seperatly, next time when some of them updated all you need to do clear specific relation cache and fetch it again. No need to fetch post, tags, images etc. over and over.