I have a Batch model, which hasMany Results and belongsTo a Project. The current status of a batch is based on the status of its most recent Result. So, in my batch model I have this:
public function allForProject($pid)
{
$batches = $this
->with(static::$relatedObjects)
->with('current_status')
->where('project_id', '=', $pid)
->get();
return $batches;
}
public function current_status()
{
return $this
->belongsToMany('BehatEditor\Models\Result')
->orderBy('created_at', 'DESC')
->limit(1)
;
}
...So by saying "->with('current_status')" I am trying to eager load only the most recent result for that batch - there may be thousands of them per batch that I do not want to return to the front end.
Now, this doesn't break, but the "limit(1)" actually seems to limit the number of Batches that get returned with a Result. Even though each one of my Batches has 2 results with my test data, when I use limit(1) only one of them comes back with any data. When I use limit(2), only 2 batches come back with a single current_status record (desired) but the rest have an empty array for current_status.
This is a Silex project using Eloquent as an ORM, so Laravel specific methods won't work.
Any help is much appreciated!
UPDATE:
It looks like Eloquent just doesn't support this. see http://irclogs.julien-c.fr/2013-12-19/01:48#log-52b25061a599aafb54008650. I would like to update my question to be how can I cleanly add the raw SQL I need to my query? Can I supply my own method that holds only the SQL needed, or do I need to replace all ORM usage in allForProject()?
Instead of ->limit(1), use ->first().
Update: Misread what you said, this should work.
Related
I really tried to understand the difference between the with() method and the load() method, but couldn't really understand.
As I see it, using the with() method is "better" since I eager load the relation. It seems that if I use load() I load the relation just as if I would use the hasMany() (or any other method that relates to the relation between objects).
Do I get it wrong?
Both accomplish the same end results—eager loading a related model onto the first. In fact, they both run exactly the same two queries. The key difference is that with() eager loads the related model up front, immediately after the initial query (all(), first(), or find(x), for example); when using load(), you run the initial query first, and then eager load the relation at some later point.
"Eager" here means that we're associating all the related models for a particular result set using just one query, as opposed to having to run n queries, where n is the number of items in the initial set.
Eager loading using with()
If we eager load using with(), for example:
$users = User::with('comments')->get();
...if we have 5 users, the following two queries get run immediately:
select * from `users`
select * from `comments` where `comments`.`user_id` in (1, 2, 3, 4, 5)
...and we end up with a collection of models that have the comments attached to the user model, so we can do something like $users->comments->first()->body.
"Lazy" eager loading using load()
Or, we can separate the two queries, first by getting the initial result:
$users = User::all();
which runs:
select * from `users`
And later, if we decide that we need the related comments for all these users, we can eager load them after the fact:
$users = $users->load('comments');
which runs the 2nd query:
select * from `comments` where `comments`.`user_id` in (1, 2, 3, 4, 5)
...and we end up with the same result, just split into two steps. Again, we can call $users->comments->first()->body to get to the related model for any item.
Why use load() vs. with()? load() gives you the option of deciding later, based on some dynamic condition, whether or not you need to run the 2nd query. If, however, there's no question that you'll need to access all the related items, use with().
The alternative to either of these would be looping through the initial result set and querying a hasMany() relation for each item. This would end up running n+1 queries, or 6 in this example. Eager loading, regardless of whether it's done up-front with with() or later with load(), only runs 2 queries.
As #damiani said, Both accomplish the same end results—eager loading a related model onto the first. In fact, they both run exactly the same two queries. The key difference is that with() eager loads the related model up front, immediately after the initial query (all(), first(), or find(x), for example); when using load(), you run the initial query first, and then eager load the relation at some later point.
There is one more difference between With() & load(), you can put the conditions when using with() but you can't do the same in case of load()
For example:
ProductCategory::with('children')
->with(['products' => function ($q) use($SpecificID) {
$q->whereHas('types', function($q) use($SpecificID) {
$q->where('types.id', $SpecificID)
});
}])
->get();
#damiani Explanied difference between load() and with() as well but he said load() is not cacheable so I wanna say couple words about it.
Let assume we have a blog post and related with comments. And we're fetching together and caching it.
$post = Cache::remember("post.".$slug,720,function()use($slug){
return Post::whereSlug($slug)->with("comments")->first();
});
But if there is a new comment and we want to display it immediately, we have to clear post cache and fetch post and comments together again. And that causes unnecessary queries. Lets think there are another queries for tags, media, contributors of the post etc. it will increase amount of resource usage..
public function load($relations)
{
$query = $this->newQueryWithoutRelationships()->with(
is_string($relations) ? func_get_args() : $relations
);
$query->eagerLoadRelations([$this]);
return $this;
}
As you can see above when we use the method it loads given relation and returns model with fetched relation. So you can return it outside of a callback.
$post = Cache::remember("post.".$slug,720,function()use($slug){
return Post::whereSlug($slug)->first();
});
$post = Cache::remember("post.relation.images.".$slug,720,function()use($post){
return $post->load("images");
});
$post = Cache::remember("post.relation.comments".$slug,720,function()use($post){
return $post->load("comments");
});
So if we load them seperatly, next time when some of them updated all you need to do clear specific relation cache and fetch it again. No need to fetch post, tags, images etc. over and over.
I have a basic query set up in the show method of a laravel resource
public function show($id){
$results = Student::find($id);
$drives= Drive:: where('student_id', $id);
}
The query for $results works perfectly. The query for $drives does not work unless I do ->get() at the end of it. Why is this? what's the difference between the two queries so that one requires the ->get() and the other does not? Solving this problem took me like 5 hrs and i'm just curious as to the functionality behind it so i can avoid this headache in the future.
Some eloquent expressions have a get implicitly. Those ones who are made by a Query Builder will need a ->get() call, find(), findOne()... won't need a get().
https://laravel.com/docs/5.6/eloquent#retrieving-models
https://laravel.com/docs/5.6/queries
use get to execute a builder query. unless you run the get() query wont be executed. get will return a collection.
1 - Use query builder to build queries however you want.
$drives= Drive:: where('student_id', $id);
dd($drives); // will return a query builder, you can use it to build query by chaining
2 - when you are ready to execute the query call get()
$drives= Drive:: where('student_id', $id);
$result = $drives->get()
dd($result); // will return a database query result set as a collection object
If you want to get a single object by id use find, to get a single object
$results = Student::find($id);
dd($result); will return a single model
Using the function find() on a model gets a query result based on the primary key of the model, id in this case.
When using where(), it gets a collection (an object of all query results), so if you only want the first result you must call $drives=Drive::where('student_id', $id)->first();
Here is a more in-depth explanation: the difference of find and get in Eloquent
We are developing an API with LUMEN.
Today we had a confused problem with getting the collection of our "TimeLog"-model.
We just wanted to get all time logs with additional informationen from the board model and task model.
In one row of time log we had a board_id and a task_id. It is a 1:1 relation on both.
This was our first code for getting the whole data. This took a lot of time and sometimes we got a timeout:
BillingController.php
public function byYear() {
$timeLog = TimeLog::get();
$resp = array();
foreach($timeLog->toArray() as $key => $value) {
if(($timeLog[$key]->board_id && $timeLog[$key]->task_id) > 0 ) {
array_push($resp, array(
'board_title' => isset($timeLog[$key]->board->title) ? $timeLog[$key]->board->title : null,
'task_title' => isset($timeLog[$key]->task->title) ? $timeLog[$key]->task->title : null,
'id' => $timeLog[$key]->id
));
}
}
return response()->json($resp);
}
The TimeLog.php where the relation has been made.
public function board()
{
return $this->belongsTo('App\Board', 'board_id', 'id');
}
public function task()
{
return $this->belongsTo('App\Task', 'task_id', 'id');
}
Our new way is like this:
BillingController.php
public function byYear() {
$timeLog = TimeLog::
join('oc_boards', 'oc_boards.id', '=', 'oc_time_logs.board_id')
->join('oc_tasks', 'oc_tasks.id', '=', 'oc_time_logs.task_id')
->join('oc_users', 'oc_users.id', '=', 'oc_time_logs.user_id')
->select('oc_boards.title AS board_title', 'oc_tasks.title AS task_title','oc_time_logs.id','oc_time_logs.time_used_sec','oc_users.id AS user_id')
->getQuery()
->get();
return response()->json($timeLog);
}
We deleted the relation in TimeLog.php, cause we don't need it anymore. Now we have a load time about 1 sec, which is fine!
There are about 20k entries in the time log table.
My questions are:
Why is the first method out of range (what causes the timeout?)
What does getQuery(); exactly do?
If you need more information just ask me.
--First Question--
One of the issues you might be facing is having all those huge amount of data in memory, i.e:
$timeLog = TimeLog::get();
This is already enormous. Then when you are trying to convert the collection to array:
There is a loop through the collection.
Using the $timeLog->toArray() while initializing the loop based on my understanding is not efficient (I might not be entirely correct about this though)
Thousands of queries are made to retrieve the related models
So what I would propose are five methods (one which saves you from hundreds of query), and the last which is efficient in returning the result as customized:
Since you have many data, then chunk the result ref: Laravel chunk so you have this instead:
$timeLog = TimeLog::chunk(1000, function($logs){
foreach ($logs as $log) {
// Do the stuff here
}
});
Other way is using cursor (runs only one query where the conditions match) the internal operation of cursor as understood is using Generators.
foreach (TimeLog::where([['board_id','>',0],['task_id', '>', 0]])->cursor() as $timelog) {
//do the other stuffs here
}
This looks like the first but instead you have already narrowed your query down to what you need:
TimeLog::where([['board_id','>',0],['task_id', '>', 0]])->get()
Eager Loading would already present the relationship you need on the fly but might lead to more data in memory too. So possibly the chunk method would make things more easier to manage (even though you eagerload related models)
TimeLog::with(['board','task'], function ($query) {
$query->where([['board_id','>',0],['task_id', '>', 0]]);
}])->get();
You can simply use Transformer
With transformer, you can load related model, in elegant, clean and more controlled methods even if the size is huge, and one greater benefit is you can transform the result without having to worry about how to loop round it
You can simply refer to this answer in order to perform a simple use of it. However incase you don't need to transform your response then you can take other options.
Although this might not entirely solve the problem, but because the main issues you face is based on memory management, so the above methods should be useful.
--Second question--
Based on Laravel API here You could see that:
It simply returns the underlying query builder instance. To my observation, it is not needed based on your example.
UPDATE
For question 1, since it seems you want to simply return the result as response, truthfully, its more efficient to paginate this result. Laravel offers pagination The easiest of which is SimplePaginate which is good. The only thing is that it makes some few more queries on the database, but keeps a check on the last index; I guess it uses cursor as well but not sure. I guess finally this might be more ideal, having:
return TimeLog::paginate(1000);
I have faced a similar problem. The main issue here is that Elloquent is really slow doing massive task cause it fetch all the results at the same time so the short answer would be to fetch it row by row using PDO fetch.
Short example:
$db = DB::connection()->getPdo();
$query_sql = TimeLog::join('oc_boards', 'oc_boards.id', '=', 'oc_time_logs.board_id')
->join('oc_tasks', 'oc_tasks.id', '=', 'oc_time_logs.task_id')
->join('oc_users', 'oc_users.id', '=', 'oc_time_logs.user_id')
->select('oc_boards.title AS board_title', 'oc_tasks.title AS task_title','oc_time_logs.id','oc_time_logs.time_used_sec','oc_users.id AS user_id')
->toSql();
$query = $db->prepare($query->sql);
$query->execute();
$logs = array();
while ($log = $query->fetch()) {
$log_filled = new TimeLog();
//fill your model and push it into an array to parse it to json in future
array_push($logs,$log_filled);
}
return response()->json($logs);
I really tried to understand the difference between the with() method and the load() method, but couldn't really understand.
As I see it, using the with() method is "better" since I eager load the relation. It seems that if I use load() I load the relation just as if I would use the hasMany() (or any other method that relates to the relation between objects).
Do I get it wrong?
Both accomplish the same end results—eager loading a related model onto the first. In fact, they both run exactly the same two queries. The key difference is that with() eager loads the related model up front, immediately after the initial query (all(), first(), or find(x), for example); when using load(), you run the initial query first, and then eager load the relation at some later point.
"Eager" here means that we're associating all the related models for a particular result set using just one query, as opposed to having to run n queries, where n is the number of items in the initial set.
Eager loading using with()
If we eager load using with(), for example:
$users = User::with('comments')->get();
...if we have 5 users, the following two queries get run immediately:
select * from `users`
select * from `comments` where `comments`.`user_id` in (1, 2, 3, 4, 5)
...and we end up with a collection of models that have the comments attached to the user model, so we can do something like $users->comments->first()->body.
"Lazy" eager loading using load()
Or, we can separate the two queries, first by getting the initial result:
$users = User::all();
which runs:
select * from `users`
And later, if we decide that we need the related comments for all these users, we can eager load them after the fact:
$users = $users->load('comments');
which runs the 2nd query:
select * from `comments` where `comments`.`user_id` in (1, 2, 3, 4, 5)
...and we end up with the same result, just split into two steps. Again, we can call $users->comments->first()->body to get to the related model for any item.
Why use load() vs. with()? load() gives you the option of deciding later, based on some dynamic condition, whether or not you need to run the 2nd query. If, however, there's no question that you'll need to access all the related items, use with().
The alternative to either of these would be looping through the initial result set and querying a hasMany() relation for each item. This would end up running n+1 queries, or 6 in this example. Eager loading, regardless of whether it's done up-front with with() or later with load(), only runs 2 queries.
As #damiani said, Both accomplish the same end results—eager loading a related model onto the first. In fact, they both run exactly the same two queries. The key difference is that with() eager loads the related model up front, immediately after the initial query (all(), first(), or find(x), for example); when using load(), you run the initial query first, and then eager load the relation at some later point.
There is one more difference between With() & load(), you can put the conditions when using with() but you can't do the same in case of load()
For example:
ProductCategory::with('children')
->with(['products' => function ($q) use($SpecificID) {
$q->whereHas('types', function($q) use($SpecificID) {
$q->where('types.id', $SpecificID)
});
}])
->get();
#damiani Explanied difference between load() and with() as well but he said load() is not cacheable so I wanna say couple words about it.
Let assume we have a blog post and related with comments. And we're fetching together and caching it.
$post = Cache::remember("post.".$slug,720,function()use($slug){
return Post::whereSlug($slug)->with("comments")->first();
});
But if there is a new comment and we want to display it immediately, we have to clear post cache and fetch post and comments together again. And that causes unnecessary queries. Lets think there are another queries for tags, media, contributors of the post etc. it will increase amount of resource usage..
public function load($relations)
{
$query = $this->newQueryWithoutRelationships()->with(
is_string($relations) ? func_get_args() : $relations
);
$query->eagerLoadRelations([$this]);
return $this;
}
As you can see above when we use the method it loads given relation and returns model with fetched relation. So you can return it outside of a callback.
$post = Cache::remember("post.".$slug,720,function()use($slug){
return Post::whereSlug($slug)->first();
});
$post = Cache::remember("post.relation.images.".$slug,720,function()use($post){
return $post->load("images");
});
$post = Cache::remember("post.relation.comments".$slug,720,function()use($post){
return $post->load("comments");
});
So if we load them seperatly, next time when some of them updated all you need to do clear specific relation cache and fetch it again. No need to fetch post, tags, images etc. over and over.
I'm working in Laravel 4, and I have a Child model with multiple EducationProfiles:
class Child extends EloquentVersioned
{
public function educationProfiles()
{
return $this->hasMany('EducationProfile');
}
}
If I wanted to get all the EducationProfiles for each kid under age 10 it would be easy:
Child::where('date_of_birth','>','2004-03-27')->with('educationProfiles')->all();
But say (as I do) that I would like to use with() to grab a calculated value for the Education Profiles of each of those kids, something like:
SELECT `education_profiles`.`child_id`, GROUP_CONCAT(`education_profiles`.`district`) as `district_list`
In theory with() only works with relationships, so do I have any options for associating the district_list fields to my Child models?
EDIT: Actually, I was wondering whether with('educationProfiles') generates SQL equivalent to:
EducationProfile::whereIn('id',array(1,2,3,4))
or whether it's actually equivalent to
DB::table('education_profiles')->whereIn('id',array(1,2,3,4))
The reason I ask is that in the former I'm getting models, if it's the latter I'm getting unmodeled data, and thus I can probably mess it up as much as I want. I assume with() generates an additional set models, though. Anybody care to correct or confirm?
Ok, I think I've cracked this nut. No, it is NOT possible to eager load arbitrary queries. However, the tools have been provided by the Fluent query builder to make it relatively easy to replicate eager loading manually.
First, we leverage the original query:
$query = Child::where('date_of_birth','>','2004-03-27')->with('educationProfiles');
$children = $query->get();
$eagerIds = $query->lists('id');
Next, use the $eagerIds to filterDB::table('education_profile') in the same way that with('educationProfiles') would filter EducationProfile::...
$query2 = DB::table('education_profile')->whereIn('child_id',$eagerIds)->select('child_id', 'GROUP_CONCAT(`education_profiles`.`district`) as `district_list`')->groupBy('child_id');
$educationProfiles = $query2->lists('district_list','child_id');
Now we can iterate through $children and just look up the $educationProfiles[$children->id] values for each entry.
Ok, yes, it's an obvious construction, but I haven't seen it laid out explicitly anywhere before as a means of eager loading arbitrary calculations.
You can add a where clause to your hasMany() call like this:
public function educationProfilesUnderTen() {
$ten_years_ago = (new DateTime('10 years ago'))->format('Y-m-d');
return $this->hasMany('EducationProfile')->where('date_of_birth', '>', $ten_years_ago)
}