In my model I am appending a SUM calculation of a relationship which causes 80 queries to be run for the projectHours attributes.
If I don't append projectHours I get 6 queries.
I believe this is N+1 issue within the model for a relationship.
Is there some way to use eager loading within the model to reduce my queries?
Or should I be going about this a different way? I was suggested to refactor this into a Resource and wrapping the eager-load query in a scope so you could do this in your controller, but I thought resources were more for API endpoints.
Appreciated the help.
class Project extends Model
{
protected $appends = ['projectHours'];
public function jobs()
{
return $this->hasMany('App\JobItem', 'project_id', 'id');
}
public function getProjectHoursAttribute()
{
return $this->jobs()->sum('hours');
}
}
So your (N + 1) is coming from right here:
$this->jobs()->sum('hours');
It is caused accessing by your jobs relationship as a query builder instance jobs()
If you want to preload the relationship and then sum the results, you can do it like this:
$this->jobs->sum('hours');
This then uses the Eloquent Collection sum method
Example
$project = Project::with('jobs')->find(1);
$hours = $project->projectHours;
Related
I got different result from getReward1 and getReward2:
Model:
class User extends Authenticatable
{
public function Products()
{
return $this->hasMany('App\Product', 'user_id');
}
public function getReward1()
{
return $this
->Products
->where('reward', '>', 0)
->where('status', 0)
->sum('reward'); // sum = 7,690,000
}
public function getReward2()
{
return $this
->Products()
->where('reward', '>', 0)
->where('status', 0)
->sum('reward'); // sum = 7,470,000
}
}
getReward1 return 7,690,000 and getReward2 return 7,470,000 (Two different values)
What is difference between $this->Products and $this->Products() ?
$this->products;
// Returns a Collection
$this->products();
// Returns a Relation instance, which is a query builder and can be of type HasMany, BelongsTo...
$this->products()->get();
// Is EXACTLY like doing $this->products for the first time.
The main difference is that products() is just a query that hasn't been executed yet, whereas products are the actual results of this query.
Honestly, even if the name is the same and can be confusing, there are no other similarities between them.
A simple analogy:
DB::table('products')->where('user_id', 18); //could be the $user->products()
DB::table('products')->where('user_id', 18)->get(); //could be $user->products
It's just an analogy, it's not exactly like this internally, but you get the point.
To add more confusion on top of it, Collection methods are ofter similar to those you find in queries; both have where(), first()...
The main thing to remember is that with parentheses, you are still building a query. Until you call get or first, you remain in a query builder.
Without, you already have your results, you are in a Collection (https://laravel.com/docs/8.x/collections).
About the difference you get between getReward1 and getReward2, it's hard to tell exactly what's happening without seeing your database structure.
It can be a lot of things, but when you are calling the sum method, you are calling it on a Collection instance in getReward1 and on a query builder in getReward2 (you are actually executing a query with SELECT SUM(reward)...).
$this->Products() will return an instance of the query builder. The subsequence where clauses will constrain the DB query and then return only the product that you want. These will not be stored in the model instance.
$this->Products will get all of the products from the DB and store them in the model instance as an Eloquent Collection. The subsequent where clauses will then be performed on the Eloquent Collection.
Essentially, the method is doing everything in the DB, whereas, the property is fetching all of the rows and then limiting it with PHP.
I need to get counts of all the records based on belongsToMany relationship. normally I can use groupBy() in a function inside the model. but if I use count() or withCount() inside a model function, i get the error as followed:
function code:
public function TaskCount(){
return $this->belongsToMany(User::class)->count();
}
Error message:
Symfony\Component\Debug\Exception\FatalThrowableError: Call to a member function addEagerConstraints() on int in file /Users/dragonar/Dev/iyw/vendor/laravel/framework/src/Illuminate/Database/Eloquent/Builder.php on line 560
If I do the following...
public function TaskCount(){
return $this->belongsToMany(User::class)->Count();
}
//expected record is 4(int)
//output is 4(array) user records.
...it gives me data but like 4 records of the user instead of a number 4. The user data is useless. The only thing needed is totalCount for those records.
Relationship methods have to return Relation type objects. You are returning the result of a query, count() returns a number not the Relation object / Builder. Remove the count from that statement you are returning. Renamed the relationship tasks here.
public function tasks()
{
return $this->belongsToMany(User::class);
// this returns a Relation type object a BelongsToMany object
}
Where you need to use that relationship you can then use count:
$something->tasks()->count();
Or you can load the count of the relationship using loadCount:
$something->loadCount('tasks');
$something->tasks_count;
Or via eager loading for a collection:
$results = Something::withCount('tasks')->get();
foreach ($results as $model) {
echo $model->tasks_count;
}
If you really wanted to you could create an accessor to get the count as well, you just may want to avoid the N+1 issue by preloading the relationship and using the dynamic property to access it in the accessor.
These relation objects are Builders. When you called groupBy on it previously that is returning the Builder, it isn't executing the query. You can add where conditions and order by statements because they are just building the query, not executing it, they return the builder you are calling the method on.
Laravel 6.x Docs - Eloquent - Relationships - Counting Related Models withCount loadCount
Why not use: Task::all()->count(); ?
you can use the withCount method while calling relation like this
User::withCount('images')->get();
You can add get the data and just count it.
public function TaskCount(){
return $this->belongsToMany(User::class)->get()->count();
}
You can call it like
$taskCount = $task->TaskCount();
I have various parent/child relationships, drilling down a few levels. What I want to know is if its possible to do something like this:
$student = Student::find(1);
$student->bursaries()->enrolments()->courses()->where('course','LIKE','%B%');
(With the end goal of selecting the course which is like '%B%'), or if I would have to instead use the DB Query builder with joins?
Models / Relationships
Student:
public function bursaries() {
return $this->hasMany('App\StudentBursary');
}
StudentBursary:
public function enrolments() {
return $this->hasMany('App\StudentBursaryEnrolment');
}
If what you want is to query all courses, from all enrollments, from all bursaries, from a students, then, unfortunately, you are one table too many from getting by with the Has Many Through relationship, because it supports only 3 tables.
Online, you'll find packages that you can import / or answers that you can follow to provide you more though of solutions, for example:
1) How to use Laravel's hasManyThrough across 4 tables
2) https://github.com/staudenmeir/eloquent-has-many-deep
Anyhow, bellow's something you can do to achieve that with Laravel alone:
// Eager loads bursaries, enrolments and courses, but, condition only courses.
$student = Student::with(['bursaries.enrolments.courses' => function($query) {
$query->where('course','LIKE','%B%');
}])->find(1);
$enrolments = collect();
foreach($student->bursaries as $bursary) {
$enrolments = $enrolments->merge($bursary->enrolments);
}
$courses = collect();
foreach ($enrolments as $enrolment) {
$courses = $courses->merge($enrolment->courses);
}
When you do $student->bursaries() instead of $student->bursaries, it returns a query builder instead of relationship map. So to go to enrolments() from bursaries() you need to do a bursaries()->get(). It should look like this.
$student->bursaries()->get()[0]->enrolments(), added the [0] because im using get(), you can use first() to avoid the [0]
$student->bursaries()->first()->enrolments()
But I'm not sure if it will suffice your requirement or not.
after some digging I still could not find any solid way to retrieve the inverse of a many-to-many polymorphic relation that allows mixed models results.
Please consider the following:
I have several models that can be "tagged". While it is trivial to retrieve for example $item->tags, $article->tags and the inverse with $tag->articles and $tag->items I have no easy way to do something like $tag->taggables to return both articles and items in the same collection. Things get even bumpier as I need to use pagination/simple pagination to the query.
I have tried a few workarounds but the best I could put together still looks crappy and limited. Basically:
I queried the DB once per "taggable";
put all in a single big collection;
passed the collection to a phpleague/fractal transformer (my API uses it) that returns different json values depending on the parsed models.
The limits of this approach is that building a pagination is a nightmare and fractal "include" options can't be used out of the box.
Can anyone help me? I'm currently using Laravel 5.1.
There is not much magic in my current code. Faking and simplifying it to make it short:
From the api controller:
$tag = Tag::findOrDie($tid);
$articles = $tag->cms_articles()->get();
$categories = $tag->cms_categories()->get();
$items = $tag->items()->simplePaginate($itemsperpage);
$taggables = Collection::make($articles)->merge($categories);
// Push items one by one as pagination would dirt the collection struct.
foreach ($items as $item) {
$taggables->push($item);
}
return $this->respondWithCollection($taggables, new TaggableTransformer);
Note: using simplePaginate() is there only because I would like all articles and categories to be shown on first page load while the number of items are so many that need pagination.
From the Transformer class:
public function transform($taggable)
{
switch (get_class($taggable)) {
case 'App\Item':
$transformer = new ItemTransformer;
break;
case 'App\CmsArticle':
$transformer = new CmsArticleTransformer;
break;
case 'App\CmsCategory':
$transformer = new CmsCategoryTransformer;
break;
}
return $transformer->transform($taggable);
}
Please consider that the other transformers are simply returning arrays of data about the models they correlate with. If you use Fractal you would easily spot that nested "included" models would not be applied.
Nothing fancy for the Tag model:
class Tag extends Model
{
protected $morphClass = 'Tag';
protected $fillable = array('name', 'language_id');
public function cms_articles() {
return $this->morphedByMany('App\CmsArticle', 'taggable');
}
public function cms_categories() {
return $this->morphedByMany('App\CmsCategory', 'taggable');
}
public function items() {
return $this->morphedByMany('App\Item', 'taggable');
}
// Would love something like this to return inverse relation!! :'(
public function taggables() {
return $this->morphTo();
}
}
I am also considering the option to do 3 separate calls to the API to retrieve articles, categories and items in three steps. While in this particular scenario this might make sense after all, I would still need to deal with this particular inverse relation headache with another part of my project: notifications. In this particular case, notifications would have to relate to many different actions/models and I would have to retrieve them all in batches (paginated) and sorted by model creation date...
Hope this all makes sense. I wonder if a completely different approach to the whole inverse "polymorphic" matter would help.
Kind regards,
Federico
Ah yes. I was down your path not all that long ago. I had the same nightmare of dealing with resolving the inverse of the relationship of polymorphic relationships.
Unfortunately polymorphic relationships haven't been given much attention in the Laravel ecosystem. From afar they look like unicorns and rainbows but soon you're fighting things like this.
Can you post an example of a $thing->taggable for a better picture? Think it may be solvable with a dynamic trait + accessor magic.
In Laravel 4; I have model Project and Part, they have a many-to-many relationship with a pivot table project_part. The pivot table has a column count which contains the number of a part ID used on a project, e.g.:
id project_id part_id count
24 6 230 3
Here the project_id 6, is using 3 pieces of part_id 230.
One part may be listed multiple times for the same project, e.g.:
id project_id part_id count
24 6 230 3
92 6 230 1
When I show a parts list for my project I do not want to show part_id twice, so i group the results.
My Projects model has this:
public function parts()
{
return $this->belongsToMany('Part', 'project_part', 'project_id', 'part_id')
->withPivot('count')
->withTimestamps()
->groupBy('pivot_part_id')
}
But of course my count value is not correct, and here comes my problem: How do I get the sum of all grouped parts for a project?
Meaning that my parts list for project_id 6 should look like:
part_id count
230 4
I would really like to have it in the Projects-Parts relationship so I can eager load it.
I can not wrap my head around how to do this without getting the N+1 problem, any insight is appreciated.
Update: As a temporary work-around I have created a presenter method to get the total part count in a project. But this is giving me the N+1 issue.
public function sumPart($project_id)
{
$parts = DB::table('project_part')
->where('project_id', $project_id)
->where('part_id', $this->id)
->sum('count');
return $parts;
}
Try to sum in Collection,
$project->parts->sum('pivot.count');
This is best way I found. It's clean (easy to read) and able to re-use all of your scope, ordering and relation attribute caching in parts many-to-many defination.
#hebron No N+1 problem for this solution if you use with('parts') to eager load. Because $project->parts (without funtion call) is a cached attribute, return a instance of Collection with all your data. And sum('pivot.count') is a method of Collection which contains pure funcional helpers (not relative to database, like underscore in js world).
Full example:
Definition of relation parts:
class Project extends Model
{
public function parts()
{
return $this->belongsToMany('Part', 'project_part', 'project_id', 'part_id')
->withPivot('count')
->withTimestamps();
}
}
When you use it (note that eager load is important to avoid N+1 problem),
App\Project::with('parts')->get()->each(function ($project) {
dump($project->parts->sum('pivot.count'));
});
Or you can define the sum function in Project.php,
class Project extends Model
{
...
/**
* Get parts count.
*
* #return integer
*/
public function partsCount()
{
return $this->parts->sum('pivot.count');
}
}
If you want to avoid with('parts') on caller side (eager load parts by default), you can add a $with attribute
class Project extends Model
{
/**
* The relations to eager load on every query.
*
* #var array
*/
protected $with = ['parts'];
...
}
From the code source:
We need to alias all of the pivot columns with the "pivot_" prefix so we can easily extract them out of the models and put them into the pivot relationships when they are retrieved and hydrated into the models.
So you can do the same with select method
public function parts()
{
return $this->belongsToMany('Part', 'project_part', 'project_id', 'part_id')
->selectRaw('parts.*, sum(project_part.count) as pivot_count')
->withTimestamps()
->groupBy('project_part.pivot_part_id')
}
The best way that you can use is:
$project->parts->sum('pivot.count');
I faced the same problem, but this solved my issue.