I've got a Symfony entity, which has a OneToMany mapping with an OrderBy clause like this:
/**
* #ORM\OneToMany(targetEntity="App\Entity\News", mappedBy="category", orphanRemoval=true)
* #ORM\OrderBy({"id" = "DESC"})
*/
private $news;
Assuming I would like to only display n entries in Twig, I would have the options to either loop over it and ignoring every thing after loop.index n or rather use slice. However these options do have the downside, that if there are a lot of news entries, all of them will be loaded, which isn't very efficient.
Another option would be to use a Criteria in the controller or the entity to limit the amount of loaded entities. If I understood it here correctly, it should modify the doctrine query directly and thus not have any performance impact. Is this the best practice, or would it be better to have a custom query builder in the controller or a function in the repository?
Actually you can set $news relationship as EXTRA_LAZY and use $news->slice() function without triggering a full load as stated in official documentation:
If you mark an association as extra lazy the following methods on
collections can be called without triggering a full load of the
collection:
Collection#contains($entity)
Collection#containsKey($key) (available with Doctrine 2.5)
Collection#count()
Collection#get($key) (available with Doctrine 2.4)
Collection#slice($offset, $length = null)
Therefore your declaration should look like the following:
/**
* #ORM\OneToMany(targetEntity="App\Entity\News", mappedBy="category", orphanRemoval=true, fetch="EXTRA_LAZY")
* #ORM\OrderBy({"id" = "DESC"})
*/
private $news;
Related
Since Laravel Scout doesn't support more complex where clauses than simple numeric comparisons.
I checked the source code and I found the following lines
if (!empty($models = $model->getScoutModelsByIds($builder, $modelKeys))) {
$instances = $instances->merge($models->load($searchable->getRelations($modelClass)));
}
The instances is what is returned from Algolia search, so for example the following search essentially returns the $instances variable.
Mode::search('something')->get();
the $model is the searchable model and the getScoutModelsByIds what It basically does is a query to the database like
public function getScoutModelsByIds(){
$model->whereIn('id', $modelKeys)->get();
}
I was wondering if I apply any kind of where clauses or addSelect, or with eager loading, on the model before actually retrieving the data from the database, is it a good idea ?
For example
$model->where('some condition')->whereIn('id', $modelKeys)->get();
and instead of using lazy loading
$instances = $instances->merge($models->load($searchable->getRelations($modelClass)));
use the with function before retrieving the data from db.
For example
$model->where('some condition')->whereIn('id', $modelKeys)->with('relationships')->get();
Doctrine 2.1 brings a new feature EXTRA_LAZY loading for associations: https://www.doctrine-project.org/projects/doctrine-orm/en/latest/tutorials/extra-lazy-associations.html
This feature creates a new method slice($offset, $length) to query just a page of the association and is very useful for pagination of large data sets.
However, behind the scene the SQL query uses the classic LIMIT XX OFFSET XX syntax which is slow for large data sets (https://www.eversql.com/faster-pagination-in-mysql-why-order-by-with-limit-and-offset-is-slow/)
Is there a way to use the pagination with a WHERE clause?
If not, how may I extend the instance of Doctrine\ORM\PersistentCollection to create a method sliceWithCursor($columnName, $cursor, $length)?
My main goal is to implement a faster pagination while using the very convenient magic of Doctrine for associations.
Thanks !
You can use the matching function of Doctrine\ORM\PersistentCollection, providing the criteria to filter, e.g.:
use Doctrine\Common\Collections\Criteria;
$group = $entityManager->find('Group', $groupId);
$userCollection = $group->getUsers();
$criteria = Criteria::create()
->where(Criteria::expr()->eq("birthday", "1982-02-17"))
->orderBy(array("username" => Criteria::ASC));
$birthdayUsers = $userCollection->matching($criteria);
matching() returns a Doctrine\ORM\LazyCriteriaCollection, if your association is defined as EXTRA_LAZY.
You can paginate with the latter:
$birthdayUsers->slice($offset, $length);
Using cursor pagination
In some cases, it is required to use cursor pagination. You could do this by extending Doctrine\ORM\PersistentCollection, as suggested:
use Doctrine\Common\Collections\Criteria;
public function sliceWithCursor($criteria, $cursorEntity, $limit) {
$orderBy = $criteria->getOrderings();
foreach ($orderBy as $columnName => $direction) {
if ($direction === Criteria::ASC) {
$criteria->andWhere(Criteria::expr()->gte($columnName, $cursorEntity->{$columnName}));
} else {
$criteria->andWhere(Criteria::expr()->lte($columnName, $cursorEntity->{$columnName}));
}
}
// exclude cursor entity from the results
$criteria->andWhere(Criteria::expr()->neq("id", $cursorEntity->id));
$criteria->setMaxResults($limit);
return $this->matching($criteria);
}
The idea of cursor based pagination is to use a result row as starting point, instead of an offset, and get the next rows. As stated at alternative for using OFFSET, the idea is to substitute offset, with conditions from the order by clause.
One of my models contains the following:
public function from()
{
return $this->belongsTo(Station::class, 'from_station_id');
}
public function to()
{
return $this->belongsTo(Station::class, 'to_station_id');
}
In order to use this I'm using the with('to', 'from') method. Which results in the following:
select * from "stations" where "stations"."id" in ('1')
select * from "stations" where "stations"."id" in ('2')
Two cached queries one for "to's" and one for "from's". At the moment with 1 record they are "useful". But in the future they will have a lot of duplicate IDs..
Does Laravel offer an option to combine these?
Assuming you'll need to access them from the model by the relation, like $model->from->first() or $model->to->count(), your best option would be to stick with 2 queries. A query with where in clause is not that heavy and you can additionally cache them to speed up.
I'm working in Laravel 4, and I have a Child model with multiple EducationProfiles:
class Child extends EloquentVersioned
{
public function educationProfiles()
{
return $this->hasMany('EducationProfile');
}
}
If I wanted to get all the EducationProfiles for each kid under age 10 it would be easy:
Child::where('date_of_birth','>','2004-03-27')->with('educationProfiles')->all();
But say (as I do) that I would like to use with() to grab a calculated value for the Education Profiles of each of those kids, something like:
SELECT `education_profiles`.`child_id`, GROUP_CONCAT(`education_profiles`.`district`) as `district_list`
In theory with() only works with relationships, so do I have any options for associating the district_list fields to my Child models?
EDIT: Actually, I was wondering whether with('educationProfiles') generates SQL equivalent to:
EducationProfile::whereIn('id',array(1,2,3,4))
or whether it's actually equivalent to
DB::table('education_profiles')->whereIn('id',array(1,2,3,4))
The reason I ask is that in the former I'm getting models, if it's the latter I'm getting unmodeled data, and thus I can probably mess it up as much as I want. I assume with() generates an additional set models, though. Anybody care to correct or confirm?
Ok, I think I've cracked this nut. No, it is NOT possible to eager load arbitrary queries. However, the tools have been provided by the Fluent query builder to make it relatively easy to replicate eager loading manually.
First, we leverage the original query:
$query = Child::where('date_of_birth','>','2004-03-27')->with('educationProfiles');
$children = $query->get();
$eagerIds = $query->lists('id');
Next, use the $eagerIds to filterDB::table('education_profile') in the same way that with('educationProfiles') would filter EducationProfile::...
$query2 = DB::table('education_profile')->whereIn('child_id',$eagerIds)->select('child_id', 'GROUP_CONCAT(`education_profiles`.`district`) as `district_list`')->groupBy('child_id');
$educationProfiles = $query2->lists('district_list','child_id');
Now we can iterate through $children and just look up the $educationProfiles[$children->id] values for each entry.
Ok, yes, it's an obvious construction, but I haven't seen it laid out explicitly anywhere before as a means of eager loading arbitrary calculations.
You can add a where clause to your hasMany() call like this:
public function educationProfilesUnderTen() {
$ten_years_ago = (new DateTime('10 years ago'))->format('Y-m-d');
return $this->hasMany('EducationProfile')->where('date_of_birth', '>', $ten_years_ago)
}
I'm encountering an issue because I'm sure I'm not doing this correctly with my programming. I have created a custom model in Magento.
In the database table of my model there are several entities with the same attributes...
I need to pick just one from all these entities with the same attribute that I have. For the moment I did this:
$myvariable = Mage::getModel('test/test')->getCollection()
->setOrder('idserialkeys', 'asc')
->addFilter('idproduit', 1)
->addFilter('utilise', 0)
->addFilter('customerid', 0)
->addFilter('numcommande', 0)
From this loading I have around a hundred results but I need to update only one of these, so just after I'm doing:
->setPageSize(1);
The problem is that I need a foreach after to update my entity
foreach($mavaribale as $modifiemoi) {
// Update of my entity because of course there is only one
}
As you can see I'm obliged to do a loop (for each) even if I have a setPagesize... I would like to avoid this loop to optimize my code.
When you have a collection, and you only need one element, use the getFirstItem method. Try this:
$modifiemoi = $myvariable->getFirstItem();
Make sure that you also use your setPageSize call so that you only transfer data for one item.
All collections are Varien_Data_Collection objects so you can use getFirstItem:
$modifiemoi = $mavaribale->getFirstItem();