I have this chunk of code:
//to-do
public function searchVehicles($terms, $offset=1, $order='ASC')
{
if (trim($terms) == '') {
return array();
}
$query = $this->_getQuery($terms);
$query->setStoreId(1);
if ($query->getId()) {
$query->setPopularity($query->getPopularity()+1);
}
else {
$query->setPopularity(1);
}
$query->prepare();
$query->save();
$collection = Mage::getResourceModel('catalog/product_collection');
$collection->getSelect()->joinInner(
array('search_result' => $collection->getTable('catalogsearch/result')),
$collection->getConnection()->quoteInto(
'search_result.product_id=e.entity_id AND search_result.query_id=?',
$query->getId()
),
array('relevance' => 'relevance')
);
$collection->setStore(1);
//Mage::getSingleton('catalog/product_status')->addVisibleFilterToCollection($collection);
//Mage::getSingleton('catalog/product_visibility')->addVisibleInSearchFilterToCollection($collection);
return $this->_listProductCollection($collection, $offset, $order);
}
Which is inside a Resource class and reachable via SOAP.
Before we start: Yes, I remember to do the cache flushing and recompiling process - I clarify because this is an usual issue to newbies like me xDDD.
Now: I can access such method but it returns [].
SPECIAL NOTE: $this->_listProductCollection($collection, $offset, $order); WORKS since i'm using the same method in other collections fetched from other methods in the same resource, and have no trouble at all.
Let me review the intention of my code since I'm a newbie at Magento (I'm using version 1.6.2).
The code is based on the CatalogSearch/ResultController controller's indexAction() method, and tried to learn about it.
An empty query will yield an empty result and will not bother the Magento search engine.
There's only a Store (id = 1) in the site and the search query is created like this:
private function _getQuery($terms)
{
$query = Mage::getModel('catalogsearch/query')->loadByQuery($terms);
if (!$query->getId()) {
$query->setQueryText($terms);
}
return $query;
}
The query increases it's popularity (I took this code from the controller. I assume this is for statistical purposes only).
The query is prepared (I think this means: the MySQL internal query is prepared) so I can fetch it later.
The query is saved - AFAIK this means that the query results are iterated and cached so a subsequent same query will only fetch the stored results instead of processing the search again.
At this point the query will have an ID.
I get the whole Product collection, and join it with the search result table. SEEMS that the results table has - at least (queryId, matchedProductId). I only keep the products having IDs in the matched results, and from store 1.
I list the products.
Note that the filters are currently commented.
However, the returned list is [] (an empty list) when I hit this API entry point, althought searching in the usual search bar gives me the expected result.
Question: What am I missing? What did I misunderstood in the process?
Related
This is more a "best practice" question.
My website has articles, and articles can have comments.
public function comments() {
return $this->hasMany('App\Comment', 'submission_id');
}
On the front page, where the articles are listed, the amount of comments are shown:
#if ($article->comments->count())
{{ $article->comments->count() }}
{{ $article->comments->count() == 1 ? 'comment' : 'comments' }}
#endif
And I was thinking to myself, in this example, is my database being queried 3 times for this simple code snippet?
Every time I get the comment count through this relationship, is it loading the entire array of comments and all of its columns? Because sometimes articles can have upwards of 1k+ comments.
If so, would it not be best practice to simply have a column in my articles database table that increments every time a comment is posted and then fetching that directly rather than through the relationship?
Not the way you're doing it. It depens on when you're using ->count(), and whether or not it's the Collection version, or the Builder version. See the following examples:
Example one - new query with comments() method:
$article = Article::first();
$article->comments()->count();
In this example, using $article->comments() causes a new query to be started (using the Builder class). No matter how you finish this query, i.e. using ->first(), ->get() or ->count() a new query will be executed.
Now, example two - new query with comments property:
$article = Article::first();
$article->comments->count();
In this example, $article->comments will be executing a new query, as relationship accessors act differently if the relationship has been loaded or not. In this case, it hasn't been loaded, so $article->comments returns a Builder instance instead of a Collection, so finishing the query will execute a new query.
Finally, example three - Collection access with comments property:
$article = Article::with(["comments"])->first();
$article->comments->count();
In this last example, $article->comments has been specified as loaded via the ->with(["comments"]) clause, so it is a Collection, and not a Builder instance. For this reason, calling ->count() is using the Collection count method, and does not perform an additional query
Doctrine 2.1 brings a new feature EXTRA_LAZY loading for associations: https://www.doctrine-project.org/projects/doctrine-orm/en/latest/tutorials/extra-lazy-associations.html
This feature creates a new method slice($offset, $length) to query just a page of the association and is very useful for pagination of large data sets.
However, behind the scene the SQL query uses the classic LIMIT XX OFFSET XX syntax which is slow for large data sets (https://www.eversql.com/faster-pagination-in-mysql-why-order-by-with-limit-and-offset-is-slow/)
Is there a way to use the pagination with a WHERE clause?
If not, how may I extend the instance of Doctrine\ORM\PersistentCollection to create a method sliceWithCursor($columnName, $cursor, $length)?
My main goal is to implement a faster pagination while using the very convenient magic of Doctrine for associations.
Thanks !
You can use the matching function of Doctrine\ORM\PersistentCollection, providing the criteria to filter, e.g.:
use Doctrine\Common\Collections\Criteria;
$group = $entityManager->find('Group', $groupId);
$userCollection = $group->getUsers();
$criteria = Criteria::create()
->where(Criteria::expr()->eq("birthday", "1982-02-17"))
->orderBy(array("username" => Criteria::ASC));
$birthdayUsers = $userCollection->matching($criteria);
matching() returns a Doctrine\ORM\LazyCriteriaCollection, if your association is defined as EXTRA_LAZY.
You can paginate with the latter:
$birthdayUsers->slice($offset, $length);
Using cursor pagination
In some cases, it is required to use cursor pagination. You could do this by extending Doctrine\ORM\PersistentCollection, as suggested:
use Doctrine\Common\Collections\Criteria;
public function sliceWithCursor($criteria, $cursorEntity, $limit) {
$orderBy = $criteria->getOrderings();
foreach ($orderBy as $columnName => $direction) {
if ($direction === Criteria::ASC) {
$criteria->andWhere(Criteria::expr()->gte($columnName, $cursorEntity->{$columnName}));
} else {
$criteria->andWhere(Criteria::expr()->lte($columnName, $cursorEntity->{$columnName}));
}
}
// exclude cursor entity from the results
$criteria->andWhere(Criteria::expr()->neq("id", $cursorEntity->id));
$criteria->setMaxResults($limit);
return $this->matching($criteria);
}
The idea of cursor based pagination is to use a result row as starting point, instead of an offset, and get the next rows. As stated at alternative for using OFFSET, the idea is to substitute offset, with conditions from the order by clause.
We are developing an API with LUMEN.
Today we had a confused problem with getting the collection of our "TimeLog"-model.
We just wanted to get all time logs with additional informationen from the board model and task model.
In one row of time log we had a board_id and a task_id. It is a 1:1 relation on both.
This was our first code for getting the whole data. This took a lot of time and sometimes we got a timeout:
BillingController.php
public function byYear() {
$timeLog = TimeLog::get();
$resp = array();
foreach($timeLog->toArray() as $key => $value) {
if(($timeLog[$key]->board_id && $timeLog[$key]->task_id) > 0 ) {
array_push($resp, array(
'board_title' => isset($timeLog[$key]->board->title) ? $timeLog[$key]->board->title : null,
'task_title' => isset($timeLog[$key]->task->title) ? $timeLog[$key]->task->title : null,
'id' => $timeLog[$key]->id
));
}
}
return response()->json($resp);
}
The TimeLog.php where the relation has been made.
public function board()
{
return $this->belongsTo('App\Board', 'board_id', 'id');
}
public function task()
{
return $this->belongsTo('App\Task', 'task_id', 'id');
}
Our new way is like this:
BillingController.php
public function byYear() {
$timeLog = TimeLog::
join('oc_boards', 'oc_boards.id', '=', 'oc_time_logs.board_id')
->join('oc_tasks', 'oc_tasks.id', '=', 'oc_time_logs.task_id')
->join('oc_users', 'oc_users.id', '=', 'oc_time_logs.user_id')
->select('oc_boards.title AS board_title', 'oc_tasks.title AS task_title','oc_time_logs.id','oc_time_logs.time_used_sec','oc_users.id AS user_id')
->getQuery()
->get();
return response()->json($timeLog);
}
We deleted the relation in TimeLog.php, cause we don't need it anymore. Now we have a load time about 1 sec, which is fine!
There are about 20k entries in the time log table.
My questions are:
Why is the first method out of range (what causes the timeout?)
What does getQuery(); exactly do?
If you need more information just ask me.
--First Question--
One of the issues you might be facing is having all those huge amount of data in memory, i.e:
$timeLog = TimeLog::get();
This is already enormous. Then when you are trying to convert the collection to array:
There is a loop through the collection.
Using the $timeLog->toArray() while initializing the loop based on my understanding is not efficient (I might not be entirely correct about this though)
Thousands of queries are made to retrieve the related models
So what I would propose are five methods (one which saves you from hundreds of query), and the last which is efficient in returning the result as customized:
Since you have many data, then chunk the result ref: Laravel chunk so you have this instead:
$timeLog = TimeLog::chunk(1000, function($logs){
foreach ($logs as $log) {
// Do the stuff here
}
});
Other way is using cursor (runs only one query where the conditions match) the internal operation of cursor as understood is using Generators.
foreach (TimeLog::where([['board_id','>',0],['task_id', '>', 0]])->cursor() as $timelog) {
//do the other stuffs here
}
This looks like the first but instead you have already narrowed your query down to what you need:
TimeLog::where([['board_id','>',0],['task_id', '>', 0]])->get()
Eager Loading would already present the relationship you need on the fly but might lead to more data in memory too. So possibly the chunk method would make things more easier to manage (even though you eagerload related models)
TimeLog::with(['board','task'], function ($query) {
$query->where([['board_id','>',0],['task_id', '>', 0]]);
}])->get();
You can simply use Transformer
With transformer, you can load related model, in elegant, clean and more controlled methods even if the size is huge, and one greater benefit is you can transform the result without having to worry about how to loop round it
You can simply refer to this answer in order to perform a simple use of it. However incase you don't need to transform your response then you can take other options.
Although this might not entirely solve the problem, but because the main issues you face is based on memory management, so the above methods should be useful.
--Second question--
Based on Laravel API here You could see that:
It simply returns the underlying query builder instance. To my observation, it is not needed based on your example.
UPDATE
For question 1, since it seems you want to simply return the result as response, truthfully, its more efficient to paginate this result. Laravel offers pagination The easiest of which is SimplePaginate which is good. The only thing is that it makes some few more queries on the database, but keeps a check on the last index; I guess it uses cursor as well but not sure. I guess finally this might be more ideal, having:
return TimeLog::paginate(1000);
I have faced a similar problem. The main issue here is that Elloquent is really slow doing massive task cause it fetch all the results at the same time so the short answer would be to fetch it row by row using PDO fetch.
Short example:
$db = DB::connection()->getPdo();
$query_sql = TimeLog::join('oc_boards', 'oc_boards.id', '=', 'oc_time_logs.board_id')
->join('oc_tasks', 'oc_tasks.id', '=', 'oc_time_logs.task_id')
->join('oc_users', 'oc_users.id', '=', 'oc_time_logs.user_id')
->select('oc_boards.title AS board_title', 'oc_tasks.title AS task_title','oc_time_logs.id','oc_time_logs.time_used_sec','oc_users.id AS user_id')
->toSql();
$query = $db->prepare($query->sql);
$query->execute();
$logs = array();
while ($log = $query->fetch()) {
$log_filled = new TimeLog();
//fill your model and push it into an array to parse it to json in future
array_push($logs,$log_filled);
}
return response()->json($logs);
This may be a trivial question but I am wondering if Laravel recommends a certain way to check whether an Eloquent collection returned from $result = Model::where(...)->get() is empty, as well as counting the number of elements.
We are currently using !$result to detect empty result, is that sufficient? As for count($result), does it actually cover all cases, including empty result?
When using ->get() you cannot simply use any of the below:
if (empty($result)) { }
if (!$result) { }
if ($result) { }
Because if you dd($result); you'll notice an instance of Illuminate\Support\Collection is always returned, even when there are no results. Essentially what you're checking is $a = new stdClass; if ($a) { ... } which will always return true.
To determine if there are any results you can do any of the following:
if ($result->first()) { }
if (!$result->isEmpty()) { }
if ($result->count()) { }
if (count($result)) { }
You could also use ->first() instead of ->get() on the query builder which will return an instance of the first found model, or null otherwise. This is useful if you need or are expecting only one result from the database.
$result = Model::where(...)->first();
if ($result) { ... }
Notes / References
->first() http://laravel.com/api/4.2/Illuminate/Database/Eloquent/Collection.html#method_first
isEmpty() http://laravel.com/api/4.2/Illuminate/Database/Eloquent/Collection.html#method_isEmpty
->count() http://laravel.com/api/4.2/Illuminate/Database/Eloquent/Collection.html#method_count
count($result) works because the Collection implements Countable and an internal count() method: http://laravel.com/api/4.2/Illuminate/Database/Eloquent/Collection.html#method_count
Bonus Information
The Collection and the Query Builder differences can be a bit confusing to newcomers of Laravel because the method names are often the same between the two. For that reason it can be confusing to know what one you’re working on. The Query Builder essentially builds a query until you call a method where it will execute the query and hit the database (e.g. when you call certain methods like ->all() ->first() ->lists() and others). Those methods also exist on the Collection object, which can get returned from the Query Builder if there are multiple results. If you're not sure what class you're actually working with, try doing var_dump(User::all()) and experimenting to see what classes it's actually returning (with help of get_class(...)). I highly recommend you check out the source code for the Collection class, it's pretty simple. Then check out the Query Builder and see the similarities in function names and find out when it actually hits the database.
Laravel 5.2 Collection Class
Laravel 5.2 Query Builder
I think you are looking for:
$result->isEmpty()
This is different from empty($result), which will not be true because the result will be an empty collection. Your suggestion of count($result) is also a good solution. I cannot find any reference in the docs
I agree the above approved answer. But usually I use $results->isNotEmpty() method as given below.
if($results->isNotEmpty())
{
//do something
}
It's more verbose than if(!results->isEmpty()) because sometimes we forget to add '!' in front which may result in unwanted error.
Note that this method exists from version 5.3 onwards.
There are several methods given in Laravel for checking results count/check empty/not empty:
$result->isNotEmpty(); // True if result is not empty.
$result->isEmpty(); // True if result is empty.
$result->count(); // Return count of records in result.
I think better to used
$result->isEmpty();
The isEmpty method returns true if the collection is empty; otherwise,
false is returned.
According to Laravel Documentation states you can use this way:
$result->isEmpty();
The isEmpty method returns true if the collection is empty; otherwise, false is returned.
I think you try something like
#if(!$result->isEmpty())
// $result is not empty
#else
// $result is empty
#endif
or also use
if (!$result) { }
if ($result) { }
You can do
$result = Model::where(...)->count();
to count the results.
You can also use
if ($result->isEmpty()){}
to check whether or not the result is empty.
so Laravel actually returns a collection when just using Model::all();
you don't want a collection you want an array so you can type set it.
(array)Model::all(); then you can use array_filter to return the results
$models = (array)Model::all()
$models = array_filter($models);
if(empty($models))
{
do something
}
this will also allow you to do things like count().
You can use: $counter = count($datas);
The in_array() checks if a value exists in an array.
public function isAbsolutelyEmpty($value)
{
return in_array($value, ["", "0", null, 0, 0.0], true);
}
You want to check these two cases of count().
#1
If the result contains only a single row (one record) from the database by using ->first().
if(count($result)) {
// record is exist true...
}
#2
If result contain set of multiple row (multiple records) by using ->get() or ->all().
if($result->count()) {
//record is exist true...
}
I have two models: Plans and PlanDetails.
Relationship is: PlanDetails hasMany Plans. Plans belongTo PlanDetails.
In the PlanDetails view.ctp, I am pulling in related Plans.
I am trying to sort the Plans by ANY field (I've tried them all), and I cannot get it working. I assume I am overlooking something very simple.
Here is my base code:
PlanDetail >> view.ctp:
...foreach ($planDetail['Plan'] as $plan_edit) :
$class = null;
if ($i++ % 2 == 0) {
$class = ' class="altrow"';
}...
<?php echo $this->Paginator->sort('Plan ID', 'Plan.id'); ?>...
...<?php echo $plan_edit['id']; ?>
plan_details_controller.php:
...function view($id = null) {
if (!$id) {
$this->Session->setFlash(__('Invalid plan detail', true));
$this->redirect(array('action' => 'index'));
}
$this->PlanDetail->recursive = 2; // To run the editable form deeper.
$this->set('planDetail', $this->PlanDetail->read(null, $id));
$this->set('plan', $this->paginate('Plan'));
}...
I should add, no errors are being thrown and the sort() arrows on the ID field are showing as expected, but the sort order DOES not change when clicked either way.
Sorry, I'm not able to comment on the question itself, but I've noticed that in your action, you set planDetail to be the PlanDetail record you read (with recursive set to 2), and then you set plan to be the result of the paginate call.
Then, in your view template, you're iterating over $planDetail's contained Plan association, like this:
foreach ($planDetail['Plan'] as $plan_edit):
But in order to get the sorting and pagination done, you need to be displaying the results of the paginate call i.e. iterate over the records contained in $plan.
Do a debug($plan) in your view template to see what results you get there and to see if the records' ordering changes when you sort by different fields.
Also, perhaps you're using syntax I'm not aware of, but if you simply call $this->paginate('Plan') in your controller, I don't know that you're going to get only the related Plan records for your particular PlanDetail record. (There's nothing tying the $id passed into your view action with the Plan records.) You might need to add some conditions to the paginate call, like so:
$this->paginate['Plan'] = array('conditions' => array('Plan.plan_detail_id' => $id));
$this->set('plans', $this->paginate('Plan'));
Here is what I did to solve this. Based on some helpful direction from johnp & tokes.
plan_details/view.ctp:
...$i = 0;
foreach ($plan as $plan_edit) : // note $plan here.
}...
In my plan_details_controller.php view action:
$conditions = array("Plan.plan_detail_id" => "$id");
$this->set('plan', $this->paginate('Plan', $conditions)); // note "plan" in the first argument of set (this is required to pass the results back to the view). Also. The $condition is set to return ONLY plans that match the plan_detail_id of id (on the plan_details table).
And in my view, in order to get my results (because I changed the array name), I had to change the way I was getting the values to:
$plan_edit['Plan']['modified'] // note I placed ['Plan'] in front of these calls as the array called for this to get the data...
Well until the next problem! SOLVED.