In my Symfony/Doctrine app, I have a query that orders by RANDOM(). I call this same method several times, but it looks like the query's result is being cached.
Here's my relevant code:
$query = $table->createQuery('p')
->select('p.*, RANDOM() as rnd')
->orderBy('rnd')
->limit(1)
->useQueryCache(null)
->useResultCache(null);
$result = $query->fetchOne();
Unfortunately, the same record is returned every time, regardless of me passing null to both useQueryCache and useResultCache. I tried using false instead of null, but that didn't work either. Lastly, I also tried calling both setResultCacheLifeSpan(0) and setResultCacheLifeSpan(-1), but neither call made a difference.
Any insight on how to prevent caching since I want a different random row to be selected each time I call this method?
Edit: I also tried calling clearResultCache(), but that just ended up causing an error stating: "Result Cache driver not initialized".
Edit 2: As requested, here's the SQL generated by calling $query->getSqlQuery():
SELECT c.id AS c__id, c.name AS c__name, c.image_url AS c__image_url,
c.level AS c__level, c.created_at AS c__created_at, c.updated_at
AS c__updated_at, RANDOM() AS c__0 FROM cards c ORDER BY c__0 LIMIT 1
It turns out I'm a moron. I tried to simplify my query for this question, and in doing so, I didn't capture the true cause. I had a where() and andWhere() call, and the combination of conditions resulted in only one possible record being matched. Thanks for taking the time to respond, everyone, sorry to have wasted your time!
Doctrine also caches entities you created in the same request/script run.
For instance:
$order = new Order();
$order->save();
sleep(10); // Edit this record in de DB in another procces.
$q = new Doctrine_Query();
$result = $q->select()
->from('Order o')
->where('o.id = '.$order->id);
$order = $result->getFirst();
print_r($order->toArray());
The print_r will not contain the changes you made during the sleep.
The following code will remove that kind of memory cache:
$manager = Doctrine_Manager::getInstance();
$connection = $manager->getCurrentConnection();
$tables = $connection->getTables();
foreach ( $tables as $table ) {
$table->clear();
}
PS: Added this answer because I found this topic trying to resolve above issue.
Related
How can I remove the limit/offset from the below query?
$query = TestModel::where('a', 'b')->limit(100);
$query->removeLimit();
I'm using a query from another module and I don't want to change the code.
You can reset the $limit property:
$query = TestModel::where('a', 'b')->limit(100);
$query->limit = null;
$unlimited = $query->get();
$query->getQuery()->limit = null;
One can reset the limit by passing the null value to the limit method.
$query->limit(null);
Works both with Eloquent\Builder and Query\Builder.
The simple answer to your question is - you cannot. Because you have already filtered the result set to a limit of 100 tuples.
What is the reason for you to avoid the change in code in the Model? Because what #Dhruv has suggested is the correct way to achieve what you want to.
In fact, if you still want to keep the code intact. You can rather define another function in your model this way and use it internally in your old function:
public function newFunction(){
return TestModel::where('a', 'b')->get();
}
public function oldFunction(){
return $this->newFunction()->limit(100);
}
Keeping your code consistent, then use newFunction() in your Controller to do whatever you want to.
get(): To get all record from table use get():
$query = TestModel::where('a', 'b')->get();
limit(): To limit the number of results returned from the query
$query = TestModel::where('a', 'b')->limit(10)->get();
We are developing an API with LUMEN.
Today we had a confused problem with getting the collection of our "TimeLog"-model.
We just wanted to get all time logs with additional informationen from the board model and task model.
In one row of time log we had a board_id and a task_id. It is a 1:1 relation on both.
This was our first code for getting the whole data. This took a lot of time and sometimes we got a timeout:
BillingController.php
public function byYear() {
$timeLog = TimeLog::get();
$resp = array();
foreach($timeLog->toArray() as $key => $value) {
if(($timeLog[$key]->board_id && $timeLog[$key]->task_id) > 0 ) {
array_push($resp, array(
'board_title' => isset($timeLog[$key]->board->title) ? $timeLog[$key]->board->title : null,
'task_title' => isset($timeLog[$key]->task->title) ? $timeLog[$key]->task->title : null,
'id' => $timeLog[$key]->id
));
}
}
return response()->json($resp);
}
The TimeLog.php where the relation has been made.
public function board()
{
return $this->belongsTo('App\Board', 'board_id', 'id');
}
public function task()
{
return $this->belongsTo('App\Task', 'task_id', 'id');
}
Our new way is like this:
BillingController.php
public function byYear() {
$timeLog = TimeLog::
join('oc_boards', 'oc_boards.id', '=', 'oc_time_logs.board_id')
->join('oc_tasks', 'oc_tasks.id', '=', 'oc_time_logs.task_id')
->join('oc_users', 'oc_users.id', '=', 'oc_time_logs.user_id')
->select('oc_boards.title AS board_title', 'oc_tasks.title AS task_title','oc_time_logs.id','oc_time_logs.time_used_sec','oc_users.id AS user_id')
->getQuery()
->get();
return response()->json($timeLog);
}
We deleted the relation in TimeLog.php, cause we don't need it anymore. Now we have a load time about 1 sec, which is fine!
There are about 20k entries in the time log table.
My questions are:
Why is the first method out of range (what causes the timeout?)
What does getQuery(); exactly do?
If you need more information just ask me.
--First Question--
One of the issues you might be facing is having all those huge amount of data in memory, i.e:
$timeLog = TimeLog::get();
This is already enormous. Then when you are trying to convert the collection to array:
There is a loop through the collection.
Using the $timeLog->toArray() while initializing the loop based on my understanding is not efficient (I might not be entirely correct about this though)
Thousands of queries are made to retrieve the related models
So what I would propose are five methods (one which saves you from hundreds of query), and the last which is efficient in returning the result as customized:
Since you have many data, then chunk the result ref: Laravel chunk so you have this instead:
$timeLog = TimeLog::chunk(1000, function($logs){
foreach ($logs as $log) {
// Do the stuff here
}
});
Other way is using cursor (runs only one query where the conditions match) the internal operation of cursor as understood is using Generators.
foreach (TimeLog::where([['board_id','>',0],['task_id', '>', 0]])->cursor() as $timelog) {
//do the other stuffs here
}
This looks like the first but instead you have already narrowed your query down to what you need:
TimeLog::where([['board_id','>',0],['task_id', '>', 0]])->get()
Eager Loading would already present the relationship you need on the fly but might lead to more data in memory too. So possibly the chunk method would make things more easier to manage (even though you eagerload related models)
TimeLog::with(['board','task'], function ($query) {
$query->where([['board_id','>',0],['task_id', '>', 0]]);
}])->get();
You can simply use Transformer
With transformer, you can load related model, in elegant, clean and more controlled methods even if the size is huge, and one greater benefit is you can transform the result without having to worry about how to loop round it
You can simply refer to this answer in order to perform a simple use of it. However incase you don't need to transform your response then you can take other options.
Although this might not entirely solve the problem, but because the main issues you face is based on memory management, so the above methods should be useful.
--Second question--
Based on Laravel API here You could see that:
It simply returns the underlying query builder instance. To my observation, it is not needed based on your example.
UPDATE
For question 1, since it seems you want to simply return the result as response, truthfully, its more efficient to paginate this result. Laravel offers pagination The easiest of which is SimplePaginate which is good. The only thing is that it makes some few more queries on the database, but keeps a check on the last index; I guess it uses cursor as well but not sure. I guess finally this might be more ideal, having:
return TimeLog::paginate(1000);
I have faced a similar problem. The main issue here is that Elloquent is really slow doing massive task cause it fetch all the results at the same time so the short answer would be to fetch it row by row using PDO fetch.
Short example:
$db = DB::connection()->getPdo();
$query_sql = TimeLog::join('oc_boards', 'oc_boards.id', '=', 'oc_time_logs.board_id')
->join('oc_tasks', 'oc_tasks.id', '=', 'oc_time_logs.task_id')
->join('oc_users', 'oc_users.id', '=', 'oc_time_logs.user_id')
->select('oc_boards.title AS board_title', 'oc_tasks.title AS task_title','oc_time_logs.id','oc_time_logs.time_used_sec','oc_users.id AS user_id')
->toSql();
$query = $db->prepare($query->sql);
$query->execute();
$logs = array();
while ($log = $query->fetch()) {
$log_filled = new TimeLog();
//fill your model and push it into an array to parse it to json in future
array_push($logs,$log_filled);
}
return response()->json($logs);
I have a query which I use all over my routes.php under almost every get request and also use the results in many of my views. It makes more sense at this point for me to call the query once and be able to use it globally without ever having to call it again.
Here's the query:
$followers = Follower::where('user_id', '1')
->get();
How can I do this?
Why not just execute the query once in an init function and store the result into a global variable?
global $followers = Follower::where('user_id', '1')
->get();
you can store it to the session every time the user logs in
Like this exemple
$followers = Follower::where('user_id', '1')
->first();
Session::put('followers', 'value');
whenever you want that you can access it like this
$value = Session::get('followers');
The another answer with session is a simple solution but
I would suggest you to use Laravel Cache for this purpose (because this is the standard practice).
The Laravel Cache::remember accepts three parameters.
key: make an md5 key of 'followers' and 'user id'
time: time in minutes you want to cache the values (depending how frequently your values will be changed)
A closure function which runs when no value is found corresponding to the key. (this method will query once, in this case, and store the value in your cache)
Just do the following in your BaseController's constructor:
$id = 1; //User id
$key = md5('followers'.$id);
$minutes = 60; //cache for 1 hour, change it accordingly
$followers = Cache::remember($key, $minutes, function() use ($id) {
return Follower::where('user_id', $id)->get();
});
Now of course to use Cache you need to use some Cache driver like Redis.
If you don't have how to setup it Read my other answer.
Though it may be little longer solution for your problem, and may take you 15-20 min to set up and run everything, but believe me once you start using cache you will love it.
i created a Doctrine_Query and executes it but i wanna know what params i can pass to it.
$q = Doctrine_Query::create()
->select('cl.id, cl.name')
->from('ContactList cl');
$contactLists = $q->execute($params, $hydrationMode);
from the api documentation:
execute($params = array(), $hydrationMode = null)
where do they tell me about the params? and also hydration mode.
seems like i cannot find anything in the documentations. would be great if they had a reference for everything.
thanks
I beleive the params are an array of values to bind to the query - similar to a prepeared statement - for example:
$q = Doctrine_Query::create()
->select('cl.id, cl.name')
->from('ContactList cl')
->where('cl.name = ?');
$q->execute(array('fayer'));
The hydration mode is one of the hydrator constants from Doctrine_Core and determines how the result set is hydrated (Array, object, etc..) You can also write custom hydrators if you need to.
I followed doctrine documnetation to get started. Here is the documentation.
My code is
$User = Doctrine_Core::getTable("User")->find(1);
when I access relations by $User->Phonenumbers, it works. When I convert User object to array by using toArray() method, it does not convert relations to array. It simply display $User data.
Am I missing something?
By using the find method you've only retrieved the User data which is why the return of toArray is limited to that data. You need to specify the additional data to load, and the best place to do this is usually in the original query. From the example you linked to, add the select portion:
$q = Doctrine_Query::create()
->select('u.*, e.*, p.*') // Example only, select what you need, not *
->from('User u')
->leftJoin('u.Email e')
->leftJoin('u.Phonenumbers p')
->where('u.id = ?', 1);
Then when toArray'ing the results from that, you should see the associated email and phonenumber data as well.
I also noticed an anomaly with this where if you call the relationship first then call the ToArray, the relationship somehow gets included. what i mean is that, taking your own eg,
$User = Doctrine_Core::getTable("User")->find(1);
$num= $User->Phonenumbers->office; // assumed a field 'office' in your phone num table
$userArray = $user->toArray(true);
In the above case, $userArray somehow contains the whole relationship. if we remove the $num assignment it doesn't.
am guessing this is due to doctrine only fetching the one record first, and it's only when you try to access foreign key values that it fetches the other related tables