I am running to an issue where the following code fails in testing only, while behaving correctly in the browser.
The code should provide a CursorPaginator for an Eloquent collection and return it as JSON.
Test Code:
public function testMoreNotesAreReturnedIfRequested()
{
$item = Item::factory()->has(Entry::factory()->for($this->user)->count(6))->create();
// Get page one of paginator, and request page 2 using the URL it returns
$response = $this->actingAs($this->user)->json('get',"/items/{$item->id}/notes/more");
$next = $this->actingas($this->user)->json('get',$response->decodeResponseJson()['next_page_url']);
$next->assertJson(fn (AssertableJson $json) => $json
->has('data')
->has('paginator'));
}
The above fails with the following exception:
Only arrays and objects are supported when cursor paginating items.
I have confirmed that there are 6 entries created in line 1, and that they all have different millisecond creation times as suggested when googling the problem.
Any ideas?
Controller Code:
return $item->entries()
->latest()
->with(['content','user'])
->cursorPaginate(5)
->withPath("/items/$item->id/notes/more")
->through( static function ($item) use ($request) {
$item->setAttribute('can_edit',$request->user()->can('update',$item->content));
$item->setAttribute('can_delete',$request->user()->can('delete',$item->content));
return $item;
});
Edit - the test fails every time, whereas the browser only fails when the number of items in the collection is 6, 7, 16 or 17. I am now even more confused.
This seems to be caused by using latest() for ordering. The problem was resolved by changing to
->orderBy('id','desc')
Related issues that people are having seem to suggest there is a problem with the precision of datestamps (in my case MariaDB providing 6 decimal places after the second). See here and here.
I wonder if this is a bug in CursorPaginator.
Related
I`ve got code that shows all of the placements for a user. Is there a way to narrow this down to only show placements that are in the future? I've tried to do this using where and carbon::now to no avail.
My current code to show all of the placements :
$placements = Auth::user()->placementsAuthored;
$placements->load('keystage', 'subject', 'dates');
Placements Authored connection to connect a user to a placement :
public function placementsAuthored()
{
return $this->hasMany(Placement::class, 'author_id');
}
My attempt at trying to do this. I get no errors but the code doesn't work. It doesn't seem to take any effect of my where clause any ideas?
$placements ->where('date','>',Carbon::now()->format('Y-m-d'));
After a bit of tweaking, I found that this works but I don't understand why this works and the above doesn't. In my mind, they do the same but this is a longer way of doing it. Any idea why this works and the above doesn't?
// Only load future placements
$placements = Placement::whereHas( 'dates',
function ($q) {
$user_id = Auth::user()->id;
$q->where('author_id', $user_id)->where('date','>=' ,Carbon::now()->format('Y-m-d'));})->get();
You should take a different name for the date column because the date keyword is already reserved in the PHP function this is not a standard way
I'm facing a weird problem which only happen in the production server (It's 100% normal in my local). I'm using Laravel 8, and have a model named Student. Inside the model, I created several scope methods like this:
public function scopeWhereAlreadySubmitData($query)
{
return $query->where('status_data_id', '!=', 1);
}
public function scopeWhereAlreadySubmitDocument($query)
{
return $query->where('status_document_id', '!=', 1);
}
Then I use the methods above in an AJAX controller to chain it with count method. Something like:
public function getAggregates()
{
$student = Student::with(['something', 'somethingElse']);
$count_student_already_submit_data = $student->whereAlreadySubmitData()->count();
$count_student_already_submit_document = $student->whereAlreadySubmitDocument()->count();
return compact('count_student_already_submit_data', 'count_student_already_submit_document');
}
Here's is where the weird thing happens: while the controller above produce the correct value in my local, in production count_student_already_submit_data has a correct value but count_student_already_submit_document has zero value. Seeing at the database directly, count_student_already_submit_document should have value more than zero as there are many records has status_document_id not equals to 1.
I've also tried to use the scopeWhereAlreadySubmitDocument method in tinker. Both in local and production, it shows the correct value, not just zero.
Another thing to note is that Student model actually had 4 scope methods like the above, not just 2. 3 of them is working correctly, and only 1 is not. Plus, there's another controller using all the scope methods above and all of them are showing the correct value.
Have you ever face such thing? What could be the problem behind this? Your input is appreciated.
Turns out this is because the query builder is effected by the given chain. The weird thing I mentioned about "local vs production" is likely doesn't gets noticed in the local because its only has a few records. I should've use clone() when chaining the builder. See the topic below for more information.
Laravel query builder - re-use query with amended where statement
I have two forms; the first is the form formers with two fields (name, firstname).
I also have the form trainings with two fields (date_sitting, fk_former).
My problem, if I want to add the other sitting today (07/07/2019), I would like to see only the formers who have no training today.
Here, I retrieve a former who has a sitting today.
Do you think it's possible to get only the formers who have no of sitting for now?
Edit: 10/07/2019
Controller Training
public function index()
{
$trainings = Training::oldest()->paginate(5);
$formersNoTrainingToday = Training::whereDate('date_sitting', "!=", Carbon::today())
->orWhere('date_sitting', null)->get();
return view('admin.trainings.index', compact('trainings', 'formersNoTrainingToday'))
->with('i', (request()->input('page',1) -1)*5);
}
And
public function create()
{
$formers = Former::all();
return view('admin.trainings.create', compact('formers','trainings'));
}
I would like to see only the formers who have no training today.
Sure - you can determine your correct list of candidates to show by using the following query:
$formersNoTrainingToday = Training::whereDate('date_sitting', "!=", Carbon::today())
->orWhere('date_sitting', null)->get();
This should work... but it assumes a few things within your code / db. If this fails, consider a few options to replace the whereDate section above:
Using where:
->where('date_sitting', '!=', \Carbon::today()->toDateString())
Using formatted date if that column on the DB is a different format than Carbon:
->whereDate('date_sitting', "!=", Carbon::now()->format('m/d/Y'))
If you're not using Carbon for some reason, you can try the raw query route for today:
->whereDate('date_sitting', "!=", DB::raw('CURDATE()'))
Bottom line, here are a number of ways to get close to this. But you may need to tweak this on your own to suit your needs. You may need to take Timezone or some hours of difference into account, so you may need to add a range or buffer. But the above should get you close if not all the way there.
HTH
We are developing an API with LUMEN.
Today we had a confused problem with getting the collection of our "TimeLog"-model.
We just wanted to get all time logs with additional informationen from the board model and task model.
In one row of time log we had a board_id and a task_id. It is a 1:1 relation on both.
This was our first code for getting the whole data. This took a lot of time and sometimes we got a timeout:
BillingController.php
public function byYear() {
$timeLog = TimeLog::get();
$resp = array();
foreach($timeLog->toArray() as $key => $value) {
if(($timeLog[$key]->board_id && $timeLog[$key]->task_id) > 0 ) {
array_push($resp, array(
'board_title' => isset($timeLog[$key]->board->title) ? $timeLog[$key]->board->title : null,
'task_title' => isset($timeLog[$key]->task->title) ? $timeLog[$key]->task->title : null,
'id' => $timeLog[$key]->id
));
}
}
return response()->json($resp);
}
The TimeLog.php where the relation has been made.
public function board()
{
return $this->belongsTo('App\Board', 'board_id', 'id');
}
public function task()
{
return $this->belongsTo('App\Task', 'task_id', 'id');
}
Our new way is like this:
BillingController.php
public function byYear() {
$timeLog = TimeLog::
join('oc_boards', 'oc_boards.id', '=', 'oc_time_logs.board_id')
->join('oc_tasks', 'oc_tasks.id', '=', 'oc_time_logs.task_id')
->join('oc_users', 'oc_users.id', '=', 'oc_time_logs.user_id')
->select('oc_boards.title AS board_title', 'oc_tasks.title AS task_title','oc_time_logs.id','oc_time_logs.time_used_sec','oc_users.id AS user_id')
->getQuery()
->get();
return response()->json($timeLog);
}
We deleted the relation in TimeLog.php, cause we don't need it anymore. Now we have a load time about 1 sec, which is fine!
There are about 20k entries in the time log table.
My questions are:
Why is the first method out of range (what causes the timeout?)
What does getQuery(); exactly do?
If you need more information just ask me.
--First Question--
One of the issues you might be facing is having all those huge amount of data in memory, i.e:
$timeLog = TimeLog::get();
This is already enormous. Then when you are trying to convert the collection to array:
There is a loop through the collection.
Using the $timeLog->toArray() while initializing the loop based on my understanding is not efficient (I might not be entirely correct about this though)
Thousands of queries are made to retrieve the related models
So what I would propose are five methods (one which saves you from hundreds of query), and the last which is efficient in returning the result as customized:
Since you have many data, then chunk the result ref: Laravel chunk so you have this instead:
$timeLog = TimeLog::chunk(1000, function($logs){
foreach ($logs as $log) {
// Do the stuff here
}
});
Other way is using cursor (runs only one query where the conditions match) the internal operation of cursor as understood is using Generators.
foreach (TimeLog::where([['board_id','>',0],['task_id', '>', 0]])->cursor() as $timelog) {
//do the other stuffs here
}
This looks like the first but instead you have already narrowed your query down to what you need:
TimeLog::where([['board_id','>',0],['task_id', '>', 0]])->get()
Eager Loading would already present the relationship you need on the fly but might lead to more data in memory too. So possibly the chunk method would make things more easier to manage (even though you eagerload related models)
TimeLog::with(['board','task'], function ($query) {
$query->where([['board_id','>',0],['task_id', '>', 0]]);
}])->get();
You can simply use Transformer
With transformer, you can load related model, in elegant, clean and more controlled methods even if the size is huge, and one greater benefit is you can transform the result without having to worry about how to loop round it
You can simply refer to this answer in order to perform a simple use of it. However incase you don't need to transform your response then you can take other options.
Although this might not entirely solve the problem, but because the main issues you face is based on memory management, so the above methods should be useful.
--Second question--
Based on Laravel API here You could see that:
It simply returns the underlying query builder instance. To my observation, it is not needed based on your example.
UPDATE
For question 1, since it seems you want to simply return the result as response, truthfully, its more efficient to paginate this result. Laravel offers pagination The easiest of which is SimplePaginate which is good. The only thing is that it makes some few more queries on the database, but keeps a check on the last index; I guess it uses cursor as well but not sure. I guess finally this might be more ideal, having:
return TimeLog::paginate(1000);
I have faced a similar problem. The main issue here is that Elloquent is really slow doing massive task cause it fetch all the results at the same time so the short answer would be to fetch it row by row using PDO fetch.
Short example:
$db = DB::connection()->getPdo();
$query_sql = TimeLog::join('oc_boards', 'oc_boards.id', '=', 'oc_time_logs.board_id')
->join('oc_tasks', 'oc_tasks.id', '=', 'oc_time_logs.task_id')
->join('oc_users', 'oc_users.id', '=', 'oc_time_logs.user_id')
->select('oc_boards.title AS board_title', 'oc_tasks.title AS task_title','oc_time_logs.id','oc_time_logs.time_used_sec','oc_users.id AS user_id')
->toSql();
$query = $db->prepare($query->sql);
$query->execute();
$logs = array();
while ($log = $query->fetch()) {
$log_filled = new TimeLog();
//fill your model and push it into an array to parse it to json in future
array_push($logs,$log_filled);
}
return response()->json($logs);
I have laravel controller like this:
public function postSessionTopic() {
$article_id = Input::get('article_id', 0);
$comment_id = Input::get('comment_id', 0);
\Session::set('page_topic_id', $article_id);
\Session::set('page_comment_id', $comment_id);
\\comment - I have tried \Session::put too, but that doesn't change anything
}
I use it, when user click on a article. I print_r out my session variable in this controller and everything looks fine. But after that I refresh my page, and there I read value from session, and sometimes it load old value or doesn't load anything. I can't understand why, because in controller i can see, that correct value is saved!
In my page, i get that value like this:
\Session::get('page_topic_id', 0)
Probably you do something wrong. You should make sure that in both cases you uses exactly same domain (with or without www).
In this controller when you don't have any input you set to session variables 0. This can also be an issue if you launch this method when you don't have any input.
You could try with adding this basic route:
Route::get('/session', function() {
$page_topic = Session::get('page_topic_id', 1);
$page_comment = Session::get('page_comment_id', 1);
echo $page_topic.' '.$page_comment.'<br />';
$article_id = $page_topic * 2;
$comment_id = $page_comment * 3;
Session::set('page_topic_id', $article_id);
Session::set('page_comment_id', $comment_id);
});
As you see it's working perfectly (but you need to remove session cookie before trying with this path).
You get
1 1
2 3
4 9
8 27
and so on. Everything as expected
Answer was - two ajax at one time. Don't do that, if you store something in session.
The session in Laravel doesn't consider changes permanent unless you generate a response (and that's the result of using symphony as it's base). So make sure your app->run() ends properly and returns a response before refreshing. Your problem is mostly caused by a die() method somewhere along your code or an unexpected exit of PHP instance/worker.
This is probably not your issue but if you are storing your laravel session in the database their is a limit on how large that value can be. The Laravel session migration has a field called "payload" that is a text type. If you exceed the limit on that field the entire session gets killed off. This was happening to me as I was dynamically adding json model data to my session.
Schema::create('sessions', function (Blueprint $table) {
$table->string('id')->unique();
$table->text('payload');
$table->integer('last_activity');
});
How much UTF-8 text fits in a MySQL "Text" field?