Laravel: Cache with pagination clear issue - laravel

I have laravel (7.x) application. I recently added the cache functionality for the performance boost. After implementing the cache functionality, I was having trouble with the pagination while loading the data in grid format, so I googled for the solution and found this Pagination with cache in Laravel.
Although, it did solve my problem. But, the case is that I have about 100 pages and due to the solution I found, each page has it's own cache. Now, if I create or update any record then it doesn't reflect in the grid because the data is loaded from the cache.
PostController.php:
...
$arraySearch = request()->all();
# calculating selected tab
$cache = (!empty(request()->inactive)) ? 'inactive' : 'active';
$cacheKey = strtoupper("{$this->controller}-index-{$cache}-{$arraySearch['page']}");
# caching the fetch data
$arrayModels = cache()->remember($cacheKey, 1440, function() use ($arraySearch) {
# models
$Post = new Post();
# returning
return [
'active' => $Post->_index(1, 'active', $arraySearch),
'inactive' => $Post->_index(0, 'inactive', $arraySearch),
];
});
...
Post.php:
public function _index($status = 1, $page = null, $arraySearch = null)
{
...
$Self = self::where('status', $status)
->orderBy('status', 'ASC')
->orderBy('title', 'ASC')
->paginate(10);
...
return $Self;
}
How do I clear all this cache to show the newly created or updated record to with the updated values.?

1. Store All pages under the same tag:
As seen on the documentation: https://laravel.com/docs/master/cache#storing-tagged-cache-items
You can use tags to group cached items.
$cacheTag = strtoupper("{$this->controller}-index-{$cache}");
$arrayModels = cache()->tags([$cacheTag])->remember($cacheKey, 1440, function() use ($arraySearch) {
...
2. Set an event listener on Post to clear the tag
You can run an Event listener on your Post update() or create() events.
https://laravel.com/docs/7.x/eloquent#events-using-closures
You can then clear the tag cache using
Cache::tags([$cacheTag])->flush();

I know this isn't the proper solution. But, until I find the proper way to do it, this is the option I am kind of stuck with.
PostController.php:
public function index()
{
...
$arraySearch = request()->all();
# calculating selected tab
$cache = (!empty(request()->inactive)) ? 'inactive' : 'active';
$cacheKey = strtoupper("{$this->controller}-index-{$cache}-{$arraySearch['page']}");
# caching the fetch data
$arrayModels = cache()->remember($cacheKey, 1440, function() use ($arraySearch) {
# models
$Post = new Post();
# returning
return [
'active' => $Post->_index(1, 'active', $arraySearch),
'inactive' => $Post->_index(0, 'inactive', $arraySearch),
];
});
...
}
public function store()
{
...
Artisan::call('cache:clear');
...
}
I'll post the proper solution when I find one. Till then I am using this one.

There is a method in Laravel Model class called booted (not boot, which is having a different purpose). This method runs every time something is "saved" (including "updated") or "deleted".
I have used this as following (in a Model; or a Trait, included in a Model):
protected static function booted(): void
{
$item = resolve(self::class);
static::saved(function () use ($item) {
$item->updateCaches();
});
static::deleted(function () use ($item) {
$item->updateCaches();
});
}
"updateCaches" is a method in the Trait (or in the Model), that can have the code to update the cache.

Related

How to check data exists in the database

I have a function to add new property. But i want to check for duplicate data at column "code" before add new data into database. If data exists will appear a message error.
function addPro(Request $req)
{
$id = $req->type_id;
$type = AssetType::find($id);
if($req->save == 'save'){
$pro = new TypeProperties;
$pro->name = $req->name;
$pro->code = $req->code;
$pro->type = $req->type;
$pro->assettype_id = $req->type_id;
$pro->save();
Schema::table($type->code, function ($table) use ($pro) {
if ($pro->type == "textbox")
$table->string($pro->code )->nullable();
if ($pro->type == "textarea")
$table->text($pro->code )->nullable();
});
return redirect(url($type->id.'/add/property'))->with('message','Save successful');
}
return redirect(url('asset/type/'.$type->id));
}
You can use laravel Request Validation
function addPro(Request $req)
{
$id = $req->type_id;
$type = AssetType::find($id);
if($req->save == 'save'){
$req->validate([
'code' => 'required|unique:tablename'
]);
$pro = new TypeProperties;
$pro->name = $req->name;
$pro->code = $req->code;
$pro->type = $req->type;
$pro->assettype_id = $req->type_id;
$pro->save();
Schema::table($type->code, function ($table) use ($pro) {
if ($pro->type == "textbox")
$table->string($pro->code )->nullable();
if ($pro->type == "textarea")
$table->text($pro->code )->nullable();
});
return redirect(url($type->id.'/add/property'))->with('message','Save successful');
}
return redirect(url('asset/type/'.$type->id));
}
The most simple way to do this is by checking if code is_null :
if (is_null($pro->code)) {
// It does not exist
} else {
// It exists
}
The other way is to make a validation using Laravel's built in ValidateRequest class. The most simple use-case for this validation, is to call it directly in your store() method like this:
$this->validate($req, [
'code' => 'required|unique,
//... and so on
], $this->messages);
With this, you're validating users $req by saying that specified columns are required and that they need to be unique, in order for validation to pass. In your controller, you can also create messages function to display error messages, if the condition isn't met:
private $messages = [
'code.required' => 'Code is required',
'code.unique' => 'Code already exists',
//... and so on
];
You can also achieve this by creating a new custom validation class:
php artisan make:request StorePro
The generated class will be placed in the app/Http/Requests directory. Now, you can add a few validation rules to the rules method:
public function rules()
{
return [
'code' => 'required|unique,
//... and so on
];
}
All you need to do now is type-hint the request on your controller method. The incoming form request is validated before the controller method is called, meaning you do not need to clutter your controller with any validation logic:
public function store(StorePro $req)
{
// The incoming request is valid...
// Retrieve the validated input data...
$validated = $req->validated();
}
If you have any additional question about this, feel free to ask. Source: Laravel official documentation.
What does your migration look like for AssetType?
I ask because you can do this in the schema with ->unique() added to the column on the creation or make a migration to add the constraint.
You can also check with something like this:
// Search database table for entry
$entry = AssetType::where('code', '=', $pro->code)->first();
// If not found
if ($entry === null) {
// Save method here.
}
Otherwise, you can use the manual validator or create a Request with validation
References:
https://laravel.com/docs/5.8/queries#where-clauses
https://laravel.com/docs/5.8/validation#creating-form-requests
https://laravel.com/docs/5.8/validation#manually-creating-validators

Fetching a cached Eloquent collection of 10 items times out

I'm building a search functionality that returns large collections which are paginated using a LengthAwarePaginator. I'm trying to cache results using a key called $searchFilter_$query_$offsetPages for a single page of returned results (10 items). It goes into the cache just fine. However, it times out when I try to check using Cache::has($key) or fetch using Cache::get($key).
The same problem occurs in the browser as well as in artisan Tinker. Strangely, when I put a random set of 10 items into the cache in Tinker and fetch them back, everything works fine. I'm using Redis as the cache driver.
Here is my controller method:
public function search($filter, $query, $layout, Request $request) {
if($layout == "list-map") {
return view("list-map")->with(['filter' => $filter, 'query' => $query, 'layout' => 'list-map']);
} else {
$offsetPages = $request->input('page', 1) - 1;
$cacheKey = $filter . "_" . $query . "_" . $offsetPages;
if(Cache::has($cacheKey)) {
\Log::info("fetching results from cache");
$data = Cache::get($cacheKey);
$totalCt = $data[0];
$results = $data[1];
} else {
$results = $this->getResults($filter, $query);
$totalCt = $results->count();
$results = $results->slice($offsetPages, $this->resultsPerPage);
\Log::info("caching results");
Cache::put($cacheKey, [$totalCt, $results], 5);
}
$results = new LengthAwarePaginator($results,
$totalCt,
$this->resultsPerPage,
$request->input('page', 1),
['path' => LengthAwarePaginator::resolveCurrentPath()]
);
return view($layout)->with(['filter' => $filter, 'query' => $query, 'layout' => $layout, 'results' => $results]);
}
}
So, the issue was that many of the models in the collection returned from my getResults() method were obtained via relationship queries. When I would dd($results) on the single page of 10 results, I could see that there was a "relations" field on each model. Inside that array were thousands of recursively related models based on the relationship I originally queried. I was unable to find any information about an option to not eager load these related models. Instead I came up with a bit of a hacky workaround to fetch the models directly:
$results = $results->slice($offsetPages, $this->resultsPerPage);
//load models directly so they don't include related models.
$temp = new \Illuminate\Database\Eloquent\Collection;
foreach($results as $result) {
if(get_class($result) == "App\Doctor") {
$result = Doctor::find($result->id);
} else if(get_class($result == "App\Organization")) {
$result = Organization::find($result->id);
}
$temp->push($result);
}
$results = $temp;
\Log::info("caching results");
Cache::put($cacheKey, [$totalCt, $results], 5);
If anyone knows the best practice in this situation, please let me know. Thanks!
Edit:
I've found a better solution instead of the above workaround. If I query my relationships like this: $taxonomy->doctors()->get() rather than $taxonomy->doctors, it does not load in the huge recusive relations.
I dont really see why your code doesn't work. The only potential problems I see are the cache keys, which could contain problematic characters, as well as the way you check for a cached value. As you are using Cache::has($key) before Cache::get($key), you could end up with a race condition where the first call returns true and the latter null because the cached value timed out just between the two calls.
I tried to address both issues in the following snippet:
public function search($filter, $query, $layout, Request $request)
{
if($layout == "list-map") {
return view("list-map")->with(['filter' => $filter, 'query' => $query, 'layout' => 'list-map']);
} else {
$offsetPages = $request->input('page', 1) - 1;
$cacheKey = md5("{$filter}_{$query}_{$offsetPages}");
$duration = 5; // todo: make this configurable or a constant
[$totalCount, $results] = Cache::remember($cacheKey, $duration, function () use ($filter, $query) {
$results = $this->getResults($filter, $query);
$totalCount = $results->count();
$filteredResults = $results->slice($offsetPages, $this->resultsPerPage);
return [$totalCount, $filteredResults];
});
$results = new LengthAwarePaginator($results,
$totalCount,
$this->resultsPerPage,
$request->input('page', 1),
['path' => LengthAwarePaginator::resolveCurrentPath()]
);
return view($layout)->with(compact('filter', 'query', 'layout', 'results'));
}
}
The inbuilt function Cache::remember() doesn't use Cache::has() under the hood. Instead, it will simply call Cache::get(). As this function will return null as default if no cache was hit, the function can easily determine if it has to execute the closure or not.
I also wrapped the $cacheKey in md5(), which gives a consistently valid key.
Looking at the following part of your code
$results = $this->getResults($filter, $query);
$totalCount = $results->count();
$filteredResults = $results->slice($offsetPages, $this->resultsPerPage);
I am quite sure the whole search could be improved (independently of the caching). Because it seems you are loading all results for a specific search into memory, even if you throw away most parts of it. There is certainly a better way to do this.

Optimizing code with chunk or cursor in laravel

I'm having Company Model and Contact Model defined in my Laravel 5.4 application, both of them have many to many relationship. So, for example contacts model has:
public function company()
{
return $this
->belongsToMany('App\Company', 'company_contact','contact_id', 'company_id')->withTimestamps();
}
Now I've a data set where I want to pull all contacts data and there company details so I was using:
public function getData()
{
$contacts = Contact::all();
foreach($contacts as $contact)
{
$getCompany = $contact->company()->withPivot('created_at')->orderBy('pivot_created_at', 'desc')->first();
$getCompany->contacts = Company::find($getCompany->id)->contacts;
$contact->company = $getCompany;
$contact->companies_interested = json_decode($contact->companies_interested);
$companies = [];
if($contact->companies_interested)
{
foreach($contact->companies_interested as $companiesInterested)
{
$getCompany = Company::withTrashed()->find($companiesInterested);
$companies[] = array(
'value' => $getCompany->id,
'label' => $getCompany->name
);
}
$contact->companies_interested = json_encode($companies);
}
}
return response()->json([
'model' => $contacts
], 200);
}
This works perfectly fine for small data set, but while using large number of data it fails (approx 10,000 fields), I guess php memory fails to load when it comes to large data set. I was going through Laravel docs to find out the solution and came to know about chunk() and cursor() methods, Can someone guide me what can be done in this problem or what can be the approach to overcome this.
Thanks
I recommend you to test both methods for some quirkiness of your system.
Chunk:
It will "paginate" your query, this way you use less memory.
Uses less memory
It takes longer
`
public function getData() {
Contact::chunk(1000, function ($contacts) {
foreach ($contacts as $contact) {
//rest of your code...
}
});
}
`
Cursor:
You will use PHP Generators to search your query items one by one.
It takes less time
Uses more memory
`
public function getData() {
foreach (Contact::cursor() as $contact) {
//rest of your code...
}
}
`
For a more detailed explanation see this answer: What is the difference between laravel cursor and laravel chunk method?
For performance testing see this post: https://translate.google.com/translate?hl=en&sl=auto&tl=en&u=http%3A%2F%2Fqiita.com%2Fryo511%2Fitems%2Febcd1c1b2ad5addc5c9d

Yii2 ActiveRecord cache

How to use ActiveRecotd cache for Yii 2? I did't find any examples in official docs. In Google I found 2 examples, first is:
$db = self::getDb();
$object = $db->cache(function ($db) use($id) {
return self::findOne($id);
});
But it doesn't work for Model, I tested with updated framework. Other example is:
$data = \Yii::$app->cache->get('some_var_' . $id);
if ($data === false)
{
$data = self::findOne($id);
\Yii::$app->cache->set('some_var_' . $id, $data, 60);
}
It's working fine, but it's not ActiveRecord caching it's data caching, So we haven't got ActiveRecord caching in Yii 2?
1) Use cache like that:
$db = Yii::$app->db;// or Category::getDb()
$result = $db->cache(function ($db) use ($id) {
return Category::find()->where(['id' => $id])->all();
}, CACHE_TIMEOUT);
2) If you may use query dependency, use like that:
$db = Yii::$app->db;// or Category::getDb()
$dep = new DbDependency();
$dep->sql = 'SELECT count(*) FROM category';
$result = $db->cache(function ($db) use ($id) {
return Category::find()->where(['id' => $id])->all();
}, CACHE_TIMEOUT, $dep);
I too am having trouble with this. Here's my workaround for the time being for a hasOne() relationship.
public function getGroup()
{
if(isset(static::$_getGroup[$this->id])) {
return static::$_getGroup[$this->id];
}
$Group = $this->hasOne(BillChargesGroup::className(), ['id' => 'group_id'])->one();
static::$_getGroup[$this->id] = $Group;
return $Group;
}
I only want to cache data for the current request, so this works. However because I'm using ->one(); it does not return the ActiveQuery object if we call $model->getGroup() (which I found is good for extending queries)
Unfortunately if I do return the ActiveQuery object, Yii2 does some "magic" on it and always does a SELECT * which I can't control.
Since 2.0.14 you can use the following shortcuts:
(new Query())->cache(7200)->all();
// and
User::find()->cache(7200)->all();
Source: https://www.yiiframework.com/doc/guide/2.0/en/caching-data

How to approach caching in ZF2

I am just starting to get my head into caching as a whole. I have a simple indexAction() that fetches all given Datasets. My approach is:
check for existing key 'controllername-index-index'
if existing: return the value of the key
if not existing, do the normal action and add the key
The value inside the key should be the ViewModel that will be generated and populated with my data.
Here's what i have done so far:
<?php
public function indexAction()
{
$sl = $this->getServiceLocator();
// $cache = $sl->get('cache');
// $key = 'kennzahlen-index-index';
//
// if ($cache->hasItem($key)) {
// return $cache->getItem($key);
// }
$viewModel = new ViewModel();
$viewModel->setTemplate('kennzahlen/index/index');
$entityService = $sl->get('kennzahlen_referenzwert_service');
$viewModel->setVariable('entities', $entityService->findAll());
// $cache->setItem($key, $viewModel);
return $viewModel;
}
The Caching parts are commented out for testing purposes, but basically this is all that i am doing. The Caching config/service looks like the following:
<?php
'cache' => function () {
return \Zend\Cache\StorageFactory::factory(array(
'adapter' => array(
'name' => 'filesystem',
'options' => array(
'cache_dir' => __DIR__ . '/../../data/cache',
'ttl' => 100
),
),
'plugins' => array(
array(
'name' => 'serializer',
'options' => array(
)
)
)
));
},
The serialization and caching works quite well, but i am surprised by the missing results. Going by what the ZendDevelopersToolbar tells me, the times WITHOUT caching range between 1.8s to 2.5s. Having the caching parts uncommented (enabled) doesn't really improve the loading time of my page at all.
So my question is: Is this approach completely wrong? Are there different, more speedy parts, that can be saved with some neat configuration tricks?
I Feel that a 2 second load time of a page is DEFINITELY too slow. 1s to me is the maximum given a huge amount of data, but certainly not anything more than that :S
All help/hints/suggestions will be greatly appreciated. Thanks in advance!
One option would be to cache the complete output of your page, for example based on the route match. You need to listen between routing and dispatching which route has been found as match and then act accordingly:
namespace MyModule;
use Zend\Mvc\MvcEvent;
class Module
{
public function onBootstrap(MvcEvent $e)
{
// A list of routes to be cached
$routes = array('foo/bar', 'foo/baz');
$app = $e->getApplication();
$em = $app->getEventManager();
$sm = $app->getServiceManager();
$em->attach(MvcEvent::EVENT_ROUTE, function($e) use ($sm) {
$route = $e->getRouteMatch()->getMatchedRouteName();
$cache = $sm->get('cache-service');
$key = 'route-cache-' . $route;
if ($cache->hasItem($key)) {
// Handle response
$content = $cache->getItem($key);
$response = $e->getResponse();
$response->setContent($content);
return $response;
}
}, -1000); // Low, then routing has happened
$em->attach(MvcEvent::EVENT_RENDER, function($e) use ($sm, $routes) {
$route = $e->getRouteMatch()->getMatchedRouteName();
if (!in_array($route, $routes)) {
return;
}
$response = $e->getResponse();
$content = $response->getContent();
$cache = $sm->get('cache-service');
$key = 'route-cache-' . $route;
$cache->setItem($key, $content);
}, -1000); // Late, then rendering has happened
}
}
The second listener checks at the render event. If that happens, the result of the response will be cached.
This system (perhaps not with 100% copy/paste, but the concept) works because if you return a Response during the route or dispatch event, the application will short circuit the application flow and stop further triggering listeners. It will then serve this response as it is.
Bear in mind it will be the complete page (including layout). If you don't want that (only the controller), move the logic to the controller. The first event (now route) will be dispatch of the controller. Listen to that early, so the normal execution of the action will be omitted. To cache the result, check the render event for the view layer to listen to.
/update: I wrote a small module to use this DRY in your app: SlmCache

Resources