I am just starting to get my head into caching as a whole. I have a simple indexAction() that fetches all given Datasets. My approach is:
check for existing key 'controllername-index-index'
if existing: return the value of the key
if not existing, do the normal action and add the key
The value inside the key should be the ViewModel that will be generated and populated with my data.
Here's what i have done so far:
<?php
public function indexAction()
{
$sl = $this->getServiceLocator();
// $cache = $sl->get('cache');
// $key = 'kennzahlen-index-index';
//
// if ($cache->hasItem($key)) {
// return $cache->getItem($key);
// }
$viewModel = new ViewModel();
$viewModel->setTemplate('kennzahlen/index/index');
$entityService = $sl->get('kennzahlen_referenzwert_service');
$viewModel->setVariable('entities', $entityService->findAll());
// $cache->setItem($key, $viewModel);
return $viewModel;
}
The Caching parts are commented out for testing purposes, but basically this is all that i am doing. The Caching config/service looks like the following:
<?php
'cache' => function () {
return \Zend\Cache\StorageFactory::factory(array(
'adapter' => array(
'name' => 'filesystem',
'options' => array(
'cache_dir' => __DIR__ . '/../../data/cache',
'ttl' => 100
),
),
'plugins' => array(
array(
'name' => 'serializer',
'options' => array(
)
)
)
));
},
The serialization and caching works quite well, but i am surprised by the missing results. Going by what the ZendDevelopersToolbar tells me, the times WITHOUT caching range between 1.8s to 2.5s. Having the caching parts uncommented (enabled) doesn't really improve the loading time of my page at all.
So my question is: Is this approach completely wrong? Are there different, more speedy parts, that can be saved with some neat configuration tricks?
I Feel that a 2 second load time of a page is DEFINITELY too slow. 1s to me is the maximum given a huge amount of data, but certainly not anything more than that :S
All help/hints/suggestions will be greatly appreciated. Thanks in advance!
One option would be to cache the complete output of your page, for example based on the route match. You need to listen between routing and dispatching which route has been found as match and then act accordingly:
namespace MyModule;
use Zend\Mvc\MvcEvent;
class Module
{
public function onBootstrap(MvcEvent $e)
{
// A list of routes to be cached
$routes = array('foo/bar', 'foo/baz');
$app = $e->getApplication();
$em = $app->getEventManager();
$sm = $app->getServiceManager();
$em->attach(MvcEvent::EVENT_ROUTE, function($e) use ($sm) {
$route = $e->getRouteMatch()->getMatchedRouteName();
$cache = $sm->get('cache-service');
$key = 'route-cache-' . $route;
if ($cache->hasItem($key)) {
// Handle response
$content = $cache->getItem($key);
$response = $e->getResponse();
$response->setContent($content);
return $response;
}
}, -1000); // Low, then routing has happened
$em->attach(MvcEvent::EVENT_RENDER, function($e) use ($sm, $routes) {
$route = $e->getRouteMatch()->getMatchedRouteName();
if (!in_array($route, $routes)) {
return;
}
$response = $e->getResponse();
$content = $response->getContent();
$cache = $sm->get('cache-service');
$key = 'route-cache-' . $route;
$cache->setItem($key, $content);
}, -1000); // Late, then rendering has happened
}
}
The second listener checks at the render event. If that happens, the result of the response will be cached.
This system (perhaps not with 100% copy/paste, but the concept) works because if you return a Response during the route or dispatch event, the application will short circuit the application flow and stop further triggering listeners. It will then serve this response as it is.
Bear in mind it will be the complete page (including layout). If you don't want that (only the controller), move the logic to the controller. The first event (now route) will be dispatch of the controller. Listen to that early, so the normal execution of the action will be omitted. To cache the result, check the render event for the view layer to listen to.
/update: I wrote a small module to use this DRY in your app: SlmCache
Related
I have laravel (7.x) application. I recently added the cache functionality for the performance boost. After implementing the cache functionality, I was having trouble with the pagination while loading the data in grid format, so I googled for the solution and found this Pagination with cache in Laravel.
Although, it did solve my problem. But, the case is that I have about 100 pages and due to the solution I found, each page has it's own cache. Now, if I create or update any record then it doesn't reflect in the grid because the data is loaded from the cache.
PostController.php:
...
$arraySearch = request()->all();
# calculating selected tab
$cache = (!empty(request()->inactive)) ? 'inactive' : 'active';
$cacheKey = strtoupper("{$this->controller}-index-{$cache}-{$arraySearch['page']}");
# caching the fetch data
$arrayModels = cache()->remember($cacheKey, 1440, function() use ($arraySearch) {
# models
$Post = new Post();
# returning
return [
'active' => $Post->_index(1, 'active', $arraySearch),
'inactive' => $Post->_index(0, 'inactive', $arraySearch),
];
});
...
Post.php:
public function _index($status = 1, $page = null, $arraySearch = null)
{
...
$Self = self::where('status', $status)
->orderBy('status', 'ASC')
->orderBy('title', 'ASC')
->paginate(10);
...
return $Self;
}
How do I clear all this cache to show the newly created or updated record to with the updated values.?
1. Store All pages under the same tag:
As seen on the documentation: https://laravel.com/docs/master/cache#storing-tagged-cache-items
You can use tags to group cached items.
$cacheTag = strtoupper("{$this->controller}-index-{$cache}");
$arrayModels = cache()->tags([$cacheTag])->remember($cacheKey, 1440, function() use ($arraySearch) {
...
2. Set an event listener on Post to clear the tag
You can run an Event listener on your Post update() or create() events.
https://laravel.com/docs/7.x/eloquent#events-using-closures
You can then clear the tag cache using
Cache::tags([$cacheTag])->flush();
I know this isn't the proper solution. But, until I find the proper way to do it, this is the option I am kind of stuck with.
PostController.php:
public function index()
{
...
$arraySearch = request()->all();
# calculating selected tab
$cache = (!empty(request()->inactive)) ? 'inactive' : 'active';
$cacheKey = strtoupper("{$this->controller}-index-{$cache}-{$arraySearch['page']}");
# caching the fetch data
$arrayModels = cache()->remember($cacheKey, 1440, function() use ($arraySearch) {
# models
$Post = new Post();
# returning
return [
'active' => $Post->_index(1, 'active', $arraySearch),
'inactive' => $Post->_index(0, 'inactive', $arraySearch),
];
});
...
}
public function store()
{
...
Artisan::call('cache:clear');
...
}
I'll post the proper solution when I find one. Till then I am using this one.
There is a method in Laravel Model class called booted (not boot, which is having a different purpose). This method runs every time something is "saved" (including "updated") or "deleted".
I have used this as following (in a Model; or a Trait, included in a Model):
protected static function booted(): void
{
$item = resolve(self::class);
static::saved(function () use ($item) {
$item->updateCaches();
});
static::deleted(function () use ($item) {
$item->updateCaches();
});
}
"updateCaches" is a method in the Trait (or in the Model), that can have the code to update the cache.
I'm building a search functionality that returns large collections which are paginated using a LengthAwarePaginator. I'm trying to cache results using a key called $searchFilter_$query_$offsetPages for a single page of returned results (10 items). It goes into the cache just fine. However, it times out when I try to check using Cache::has($key) or fetch using Cache::get($key).
The same problem occurs in the browser as well as in artisan Tinker. Strangely, when I put a random set of 10 items into the cache in Tinker and fetch them back, everything works fine. I'm using Redis as the cache driver.
Here is my controller method:
public function search($filter, $query, $layout, Request $request) {
if($layout == "list-map") {
return view("list-map")->with(['filter' => $filter, 'query' => $query, 'layout' => 'list-map']);
} else {
$offsetPages = $request->input('page', 1) - 1;
$cacheKey = $filter . "_" . $query . "_" . $offsetPages;
if(Cache::has($cacheKey)) {
\Log::info("fetching results from cache");
$data = Cache::get($cacheKey);
$totalCt = $data[0];
$results = $data[1];
} else {
$results = $this->getResults($filter, $query);
$totalCt = $results->count();
$results = $results->slice($offsetPages, $this->resultsPerPage);
\Log::info("caching results");
Cache::put($cacheKey, [$totalCt, $results], 5);
}
$results = new LengthAwarePaginator($results,
$totalCt,
$this->resultsPerPage,
$request->input('page', 1),
['path' => LengthAwarePaginator::resolveCurrentPath()]
);
return view($layout)->with(['filter' => $filter, 'query' => $query, 'layout' => $layout, 'results' => $results]);
}
}
So, the issue was that many of the models in the collection returned from my getResults() method were obtained via relationship queries. When I would dd($results) on the single page of 10 results, I could see that there was a "relations" field on each model. Inside that array were thousands of recursively related models based on the relationship I originally queried. I was unable to find any information about an option to not eager load these related models. Instead I came up with a bit of a hacky workaround to fetch the models directly:
$results = $results->slice($offsetPages, $this->resultsPerPage);
//load models directly so they don't include related models.
$temp = new \Illuminate\Database\Eloquent\Collection;
foreach($results as $result) {
if(get_class($result) == "App\Doctor") {
$result = Doctor::find($result->id);
} else if(get_class($result == "App\Organization")) {
$result = Organization::find($result->id);
}
$temp->push($result);
}
$results = $temp;
\Log::info("caching results");
Cache::put($cacheKey, [$totalCt, $results], 5);
If anyone knows the best practice in this situation, please let me know. Thanks!
Edit:
I've found a better solution instead of the above workaround. If I query my relationships like this: $taxonomy->doctors()->get() rather than $taxonomy->doctors, it does not load in the huge recusive relations.
I dont really see why your code doesn't work. The only potential problems I see are the cache keys, which could contain problematic characters, as well as the way you check for a cached value. As you are using Cache::has($key) before Cache::get($key), you could end up with a race condition where the first call returns true and the latter null because the cached value timed out just between the two calls.
I tried to address both issues in the following snippet:
public function search($filter, $query, $layout, Request $request)
{
if($layout == "list-map") {
return view("list-map")->with(['filter' => $filter, 'query' => $query, 'layout' => 'list-map']);
} else {
$offsetPages = $request->input('page', 1) - 1;
$cacheKey = md5("{$filter}_{$query}_{$offsetPages}");
$duration = 5; // todo: make this configurable or a constant
[$totalCount, $results] = Cache::remember($cacheKey, $duration, function () use ($filter, $query) {
$results = $this->getResults($filter, $query);
$totalCount = $results->count();
$filteredResults = $results->slice($offsetPages, $this->resultsPerPage);
return [$totalCount, $filteredResults];
});
$results = new LengthAwarePaginator($results,
$totalCount,
$this->resultsPerPage,
$request->input('page', 1),
['path' => LengthAwarePaginator::resolveCurrentPath()]
);
return view($layout)->with(compact('filter', 'query', 'layout', 'results'));
}
}
The inbuilt function Cache::remember() doesn't use Cache::has() under the hood. Instead, it will simply call Cache::get(). As this function will return null as default if no cache was hit, the function can easily determine if it has to execute the closure or not.
I also wrapped the $cacheKey in md5(), which gives a consistently valid key.
Looking at the following part of your code
$results = $this->getResults($filter, $query);
$totalCount = $results->count();
$filteredResults = $results->slice($offsetPages, $this->resultsPerPage);
I am quite sure the whole search could be improved (independently of the caching). Because it seems you are loading all results for a specific search into memory, even if you throw away most parts of it. There is certainly a better way to do this.
I made a view helper that checks if an external URL exists before outputting it. Some of those URLs are in my main layout, so that check is quite slowing down my site by calling all those urls all the times, to check if they exist. I would like to save the output of that function so that it only checks an URL if the same one hasn't been checked already in less than an hour, or a day. I believe I should use Zend Cache to do that? But I have no idea where to start, do you have any suggestions, easy solutions or some basic tutorial to learn? Thanks!
Add global config for cache service, like here:
config/autoload/global.php
'service_manager' => array(
'abstract_factories' => array(
'Zend\Cache\Service\StorageCacheAbstractServiceFactory',
)
),
config/autoload/cache.global.php
return array(
'caches' => array(
// Cache config
)
)
Use factory to create your View Helper:
Application/Module.php::getViewHelperConfig()
'LinkHelper' => function ($sm) {
$locator = $sm->getServiceLocator();
return new LinkHelper($locator->get('memcached'))
}
Use cache service in your View Helper:
LinkHelper.php
protected $cache;
public function __construct($cache)
{
$this->cache = $cache;
}
public function __invoke($url)
{
$cacheKey = md5($url);
if ($this->cache->hasItem($cacheKey) {
return $this->cache->getItem($cacheKey);
}
$link = ''; // Parse link
$this->cache->setItem($cacheKey, $link);
return $link;
}
in my application i have a route (Literal or segment) for every action. i am not using one global route for everything so as a result the number of routes has grown hugely with 44+ modules (and more in future) .
It is my understanding (from what i have seen in the code) that in every page request zend goes throw all this routes in a array ans searches for a match witch could be bottleneck for the application (am i right?)
So i was thinking why not cache the matched routes in a db table with index to speed up the search ?
FIRST QUESTION : would this make the systems performance better?
so my first problem is skipping the system route matching mechanism. this is what i tried but it did not work :
public function onBootstrap(MvcEvent $e)
{
$em = StaticEventManager::getInstance();
$em->attach('Zend\Mvc\Application', MvcEvent::EVENT_ROUTE, array($this, 'onRoute'), 100);
}
public function onRoute(MvcEvent $e)
{
//var_dump($e->getRouteMatch());//->null routing has not been done yet
/* #var $router \Zend\Mvc\Router\Http\TreeRouteStack */
$router = $e->getRouter();
//-------------------------------------created a dummy route
$routeMatch = new \Zend\Mvc\Router\RouteMatch(array(
'controller' => 'Links\Controller\Items',
'action' => 'view',
'catId' => 0
));
$routeMatch->setMatchedRouteName('app/links');
$e->setRouteMatch($routeMatch);//set the dummy route
//--------------------------------------------PROBLEM HERE
//detach the onRoute event from routeListener
$e->getApplication()
->getServiceManager()
->get('RouteListener')
->detach($e->getApplication()->getEventManager());
}
the detach method is executed but the onRoute event still gets executed and matches the url to the correct route. so how to bypass(skip|detach) route matching ?
The reason you can't detach the listener is because you're already in the route event. By that point all the listeners have been marshalled together and queued for execution.
Why not instead take the listener out of the equation beforehand?
public function onBootstrap(MvcEvent $e)
{
$app = $e->getApplication();
$events = $app->getEventManager();
$shared = $events->getSharedManager();
$services = $app->getServiceManager();
$routeListener = $services->get('RouteListener');
$routeListener->detach($events);
$shared->attach('Zend\Mvc\Application', MvcEvent::EVENT_ROUTE, array($this, 'onRoute'), 100);
}
I am using the SecurityServiceProvider to secure my Silex application and would like to display a message after the user has logged out by navigating to the logout_path route.
The message should be stored in the sessions flash bag so that my template can automatically display it after.
I have tried adding an application middleware, but where not able to hook my code in. The before hook doesn't seem to work, because it happens after security and thus after the security's redirected back to my home page.
The before hook with the Application::EARLY_EVENT seems to be to early because as far as I know does the Security provider destroy the session after logout.
Before I keep trying to find a sort of working but probably dirty solution I would like to ask what the best/cleanest solution for this case would be?
UPDATE: After npms hint for a logout event handler I found this article on Google, which describes how to tackle the problem in Symfony very well.
In Silex things are slightly different though and after reading the source of the SecurityServiceProvider I came up with this solution.
$app['security.authentication.logout_handler._proto'] = $app->protect(function ($name, $options) use ($app) {
return $app->share(function () use ($name, $options, $app) {
return new CustomLogoutSuccessHandler(
$app['security.http_utils'],
isset($options['target_url']) ? $options['target_url'] : '/'
);
});
});
class CustomLogoutSuccessHanler extends DefaultLogoutSuccessHandler {
public function onLogoutSuccess(Request $request)
{
$request->getSession()->getFlashBag()->add('info', "Logout success!");
return $this->httpUtils->createRedirectResponse($request, $this->targetUrl);
}
}
The problem however is, that the flashbag message doesn't exist anymore after the redirect. So it seems that the session is being destroyed after the logout success handler is executed... or am I missing something? Is this even the right way to do it?
UPDATE: Still haven't found a proper solution yet. But this works.
I have added a parameter to the target url of the logout and use it to detect if a logout was made.
$app->register( new SecurityServiceProvider(), array(
'security.firewalls' => array(
'default' => array(
'pattern'=> '/user',
'logout' => array(
'logout_path' => '/user/logout',
'target_url' => '/?logout'
),
)
)
));
I had the same problem and your thoughts leaded me to a solution, thank you!
First define logout in the security.firewall:
$app->register(new Silex\Provider\SecurityServiceProvider(), array(
'security.firewalls' => array(
'general' => array(
'logout' => array(
'logout_path' => '/admin/logout',
'target_url' => '/goodbye'
)
)
),
));
Create a CustomLogoutSuccessHandler which handles the needed GET parameters for the logout, in this case redirect, message and pid:
class CustomLogoutSuccessHandler extends DefaultLogoutSuccessHandler
{
public function onLogoutSuccess(Request $request)
{
// use another target?
$target = $request->query->get('redirect', $this->targetUrl);
$parameter = array();
if (null != ($pid = $request->query->get('pid'))) {
$parameter['pid'] = $pid;
}
if (null != ($message = $request->query->get('message'))) {
$parameter['message'] = $message;
}
$parameter_str = !empty($parameter) ? '?'.http_build_query($parameter) : '';
return $this->httpUtils->createRedirectResponse($request, $target.$parameter_str);
}
}
Register the handler:
$app['security.authentication.logout_handler.general'] = $app->share(function () use ($app) {
return new CustomLogoutSuccessHandler(
$app['security.http_utils'], '/goodbye');
});
The trick to make this working as expected is to use another route to logout:
$app->get('/logout', function() use($app) {
$pid = $app['request']->query->get('pid');
$message = $app['request']->query->get('message');
$redirect = $app['request']->query->get('redirect');
return $app->redirect(FRAMEWORK_URL."/admin/logout?pid=$pid&message=$message&redirect=$redirect");
});
/logout set the needed parameters and execute the regular logout /admin/logout
Now you can use
/logout?redirect=anywhere
to redirect to any other route after logout or
/logout?message=xyz
(encoded) to prompt any messages in the /goodbye dialog.