Have problem with loading homepage of my site. PHP execution time is about 14sec !!! Other pages like categories have 1,5sec to execution. Cache are enabled also I clean DB with no results.
The cache being enabled doesn't help if the blocks you have on your homepage aren't setup to be cached. If you have any custom blocks generating content on your homepage you should set the cache lifetime and override the getCacheKeyInfo. Especially if these blocks are doing large DB queries involving joins across many EAV tables. Obviously there's not much anyone can do with the limited information you've given, but start by looking at your homepage layout XML to see which blocks are being used, and investigate whether they are being cached.
Install FPC.
Pre-warm Cache.
Make sure if any listing then flat tables are used.
No block level code in .phtml
In short you need to know how to optimise magenta for performance
Solved problem with Full Page Cache Thanks!
Related
I am using Magnolia v5.7.1 and just configured the advanced cache module for site aware caching. Before that, the default behavior was to flush all caches in case of any (activation, import, edit) in a workspace. Using the advanced cache module, if any content on a specific site is changed, only the corresponding caches are flushed. So far, so good.
Now, let's say pages A and B are cached. If page A is changed, this will flush the cache for page A and B (as long as both pages are on the same site). I am wondering if there is a good reason that the default behavior isn't the following: If page A is changed, only the cache for page A gets flushed.
I know it's possible to implement my own FlushPolicy, however, this seems to be a difficult task and perhaps I miss a good reason why "page aware" caching can't be done.
The good reason to flush all is that change in one page might affect others. It is quite common to eg. generate menu from page strucuture, thus renaming one page would affect all pages that show menu. Or eg. have teaser component in a page that will take abstract text and image from the page it is teasing. And so on. In short, without calculating dependency graph there is no way for system to know which pages rendering might be affected by which other page changes. And in some cases it can be near impossible to know. Imagine event calendar page, with subpages for each event. And calendar being composed by query that searches for all events in current month. Since dynamic query is involved, calculating dependency graph just gets even more complicated.
That said it would still be possible to calculate dependencies and flush only affected pages, but in reality, for most cases, effort (cpu time) to calculate such graph is bigger than simply flushing all and re-rendering the pages as rendering is cheap (except special cases). On top of that it's also much faster to drop all items in the cache than retrieving them one by one and flushing only those that need to be flushed.
TLDR; For majority of the websites, it's not worth the effort to try to do very smart page cache management as costs outweigh the benefits w/o really affecting the performance.
We increased our products from 12k to 50k.
Now we're faceing a huge loading time issue. Sites are loading over 1-2 minutes. It seems like the first ttfb takes much to long and our developers can't find a solution/reason for that. someone knows a solution for this or someone who eveb could fix it? (paid)
Loading Time in Screenshot
Slow example url would be: https://gear2game.ch/unterhaltungselektronik/tv-home-cinema/tv.html
Also good to know, the homepage is fast, but all categories and products are slow :(
Best regards
Sandro
When the homepage is fast, but categories and products are slow, that is frequently an indication that way too much data is being loaded somewhere in a custom module, or you're not making use of indexers (which are a necessity with 50k products).
Slow ttfb also indicates that it's an issue on your backend (and given you just added a bunch of products, I'd say it's because some of the settings in Magento to handle large amounts of data are not enabled).
These are the things I'd start with:
Have you reindexed?
Are you using flat tables?
Are your caches enabled?
Are you using the Magento swatch module? This is notoriously bad with many products.
With a 50K products you should really think about Varnish.
Here are some useful links :
https://varnish-cache.org/
https://www.magentocommerce.com/magento-connect/turpentine-varnish-cache.html
I was simple cruising through the modx options and i noticed the option to cache snippets. I was wondering what kind of effect this would have (downsides) to my site. I know that caching would improve the loading time of the site by keeping them 'cached' after the first time and then only reloading the updates but this all seems to good to be true. My question is simple: are there any downsides to caching snippets? Cheers, Marco.
Great question!
The first rule of Modx is (almost) always cache. They've said so in their own blog.
As you said, the loading time will be lower. Let's just get the basics on the floor first. When you chose to cache a page, the page with all the output is stored as a file in your cache-folder. If you have a small and simple site, you might not see the biggest difference in caching and not, but if you have a complex one with lots of chunks-in-chunks, snippets parsing chunks etc, the difference is enormous. Some of the websites I've made goes down 15-30 levels to parse the content in come sections. Loading all this fresh from the database can take up to a coupe of seconds, while loading a flat-file would take only a few microseconds. There is a HUGE difference (remember that).
Now. You can cache both snippets and chunks. Important to remember. You can also cache one chunk while uncache the next level. Using Modx's brilliant markup, you can chose what to cache and what to uncache, but in general you want as much as possible cached.
You ask about the downside. There are none, but there are a few cases where you can't use cached snippets/chunks. As mentioned earlier, the cached response is divided into each page. That means that if you have a page (or url or whatever you want to call it), where you display different content based on for example GET-parameters. You can't cache a search-result (because the content changes) or a page with pagination (?page=1, ?page=2 etc would produce different output on the same page). Another case is when a snippet's output is random/different every time. Say you put a random quotes in your header, this needs to be uncached, or you will just see the first random result every time. In all other cases, use caching.
Also remember that every time you save a change in the manager, the cache will be wiped. That means that if you for example display the latest news-articles on your frontpage, this can still be cached because it will not display different content until you add/edit a resource, and then the cache will be cleared.
To sum it all up. Caching is GREAT and you should use it as much as possible. I usually make all my snippets/chunks cached, and if I crash into problems, that is the first thing I check.
Using caching makes your webserver respond quicker (good for the user) and produces fewer queries to the database (good for you). All in all. Caching is a gift. Use it.
There's no downsides to caching and honestly I wonder what made you think there were downsides to it?
You should always cache everything you can - there's no point in having something be executed on every page load when it's exactly the same as before. By caching the output and the source, you bypass the need for processing time and improve performance.
Assuming MODX Revolution (2.x), all template tags you use can be called both cached and uncached.
Cached:
[[*pagetitle]]
[[snippet]]
[[$chunk]]
[[+placeholder]]
[[%lexicon]]
Uncached:
[[!*pagetitle]] - this is pointless
[[!snippet]]
[[!$chunk]]
[[!+placeholder]]
[[!%lexicon]]
In MODX Evolution (1.x) the tags are different and you don't have as much control.
Some time ago I wrote about caching in MODX Revolution on my blog and I strongly encourage you to check it out as it provides more insight into why and how to use caching effectively: https://www.markhamstra.com/modx/2011/10/caching-guidelines-for-modx-revolution/
(PS: If you have MODX specific questions, I'd suggest posting them on forums.modx.com - there's a larger MODX audience there that can help)
i'm looking on performance (server load time) of magento site and i'm trying to tune search result pages. I realized that when I disabled all heavy things like top navigation, lev layered navigation and product listing and I cleared all cache then after this magento core does like 60 SQL queries agains a database. Does anyone have any procedure how to rid of them or how to reduce them to some acceptable amount?
Also can I somehow reduce a time spent during creating of blocks?
Thank you very much,
Jaro.
Magento is a extremely flexible ecommerce framework, but that flexibility comes with a price: performance. This answer is a collection of pointers and some details on caching (especially for blocks).
One thing to consider is the Magento environment, e.g. tuning the php, the web server (favor nginx over Apache), and MySQL. Also, set up a good caching backend for Magento. All these are covered e.g. in the Magento Performance Whitepaper that applies also to the CE.
After the environment is set up, the other side of things is the code.
Reducing the number of queries is possible for some pages by enabling the flat table catalog (System > Configuration > Catalog > Frontend), but you will always have a high number of queries.
You also can't really reduce the time spent creating the blocks except by tuning the environment (APC, memory, CPU). So as the other commenters said, your best choice is utilizing the caching functionality that Magento has built in.
Magento Block Caching
Because you specifically mentioned blocks in the question, I'll elaborate a bit more on block caching. Block caching is governed by three properties:
cache_lifetime
cache_key
cache_tags
All these properties can be set in the _construct() method of a block using setData() or magic setters, or by implementing the associated getter methods (getCacheLifetime(), getCacheKey(), getCacheTags()).
The cache_lifetime is specified in (integer) seconds. If it is set to false(boolean), the block will be cached for ever (no expiry). If it is set to nullthe block will not be cached (this is the default in Mage_Core_Block_Abstract).
The cache_key is the unique string that is used to identify the cache record in the cache pool. By default it is constructed from the array returned by the method getCacheKeyInfo().
// Mage_Core_Block_Abstract
public function getCacheKeyInfo()
{
return array(
$this->getNameInLayout()
);
}
public function getCacheKey()
{
if ($this->hasData('cache_key')) {
return $this->getData('cache_key');
}
/**
* don't prevent recalculation by saving generated cache key
* because of ability to render single block instance with different data
*/
$key = $this->getCacheKeyInfo();
//ksort($key); // ignore order
$key = array_values($key); // ignore array keys
$key = implode('|', $key);
$key = sha1($key);
return $key;
}
The best way to customize the cache key in custom blocks is to override the getCacheKeyInfo() method and add the data that you need to uniquely identify the cached block as needed.
For example, in order to cache a different version of a block depending on the customer group you could do:
public function getCacheKeyInfo()
{
$info = parent::getCacheKeyInfo();
$info[] = Mage::getSingleton('customer/session')->getCustomerGroupId()
return $info;
}
The cache_tags are an array that enable cache segmentation. You can delete sections of the cache matching one or more tags only.
In the admin interface under System > Cache Management you can see a couple of the default cache tags that are available (e.g. BLOCK_HTML, CONFIG, ...). You can use custom cache tags, too, simply by specifying them.
This is part of the Zend_Cache implementation, and needs to be customized far less frequently compared to the cache_lifetime and the cache_key.
Other Caching
Besides blocks Magento caches many other things (collection data, configuration, ...).
You can cache your own data using Mage::app()->saveCache(), Mage::app()->loadCache(), Mage::app()->cleanCache() and Mage::app()->removeCache(). Please look in Mage_Core_Model_App for details on these methods, they are rather straight forward.
You will also want to use a full page cache module. If you are using the Magento EE, you already have one. Otherwise search Magento Connect - there are many options (commercial).
Some of those modules also tune various parts of Magento for you beyond the full page caching aspect, e.g. Nitrogento (commercial).
Using a reverse proxy like Varnish is also very beneficial.
There are quite a number of blog posts on this subject. Here is one post by the publishers of the Nitrogento extension.
If you are running Magento on a more low-scale environment, check out my post on the optimization of the file cache backend on magebase.com.
I am adding additional comments for speed:
Instead of using Apache use nginx or litespeed.
Make sure flat catalog is used.
If possible use FPC.
compiler mode to be set on.
Merge css and js(Fooman Speedster ).
Use image sprites to reduce number of request.
Use query cache but avoid size greater then 64 MB.
Remove all modules not in use by removing there xml.Just disabling will not do.
Session to be on Ram.
Use of APC recommended.
Your cron should be run in offpeak hours.
Delete additional stores if not in use.
Delete cart rules if not in use.
optimize image for size.
Use Ajax where ever possible.
CMS blocks take more time then a magento block so unless you want a block to be modified do not use CMS blocks.
Do not use collection count use collection getSize to get what is the collection size.
Minimize number of searchable attributes as these result in columns in flat catalog table and will slow down your search.
Use of Solr search is recommended. It comes with EE version but it can be installed with CE as well.
Minimize customer group as suggested in comment.
Enable compression in .htaccess (mod_gzip for Apache 1.3, mod_deflate for Apache 2)
Remove staging stores if on EE.
Use Apache mod_expires and be sure to set how long files should be cached.In case you are on Apache server.
Use a Content Delivery Network (CDN).
Enable Apache KeepAlives.
Make your output W3C compliant
Use of getChildHtml('childName') is recommended as this will cache block against direct use of block code in .phtml file.
Make sure cron is run so as to clean logs stored in data base.
Number of days log should be minimized as per requirement.
Load cache on RAM if memory permits.
Reduce hard disc file reads and try reads from ram as this is faster.
Upgrade PHP version to above 5.3
If on EE make sure that most pages are delivered without application initialization.Even if one container needs application initialization its going to effect execution speed as apart form URL rewrites most of the other code will get executed.
Check XML for blocks placed in default handle and if those blocks not on specific page then move those XML values from default handle to specific handles.It has been observed that lots of blocks are executed that are not displayed.
If using FPC make sure your containers are cached and repeat request for container is delivered via cache.Improper placeholder definition results in container cache not being used but each time new container content getting generated.
Analyze page blocks and variables and if possible add those variables/blocks to cache.
Switch off Logs writing in Magento.
Remove Admin notification module.
Use of image sprites.
Use some web test tool to analyse number of requests and other html related parameters responsible for download time and act accordingly.
Remove attributes if not needed.With proper care we can even remove system attributes if not in use.
42: If on enterprise make sure partial indexing is effectively used.
Write your own solr search populate to bypass Magento search indexing.
Clean _cl tables or reduce _cl table rows.
I would add into list: try to avoid file cache if possible, replace it by apc / redis / memcache( As suggested by Jaro)
Remove system attributes not in use( Be careful,do a thorough check before removing).
There are some cron tab jobs that are not applicable to all stores so depending on your store features those can be removed.
Optimization by proper attribute management like setting required attribute to yes or is searchable or required in listing etc.
Some observers are not required for all stores so in case those observers are not applicable to a specific Magento site then they should be removed.
Make sure FPC is applicable to most of the site pages. Specially when you added some new controllers to delivering a page.
Magento has lots of features.For this it has many events and associated observers.There are few features that are not used by a store so any observer related to that feature should be removed.e.g : If you check enterprise version there is category permission concept which if not used, then its recommended that on save after events permission related observers to be removed.
If a specific attribute is to be save for a product then instead of call $product->save call a function that will save specific attribute.
In EE version that has partial indexing and triggers modify triggers to avoid multiple entries to_cl tables.
No.phtml files bypasses blocks and use modules or resources directly.As this will result in no caching which in-turn means more work for Magento.
Delivering images depending on device in use.
Some of the FPC recommended for community : Lesti( Free as on date ),Amasty( commercial),extender(commercial ) and Bolt(commercial).
Warming Cache.
Controlling bots by .htaccess during peak hrs.
Pre-populating values in a custom table for Layered Navigation via a custom script that executes daily by cron.
Making sure to avoid unwanted Keys to reduce cache size.
Using a higher PHP version 5.4+
Using a higher Mysql version( 5.5 +)
Reduce number of Dom elements.
Move all js out from html pages.
Remove commented html.
Modify triggers if on enterprise version(1.13 or higher) so as to reduct _cl table entries.As these entries results in cache flushing which in turn results in lower cache hit,hence more TTFB time.
Use Magmi to import products.
As Vinai said, Magento is all about extensibility and raw performance is secondary but remedied by things like indexing and caching. Significantly improving performance without caching is going to be very difficult. Short of full-page caching, enabling block caching is a good method of improving performance but proper cache invalidation is key. Many blocks are cacheable but not already configured to be cached by default so identify the slowest ones using profiling use Vinai's guide for enabling caching. Here are a few additional things to keep in mind with block caching:
Any block that lists product info should have the product's tag which is 'catalog_product_'.$productId. Similarly, use 'catalog_category_'.$categoryId for categories. This will ensure the cache is invalidated when the product or category is saved (edited in backend). Don't set these in the constructor, set them in an overridden getCacheTags() so that they are only collected when the block is saved and not when it is loaded from cache (since that would defeat the purpose of caching it).
If you use https and the block can appear on an https page and includes static resources, make sure the cache key includes Mage::app()->getRequest()->isSecure() or else you'll end up with http urls on https pages and vice versa.
Make sure your cache backend has plenty of capacity and avoid needless cache flushes.
Don't cache child blocks of a block that is itself cached unless the parent changes much more frequently than the child blocks or else you're just cluttering your cache backend.
If you do cache tagging properly you should be able to use a very long default cache lifetime safely. I believe setting "false" as the lifetime actually uses the default, not infinite. The default is 7200 seconds but can be configured in local.xml.
Using the redis backend in most cases will give you the best and most consistent performance. When using Redis you can monitor used memory size using this munin plugin.
Just to follow on from Mark... most of the tables in the Magento database are InnoDB. Whilst the query cache can be used in a few specific places, the following are more directly relevant...
innodb_buffer_pool_size
innodb_thread_concurrency
innodb_flush_method
innodb_flush_log_at_trx_commit
I also use
innodb_file_per_table
as this can be beneficial in reorganising specific tables.
If you give the database enough resource, (within reason) the amount of traffic really doesn't load the server up at all as the majority of queries are repeats anyway, and are delivered out of database cache.
In other words, you're probably worrying about nothing...
Make sure mysql query cache is turned on. And set these variables in mysql (maybe need tweaking depending on your setup).
query_cache_type=1
query_cache_size=64M
i found a very interesting blog post about Magento Performance optimization, there are many configuration settings for your server and your magento store, was very helpful for me.
http://www.mgt-commerce.com/blog/magento-on-steroids-best-practice-for-highest-performance/
First you need to audit and optimize time to first byte (TTFB).
Magento has profiler built-in that will help you identify unoptimized code blocks.
Examine your template files and make sure you DO NOT load product models inside a loop (common performance hog):
foreach($collection as $_product){
$_product = Mage::getModel('catalog/product')->load($_product->getId()
I see this code often in product/list.phtml
I wrote a step-by-step article on how to optimize TTFB
Disclaimer: the link points to my own website
I'm currently using CakePHP for my training plan application but I have rendering times of 800 ms per page.
Do you have any tips on improving the performance?
Thanks in advance.
br, kms
In general the tool that will give you the greatest speed boost is caching, and CakePHP has a dedicated helper. The second most useful thing to do is optimizing the queries, only asking for the data you need - this can be done with the containable behavior.
You can find more detailed tips in this post.
Install APC, if you dont have it. that will instantly make it < 500ms. also without touching a single line of code.
Make sure your tables have all the proper indexes so that queries are as fast as they can be.
Next up look at some things about caching routes / urls as that is a huge drain.
Those will give you the most speed boost for the least amount of work
Have you tried any of the CSS/JS asset combiners for CakePHP? They combine/compress/minify your CSS/JS scripts and cache them where applicable. Here's one that's quite recent.
Not specific to CakePHP but you could go through all the factors in Google Page Speed, it will help you speed up your page loading times by suggesting what scripts you could combine and advice on how to reduce requests.
Other than that, look into the containable behaviour, see if you can cut out any necessary info/queries by only selecting what you need at any time.
This question is well-populated with information on speeding Cake up.