I am currently combing through the Magento Performance White paper to optimize our Magento experience before the start of the holiday season. For 2 months we get heavy traffic spikes and want to make sure we are running smoothly. We are using Magento EE 1.8 and by default utilizing Full Page Cache. In addition to FPC I have enabled GZIP, Cache-Control headers, and made all the mySQL tuning adjustments recommended by the white paper. We will also be employing a CDN to serve static content.
We are currently using filesystem cache and I am confused on where APC would fall into this stack, if at all. Is it worth installing APC when we are using FPC already? I know that the Magento cache and APC cache are 2 different things so if I edit products which invalidate the Magento cache and need a refresh, do I in turn have to refresh the APC cache each time? Any help is appreciated!
APC is opcode cache geared towards PHP itself, FPC is more geared towards caching of content geared towards Magento itself. You should not need to refresh APC when making any product changes. Only FPC will need to be refreshed upon a backend type of edit, Magento will normally notify you about it being invaldated. APC will only need to be refreshed if a PHP or template (.phtml) file has been modified or changed. Keep in mind that APC stores cache for both CLI and Web based differently so employing some type of wget or cURL request to a custom internal URL that will clear the cache is best.
For example something like
system('wget --spider --quiet http://localhost/apc/clear_apc_cache.php');
Also, I'd highly recommend setting Varnish reverse proxy server in front of the stack as its performance is highly noticable on high traffic systems. You can also use memcache to keep database loads on checkout or dynamic requests outside of Varnish or FPC.
Getting Varnish To Work on Magento
Hope this helps!
You could use APC for your 2 level caching (see _getBackendOptions() in Mage_Core_Model_Cache). Though APC isn't the best option for that.
Depending on whether you run a single or multiple webnodes you could either choose for memcache (multiple webnodes) or tempfs (single webnode). With memcache configured in Magento you can still use APC to cache opcodes.
After editing a product you only need to hit the 'flush cache storage' in the cache management admin to invalidate all entries in memcache. APC doesn't need any flushing.
Where are you currently storing your sessions? If it's also on disk, you might also want to migrate them to tempfs or memcache. Though, be cautious when using memcache: use a different port/pool for your sessions than the one that is used for FPC. Otherwise you would flush all sessions when flushing memcache, logging out all users and flushing their carts in the progress.
Related
I suddenly got a spike in CPU usage on shared hosting and my website has been taken offline for the past 48 hours. I have already tried the following, and it did not help (or only a bit):
Updated Drupal 7 and all modules to the latest version
activated CloudFlare
disabled any unused modules
Caching was already on (min cache lifetime: none; expiration cached pages: 1 hour; bandwidth optimization: all options checked). I had no problems before, and visitor numbers have remained fairly constant at 1500-2000 a day. Could it be hacked? Spam? The logs only show lots of Google bot "access denied" activity, but somehow doubt that can take the site down.
Thanks in advance for any advice!
Use boost module, this will help you improve the performance by creating HTML files.
Boost provides static page caching for Drupal enabling a very
significant performance and scalability boost for sites that receive
mostly anonymous traffic. For shared hosting this is your best option
in terms of improving performance. On dedicated servers, you may want
to consider Varnish instead.
Apache is fully supported, with Nginx, Lighttpd and IIS 7
semi-supported. Boost will cache & gzip compress html, xml, ajax, css,
& javascript. Boosts cache expiration logic is very advanced; it's
fairly simple to have different cache lifetimes for different parts of
your site. The built in crawler makes sure expired content is quickly
regenerated for fast page loading.
I'm try to improve Magento performances ( soon or later "MageDev" hit this point :)
I did some research and I found a lot of good, but not homogeneous, guides.
What I got is that:
MemCache or Redis are generic cache system, they cache data and they can be integrated directly with Magento (local.xml)
APC is a cache for the php code itself can only be integrated at server level.
Varnish is a reverse proxy, it cache the response can only be integrated at server level. ( there is a extension for Magento, turpentine, but I'm not sure what exactly does )
After all this good reading I'm still a bit confused about what of the above cache systems is possible to use in combinations, for EX:
MemCache + APC ?
Redis + APC ?
can I add Varnish to one of the above configuration ?
Just to be clear the question is not about how to configure Magento or the server but what
are the possibility allowed and some clearance about how to mix cache systems. ( beside that if anyone can come with a good recommendation I would appreciate it thanks. )
All these are different things, so they don't depend on each other.
APC caches the compiled code, MemCache stores data in memory and Redis is a persistent storage for different data structures.
Question is not clear, because you didn't point where is the bottleneck of your project
I was a bit tricky and I also posted the same question here: https://magento.stackexchange.com/questions/48003/magento-cache-confusion-about-varnish-redis-apc-memcache
I got an very good answer from #sonassi.
If the double question is a against Stack policy let me know and I will close this one.
I am new with Magento and have developed a website using CE 1.7.0.2. Now its ready to go live but I have issues with slow page load.
My website product home, list and detail page initially takes time to 10-13 sec to page load but after that first load it only takes 1-2 second to page load.
Also I have installed APC, Memcache and CDN on server and full page cache extension for website but yet it is slow. I am so frustrated why this happening with my website?
If anybody knows how can we resolve speed up issues that will be helpful for me.
Thanks!
You can go through the below steps for Magento Optimization:
High Performance Dedicated Server. Ex: Amazon EC2 cloud
Swap Apache for NGINX
Minimize Javascript use
Minify and Compressed CSS files
‘Combine CSS’ seeks to reduce the number of HTTP requests made by a
browser
Optimize images
Use lazyload for images
Specify Image dimensions
Combine images into CSS sprites
Use a Content Delivery Network (CDN) for delivering static files
like JS, CSS and Images to offload your server
Disable modules/extension which are not required
Enable all Magento Caches
Use a Full Page Cache / Varnish Cache / Memcache / RedisCache
Don’t use layered navigation if you don’t really need it, it needs
a lot of resources
Enable Compilation
Limit the number of products on a product overview page.
Set only those attribute frontend properties to ‘Yes’ that you’re
actually going to use. Set all other to ‘No’.
Don’t use in quick search, advanced search compare, etc. : Catalog
-> Attributes -> Manage Atributes -> Frontend Properties.
Install Google Page Speed Module
Minimize redirects – Minimizing HTTP redirects from one URL to
another cuts out wait time for users.
Prefer asynchronous resources – Fetching resources asynchronously
prevents those resources from blocking the page load.
This list may help you for Magento performance improvement:
Enable Magento caching
This is ofcourse the first step in optimization: Enable all the available caches in the Magento Admin Panel.
Compress images
Many people forget that images (PNG, JPG) can be compressed, which lowers the bandwidth between the browser and the webserver. Not only the images used by the Magento skin need optimizing, but catalog images as well. Various tools allow you to compress batches of images, for instance the online tool Smush.It.
Disable unneeded Magento modules
By disabling Magento modules that you do not need, less resources are needed – as simple as that. Modules could be disabled through the configuration in the Magento Admin Panel, or by editing XML-files in app/etc/modules. For instance, disable Mage_Log which performs queries on every request, but is not needed if you gather site statistics using external programs.
Enable flat catalogs for smaller webshops
For smaller webshops switching from the complex EAV-structure to a flat catalog could save time. This optimization is dubious and depends on many parameters, so do not take this step lightly.
W3C compliance
While it could be argued that this is less important with the coming of HTML5, it is still a fact that if your webpages are filled with ugly errors, the browser will have a harder time interpreting it. If you stick to W3C compliance, it is made sure the browser engine has an easy job parsing your HTML-code.
Compress output in general
By enabling the PHP-setting zlib.output_compression the output generated by PHP is compressed when sent to the browser. This saves bandwidth. Instead of using this, you could use the Apache mod_deflate module as well, which allows also for compression of non-PHP output (CSS, JavaScript, other plain text-files).
Configure PHP options
Most PHP settings actually do not influence the performance of Magento, but just set certain limits. For instance, settings like memory_limit and max_execution_time do not increase the page load but just make sure that certain actions do not timeout or run into memory problems.
Session storage
With Magento, sessions could be stored in files or in the database (by configuring app/etc/local.xml). Which option performs best, really depends on how the hosting environment is setup. If MySQL databases perform well, session storage in the database could benefit your site. But if MySQL is not setup correctly, the best choice might be files.
Use a PHP accelerator
By opcode caching, PHP-execution could be fastened. There are various PHP accelerators doing this job (APC, ZendOptimizer+, eAccelerator, XCache). Both APC and ZendOptimizer+ are working flawless with Magento.
Tune PHP realpath_cache
By tuning the PHP realpath_cache_size to for instance 128K (default is 16K) and the realpath_cache_ttl to 86400, things might be speeding up. Make sure you don’t run out of memory, because every Apache child will consume the configured caching size.
Use Apache mod_expires
By telling the browser which files to keep in cache for how long, you can optimize the browser cache. For instance, JavaScript files tend to change much less then CSS files (at least in the first stages of the site), but perhaps after the site is running smooth for some months you can maximize the expiration date.
Beware 404 errors
Every time a file (like a stylesheet or image) is not found, this generates a 404 error. Because the default 404 of Magento is caught by the application of Magento itself, this causes the Magento application to start for every 404 encountered. Check your Apache logs to make sure all 404 errors are solved.
Disable Magento logging
Within the Magento configuration, you can enable logging under the Developers-tab. Depending on the Magento modules this could lead to more logs needed to be written to the filesystem, slowing down your shop. Because the logging-abilities are only needed to debug something, in daily usage, it’s best to disable logging altogether.
MySQL table optimization
Through phpMyAdmin, you can perform the command OPTIMIZE TABLE on specific Magento database-tables. When a table is cluttered, this could lead to improved performance. This does not only count for the complex EAV-tables, but also for regular MySQL tables that are used frequently (for instance, core_config_data).
Merge CSS and JavaScript
By merging all CSS and JavaScript files together as one big file, only one single HTTP-request is needed by the browser to fetch this content. This saves bandwidth. For this merging, the FooMan Speedster module could be used. Magento 1.4 contains an option to merge CSS, while JavaScript-files are merged by default.
Besides merging, crunching is also an option offered by FooMan Speedster: It removes whitespaces from the output, but when compression is already applied to CSS, this option is less needed.
Use Magento Compiler module
The Magento Compiler module limits the number of directories that PHP has to search through when looking for PHP-files. This decreases the PHP execution-time, which speeds up the Magento application in general.
Be warned that you need to be careful when making changes to your Magento system, while the Magento Compiler is enabled. Upgrades should only be undertaken when the compiler is (temporarily) disabled.
One very neat trick that speeds up things tremenduously is to create a tmpfs-mount specifically for the includes/src folder. Note that this tmpfs-mount needs to be at least 100Mb – preferably 200Mb.
MySQL server tuning
The default MySQL setup is a lot of times sufficient to run a general hosting environment, but not all optimized for Magento. Tuning settings like query_cache_size could dramatically increase performance, but is also dangerous because it hugely depends on other variables (number of databases, number of tables per database, number of queries, peak usage).
Serve static content through a CDN
Static content like images, CSS-stylesheets or JavaScript-files, could be served through other servers that are more optimized for static content. For instance, a CDN could be used so that the static content is always served from a location that is closest to the visitor. This is vital for webshops serving customers worldwide.
Disable local Magento modules
If your site does not need local Magento modules, you could choose to skip the search for local modules alltogether. Within the app/etc/local.xml file, you will find an XML-tag allowing you to do so.
Be carefull with HTTPS
Every time you use SSL between webserver and browser, the process of encrypting and decrypting is added on both sides. Also there is a slight overhead in bandwidth. The Magento site runs slightly faster if you disable SSL for all or just a few pages. However, this “win” is so small compared to the other wins on this page, that it should only be handled with caution. The gained bandwidth is non-vital, while almost all computers nowadays have CPU-power with which the encryption/decryption process takes place in microseconds.
Magento in the cloud
While CDNs could be used to optimize the bandwidth for static content, the Magento application could also be optimized in the same way by using cloud computing.
Memory-based filesystem for dynamic data
By storing dynamic data (var/cache, var/session) on a memory-based filesystem like RAMdisk or tmpfs, the disk I/O is decreased.
Disable Apache htaccess-files
When allowing the usage of htaccess-files, Apache needs to inspect every directory in its path to see if an htaccess-file is present. By moving the Apache configuration-directives from the htaccess-file to the VirtualHost configuration-file, and disabling htaccess-files all together, the Apache execution-time is decreased. This tip probably applies in most cases only to dedicated servers.
Use Nginx or Litespeed
While the Apache webserver is very flexible in its configurations, there are other webservers that are better optimized regarding memory usage: By replacing Apache with either Nginx or Litespeed, you could speed up the Magento scripts even more. Both webservers require manual configuration to allow for SEF URLs.
Use lazyload for images
When a page is loading, a visitor is often waiting for images on that page to load. Depending on the number and size of these images, this can take some time. Instead of loading images at once when the page is loaded, you can also add the LazyLoad JavaScript effect that makes sure only visible images (within the browser screen) are loaded, while remaining images are only loaded once the visitor scrolls down.
Minimize Apache logging
If Apache logging is minimized, less file operations are needed for every incoming request. Ofcourse less logging also means less insight when something goes wrong. An alternative is to optimize the filesystem on which Apache logs are stored. By default, Apache logs to the /var filesystem – but there’s no need to enable things like journalling for that filesystem.
ref: http://magentotutorialbeginners.blogspot.in/2014/05/magento-performance-improvement.html
If you have done all kind of server and caching optimization.Go to code level.
1) See are you loading a collection with in a foreach loop.
2) Try to optimize the code.
3) If you are loading a collection, filter the collection for the required attributes only.
4) Check for your product images.Use png images and try to keep images size under 500 KB.
5) Try commenting the custom functionality that you are providing on that page.And check with some tool like GTMetrix. How much time you achieve to load the page.Try to find out the code which is taking long time to load.
6) Keep only necessary attributes used for filter purpose in Layered navigation.
7) Try disabling unnecessary modules.
8) Try after enabling the compilation.
Hope these suggestions will work for you.
I am trying to improve the load speed of my Magento store http://www.jinkou.info the load time for the front page on pingdom tools is 13 seconds! It seems that the delay in the pingdom readout is the green section after "connect"
What is likely to be causing this bottleneck?
First of all, you need to make sure that Magento cache is enabled. You can do it in Admin -> System -> Cache Management. All cache data should be enabled there.
If it didn't help, you need to enable Profiler, and this will help you to see bottlenecks as a table in the footer. You can do it in 2 steps:
Go to Admin -> System -> Configuration -> Developer -> Debug, and enable Profiler.
Edit Magento's index.php file, and uncomment this line:
Varien_Profiler::enable();
After steps 1 and 2 has been done -- go to any page, a look at the footer.
Magento certainly does like to use the database a lot and every query means a roundtrip to the disc while nothing else is being done. Make sure MySQL has query cache turned on and has lots of memory assigned to it, for large sites several gigabytes are needed. This is why a dedicated server is best.
From tests I find that the biggest CPU cost is running PHP. Even when using an opcode cache it is still an interpreted language and that is expensive. Run PHP as a separate FastCGI process. If following the traditional three-tier approach this would also be a dedicated server.
Only use Memcache if you have more than one PHP server, otherwise just mount var/cache/ as tmpfs.
Lastly as rpSetzer has said - and no doubt others will say - use block caching.
More block caching.
Page caching.
Hard to say, here are a few tips that can certainly help:
use APC
use Memcache for caching
make use of block caching
Magento caching must be enabled when the site gets loaded. But what really makes a difference is a PHP cache. We use APC and its a major difference.
Also make sure to tune your database! MySQL tuning makes a uge improvement. Google for MySQL tuning and you will find.
with the help of magento speed analyzer module you can able to find each block load time.
with the help of speed analyzer module you can able to find each block load time.
http://www.magentocommerce.com/magento-connect/speed-analyzer.html
I have a server which is running more than 15 Magento stores, but they are not performing well, though I have a giant server for hosting them. My server configuration is - 8 CPU's Quad Core 24GB RAM and 2 TB HDD.
My current page load is 1.6sec. I want it under 600ms. I have already installed APC, & eAccelerator and tuned Apache's parameters. I am using the latest Magento version.
Please suggest.
-Ramesh
You might look at enabling block caching as explained here. It should work pretty well on the category and product pages but you have to be extra careful to apply proper cache tags and identifiers to make sure the content you are displaying is always up to date...
First things first, what is actually bottlenecked? Optimization is always about tradeoffs, and you may just make things worse if you're looking in the wrong places. Make use of top (assuming you're on Linux here) and see what your processor/memory usage look like.
I'm going to take a stab in the dark here and say that, if you have already added an opcode, you may be waiting on other HTTP requests for page load. Use YSlow on Firefox and see if you are trying to load excessive amounts of data. Optimizing image size and setting proper caching parameters for images may solve the problem there.
If not, silvo's suggestion is a very good one. Using either block-level or page-level caching can really speed up a site. This topic has been covered previously, so see those posts, too.
Hope that helps!
Thanks,
Joe
I'm not sure that you will see a benefit of using APC and eAccelerator on the same server. They pretty much do the same thing.
A page load of 1.6 seconds is fairly typical for a Magento install. They easiest way to lower your page load time (after basic Apache and MySQL tuning and APC) is use a Full Page Cache. There are a few out on the market right now. We have written a Full Page Cache that has gotten page loads down to the .1 - .3 second range for most users, http://ecommerce.brimllc.com/performance/full-page-cache-magento.html