I'm try to improve Magento performances ( soon or later "MageDev" hit this point :)
I did some research and I found a lot of good, but not homogeneous, guides.
What I got is that:
MemCache or Redis are generic cache system, they cache data and they can be integrated directly with Magento (local.xml)
APC is a cache for the php code itself can only be integrated at server level.
Varnish is a reverse proxy, it cache the response can only be integrated at server level. ( there is a extension for Magento, turpentine, but I'm not sure what exactly does )
After all this good reading I'm still a bit confused about what of the above cache systems is possible to use in combinations, for EX:
MemCache + APC ?
Redis + APC ?
can I add Varnish to one of the above configuration ?
Just to be clear the question is not about how to configure Magento or the server but what
are the possibility allowed and some clearance about how to mix cache systems. ( beside that if anyone can come with a good recommendation I would appreciate it thanks. )
All these are different things, so they don't depend on each other.
APC caches the compiled code, MemCache stores data in memory and Redis is a persistent storage for different data structures.
Question is not clear, because you didn't point where is the bottleneck of your project
I was a bit tricky and I also posted the same question here: https://magento.stackexchange.com/questions/48003/magento-cache-confusion-about-varnish-redis-apc-memcache
I got an very good answer from #sonassi.
If the double question is a against Stack policy let me know and I will close this one.
Related
This question is directed towards Jeroen and is a follow-up to this answer: https://stackoverflow.com/a/12482918/177984
Jeroen wrote "the server does caching" .. "so if enough memory is available it will automatically be available from memory."
How can I confirm if an object is cached 'in-memory' or not? From what I can tell (by performance) all of my objects are being read from disk. I'd like to have things read from memory to speed up data load times. Is there a way to view what's in the in-memory cache? Is there a way to force caching objects in-memory?
Thanks for your help.
The OpenCPU project is rapidly evolving. Things have changed in OpenCPU 1.0. Have a look at the website for the latest information: http://www.opencpu.org.
The answer that you cited is outdated. Currently indeed all the caching is done on disk. In a previous version, OpenCPU used Varnish to do caching, which is completely in-memory. However this turned out to make things more complicated (especially https), and performance was a bit disappointing (especially in comparison with fast disks these days). So now we're back at nginx which caches on disk, but is much more mature and configurable as a web server, and has other performance benefits.
I am currently combing through the Magento Performance White paper to optimize our Magento experience before the start of the holiday season. For 2 months we get heavy traffic spikes and want to make sure we are running smoothly. We are using Magento EE 1.8 and by default utilizing Full Page Cache. In addition to FPC I have enabled GZIP, Cache-Control headers, and made all the mySQL tuning adjustments recommended by the white paper. We will also be employing a CDN to serve static content.
We are currently using filesystem cache and I am confused on where APC would fall into this stack, if at all. Is it worth installing APC when we are using FPC already? I know that the Magento cache and APC cache are 2 different things so if I edit products which invalidate the Magento cache and need a refresh, do I in turn have to refresh the APC cache each time? Any help is appreciated!
APC is opcode cache geared towards PHP itself, FPC is more geared towards caching of content geared towards Magento itself. You should not need to refresh APC when making any product changes. Only FPC will need to be refreshed upon a backend type of edit, Magento will normally notify you about it being invaldated. APC will only need to be refreshed if a PHP or template (.phtml) file has been modified or changed. Keep in mind that APC stores cache for both CLI and Web based differently so employing some type of wget or cURL request to a custom internal URL that will clear the cache is best.
For example something like
system('wget --spider --quiet http://localhost/apc/clear_apc_cache.php');
Also, I'd highly recommend setting Varnish reverse proxy server in front of the stack as its performance is highly noticable on high traffic systems. You can also use memcache to keep database loads on checkout or dynamic requests outside of Varnish or FPC.
Getting Varnish To Work on Magento
Hope this helps!
You could use APC for your 2 level caching (see _getBackendOptions() in Mage_Core_Model_Cache). Though APC isn't the best option for that.
Depending on whether you run a single or multiple webnodes you could either choose for memcache (multiple webnodes) or tempfs (single webnode). With memcache configured in Magento you can still use APC to cache opcodes.
After editing a product you only need to hit the 'flush cache storage' in the cache management admin to invalidate all entries in memcache. APC doesn't need any flushing.
Where are you currently storing your sessions? If it's also on disk, you might also want to migrate them to tempfs or memcache. Though, be cautious when using memcache: use a different port/pool for your sessions than the one that is used for FPC. Otherwise you would flush all sessions when flushing memcache, logging out all users and flushing their carts in the progress.
I would like to know if any of you had, or may propose a solution to increase efficiency for online store based on Magento platform.
We currently use multifront architecture (front == each separate server) using load-balancing and two Memcache servers.
We're considering connection for each separate front an Memcache server, but at this point a problem arises with memcache synchronization, so that each store the same value.
Any advice appreciated :)
If you are running memcached on its own hardware, there is no benefit to giving each store its own memcached server.
Configure all front ends to use both memcached instances. This way, all front ends will go to the same memcached instance for a given key. Plus, you get automatic fail-over if one instance croaks, and you can scale up almost linearly as the demand for cache increases.
One of the biggest improvements I've seen for more advanced setups like these is to use a PHP Opcode cache like Xcode. Since Magento uses many PHP files, the opcode cache will end up saving a lot of compilation between runs. Also make sure that all your caches are turned on and leveraged as much as possible. Enable flat catalog and flat product tables as well.
At work, we've recently started designing an application to me "large scale" (we're engineering for the potential to serve up many millions of hits a day). One of the senior devs and the sysadmin have set up memcache on the server.
As I understand it, Memcache will hold query results and certain tables in memory for X amount of time and keep everything hunky dory.
A drawback of memcache it seems is that I just can't for the life of me manage to set it up on my local dev environment. I've followed a few different instructionals on how to compile it for yourself. Most, if not all of the steps seem to work properly but get this error on PHPLoad:
[11-Sep-2010 16:02:30] PHP Warning: PHP Startup: Unable to load dynamic library '/Applications/MAMP/bin/php5.3/lib/php/extensions/no-debug-non-zts-20090626/memcached.so' - dlopen(/Applications/MAMP/bin/php5.3/lib/php/extensions/no-debug-non-zts-20090626/memcached.so, 9): image not found in Unknown on line 0
Not the primary question but incedentally, if you've been able to compile Memcache for MAMP 1.9 on Snow Leopard, please let me know the trick.
My primary question is about what the differences are between the various web caching technologies. I've seen mention of Memcache, APC and Xcache (here: Cache results of a mysql query manually to a txt file) but don't know the pros, cons and differences between each.
To my mind, Memcache has the advantage of being the one that the project's lead dev and our sysadmin chose. It has the disadvantage of being utter foobar to try and set up and compile on a Mac. :-^)
Anyone who I'd love to hear from anyone who can enumerate the pros and cons of each (or even one of) the other cachine technologies. Where are they best used, how are they best used. And so on.
It's all useful information I think.
Thanks so much for lending your time to expanding my knowledge.
- Alex.
First, a list of opcode cachers for php.
Second Memcache/MemcacheD is not an Opcode Cacher. It is a distributed memory caching system. It does not improve the speed/performance of your PHP code. It can be used to store data only.
APC, EAccelerator, XCache and the others are non distributed, meaning you can only store data on the local web-server. However all of these are opcode cachers and can improve the performance of your PHP app. Most, excluding EAccelerator (in the current version) can also store data.
I generally choose APC for the opcode cacher (It reportedly will be included into the core of PHP 6). However if I also have more than one web-server for the site I will also make use of MemcacheD.
Edit 1 I agree it is very annoying to setup APC, Memcache on MAMP. There are however tutorials out there dealing with such.
Edit 2 Also with regards to the best Opcode Cacher for your app really depends on which server you are using. Some work better on some systems. It also depends on the size and scale of your app as to how the cachers perform.
Edit 3 Very interesting article here about comparing performance of a few different cachers. (This article appears to be written in 2006 and should not really be used for current reference)
APC is a opcode cache. It will store parsed PHP code so that every time your PHP files do not need to get parsed.
Memcache is a data cache. It will store data as a key value pair.
We've been running eAccelerator on each of 3 webservers and are looking to move to a memcache pool across all 3, hopefully reducing by about 2/3 our db lookups.
One of the handy things about eAccelerator is the web-based control interface (control.php), which has proved very useful when we've had to flush the cache unexpectedly, quickly monitor which scripts are in cache, etc.
We've been looking but haven't found anything that offers the same type of functionality for memcache - does anyone know if such a thing exists?
Obviously flushing cache etc is easy enough with memcache on the console, but our particular set-up means we may have guys monitoring our front-end and needing to flush the cache who will not necessarily have shell access on the servers.
I know this is a late addition to an old question but none of the answers were a simple plain solution, so i created one and put it up on github for you to enjoy:
Screenshoots
memcache.php may be what you're looking for.
memcache.php that you can get stats and dump from multiple memcache servers.
Can delete keys and flush servers.
PHPMemcacheAdmin - http://code.google.com/p/phpmemcacheadmin/
If all you need to do is to be able to flush the cache from a web-application, you could create a simple php-page and then use the system() call...
Cache flushing is part of what we're looking for, but also a way to monitor what scripts are currently in there, how much data is in there, etc - basically the same stuff available on the EA control panel page.
We've played around with munin plugins for showing data usage, and were thinking we'd have to go down the line suggested above (system calls, etc), but were hoping that someone, somewhere would have rolled something similar already!