I am using memcache.
I want to understand what is stored in Magento cache and how?
Do magento stores cache variable with website scope or store scope?
I have googled and greped the code but couldnt conclude anything,
Please if someone can direct me to correct links and path
Thanks & Regards,
Saurabh
If you go to the Cache Management section of the admin area you can see what it caches (configuration, layout configuration, block html output, translations, eav types, etc). I am no expert on Magento's caching mechanisms but here are a few random tidbits that might be helpful (maybe). (Also note that I am only familiar with Magento 1.3.x, not 1.4.x so things could have changed).
The caching is actually stored in the var/cache directory. There are a ton of directories in there (mage--0, mage--1, mage--2) and each directory has the cache files. Do a ls var/cache/mage*/* to see all the files.
Configuration - This source for the configuration is varied. Your app/etc/local.xml, and all of the config.xml files (that are in each module's etc dir) are combined together to make one big configuration object. Then Magento reads from the core_config_data table to update the configuration object. Then the configuration is written to a cache file so that next time a request is made it doesn't need to open a ton of config files and hit the database. Somehow this info gets stored in a bunch of files under var/cache. For some insight do a ls var/cache/mage*/*CONF*.
Layout - This is a lot like the configuration... there are a bunch of xml files in the app/design/frontendOrAdminhtml/yournamespace/layout/ directory and all these are merged into one layout configuration object, then cached in the cache directory.
Block HTML - The actual html generated by a block is cached. Each block is able to decide how long it is going to be cached.
Lastly, to (not really) answer your question about if the cache is per website or store, I can't really say since I haven't had the need to setup a multi-website/multi-store shop yet. It looks like there may be some store/website-specific files, but I can't see that they are really organized in a logical way. For example, in one of my instances I see a var/cache/mage--f/mage---LAYOUT_FRONTEND_STORE0_DEFAULT_BLANK_SEO file and a var/cache/mage--f/mage---LAYOUT_FRONTEND_STORE1_DEFAULT_BLANK_SEO... but then again, I only have one store configured and those two files have the same contents. Good luck with that!
You could also use some of the very great memcached analysis and reporting tools available
http://code.google.com/p/memcached/wiki/Tools
The best solution I have come up with is to use a two level cache.
Consult app/etc/local.xml.additional to see how to put memcached server nodes in there. Note that within the <servers> tag you will have to have tags like <server1> and <server2> encapsulating each memcached node's settings.
<cache>
<backend>memcached</backend>
<slow_backend>database</slow_backend>
</cache>
In this way all cache is shared.
To clear it the way I do it is to:
1. shut down apache
2. connect to mysql and connect to the magento db and run truncate core_cache; truncate core_cache_tag.
3. I then bounce the memcached nodes.
4. I restart apache but I keep it out of the load balancer until I have hit it at least once to generate the APC opcode cache. Otherwise the load can shot up through the roof.
This all seems extreme but I have found it works for me. Clearing cache using the backend is REALLY slow. I have around 100k entries in the core_cache table and close to 1 million entries in core_cache_tag. If I don't do it this way sometimes I get strange behavior.
Your Memcache configuration in ./app/etc/local/xml will dictate what Memcache is actually caching.
If you are only using a the single-level cache (without ), then Magento will store its cache (in its entirety) in Memcache.
HOWEVER without the slow_backend defined - it is caching content, without cache_tags - ie. without the ability to differentiate cache items
Eg. configuration, block, layouts, translations etc.
So, without the defined, you cannot refresh caches individually, in-fact, you'll almost always have to rely on "Flush Cache Storage" to actually see updates take effect.
We wrote a nice article here which covers your very issue - http://www.sonassi.com/knowledge-base/magento-knowledge-base/what-is-memcache-actually-caching-in-magento/
Memcached is a distributed memory caching system. It speeds up websites having large dynamic databases by storing database objects in Dynamic Memory to reduce the pressure on a server whenever an external data source requests a read. A Memcached layer reduces the number of times database requests are made.
The caching is actually stored in the var/cache directory. There are a ton of directories in there (mage--0, mage--1, mage--2) and each directory has the cache files. Do a ls var/cache/mage*/* to see all the files.
Configure Memcache Magento 2
Magento 2 also supports Memcached for caching objects but it isn’t enabled by default. You need to make simple changes to the $Magento2Root/app/etc/env.php file to enable it.
In env.php, you will see a large number of PHP arrays with different settings and configurations. Open the file in your favorite code editor and locate the following code:
array (
session' =>
'save' => 'files',
),
Modify this chunk as:
'session' =>
array (
'save' => 'memcached',
'save_path' => '<memcache ip or host>:<memcache port>'
),
Note that the default value for memcache ip is 127.0.0.1:11211. Similarly, the default value for memcache port is 11211.
For complete manual please look into it:
https://www.cloudways.com/blog/magento-2-memcached/
https://devdocs.magento.com/guides/v2.4/config-guide/memcache/memcache_magento.html
Related
Laravel 8 has config files for auth, mail, broadcasting, queue, services, session .etc, but I am not using those functionalities for my specific application.
Is it fine to leave the config files (and corresponding .env settings) untouched, or is it better to delete those files?
I am talking about both in terms of performance, execution correctness and security, but not in terms of code readability here. In other words I am talking about "real effects to my application".
Short answer:
it's not recommended to do something like that if you don't wanna go out from laravel's ecosystem.
TL;DR:
Since Laravel version 5.1 there was some general changes about the configurations. It means it for your v8 as well.
In every today's laravel project you have bootstrap/cache folder, which includes some bootstrap/cache/.gitignore file for ignoring other files. That ignored (3-4) files are actually cached files, which were creating automatically, and you can't do anything about that in-the-box.
The bootstrap/cache/config.php file is responsible for all the configurations, and it's creating as from config/*.php config files, as well as from all the config files from vendored dependencies. It means, everytime when you're using your laravel app, that file will be created automatically using
all the project-specific configurations
all the vendored configurations
environment variables from .env.
Note: there might be a case, when some in-box package (like laravel's default mail.php config file) or some 3rd-party package (unlikely, but anyway) could not have their own config file, so in vendor core codes there may be a confidence, that it can get some configs exactly from their apropriate config/*.php file, so in that case it's not recommended to delete that. For example, when you're deleting config/mail.php, and you're using that, in both cases you'll have approximately the same cached config, except some small mail-config-features (optimization there only approx. ~60 lines of human readable lines), but in that case you can't use laravel's Mailing functionality.
The sense is that when you want to override some configs, you're just creating some own config file(s) (in general with the same name as it's in vendor's appropriate location), so that laravel can do caching from config/*.php, but not from vendor/username/package/path/to/config.php.
So for optimization, laravel doing that caching process once, and after whole usage it will retrieve conifigs only from bootsrapped bootsrap/cache/config.php configs.
That's why everytime you're changing something in some config/*.php file or/and in .env file, you need to clean (manually) and it will be created again automatically
OR just:
there's a in-build command like:
# this will clean all the application caches
php artisan optimize
# or only for cleaning and recreating config
# this is an old-school version, but still used
php artisan config:cache
All this means, that you don't need to delete some config file from config/*.php, because laravel will check is the cached file exists, (if not, then it will create caches anyway from vendor configs), and all the time it will read them from cached files.
Conclusion: All this means, that
that is not related with security and there's nothing to do about that
if you feel that decreasing the cached bootstrap/config.php file content from 1566 lines to 1500 lines it's a way of optimization, then you're free to do that, but you should know that it could make some problems in future.
Subjective recommendation: in human language, i don't recommend to do something like that. Anyway you can do that, may you can improve something, that couldn't do another contributors yet. But it will not give a real effect to your app.
I'm using phpFastCache in a frontend-application, setting the path to the server's "/tmp/" directory:
phpFastCache::setup('path',"/tmp/");
I do not want to use phpFastCache's automatically found cache-directory, because it clutters my home directory with an extra directoy for every domain through which users are reaching the application (several are connected).
In the backend I would like to display cache-statistics and be able to clear the cache. This doesn't work anymore, now that I have set /tmp/ as the cache path. Statistics show up empty and the cache is not cleared. I did configure the cache-directoy to the same "/tmp/" in the backend-application as well.
How can phpFastCache be configured to be able to achieve this?
After looking at the phpFastCache-code, I'm able to answer my own question:
To achieve what I wanted (have only ONE cache-directory, regardless of domain used; be able to list statistics and clear cache from a separate application) I had to make two config-settings:
phpFastCache::setup('path', '/path-to-my-home-dir');
phpFastCache::setup('securityKey', 'phpfastcache');
I'm setting these identically in both my frontend- and backend-applications.
This will make phpFastCache use /path-to-my-home-dir/phpfastcache as its only cache-directory.
Had I not set the 'securityKey', phpFastCache would have generated one from the current domain (in most cases), therefore my backend application would have only "seen" that part of the cache residing in the directory for the currently used domain.
I am getting errors on my website and my website inodes count is overload. The hosting inodes limit is 200,000 but my website inodes count is 909,496 and I can't even open phpMyAdmin. The hosting support asked me to remove unused files. How can I decrease the inodes count and which files are unused in Magento based website?
Usually an indicator that you need a more capable hosting provider.
The major places that Magento creates files during operation are in the var/ folder and your product image cache.
If you've never checked before, the following areas can accumulate a phenomenal amount of detritus. Using an ftp client, check the following areas in your var/ folder:
Check that you don't have a bazillion sessions files in var/session, remove anything older than current date
Check that there aren't an excessive amount of files in var/report, you might want to find out why Magento is generating them and fix the issue. Delete them all.
Logging will generate over time several huge files in var/log, delete them and then look at the new ones to find out what errors are being generated.
Imports and other stuff can cause temporary files to accumulate in var/tmp, delete them. Also check in var/import for old imports that can be deleted
Stored database backups are kept in var/backup, using the admin backend System > Tools > Backups:
Download the latest database backups to a local workstation and delete all backups.
Magento uses a lot of caching to store information, the biggest will be the Image Cache if you have a large catalog, and it will contain cached images from the beginning of time, and lots of useless ones if you've deleted product over time. Using the Admin backend, go into System > Cache Management:
Clear the Magento Cache.
Flush Catalog Images Cache.
Magento does not delete product images when you delete product. In fact Magento would be a prime candidate for appearing on one of those Hoarder programs that were prevalent on TV there for a while.
After you get the site working, consider installing ImageClean.
Hopefully this will have reduced your inode count enough to carry out the following operations. Before proceeding, do a couple database backups and store off server!!!
Next step is to ask your hosting provider if they include your database in that inode table count. If they do, you are kind of stuck as Magento uses innodb and likely, they've cheaply not set up MySQL to use files-per-table so you can resize the innodb file size by optimizing each table. Ask them if they use files-per-table when they set up MySQL, if they don't know what it is, develop that sinking feeling in the pit of your stomach.
Some tables that get excessively huge, especially if you've haven't properly set set up the Magento master cron job trigger in your cPanel and checked to make sure log table cleaning is enabled in System > Configuration > Advanced > System > Log Cleaning. These tables are as follows:
'dataflow_batch_export',
'dataflow_batch_import',
'log_customer',
'log_quote',
'log_summary',
'log_summary_type',
'log_url',
'log_url_info',
'log_visitor',
'log_visitor_info',
'log_visitor_online',
'index_event',
'report_event',
'report_viewed_product_index',
'report_compared_product_index',
'catalog_compare_item',
'catalogindex_aggregation',
'catalogindex_aggregation_tag',
'catalogindex_aggregation_to_tag'
Magento has a built-in script to clean the logs. If running this crashes with a memory error because you've never set the cron job up and there's too much bloat to clean out, Crucial Web Host has a script that can be run to manually delete all log file contents. including the dataflow tables which won't be cleaned out by the Magento log cleaning process. If you use dataflow import/export a lot, Nexcess has a script that can check on the dataflow tables size and clear them as well.
After cleaning the database, you will need to use phpMyAdmin to optimize each table in your Magento database. If the hosting provider hasn't set up files-per-table in MySQL, it will do squat for reducing your inode count.
After all that, don't bother messing with deleting application files or anything else Magento uses. It doesn't really accumulate that much aside from the var/ folders and the Image cache and you likely will end up with a dead website.
At this point, you're at the mercy of a shared server hosting plan that has decided to be fair to everyone by limiting what can be done in each account and doesn't allow enough resources to run Magento. Start looking for a hosting provider that supports Magento, often they don't bother limiting your inode count (a cheap trick to allow too many people to share a hard drive) as they offer plenty of disk space for you to run your e-commerce website.
I'm working on building out a standard set of configurations for our cache clusters within App Fabric. My goal is to have a repeatable cache settings configuration when we load up a new environment (so server names are different, number of hosts, and other environmental factors).
My initial pass was to utilize the XML available from Export-CacheClusterConfig and simply change server names and size attributes in the <hosts> section, but I'm not sure what else is automatically registered with those values (the hostId parameter, for example).
My next approach that I've considered is a PowerShell script to simply build up the various caches with the correct parameters passed in that would simply run as a post-deploy step.
Anyone else have experience with repeatable AppFabric cache cluster deployments?
After trying both, the more successful option seems to be a combination of two factors. Management of the Cache Cluster (host information) is primarily an operations concern and is managed best by the operations team (i.e. those guys that read Server Fault). Since this information is stored in the configuration as well (and would require an XML file obtained from Export-CacheClusterConfig for each environment) it's best left to the operations team on how they want to manage it. Importing the wrong file (with the incorrect host information) has led to a number of issues.
So, we're left with PowerShell scripts. Here's a sample that I have. It could be cleaned up (check for Cache existence first) but you get the general idea. It's also much easier to store in source control (as it's just one file).
New-Cache -CacheName CRMTickets -Eviction None -Expirable false -NotificationsEnabled true
New-Cache -CacheName ConsultantCache -Eviction Lru -Expirable true -TimeToLive 60
New-Cache -CacheName WorkitemCache -Eviction None -Expirable true -TimeToLive 60
What is the best way to save data in session variables in a classic web site?
I am maintaining a classic web site and want to be able to allow my users to demo all functionality of the site, this means allowing them to delete records.
The closet example I have seen so far are the demos of Telerik controls where they are saving the dataset in sessions on first load and allowing the user to manipulate the data.
How can I achieve the same in ASP with an MS Access backend?
If you want to persist the state over multiple pages (e.g. to demo you complete application) then it's a bit tricky.
I would suggest copying the MDB file for each session and using the copied version. This would ensure that every session uses its own data.
create a version of your access db which will be used as a fresh template for each user
on session copy the template and name it after the users session ID
use the individual MDB
Note: Then only drawback I can see here is that you need to remove the unused MDB files as it can get a lot after sometime. You could do it with a scheduled task or even on session start before you create a new one.
I am not sure what you can use to check if it's used or not but check the files creation date or maybe the LDF file can help you as well (if it does not exist = unused).
You can store a connection or inclusive an object in a session variable as far you remember what kind of variable are you storing at the retrieving time. I had never stored a dataset in a session variable but I had stored a lot of arrays in session variables so you can use the ADO Getrows method to locate a complete dataset into a session variable.
How big is the Access database? If your database is small enough (relative to the server capacity, expected number of users, and so forth) then I like the idea of using a fresh copy of the database for each user that runs the demo.
With this approach, you simplify your possible code paths. Otherwise this "are we in demo mode or not?" logic will permeate a heck of a lot of your code.
I'd do it like this...
When the user begins the demo, make a copy of the Access DB for that user to use. If your db is foo.mdb, copy it to /tempdb/foo_1234567890.mdb where 1234567890 is the user's session ID.
Alter the user's connection string to point to the fresh database copy. From this point on, your app can operate like "normal" with no further modifications.
Have a scheduled task that deletes all files in /tempdb with last-modified times more than __ hours in the past. If you don't have the ability to schedule tasks on the server (perhaps you're in a shared hosting environment, etc) then you could do this at the same time you do step #1.