Let's say I have a bunch of users who all access the same set of files, that have permission system:anyuser. User1 logs in and accesses some files, and then logs out. When User2 logs in and tries to access the same files, will the cache serve the files, or will it be cleared between users?
The cache should serve the files (in the example above).
How long a file will persist in the OpenAFS cache manager depends on how the client is configured, variables include the configured size of the cache, whether or not the memcache feature is enabled, and how "busy" the client is.
If OpenAFS memcache (cache chunks stored in RAM) is enabled, then the cache is cleared upon reboot. With the more traditional disk cache, the cache can persist across reboots. Aside from that key difference files persist in the cache following the same basic rules. The cache is a fixed size stack, recently accessed files stay in the cache and older files are purged as needed when newer files are requested.
More details are available in the OpenAFS wiki:
http://wiki.openafs.org/
Related
Is there a need to have Redis snapshots when only using it as a session replication service with Spring Session? I read about Redis Persistence but it seems "persistence" only means backups, and that it is not fully required.
I have a problem in my application that no matter how many times I will call FLUSHALL, it will keep reloading old sessions somehow. I suspect from my RDB file.
Can I just define everything as memory only? Is there any reliability/performance benefits to use an RDB file at all?
I have a problem in my application that no matte how many times I will call FLUSHALL, it will keep reloading old sessions somehow. I suspect from my RDB file.
FLUSHALL will also remove data from RDB file. I think the data might be written by other processes after you called FLUSHALL.
Can I just define everything as memory only?
Yes, you can disable saving db to disk with config file. By default, Redis will save data to disk with some save directives in the config file. For example:
save 900 1
save 300 10
save 60 10000
To disable saving, just comment these save directives or add an empty save directive (i.e. save "").
# save 900 1
# save 300 10
# save 60 10000
Also you can disable AOF persistence logs with config file:
appendonly no
Is there any reliability/performance benefits to use an RDB file at all?
Reliability: Since Redis saves your data to disk, you can recover the data, when you restart Redis. However, if Redis is down accidently, you might lose some data, depends on your save directives in config file.
Performance: There's a performance penalty, especially when you are using RDB persistent with big data in memory.
You can get more details on Redis Persistence from its website.
We are using AppFabric Caching services with local cache enabled.
the in-memory operations being performed on data (acquired from Cache) seem to be getting saved in local cache (without explicitly placing the updated objects in cache).
What I've read in documentation, local cache just holds a local copy and there are mechanisms to invalidate local Cache as well.
What can I do in order to over-write this local-cache behavior (as per my initial understanding, local-cache contents are read-only, which does not seem the case here)
this the configuration being used:
DataCacheFactoryConfiguration configuration = new DataCacheFactoryConfiguration();
configuration.TransportProperties.MaxBufferSize = int.MaxValue;
configuration.TransportProperties.MaxBufferPoolSize = long.MaxValue;
configuration.MaxConnectionsToServer = MaxConnectionsToServer;
DataCacheServerEndpoint server = new DataCacheServerEndpoint(host, port);
List<DataCacheServerEndpoint> servers = new List<DataCacheServerEndpoint>();
servers.Add(server);
configuration.Servers = servers;
DataCacheLocalCacheProperties localCacheProperties = new DataCacheLocalCacheProperties(MaxObjectCount, LocalCacheTimeout, DataCacheLocalCacheInvalidationPolicy.TimeoutBased);
configuration.LocalCacheProperties = localCacheProperties;
How can I overwrite this behavior of local cache (not using local cache is not an option due to lot of read operations going on) ?
Thanks in advance,
I think this is explained in the documentation (emphasis added):
Locally cached objects are stored within the same process space as the
cache client process. When a cache client requests a locally cached
object, the client receives a reference to the locally cached object
rather than a copy
and
After objects are stored in the local cache, your application
continues to use those objects until they are invalidated, regardless
of whether those objects are updated by another client on the cache
cluster. For this reason, it is best to use local cache for data that
changes infrequently.
I am getting errors on my website and my website inodes count is overload. The hosting inodes limit is 200,000 but my website inodes count is 909,496 and I can't even open phpMyAdmin. The hosting support asked me to remove unused files. How can I decrease the inodes count and which files are unused in Magento based website?
Usually an indicator that you need a more capable hosting provider.
The major places that Magento creates files during operation are in the var/ folder and your product image cache.
If you've never checked before, the following areas can accumulate a phenomenal amount of detritus. Using an ftp client, check the following areas in your var/ folder:
Check that you don't have a bazillion sessions files in var/session, remove anything older than current date
Check that there aren't an excessive amount of files in var/report, you might want to find out why Magento is generating them and fix the issue. Delete them all.
Logging will generate over time several huge files in var/log, delete them and then look at the new ones to find out what errors are being generated.
Imports and other stuff can cause temporary files to accumulate in var/tmp, delete them. Also check in var/import for old imports that can be deleted
Stored database backups are kept in var/backup, using the admin backend System > Tools > Backups:
Download the latest database backups to a local workstation and delete all backups.
Magento uses a lot of caching to store information, the biggest will be the Image Cache if you have a large catalog, and it will contain cached images from the beginning of time, and lots of useless ones if you've deleted product over time. Using the Admin backend, go into System > Cache Management:
Clear the Magento Cache.
Flush Catalog Images Cache.
Magento does not delete product images when you delete product. In fact Magento would be a prime candidate for appearing on one of those Hoarder programs that were prevalent on TV there for a while.
After you get the site working, consider installing ImageClean.
Hopefully this will have reduced your inode count enough to carry out the following operations. Before proceeding, do a couple database backups and store off server!!!
Next step is to ask your hosting provider if they include your database in that inode table count. If they do, you are kind of stuck as Magento uses innodb and likely, they've cheaply not set up MySQL to use files-per-table so you can resize the innodb file size by optimizing each table. Ask them if they use files-per-table when they set up MySQL, if they don't know what it is, develop that sinking feeling in the pit of your stomach.
Some tables that get excessively huge, especially if you've haven't properly set set up the Magento master cron job trigger in your cPanel and checked to make sure log table cleaning is enabled in System > Configuration > Advanced > System > Log Cleaning. These tables are as follows:
'dataflow_batch_export',
'dataflow_batch_import',
'log_customer',
'log_quote',
'log_summary',
'log_summary_type',
'log_url',
'log_url_info',
'log_visitor',
'log_visitor_info',
'log_visitor_online',
'index_event',
'report_event',
'report_viewed_product_index',
'report_compared_product_index',
'catalog_compare_item',
'catalogindex_aggregation',
'catalogindex_aggregation_tag',
'catalogindex_aggregation_to_tag'
Magento has a built-in script to clean the logs. If running this crashes with a memory error because you've never set the cron job up and there's too much bloat to clean out, Crucial Web Host has a script that can be run to manually delete all log file contents. including the dataflow tables which won't be cleaned out by the Magento log cleaning process. If you use dataflow import/export a lot, Nexcess has a script that can check on the dataflow tables size and clear them as well.
After cleaning the database, you will need to use phpMyAdmin to optimize each table in your Magento database. If the hosting provider hasn't set up files-per-table in MySQL, it will do squat for reducing your inode count.
After all that, don't bother messing with deleting application files or anything else Magento uses. It doesn't really accumulate that much aside from the var/ folders and the Image cache and you likely will end up with a dead website.
At this point, you're at the mercy of a shared server hosting plan that has decided to be fair to everyone by limiting what can be done in each account and doesn't allow enough resources to run Magento. Start looking for a hosting provider that supports Magento, often they don't bother limiting your inode count (a cheap trick to allow too many people to share a hard drive) as they offer plenty of disk space for you to run your e-commerce website.
I am new to symfony and am responsible for a site that I didn't build. For some reason the site is on a live server but running in dev mode. - Im not sure why??
That aside - The website keeps writing _sess files to my /tmp directory. The contents of each _sess file is exactly the same. See below:
_symfony2|a:3:{s:10:"attributes";a:0:{}s:7:"flashes";a:0:{}s:6:"locale";s:2:"en";}
Do I really need all of these files? Can anyone suggest a way of disabling this feature?
Thanks in advance
The default session storage of Symfony2 writes the session information to file(s). The location these files are written to is determined by the config parameter framework.session.save_path. The default value for this is %kernel.cache.dir%/sessions. This means that in a default installation of symfony the session files would be written to the cache directory for the environment.
However, this can be a problem as the cache directory has to be cleared each time an app is deployed, thus logging all the users out. Therefore presumably your app has been configured (most likely in config.yml) to store the session files in /tmp.
As I understand it, sessions that have expired should be garbage-collected at some point. Symfony also has some config params that affect this - see the FrameworkBundle Configuration. I don't know how much traffic your website has but obviously you do need the session files for active sessions. If you think you have a lot of expired sessions you could try tweaking the gc config params.
Alternatively, if having the session files in /tmp is specifically the problem you could relocate them (by changing the value of the framework.session.save_path) or use PDOSessionHandler to store sessions in the database.
I have this problem with symfony 1.4.20 on a web site I inherited.
It is writing files to
/var/lib/php/sessions
every second, until the server runs out of iNodes.
I've tried changing settings in settings.yml. app.yml and PHP session variables.
Nothing sees to be working though, the only way I can stop it is to change the ownership of /var/lib/php/sessions to root and that prevents any session files being created.
I am using memcache.
I want to understand what is stored in Magento cache and how?
Do magento stores cache variable with website scope or store scope?
I have googled and greped the code but couldnt conclude anything,
Please if someone can direct me to correct links and path
Thanks & Regards,
Saurabh
If you go to the Cache Management section of the admin area you can see what it caches (configuration, layout configuration, block html output, translations, eav types, etc). I am no expert on Magento's caching mechanisms but here are a few random tidbits that might be helpful (maybe). (Also note that I am only familiar with Magento 1.3.x, not 1.4.x so things could have changed).
The caching is actually stored in the var/cache directory. There are a ton of directories in there (mage--0, mage--1, mage--2) and each directory has the cache files. Do a ls var/cache/mage*/* to see all the files.
Configuration - This source for the configuration is varied. Your app/etc/local.xml, and all of the config.xml files (that are in each module's etc dir) are combined together to make one big configuration object. Then Magento reads from the core_config_data table to update the configuration object. Then the configuration is written to a cache file so that next time a request is made it doesn't need to open a ton of config files and hit the database. Somehow this info gets stored in a bunch of files under var/cache. For some insight do a ls var/cache/mage*/*CONF*.
Layout - This is a lot like the configuration... there are a bunch of xml files in the app/design/frontendOrAdminhtml/yournamespace/layout/ directory and all these are merged into one layout configuration object, then cached in the cache directory.
Block HTML - The actual html generated by a block is cached. Each block is able to decide how long it is going to be cached.
Lastly, to (not really) answer your question about if the cache is per website or store, I can't really say since I haven't had the need to setup a multi-website/multi-store shop yet. It looks like there may be some store/website-specific files, but I can't see that they are really organized in a logical way. For example, in one of my instances I see a var/cache/mage--f/mage---LAYOUT_FRONTEND_STORE0_DEFAULT_BLANK_SEO file and a var/cache/mage--f/mage---LAYOUT_FRONTEND_STORE1_DEFAULT_BLANK_SEO... but then again, I only have one store configured and those two files have the same contents. Good luck with that!
You could also use some of the very great memcached analysis and reporting tools available
http://code.google.com/p/memcached/wiki/Tools
The best solution I have come up with is to use a two level cache.
Consult app/etc/local.xml.additional to see how to put memcached server nodes in there. Note that within the <servers> tag you will have to have tags like <server1> and <server2> encapsulating each memcached node's settings.
<cache>
<backend>memcached</backend>
<slow_backend>database</slow_backend>
</cache>
In this way all cache is shared.
To clear it the way I do it is to:
1. shut down apache
2. connect to mysql and connect to the magento db and run truncate core_cache; truncate core_cache_tag.
3. I then bounce the memcached nodes.
4. I restart apache but I keep it out of the load balancer until I have hit it at least once to generate the APC opcode cache. Otherwise the load can shot up through the roof.
This all seems extreme but I have found it works for me. Clearing cache using the backend is REALLY slow. I have around 100k entries in the core_cache table and close to 1 million entries in core_cache_tag. If I don't do it this way sometimes I get strange behavior.
Your Memcache configuration in ./app/etc/local/xml will dictate what Memcache is actually caching.
If you are only using a the single-level cache (without ), then Magento will store its cache (in its entirety) in Memcache.
HOWEVER without the slow_backend defined - it is caching content, without cache_tags - ie. without the ability to differentiate cache items
Eg. configuration, block, layouts, translations etc.
So, without the defined, you cannot refresh caches individually, in-fact, you'll almost always have to rely on "Flush Cache Storage" to actually see updates take effect.
We wrote a nice article here which covers your very issue - http://www.sonassi.com/knowledge-base/magento-knowledge-base/what-is-memcache-actually-caching-in-magento/
Memcached is a distributed memory caching system. It speeds up websites having large dynamic databases by storing database objects in Dynamic Memory to reduce the pressure on a server whenever an external data source requests a read. A Memcached layer reduces the number of times database requests are made.
The caching is actually stored in the var/cache directory. There are a ton of directories in there (mage--0, mage--1, mage--2) and each directory has the cache files. Do a ls var/cache/mage*/* to see all the files.
Configure Memcache Magento 2
Magento 2 also supports Memcached for caching objects but it isn’t enabled by default. You need to make simple changes to the $Magento2Root/app/etc/env.php file to enable it.
In env.php, you will see a large number of PHP arrays with different settings and configurations. Open the file in your favorite code editor and locate the following code:
array (
session' =>
'save' => 'files',
),
Modify this chunk as:
'session' =>
array (
'save' => 'memcached',
'save_path' => '<memcache ip or host>:<memcache port>'
),
Note that the default value for memcache ip is 127.0.0.1:11211. Similarly, the default value for memcache port is 11211.
For complete manual please look into it:
https://www.cloudways.com/blog/magento-2-memcached/
https://devdocs.magento.com/guides/v2.4/config-guide/memcache/memcache_magento.html