I'm trying to compare the performance of different Solr queries. In order to get a fair test, I want to clear the cache between queries.
How is this done? Of course, one can restart the server, I was curious if there is a quicker way.
I'm using version 4.2.1 and even with autowarmCount="0" the cache is not updated after doing a Dataimport.
In that case, on Solr Admin (usually http://localhost:8983/)
Go to Core Admin and click Reload.
When refreshed, you should see a green check mark on the "current" field.
Disable all the caches from solrconfig.xml.
Note that the Lucene FieldCache will still be enabled.
Just for thorough details
If you want to disable a cache (or all), comment out those sections in solrconfig.xml and restart solr.
example diable queryResultCache
<!--
<queryResultCache class="solr.LRUCache"
size="5000"
initialSize="5000"
autowarmCount="2000"/>
-->
Related
Now I am using gradle mlReloadSchemas tasks to reload TDE templates.
I guess even if the change is for one tde file only, the reload schemas task may delete all in DB and load all TDE templates to ML DB.
I wonder whether it will cause a performance issue for ML. Will that trigger indexing even for the TDE files that have not yet changed?
I am using DevOps pipeline to trigger the schema reload from GIT repository. As such, I could not load only the change TDE file. I have to reload everything. If there is performance issue, how to load only changed file with the pipeline?
Redeploying TDE can cause reindexing. How many records to be reindexed depends upon the context matching for those TDE.
A properly resourced cluster should be able to handle the load of reindexing.
That being said, the merging activities can compete with online traffic and query demands. You can help minimize the impact by setting the reindex throttle to a lower level (1-5 with 1 being the lowest), and you can set a background-io limit to restrict the amount of IO any node will use for background activities such as merges and backups.
You can also choose when to enable/disable reindexing, and adjust the reindexing level to a higher/lower level at different periods.
https://help.marklogic.com/Knowledgebase/Article/View/how-reindexing-works-and-its-impact-on-performance
https://help.marklogic.com/Knowledgebase/Article/View/indexing-best-practices
When I use the search function on my website, it shows results with old content. I tried to clear all caches but it doesn't solve the issue.
I am not sure if I should truncate some tables in the DB and how safe this is.
By the way I don't have any "indexed search" option in the Backend under "Info".
Which TYPO3 Version are you using? With TYPO3 8.7 (not sure if also with 7.6) the module moved to an own one called "Indexing".
Anyway you shouldn't get any problem truncating the indexed_search tables in the DB as they get shipped without data when you install indexed_search.
To be sure, just make a backup of the tables.
I have a requirement for a document management system to handle pdf,word,xls,ppt with semantic search.
I started looking into elasticsearch for the same and stumbled on Apache JacKrabbit and subsequently on OpenKM and Hippo. Even though core features like versioning exists in Jackrabbit, I need some pointers on how to go about this.
I need help navigating through the following concerns:
Should I just use elasticsearch and elasticsearch attachment plugin or use Jackrabbit with MySQL backend and use Elasticsearch to index the documents.
Or should I use OpenKM?
Any pointers would be greatly appreciated. This would finally require App integration.
Update Logically, using ElasticSearch for Search makes sense. But I figure that I cannot use that as primary datasource. What are the best options from storage(primary) Apache JackRabbit with MySQL? As all features are prebuilt in OpenKM, would this be a better option?.
What is it you want to achieve? Are you looking to manage making the documents available, is it about managing the content in documents? ES, or any search engine, is generally not a primary data source.
I can't give you any advice wrt OpenKM (neither for or against). Whether Hippo is a match depends on your case which I need to know more about.
I am using Pentaho 5. My dimensions keep changing frequently and I need the changes to be applied to the dashboard, this is not possible because Pentaho keeps caching. I have created the cube using the datasource wizard and the querys using mdx over mondrian jndi. Even though I set the property Cache to false or set cache duration doesn't seem to work. Is there a API that I can use with mondrian jndi to clear cache? Or are there any property files that I should change? Please help.
In Pentahoo 7 the "Clear Cache" option is in a different Menu:
Tools -> Database -> Clear Cache
If you are using Database lookup and if you are getting older fields, cleaning cache can actually solved the problem.
It worked for me.
You can do it manually inside Pentaho User Console: Tools -> Refresh -> Mondrian Schema Cache.
Or you can make schedule for refreshing cache: find clear_mondrian_schema_cache.xaction inside your installation and schedule it.
Option 1:
You can read http://javadoc.pentaho.com/bi-platform500/webservice500/ for api details.
And also you can refresh Reporting Metadata Cache via web service, you can use the following web service call:
http://localhost:8080/pentaho/api/system/refresh/metadata
Option2: You can navigate to \biserver-ee\tomcat\webapps\pentaho\WEB-INF\classes
and change the configuration file "ehcache.xml"
<cache name="report-dataset-cache"
maxElementsInMemory="50"
eternal="false"
overflowToDisk="false"
timeToIdleSeconds="1"
timeToLiveSeconds="2"
diskPersistent="false"
diskExpiryThreadIntervalSeconds="1"
/>
If you have done that and still no positive answer, I think you have not restarted the BA server. If the issue still exists comment below.
There are 2 options
One is to schedule "clear mondrian schema" on ba server but for that to happen , you need to get clear_mondrian_schema.xml from pentaho-solutions/systems folder and upload it in some folder that you can access on ba server. You can then use normal schedule file options to achieve you want. This put a lof of load on BA server though.
My second recommendation is if you are using cubes/schema and building using schema workbench, you can turn the caching off. If your database is architecturally good and your schemas all well defined, user will get updates/new data as soon as they refresh.
I am using memcache.
I want to understand what is stored in Magento cache and how?
Do magento stores cache variable with website scope or store scope?
I have googled and greped the code but couldnt conclude anything,
Please if someone can direct me to correct links and path
Thanks & Regards,
Saurabh
If you go to the Cache Management section of the admin area you can see what it caches (configuration, layout configuration, block html output, translations, eav types, etc). I am no expert on Magento's caching mechanisms but here are a few random tidbits that might be helpful (maybe). (Also note that I am only familiar with Magento 1.3.x, not 1.4.x so things could have changed).
The caching is actually stored in the var/cache directory. There are a ton of directories in there (mage--0, mage--1, mage--2) and each directory has the cache files. Do a ls var/cache/mage*/* to see all the files.
Configuration - This source for the configuration is varied. Your app/etc/local.xml, and all of the config.xml files (that are in each module's etc dir) are combined together to make one big configuration object. Then Magento reads from the core_config_data table to update the configuration object. Then the configuration is written to a cache file so that next time a request is made it doesn't need to open a ton of config files and hit the database. Somehow this info gets stored in a bunch of files under var/cache. For some insight do a ls var/cache/mage*/*CONF*.
Layout - This is a lot like the configuration... there are a bunch of xml files in the app/design/frontendOrAdminhtml/yournamespace/layout/ directory and all these are merged into one layout configuration object, then cached in the cache directory.
Block HTML - The actual html generated by a block is cached. Each block is able to decide how long it is going to be cached.
Lastly, to (not really) answer your question about if the cache is per website or store, I can't really say since I haven't had the need to setup a multi-website/multi-store shop yet. It looks like there may be some store/website-specific files, but I can't see that they are really organized in a logical way. For example, in one of my instances I see a var/cache/mage--f/mage---LAYOUT_FRONTEND_STORE0_DEFAULT_BLANK_SEO file and a var/cache/mage--f/mage---LAYOUT_FRONTEND_STORE1_DEFAULT_BLANK_SEO... but then again, I only have one store configured and those two files have the same contents. Good luck with that!
You could also use some of the very great memcached analysis and reporting tools available
http://code.google.com/p/memcached/wiki/Tools
The best solution I have come up with is to use a two level cache.
Consult app/etc/local.xml.additional to see how to put memcached server nodes in there. Note that within the <servers> tag you will have to have tags like <server1> and <server2> encapsulating each memcached node's settings.
<cache>
<backend>memcached</backend>
<slow_backend>database</slow_backend>
</cache>
In this way all cache is shared.
To clear it the way I do it is to:
1. shut down apache
2. connect to mysql and connect to the magento db and run truncate core_cache; truncate core_cache_tag.
3. I then bounce the memcached nodes.
4. I restart apache but I keep it out of the load balancer until I have hit it at least once to generate the APC opcode cache. Otherwise the load can shot up through the roof.
This all seems extreme but I have found it works for me. Clearing cache using the backend is REALLY slow. I have around 100k entries in the core_cache table and close to 1 million entries in core_cache_tag. If I don't do it this way sometimes I get strange behavior.
Your Memcache configuration in ./app/etc/local/xml will dictate what Memcache is actually caching.
If you are only using a the single-level cache (without ), then Magento will store its cache (in its entirety) in Memcache.
HOWEVER without the slow_backend defined - it is caching content, without cache_tags - ie. without the ability to differentiate cache items
Eg. configuration, block, layouts, translations etc.
So, without the defined, you cannot refresh caches individually, in-fact, you'll almost always have to rely on "Flush Cache Storage" to actually see updates take effect.
We wrote a nice article here which covers your very issue - http://www.sonassi.com/knowledge-base/magento-knowledge-base/what-is-memcache-actually-caching-in-magento/
Memcached is a distributed memory caching system. It speeds up websites having large dynamic databases by storing database objects in Dynamic Memory to reduce the pressure on a server whenever an external data source requests a read. A Memcached layer reduces the number of times database requests are made.
The caching is actually stored in the var/cache directory. There are a ton of directories in there (mage--0, mage--1, mage--2) and each directory has the cache files. Do a ls var/cache/mage*/* to see all the files.
Configure Memcache Magento 2
Magento 2 also supports Memcached for caching objects but it isn’t enabled by default. You need to make simple changes to the $Magento2Root/app/etc/env.php file to enable it.
In env.php, you will see a large number of PHP arrays with different settings and configurations. Open the file in your favorite code editor and locate the following code:
array (
session' =>
'save' => 'files',
),
Modify this chunk as:
'session' =>
array (
'save' => 'memcached',
'save_path' => '<memcache ip or host>:<memcache port>'
),
Note that the default value for memcache ip is 127.0.0.1:11211. Similarly, the default value for memcache port is 11211.
For complete manual please look into it:
https://www.cloudways.com/blog/magento-2-memcached/
https://devdocs.magento.com/guides/v2.4/config-guide/memcache/memcache_magento.html