We have multiple instances of Ofbiz/Opentaps running. All the instances talk to the same database. There are many tables that are rarely updated hence they are cached and all the instances maintain their individual copies of cache as a standard Ofbiz cache mechanism. But in rare situations when we update some entity using one of many instances then all other instances keep showing dirty cache data. So it requires a manual action to go and clear all the cache copies on other instances as well.
I want this cache clearing operation on all the instances to happen automatically. On Ofbiz confluence page here there is a very brief mention of "Distributed cache clearing". It relies on JMS it seems so whenever an instance's cache is cleared it sends notification over JMS to a topic and other instances subscribing to the same JMS topic clear their corresponding copies of cache upon this notification. But I could not find any other reference or documentation on how to do that? What are the files that need to be updated to set it all up in Ofbiz? An example page/link is what I'm looking for.
Alright I believe I've figured it all out. I have used ActiveMQ as my JMS broker to set it up so here are the steps in Ofbiz to make it working:
1. Copy activemq-all.jar to framework/base/lib folder inside your Ofbiz base directory.
2. Edit File base/config/jndiservers.xml: Add following definition inside <jndi-config> tag:
<jndi-server name="activemq"
context-provider-url="failover:(tcp://jms.host1:61616,tcp://jms.host2:61616)?jms.useAsyncSend=true&timeout=5000"
initial-context-factory="org.apache.activemq.jndi.ActiveMQInitialContextFactory"
url-pkg-prefixes=""
security-principal=""
security-credentials=""/>
3. Edit File base/config/jndi.properties: Add this line at the end:
topic.ofbiz-cache=ofbiz-cache
4. Edit File service/config/serviceengine.xml: Add following definition inside <service-engine> tag:
<jms-service name="serviceMessenger" send-mode="all">
<server jndi-server-name="activemq"
jndi-name="ConnectionFactory"
topic-queue="ofbiz-cache"
type="topic"
listen="true"/>
</jms-service>
5. Edit File entityengine.xml: Change default delegator to enable distributed caching:
<delegator name="default" entity-model-reader="main" entity-group-reader="main" entity-eca-reader="main" distributed-cache-clear-enabled="true">
6. Edit File framework/service/src/org/ofbiz/service/jms/AbstractJmsListener.java: This one is probably a bug in the Ofbiz code
Change following line from:
this.dispatcher = GenericDispatcher.getLocalDispatcher("JMSDispatcher", null, null, this.getClass().getClassLoader(), serviceDispatcher);
To:
this.dispatcher = GenericDispatcher.getLocalDispatcher("entity-default", null, null, this.getClass().getClassLoader(), serviceDispatcher);
7. And finally build the serviceengine code by issuing following command:
ant -f framework/service/build.xml
With this entity data changes in Ofbiz on one instances are immediately propagated to all the other Ofbiz instances clearing cache line item on its own without any need of manual cache clearing.
Cheers.
I have a added a page on this subject in OFBiz wiki https://cwiki.apache.org/OFBIZ/distributed-entity-cache-clear-mechanism.html. Though it's well explained here, the OFBiz wiki page adds other important information.
Note that the bug reported here has been fixed since, but another is currently pending, I should fix it soon https://issues.apache.org/jira/browse/OFBIZ-4296
Jacques
Yes, I fixed this behaviour sometimes ago at http://svn.apache.org/viewvc?rev=1090961&view=rev. But it still needs another fix related to https://issues.apache.org/jira/browse/OFBIZ-4296.
The patch below fixes this issue locally, but still creates 2 listeners on clusters, not sure why... Still investigating (not a priority)...
Index: framework/entity/src/org/ofbiz/entity/DelegatorFactory.java
===================================================================
--- framework/entity/src/org/ofbiz/entity/DelegatorFactory.java (revision 1879)
+++ framework/entity/src/org/ofbiz/entity/DelegatorFactory.java (revision 2615)
## -39,10 +39,10 ##
if (delegator != null) {
+ // setup the distributed CacheClear
+ delegator.initDistributedCacheClear();
+
// setup the Entity ECA Handler
delegator.initEntityEcaHandler();
//Debug.logInfo("got delegator(" + delegatorName + ") from cache", module);
-
- // setup the distributed CacheClear
- delegator.initDistributedCacheClear();
return delegator;
Please notify me using #JacquesLeRoux in your post, if ever you have something new to share.
Related
Longtime D7 user, first time with D9. I am writing my first custom module and having a devil of a time. My routing calls a controller that simple does this:
\Drupal::service('page_cache_kill_switch')->trigger();
die("hello A - ". rand());
I can refresh the page over and over and get a new random number each
time. But, when I change the code to:
\Drupal::service('page_cache_kill_switch')->trigger();
die("hello B - ". rand());
I still get "hello A 34234234" for several minutes. Clearing the cache doesn't help, all I can do is wait, it's normally about two minutes. I am at my wits end.
I thought it maybe an issue with my docker instance. So I generated a simple HTML file but if I edit then reload that file changes are reflected immediately.
In my settings.local.php I have disabled the render cache, caching for migrations, Internal Page Cache, and Dynamic Page Cache.
In my mymod.routing.yml I have:
options:
_admin_route: TRUE
no_cache: TRUE
Any hint on what I am missing would be deeply appreciated.
thanks,
summer
I have a scheduled script execution that needs to persist a value between runs. It is updated with each run. Using gs.setProperty seemed like the natural place until I came across this:
Care should be taken when setting system properties (sys_properties)
using this method as it causes a system-wide cache flush. Each flush
can cause system degradation while the caches rebuild. If a value must
be updated often, it should not be stored as a system property. In
general, you should only place values in the sys_properties table that
do not frequently change.
Creating a separate table to store a single scalar value seems like overkill. Is there a better place to store it?
You could set a preference if you need it in the instance. Another place could be the events table. Log the event with the data in parm1 or parm2 and on next run query the most recent event.
I'd avoid making a table as that has cost implications for some clients. I agree with the sys_properties.
var encrypter = new GlideEncrypter();
var encrypted = encrypter.encrypt('Super Secret Phrase');
gs.info('encrypted: ' + encrypted);
var decrypted = encrypter.decrypt(encrypted);
gs.info('decrypted: ' + decrypted);
/**
*** Script: encrypted: g/bXLJHa7xNRMKZEo5q/YtLMEdse36ED
*** Script: decrypted: Super Secret Phrase
*/
This way only administrators could really read this data. Also if I recall correctly, the sysevent table is cleared after 7 days. You could have the job remove the event as soon as it has it in memory.
I'm working on a simple script in a custom theme in Drupal 7 that is supposed to just rotate through different background image each time a user loads the page. This is my code in [view].tpl.php that picks which image to use.
$img_index = (!isset($_SESSION["img_index"]) || is_null($_SESSION["img_index"])) ? 1 : $_SESSION["img_index"] + 1;
if ($img_index > 2) {
$img_index = 0;
}
$_SESSION["img_index"] = $img_index;
Pretty simple stuff, and it works fine as long as Drupal starts up a session. However, if I delete my session cookie, then always shows the same image, a session is never started.
I'm assuming that since this code is in the view file that the view code is being cached for anonymous users and hence the session is never started, but I can't figure out how to otherwise do what I want.
Don't mess with session like /u/maiznieks mentioned on Reddit. It's going to affect performance.
I've had to do something similar in the past and went with an approach like /u/maiznieks mentions. It's something like this,
Return all the URLs in an array via JS on Drupal.settings.
Check if a cookie is set.
If it's not, set it and set it's value to 0.
If it's set, get the value, increase the value by one, save it to the cookie.
With that value, now you have an index.
Check if image[index] exists
If it does, show that to the user.
If it doesn't, reset index to 0 and show that. Save 0 to the cookie.
You keep caching. You keep showing the user new images on every page load.
You could set your current view to do a random sort every 5 mins. You would then only have to update the logic above to replace that image. That way you can keep something similar working for users with no JS but still keep this functionality for the rest.
You can replace cookies above with HTML5 local storage if you'd like.
#hobberwickey, I will suggest to create a custom module and implement hook_boot() in module. As per drupal bootstrap process session layer will call after cache layer everytime. hook_boot can be called in cache pages and before bootstrap process also. You can take more information here.
I've got spark.history.fs.cleaner.enabled = true for both my Spark2 and Spark configuration. It works for keeping /spark-history/ clean, but fails to do anything for /spark2-history. Any thoughts on why it's not working?
spark.history.fs.cleaner.enabled Spark property controls a task that periodically cleans event logs on disk.
In your question it's spark.history.fs.cleaner=enabled so I think the issue is with the = character.
The other spark.history.fs.cleaner.interval Spark property (with 1d default value) controls how often the cleaner checks for event logs to delete. Make sure that it's often enough.
Using the devel module I can see a lot of calls to cache_get() and cache_set(). After how long does a cached value need to be refreshed? Does the cache get invalidated every few minutes?
The module that is using cache_set sets the expiration in the call. Some things have explicit durations, others have permanent or semi-permanent lifetimes, based on the situation.
Caches get explicitly cleared when you invoke the method through the admin interface (or drush), or otherwise through the use of drupal_flush_all_caches or cache_clear_all.
Lately, I have been using a hook_cron to clear certain cache tables each night.
EDIT to answer comment:
To see which cache, I usually put this in a separate script somewhere:
require_once './includes/bootstrap.inc';
drupal_bootstrap(DRUPAL_BOOTSTRAP_FULL);
header("Content-Type: text/plain; encoding=utf-8");
$user = user_load(1);
print "Modules implementing hook_cron:\n" . implode("\n", module_implements('cron'));
To see expirations, examine the various cache tables in the database and look at the expire column. Modules can set expirations on each individual call to cache_set, so it can vary entry by entry.