I have a strange behavior with caching and JPA Entities (EclipseLink 2.4.1 ) + GUICE PERSIST
I will not use caching, nevertheless I get randomly an old instance that has already changed in MySQL database.
I have tried the following:
Add # Cacheable (false) to the JPA Entity.
Disable Cache properties in the persistence.xml file :
<class>MyEntity</class>
<shared-cache-mode>NONE</shared-cache-mode>
<properties>
<property name="eclipselink.cache.shared.default" value="false"/>
<property name="eclipselink.cache.size.default" value="0"/>
<property name="eclipselink.cache.type.default" value="None"/>
<property name="eclipselink.refresh" value="true"/>
<property name="eclipselink.query-results-cache" value="false"/>
<property name="eclipselink.weaving" value="false"/>
</properties>
Even activating trace EclipseLink, i see the JPQL query:
ReadObjectQuery Execute query (name = "readObject" referenceClass = XX sql = "... (just making a call" find "the entityManager
but, However randomly returns an old value of that class.
Note
Perhaps happens for using different instances of EntityManager and everyone has their cache?
I have seen the following related post : Disable JPA EclipseLink 2.4 cache
If so, is possible to clear the cache of ALL EntityManager whithout using : ????
em.getEntityManagerFactory().getCache().evictAll();
Is it possible to clear ALL caches whithout using evictALL ??
Evict all is for the shared cache which you have disabled already anyway. EntityManager instances are required by default to have a first level cache of their own to keep track of all managed instances they have created. An entityManager is meant to represent logical transactions and so should not be long lived. You need to throw away your EntityManagers and re obtain them or just clear them at logical points rather than let the number of managed entitites in its cache grow endlessly. This will also help limit the stale data issue, though nothing other than pessimistic locking can eliminate it. I recommend using optimistic locking if you aren't already to avoid overwriting with stale data.
Related
I need to understand if the following configuration of Ignite will serve my data from memory or from disk.
Ignite configuration:
<property name="dataStorageConfiguration">
<bean class="org.apache.ignite.configuration.DataStorageConfiguration">
<property name="defaultDataRegionConfiguration">
<bean class="org.apache.ignite.configuration.DataRegionConfiguration">
<property name="persistenceEnabled" value="true"/>
</bean>
</property>
</bean>
</property>
Java Code:
ClientConfiguration cfg = new ClientConfiguration().setAddresses("127.0.0.1:10800");
try (IgniteClient client = Ignition.startClient(cfg)) {
ClientCache<Long, SensorsWaiting> cache = client.cache("SQL_PUBLIC_FOO");
FieldsQueryCursor<List<?>> query = cache.query(new SqlFieldsQuery("select * from foo"));
}
Background of the question:
I expect a large number of queries and need the results being served from memory. At the same time I need the data to be stored to disk in case the Ignite Server crashes or needs to be restarted.
Is my understanding correct, that in this case my data is served from memory?
What if I use the JDBC driver? Is it still the same? What is the difference between the cache and the jdbc driver?
It'll be served from memory if it's in memory, otherwise it will be pulled in from disk. (This distinction is important if you have more data than you have memory or for when the cluster starts up.)
The different APIs all access the same data. Whether you use the JCache (get, put), SqlFieldsQuery or JDBC/ODBC, it's all the same.
In my xml file, I have something as follow:
<bean id="transactionManager"
class="org.springframework.transaction.jta.WebSphereUowTransactionManager"
p:defaultTimeout="60" />
<bean id="sharedTransactionTemplate"
class="org.springframework.transaction.support.TransactionTemplate">
<constructor-arg>
<ref bean="transactionManager" />
</constructor-arg>
<property name="isolationLevelName" value="${sharedTransactionTemplate.isolationlevel:ISOLATION_READ_UNCOMMITTED}"/>
<property name="timeout" value="60"/>
</bean>
With the value 60, my program will hit timeout if the response from db taking more than 60 seconds. This is correct and also what I expected.
And I found that there is some transaction time out value setting in WAS Console as well:
Server --> WebSphere application servers --> my server
Under Container Settings --> click on Container Services --> Transaction service
Inside Transaction service page, there is a value call "Total transaction lifetime timeout ". I set the value to 80.
In my application, I have a part that will trigger Spring SimpleJobLauncher to run a spring batch in my application. In my Spring batch, I have some for loop which is write some data in log file, and it does not have any interaction with DB.
I found that, my for loop will not hit the 60 seconds time out after 60 seconds. It will only hit the 80 seconds time out. I believe that it is because of it didn't call db.
My code is something as follow:
#Autowired
#Qualifier("sharedTransactionTemplate")
private TransactionTemplate transactionTemplate;
transactionTemplate.execute( new TransactionCallbackWithoutResult( ) {
// In here I trigger the spring batch
} );
I would like to edit this value to for example 70 seconds base on code in xml or any way. I do not want to edit it in WAS Console because I still want other method still using the 80 seconds.
Any ideas?
Here is what my spring batch doing:
Call db, update something. (done with no error)
reader, read data from db. (done with no error)
Before write, i got some for loop which is not call db. --> hit timeout here, I found that the timeout value is the value that set in WAS Console, instead of the value set in xml.
and so on...
I actually want to do something that I can code in xml, so that this spring batch can use my own value set in xml. SO that my step 3 can use my own value.
Additional question, are these following class only applicable for transaction that involve connection to database?
class="org.springframework.transaction.jta.WebSphereUowTransactionManager"
class="org.springframework.transaction.support.TransactionTemplate"
It is unclear to me from the info you've provided why you are executing your Spring Batch job transactionally, you may want to consider whether you need to. Although not a duplicate, this question is similar to this one in which you can see one possible solution is to start a UserTransaction for your spring batch job which you can control the timeout. As pointed out in that answer and subsequent comments, there are some limitations and considerations about using this method.
I am at a loss with this and can't seem to find an answer in the docs. I am observing the following behaviour. I have this rule:
import function util.CSVParser.parse;
declare Passenger
#role(event)
#expires(24h)
end
rule "Parse and Insert CSV"
when
CSVReadyEvent( $csv_location : reader ) from entry-point "CSVReadyEntryPoint";
$p : Passenger() from parse($csv_location);
then
insert( $p );
end
I can then enter my CSVReadyEvent into my session and call fireAllRules and it executes correctly. It hits the safe point at the end, and all is cool.
I then restart my app and load the session like this:
KieSession loadedKieSession = kieServices.getKieService().getStoreServices().loadKieSession(session.getId(), kieBase, ksConf, kieServices.getEnvironment());
The base and config I take from my kmodule.xml.
What happens now is that WITHOUT calling fireAllRules() loading the session somehow triggers fireing all rules.
I do not understand how unmarshalling triggers rule execution but this is obviously wrong. I have already executed that rule, and it should not be executed twice.
In a test case (my tests do NOT create persistent sessions because I only want the rules to be tested) I can call fireAllRules() twice, and the second time does not trigger any matched rules. I am not exactly sure what goes wrong, but the persistent session seems to be loaded in an odd way. Or the persisting of the session is wonky and forgets that it had executed the rule already.
Does anyone have inside in this? I am more than happy to share any code.
Here's my persistence.xml:
<persistence-unit name="org.jbpm.persistence.jpa" transaction-type="JTA">
<provider>org.hibernate.ejb.HibernatePersistence</provider>
<class>org.drools.persistence.info.SessionInfo</class>
<class>org.drools.persistence.info.WorkItemInfo</class>
<exclude-unlisted-classes>true</exclude-unlisted-classes>
<properties>
<property name="hibernate.dialect" value="org.hibernate.dialect.MySQLDialect" />
<property name="hibernate.max_fetch_depth" value="30" />
<property name="hibernate.hbm2ddl.auto" value="update" />
<property name="hibernate.show_sql" value="true" />
<property name="hibernate.transaction.jta.platform" value="org.hibernate.service.jta.platform.internal.JBossStandAloneJtaPlatform" />
</properties>
</persistence-unit>
Thanks!
An update/answer from a painful painful painful day of debugging and testing and running stuff:
I suspected my hibernate setup was wrong, so the wrong thing got persisted. I ended up throwing that approach away and writing a manual marshalling/de-marshalling thing.
After creating/loading/recreating/loading I can confirm the session NEVER changes on file.
This was interesting to me because I could swear that the rules are executed, and I was half right:
The WHEN part is executed when the session is loaded. Why? I have not the slightest idea...
I was chasing a red hearing because I am calling a function in my when part (as you can see in the rule) to iterate and insert all facts based on that event I am receiving.
My parse function obviously has logging, so each time I reload the session, I get a storm of log flying through my terminal hinting that my rules are being executed.
I then changed my rules to be very very specific (as in output everywhere I possible can). I debugged as deep as I could and I still can't seem to be able to pinpoint as to why on earth recreating the session is executing the when part of a rule. I settled on this: Magic. And with a little more detail:
The documentation of drools persistence https://docs.jboss.org/jbpm/v6.2/userguide/jBPMPersistence.html states that the guys implemented their own serialze/deserialize strategy in order to speed up the process. I resolve to blame this custom strategy on what I am seeing.
Lesson learned:
Do NOT create objects in the when part (because this will slow you down when loading a session since all when parts are executed)
Chasing red herrings is a pain in my butt.
So to sum up: I believe (up to say 99%) that loading a session is NOT executing the rules.
Using events in real mode and in a STREAM session running due to fireUntilHalt on the one hand and saving and restarting sessions with fireAllRules are somewhat contradictory paradigms.
If you have events, I suggest that you use the API to set up and start a (stateful) session in a thread, and insert facts (events) as they arrive.
Sorry in advance for the long post however I wanted to be precise in my post.
I have a spring mvc 3.1 web application and have just built ehcache into my web application which I am using to cache all my list of values (drop downs) built from the database.
Here is an example of my settings...
<!-- Cache -->
<dependency>
<groupId>net.sf.ehcache</groupId>
<artifactId>ehcache-core</artifactId>
<version>2.5.0</version>
</dependency>
...
<ehcache>
<diskStore path="java.io.tmpdir"/>
<cache
name="lovStore"
maxElementsInMemory="512"
eternal="false"
timeToIdleSeconds="60"
timeToLiveSeconds="60"
overflowToDisk="false"
memoryStoreEvictionPolicy="LRU"/>
</ehcache>
...
<cache:annotation-driven />
<bean id="cacheManager"
class="org.springframework.cache.ehcache.EhCacheCacheManager"
p:cache-manager-ref="ehcache"/>
<bean id="ehcache"
class="org.springframework.cache.ehcache.EhCacheManagerFactoryBean"
p:config-location="classpath:ehcache.xml"/>
...
#Cacheable(value = "lovStore", key = "#code")
public Map<String, String> getLov(String code) throws ReportingManagerException {
return MockLOVHelper.getInstance().getLov(code);
}
A lot of the tutorials on the net talk about evicting the cache using a #cacheEvict method however this does not suit me. I think I am better suited off using the timeToLiveSeconds option to evict the cache.
When i look in the logs the cache is definitely working however the eviction of the cache is not. I've read some other articles on the net about how timeToLiveSeconds doesn't truly evict the cache but other articles like this one http://code.google.com/p/ehcache-spring-annotations/wiki/EvictExpiredElements which say there are special settings you have to create to get the cache to evict.
Can someone please help me understand if my cache should be evicting and also how I can evict because what is mentioned in the article is not something i was able to understand how to implement.
Here are what my logs look like. But there are no signs of eviction...
2014-01-20 13:32:41,791 DEBUG [AnnotationCacheOperationSource] - Adding cacheable method 'getLov' with attribute: [CacheableOperation[public java.util.Map com.myer.reporting.dao.mock.MockLovStoreDaoImpl.getLov(java.lang.String) throws com.myer.reporting.exception.ReportingManagerException] caches=[lovStore] | condition='' | key='#code']
thanks
Ecache does not evict elements until there's need to do that. And that's reasonable since there is no need to waste CPU resources for operation(evicting elemets) that wouldn't make much difference.
If someone would try to get element from cache and the element's time to live/idle would be expired, then ehcache would evict that element and return null. In case max elements or memory for cache/pool would be reached then ehcache would evict elements that are expired, based on eviction policy.
And if I understand correctly it's not guaranteed that all expired element's would be evicted since only sample elements are selected to evict(from documentation):
#param sampledElements this should be a random subset of the population
I am testsing eclipselink to make bulk data insert into derby.
Compared by the same set of data, eclipse link take double time of jdbc batch update.
I have enabled the batchupdate feature of eclipse link, and the other properties:
<property name="eclipselink.jdbc.batch-writing" value="JDBC"/>
<property name="eclipselink.jdbc.batch-writing.size" value="1000"/>
<property name="eclipselink.jdbc.cache-statements" value="true"/>
<property name="eclipselink.jdbc.cache-statements.size" value="30"/>
<property name="eclipselink.cache.shared.default" value="false"/>
<property name="eclipselink.jdbc.read-connections.max" value="20"/>
<property name="eclipselink.jdbc.read-connections.min" value="1"/>
<property name="eclipselink.jdbc.write-connections.min" value="1"/>
<property name="eclipselink.jdbc.write-connections.max" value="30"/>
The question is how to make eclipse link be faster?
Please include the code, and the SQL log. Also include your JDBC code, and make sure it is kosher (closing statements, etc.).
Are you using sequence preallocation? If not then you will not be getting any batching (check your SQL log to see if the batch is occurring).
I would not change the connection pooling defaults, your are less efficient than the default (initial 1, min 32, max 32, no separate read/write pool). Having a different min/max will cause connection throttling, which is bad.
See,
http://java-persistence-performance.blogspot.com/2011/06/how-to-improve-jpa-performance-by-1825.html
Since JPA operates on top of JDBC, is will always take more time than fully optimized JDBC code. But does have the advantage of letting you use Java objects and not write JDBC code, and make major optimization such as batch writing just by changing a flag, instead of rewriting the code.