Have a Spring/Tomcat application and I'm trying to get ehcache working as Hibernate second-level entity cache. It is enabled and active. I can see in the debug log output how many entities are cached. The problem is that every few seconds, the cache empties. I even pinned the cache (just to see if that would alter the behavior) and now I see this:
05-08-2017 16:05:21.550 [taskExecutor-12] {env=''} WARN n.s.e.Cache: Data availability impacted:
****************************************************************************************
************************** removeAll called on a pinned cache **************************
05-08-2017 16:05:21.550 [taskExecutor-12] {env=''} WARN n.s.e.Cache: Data availability impacted:
****************************************************************************************
************************** removeAll called on a pinned cache **************************
So I think this is the problem. Any idea what would cause removeAll to be implicitly called over and over?
ehcache.xml
<ehcache xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="ehcache.xsd"
updateCheck="true"
monitoring="autodetect"
maxBytesLocalHeap="24G"
dynamicConfig="true">
<defaultCache eternal="true" maxBytesLocalHeap="1" /> <!-- not used, but required per ehcache configuration rules -->
<!-- cache for foo -->
<cache name="foo"
maxBytesLocalHeap="8G"
eternal="false"
memoryStoreEvictionPolicy="LRU"
timeToLiveSeconds="12000"
transactionalMode="off">
<pinning store="inCache" />
<persistence strategy="none" />
</cache>
</ehcache>
persistence.xml
<properties>
<property name="hibernate.dialect" value="org.hibernate.dialect.PostgreSQLDialect" />
<property name="hibernate.connection.driver_class" value="org.postgresql.Driver" />
<property name="hibernate.show_sql" value="false" />
<property name="hibernate.hbm2ddl.auto" value="update" />
<property name="hibernate.ejb.naming_strategy" value="org.hibernate.cfg.ImprovedNamingStrategy" />
<property name="hibernate.cache.provider_class" value="org.hibernate.cache.EhCacheProvider" />
<property name="hibernate.cache.use_second_level_cache" value="true" />
<property name="hibernate.cache.use_query_cache" value="false" />
<property name="hibernate.generate_statistics" value="true" />
<property name="hibernate.cache.use_structured_entries" value="true" />
<property name="hibernate.cache.region.factory_class" value="org.hibernate.cache.ehcache.SingletonEhCacheRegionFactory" />
</properties>
And these annotations are on the entity model class:
#Cacheable
#Cache(region = "foo", usage = CacheConcurrencyStrategy.READ_WRITE)
I figure it out. Native SQL DELETE queries were causing hibernate to frequently dump the entire cache (the removeAll calls above). I did as suggested below, and everything is working now.
2nd level cache invalidation with native queries
Related
For a simple use-case, I have two bundles in Karaf as:
Bundle-Shared-Library
Bundle-My-Implementation -> depends on (1) Bundle-Shared-Library, mentioned in blueprint via depends-on attribute of the bean
Steps that I am following:
Load Bundle(1)
Load Bundle(2)
Shutdown karaf
It is evident that (2) is dependent on (1). While shutting down Karaf, Bundle(1) gets removed first, this results in the loss of in-flight messages as it cannot find its dependency and cannot process further.
I tried setting the start level of (1) both higher and lower than that of bundle(2). This did not help.
Also, I tried setting org.apache.aries.blueprint.preemptiveShutdown=false in etc/config.properties. This did not help as well.
Am I missing something here?
-- EDIT 1 --
After looking a bit over here and there, I found out that we can set a custom value for timeout in DefaultShutdownStrategy. So, as a workaround, I have done following:
<bean id="shutdownStrategy" class="org.apache.camel.impl.DefaultShutdownStrategy">
<property name="timeout" value="1" />
</bean>
I understand that this is not a clean and optimal way to do. For the time being, it somehow helps me from losing a lot of in-flight messages.
But, any suggestion or feedback would be great.
-- EDIT 2 -- Adding Blueprint File of Bundle 1
<?xml version="1.0" encoding="UTF-8"?>
<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:cm="http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.3.0"
xmlns:camel="http://camel.apache.org/schema/blueprint"
xsi:schemaLocation="
http://www.osgi.org/xmlns/blueprint/v1.0.0 https://www.osgi.org/xmlns/blueprint/v1.0.0/blueprint.xsd
http://camel.apache.org/schema/blueprint http://camel.apache.org/schema/blueprint/camel-blueprint.xsd
http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.3.0 http://aries.apache.org/schemas/blueprint-cm/blueprint-cm-1.3.0.xsd
">
<!-- persistent-id for shared context -->
<cm:property-placeholder persistent-id="xxx.shared" update-strategy="none" >
<cm:default-properties>
...
</cm:default-properties>
</cm:property-placeholder>
<!--jms broker connection-->
<reference id="jmsProducerConnectionFactory" interface="javax.jms.ConnectionFactory" filter="(osgi.jndi.service.name=jms/xxx.producer)" availability="mandatory" />
<bean id="xxx-producer" class="org.apache.camel.component.jms.JmsComponent">
<property name="connectionFactory" ref="jmsProducerConnectionFactory"/>
</bean>
<!-- Source DB Reference -->
<reference id="XDataSource" interface="javax.sql.DataSource" filter="(osgi.jndi.service.name=xxx-datasource)" availability="mandatory"/>
<!-- Datawarehouse DB Reference -->
<reference id="dw" interface="javax.sql.DataSource" filter="(osgi.jndi.service.name=xdb-datasource)" availability="mandatory"/>
<bean id="prs" class="org.apache.camel.component.sql.SqlComponent">
<property name="dataSource" ref="XDataSource"/>
</bean>
<bean id="timestamp" class="org.library.shared.TimestampImplementation" destroy-method="synchronize" depends-on="dw">
<argument index="0" ref="dw" />
<argument index="2" value="Orders" />
</bean>
<bean id="shutdownStrategy" class="org.apache.camel.impl.DefaultShutdownStrategy">
<property name="timeout" value="1" />
</bean>
<camelContext id="camel-context" xmlns="http://camel.apache.org/schema/blueprint" depends-on="timestamp">
<packageScan>
<package>org.xyz.orders.routing</package>
</packageScan>
</camelContext>
</blueprint>
The other bundle is simply a collection of few classes (a shared library), which I have included as a maven dependency. There is no blueprint file for that bundle.
Due to organizational issues, I have used dummy names at some places.
We have developed two web services (A,B) with Spring 3, Hibernate 4 & JPA.
Inside a method of service A we need to call to an method of service B. The access is through RMI.
The problem is that the two of them need to be in the same transaction. I think we followed all of the rules dealing with transactions but we cannot rollback the method in service B when service A crashes after method in service B is called.
If an exception is thrown in service A after service B is called the data that service A has changed is rolled back, but not the data changed by service B. It seems that the transaction is renewed in service B and not included in that created in service B.
We use a jndi datasource created in weblogic.
The transactional methods are annotated with:
*#Transactional( readOnly = false, isolation=Isolation.SERIALIZABLE,
rollbackFor=Exception.class, propagation = Propagation.REQUIRED )*
Here is my configuration dealing with transactions:
spring-server.xml (the only data that differs from service to service is the persistence-unit name):
*<bean class="org.springframework.orm.jpa.support.PersistenceAnnotationBeanPostProcessor" />
<bean id="entityManagerFactory" class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
<property name="persistenceUnitName" value="SAPOY/SANOT" />
</bean>
<bean id="jtaTransactionManager" class="org.springframework.transaction.jta.JtaTransactionManager">
<property name="allowCustomIsolationLevels" value="true"/>
<property name="globalRollbackOnParticipationFailure" value="true" />
</bean>
<tx:annotation-driven transaction-manager="jtaTransactionManager" proxy-target-class="true" />
<!-- Servicio SAPOY.SANOT -->
<bean id="sanot" class="com.sems.sapoy.sanot.services.impl.Sanot">
<property name="sanotBiz" ref="sanotBiz" />
</bean>*
persistence.xml (the only data that differs from service to service is the persistence-unit name):
*<?xml version="1.0" encoding="UTF-8"?>
<persistence version="2.0" xmlns="http://java.sun.com/xml/ns/persistence"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/persistence
http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd">
<persistence-unit name="SAPOY/SANOT" transaction-type="JTA">
<description>Unidad de Persistencia del modulo SANOT</description>
<provider>org.hibernate.ejb.HibernatePersistence</provider>
<jta-data-source>jdbc/xaDS</jta-data-source>
<properties>
<!--
################################################################
HIBERNATE
################################################################
-->
<property name="hibernate.archive.autodetection" value="class" />
<property name="hibernate.show_sql" value="true" />
<property name="hibernate.query.factory_class"
value="org.hibernate.hql.classic.ClassicQueryTranslatorFactory" />
<property name="hibernate.format_sql" value="true" />
<property name="hibernate.dialect" value="org.hibernate.dialect.Oracle10gDialect" />
<property name="hibernate.autoReconnect" value="true" />
<property name="hibernate.autoReconnectForPools" value="true" />
<property name="hibernate.transaction.manager_lookup_class" value="org.hibernate.transaction.WeblogicTransactionManagerLookup" />
<property name="hibernate.transaction.factory_class" value="org.hibernate.transaction.JTATransactionFactory" />
<!--
################################################################
CONNECTION
################################################################
-->
<property name="hibernate.connection.autocommit" value="false"/>
<!--
################################################################
CACHE
################################################################
-->
<property name="hibernate.cache.use_second_level_cache" value="true" />
<property name="hibernate.cache.region.factory_class" value="net.sf.ehcache.hibernate.EhCacheRegionFactory" />
<property name="hibernate.cache.use_query_cache" value="true" />
</properties>
</persistence-unit>
</persistence>*
Your DataSource/TransactionManager configurations are just fine, the only problem is that you call your other service through RMI.
The global transactions must be coordinated from the same running Thread that may enlist one or more Resources (JDBC connections/JMS queues), but it cannot span over multiple threads.
The moment you call a RMI method, that call is going to be served by a RMI server which assigns a working thread to service your RMI request. Even if that method is #Transactional the transaction will be defined at the RMI working thread boundary.
Try removing the RMI call and it will work. Implementing it over multiple servers is not a trivial thing to do. You'd probably have to use something like the Command pattern, so that the Service A can call Service B to undo when Service A fails to commit. Like I said, this is not as trivial as a global XA transaction. It requires careful design and it may lead to inconsistencies to due undos happening after other transaction already used the already committed changes.
For that you need WS-Transactions and a Application server to support it. By itself JTA manages global transactions on a Thread basis. To span transactions over multiple Threads you need a higher-level protocol like WS-Transactions.
One of our projects use the cluster of spring 3.2 plus tomcat 7.0, and use ehcache2.8.0 for better performance, also we use RMI for replication like http://ehcache.org/documentation/replication/rmi-replicated-caching. Config like:
<bean id="method-cache-key-generator" class="com.services.cache.MethodCacheKeyGenerator" />
<bean id="service-cache-manager" class="org.springframework.cache.ehcache.EhCacheCacheManager">
<property name="cacheManager" ref="ehcache-manager" />
</bean>
<bean id="ehcache-manager"
class="org.springframework.cache.ehcache.EhCacheManagerFactoryBean" />
and:
<cache:advice id="user-service-read-cache-advice"
cache-manager="service-cache-manager" key-generator="method-cache-key-generator">
<cache:caching cache="userCache">
<cache:cacheable method="get*" />
<cache:cacheable method="find*" />
</cache:caching>
</cache:advice>
<cache:advice id="user-service-write-cache-advice"
cache-manager="service-cache-manager" key-generator="method-cache-key-generator">
<cache:caching cache="userCache">
<cache:cache-evict method="create*" all-entries="true" />
<cache:cache-evict method="update*" all-entries="true" />
<cache:cache-evict method="remove*" all-entries="true" />
<cache:cache-evict method="reset*" all-entries="true" />
<cache:cache-evict method="change*" all-entries="true" />
</cache:caching>
</cache:advice>
<bean id="cached-user-info-service" class="org.springframework.aop.framework.ProxyFactoryBean">
<property name="target" ref="original-user-info-service" />
<property name="interceptorNames">
<list>
<value>user-service-clone-advice</value>
<value>user-service-read-cache-advice</value>
</list>
</property>
</bean>
<bean id="cached-user-service" class="org.springframework.aop.framework.ProxyFactoryBean">
<property name="target" ref="original-user-service" />
<property name="interceptorNames">
<list>
<value>user-service-write-cache-advice</value>
</list>
</property>
</bean>
We found that end-users cannot log on after using the system for a short time, the tomcat has to be restarted to solve the problem. However, we can’t reproduce this problem by our testers on the very same system servers.
We use jconsole to check the servers and find that a large amount of threads are waiting for a thread lock, and this thread deadlock on ehcache.
part of stack is:
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:485)
net.sf.ehcache.util.concurrent.ConcurrentHashMap$Node.tryAwaitLock(ConcurrentHashMap.java:687)
net.sf.ehcache.util.concurrent.ConcurrentHashMap.internalClear(ConcurrentHashMap.java:2016)
net.sf.ehcache.util.concurrent.ConcurrentHashMap.clear(ConcurrentHashMap.java:2680)
net.sf.ehcache.store.cachingtier.OnHeapCachingTier.clear(OnHeapCachingTier.java:213)
net.sf.ehcache.store.CacheStore.removeAll(CacheStore.java:250)
net.sf.ehcache.Cache.removeAll(Cache.java:2451)
net.sf.ehcache.Cache.removeAll(Cache.java:2436)
org.springframework.cache.ehcache.EhCacheCache.clear(EhCacheCache.java:74)
org.springframework.cache.interceptor.CacheAspectSupport.inspectCacheEvicts(CacheAspectSupport.java:243)
org.springframework.cache.interceptor.CacheAspectSupport.inspectAfterCacheEvicts(CacheAspectSupport.java:227)
org.springframework.cache.interceptor.CacheAspectSupport.execute(CacheAspectSupport.java:212)
org.springframework.cache.interceptor.CacheInterceptor.invoke(CacheInterceptor.java:66)
For some reasons, we cannot ask our end-users to help us reproduce this bug. My question is under what kind of conditions ehcache will not release the write lock?
Any suggestions for us to find the method to solve this problem would be much appreciated.
I have read many tutorials to config spring with JPA. I am using a local MySQL database and i have this context:
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:p="http://www.springframework.org/schema/p" xmlns:tx="http://www.springframework.org/schema/tx"
xmlns:context="http://www.springframework.org/schema/context"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
http://www.springframework.org/schema/tx
http://www.springframework.org/schema/tx/spring-tx-3.0.xsd
http://www.springframework.org/schema/context
http://www.springframework.org/schema/context/spring-context-3.0.xsd">
<!-- DataSource Setup -->
<bean id="dataSource"
class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="com.mysql.jdbc.Driver" />
<property name="url" value="jdbc:mysql://localhost:3306/test" />
<property name="username" value="vitornobrega" />
<property name="password" value="" />
</bean>
<!-- Entity Manager Factory -->
<bean id="entityManagerFactory"
class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="packagesToScan" value="com.vitornobrega.myapp.entities" />
<property name="jpaVendorAdapter">
<bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter">
<property name="showSql" value="true" />
<property name="generateDdl" value="true" />
<property name="databasePlatform" value="org.hibernate.dialect.MySQLDialect" />
</bean>
</property>
</bean>
<bean id="txManager" class="org.springframework.orm.jpa.JpaTransactionManager">
<property name="entityManagerFactory" ref="entityManagerFactory" />
</bean>
<tx:annotation-driven transaction-manager="txManager" />
</beans>
I have a test on JUnit to test my DAO and works nice but when i try to persist an entity with this config, i never the in hibernate log the insert command but in test case i see it. If i try to make a entitymanager.flush i get an exception because any transaction is running.
What i should change to can make persist on my local database with this entities?
thanks
If i try to make a entitymanager.flush i get an exception because any transaction is running.
If I'm reading that right, the problem is you're trying to save something without starting a transaction. Transactions are required when changing persistent state.
In order to be able to persist in database it is mandatory to be within a transaction. So the better you can solve it is to annotate #Transactional ... e.g. on the method
This also make me crazy for couple of hours some time ago.
are you using annotations in your classes? if so please make sure you
<context:component-scan base-package="com.vitornobrega.myapp"> because arpart from the jpa configurations i can see where you are injecting your dependencies.
I am using Hibernate in combination with Spring. As database I am currently using HSQL, which stores its data in a file (like SQLite). The path to the HSQL file is currently hard-coded in the persistence.xml. How can I access and change this value at runtime, so a user can load and save from/to an arbitrary HSQL file?
persistence.xml:
<persistence xmlns="http://java.sun.com/xml/ns/persistence"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/persistence
http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd"
version="1.0">
<persistence-unit name="something-unit">
<provider>org.hibernate.ejb.HibernatePersistence</provider>
<properties>
<property name="hibernate.connection.driver_class" value="org.hsqldb.jdbcDriver" />
<property name="hibernate.connection.url" value="jdbc:hsqldb:file:~/something-db/somethingdb" />
<property name="hibernate.connection.username" value="sa" />
<property name="hibernate.dialect" value="org.hibernate.dialect.HSQLDialect" />
<property name="hibernate.hbm2ddl.auto" value="update" />
</properties>
</persistence-unit>
</persistence>
Spring applicationContext.xml
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:context="http://www.springframework.org/schema/context"
xmlns:data="http://www.springframework.org/schema/data/jpa"
xmlns:tx="http://www.springframework.org/schema/tx"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/context
http://www.springframework.org/schema/context/spring-context-2.5.xsd
http://www.springframework.org/schema/data/jpa
http://www.springframework.org/schema/data/jpa/spring-jpa.xsd
http://www.springframework.org/schema/tx
http://www.springframework.org/schema/tx/spring-tx-3.0.xsd">
<!-- Database Setup -->
<bean id="entityManagerFactory" class="org.springframework.orm.jpa.LocalEntityManagerFactoryBean">
<property name="persistenceUnitName" value="something-unit" />
</bean>
<data:repositories base-package="com.something.playlist"/>
<!-- Transaction Setup -->
<tx:annotation-driven/>
<bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager">
<property name="entityManagerFactory" ref="entityManagerFactory"/>
</bean>
</beans>
Thanks for any hint!
You can specify a JNDI datasource and pass it to Hibernate. Or you can define your own plugin strategy for obtaining JDBC connections by implementing the interface org.hibernate.connection.ConnectionProvider
For more hints see: http://docs.jboss.org/hibernate/orm/3.3/reference/en/html/session-configuration.html
Edit 2/16: There is an example on StackOverflow on creating a custom ConnectionProvider: How can I set Datasource when I'm creating Hibernate SessionFactory?
If you are going to change the data source on the fly, rather than at the startup, you will have to restart the Hibernate session factory. To do it correctly, you will have to make sure that no transactions are running in it at the time of the restart. Following question/answers would help you with that: Hibernate Sessionfactory restart | Spring
A commonly used strategy is to define all runtime configurations in one or several *.properties files and use spring's PropertyPlaceholderConfigurer to load the values and substitute the placeholder in applicationContext.xml, read more here: Best ways to deal with properties values in XML file in Spring, Maven and Eclipses.
app.properties:
# Dadabase connection settings:
hibernate.connection.driver_class=org.hsqldb.jdbcDriver
hibernate.connection.url=jdbc:hsqldb:file:~/something-db/somethingdb
hibernate.connection.username=sa
hibernate.connection.password=changeit
hibernate.dialect=org.hibernate.dialect.HSQLDialect
hbm2ddl.auto=update
... ...
applicationContext-dataStore.xml:
<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="locations">
<list>
<!-- Default location inside war file -->
<value>classpath:app.properties</value>
<!-- Environment specific location, a fixed path on deployment server -->
<value>file:///opt/my-app/conf/app.properties</value>
</list>
</property>
<property name="ignoreResourceNotFound" value="true"/>
</bean>
... ...
<bean id="dataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="${hibernate.connection.driver_class}" />
<property name="url" value="${hibernate.connection.url}" />
<property name="username" value="${hibernate.connection.username}" />
<property name="password" value="${hibernate.connection.password}" />
</bean>
One problem here is the PropertyPlaceholderConfigurer doesn't parse persistence.xml, the solution is to move all hibernate configuration into Spring's applicationContext.xml, as it is not necessary to set them in persistence.xml. read more here: loading .properties in spring-context.xml and persistence.xml.
persistence.xml:
<?xml version="1.0" encoding="UTF-8"?>
<persistence xmlns="http://java.sun.com/xml/ns/persistence"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd"
version="1.0">
<persistence-unit name="JPAService" transaction-type="RESOURCE_LOCAL"/>
</persistence>
applicationContext-datSource.xml:
<bean id="dataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="${hibernate.connection.driver_class}"/>
<property name="url" value="${hibernate.connection.url}"/>
<property name="username" value="${hibernate.connection.username}"/>
<property name="password" value="${hibernate.connection.password}"/>
</bean>
... ...
<bean id="entityManagerFactory" class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
<property name="persistenceXmlLocation" value="classpath:./META-INF/persistence.xml"/>
<property name="persistenceUnitName" value="JPAService"/>
<property name="dataSource" ref="dataSource"/>
<property name="jpaVendorAdapter">
<bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter">
<property name="databasePlatform" value="${hibernate.dialect}"/>
<property name="showSql" value="true" />
<property name="generateDdl" value="true"/>
</bean>
</property>
<property name="jpaProperties">
<!-- set extra properties here, e.g. for Hibernate: -->
<props>
<prop key="hibernate.hbm2ddl.auto">${hbm2ddl.auto}</prop>
</props>
</property>
</bean>
Note that the web application need to be restarted every time you alter the configuration in /opt/my-app/conf/app.properties, in order to make changes take effect.
Hope this helps.
If you wish to use hibernate via the JPA Abstraction you can we-write your code or service to use an javax.persistence.EntityManagerFactory. Autowire one of these and call createEntityManager(Map map); You can provide a datasource in the map. You could wrap the entity manager with your own implementation that pulls the parameter off a thread-local for creating the datasource.
EDIT: Mis-read the context and saw you are using an EntityManagerFactory. In which case just read the last part where you wrap the Factory with a delegate that creates the correct datasource from a threadlocal.