Change isolation level with Spring 4.0 & Hibernate 4.2.8 - spring

we are developing an app with Spring 4.0.0, Hibernate 4.2.8 and Ms SQL Server 8, which uses a custom sequence backed with a DB table and mapped with a Hibernate VO (CustomSequence)
This sequence is accesed within a service call:
Main service starts its own transaction
Execute code, do some things, queries...
Calls sequence service for a sequence value (SequenceService)
SequenceService starts its own transaction (REQUIRES_NEW)
SequenceService finds object, returns value and saves next value
Main service gets value, sets in a business object and saves (at this point the sequence value is already commited by the inner new transaction)
Exits
Snippet of the service which manages custom sequence:
#Transactional(propagation = Propagation.REQUIRES_NEW, isolation = Isolation.SERIALIZABLE)
#Service("sequenceService")
public class SequenceService implements ISequenceService {
#Autowired
private ISequenceDao sequenceDao;
private Integer getSequence() {
CustomSequence sequenceOut = sequenceDao.find();
final Integer nextVal = sequenceOut.getNextVal();
sequenceOut.setNextVal(nextVal + 1);
sequenceDao.save(sequenceOut);
return nextVal;
}
}
Our problem is serializable attribute is completely ignored so 2 concurrent threads access getSequence method and obtain the same value.
If we check isolation with TransactionSynchronizationManager the value seems correct for serializable (value=8):
...
Integer isolation = TransactionSynchronizationManager.getCurrentTransactionIsolationLevel();
...
Our spring xml file is this one:
<context:annotation-config />
<context:component-scan base-package="dev.app"/>
<tx:annotation-driven />
<bean name="transactionManager" class="org.springframework.orm.hibernate4.HibernateTransactionManager">
<property name="sessionFactory" ref="sessionFactory" />
</bean>
<bean id="dataSource" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiName" value="java:comp/env/jdbc/appDatasource"/>
</bean>
<bean id="sessionFactory" class="org.springframework.orm.hibernate4.LocalSessionFactoryBean" lazy-init="false" >
<property name="dataSource"> <ref bean="dataSource" /></property>
<property name="packagesToScan" value="dev.app.model"/>
<property name="hibernateProperties">
<props>
<prop key="hibernate.dialect">org.hibernate.dialect.SQLServerDialect</prop>
<prop key="hibernate.show_sql">true</prop>
<prop key="hibernate.format_sql">true</prop>
<!-- Disable LOB creation as connection -->
<prop key="hibernate.temp.use_jdbc_metadata_defaults">false</prop>
</props>
</property>
</bean>
I've checked database serializable capabilities with MS SQL Management Studio with those commands and then execute app code, and it worked (blocked code until studio did commit):
SET TRANSACTION ISOLATION LEVEL SERIALIZABLE
BEGIN TRAN
UPDATE CUSTOM_SEQUENCE set NEXTVAL = 1000;
WAITFOR DELAY '00:1:00'
COMMIT
¿any clues for what's going on? I've read lot of info on internet but to no avail
lot of thanks in advance!

According to the code of HibernateTransactionManager, this could be because something is setting the prepareConnection flag of the transaction manager to false:
/**
* Set whether to prepare the underlying JDBC Connection of a transactional
* Hibernate Session, that is, whether to apply a transaction-specific
* isolation level and/or the transaction's read-only flag to the underlying
* JDBC Connection.
* <p>Default is "true". If you turn this flag off, the transaction manager
* will not support per-transaction isolation levels anymore. ...
*/
public void setPrepareConnection(boolean prepareConnection) {
this.prepareConnection = prepareConnection;
}
Try and put a breakpoint to see if it's the case. Also this flag is always used together with isSameConnectionForEntireSession(session):
if (this.prepareConnection && isSameConnectionForEntireSession(session)) {
....
}
isSameConnectionForEntireSession says:
/**
* Return whether the given Hibernate Session will always hold the same
* JDBC Connection. This is used to check whether the transaction manager
* can safely prepare and clean up the JDBC Connection used for a transaction.
* <p>The default implementation checks the Session's connection release mode
* to be "on_close".
*/
protected boolean isSameConnectionForEntireSession(Session session) ...
This means a custom isolation level can only be applied if: the flag to do so is enabled at the transaction manager AND there is the guarantee that the same database connection will always be used for the same hibernate session.
If it's not the case, then the transaction manager does not change the isolation settings, because if a session could use multiple sessions for different queries, the transaction manager would not know when the sessions are being sent back to the pool.
This basically means the transaction manager only changes the isolation settings of a database session if it has the guarantee that it can cleanup those same settings before the session is sent to the pool.

Related

Spring JMS / Integration thread routing

I have a Spring JMS-to-database microservice using Spring Integration to route incoming messages through a sequence of filter, transform, and service activator endpoints (the last persisting the filtered and transformed messages to an Oracle database). Messages have a primary key used in the database. We have a 5 concurrent consumer set up, based on MQ constraints in prod. All of this works.
Recently we found that the upstream producer can send two or more messages with the same primary key back-to-back. Since two separate threads are given the two messages, even MERGE SQL fails with a SQLIntegrityConstraintViolationException (both threads attempt the ON NOT MATCHED INSERT... portion of the MERGE with the database server).
So we thought to change the listener to guarantee that a specific thread would get a message with any given primary key, using a Hash of the key moduloed by the concurrent consumer count. We need to ACK the message once it is saved to the database, not before, so I want to preserve transaction semantics. But I have been unable to get this working with either a Router, or a LoadBalancer on the channel coming from our DefaultMessageListenerContainer. I've also looked at sub-classing DefaultMessageListenerContainer, but don't see the appropriate point to select a given thread.
Any recommendations?
Bean/channel XML setup, our classes renamed for brevity:
...
<bean id="listenerContainer"
class="org.springframework.jms.listener.DefaultMessageListenerContainer"
p:connectionFactory-ref="consumerConnectionFactory"
p:concurrentConsumers="5"
p:maxConcurrentConsumers="5"
p:destinationName="IN_QUEUE"
p:idleConsumerLimit="5"
p:sessionTransacted="true"
p:errorHandler-ref="errorHandler"
p:autoStartup="true"
p:recoveryInterval="60000"
p:receiveTimeout="5000"/>
...
`<int-jms:message-driven-channel-adapter id="xxx-message-in"
container="listenerContainer"
error-channel="errorChannel"/>
<int:transformer id="unmarshaller"
method="unmarshalPayload"
input-channel="message-in" output-channel="filterIt">
<bean class="com.db.xyz.endpoint.UnmarshallerEndpoint"/>
</int:transformer>
<int:filter id="identifyIt" input-channel="filterIt"
output-channel="transformItC" method="isItOurs">
<bean class="com.db.xyz.service.endpoint.ItFilterEndpoint"/>
</int:filter>
<int:transformer id="transformIt"
input-channel="transformItC"
method="transform" output-channel="persistItC">
<bean class="com.db.xzy.service.endpoint.ItTransformEndpoint"/>
</int:transformer>
<int:service-activator id="persistIt" method="publish"
input-channel="persistItC">
<bean class="com.db.xyz.ilr.service.endpoint.PersistEndpoint"/>
</int:service-activator>
Along with error channel definitions, etc. The consumerConnectionFactory is just a class that selects a Solace JMS, MQ Series, or Active MQ factory based on properties.

Why PollSkipStrategy.skipPoll method is getting called on every message polled from queue?

I'm using inbound poller to process failed requests from backout queue. For scheduling, I'm using corn expression '0 0/2 * * * *' i.e. execute poller every two minutes. The scheduling is working fine as per corn, but PollSkipStrategy.skipPoll method is getting called for every message polled. I was under impression is, poll skip strategy will be execute once for each poll and not for each record polled. I have implementation for PollSkipStrategy.skipPoll, which returns true or false based on peoperty. I'm missing something here? Here is my configuration
<bean id="RegistrationEventPoller"
class="com.poller.RegistrationEventPoller">
<property name="RegistrationEventRetryCount" value="$env{RegistrationEventRetryCount}"/>
</bean>
<bean id="PollSkipAdvice" class="org.springframework.integration.scheduling.PollSkipAdvice">
<constructor-arg ref="PollSkipStrategy"/>
</bean>
<bean id="PollSkipStrategy"
class="com..poller.PollSkipStrategy">
<property name="RegistrationPollerOnOff" value="$env{RegistrationPollerOnOff}"/>
</bean>
The advice is an around advice on the whole flow (MessageSource.receive() and sending the message). When the poller fires it calls the flow for up to maxMessagesPerPoll so, yes, the advice is actually called for each message found within the poll, not just on the first poll. It simply provides a mechanism to stop calling the message source if some condition prevents you from handling messages.
A more sophisticated Smart Polling feature was added in 4.2 which gives you much more flexibility.

Does applying ExpressionEvaluatingRequestHandlerAdvice supress the error?

adapter going out to a jms queue. I have some logic that needs to trigger both on successful deliver and on failover so i've hooked the adapter to the ExpressionEvaluatingRequestHandlerAdvice.
<jms:outbound-channel-adapter id="101Out"
channel="101DestinationChannel"
connection-factory="101Factory"
destination-expression="headers.DESTINATION_NAME"
destination-resolver="namingDestinationResolver"
explicit-qos-enabled="true">
<jms:request-handler-advice-chain>
<beans:bean class="org.springframework.integration.handler.advice.ExpressionEvaluatingRequestHandlerAdvice">
<beans:property name="onSuccessExpression" ref="success"/>
<beans:property name="successChannel" ref="normalOpsReplicationChannel"/>
<beans:property name="onFailureExpression" ref="failure"/>
<beans:property name="failureChannel" ref="failoverInitiationChannel" />
</beans:bean>
<beans:bean id="retryAdvice" class="org.springframework.integration.handler.advice.RequestHandlerRetryAdvice">>
<beans:property name="retryTemplate" ref="retryTemplate"/>
</beans:bean>
</jms:request-handler-advice-chain>
</jms:outbound-channel-adapter>
Now both of these methods are triggering appropriately and the replication/failover logic is executing fine. But on failure (when i stop the queue manager) once the process hooked up to the failureChannel completes, i see that the error is propagating back to the source of the call (an HTTP endpoint in this case).
The advice IS supposed to stop the error from propagating right?
<service-activator input-channel="failoverInitiationChannel"
ref="failoverInitiator" />
I have a service activator hooked up to the failureChannel which just mutates a singleton. Nothing i do here can have triggered the error. Moreover, the error coming back is definitely for the queue access so it can't be anything i did after the failoverInitiator activated.
org.springframework.jms.IllegalStateException: JMSWMQ0018: Failed to connect to queue manager 'APFDEV1' with connection mode 'Client' and host name 'localhost(1510)'.
I'm a very confused if i'm supposed to use the recoveryCallback on the RequestHandlerRetryAdvice or this one to actually stop the error. But i do need an action taken even on success so the ExpressionEvaluatingAdvice is a better fit to my scenario.
Thanks for the help in advance :-)
That is the default behavior. Please see the ExpressionEvaluatingRequestHandlerAdvice javadocs for the trapException property...
/**
* If true, any exception will be caught and null returned.
* Default false.
* #param trapException true to trap Exceptions.
*/
public void setTrapException(boolean trapException) {
this.trapException = trapException;
}
I will add a note the reference manual.

How do you test Spring #Transactional without just hitting hibernate level 1 cache or doing manual session flush?

Using Spring + Hibernate and transactional annotations.
I'm trying to test the following:
call a method that changes a User object then calls a #Transactional service method to persist it
read the object back from the DB and insure it's values are correct after the method
The first problem I had was reading the User object in step 2 just returned the one in the Hibernate level 1 cache and did not actually read from the database.
I therefore manually evicted the object from the cache using the Session to force a read from the database. However, when I do this, the object values are never persisted within the unit test (I know it rolls back after the test is complete because of the settings I specified).
I tried manually flushing the session after the call to the #Transactional service method, and that DID commit the changes. However, that was not what I expected. I thought that a #Transactional service method would insure the transaction was committed and session flushed before it returned.
I know in general Spring will decide when to do this management, but I thought the "unit of work" in a #Transactional method was that method.
In any case, now I'm trying to figure out how I would test a #Transactional method in general.
Here's a junit test method that is failing:
#RunWith(SpringJUnit4ClassRunner.class)
#Transactional
#TransactionConfiguration(transactionManager = "userTransactionManager", defaultRollback = true)
#WebAppConfiguration()
#ContextConfiguration(locations = { "classpath:test-applicationContext.xml",
"classpath:test-spring-servlet.xml",
"classpath:test-applicationContext-security.xml" })
public class HibernateTest {
#Autowired
#Qualifier("userSessionFactory")
private SessionFactory sessionFactory;
#Autowired
private UserService userService;
#Autowired
private PaymentService paymentService;
#Autowired
private QueryService queryService;
#Autowired
private NodeService nodeService;
#Autowired
private UserUtils userUtils;
#Autowired
private UserContext userContext;
#Test
public void testTransactions() {
// read the user
User user1 = userService.readUser(new Long(77));
// change the display name
user1.setDisplayName("somethingNew");
// update the user using service method that is marked #Transactional
userService.updateUserSamePassword(user1);
// when I manually flush the session everything works, suggesting the
// #Transactional has not flushed it at the end of the method marked
// #Transactional, which implies it is leaving the transaction open?
// session.flush();
// evict the user from hibernate level 1 cache to insure we are reading
// raw from the database on next read
sessionFactory.getCurrentSession().evict(user1);
// try to read the user again
User user2 = userService.readUser(new Long(77));
System.out.println("user1 displayName is " + user1.getDisplayName());
System.out.println("user2 displayName is " + user2.getDisplayName());
assertEquals(user1.getDisplayName(), user2.getDisplayName());
}
}
If I manually flush the session, then the test succeeds. However, I would have expected the #Transactional method to take care of committing and flushing the session.
The service method for updateUserSamePassword is here:
#Transactional("userTransactionManager")
#Override
public void updateUserSamePassword(User user) {
userDAO.updateUser(user);
}
The DAO method is here:
#Override
public void updateUser(User user) {
Session session = sessionFactory.getCurrentSession();
session.update(user);
}
SesssionFactory is autowired:
#Autowired
#Qualifier("userSessionFactory")
private SessionFactory sessionFactory;
I'm using XML application context configuration. I have:
<context:annotation-config />
<tx:annotation-driven transaction-manager="userTransactionManager" />
And
<bean id="userDataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource" destroy-method="close">
<property name="driverClass" value="${user.jdbc.driverClass}"/>
<property name="jdbcUrl" value="${user.jdbc.jdbcUrl}" />
<property name="user" value="${user.jdbc.user}" />
<property name="password" value="${user.jdbc.password}" />
<property name="initialPoolSize" value="3" />
<property name="minPoolSize" value="1" />
<property name="maxPoolSize" value="17" />
</bean>
<bean id="userSessionFactory"
class="org.springframework.orm.hibernate4.LocalSessionFactoryBean">
<property name="dataSource" ref="userDataSource" />
<property name="configLocation" value="classpath:user.hibernate.cfg.xml" />
</bean>
<bean id="userTransactionManager"
class="org.springframework.orm.hibernate4.HibernateTransactionManager">
<property name="dataSource" ref="userDataSource" />
<property name="sessionFactory" ref="userSessionFactory" />
</bean>
There is also a component scan on the services and dao classes. As I said, this is working in production.
I thought that if I have a method marked #Transactional that by the end of that method (e.g. the update method here), Spring would have forced the Session to commit and flush.
I can see only a few options:
I misconfigured something, even though this is working for me in general (just not the unit tests). Any guesses? Any ideas for how to test this?
Something about the unit test config themselves is not behaving the way the app would.
Transactions and sessions don't work like that. My only deduction is that Spring is leaving the transaction and/or the session open after calling that update method. So when I manually evict the user on the Session object, those changes haven't been committed yet.
Can anyone confirm if this is expected behavior? Shouldn't #Transaction have forced commit and flush on session? If not, then how would one test a method marked #Transactional and that the methods actually work with transactions?
I.e., how should I rewrite my Unit test here?
Any other ideas?
Here's what I was running into. Consider this code in a test method:
String testDisplayNameChange = "ThisIsATest";
User user = userService.readUser(new Long(77));
user.setDisplayName(testDisplayNameChange);
user = userService.readUser(new Long(77));
assertNotEquals(user.getDisplayName(), testDisplayNameChange);
Note that the method userService.readUser is marked #Transactional in the service class.
If that test method is marked #Transactional the test fails. If it is NOT, it succeeds. Now I'm not sure if/when the Hibernate cache is actually getting involved. If the test method is transactional then each read happens in one transaction, and I believe they only hit the Hibernate level 1 cache (and don't actually read from the database). However, if the test method is NOT transactional, then each read happens in it's own transaction, and each does hit the database. Thus, the hibernate level 1 cache is tied to the session / transaction management.
Take aways:
Even if a test method is calling multiple transactional methods in another class, if that test method itself is transactional, all of those calls happen in one transaction. The test method is the "unit of work". However, if the test method is NOT transactional, then each call to a transactional method within that test executes within it's own transaction.
My test class was marked #Transactional, therefore every method will be transactional unless marked with an overriding annotation such as #AfterTransaction. I could just as easily NOT mark the class #Transactional and mark each method #Transactional
Hibernate level 1 cache seems tied to the transaction when using Spring #Transactional. I.e. subsequent reads of an object within the same transaction will hit the hibernate level 1 cache and not the database. Note there is a level 2 cache and other mechanisms that you can tweak.
I was going to have a #Transactional test method then use #AfterTransaction on another method in the test class and submit raw SQL to evaluate values in the database. This would circumvent the ORM and hibernate level 1 cache altogether, insuring you were comparing actual values in the DB.
The simple answer was to just take #Transactional off of my test class. Yay.
Q
Can anyone confirm if this is expected behavior? Shouldn't
#Transaction have forced commit and flush on session? If not, then how
would one test a method marked #Transactional and that the methods
actually work with transactions?
A
It is the expected behavior. The spring aware transactional unit test support by design rollsback the transaction after the test is finished. This is by design.
An implicit transaction boundary is created per test (each method with #Test) and once the test is finished a rollback is done.
The consequence of this is after all tests are finished there is no data that is actually changed. That is the goal is to be more "unit" like and less "integration" like. You should read the spring documentation as to why this is advantageous.
If you really want to test data being persisted and to view that data outside of the transaction boundary I recommend a more end-to-end test like a functional/integration test such as selenium or hitting your external WS/REST API if you have one.
Q
I.e., how should I rewrite my Unit test here?
A
Your unit test does not work because you session.evict and not flush. Evicting will cause the changes not to be synchronized even though you already called session.update your in a transaction and hibernate batches operations. This is more clear with raw SQL as hibernate defers persisting to batch up all the operations so it waits until the session is flushed or closed to communicate with the database for performance. If you session.flush though the SQL will be executed right away (ie update will really happen) and then you can evict if you like to force a reread but your rereading within the transaction. I'm actually pretty sure flush will cause eviction so there is no need to call evict to force a reread but I might be wrong.

Spring integration channel message count

I have 2 questions. Here they are:
Is there any way to determine the number of messages waiting to be
processed in a spring integration channel while the number of
customers keep increasing along the time?
In the app context, I want to be able to define x instances of
bean y which x and y, both consumes from channel p, programmatically
increase or decrease the consumers based on the load.
There is an example showed in spring 2gx but it used rabbitmq to determine load.
For 1), Spring Integration channels are just beans like any other bean. Assuming that you're using a standard pollable channel, you can autowire it in by name and then get the number of messages waiting:
http://docs.spring.io/spring-integration/api/org/springframework/integration/channel/QueueChannel.html#getQueueSize()
You may also be able to introspect this through JMX if you want.
For 2)... To do that, you'll want to use an AOP proxy on top of an object pool, then wire the proxy into (what I assume is a) Service Activator. Those properties can be externalized with a PropertyPlaceholderConfigurer. So, as an example:
<bean id="propertyPlaceholder"
class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="systemPropertiesMode" value="2" />
<property name="locations" value="classpath:my_config_params.properties" />
</bean>
<bean id="myBusinessObjectImpl" class="com.mycompany.whatever.impl.MyServiceImpl"
scope="prototype" autowire-candidate="false" />
<bean id="myBusinessObjPool" class="org.springframework.aop.target.CommonsPoolTargetSource">
<property name="targetBeanName" value="myBusinessObjectImpl" />
<!-- THIS IS THE KEY. myconfig.params.poolsize is the name of the property in the my_config_params.properties above. -->
<property name="maxSize" value="${myconfig.params.poolsize}" />
</bean>
<bean id="myBusinessObject" class="org.springframework.aop.framework.ProxyFactoryBean">
<property name="targetSource" ref="myBusinessObjPool" />
</bean>
<int:channel id="myInputChannel">
<int:queue size="500" />
</int:channel>
<int:service-activator inputChannel="myInputChannel" ref="myBusinessObject" method="processMessages">
<int:poller max-messages-per-poll="10" fixed-rate="5000"/>
</int:service>
This also allows you to make the Service Activator stateful.
Obligatory link to object pooling functionality:
http://docs.spring.io/spring/docs/3.0.x/spring-framework-reference/html/aop-api.html#aop-ts-pool

Resources