Testing #TransactionalEvents and #Rollback - spring

I've been trying to test out #TransactionalEvents (a feature of Spring 4.2 https://spring.io/blog/2015/02/11/better-application-events-in-spring-framework-4-2) with our existing Spring JUnit Tests (run via either #TransactionalTestExecutionListener or subclassing AbstractTransactionalUnit4SpringContextTests but, it seems like there's a forced choice -- either run the test without a #Rollback annotation, or the events don't fire. Has anyone come across a good way to test #TransactionalEvents while being able to #Rollback tests?

Stéphane Nicoll is correct: if the TransactionPhase for your #TransactionalEventListener is set to AFTER_COMMIT, then having a transactional test with automatic rollback semantics doesn't make any sense because the event will never get fired.
In other words, there is no way to have an event fired after a transaction is committed if that transaction is never committed.
So if you really want the event to be fired, you have to let the transaction be committed (e.g., by annotating your test method with #Commit). To clean up after the commit, you should be able to use #Sql in isolated mode to execute cleanup scripts after the transaction has committed. For example, something like the following (untested code) might work for you:
#Transactional
#Commit
#Sql(scripts = "/cleanup.sql", executionPhase = AFTER_TEST_METHOD,
config = #SqlConfig(transactionMode = TransactionMode.ISOLATED))
#Test
public void test() { /* ... */ }
Regards,
Sam (author of the Spring TestContext Framework)

Marco's solution works but adding REQUIRES_NEW propagation into business code is not always acceptable. This modifies business process behavior.
So we should assume that we can change only test part.
Solutin 1
#TestComponent // can be used with spring boot
public class TestApplicationService {
#Autowired
public MyApplicationService service;
#Transactional(propagation = Propagation.REQUIRES_NEW)
public void doSomething() {
service.doSomething();
}
}
This wraps the real service into test component that can be decorated with REQUIRES_NEW propagation. This solution doesn't modify other logic than testing.
Solution 2
#Transactional
#Sql(scripts = "/cleanup.sql", executionPhase = AFTER_TEST_METHOD,
config = #SqlConfig(transactionMode = TransactionMode.ISOLATED))
#Test
public void test() {
MyApplicationService target = // ...
target.doSomething();
TestTransaction.flagForCommit(); //Spring-test since 4.1 - thx for Sam Brannen
TestTransaction.end();
// the event is now received by MyListener
// assertions on the side effects of MyListener
// ...
}
This is simplest solution. We can end test transaction and mark it for commit. This forces transactional events to be handled. If test changes data, cleanup sql script must be specified, otherwise we introduce side effects with committed modified data.

Sam Brannen's solution almost works with regard to adam's comment.
Actually the methods annotated with #TransactionalEventListener are called after the test method transaction is committed. This is so because the calling method, which raises the event, is executing within a logical transaction, not a physical one.
Instead, when the calling method is executed within a new physical transaction, then the methods annotated with #TransactionalEventListener are invoked at the right time, i.e., before the test method transaction is committed.
Also, we don't need #Commit on the test methods, since we actually don't care about these transactions. However, we do need the #Sql(...) statement as explained by Sam Brannen to undo the committed changes of the calling method.
See the small example below.
First the listener that is called when the transaction is committed (default behavior of #TransactionalEventListener):
#Component
public class MyListener {
#TransactionalEventListener
public void when(MyEvent event) {
...
}
}
Then the application service that publishes the event listened to by the above class. Notice that the transactions are configured to be new physical ones each time a method is invoked (see the Spring Framework doc for more details):
#Service
#Transactional(propagation = Propagation.REQUIRES_NEW)
public class MyApplicationService {
public void doSomething() {
// ...
// publishes an instance of MyEvent
// ...
}
}
Finally the test method as proposed by Sam Brannen but without the #Commitannotation which is not needed at this point:
#Transactional
#Sql(scripts = "/cleanup.sql", executionPhase = AFTER_TEST_METHOD,
config = #SqlConfig(transactionMode = TransactionMode.ISOLATED))
#Test
public void test() {
MyApplicationService target = // ...
target.doSomething();
// the event is now received by MyListener
// assertions on the side effects of MyListener
// ...
}
This way it works like a charm :-)

Related

Without JPA #Transaction and save() when is the commit done?

When a method has a #Transaction annatotion, I know the commit is done at the end of the method. But when I don't use #Transaction, it's not clear to me when the commit is done. In my example I don't use #Transaction, do the real change in another service and don't use someRepository .save(), but it still works:
#Service
public class ServiceA {
private final SomeRepository someRepository;
private final ServiceB serviceB;
public ServiceA(SomeRepository someRepository, ) {
this.someRepository = someRepository;
this.serviceB = serviceB;
}
// Called from controller
public void doStuff() {
var someEntity = someRepository.findById(1);
serviceB.makeChange(someEntity);
}
}
#Service
public class ServiceB {
public ServiceB() {}
public void makeChange(SomeEntity someEntity) {
someEntity.setName("Test"); // this is working and committed to the database
}
}
So actually I have 2 questions:
When I don't add a #Transaction annatotion to a method when is the commit done?
I don't even have to call someRepository.save(entity)? I thought that worked only when using the #Transaction annotation?
Context:
Spring Boot 2.2.6
"spring-boot-starter-data-jpa" as dependency
first one clarification: the #Transactional annotation does not mean there is a commit at end of the method. It means the method joins the transaction (or start a new one - this depends on the propagation attributes to be precise), so the commit (or rollback) will be performed at the end of the transaction, which can (and often does) involve multiple methods with various DB access.
Normally Spring (or another transaction manager) takes care of this (ie disabling auto-commit).
#Transactional missing
There is no transactional context so the commit is performed immediately as the database in modified. There is no rollback option and, if there is an error, the data integrity might be violated,
#Transactional defined
During the transactions the JPA entities are in managed-state, at the end of the transaction the state is automatically flushed to the DB (no need to call someRepository.save(entity)

What's the "Right Way" to send a data changed websocket event and ensure the database is committed in Spring Boot

Note: read the end of the answer for the way I implemented #Nonika's suggestions
What's the "right way" to send a websocket event on data insert?
I'm using a Spring Boot server with SQL/JPA and non-stomp websockets. I need to use "plain" websockets as I'm using Java clients where (AFAIK) there's no stomp support.
When I make a change to the database I need to send the event to the client so I ended up with an implementation like this:
#Transactional
public void addEntity(...) {
performActualEntityAdding();
sendEntityAddedEvent(eventData);
}
#Transactional
public void sendEntityAddedEvent(String eventData) {
TransactionSynchronizationManager.registerSynchronization(new TransactionSynchronizationAdapter() {
#Override
public void afterCommit() {
sendEntityAddedEventAsync(eventData);
}
});
}
#Async
public void sendEntityAddedEventAsync(String eventData) {
// does the websocket session sending...
}
This works. If I would just call the sendEntityAddedEventAsync it would also work for real world scenarios but it fails on unit tests because the event would arrive before transaction commit. As such when the unit test invokes a list of the entities after the event it fails.
This feels like a hack that shouldn't be here. Is there a better way to ensure a commit?
I tried multiple alternative approaches and the problem is that they often worked for 10 runs of the unit tests yet failed every once in a while. That isn't acceptable.
I tried multiple approaches to solve this such as different transaction annotations and splitting the method to accommodate them. E.g read uncommitted, not supported (to force a commit) etc. Nothing worked for all cases and I couldn't find an authoritative answer for this (probably common) use case that wasn't about STOMP (which is pretty different).
Edit
One of my original attempts looked something like this:
// this shouldn't be in a transaction
public void addEntity(...) {
performActualEntityAdding();
sendEntityAddedEvent(eventData);
}
#Transactional
public void performActualEntityAdding(...) {
//....
}
#Async
public void sendEntityAddedEventAsync(String eventData) {
// does the websocket session sending...
}
The assumption here is that when sendEntityAddedEventAsync is invoked the data would already be in the database. It wasn't for a couple of additional milliseconds.
A few additional details:
Test environment is based on h2 (initially I mistakenly wrote hsql)
Project is generated by JHipster
Level 2 cache is used but disabled as NONE for these entities
Solution (based on #Nonika's answer):
The solution for me included something similar to this:
public class WebEvent extends ApplicationEvent {
private ServerEventDAO event;
public WebEvent(Object source, ServerEventDAO event) {
super(source);
this.event = event;
}
public ServerEventDAO getEvent() {
return event;
}
}
#Transactional
public void addEntity(...) {
performActualEntityAdding();
applicationEventPublisher.publishEvent(new WebEvent(this, evtDao));
}
#Async
#TransactionalEventListener
public void sendEntityAddedEventAsync(WebEvent eventData) {
// does the websocket session sending...
}
This effectively guarantees that the data is committed properly before sending the event and it runs asynchronously to boot. Very nice and simple.
Spring is using AdviceMode.PROXY for both #Async and #Transactional this is quote from the javadoc:
The default is AdviceMode.PROXY. Please note that proxy mode allows
for interception of calls through the proxy only. Local calls within
the same class cannot get intercepted that way; an Async annotation on
such a method within a local call will be ignored since Spring's
interceptor does not even kick in for such a runtime scenario. For a
more advanced mode of interception, consider switching this to
AdviceMode.ASPECTJ.
This rule is common for almost all spring annotations which requires proxy to operate.
Into your first example, you have a #Transactional annotation on both addEntity(..) and performActualEntityAdding(..). I suppose you call addEntity from another class so #Transactional works as expected. process in this scenario can be described in this flow
// -> N1 transaction starts
addEntity(){
performActualEntityAdding()//-> we are still in transaction N1
sendEntityAddedEvent() // -> call to this #Async is a class local call, so this advice is ignored. But if this was an async call this would not work either.
}
//N1 transaction commits;
That's why the test fails. it gets an event that there is a change into the db, but there is nothing because the transaction has not been committed yet.
Scenario 2.
When you don't have a #Transactional addEntity(..) then second transaction for performActualEntityAdding not starts as there is a local call too.
Options:
You can use some middleware class to call these methods to trigger
spring interceptors.
you can use Self injection with Spring
if you have Spring 5.0 there is handy #TransactionalEventListener(phase = TransactionPhase.AFTER_COMMIT)

Spring nested transactions

In my Spring Boot project I have implemented following service method:
#Transactional
public boolean validateBoard(Board board) {
boolean result = false;
if (inProgress(board)) {
if (!canPlayWithCurrentBoard(board)) {
update(board, new Date(), Board.AFK);
throw new InvalidStateException(ErrorMessage.BOARD_TIMEOUT_REACHED);
}
if (!canSelectCards(board)) {
update(board, new Date(), Board.COMPLETED);
throw new InvalidStateException(ErrorMessage.ALL_BOARD_CARDS_ALREADY_SELECTED);
}
result = true;
}
return result;
}
Inside this method I use another service method which is called update:
#Transactional(propagation = Propagation.REQUIRES_NEW)
public Board update(Board board, Date finishedDate, Integer status) {
board.setStatus(status);
board.setFinishedDate(finishedDate);
return boardRepository.save(board);
}
I need to commit changes to database in update method independently of the owner transaction which is started in validateBoard method. Right now any changes is rolling back in case of any exception.
Even with #Transactional(propagation = Propagation.REQUIRES_NEW) it doesn't work.
How to correctly do this with Spring and allow nested transactions ?
This documentation covers your problem - https://docs.spring.io/spring-framework/docs/current/spring-framework-reference/data-access.html#transaction-declarative-annotations
In proxy mode (which is the default), only external method calls coming in through the proxy are intercepted. This means that self-invocation, in effect, a method within the target object calling another method of the target object, will not lead to an actual transaction at runtime even if the invoked method is marked with #Transactional. Also, the proxy must be fully initialized to provide the expected behaviour so you should not rely on this feature in your initialization code, i.e. #PostConstruct.
However, there is an option to switch to AspectJ mode
Using "self" inject pattern you can resolve this issue.
sample code like below:
#Service #Transactional
public class YourService {
//... your member
#Autowired
private YourService self; //inject proxy as an instance member variable ;
#Transactional(propagation= Propagation.REQUIRES_NEW)
public void methodFoo() {
//...
}
public void methodBar() {
//call self.methodFoo() rather than this.methodFoo()
self.methodFoo();
}
}
The point is using "self" rather than "this".
The basic thumb rule in terms of nested Transactions is that they are completely dependent on the underlying database, i.e. support for Nested Transactions and their handling is database dependent and varies with it.
In some databases, changes made by the nested transaction are not seen by the 'host' transaction until the nested transaction is committed. This can be achieved using Transaction isolation in #Transactional (isolation = "")
You need to identify the place in your code from where an exception is thrown, i.e. from the parent method: "validateBoard" or from the child method: "update".
Your code snippet shows that you are explicitly throwing the exceptions.
YOU MUST KNOW::
In its default configuration, Spring Framework’s transaction
infrastructure code only marks a transaction for rollback in the case
of runtime, unchecked exceptions; that is when the thrown exception is
an instance or subclass of RuntimeException.
But #Transactional never rolls back a transaction for any checked exception.
Thus, Spring allows you to define
Exception for which transaction should be rollbacked
Exception for which transaction shouldn't be rollbacked
Try annotating your child method: update with #Transactional(no-rollback-for="ExceptionName") or your parent method.
Your transaction annotation in update method will not be regarded by Spring transaction infrastructure if called from some method of same class. For more understanding on how Spring transaction infrastructure works please refer to this.
Your problem is a method's call from another method inside the same proxy.It's self-invocation.
In your case, you can easily fix it without moving a method inside another service (why do you need to create another service just for moving some method from one service to another just for avoid self-invocation?), just to call the second method not directly from current class, but from spring container. In this case you call proxy second method with transaction not with self-invocatio.
This principle is useful for any proxy-object when you need self-invocation, not only a transactional proxy.
#Service
class SomeService ..... {
-->> #Autorired
-->> private ApplicationContext context;
-->> //or with implementing ApplicationContextAware
#Transactional(any propagation , it's not important in this case)
public boolean methodOne(SomeObject object) {
.......
-->> here you get a proxy from context and call a method from this proxy
-->>context.getBean(SomeService.class).
methodTwo(object);
......
}
#Transactional(any propagation , it's not important in this case)public boolean
methodTwo(SomeObject object) {
.......
}
}
when you do call context.getBean(SomeService.class).methodTwo(object); container returns proxy object and on this proxy you can call methodTwo(...) with transaction.
You could create a new service (CustomTransactionalService) that will run your code in a new transaction :
#Service
public class CustomTransactionalService {
#Transactional(propagation= Propagation.REQUIRES_NEW)
public <U> U runInNewTransaction(final Supplier<U> supplier) {
return supplier.get();
}
#Transactional(propagation= Propagation.REQUIRES_NEW)
public void runInNewTransaction(final Runnable runnable) {
runnable.run();
}
}
And then :
#Service
public class YourService {
#Autowired
private CustomTransactionalService customTransactionalService;
#Transactional
public boolean validateBoard(Board board) {
// ...
}
public Board update(Board board, Date finishedDate, Integer status) {
this.customTransactionalService.runInNewTransaction(() -> {
// ...
});
}
}

#TransactionalEventListener is never being called in Spring Framework 4.2

I am implementing #EventListener from Spring 4.2 and it works fine. When I try to implement a #TransactionalEventListener it never gets called.
The ApplicationListenerMethodTransactionalAdapter makes the check:
if(TransactionSynchronizationManager.isSynchronizationActive())
and it is always false so it skips running the event because it says it is not in a transaction.
The code for the event listener is simply:
#TransactionalEventListener()
public void handleTransactionalAddEvent(Event event)
{
logger.info("Add Event: {}");
}
The code which publishes the event is as follows:
#Override
#Transactional
public Order addToOrder(String username, Long orderId)
{
Order order = getOrder(orderId, username);
publisher.publishEvent(new Event(order, Event.EventType.ADD));
... Code to do stuff to the order ...
updateOrder(order);
return order;
}
If I change the #TransactionEventListener to just be #EventListener, or I add the fallbackExecution attribute it will run fine, but with the #TransactionEventListener it never gets called.
I found out that we had a custom ApplicationEventMulticaster which was conflicting with the transactional event listeners. I removed it and now it works fine.
#TransactionalEventListener is used when its method should be within the transaction.
If no transaction is running, the listener is not invoked at all since we can’t honor the required semantics.

TransactionTemplate vs #Transactional(propagation = Propagation.REQUIRES_NEW)

could someone please explain why the first unit test class works whereas the second test class fails with a lock wait timeout error?
First test class:
public class Test1 extends AbstractTransactionalJUnit4SpringContextTests {
#Before
public void setUp() {
// do stuff through hibernate to populate database with test data
}
#Test
#Transactional(propagation = Propagation.NEVER)
public void testDeleteObject() {
TransactionTemplate transactionTemplate = new TransactionTemplate(transactionManager);
transactionTemplate.execute(new TransactionCallbackWithoutResult() {
#Override
protected void doInTransactionWithoutResult(TransactionStatus status) {
try {
// execute sql that deletes data populated in setUp() [i.e. this will require locks on the objects].
}
});
}
}
Second Test class [Get a lock wait timeout error]:
public class Test2 extends AbstractTransactionalJUnit4SpringContextTests {
#Before
public void setUp() {
// do stuff through hibernate to populate database with test data
}
#Test
#Transactional(propagation = Propagation.REQUIRES_NEW)
public void testObject() {
// execute sql that deletes data populated in setUp() [i.e. this will require locks on the objects].
}
}
I understand that the second test class fails because the two transaction are fighting for the same locks but neither can give up the locks due to its in_progress transaction state. What I'm confused about is why the first test class succeeds in executing the sql. I'm probably understanding this wrong, but doesn't a new transaction also gets created when the transactionTemplate executes the transaction callback? In that case, shouldn't the same thing happen (lock wait timeout)?
TransactionTemplate does not create a new transaction by default. The default propagation behaviour is PROPAGATION_REQUIRED. In your first case there are no locking issues, because setup and deletion are done in the same transaction.
You can see when new transactions are created or existing transactions reused by setting log level to DEBUG for class org.springframework.orm.jpa.JpaTransactionManager (or even for the whole org.springframework package).
Javadoc for propagation behaviour
A deadlock occurs only if there are two or more connections accessing the same data. In case of test case annotated with propagation NEVER you've got only one transaction, one created by TransactionTemplate.
The second case is a bit fuzzy to me. An exception means there are two concurrent connections/transactions - one for setUp and one for testObject. Propagation REQUIRES_NEW indeed enforces another connection even if there is one detected but I would expect setUp to be launched within this transaction as well. You may try to get rid of #Transactional on testObject. AbstractTransactionalJUnit4SpringContextTests is annotated with #Transactional itself with default propagation which is REQUIRED I believe.

Resources