From the title you'll probably tell me "use the #TestTransaction". This is not my question. As I undertand this annotation, it will rollback any changes in the db, in this case:
#QuarkusTest
class Test {
#Inject val repository: Repository
#Test
#TestTransaction
fun test() {
repository.persist(MyClass(...))
}
Now, in the case where you want - from the test - to persist some changes and call an endpoint that also persists updates, how are we supposed to handle it?
#Test
#TestTransaction
fun testWithEndpoint() {
repository.persist(MyClass(...))
given().body(MyClass(...)).post("/save-my-class").then()...
}
This gives me a pessimistic lock exception, I guess because the test starts one blocking transaction, and the server cannot then get the lock. Is there any way the wrap the whole into a single transaction that would be rollbacked at the end?
Related
Note: read the end of the answer for the way I implemented #Nonika's suggestions
What's the "right way" to send a websocket event on data insert?
I'm using a Spring Boot server with SQL/JPA and non-stomp websockets. I need to use "plain" websockets as I'm using Java clients where (AFAIK) there's no stomp support.
When I make a change to the database I need to send the event to the client so I ended up with an implementation like this:
#Transactional
public void addEntity(...) {
performActualEntityAdding();
sendEntityAddedEvent(eventData);
}
#Transactional
public void sendEntityAddedEvent(String eventData) {
TransactionSynchronizationManager.registerSynchronization(new TransactionSynchronizationAdapter() {
#Override
public void afterCommit() {
sendEntityAddedEventAsync(eventData);
}
});
}
#Async
public void sendEntityAddedEventAsync(String eventData) {
// does the websocket session sending...
}
This works. If I would just call the sendEntityAddedEventAsync it would also work for real world scenarios but it fails on unit tests because the event would arrive before transaction commit. As such when the unit test invokes a list of the entities after the event it fails.
This feels like a hack that shouldn't be here. Is there a better way to ensure a commit?
I tried multiple alternative approaches and the problem is that they often worked for 10 runs of the unit tests yet failed every once in a while. That isn't acceptable.
I tried multiple approaches to solve this such as different transaction annotations and splitting the method to accommodate them. E.g read uncommitted, not supported (to force a commit) etc. Nothing worked for all cases and I couldn't find an authoritative answer for this (probably common) use case that wasn't about STOMP (which is pretty different).
Edit
One of my original attempts looked something like this:
// this shouldn't be in a transaction
public void addEntity(...) {
performActualEntityAdding();
sendEntityAddedEvent(eventData);
}
#Transactional
public void performActualEntityAdding(...) {
//....
}
#Async
public void sendEntityAddedEventAsync(String eventData) {
// does the websocket session sending...
}
The assumption here is that when sendEntityAddedEventAsync is invoked the data would already be in the database. It wasn't for a couple of additional milliseconds.
A few additional details:
Test environment is based on h2 (initially I mistakenly wrote hsql)
Project is generated by JHipster
Level 2 cache is used but disabled as NONE for these entities
Solution (based on #Nonika's answer):
The solution for me included something similar to this:
public class WebEvent extends ApplicationEvent {
private ServerEventDAO event;
public WebEvent(Object source, ServerEventDAO event) {
super(source);
this.event = event;
}
public ServerEventDAO getEvent() {
return event;
}
}
#Transactional
public void addEntity(...) {
performActualEntityAdding();
applicationEventPublisher.publishEvent(new WebEvent(this, evtDao));
}
#Async
#TransactionalEventListener
public void sendEntityAddedEventAsync(WebEvent eventData) {
// does the websocket session sending...
}
This effectively guarantees that the data is committed properly before sending the event and it runs asynchronously to boot. Very nice and simple.
Spring is using AdviceMode.PROXY for both #Async and #Transactional this is quote from the javadoc:
The default is AdviceMode.PROXY. Please note that proxy mode allows
for interception of calls through the proxy only. Local calls within
the same class cannot get intercepted that way; an Async annotation on
such a method within a local call will be ignored since Spring's
interceptor does not even kick in for such a runtime scenario. For a
more advanced mode of interception, consider switching this to
AdviceMode.ASPECTJ.
This rule is common for almost all spring annotations which requires proxy to operate.
Into your first example, you have a #Transactional annotation on both addEntity(..) and performActualEntityAdding(..). I suppose you call addEntity from another class so #Transactional works as expected. process in this scenario can be described in this flow
// -> N1 transaction starts
addEntity(){
performActualEntityAdding()//-> we are still in transaction N1
sendEntityAddedEvent() // -> call to this #Async is a class local call, so this advice is ignored. But if this was an async call this would not work either.
}
//N1 transaction commits;
That's why the test fails. it gets an event that there is a change into the db, but there is nothing because the transaction has not been committed yet.
Scenario 2.
When you don't have a #Transactional addEntity(..) then second transaction for performActualEntityAdding not starts as there is a local call too.
Options:
You can use some middleware class to call these methods to trigger
spring interceptors.
you can use Self injection with Spring
if you have Spring 5.0 there is handy #TransactionalEventListener(phase = TransactionPhase.AFTER_COMMIT)
I wrote test that require transactions, it looks like :
#RunWith(SpringRunner.class)
#SpringBootTest(classes = ExchangeApp.class)
#EnableTransactionManagement(proxyTargetClass = true, mode = AdviceMode.PROXY)
#ActiveProfiles({JHipsterConstants.SPRING_PROFILE_TEST})
public abstract class AbstractServiceTest {
So when I run single test method : mvn test -Dtest=TestClassName#method1
works as expected, but
mvn test -Dtest=TestClassName
failed, with weird exceptions, exception says constraint violation in #OneToMany and exceptions with decimal calculations during divide in BigDecimal. Same exceptions when I run in IDE.
It looks like transaction managing missed. Any ideas ?
java.lang.ArithmeticException: Non-terminating decimal expansion; no exact representable decimal result.
org.springframework.dao.DataIntegrityViolationException: could not execute statement; SQL [n/a]; constraint ["FK_OPEN_EXEC_ID: PUBLIC.ORDER_PAIR_OPEN_EXEC FOREIGN KEY(EXECUTIONS_ID) REFERENCES PUBLIC.ORDER_PAIR_OPEN(ID) (2)"; SQL statement:
delete from order_pair_open where id=? [23503-197]]; nested exception is org.hibernate.exception.ConstraintViolationException: could not execute statement
UPD: also I already tried
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven-surefire-plugin.version}</version>
<configuration>
<!-- Force alphabetical order to have a reproducible build -->
<runOrder>alphabetical</runOrder>
<parallel>classes</parallel>
<threadCountClasses>1</threadCountClasses>
<threadCountMethods>1</threadCountMethods>
<threadCountSuites>1</threadCountSuites>
</configuration>
</plugin>
UPD: it is specific to my case. I am trying to test service with #Async method inside, so it seems I have to mark #Transactional on test method name, in order to enable transaction support, thats why I tried to use #EnableTransactionManagement(proxyTargetClass = true, mode = AdviceMode.PROXY) to enable transactions managing during class test. Here it is pseudo code :
class Service1Test extends AbstractServiceTest {
Service1 service1;
Repo1 repo1;
//Does not works with class call, but works with method call
//when I mark this method with #Transactional, mentioned exceptions are gone,
// but I cant check result since "registerSynchronization" were not called
#Test
public void test1() throws InterruptedException {
service1.method1();
synchronized (this) {
wait(2000l);
}
assertThat( repo1.findAll().size()).isEqualTo(1);
//repoN check
}
}
#Service
#Transactional
class Service1 {
Service2 service2;
#Async
public void method1() {
//DB operations...
TransactionSynchronizationManager.registerSynchronization(new TransactionSynchronizationAdapter() {
#Override
public void afterCommit() {
service2.method2();
}
});
}
}
#Service
class Service2 {
Repo1 repo1;
public void method2() {
repo1.save(new Entity1());
}
}
#Service
class Service3 {
#Autowired
private ScheduledExecutorService scheduler;
public void method3() {
scheduler.schedule(() -> {
//other transactional services call
}, 1l, TimeUnit.SECONDS);
}
}
#Repository
interface Repo1 extends JpaRepository<Entity1, Long> {
}
#Entity
class Entity1{
}
I can't comment on JHipster side, but one possible reason is that "Test Transactional" behavior is not applied to the test code.
Just to clarify, by default if you make a test as #Transactional, spring opens one transaction, the test runs, and when its done (regardless whether it has passed or failed) the transaction does the rollback effectively cleaning up the database.
Now, I don't see this in tests, so probably you don't use this behavior.
But this is pure spring, not a spring boot.
Now regarding the spring boot part.
If you use #SpringBootTest with a concrete configuration ExchangeAppin this case, the chances are that it won't load any autoconfiguration-s (for example those that define configurations working with transactions, data source management, etc.).
If you want to "mimic" the load of the microservice, you should run #SpringBootTest without configurations but it's beyond the scope of the question.
A "by the book" spring boot way to test DAO with hibernate is using #DataJpaTest that loads only the database related stuff, but it can't be used with #SpringBootTest - you should choose one.
So for me it's clear that the test does something tricky and definitely not something that follows spring boot conventions, so probably spring/spring boot strikes back :)
Now, regarding the Asynchronous stuff. This can also contribute to the mess because transaction support in spring relies heavily on Thread Local concept, so when the new thread gets executed (on another thread pool or something) the information about transaction does not propagate, so spring can't understand that its still in the same transaction. I see that you use the TransactionSynchronizationManager but without debugging its hard to tell what happens.
Now in order to check why doesn't the transaction get propagated, I think you should debug the application and see:
Whether the services are wrapped in proxy that support transactions (that's what #Transactional does, assuming the relevant BeanPostProcessor was applied)
Check that during each step you're in the transaction
Consider using #Transactional on test / test case, so that it would clean up the changes that have been applied during the test
I've been trying to test out #TransactionalEvents (a feature of Spring 4.2 https://spring.io/blog/2015/02/11/better-application-events-in-spring-framework-4-2) with our existing Spring JUnit Tests (run via either #TransactionalTestExecutionListener or subclassing AbstractTransactionalUnit4SpringContextTests but, it seems like there's a forced choice -- either run the test without a #Rollback annotation, or the events don't fire. Has anyone come across a good way to test #TransactionalEvents while being able to #Rollback tests?
Stéphane Nicoll is correct: if the TransactionPhase for your #TransactionalEventListener is set to AFTER_COMMIT, then having a transactional test with automatic rollback semantics doesn't make any sense because the event will never get fired.
In other words, there is no way to have an event fired after a transaction is committed if that transaction is never committed.
So if you really want the event to be fired, you have to let the transaction be committed (e.g., by annotating your test method with #Commit). To clean up after the commit, you should be able to use #Sql in isolated mode to execute cleanup scripts after the transaction has committed. For example, something like the following (untested code) might work for you:
#Transactional
#Commit
#Sql(scripts = "/cleanup.sql", executionPhase = AFTER_TEST_METHOD,
config = #SqlConfig(transactionMode = TransactionMode.ISOLATED))
#Test
public void test() { /* ... */ }
Regards,
Sam (author of the Spring TestContext Framework)
Marco's solution works but adding REQUIRES_NEW propagation into business code is not always acceptable. This modifies business process behavior.
So we should assume that we can change only test part.
Solutin 1
#TestComponent // can be used with spring boot
public class TestApplicationService {
#Autowired
public MyApplicationService service;
#Transactional(propagation = Propagation.REQUIRES_NEW)
public void doSomething() {
service.doSomething();
}
}
This wraps the real service into test component that can be decorated with REQUIRES_NEW propagation. This solution doesn't modify other logic than testing.
Solution 2
#Transactional
#Sql(scripts = "/cleanup.sql", executionPhase = AFTER_TEST_METHOD,
config = #SqlConfig(transactionMode = TransactionMode.ISOLATED))
#Test
public void test() {
MyApplicationService target = // ...
target.doSomething();
TestTransaction.flagForCommit(); //Spring-test since 4.1 - thx for Sam Brannen
TestTransaction.end();
// the event is now received by MyListener
// assertions on the side effects of MyListener
// ...
}
This is simplest solution. We can end test transaction and mark it for commit. This forces transactional events to be handled. If test changes data, cleanup sql script must be specified, otherwise we introduce side effects with committed modified data.
Sam Brannen's solution almost works with regard to adam's comment.
Actually the methods annotated with #TransactionalEventListener are called after the test method transaction is committed. This is so because the calling method, which raises the event, is executing within a logical transaction, not a physical one.
Instead, when the calling method is executed within a new physical transaction, then the methods annotated with #TransactionalEventListener are invoked at the right time, i.e., before the test method transaction is committed.
Also, we don't need #Commit on the test methods, since we actually don't care about these transactions. However, we do need the #Sql(...) statement as explained by Sam Brannen to undo the committed changes of the calling method.
See the small example below.
First the listener that is called when the transaction is committed (default behavior of #TransactionalEventListener):
#Component
public class MyListener {
#TransactionalEventListener
public void when(MyEvent event) {
...
}
}
Then the application service that publishes the event listened to by the above class. Notice that the transactions are configured to be new physical ones each time a method is invoked (see the Spring Framework doc for more details):
#Service
#Transactional(propagation = Propagation.REQUIRES_NEW)
public class MyApplicationService {
public void doSomething() {
// ...
// publishes an instance of MyEvent
// ...
}
}
Finally the test method as proposed by Sam Brannen but without the #Commitannotation which is not needed at this point:
#Transactional
#Sql(scripts = "/cleanup.sql", executionPhase = AFTER_TEST_METHOD,
config = #SqlConfig(transactionMode = TransactionMode.ISOLATED))
#Test
public void test() {
MyApplicationService target = // ...
target.doSomething();
// the event is now received by MyListener
// assertions on the side effects of MyListener
// ...
}
This way it works like a charm :-)
I have a test like this:
#RunWith(SpringJUnit4ClassRunner.class),
#ContextConfiguration(locations = { "file:war/WEB-INF/application-context.xml" })
#Transactional
public class ServiceImplTest extends AbstractTestNGSpringContextTests
{
#Autowired
private Service service;
#Test
#Rollback(false)
public void testCreate()
{
.....
//save an entity to table_A
service.save(a);
}
}
It seems that the table_A will be cleaned up before each test running(not roolback after test ran),because after each test,all old data entries in the table are cleaned up,only new inserted entry by test is left.How to prevent this "cleaning" action?
The default behavior is to rollback the transactions in testing context. You can override this behavior using the #Rollback(false) annotation on a test method to not rollback the changes made to the DB during that particular test.
That said, it is recommended that each test case is independent and should have its own scenario setup, scenario execution and scenario tear down. Otherwise, the test failure behavior would be difficult to analyze if there are inter-dependencies among tests.
could someone please explain why the first unit test class works whereas the second test class fails with a lock wait timeout error?
First test class:
public class Test1 extends AbstractTransactionalJUnit4SpringContextTests {
#Before
public void setUp() {
// do stuff through hibernate to populate database with test data
}
#Test
#Transactional(propagation = Propagation.NEVER)
public void testDeleteObject() {
TransactionTemplate transactionTemplate = new TransactionTemplate(transactionManager);
transactionTemplate.execute(new TransactionCallbackWithoutResult() {
#Override
protected void doInTransactionWithoutResult(TransactionStatus status) {
try {
// execute sql that deletes data populated in setUp() [i.e. this will require locks on the objects].
}
});
}
}
Second Test class [Get a lock wait timeout error]:
public class Test2 extends AbstractTransactionalJUnit4SpringContextTests {
#Before
public void setUp() {
// do stuff through hibernate to populate database with test data
}
#Test
#Transactional(propagation = Propagation.REQUIRES_NEW)
public void testObject() {
// execute sql that deletes data populated in setUp() [i.e. this will require locks on the objects].
}
}
I understand that the second test class fails because the two transaction are fighting for the same locks but neither can give up the locks due to its in_progress transaction state. What I'm confused about is why the first test class succeeds in executing the sql. I'm probably understanding this wrong, but doesn't a new transaction also gets created when the transactionTemplate executes the transaction callback? In that case, shouldn't the same thing happen (lock wait timeout)?
TransactionTemplate does not create a new transaction by default. The default propagation behaviour is PROPAGATION_REQUIRED. In your first case there are no locking issues, because setup and deletion are done in the same transaction.
You can see when new transactions are created or existing transactions reused by setting log level to DEBUG for class org.springframework.orm.jpa.JpaTransactionManager (or even for the whole org.springframework package).
Javadoc for propagation behaviour
A deadlock occurs only if there are two or more connections accessing the same data. In case of test case annotated with propagation NEVER you've got only one transaction, one created by TransactionTemplate.
The second case is a bit fuzzy to me. An exception means there are two concurrent connections/transactions - one for setUp and one for testObject. Propagation REQUIRES_NEW indeed enforces another connection even if there is one detected but I would expect setUp to be launched within this transaction as well. You may try to get rid of #Transactional on testObject. AbstractTransactionalJUnit4SpringContextTests is annotated with #Transactional itself with default propagation which is REQUIRED I believe.