ActiveMQ Artemis, Spock, & Spring JMSMessagingTemplate - test is unstable - spring-boot

Question

I have a Spring Boot application I'm trying to migrate to use spring-jms dependency instead of spring-activemq.
I have a few tests written with Spock. I also want to use either spring-jms-server or artemis-junit module. These tests use a JmsMessagingTemplate to send a message on a multicast address (topic), the message is then processed by some #JmsListeners on the consumers queues of these topics.
It seems that the test fails sometimes because the message sent via a JmsMessagingTemplate fails to be acknowledged or never arrives at the listeners.
pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.7.2</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>be.fgov.minfin.esbsoa.ems.ccncsi</groupId>
<artifactId>audit</artifactId>
<version>3.10.0-SNAPSHOT</version>
<properties>
<java.version>1.8</java.version>
<common-lib.version>1.8.0-SNAPSHOT</common-lib.version>
<jacoco.version>0.8.5</jacoco.version>
<sonar.tests>src/test/groovy,src/test-integration/groovy</sonar.tests>
<spock.version>2.2-M3-groovy-3.0</spock.version>
<logbook-spring-boot-starter.version>2.14.0</logbook-spring-boot-starter.version>
<jacoco.version>0.8.5</jacoco.version>
<sonar-maven-plugin.version>3.9.1.2184</sonar-maven-plugin.version>
<maven-compiler-plugin.version>3.8.1</maven-compiler-plugin.version>
<maven-surefire-plugin.version>3.0.0-M5</maven-surefire-plugin.version>
<gmavenplus-plugin.version>1.13.1</gmavenplus-plugin.version>
</properties>
<dependencies>
<dependency>
<groupId>be.fgov.minfin.esbsoa.ems.ccncsi</groupId>
<artifactId>common-lib</artifactId>
<version>${common-lib.version}</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-artemis</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
<exclusions>
<exclusion>
<groupId>com.zaxxer</groupId>
<artifactId>HikariCP</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-validation</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-aop</artifactId>
</dependency>
<dependency>
<groupId>org.zalando</groupId>
<artifactId>logbook-spring-boot-starter</artifactId>
<version>${logbook-spring-boot-starter.version}</version>
</dependency>
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-prometheus</artifactId>
</dependency>
<dependency>
<groupId>org.apache.tomcat</groupId>
<artifactId>tomcat-jdbc</artifactId>
<version>10.0.23</version>
</dependency>
<dependency>
<groupId>com.ibm.db2</groupId>
<artifactId>jcc</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.activemq</groupId>
<artifactId>artemis-junit</artifactId>
<version>2.21.0</version>
</dependency>
<!-- <!– https://mvnrepository.com/artifact/org.apache.activemq/artemis-server –>-->
<!-- <dependency>-->
<!-- <groupId>org.apache.activemq</groupId>-->
<!-- <artifactId>artemis-server</artifactId>-->
<!-- <version>2.21.0</version>-->
<!-- </dependency>-->
<dependency>
<groupId>org.spockframework</groupId>
<artifactId>spock-spring</artifactId>
<version>${spock.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<scope>runtime</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>${maven-compiler-plugin.version}</version>
<configuration>
<source>${java.version}</source>
<target>${java.version}</target>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven-surefire-plugin.version}</version>
<configuration>
<enableAssertions>false</enableAssertions>
</configuration>
<executions>
<execution>
<id>unit-tests</id>
<phase>test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<includes>
<include>**/*Spec.java</include>
</includes>
<excludes>
<exclude>**/*ITSpec.java</exclude>
</excludes>
</configuration>
</execution>
<execution>
<id>integration-tests</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<!-- Never skip running the tests when the integration-test phase is invoked -->
<skip>false</skip>
<includes>
<!-- Include integration tests within integration-test phase. -->
<include>**/*ITSpec.java</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.codehaus.gmavenplus</groupId>
<artifactId>gmavenplus-plugin</artifactId>
<version>${gmavenplus-plugin.version}</version>
<configuration>
<testSources>
<testSource>
<directory>src/test/groovy</directory>
<includes>
<include>**/*.groovy</include>
</includes>
</testSource>
<testSource>
<directory>src/test-integration/groovy</directory>
<includes>
<include>**/*.groovy</include>
</includes>
</testSource>
</testSources>
</configuration>
<executions>
<execution>
<id>unit-test-compile</id>
<goals>
<goal>compileTests</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.jacoco</groupId>
<artifactId>jacoco-maven-plugin</artifactId>
<version>${jacoco.version}</version>
<executions>
<execution>
<goals>
<goal>prepare-agent</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.sonarsource.scanner.maven</groupId>
<artifactId>sonar-maven-plugin</artifactId>
<version>${sonar-maven-plugin.version}</version>
</plugin>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<excludes>
<exclude>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
</exclude>
</excludes>
</configuration>
<executions>
<execution>
<goals>
<goal>repackage</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
In my specification, I use either
#Rule
public EmbeddedActiveMQResource server = new EmbeddedActiveMQResource();
or the artemis-server dependency to start a server.
I also tried to use sleep() and #Retry but tests seem to hang in the send() method even when they are retried. In the logs, I can see that the sent message is sometimes not acknowledged. If this is the case once, retry hangs in the same spot.
You can also see my attempt to get a "fresh" server by using : start(), setReceiveDelay() and stop() on the server in the setup() and cleanup() fixtures.
AuditMessageArtemisSpec.groovy
package be.fgov.minfin.esbsoa.ems.ccncsi.audit.service
import be.fgov.minfin.esbsoa.ems.ccncsi.audit.domain.AuditMessage
import be.fgov.minfin.esbsoa.ems.ccncsi.audit.domain.AuditMessageHistory
import be.fgov.minfin.esbsoa.ems.ccncsi.audit.repository.AuditMessageHistoryRepository
import be.fgov.minfin.esbsoa.ems.ccncsi.audit.repository.AuditMessageRepository
import be.fgov.minfin.esbsoa.ems.ccncsi.common.domain.Headers
import be.fgov.minfin.esbsoa.ems.ccncsi.common.service.JmsService
import com.fasterxml.jackson.databind.ObjectMapper
import groovy.util.logging.Slf4j
import org.apache.activemq.artemis.junit.EmbeddedActiveMQResource
import org.junit.Rule
import org.junit.runner.RunWith
import org.springframework.beans.factory.annotation.Autowired
import org.springframework.beans.factory.annotation.Value
import org.springframework.boot.test.context.SpringBootTest
import org.springframework.context.annotation.ComponentScan
import org.springframework.test.annotation.DirtiesContext
import org.springframework.test.context.ActiveProfiles
import org.springframework.test.context.junit4.SpringRunner
import spock.lang.Retry
import spock.lang.Specification
#RunWith(SpringRunner.class)
#SpringBootTest
#ActiveProfiles("artemis")
#ComponentScan("be.fgov.minfin.esbsoa.ems.ccncsi")
#Slf4j
class AuditMessageServiceArtemisSpec extends Specification{
static final SLEEP_TIME = 3000
#Rule
public EmbeddedActiveMQResource server = new EmbeddedActiveMQResource();
#Autowired
private JmsService jmsService;
#Autowired
private AuditMessageRepository repository
#Autowired
private AuditMessageHistoryRepository historyRepository
#Value("\${ccncsi.audit.topic.toEurope}")
private String europeDestination
#Value("\${ccncsi.audit.topic.commands}")
private String commandsDestination
private ObjectMapper jsonMapper = new ObjectMapper()
//def setup() {
// server.start()
// server.setDefaultReceiveTimeout(SLEEP_TIME)
//}
#Retry(delay = 10000)
def 'Send a valid message to Europe'() {
when:
def msg = send(europeDestination,"tests/message1.json")
then: "Verify if message is inserted in table AUDIT_MESSAGE"
sleep(SLEEP_TIME)
List<AuditMessage> messages = repository.findAll().collect()
messages.size() == 1
AuditMessage auditMessage = messages.get(0)
auditMessage.id != null
auditMessage.messageId == msg.headers.get(Headers.MESSAGE_ID)
auditMessage.reference == null
auditMessage.content != null
auditMessage.content.contentType == "text/plain"
new String(Base64.decoder.decode(auditMessage.content.content)) == "Here is the body of message1"
auditMessage.contentStored
auditMessage.messageTimestamp != null
auditMessage.creationDate != null
check(auditMessage, msg, Headers.MESSAGE_ID, Headers.APPLICATION, Headers.CONTENT_TYPE, Headers.COUNTRY_CODE, Headers.QUEUE_BASE_NAME, Headers.MESSAGE_TYPE)
and: "Check method findByMessageIdOrderByCreation"
List<AuditMessage> messagesByMessageId = repository.findByMessageIdOrderByCreationDate(auditMessage.getMessageId())
messagesByMessageId.size() == 1
messagesByMessageId.get(0).id == auditMessage.id
and: "Check if a status event is sent"
sleep(SLEEP_TIME)
List<AuditMessageHistory> histories = historyRepository.findAll().collect()
histories.size() == 1
}
#Retry(delay = 10000)
def 'Send a reply message from Europe - with correlationId'() {
when:
def msg = send(europeDestination,"tests/message2.json")
sleep(SLEEP_TIME)
then: "Verify if message is inserted in table AUDIT_MESSAGE"
List<AuditMessage> messagesByMessageId = repository.findByMessageIdOrderByCreationDate(msg.headers.get(Headers.MESSAGE_ID))
messagesByMessageId.size() == 1
messagesByMessageId.get(0).reference != null
}
#Retry(delay = 10000)
def 'Send message with body null'() {
when:
def msg = send(europeDestination,"tests/message5.json")
sleep(SLEEP_TIME)
then: "Verify if message is inserted in table AUDIT_MESSAGE and contentStored is false"
List<AuditMessage> messagesByMessageId = repository.findByMessageIdOrderByCreationDate(msg.headers.get(Headers.MESSAGE_ID))
messagesByMessageId.size() == 1
!messagesByMessageId.get(0).contentStored
}
#Retry(delay = 10000)
def 'Bad message3.json -> No inserts'() {
when:
def msg = send(europeDestination,"tests/message3.json")
sleep(SLEEP_TIME)
then: "Verify if message is not inserted in table AUDIT_MESSAGE"
List<AuditMessage> messagesByMessageId = repository.findByMessageIdOrderByCreationDate(msg.headers.get(Headers.MESSAGE_ID))
messagesByMessageId.size() == 0
List<AuditMessageHistory> messageHistories = repository.findByMessageIdOrderByCreationDate(msg.headers.get(Headers.MESSAGE_ID))
messageHistories.size() == 0
}
//def cleanup() {
// server.stop()
// sleep(SLEEP_TIME)
//}
def get(String path) {
URL url = this.getClass().getClassLoader().getResource(path)
return jsonMapper.readValue(url, AuditTestMessage.class)
}
def send(String destination, String path) {
log.error("sending message from file: {} to {}", path.toString(), destination)
AuditTestMessage msg = get(path)
jmsService.send(destination, msg.getHeaders(), msg.getBody())
return msg
}
def check(AuditMessage msg1, AuditTestMessage msg2, String... keys) {
for (String key : keys) {
if (msg2.getHeaders().get(key) != msg1.getHeaders().get(key)) {
log.error("Header ["+key+"] : "+ msg1.getHeaders().get(key) + " != " + msg2.getHeaders().get(key))
return false
}
}
return true
}
}
JMSConfig.java
Below you can see an attempt at setting subdomain to true on the JMSFactory as I'm sending the messages to a topic. This did not change any behavior.
package be.fgov.minfin.esbsoa.ems.ccncsi.audit.config;
import be.fgov.minfin.esbsoa.ems.ccncsi.common.service.JmsService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.autoconfigure.jms.DefaultJmsListenerContainerFactoryConfigurer;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.jms.annotation.EnableJms;
import org.springframework.jms.config.DefaultJmsListenerContainerFactory;
import org.springframework.jms.config.JmsListenerContainerFactory;
import org.springframework.jms.core.JmsMessagingTemplate;
import org.springframework.jms.support.destination.DynamicDestinationResolver;
import org.springframework.lang.Nullable;
import org.springframework.messaging.support.MessageBuilder;
import javax.jms.ConnectionFactory;
import javax.jms.Destination;
import javax.jms.JMSException;
import javax.jms.Session;
import java.util.Map;
#Configuration
#EnableJms
public class JMSConfig {
#Bean
public JmsListenerContainerFactory<?> myFactory(ConnectionFactory connectionFactory,
DefaultJmsListenerContainerFactoryConfigurer configurer) {
DefaultJmsListenerContainerFactory factory = new DefaultJmsListenerContainerFactory();
// This provides all boot's default to this factory, including the message converter
configurer.configure(factory, connectionFactory);
//factory.setPubSubDomain(true);
// You could still override some of Boot's default if necessary.
return factory;
}
#Bean
JmsService jmsService(#Autowired JmsMessagingTemplate jmsMessagingTemplate) {
return new JmsService() {
#Override
public void send(String destination, Object payload) {
jmsMessagingTemplate.send(destination, MessageBuilder.withPayload(payload).build());
}
#Override
public void send(String destination, Map<String, Object> headers, Object payload) {
if (headers == null) send(destination, payload);
else jmsMessagingTemplate.send(destination, MessageBuilder.withPayload(payload).copyHeaders(headers).build());
}
};
}
}
MessageListener.java
package be.fgov.minfin.esbsoa.ems.ccncsi.audit.listener;
import be.fgov.minfin.esbsoa.ems.ccncsi.audit.domain.AuditMessage;
import be.fgov.minfin.esbsoa.ems.ccncsi.audit.service.AuditMessageHistoryService;
import be.fgov.minfin.esbsoa.ems.ccncsi.audit.service.AuditMessageMapper;
import be.fgov.minfin.esbsoa.ems.ccncsi.audit.service.AuditMessageService;
import be.fgov.minfin.esbsoa.ems.ccncsi.common.domain.DestinationType;
import be.fgov.minfin.esbsoa.ems.ccncsi.common.domain.command.Command;
import be.fgov.minfin.esbsoa.ems.ccncsi.common.domain.event.StatusEvent;
import be.fgov.minfin.esbsoa.ems.ccncsi.common.domain.event.StatusType;
import be.fgov.minfin.esbsoa.ems.ccncsi.common.service.JsonService;
import be.fgov.minfin.esbsoa.ems.ccncsi.common.service.StatusEventService;
import com.fasterxml.jackson.databind.JsonNode;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.jms.annotation.JmsListener;
import org.springframework.jms.core.JmsMessagingTemplate;
import org.springframework.jms.support.JmsHeaders;
import org.springframework.messaging.Message;
import org.springframework.stereotype.Component;
import javax.transaction.Transactional;
import java.io.IOException;
#Component
#Slf4j
public class MessageListener {
private final AuditMessageService service;
private final AuditMessageMapper mapper;
private final StatusEventService statusEventService;
private final JsonService jsonService;
private final AuditMessageHistoryService historyService;
private final JmsMessagingTemplate jmsMessagingTemplate;
#Value("${ccncsi.audit.queue.fromEuropeError:Consumer.ccncsiaudit.VirtualTopic.ccncsi.fromEurope.rme}")
private String fromEuropeError;
#Value("${ccncsi.audit.queue.toEuropeError:Consumer.ccncsiaudit.VirtualTopic.ccncsi.toEurope.rme}")
private String toEuropeError;
#Autowired
public MessageListener(AuditMessageService service, AuditMessageMapper mapper, StatusEventService statusEventService,
be.fgov.minfin.esbsoa.ems.ccncsi.common.service.JsonService jsonService, AuditMessageHistoryService historyService,
JmsMessagingTemplate jmsMessagingTemplate) {
this.mapper = mapper;
this.service = service;
this.statusEventService = statusEventService;
this.jsonService = jsonService;
this.historyService = historyService;
this.jmsMessagingTemplate = jmsMessagingTemplate;
}
#JmsListener(destination = "${ccncsi.audit.queue.toEurope:Consumer.ccncsiaudit.VirtualTopic.ccncsi.toEurope}", concurrency = "${ccncsi.audit.queue.concurrency:1-10}")
#Transactional(value = Transactional.TxType.REQUIRED)
public void onMessageToEurope(Message<?> message) {
save(message, DestinationType.TO_EUROPE);
}
#JmsListener(destination = "${ccncsi.audit.queue.fromEurope:Consumer.ccncsiaudit.VirtualTopic.ccncsi.fromEurope}", concurrency = "${ccncsi.audit.queue.concurrency:1-10}")
#Transactional(value = Transactional.TxType.REQUIRED)
public void onMessageFromEurope(Message<?> message) {
save(message, DestinationType.FROM_EUROPE);
}
private void save(Message<?> message, DestinationType destination) {
AuditMessage auditMessage = service.save(mapper.map(message, destination));
statusEventService.notify(message.getHeaders(), auditMessage.getId(), StatusType.AUDIT_STORE_SUCCESS);
}
#JmsListener(destination = "${ccncsi.audit.queue.events:Consumer.ccncsiaudit.VirtualTopic.ccncsi.events}", concurrency = "${ccncsi.audit.queue.concurrency:1-10}")
#Transactional(value = Transactional.TxType.REQUIRED)
public void onEventMessage(Message<?> event) throws IOException {
StatusEvent statusEvent = jsonService.fromString((String) event.getPayload(), StatusEvent.class);
historyService.save(statusEvent);
}
#JmsListener(destination = "${ccncsi.audit.queue.commands:Consumer.ccncsiaudit.VirtualTopic.ccncsi.commands}", concurrency = "${ccncsi.audit.queue.concurrency:1-10}")
#Transactional(value = Transactional.TxType.REQUIRED)
public void onCommandMessage(Message<?> command) throws Exception {
JsonNode jsonCommand = jsonService.fromString((String) command.getPayload());
service.execute((Command) jsonService.fromJSonNode(jsonCommand, Class.forName(jsonCommand.get("name").textValue())));
}
#JmsListener(destination = "DLQ.${ccncsi.audit.queue.toEurope:Consumer.ccncsiaudit.VirtualTopic.ccncsi.toEurope}")
#Transactional(value = Transactional.TxType.REQUIRED)
public void onErrorToEurope(Message<?> message) {
statusEventService.notify(message.getHeaders(), message.getHeaders().get(JmsHeaders.MESSAGE_ID), StatusType.AUDIT_STORE_FAILED);
jmsMessagingTemplate.send(toEuropeError, message);
}
#JmsListener(destination = "DLQ.${ccncsi.audit.queue.fromEurope:Consumer.ccncsiaudit.VirtualTopic.ccncsi.fromEurope}")
#Transactional(value = Transactional.TxType.REQUIRED)
public void onErrorFromEurope(Message<?> message) {
statusEventService.notify(message.getHeaders(), message.getHeaders().get(JmsHeaders.MESSAGE_ID), StatusType.AUDIT_STORE_FAILED);
jmsMessagingTemplate.send(fromEuropeError, message);
}
}
Below you can see that I modified the addresses to the artemis format. The specification sends messages to VirtualTopic.ccncsi.toEurope (topic) and are received on the consumer queues of this address.
application.yml (relevant section)
ccncsi:
common:
statusEventDestination: VirtualTopic.ccncsi.events
audit:
#Maximum size of content (in Kilobytes)
max-size-content-storage: 5000
topic:
toEurope: VirtualTopic.ccncsi.toEurope
fromEurope: VirtualTopic.ccncsi.fromEurope
commands: VirtualTopic.ccncsi.commands
queue:
toEurope: VirtualTopic.ccncsi.toEurope::Consumer.ccncsiaudit.VirtualTopic.ccncsi.toEurope
fromEurope: VirtualTopic.ccncsi.fromEurope::Consumer.ccncsiaudit.VirtualTopic.ccncsi.fromEurope
events: VirtualTopic.ccncsi.events::Consumer.ccncsiaudit.VirtualTopic.ccncsi.events
commands: VirtualTopic.ccncsi.commands::Consumer.ccncsiaudit.VirtualTopic.ccncsi.commands
concurrency: 1-30
application-junit.yml
spring:
jpa:
show-sql: true
hibernate:
ddl-auto: create
datasource:
url: jdbc:h2:mem:testdb;NON_KEYWORDS=KEY,VALUE;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE
username: sa
password: password
driverClassName: org.h2.Driver
jms:
pub-sub-domain: true
template:
default-destination: VirtualTopic.ccncsi.toEurope
# artemis:
# embedded:
# enabled: true
# persistent: true
# mode: embedded
# broker-url: vm://0?virtualTopicConsumerWildcards=Consumer.*.%3E%3B2
Can anyone tell me what I am doing wrong ?