Connection Timeout with testcontainers and redis - spring-boot

I do integration tests using Spring Boot, TestContainers, redis and Junit 5.
I am facing a weird behavior, when I all the integration tests, I keep having this log displaying :
Cannot reconnect to [localhost:55133]: Connection refused: localhost/127.0.0.1:55133
and this exception :
org.springframework.dao.QueryTimeoutException: Redis command timed out; nested exception is io.lettuce.core.RedisCommandTimeoutException: Command timed out after 1 minute(s)
at org.springframework.data.redis.connection.lettuce.LettuceExceptionConverter.convert(LettuceExceptionConverter.java:70)
But I run the tests individually, I dont have this behavior.
I use Junit5 and I am using Junit5 extension to start and stop my redis container :
public class RedisTestContainerExtension implements BeforeAllCallback, AfterAllCallback {
private GenericContainer<?> redis;
#Override
public void beforeAll(ExtensionContext extensionContext) throws Exception {
redis = new GenericContainer<>(DockerImageName.parse("redis:5.0.3-alpine"))
.withCommand("redis-server","--requirepass", "password")
.waitingFor(Wait.forListeningPort())
.withStartupTimeout(Duration.ofMinutes(2))
.withExposedPorts(6379);
redis.start();
System.setProperty("APP_REDIS_CONVERSATIONS_HOST",redis.getHost());
System.setProperty("APP_REDIS_CONVERSATIONS_PORT",redis.getFirstMappedPort().toString());
System.setProperty("APP_REDIS_CONVERSATIONS_PASSWORD","password");
System.setProperty("APP_REDIS_CONVERSATIONS_TTL","600m");
}
#Override
public void afterAll(ExtensionContext extensionContext) throws Exception {
if(redis != null){
redis.stop();
}
}
}
And I add this file as an extension to my integration test :
#ExtendWith({SpringExtension.class, RedisTestContainerExtension.class})
#SpringBootTest(classes = ConversationsApplication.class)
class MyIntegrationTest {
...
}
Can anyone help me fix this situation.

We had a similar issue. The issue was occured only when we execute all tests (or at least not only one specific)
We have another test setup - we are using a base class to manage test testcontainers - where the port-mapping of the containers was applied by overriding the properties via DynamicPropertySource
Our fix was to mark the base-test-class with #DirtiesContext that spring does not reuse the application-context over the tests-classes - see documentation of DynamicPropertySource:
NOTE: if you use #DynamicPropertySource in a base class and discover that tests in subclasses fail because the dynamic properties change between subclasses, you may need to annotate your base class with #DirtiesContext to ensure that each subclass gets its own ApplicationContext with the correct dynamic properties.
Example:
#Slf4j
#SpringBootTest
#DirtiesContext
#Testcontainers
public abstract class AbstractContainerTest {
#Container
private static final ElasticsearchContainer elasticsearchContainer = new DealElasticsearchContainer();
#Container
private static final RedisCacheContainer redisCacheContainer = new RedisCacheContainer();
#DynamicPropertySource
static void databaseProperties(DynamicPropertyRegistry registry) {
log.info("Override properties to connect to Testcontainers:");
log.info("* Test-Container 'Elastic': spring.elasticsearch.rest.uris = {}",
elasticsearchContainer.getHttpHostAddress());
log.info("* Test-Container 'Redis': spring.redis.host = {} ; spring.redis.port = {}",
redisCacheContainer.getHost(), redisCacheContainer.getMappedPort(6379));
registry.add("spring.elasticsearch.rest.uris", elasticsearchContainer::getHttpHostAddress);
registry.add("spring.redis.host", redisCacheContainer::getHost);
registry.add("spring.redis.port", () -> redisCacheContainer.getMappedPort(6379));
}
}
So maybe give it a try to use #DirtiesContext or switch to a setup which uses DynamicPropertySource to override the properties. It was especially build for this case:
Method-level annotation for integration tests that need to add properties with dynamic values to the Environment's set of PropertySources.
This annotation and its supporting infrastructure were originally designed to allow properties from Testcontainers based tests to be exposed easily to Spring integration tests. However, this feature may also be used with any form of external resource whose lifecycle is maintained outside the test's ApplicationContext.

Related

How to stop #CucumberContextConfiguration with #SpringBootTest from reloading application context between every test?

I've got this problem where my application context is reloaded between every test. I'm wiring in my actual application with functional test properties, wiremock etc. to create a functional test environment. Tests have always run fine but now we've added several it's become painfully slow due to the spring application being re-run everytime. The io.cucumber versions I'm using in my pom for cucumber-spring, cucumber-java, cucumber-junit is 7.11.1.
My Functional Test runner is annotated like this:
#RunWith(Cucumber.class)
#CucumberOptions(
features = "classpath:functional/features",
glue = {"com.iggroup.ds.functional.stepdefinitions"},
monochrome = true,
tags = "#FunctionalTest",
plugin = {"pretty", "html:target/cucumber-html-report", "junit:target/cucumber-xml-report.xml"}
)
public class FunctionalTestRunner {
#BeforeClass
public static void beforeClass() {
prepareEnvironment();
}
private static void prepareEnvironment() {
int applicationPort = SocketUtils.findAvailableTcpPort();
System.setProperty("server.port", String.valueOf(applicationPort));
System.setProperty("spring.active.profiles", "FUNCTIONAL_TEST");
System.setProperty("spring.cloud.config.enabled", "false");
System.setProperty("spring.cloud.config.server.bootstrap", "false");
}
}
Inside my glue package the Cucumber Configuration looks like this:
#AutoConfigureWireMock(port = 8089)
#CucumberContextConfiguration
#SpringBootTest(
classes = {
ServiceApplication.class,
RestClients.class
},
webEnvironment = DEFINED_PORT,
properties = {
"spring.profiles.active=FUNCTIONAL_TEST",
"spring.cloud.config.enabled = false"
}
)
public class FunctionalTestSpringCucumberConfiguration {
}
And lastly the application itself looks like this:
#EnableAsync
#EnableCaching
#EnableConfigServer
#SpringBootApplication
#EnableConfigurationProperties
public class ServiceApplication extends SpringBootServletInitializer {
public static void main(String[] args) {
SpringApplication.run(ServiceApplication.class, args);
}
}
I had read somewhere before that the presence of #MockBean was causing unexpected refreshes between context although I never found out as to why - but I have none defined. As far as I can tell across the articles I've been reading, this shouldn't refresh my context every time so wondering if there's any way I can force it not to rewire the ServiceApplication.class in between every scenario?
#AutoConfigureWireMock(port = 8089)
By using Wiremock on fixed port you are dirtying the application context. This means a new application context will be created for each test. The code responsible for this prints a warning that you can see in your logs.
if (portIsFixed(testContext)) {
if (log.isWarnEnabled()) {
log.warn("You've used fixed ports for WireMock setup - "
+ "will mark context as dirty. Please use random ports, as much "
+ "as possible. Your tests will be faster and more reliable and this "
+ "warning will go away");
}
testContext.markApplicationContextDirty(DirtiesContext.HierarchyMode.EXHAUSTIVE);
}

How to start testcontainers with specific docker image?

I have a junit test, making use of testcontainers-1.15.1. How can I start an explicit image? Because:
#SpringBootTest
public class ContainerTest {
private final JdbcDatabaseContainer DB = new MariaDBContainer("mariadb:10.5.8");
static {
DB.start();
}
#Test
public void test() {
}
}
Result: the default 10.3.6 container is started.
[][] 2021-02-04 14:32:50,741 INFO ?.3.6]: Creating container for image: mariadb:10.3.6
[][] 2021-02-04 14:32:51,597 INFO ?.3.6]: Container mariadb:10.3.6 is starting: d9ccf77f4b9165ccd1690ee5cb8437f43e7d853dfe5121d468a391d67eccef7d
application.properties:
spring.datasource.url=jdbc:tc:mariadb:///test
spring.datasource.username=test
spring.datasource.password=test
This might due to an inconsistent behavior of the constructors of the different Testcontainers modules in the past. It was fixed with this commit and should be available since Testcontainers 1.15.0.
Not sure if your sample was pseudo test code, but the following example is a valid copy-pastable example:
public class MariaDbContainerTest {
private static final JdbcDatabaseContainer DB = new MariaDBContainer("mariadb:10.5.8");
static {
DB.start();
}
#Test
public void test() {
}
}
I've tested it for both Testcontainers 1.15.0 and 1.15.1 and it works on my machine.
UPDATE: I've not seen that you also specify the JDBC support of Testcontainers inside your application.properties file in addition to your manual container definition as part of your test.
Pick either the JDBC support OR the manual container definition and your problem should be resolved.
When using the JDBC support you can also specify the version of your database: jdbc:tc:mariadb:10.5.8:///test

Problem with connection to Neo4j test container using Spring boot 2 and JUnit5

Problem with connection to Neo4j test container using Spring boot 2 and JUnit5
int test context. Container started successfully but spring.data.neo4j.uri property has a wrong default port:7687, I guess this URI must be the same when I call neo4jContainer.getBoltUrl().
Everything works fine in this case:
#Testcontainers
public class ExampleTest {
#Container
private static Neo4jContainer neo4jContainer = new Neo4jContainer()
.withAdminPassword(null); // Disable password
#Test
void testSomethingUsingBolt() {
// Retrieve the Bolt URL from the container
String boltUrl = neo4jContainer.getBoltUrl();
try (
Driver driver = GraphDatabase.driver(boltUrl, AuthTokens.none());
Session session = driver.session()
) {
long one = session.run("RETURN 1",
Collections.emptyMap()).next().get(0).asLong();
assertThat(one, is(1L));
} catch (Exception e) {
fail(e.getMessage());
}
}
}
But SessionFactory is not created for the application using autoconfiguration following to these recommendations - https://www.testcontainers.org/modules/databases/neo4j/
When I try to create own primary bean - SessionFactory in test context I get the message like this - "URI cannot be returned before the container is not loaded"
But Application runs and works perfect using autoconfiguration and neo4j started in a container, the same cannot be told about the test context
You cannot rely 100% on Spring Boot's auto configuration (for production) in this case because it will read the application.properties or use the default values for the connection.
To achieve what you want to, the key part is to create a custom (Neo4j-OGM) Configuration bean. The #DataNeo4jTest annotation is provided by the spring-boot-test-autoconfigure module.
#Testcontainers
#DataNeo4jTest
public class TestClass {
#TestConfiguration
static class Config {
#Bean
public org.neo4j.ogm.config.Configuration configuration() {
return new Configuration.Builder()
.uri(databaseServer.getBoltUrl())
.credentials("neo4j", databaseServer.getAdminPassword())
.build();
}
}
// your tests
}
For a broader explanation have a look at this blog post. Esp. the section Using with Neo4j-OGM and SDN.

Activiti Escalation Listener Configuration

I am using activiti 5.18.
Behind the scenes : There are few task which are getting routed though a workflow. Some of these tasks are eligible for escalation. I have written my escalation listener as follows.
#Component
public class EscalationTimerListener implements ExecutionListener {
#Autowired
ExceptionWorkflowService exceptionWorkflowService;
#Override
public void notify(DelegateExecution execution) throws Exception {
//Process the escalated tasks here
this.exceptionWorkflowService.escalateWorkflowTask(execution);
}
}
Now when I start my tomcat server activiti framework internally calls the listener even before my entire spring context is loaded. Hence exceptionWorkflowService is null (since spring hasn't inejcted it yet!) and my code breaks.
Note : this scenario only occurs if my server isn't running at the escalation time of tasks and I start/restart my server post this time. If my server is already running during escalation time then the process runs smoothly. Because when server started it had injected the service and my listener has triggered later.
I have tried delaying activiti configuration using #DependsOn annotation so that it loads after ExceptionWorkflowService is initialized as below.
#Bean
#DependsOn({ "dataSource", "transactionManager","exceptionWorkflowService" })
public SpringProcessEngineConfiguration getConfiguration() {
final SpringProcessEngineConfiguration config = new SpringProcessEngineConfiguration();
config.setAsyncExecutorActivate(true);
config.setJobExecutorActivate(true);
config.setDataSource(this.dataSource);
config.setTransactionManager(this.transactionManager);
config.setDatabaseSchemaUpdate(this.schemaUpdate);
config.setHistory(this.history);
config.setTransactionsExternallyManaged(this.transactionsExternallyManaged);
config.setDatabaseType(this.dbType);
// Async Job Executor
final DefaultAsyncJobExecutor asyncExecutor = new DefaultAsyncJobExecutor();
asyncExecutor.setCorePoolSize(2);
asyncExecutor.setMaxPoolSize(50);
asyncExecutor.setQueueSize(100);
config.setAsyncExecutor(asyncExecutor);
return config;
}
But this gives circular reference error.
I have also tried adding a bean to SpringProcessEngineConfiguration as below.
Map<Object, Object> beanObjectMap = new HashMap<>();
beanObjectMap.put("exceptionWorkflowService", new ExceptionWorkflowServiceImpl());
config.setBeans(beanObjectMap);
and the access the same in my listener as :
Map<Object, Object> registeredBeans = Context.getProcessEngineConfiguration().getBeans();
ExceptionWorkflowService exceptionWorkflowService = (ExceptionWorkflowService) registeredBeans.get("exceptionWorkflowService");
exceptionWorkflowService.escalateWorkflowTask(execution);
This works but my repository has been autowired into my service which hasn't been initialized yet! So it again throws error in service layer :)
So is there a way that I can trigger escalation listeners only after my entire spring context is loaded?
Have you tried binding the class to ApplicationListener?
Not sure if it will work, but equally I'm not sure why your listener code is actually being executed on startup.
Try to set the implementation type of listeners using Java class or delegate expression and then in the class implement JavaDelegate instead of ExecutionListener.

Camel: use datasource configured by spring-boot

I have a project and in it I'm using spring-boot-jdbc-starter and it automatically configures a DataSource for me.
Now I added camel-spring-boot to project and I was able to successfully create routes from Beans of type RouteBuilder.
But when I'm using sql component of camel it can not find datasource. Is there any simple way to add Spring configured datasource to CamelContext? In samples of camel project they use spring xml for datasource configuration but I'm looking for a way with java config. This is what I tried:
#Configuration
public class SqlRouteBuilder extends RouteBuilder {
#Bean
public SqlComponent sqlComponent(DataSource dataSource) {
SqlComponent sqlComponent = new SqlComponent();
sqlComponent.setDataSource(dataSource);
return sqlComponent;
}
#Override
public void configure() throws Exception {
from("sql:SELECT * FROM tasks WHERE STATUS NOT LIKE 'completed'")
.to("mock:sql");
}
}
I have to publish it because although the answer is in the commentary, you may not notice it, and in my case such a configuration was necessary to run the process.
The use of the SQL component should look like this:
from("timer://dbQueryTimer?period=10s")
.routeId("DATABASE_QUERY_TIMER_ROUTE")
.to("sql:SELECT * FROM event_queue?dataSource=#dataSource")
.process(xchg -> {
List<Map<String, Object>> row = xchg.getIn().getBody(List.class);
row.stream()
.map((x) -> {
EventQueue eventQueue = new EventQueue();
eventQueue.setId((Long)x.get("id"));
eventQueue.setData((String)x.get("data"));
return eventQueue;
}).collect(Collectors.toList());
})
.log(LoggingLevel.INFO,"******Database query executed - body:${body}******");
Note the use of ?dataSource=#dataSource. The dataSource name points to the DataSource object configured by Spring, it can be changed to another one and thus use different DataSource in different routes.
Here is the sample/example code (Java DSL). For this I used
Spring boot
H2 embedded Database
Camel
on startup spring-boot, creates table and loads data. Then camel route, runs "select" to pull the data.
Here is the code:
public void configure() throws Exception {
from("timer://timer1?period=1000")
.setBody(constant("select * from Employee"))
.to("jdbc:dataSource")
.split().simple("${body}")
.log("process row ${body}")
full example in github

Resources