How to execute integration tests of spring boot application with using Hazelcast, because when run all tests got hazelcast.core.DuplicateInstanceNameException?
I use Spring Boot 2.0.0.RC1 and Hazelcast 3.9.2
Use java configuration for hazelcast:
#Bean
public Config getHazelCastServerConfig() {
final Config config = new Config();
config.setInstanceName(hzInstance);
config.getGroupConfig().setName(hzGroupName).setPassword(hzGroupPassword);
final ManagementCenterConfig managementCenterConfig = config.getManagementCenterConfig();
managementCenterConfig.setEnabled(true);
managementCenterConfig.setUrl(mancenter);
final MapConfig mapConfig = new MapConfig();
mapConfig.setName(mapName);
mapConfig.setEvictionPolicy(EvictionPolicy.NONE);
mapConfig.setTimeToLiveSeconds(0);
mapConfig.setMaxIdleSeconds(0);
config.getScheduledExecutorConfig(scheduler)
.setPoolSize(16)
.setCapacity(100)
.setDurability(1);
final NetworkConfig networkConfig = config.getNetworkConfig();
networkConfig.setPort(5701);
networkConfig.setPortAutoIncrement(true).setPortCount(30);
final JoinConfig joinConfig = networkConfig.getJoin();
joinConfig.getMulticastConfig().setEnabled(false);
joinConfig.getAwsConfig().setEnabled(false);
final TcpIpConfig tcpIpConfig = joinConfig.getTcpIpConfig();
tcpIpConfig.addMember(memberOne)
.addMember(memberTwo);
tcpIpConfig.setEnabled(true);
return config;
}
#Bean
public HazelcastInstance getHazelCastServerInstance() {
final HazelcastInstance hazelcastInstance = Hazelcast.newHazelcastInstance(getHazelCastServerConfig());
hazelcastInstance.getClientService().addClientListener(new ClientListener() {
#Override
public void clientConnected(Client client) {
System.out.println(String.format("Connected %s %s %s", client.getClientType(), client.getUuid(), client.getSocketAddress()));
log.info(String.format("Connected %s %s %s", client.getClientType(), client.getUuid(), client.getSocketAddress()));
}
#Override
public void clientDisconnected(Client client) {
System.out.println(String.format("Disconnected %s %s %s", client.getClientType(), client.getUuid(), client.getSocketAddress()));
log.info(String.format("Disconnected %s %s %s", client.getClientType(), client.getUuid(), client.getSocketAddress()));
}
});
return hazelcastInstance;
}
I have simple test:
#RunWith(SpringRunner.class)
#SpringBootTest(classes = UpaSdcApplication.class)
#ActiveProfiles("test")
public class CheckEndpoints {
#Autowired
private ApplicationContext context;
private static final String HEALTH_ENDPOINT = "/actuator/health";
private static WebTestClient testClient;
#Before
public void init() {
testClient = org.springframework.test.web.reactive.server.WebTestClient
.bindToApplicationContext(context)
.configureClient()
.filter(basicAuthentication())
.build();
}
#Test
public void testHealth(){
testClient.get().uri(HEALTH_ENDPOINT).accept(MediaType.APPLICATION_JSON)
.exchange()
.expectStatus()
.isOk()
.expectBody()
.json("{\"status\": \"UP\"}");
}
}
If run with test class separate from other tests - it execute fine and passes.
If run wiith other tests - get exception:
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.hazelcast.core.HazelcastInstance]: Factory method 'getHazelCastServerInstance' threw exception; nested exception is com.hazelcast.core.DuplicateInstanceNameException: HazelcastInstance with name 'counter-instance' already exists!
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:185)
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:579)
... 91 more
Caused by: com.hazelcast.core.DuplicateInstanceNameException: HazelcastInstance with name 'counter-instance' already exists!
at com.hazelcast.instance.HazelcastInstanceFactory.newHazelcastInstance(HazelcastInstanceFactory.java:170)
at com.hazelcast.instance.HazelcastInstanceFactory.newHazelcastInstance(HazelcastInstanceFactory.java:124)
at com.hazelcast.core.Hazelcast.newHazelcastInstance(Hazelcast.java:58)
at net.kyivstar.upa.sdc.config.HazelcastConfiguration.getHazelCastServerInstance(HazelcastConfiguration.java:84)
at net.kyivstar.upa.sdc.config.HazelcastConfiguration$$EnhancerBySpringCGLIB$$c7da65f3.CGLIB$getHazelCastServerInstance$0(<generated>)
at net.kyivstar.upa.sdc.config.HazelcastConfiguration$$EnhancerBySpringCGLIB$$c7da65f3$$FastClassBySpringCGLIB$$b920d5ef.invoke(<generated>)
How do you solve this problem? How do you run integration tests?
I had the same problem and I solved it checking if the instance already exists or not:
#Bean
public CacheManager cacheManager() {
HazelcastInstance existingInstance = Hazelcast.getHazelcastInstanceByName(CACHE_INSTANCE_NAME);
HazelcastInstance hazelcastInstance = null != existingInstance
? existingInstance
: Hazelcast.newHazelcastInstance(hazelCastConfig());
return new HazelcastCacheManager(hazelcastInstance);
}
You can see the rest of the code here.
instanceName configuration element is used to create a named Hazelcast member and should be unique for each Hazelcast instance in a JVM. In your case, either you should set a different instance name for each HazelcastInstance bean creation in the same JVM, or you can totally remove instanceName configuration if you don't recall instances by using instance name.
Had the same issue while running my tests. In my case reason was,that spring test framework was trying to launch new context, while keeping old one cached - thus trying to create another hazelcast instance with the same name, while one was already in the cached context.
Once the TestContext framework loads an ApplicationContext (or
WebApplicationContext) for a test, that context is cached and reused
for all subsequent tests that declare the same unique context
configuration within the same test suite.
Read here to understand more about how spring test framework manages test context.
I am working at the solution at the moment, will post it later. One possible solution I can see is droping test context after each test with #DirtiesContext(classMode = DirtiesContext.ClassMode.AFTER_CLASS), but this is very expensive in terms of performance.
Related
I am using docker/testcontainers to run a postgresql db for testing. I have effectively done this for unit testing that is just testing the database access. However, I have now brought springboot testing into the mix so I can test with an embedded web service and I am having problems.
The issue seems to be that the dataSource bean is being requested before the container starts.
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSource' defined in class path resource [com/myproject/integrationtests/IntegrationDataService.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'dataSource' threw exception; nested exception is java.lang.IllegalStateException: Mapped port can only be obtained after the container is started
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [javax.sql.DataSource]: Factory method 'dataSource' threw exception; nested exception is java.lang.IllegalStateException: Mapped port can only be obtained after the container is started
Caused by: java.lang.IllegalStateException: Mapped port can only be obtained after the container is started
Here is my SpringBootTest:
#RunWith(SpringJUnit4ClassRunner.class)
#SpringBootTest(classes = {IntegrationDataService.class, TestApplication.class},
webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
public class SpringBootTestControllerTesterIT
{
#Autowired
private MyController myController;
#LocalServerPort
private int port;
#Autowired
private TestRestTemplate restTemplate;
#Test
public void testRestControllerHello()
{
String url = "http://localhost:" + port + "/mycontroller/hello";
ResponseEntity<String> result =
restTemplate.getForEntity(url, String.class);
assertEquals(result.getStatusCode(), HttpStatus.OK);
assertEquals(result.getBody(), "hello");
}
}
Here is my spring boot application referenced from the test:
#SpringBootApplication
public class TestApplication
{
public static void main(String[] args)
{
SpringApplication.run(TestApplication.class, args);
}
}
Here is the IntegrationDataService class which is intended to startup the container and provide the sessionfactory/datasource for everything else
#Testcontainers
#TestInstance(TestInstance.Lifecycle.PER_CLASS)
#EnableTransactionManagement
#Configuration
public class IntegrationDataService
{
#Container
public static PostgreSQLContainer postgreSQLContainer = (PostgreSQLContainer) new PostgreSQLContainer("postgres:9.6")
.withDatabaseName("test")
.withUsername("sa")
.withPassword("sa")
.withInitScript("db/postgresql/schema.sql");
#Bean
public Properties hibernateProperties()
{
Properties hibernateProp = new Properties();
hibernateProp.put("hibernate.dialect", "org.hibernate.dialect.PostgreSQLDialect");
hibernateProp.put("hibernate.format_sql", true);
hibernateProp.put("hibernate.use_sql_comments", true);
// hibernateProp.put("hibernate.show_sql", true);
hibernateProp.put("hibernate.max_fetch_depth", 3);
hibernateProp.put("hibernate.jdbc.batch_size", 10);
hibernateProp.put("hibernate.jdbc.fetch_size", 50);
hibernateProp.put("hibernate.id.new_generator_mappings", false);
// hibernateProp.put("hibernate.hbm2ddl.auto", "create-drop");
// hibernateProp.put("hibernate.jdbc.lob.non_contextual_creation", true);
return hibernateProp;
}
#Bean
public SessionFactory sessionFactory() throws IOException
{
LocalSessionFactoryBean sessionFactoryBean = new LocalSessionFactoryBean();
sessionFactoryBean.setDataSource(dataSource());
sessionFactoryBean.setHibernateProperties(hibernateProperties());
sessionFactoryBean.setPackagesToScan("com.myproject.model.entities");
sessionFactoryBean.afterPropertiesSet();
return sessionFactoryBean.getObject();
}
#Bean
public PlatformTransactionManager transactionManager() throws IOException
{
return new HibernateTransactionManager(sessionFactory());
}
#Bean
public DataSource dataSource()
{
BasicDataSource dataSource = new BasicDataSource();
dataSource.setDriverClassName(postgreSQLContainer.getDriverClassName());
dataSource.setUrl(postgreSQLContainer.getJdbcUrl());
dataSource.setUsername(postgreSQLContainer.getUsername());
dataSource.setPassword(postgreSQLContainer.getPassword());
return dataSource;
}
}
The error occurs on requesting the datasource bean from the sessionFactory from one of the Dao classes before the container starts up.
What the heck am I doing wrong?
Thanks!!!
The reason for your java.lang.IllegalStateException: Mapped port can only be obtained after the container is started exception is that when the Spring Context now gets created during your test with #SpringBootTest it tries to connect to the database on application startup.
As you only launch your PostgreSQL inside your IntegrationDataService class, there is a timing issue as you can't obtain the JDBC URL or create a connection on application startup as this bean is not yet properly created.
In general, you should NOT use any test-related code inside your IntegrationDataService class. Starting/stopping the database should be done inside your test setup.
This ensures to first start the database container, wait until it's up- and running, and only then launch the actual test and create the Spring Context.
I've summarized the required setup mechanism for JUnit 4/5 with Testcontainers and Spring Boot, that help you get the setup right.
In the end, this can look like the following
// JUnit 5 example with Spring Boot >= 2.2.6
#Testcontainers
#SpringBootTest(webEnvironment = WebEnvironment.RANDOM_PORT)
public class ApplicationIT {
#Container
public static PostgreSQLContainer postgreSQLContainer = new PostgreSQLContainer()
.withPassword("inmemory")
.withUsername("inmemory");
#DynamicPropertySource
static void postgresqlProperties(DynamicPropertyRegistry registry) {
registry.add("spring.datasource.url", postgreSQLContainer::getJdbcUrl);
registry.add("spring.datasource.password", postgreSQLContainer::getPassword);
registry.add("spring.datasource.username", postgreSQLContainer::getUsername);
}
#Test
public void contextLoads() {
}
}
My Spring Boot project uses JUnit 5. I'd like to setup an integration test which requires a local SMTP server to be started, so I implemented a custom extension:
public class SmtpServerExtension implements BeforeAllCallback, AfterAllCallback {
private GreenMail smtpServer;
private final int port;
public SmtpServerExtension(int port) {
this.port = port;
}
#Override
public void beforeAll(ExtensionContext extensionContext) {
smtpServer = new GreenMail(new ServerSetup(port, null, "smtp")).withConfiguration(GreenMailConfiguration.aConfig().withDisabledAuthentication());
smtpServer.start();
}
#Override
public void afterAll(ExtensionContext extensionContext) {
smtpServer.stop();
}
}
Because I need to configure the server's port I register the extension in the test class like this:
#SpringBootTest
#AutoConfigureMockMvc
#ExtendWith(SpringExtension.class)
#ActiveProfiles("test")
public class EmailControllerIT {
#Autowired
private MockMvc mockMvc;
#Autowired
private ObjectMapper objectMapper;
#Value("${spring.mail.port}")
private int smtpPort;
#RegisterExtension
// How can I use the smtpPort annotated with #Value?
static SmtpServerExtension smtpServerExtension = new SmtpServerExtension(2525);
private static final String RESOURCE_PATH = "/mail";
#Test
public void whenValidInput_thenReturns200() throws Exception {
mockMvc.perform(post(RESOURCE_PATH)
.contentType(APPLICATION_JSON)
.content("some content")
).andExpect(status().isOk());
}
}
While this is basically working: How can I use the smtpPort annotated with #Value (which is read from the test profile)?
Update 1
Following your proposal I created a custom TestExecutionListener.
public class CustomTestExecutionListener implements TestExecutionListener {
#Value("${spring.mail.port}")
private int smtpPort;
private GreenMail smtpServer;
#Override
public void beforeTestClass(TestContext testContext) {
smtpServer = new GreenMail(new ServerSetup(smtpPort, null, "smtp")).withConfiguration(GreenMailConfiguration.aConfig().withDisabledAuthentication());
smtpServer.start();
};
#Override
public void afterTestClass(TestContext testContext) {
smtpServer.stop();
}
}
The listener is registered like this:
#TestExecutionListeners(value = CustomTestExecutionListener.class, mergeMode = MERGE_WITH_DEFAULTS)
When running the test the listener gets called but smtpPort is always 0, so it seems as if the #Value annotation is not picked up.
I don't think you should work with Extensions here, or in general, any "raw-level" JUnit stuff (like lifecycle methods), because you won't be able to access the application context from them, won't be able to execute any custom logic on beans and so forth.
Instead, take a look at Spring's test execution listeners abstraction
With this approach, GreenMail will become a bean managed by spring (probably in a special configuration that will be loaded only in tests) but since it becomes a bean it will be able to load the property values and use #Value annotation.
In the test execution listener you'll start the server before the test and stop after the test (or the whole test class if you need that - it has "hooks" for that).
One side note, make sure you mergeMode = MergeMode.MERGE_WITH_DEFAULTS as a parameter to #TestExecutionListeners annotation, otherwise some default behaviour (like autowiring in tests, dirty context if you have it, etc) won't work.
Update 1
Following Update 1 in the question. This won't work because the listener itself is not a spring bean, hence you can't autowire or use #Value annotation in the listener itself.
You can try to follow this SO thread that might be helpful, however originally I meant something different:
Make a GreenMail a bean by itself:
#Configuration
// since you're using #SpringBootTest annotation - it will load properties from src/test/reources/application.properties so you can put spring.mail.port=1234 there
public class MyTestMailConfig {
#Bean
public GreenMail greenMail(#Value(${"spring.mail.port"} int port) {
return new GreenMail(port, ...);
}
}
Now this configuration can be placed in src/test/java/<sub-package-of-main-app>/ so that in production it won't be loaded at all
Now the test execution listener could be used only for running starting / stopping the GreenMail server (as I understood you want to start it before the test and stop after the test, otherwise you don't need these listeners at all :) )
public class CustomTestExecutionListener implements TestExecutionListener {
#Override
public void beforeTestClass(TestContext testContext) {
GreenMail mailServer =
testContext.getApplicationContext().getBean(GreenMail.class);
mailServer.start();
}
#Override
public void afterTestClass(TestContext testContext) {
GreenMail mailServer =
testContext.getApplicationContext().getBean(GreenMail.class);
mailServer.stop();
}
}
Another option is autowiring the GreenMail bean and using #BeforeEach and #AfterEach methods of JUnit, but in this case you'll have to duplicate this logic in different Test classes that require this behavour. Listeners allow reusing the code.
I try to use Test Containers with Oracle-XE module and Spring Boot and so far, when I launch my test, I am confronted to exception :
Caused by: java.lang.IllegalArgumentException: JDBC URL matches jdbc:tc: prefix but the database or tag name could not be identified
In my src/test/application.properties, I declared the url datatasource as :
spring.datasource.url=jdbc:tc:oracle-xe://somehostname:someport/databasename?TC_INITSCRIPT=schema-test.sql
To indicate the docker image to pull for oracle-xe, I created the file testcontainers.properties in src/test/resources :
oracle.container.image=oracleinanutshell/oracle-xe-11g:1.0.0
Do you have any idea how to make this work ?
It works flawlessly with MySQL, with the datasource url :
spring.datasource.url=jdbc:tc:mysql:5.6.23://somehostname:someport/databasename?TC_INITSCRIPT=schema-test.sql
You can make a test configuration class that redefine datasource bean with oracle xe container configuration.
public class OracleIT {
#ClassRule
public static OracleContainer oracleContainer = new OracleContainer();
#BeforeAll
public static void startup() {
oracleContainer.start();
}
#TestConfiguration
static class OracleTestConfiguration {
#Bean
DataSource dataSource() {
HikariConfig hikariConfig = new HikariConfig();
hikariConfig.setJdbcUrl(oracleContainer.getJdbcUrl());
hikariConfig.setUsername(oracleContainer.getUsername());
hikariConfig.setPassword(oracleContainer.getPassword());
return new HikariDataSource(hikariConfig);
}
}
}
I have not added cassandra template bean in my beandefinition class, but it works fine and gives me the required output while running, but while writing junit test class
it is throwing me a error "No bean named 'cassandraTemplate' available".
Why is this issue raising during running my junit test class.
This is my code:
#Configuration
#PropertySource("cassandra.properties")
#EnableCassandraRepositories(basePackages = "...repository")
public class Beandef
{
#Autowired
public Environment environment;
CassandraClusterFactoryBean cluster = new CassandraClusterFactoryBean();
#Bean(name = "clusterFactory")
public CassandraClusterFactoryBean getCluster() {
PoolingOptions poolingOptions = new PoolingOptions();
cluster.setContactPoints(environment.getProperty("cassandra.contactpoints"));
cluster.setPoolingOptions(poolingOptions);
cluster.setPort(Integer.parseInt(environment.getProperty("cassandra.port")));
poolingOptions.setNewConnectionThreshold(HostDistance.LOCAL, 50);
return cluster;
}
#Bean
#DependsOn("clusterFactory")
public CassandraSessionFactoryBean getSession() throws Exception {
CassandraSessionFactoryBean session = new CassandraSessionFactoryBean();
session.setCluster(cluster.getObject());
session.setKeyspaceName(environment.getProperty("cassandra.keyspace"));
session.setConverter(new MappingCassandraConverter(new CassandraMappingContextAware()));
session.setSchemaAction(SchemaAction.NONE);
return session;
}
If you're using Spring Boot, and if your test class doesn't have a #SpringBootApplication available in the classpath, you have to add #EnableAutoConfiguration to enable Spring Boot auto-config (even with spring-boot-starter-data-cassandra in your dependencies).
I'm trying to build a Spring 3.1 PropertySource which reads its values from Zookeeper nodes. For connecting to Zookeeper I am using Curator from Netflix.
For that I've built a custom property source which reads the value of a property from Zookeeper and returns it. This works fine when I am resolving the property like this
ZookeeperPropertySource zkPropertySource = new ZookeeperPropertySource(zkClient);
ctx.getEnvironment().getPropertySources().addLast(zkPropertySource);
ctx.getEnvironment().getProperty("foo"); // returns 'from zookeeper'
However, when I try to instantiate a bean which has a field with an #Value annotation then this fails:
#Component
public class MyBean {
#Value("${foo}") public String foo;
}
MyBean b = ctx.getBean(MyBean.class); // fails with BeanCreationException
This problem has most likely nothing to do with Zookeeper but with the way I'm registering the property sources and creating the beans.
Any insight is highly appreciated.
Update 1:
I'm creating the app context from an XML file like this:
public class Main {
public static void main(String[] args) throws Exception {
ConfigurableApplicationContext ctx = new ClassPathXmlApplicationContext("applicationContext.xml");
ctx.registerShutdownHook();
}
}
The class which connects to Zookeeper is a #Component.
#Component
public class Server {
CuratorFramework zkClient;
public void connectToZookeeper() {
zkClient = ... (curator magic) ...
}
public void registerPropertySource() {
ZookeeperPropertySource zkPropertySource = new ZookeeperPropertySource(zkClient);
ctx.getEnvironment().getPropertySources().addLast(zkPropertySource);
ctx.getEnvironment().getProperty("foo"); // returns 'from zookeeper'
}
#PostConstruct
public void start() {
connectToZookeeper();
registerPropertySource();
MyBean b = ctx.getBean(MyBean.class);
}
}
Update 2
This seems to work when I'm using XML-less configuration, i.e. #Configuration, #ComponentScan and #PropertySource in combination with an AnnotationConfigApplicationContext. Why doesn't it work with a ClassPathXmlApplicationContext?
#Configuration
#ComponentScan("com.goleft")
#PropertySource({"classpath:config.properties","classpath:version.properties"})
public class AppConfig {
#Bean
public static PropertySourcesPlaceholderConfigurer propertySourcesPlaceholderConfigurer() {
return new PropertySourcesPlaceholderConfigurer();
}
}
Answering to your Update 2: This does not work with your original configuration(registering a PropertySource using #PostConstruct) because the PropertySource is being registered very late, by this time your target bean has already been constructed and initialized.
Typically the injection of the placeholders happens via a BeanFactoryPostProcessor which is very early in the Spring lifecycle(beans have not been created at this stage) and if a PropertySource is registered at that stage, then placeholders should be resolved.
The best approach though is to use a ApplicationContextInitializer, get a handle on the applicationContext and to register the propertySource there:
public class CustomInitializer implements ApplicationContextInitializer<ConfigurableWebApplicationContext> {
public void initialize(ConfigurableWebApplicationContext ctx) {
ZookeeperPropertySource zkPropertySource = new ZookeeperPropertySource(zkClient);
ctx.getEnvironment().getPropertySources().addFirst(zkPropertySource);
}
}