I am running integration tests for my API which deals with in-memory LDAP
server. Sometimes tests gets executed properly and sometimes not. Why is this happening ?
I have tried optimizing test-cases and reducing count of test-cases. Individually each test is passing successfully.
//LDAPInMem.java
public class LdapInMem {
{
function startServer()
{
InMemoryDirectoryServer server;
InMemoryDirectoryServerConfig config = new InMemoryDirectoryServerConfig();
//some server configuration code
server.startListening();
}
}
//Integration test
import LDAPInMem
public class UserControllerIntegrationTest {
#Autowired
private WebApplicationContext webApplicationContext;
private MockMvc mockMvc;
#Before
public void setup()
{
mockMvc = MockMvcBuilders.webAppContextSetup(webApplicationContext).build();
LDAPInMem.startServer();
}
#Test
public void fun1()
{
//some mockMvc testcase which deals with in-memory server
}
#Test
public void fun2()
{
//some mockMvc testcase which deals with in-memory server
}
#Test
public void fun3()
{
//some mockMvc testcase which deals with in-memory server
}
}
These test-cases are failing sometimes even though everything else is fine. Why is this happening ? Is it thread related ? What can be done in this case to run these test-cases properly?
Related
I am currently experiencing a weird issue regarding Spring's MockMvc in combination with Spring Security in JUnit Tests.
When I run a whole class of tests, everything works fine and the tests are passing.
But when I run All Tests in the Project one test is always failing in that certain class and it doesn't matter if I remove the test method failing, then another one fails in the same class.
#SpringBootTest
#ExtendWith({RestDocumentationExtension.class, SpringExtension.class})
class AuthenticationApiTest {
#Autowired
private WebApplicationContext webApplicationContext;
private MockMvc mockMvc;
private static ObjectMapper objectMapper;
#BeforeAll
static void beforeAll() {
objectMapper = new ObjectMapper();
}
#BeforeEach
void setUp(RestDocumentationContextProvider documentation) {
this.mockMvc = MockMvcBuilders.webAppContextSetup(this.webApplicationContext)
.apply(documentationConfiguration(documentation)
.operationPreprocessors()
.withRequestDefaults(prettyPrint(),removeHeaders("Content-Length","Host","Pragma","X-XSS-Protection","Expires","X-Frame-Options","X-Content-Type-Options","Cache-Control"))
.withResponseDefaults(prettyPrint(),removeHeaders("Content-Length","Host","Pragma","X-XSS-Protection","Expires","X-Frame-Options","X-Content-Type-Options","Cache-Control")))
.apply(springSecurity())
.build();
}
The test which fails:
#Test
public void testSignUpFail() throws Exception {
SignUpBody signUpBody = new SignUpBody();
signUpBody.setPassword("demo1234");
signUpBody.setFirstname("first");
signUpBody.setLastname("lastn");
this.mockMvc.perform(post("/auth/user/signup")
.header(HttpHeaders.CONTENT_TYPE,"application/json")
.content(objectMapper.writeValueAsString(signUpBody))
).andExpect(status().is(400))
.andDo(document("user-signup-fail"));
this.mockMvc.perform(post("/auth/vendor/signup")
.header(HttpHeaders.CONTENT_TYPE,"application/json")
.content(objectMapper.writeValueAsString(signUpBody))
).andExpect(status().is(400))
.andDo(document("vendor-signup-fail"));
}
Any suggestions on how to solve this weird issue?
My Spring Boot project uses JUnit 5. I'd like to setup an integration test which requires a local SMTP server to be started, so I implemented a custom extension:
public class SmtpServerExtension implements BeforeAllCallback, AfterAllCallback {
private GreenMail smtpServer;
private final int port;
public SmtpServerExtension(int port) {
this.port = port;
}
#Override
public void beforeAll(ExtensionContext extensionContext) {
smtpServer = new GreenMail(new ServerSetup(port, null, "smtp")).withConfiguration(GreenMailConfiguration.aConfig().withDisabledAuthentication());
smtpServer.start();
}
#Override
public void afterAll(ExtensionContext extensionContext) {
smtpServer.stop();
}
}
Because I need to configure the server's port I register the extension in the test class like this:
#SpringBootTest
#AutoConfigureMockMvc
#ExtendWith(SpringExtension.class)
#ActiveProfiles("test")
public class EmailControllerIT {
#Autowired
private MockMvc mockMvc;
#Autowired
private ObjectMapper objectMapper;
#Value("${spring.mail.port}")
private int smtpPort;
#RegisterExtension
// How can I use the smtpPort annotated with #Value?
static SmtpServerExtension smtpServerExtension = new SmtpServerExtension(2525);
private static final String RESOURCE_PATH = "/mail";
#Test
public void whenValidInput_thenReturns200() throws Exception {
mockMvc.perform(post(RESOURCE_PATH)
.contentType(APPLICATION_JSON)
.content("some content")
).andExpect(status().isOk());
}
}
While this is basically working: How can I use the smtpPort annotated with #Value (which is read from the test profile)?
Update 1
Following your proposal I created a custom TestExecutionListener.
public class CustomTestExecutionListener implements TestExecutionListener {
#Value("${spring.mail.port}")
private int smtpPort;
private GreenMail smtpServer;
#Override
public void beforeTestClass(TestContext testContext) {
smtpServer = new GreenMail(new ServerSetup(smtpPort, null, "smtp")).withConfiguration(GreenMailConfiguration.aConfig().withDisabledAuthentication());
smtpServer.start();
};
#Override
public void afterTestClass(TestContext testContext) {
smtpServer.stop();
}
}
The listener is registered like this:
#TestExecutionListeners(value = CustomTestExecutionListener.class, mergeMode = MERGE_WITH_DEFAULTS)
When running the test the listener gets called but smtpPort is always 0, so it seems as if the #Value annotation is not picked up.
I don't think you should work with Extensions here, or in general, any "raw-level" JUnit stuff (like lifecycle methods), because you won't be able to access the application context from them, won't be able to execute any custom logic on beans and so forth.
Instead, take a look at Spring's test execution listeners abstraction
With this approach, GreenMail will become a bean managed by spring (probably in a special configuration that will be loaded only in tests) but since it becomes a bean it will be able to load the property values and use #Value annotation.
In the test execution listener you'll start the server before the test and stop after the test (or the whole test class if you need that - it has "hooks" for that).
One side note, make sure you mergeMode = MergeMode.MERGE_WITH_DEFAULTS as a parameter to #TestExecutionListeners annotation, otherwise some default behaviour (like autowiring in tests, dirty context if you have it, etc) won't work.
Update 1
Following Update 1 in the question. This won't work because the listener itself is not a spring bean, hence you can't autowire or use #Value annotation in the listener itself.
You can try to follow this SO thread that might be helpful, however originally I meant something different:
Make a GreenMail a bean by itself:
#Configuration
// since you're using #SpringBootTest annotation - it will load properties from src/test/reources/application.properties so you can put spring.mail.port=1234 there
public class MyTestMailConfig {
#Bean
public GreenMail greenMail(#Value(${"spring.mail.port"} int port) {
return new GreenMail(port, ...);
}
}
Now this configuration can be placed in src/test/java/<sub-package-of-main-app>/ so that in production it won't be loaded at all
Now the test execution listener could be used only for running starting / stopping the GreenMail server (as I understood you want to start it before the test and stop after the test, otherwise you don't need these listeners at all :) )
public class CustomTestExecutionListener implements TestExecutionListener {
#Override
public void beforeTestClass(TestContext testContext) {
GreenMail mailServer =
testContext.getApplicationContext().getBean(GreenMail.class);
mailServer.start();
}
#Override
public void afterTestClass(TestContext testContext) {
GreenMail mailServer =
testContext.getApplicationContext().getBean(GreenMail.class);
mailServer.stop();
}
}
Another option is autowiring the GreenMail bean and using #BeforeEach and #AfterEach methods of JUnit, but in this case you'll have to duplicate this logic in different Test classes that require this behavour. Listeners allow reusing the code.
I create a Spring 2.3 application using Spring Data REST, Hibernate, Mysql.
I created my tests, I've around 450 tests splitted in about 70 files. Because the persistence layer leans on a multi tenant approach (single db per tenant) using a Hikari connection pool, I've the need to avoid the pool is initializated for each test file but at the same time I need to use #MockBean because I need to mock up some repositories in the entire Spring test contest.
I create a custom annotation for all test in my suite:
#Target(ElementType.TYPE)
#Retention(RetentionPolicy.RUNTIME)
#SpringBootTest
#TestExecutionListeners(value = TestExecutionListener.class, mergeMode = TestExecutionListeners.MergeMode.MERGE_WITH_DEFAULTS)
#Transactional
#ActiveProfiles("test")
public #interface TestConfig {
}
Reading many posts and the doc, I know if I use #MockBean inside a test, the Spring context is reloaded and therefore a new pool connection is created in my case.
My idea is to create a #MockBean and share it with all tests in my suite so the context is not reloaded every time.
I tried several approaches:
#Log4j2
public class TestExecutionListener extends AbstractTestExecutionListener implements Ordered {
#Override
public void beforeTestMethod(TestContext testContext) throws Exception {
try {
TestDbUtils testDbUtils = (TestDbUtils) testContext.getApplicationContext().getBean(TestDbUtils.class);
testDbUtils.truncateDB();
TenantRepository tenantRepository = mock(TenantRepository.class);
testContext.setAttribute("tenantRepository", tenantRepository);
TenantContext.setCurrentTenantId("test");
when(tenantRepository.findByTenantId("test")).thenReturn(testDbUtils.fakeTenant());
} catch (Exception e) {
}
}
#Override
public int getOrder() {
return Integer.MAX_VALUE;
}
}
All my tests are annotated like this:
#TestConfig
#Log4j2
public class InvoiceTests {
#Test
public void test1(){
}
}
Unfortunately my tenantRepository.findByTenantId() is not mocked up. I also tried to create an abstract superclass:
#SpringBootTest
#TestPropertySource(locations = "classpath:application-test.properties")
#TestExecutionListeners(value = TestExecutionListener.class, mergeMode = TestExecutionListeners.MergeMode.MERGE_WITH_DEFAULTS)
#Transactional
#ActiveProfiles("test")
public abstract class AbstractIntegrationTest {
#MockBean
protected TenantRepository tenantRepository;
#MockBean
protected SubscriptionRepository subscriptionRepository;
#Autowired
protected TestDbUtils testDbUtils;
#BeforeAll
public void beforeAll() {
when(tenantRepository.findByTenantId("test")).thenReturn(testDbUtils.fakeTenant());
}
#BeforeEach
public void setup() {
testDbUtils.truncateDB();
TenantContext.setCurrentTenantId("test");
}
}
Even if my tests extended this superclass, during the run all of them were skipped (not sure why).
Is there any way to accomplish what I described?
The documentation says that writing #SpringbootTest doesnt mean we load all the configuration
We should be able to test a slice of application at a time
I have a case where one module talks to elastic search and we spin up EmbeddedElasticsearchServer in integration tests
However I have hard time figuring out how to define #Configuration classes, how to load them in test and how to make sure that elasticsearch server spins up once for all the tests
I am not sure about the #SpringBootTest. But you can use the below template for elasticsearch Integration est with embeddedserver
v7.0.0
#ESIntegTestCase.ClusterScope(scope = ESIntegTestCase.Scope.SUITE, numDataNodes = 1, numClientNodes = 0, transportClientRatio = 0, supportsDedicatedMasters = false)
#RunWith(com.carrotsearch.randomizedtesting.RandomizedRunner.class)
public class TestWatsonBulkIT extends ESIntegTestCase {
#Override
protected boolean addMockHttpTransport() {
return false;
}
#Override
protected Settings nodeSettings(int nodeOrdinal) {
Settings.Builder builder = Settings.builder()
.put(super.nodeSettings(nodeOrdinal))
.put(NetworkModule.TRANSPORT_TYPE_KEY, Netty4Plugin.NETTY_HTTP_TRANSPORT_NAME)
.put(NetworkModule.HTTP_TYPE_KEY, Netty4Plugin.NETTY_HTTP_TRANSPORT_NAME)
.put(HttpTransportSettings.SETTING_HTTP_PORT.getKey(), 9200)
.put(HttpTransportSettings.SETTING_HTTP_HOST.getKey(),"127.0.0.1");
Settings settings = builder.build();
return settings;
}
#Before
public void setUp() throws Exception {
beforeClass();
super.setUp();
}
#Test
public void test_1(){ // your integration test code here }
}
We should be able to test a slice of application at a time
This is possible by providing classes in #SpringBootTest annotation. I usually go for this option in order to not load everything. Let's say,
Controller:
SampleController -> ConsumerService (autowired)
Test
#RunWith(SpringRunner.class)
#SpringBootTest(classes = {ConsumerService.class, SampleController.class})
public class TestSample {
#Autowired
SampleController sam;
#Test
public void testSam() {
sam.sample();
}
}
I do have bunch of services but they won't be loaded and so their dependencies autowired.
If this is not what you're looking for please enhance your question by adding some sample codes.
I have one repository class which which implements CrudRepository. Then in service class I have auto wired this repositary. Then in controller class I have autowired this service.
I want to write test cases of controller Class. I am using below configuration.
#RunWith(SpringJUnit4ClassRunner.class)
#SpringBootTest(webEnvironment = WebEnvironment.RANDOM_PORT)
public class XYZControllerTest {
MockMvc mockMvc;
#Mock
private XYZController xyzController;
#Autowired
private TestRestTemplate template;
#Autowired
XYZRepository xyzRepository;
#Before
public void setup() throws Exception {
mockMvc = MockMvcBuilders.standaloneSetup(xyzController).build();
}
#Test
public void testPanelShouldBeRegistered() throws Exception {
HttpEntity<Object> xyz = getHttpEntity("{\"name\": \"test 1\", \"email\": \"test10000000000001#gmail.com\","
+ " \"registrationNumber\": \"41DCT\",\"registrationDate\":\"2018-08-08T12:12:12\" }");
ResponseEntity<XYZ> response = template.postForEntity("/api/xyz", xyz, XYZ.class);
}
}
My problem is that when I run test case, data is going to insert in DB which is used for application. Can I test it without inserting data in DB.
Conceptually when we are testing services we mock repositories instead of injection.
You need to mock your repository and setup behavior for returning data.
An example :
#MockBean
XYZRepository xyzRepository;
#Test
public void test() {
// other mocks
//
when(xyzRepository.findAll()).thenReturn(Arrays.asList(new XYZ()));
// service calls
// assertions
}