How to load Spring Batch Job using JobLauncherTestUtils for tests - spring-boot

I want to test a job that I used to load as a SpringBootTest and SpringJunit4Runner. As I upgraded to JUnit 5 the jobLauncherTestUtils class no longer loads. The project is a Spring Batch application using Spring Boot 2.2.0.RELEASE. My main configuration is called AppConfig and I have configured the step and job as beans that I can autowire in the test class. However, the application context which used to load now longer loads even. The error tells me the job is not added to the jobLauncherTestUtils. I do not understand why the job can no longer be loaded when it could before. I'd appreciate some help in fixing this issue
src/main/com/indigo/search/config/AppConfig
#Bean
public Step orderIntakeStep() {
return stepBuilderFactory.get("orderIntakeStep")
.<Order, Order>chunk(30)
.reader(orderReader())
.processor(orderProcessor())
.writer(orderWriter())
.build();
}
#Bean(name = "orderIntakeJob")
public Job orderIntakeJob() {
return jobBuilderFactory.get("orderIntakeJob")
.incrementer(new RunIdIncrementer())
.flow(orderIntakeStep())
.end()
.build();
}
...
}
#ExtendWith(SpringExtension.class)
#SpringBatchTest
#Transactional(propagation = Propagation.NOT_SUPPORTED)
class OrderIntakeJobTest {
#Autowired
private JobLauncherTestUtils jobLauncherTestUtils;
#Autowired
private JobRepositoryTestUtils jobRepositoryTestUtils;
#Autowired
private JobLauncher jobLauncher;
#Autowired
Job orderIntakeJob;
...
#Before
public void initJob(){
jobLauncherTestUtils.setLauncher(jobLauncher);
jobLauncherTestUtils.setJobLauncher(jobLauncher);
jobLauncherTestUtils.setJobRepository(jobRepository);
jobLauncherTestUtils.setJob(orderIntakeJob);
j
}

From what you shared, there is nothing that imports a configuration class that contains the job under test in OrderIntakeJobTest. You should have something like:
#ExtendWith(SpringExtension.class)
#SpringBatchTest
#Transactional(propagation = Propagation.NOT_SUPPORTED)
#ContextConfiguration(classes = MyJobConfiguration.class) // this is where the job under test is defined
class OrderIntakeJobTest {
// ...
}

Related

Spring Boot JUnit tests fail with Status expected:<200> but was:<404>

For some time I've been struggling to make JUnit tests for my rest controller. For some reason, every time I try to run them I get the error Status expected:<200> but was:<404>. Here is my controller:
#RestController
#RequestMapping("/travels")
#RequiredArgsConstructor
public class TravelController {
private final TravelService travelService;
private final TravelOutputDtoMapper travelOutputDtoMapper;
#GetMapping
public List<TravelOutputDto> getAll() {
List<Travel> travels = travelService.getAll();
return travels.stream()
.map(travelOutputDtoMapper::travelToTravelOutputDto)
.collect(Collectors.toList());
}
}
And here is my test:
#ExtendWith(SpringExtension.class)
#WebMvcTest(controllers = TravelController.class)
#ContextConfiguration(classes = {
TravelOutputDtoMapper.class,
TravelOutputDtoMapperImpl.class
})
class TravelControllerTest {
#Autowired
private MockMvc mockMvc;
#MockBean
private TravelService travelService;
#Autowired
private TravelOutputDtoMapper travelOutputDtoMapper;
#Test
void testGetAll() throws Exception {
List<Travel> travels = mockTravelList();
Mockito.when(travelService.getAll()).thenReturn(travels);
mockMvc.perform(get("/travels"))
.andExpect(status().isOk());
}
private List<Travel> mockTravelList() {
// Dummy travel list
}
}
I think the reason is connected with TravelOutputDtoMapper as if I remove it from the controller and don't try to inject it the tests are passing, but I cannot find any information why it is doing it. The autowired mapper has an instance and works just fine.
Here is the Mapper:
#Mapper(componentModel = "spring")
public interface TravelOutputDtoMapper {
#Mapping(target = "from", source = "entity.from.code")
#Mapping(target = "to", source = "entity.to.code")
TravelOutputDto travelToTravelOutputDto(Travel entity);
}
The #ContextConfiguration annotation is used for a different purpose:
#ContextConfiguration defines class-level metadata that is used to determine how to load and configure an ApplicationContext for integration tests.
Using Spring Boot and #WebMvcTest there's no need to manually specify how to load the context. That's done for you in the background.
If you'd use this annotation, you'd specify your main Spring Boot class here (your entry-point class with the #SpringBootApplication annotation).
From what I can see in your test and your question is that you want to provide an actual bean for the TravelOutputDtoMapper, but mock the TravelService.
In this case, you can use #TestConfiguration to add further beans to your sliced Spring TestContext:
// #ExtendWith(SpringExtension.class) can be removed. This extension is already registered with #WebMvcTest
#WebMvcTest(controllers = TravelController.class)
class TravelControllerTest {
#Autowired
private MockMvc mockMvc;
#MockBean
private TravelService travelService;
#Autowired
private TravelOutputDtoMapper travelOutputDtoMapper;
#TestConfiguration
static class TestConfig {
#Bean
public TravelOutputDtoMapper travelOutputDtoMapper() {
return new TravelOutputDtoMapper(); // I assume your mapper has no collaborators
}
}
// ... your MockMvc tests
}

Spring boot test doesn't Autowire all dependencies

I have a bit of a confusing error in my test scenario.
We want to refactor an Application that is not tested at all. To ensure that we have the same outcame after refactoring I'll write some integration tests for one Controller class.
#RestController
#RequestMapping("/rfq")
public class RfqController {
#Autowired
private RfqRepository rfqRepo;
#Autowired
private RfqDao rfqDao;
...
#PostMapping("/get")
public #ResponseBody BuyerRfqView getRFQ(#RequestBody SingleIdBody body) {
int id = body.getId();
Optional<Rfq> rfq = rfqRepo.getById(id);
...
}
}
In that case I want to test with testcontainers and spring-boot-test everything worked well, containers are up and running and the application starts so far. But the problem is that at runtime the spring-boot-test doesn't Autowire rfqRepo in the class under test. In the Testclass, every single dependency is in the ComponentScan or EntityScan and the repositories are also injected. I have no clue why this is not working. when the test is running I get a Nullpointer Exception by rfqRepo ...
here is the Test class:
#SpringBootTest(classes = RfqController.class, webEnvironment =
SpringBootTest.WebEnvironment.RANDOM_PORT)
#ComponentScan({...})
#EnableJpaRepositories({...})
#EntityScan({...})
#EnableAutoConfiguration
#ActiveProfiles("local")
#Testcontainers
#AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.NONE)
class RfqControllerTest {
#Container
private static OracleContainer database = new OracleContainer(
"oracleinanutshell/oracle-xe-11g:latest")
.withExposedPorts(1521, 5500)
.withPassword("...");
#InjectMocks
RfqController rfqController;
#DynamicPropertySource
static void databaseProperties(DynamicPropertyRegistry registry) {
registry.add("spring.datasource.url", database::getJdbcUrl);
registry.add("spring.datasource.username", database::getUsername);
registry.add("spring.datasource.password", database::getPassword);
}
#BeforeEach
void setUp() {
MockitoAnnotations.openMocks(this);
ScriptUtils.runInitScript(new JdbcDatabaseDelegate(database, ""), "ddl.sql");
}
#Test
void testGetRFQ() {
BuyerRfqView result = rfqController.getRFQ(new SingleIdBody(176501));
Assertions.assertEquals(new BuyerRfqView(), result);
}
}
In the SpringBootTest annotation you are only using RfqController. That's the only class then that is available during test.
#SpringBootTest(classes = RfqController.class, webEnvironment =SpringBootTest.WebEnvironment.RANDOM_PORT)
So you have to add all classes that are needed for your tests.

Asyn retry Junit is not running for 3 times as default

I am working on junit test for failure scenario for retryable. It used to work fine until we have upgraded to new version of mockito and spring boot version. We have updated to spring boot 2.4.x and started seeing this issue.
Service.java
public class RetryTest
{
#Autowired
private RetryTemplate retryTemplate;
#Async
private void retryTest(String in) {
retryTemplate.execute( invoke -> {
callMethod(in);
return null;
}, recoveryCallBack);
}
public void callMethod(String in) {
//some service call on failruew need to retry this.
someService.test(in);
}
}
Unit test:
#RunWith(SpringJUint4Runner.class)
Public class Test {
#InjectMocks
private RetryTesr retryTest;
#Mock
private RetryTemplate retryTemplate;
#Test(expected = ServiceException.class)
public void testFailure() {
when(someService.test(anyString())).ThenRetrun(new RuntimeException.class). ThenRetrun(new RuntimeException.class). ThenRetrun(new RuntimeException.class);
retryTest.retryTest(in);
}
}
When i run above even though the retryTemplate has default of 3 times, its only executing once. Expected should be it should execute 3 times and then it should throw a service exception as we are throwing in recoveryCallBack.
Can anyone please suggest.
To test any spring functionality you have to write integration test cases using #SpringBootTest annotation
#RunWith(SpringJUint4Runner.class)
#SpringBootTest
Public class Test {
#Autowired
private RetryTesr retryTest;
#MockBean
private RetryTemplate retryTemplate;
#Test(expected = ServiceException.class)
public void testFailure() {
when(someService.test(anyString())).ThenRetrun(new RuntimeException.class). ThenRetrun(new RuntimeException.class). ThenRetrun(new RuntimeException.class);
retryTest.retryTest(in);
}
}
Its wierd. After upgrading the version of spring boot ..Mockito.any() is not working, so i have to use the exact values to match the mock.

Spring #Scheduled in multiple classes, but scheduler returns to a single class

I Have a multiple classes (1,2,3,4) with #Scheduled where i have different times specified for each task/Process to run. Here I have created config class with all the schedulers beans in the config but, every time it returns the call to a single class 1.
Is there anything i need to be added in the config class? Than the below one
#Configuration
#EnableBatchProcessing
#ComponentScan(basePackages ="com.something.too.foo")
public class MyConfig {
#Autowired
public JobBuilderFactory jobBuilderFactory;
#Autowired
public StepBuilderFactory stepBuilderFactory;
#Bean
public MyTask1 myTask1(){
return new MyTask1();
}
#Bean
public MyTask2 myTask2(){
return new MyTask2();
}
#Bean
public MyTask3 myTask()3{
return new MyTask3();
}
#Bean
public MyTask4 myTask()4{
return new MyTask4();
}
Not sure what to be added to process all the Tasks in parallel processing. Help is very much Appreciated.

Spring Batch Tomcat memory leak

I use
Tomcat 8.0.26
Spring Boot 1.2.6.RELEASE
Spring 4.2.1.RELEASE
Spring Batch 3.0.5.RELEASE
In my application I have a following Spring Batch config:
#Configuration
#EnableBatchProcessing
public class ReportJobConfig {
public static final String REPORT_JOB_NAME = "reportJob";
#Autowired
private JobBuilderFactory jobBuilderFactory;
#Autowired
private StepBuilderFactory stepBuilderFactory;
#Autowired
private ReportService reportService;
#Bean(name = REPORT_JOB_NAME)
public Job reportJob() {
//#formatter:off
return jobBuilderFactory
.get(REPORT_JOB_NAME)
.flow(createRequestStep())
.on("*").to(retriveInfoStep())
.on("*").to(notifyAdminStep())
.end().build();
//#formatter:on
}
#Bean
public Step createRequestStep() {
return stepBuilderFactory.get("createRequest").tasklet(new CreateRequestTasklet(reportService)).build();
}
#Bean
public Step retrivePHIStep() {
return stepBuilderFactory.get("retriveInfo").tasklet(new RetriveInfoTasklet(reportService)).build();
}
#Bean
public Step notifyAdminStep() {
return stepBuilderFactory.get("notifyAdmin").tasklet(new NotifyAdminTasklet()).build();
}
}
This is how I run the job:
#Service
public class ReportJobServiceImpl implements ReportJobService {
final static Logger logger = LoggerFactory.getLogger(ReportJobServiceImpl.class);
#Autowired
#Qualifier(ReportJobConfig.REPORT_JOB_NAME)
private Job reportJob;
#Autowired
private JobLauncher jobLauncher;
#Override
public void runReportJob(String messageContent) throws JobExecutionAlreadyRunningException, JobRestartException,
JobInstanceAlreadyCompleteException, JobParametersInvalidException {
Map<String, JobParameter> parameters = new HashMap<>();
JobParameter reportIdParameter = new JobParameter(messageContent);
parameters.put(REPORT_ID, reportIdParameter);
jobLauncher.run(reportJob, new JobParameters(parameters));
}
}
Batch properties:
batch.jdbc.driver=com.mysql.jdbc.Driver
batch.jdbc.url=jdbc:mysql://localhost/database
batch.jdbc.user=user
batch.jdbc.password=password
batch.jdbc.testWhileIdle=true
batch.jdbc.validationQuery=SELECT 1
batch.drop.script=classpath:/org/springframework/batch/core/schema-drop-mysql.sql
batch.schema.script=classpath:/org/springframework/batch/core/schema-mysql.sql
batch.business.schema.script=classpath:/business-schema-mysql.sql
batch.database.incrementer.class=org.springframework.jdbc.support.incrementer.MySQLMaxValueIncrementer
batch.database.incrementer.parent=columnIncrementerParent
batch.lob.handler.class=org.springframework.jdbc.support.lob.DefaultLobHandler
batch.grid.size=50
batch.jdbc.pool.size=6
batch.verify.cursor.position=true
batch.isolationlevel=ISOLATION_SERIALIZABLE
batch.table.prefix=BATCH_
I deploy this application to Tomcat 8, perform some jobs and then undeploy application via Tomcat Web Application Manager.
With Java VisualVM tool I compared memory snapshots before and after and see that there are a lot of Spring Batch(org.springframework.batch.*) related objects still exist in memory:
Also, I run 1000 reportJob and got a huge memory consumption on my machine.. I have no idea what can be wrong right now..
What could be causing this issue ?
UPDATED
I have consumed ~1000 messages from AWS SQS queue. My JMS listener configured to consume 1 message at a time. During the execution I had a following heap histogram:
I really don't understand why do I need for example to have in memory 7932 instances of StepExecution.. or 5285 of JobExecution objects. Where is my mistake ?

Resources