I'm looking for some general opinions and advice on testing a Spring batch step and step execution.
My basic step reads in from an api, processes into an entity object and then writes to a DB. I have tested the happy path, that the step completes successfully. What I now want to do is test the exception handling when data is missing at the processor stage. I could test the processor class in isolation, but I'd rather test the step as a whole to ensure the process failure is reflected correctly at step/job level.
I've read the spring batch testing guidelines and if I'm honest, I'm slightly lost within it. Is it possible to use StepScopeTestUtils.doInStepScope or updating the StepExecution to test this scenario? Ideally I'd force the reader to return faulty data before the processor kicks in.
Any advice would be greatly appreciated.
The best approach depends on the scope of your test. Reading a little between the lines here, I assume you are using a Spring IT, setting up a Spring context and using the JobLauncherTestUtils to start a job or a step.
I think the easiest way is replace one of your beans with a mock that triggers the error scenario. Using Mockito, this can be done by adding something like this to your test-configuration.
#Bean
public ReaderDataRepository dataApi(){
return mock(ReaderDataRepository.class);
}
This bean then overrides the actual implementation. In the test setup you can then configure this mock very explicitly.
#Autowired
private ReaderDataRepository mockedRepository;
#Before
public void setUp() {
when(mockedRepository.getData()).thenReturn(faultyData())
}
This involves very little manipulation of Spring 'magic' and very explicitly defines the error from within the test.
Related
In my spring boot project, I am using MockMVC to test controller(web) layer. But I also have AOP(AspectJ) logic in my project, when I run unit test for controller with MockMVC, the test also triggers AOP code, how can I prevent AOP code to be triggered while running unit test for controller?
#Test
public void testMyControllerMethod() {
...
// myRequest hits an endpoint function of my controller, there is also AOP intercept the function call, how can I disable AOP to be triggered while running test?
mockMVC.perform(myRequest).andExpect(okStatus)
}
Question is in my code comment :)
I have checked this answer, I understand to use the if() expression, but I don't get TestMode.ACTIVE, there is no such thing in Spring boot. If someone could let me know how to check whether code is running unit test or not at runtime, I would know how to prevent AOP logic run as well.
What I meant in the other answer, as Simon already tried to explain to you, is something like this:
package de.scrum_master.app;
public class TestMode {
public static boolean ACTIVE = false;
}
But actually there I also listed a few other options such as environment variables and system properties. If I were you I would use one of those because in your Maven or Gradle build it would be very easy to set properties or environment variables via configuration. Your if() pointcut could access those variables.
Especially in the context of Spring there is an even simpler option: a test application configuration. Just provide a configuration without aspects to your tests. That way you can have different configurations for
production environment,
unit tests (no aspects),
integration tests (e.g. with aspects but different from unit test and production).
et cetera.
The advantage here is that you don't need any if() pointcuts or build any other knowledge about test/production environments into your aspects, which is quite ugly. My other answer only shows what you can do, it does not say it is the best solution.
I have endpoints in #RestControllers that look similar to this:
#RestController
#RequestMapping("/rest/x")
public class XApiController
{
// ...
#PostMapping(...)
#PreAuthorize("#apiAuthService.canAccessX(#headers)")
public void saveX(...)
{
// ...
}
}
These endpoints require the developer to make the HttpHeaders object available and name it correctly in the method declaration:
public void saveX(#RequestHeader HttpHeaders headers)
Our problem is that if this last step isn't done, the endpoint only fails at runtime when the endpoint is invoked. This means that issues from large refactors later (say, to change the HttpHeaders argument to HttpServletRequest) aren't easy to identify. Is there any way to tell Spring to validate these expressions are valid on app startup?
I suggest you to create integration tests and then invoke saveX from the test to verify this before you deploy an application.
I would also state my opinion that if you want to have testable code with good quality - try to get rid of SpringEL as soon as possible. In my experience this approach proved as poorly testable, hardly maintainable and also introducing unnecessary complications to your source code.
In modern spring framework there are lots of ways to avoid writing SpringEl.
Spring always validates all beans on start up. But your problem is not within validation your problem is test problem. The process of pre authorization is a runtime job. Spring can not know what to do with this expression spring just checks its syntax over SPEL rules.
You can create tests to check header.
You can increase your IDE inspection level of spring spel to error.
You can simply write a static method to get the headers without a rest parameter.
I have a simple Spring Boot application which reads from Kafka and writes to Kafka. I wrote a SpringBootTest using an EmbeddedKafka to test all that.
The main problem is: Sometimes the test fails because the test sends the Kafka message too early. That way, the message is already written to Kafka before the Spring application (or its KafkaListener to be precise) is ready. Since the listener reads from the latest offset (I do not want to change any config for my test - except bootstrap.servers), it will not receive all messages in that test.
Does anyone have an idea how I could know inside the test, that the KafkaListener is ready to receive messages?
Only way I could think of is waiting until /health comes available but I have no idea whether I can be sure that this implies the KafkaListener to be ready at all.
Any help is greatly appreciated!
Best regards.
If you have a KafkaMessageListenerContainer instance, then it is very easy to use org.springframework.kafka.test.utils.ContainerTestUtils.waitForAssignment(Object container, int partitions).
https://docs.spring.io/spring-kafka/api/org/springframework/kafka/test/utils/ContainerTestUtils.html
e.g. calling ContainerTestUtils.waitForAssignment(container, 1); in your Test setup will block until the container has gotten 1 partition assigned.
So, I just read about #PostConstruct and it turns out that you can easily use this also within the test:
#PostConstruct
public void checkApplicationReady() {
applicationReady = true;
}
Now I added an #Before method to wait until that flag is set to true.
So far this seems to work very nicely!
I am working on spring boot and batch application.
Due to batch, the application tries to connect to datasource
with spring-boot:run.
I want to stop that and tried spring.datasource.initialize=false
Along with this also put spring.batch.job.enabled=false
While the second one works fine, it seems the first one is ignored.
Could someone let me know if there is a way to stop db connection on startup?
Thanks in advance
The problem is, that spring/spring-boot loads the whole spring-context when it is "booted". This means, that all defined spring-beans are loaded into the spring-context during this boot-phase. In the case of spring-batch, this also means that the datasource bean is loaded and, if not turned off by "spring.batch.initializer.enabled=false", the spring-batch tables are initialized.
Generally, you cannot prevent this from happening as soon as you have added your spring-batch-starter to your maven dependencies.
Moreover, I don't understand why you want to prevent this from happening. It is just initialisation taking place and, provided that everything is configured correctly, this shouldn't be a problem at all.
Nonetheless, if you really want to stop the datasource from beeing initialized, you could try the following approach. However, I don't know if this will work.
Spring-Batch needs a datasource that is registered under the name "dataSource" in the spring-context. If no spring-bean with that name is found, it creates its own. But if you provide your own implementation/configuration for it, it will use your spring bean.
What you need to do is, to provide a proxy for a datasource that is loaded lazily and then register it under the name "dataSource" in the context:
#Configuration
public class MySetUp {
#Bean
public DataSource dataSource() {
// ... create your "lazy initializing" datasource
}
}
But - and let me stress that - this nothing that I would recommend and I don't see a good reason, why this should be necessary to do.
Furthermore, you mention that you only want load "initial static index page" (I assume, you are talking about html, right?). However, I don't see a "batch" use case, which should display html-pages. It would probably be better to have two different applications in this case.
Probably you could provide some more information about your use case.
As I understand, you don't want to prevent database connection during application startup.
Instead, you want to prevent execution of batch scripts.
Correct me, please, if I got it wrong.
To prevent execution of batch scripts set:
spring.batch.initializer.enabled=false
In my application I have a set of of DAOs which I inject into my application layer. For an acceptance test I'm writing, I want to preload the dev_server datastore with data, so I use the same Spring config in my JUnit test (using the #ContextConfiguration annotation) to inject an instance of the relevant DAO into my test. When I actually go to store some data eg:
dao.add(entity)
I get the dreaded "No API environment is registered for this thread."
Caused by: java.lang.NullPointerException: No API environment is registered for this thread.
at com.google.appengine.api.datastore.DatastoreApiHelper.getCurrentAppId(DatastoreApiHelper.java:108)
at com.google.appengine.api.datastore.DatastoreApiHelper.getCurrentAppIdNamespace(DatastoreApiHelper.java:118)
....
This is probably because my test case hasn't read in the GAE application-web.xml with the app details (although I'm guessing here I could really be wrong); so it doesn't know to write to the same datastore that the app running on the dev_server is reading/writing to.
How can I get my test to "point" to the same datastore as the app? Is there some "datasource" mechanism that I can inject both into the app and the test? Is there a way to get my test to force the datastore api to read the needed config?
Here is a page that talks about how to do unit tests that connect to a dev datastore. Is this the kind of thing you're looking for? Basically it talks about two classes, LocalServiceTestHelper and LocalDatastoreServiceTestConfig that you can use to set up an environment for testing. While the example given is for unit tests, I believe it will also work for your situation.
You can then configure things like whether the dev datastore is written to disk or just kept in memory (for faster tests). If you want this data to go to the same place as your dev server, you will probably want to adjust this, as I think the default is the "in memory" option. If you look at the javadoc there is a "setBackingStoreLocation" method where you can point to whatever file you want.
I've found the solution!!!!
For some reason the Namespace, AppID and the AuthDomain fields of the test datastore have to match that of the dev_server, then the dev_server can see the entities inserted by the test.
You can see the values for the environment (dev_server or test code) with the following statements
System.out.println(NamespaceManager.get());
System.out.println(ApiProxy.getCurrentEnvironment().getAppId());
System.out.println(ApiProxy.getCurrentEnvironment().getAuthDomain());
In your instance of LocalServiceTestHelper (eg: gaeHelper), you can set the values for the test environment
// the NamespaceManager is thread local.
NamespaceManager.set(NamespaceManager.getGoogleAppsNamespace());
gaeHelper.setEnvAppId(<the name of your app in appengine-web.xml>);
gaeHelper.setEnvAuthDomain("gmail.com");
Then the dev_server will see your entities. However because of synchronisation issues, if the test writes to the datastore after the dev_server has been started the dev_server wont see it unless it can be forced to reread the file (which I haven't figured out yet). Else the server has to be restarted.
I've found a workaround, although it's not very nice because each test method doesn't clean up the Datastore, as explained in the article Local Unit Testing for Java, however, the Datastore starts clean each time the Test class is run, so it's not so bad, provided that you're careful about that.
The problem is, that when using SpringJUnit4ClassRunner, the spring environment is created before the #Before annotation can be run, the solution is use #BeforeClass and use a static variable for LocalServiceTestHelper, to have them created before the Spring Environment is set up.
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration("classpath:META-INF/spring/context-test.xml")
#Transactional
public class MyTest {
#Inject
private MyService myService;
private static final LocalServiceTestHelper helper =
new LocalServiceTestHelper(new LocalDatastoreServiceTestConfig());
#BeforeClass
public static void beforeClass() {
helper.setUp();
}
#AfterClass
public static void afterClass() {
helper.tearDown();
}
If anyone has a better solution, I'll be glad to hear!