I use spring batch for processing a file. The configuration of all components is made programatically.
I have a job that contains several TaskletSteps:
#Bean
#Named(SEEC_JOB)
public Job seecJob() {
return jobBuilderFactory.get(SEEC_JOB).start(seecMoveToWorkingStep()).next(seecLoadFileStep())
.on(ExitStatus.COMPLETED.getExitCode()).to(seecFlowMoveToArchiveOk()).from(seecLoadFileStep())
.on(ExitStatus.FAILED.getExitCode()).to(seecFlowMoveToArchiveKo()).end().build();
}
My question focus on seecLoadFileStep(), the detail bellow:
#Bean
public TaskletStep seecLoadFileStep() {
TaskletStep build = stepBuilderFactory.get(SEEC_LOAD_FILE_STEP)
.<SeecMove, SeecMove>chunk(cormoranProperties.seec.batchSize.get()).reader(seecItemReader())
.writer(seecItemWriter()).build();
return build;
}
I would like to throw a specific exception if a reading error hapens (by reading error I mean: the file is corrupted for example or it is wrong, absent xml tag...).
I have been reading spring batch doc and I think ItemReadListener is my guy:
public interface ItemReadListener<T> extends StepListener {
void beforeRead();
void afterRead(T item);
void onReadError(Exception ex);
}
but, I don't know how to use it! I have tried doing my seecItemReader() implements this interface but onReadError method is never called.
I don't know how to declare/register in the taskletStep the ItemReadListener.
Here a bit of spring doc:
Any class that implements one of the extensions of StepListener (but
not that interface itself since it is empty) can be applied to a step
via the listeners element. The listeners element is valid inside a
step, tasklet or chunk declaration. It is recommended that you declare
the listeners at the level which its function applies, or if it is
multi-featured (e.g. StepExecutionListener and ItemReadListener) then
declare it at the most granular level that it applies (chunk in the
example given).
An ItemReader, ItemWriter or ItemProcessor that itself implements one
of the StepListener interfaces will be registered automatically with
the Step if using the namespace element, or one of the the
*StepFactoryBean factories. This only applies to components directly injected into the Step: if the listener is nested inside another
component, it needs to be explicitly registered (as described above).
Could you please help me?
Thanks in advance!
As I guessed it was easier than I thougth, for registering programatically the ItemReadListener is via listener method in the tasklet configuration:
#Bean
public TaskletStep seecLoadFileStep() {
TaskletStep build = stepBuilderFactory.get(SEEC_LOAD_FILE_STEP)
.<SeecMove, SeecMove>chunk(cormoranProperties.seec.batchSize.get()).reader(seecItemReader()).listener(seecItemReaderListener())
.writer(seecItemWriter()).build();
return build;
}
And now the onError method is called when an Exception happens.
Related
I am running with Spring 3.2.18 (I know, it's old) with Spring MVC. My issue is that there is at least one default request parameter conversion (String -> List) that fails when there is actually only one item in the array, but it has commas. This is because the default conversion built into Spring will see it as a comma-separated list.
I am NOT using Spring Boot, so please avoid answers that specifically reference solutions using it.
I tried adding a #PostConstruct method to a #Confuguration class as follows:
#Configuration
public class MyConfig {
#Autowired
private ConfigurableEnvironment env;
#PostConstruct
public void removeConverters() {
ConfigurableConversionService conversionService = env.getConversionService();
conversionService.removeConvertible(String.class, Collection.class);
}
}
This runs on startup but the broken conversion still occurs. I put a breakpoint in this method, and it is called only once on startup. I verified that it removed a converter that matched the signature of String -> Collection.
The following works, but when I put a breakpoint in the #InitBinder method it acts like it gets called once for every request parameter on every request.
#ControllerAdvice
public class MyController {
#InitBinder
public void initBinder(WebDataBinder binder) {
GenericConversionService conversionService = (GenericConversionService) binder.getConversionService();
conversionService.removeConvertible(String.class, Collection.class);
}
}
As I said the second one works, but it makes no sense that the offending converter has to be removed for every request made to that method - let alone for every request parameter that method takes.
Can someone please tell me why this only works when the removal is incredibly redundant? Better yet, please tell me how I'm doing the one-time, application-scope removal incorrectly.
I'm able to make Spring+Rabbit work with the non-functional way (prior to 2.0?), but I'm trying to use with the functional pattern as the previous one is deprecated.
I've been following this doc: https://docs.spring.io/spring-cloud-stream/docs/3.1.0/reference/html/spring-cloud-stream.html#_binding_and_binding_names
The queue (consumer) is not being created in Rabbit with the new method. I can see the connection being created but without any consumer.
I have the following in my application.properties:
spring.cloud.stream.function.bindings.approved-in-0=approved
spring.cloud.stream.bindings.approved.destination=myTopic.exchange
spring.cloud.stream.bindings.approved.group=myGroup.approved
spring.cloud.stream.bindings.approved.consumer.back-off-initial-interval=2000
spring.cloud.stream.rabbit.bindings.approved.consumer.queueNameGroupOnly=true
spring.cloud.stream.rabbit.bindings.approved.consumer.bindingRoutingKey=myRoutingKey
which is replacing:
spring.cloud.stream.bindings.approved.destination=myTopic.exchange
spring.cloud.stream.bindings.approved.group=myGroup.approved
spring.cloud.stream.bindings.approved.consumer.back-off-initial-interval=2000
spring.cloud.stream.rabbit.bindings.approved.consumer.queueNameGroupOnly=true
spring.cloud.stream.rabbit.bindings.approved.consumer.bindingRoutingKey=myRoutingKey
And the new class
#Slf4j
#Service
public class ApprovedReceiver {
#Bean
public Consumer<String> approved() {
// I also saw that it's recommended to not use Consumer, but use Function instead
// https://docs.spring.io/spring-cloud-stream/docs/3.1.0/reference/html/spring-cloud-stream.html#_consumer_reactive
return value -> log.info("value: {}", value);
}
}
which is replacing
// BindableApprovedChannel.class
#Configuration
public interface BindableApprovedChannel {
#Input("approved")
SubscribableChannel getApproved();
}
// ApprovedReceiver.class
#Service
#EnableBinding(BindableApprovedChannel.class)
public class ApprovedReceiver {
#StreamListener("approved")
public void handleMessage(String payload) {
log.info("value: {}", payload);
}
}
Thanks!
If you have multiple beans of type Function, Supplier or Consumer (which could be declared by third party libraries), the framework does not know which one to bind to.
Try setting the spring.cloud.function.definition property to approved.
https://docs.spring.io/spring-cloud-stream/docs/3.1.3/reference/html/spring-cloud-stream.html#spring_cloud_function
In the event you only have single bean of type java.util.function.[Supplier/Function/Consumer], you can skip the spring.cloud.function.definition property, since such functional bean will be auto-discovered. However, it is considered best practice to use such property to avoid any confusion. Some time this auto-discovery can get in the way, since single bean of type java.util.function.[Supplier/Function/Consumer] could be there for purposes other then handling messages, yet being single it is auto-discovered and auto-bound. For these rare scenarios you can disable auto-discovery by providing spring.cloud.stream.function.autodetect property with value set to false.
Gary's answer is correct. If adding the definition property alone doesn't resolve the issue I would recommend sharing what you're doing for your supplier.
This is also a very helpful general discussion for transitioning from imperative to functional with links to repos with more in depth examples: EnableBinding is deprecated in Spring Cloud Stream 3.x
I'm trying to run a few methods after the Spring Boot project starts. I'm using #EventListener(ApplicationReadyEvent.class) annotation above the methods I want ran after the project launches. But it's only starting for one method at a time. I want multiple methods started at once. Is that the expected behavior for #EventListener(ApplicationReadyEvent.class)?
Its OK to place several (more than one) methods annotated with #EventListener all of them will be executed:
#Configuration
public class SampleConfiguration {
#Bean
public SampleBean sampleBean() {return new SampleBean();}
#EventListener
public void onApplicationReadyEvent(ApplicationReadyEvent event) {
System.out.println("Hello");
}
#EventListener
public void onApplicationReadyEvent2(ApplicationReadyEvent event) {
System.out.println("How are you");
}
}
This will print both "Hello" and "How are you" upon the succesfull start of the Application Context.
Now, its true that spring doesn't invoke them concurrently, it resolves all the listeners and calls them sequentially.
If you need a parallel execution you can create one listener that will be an "entry point" for the logical tasks that must be run in parallel and use Threads / Thread Pool Executors to run the code of your choice in parallel
Did you try adding #Async also above the method?
This listener is invoked synchronously. You can make it asynchronous by simply adding #Async annotation.
You can have the event listeners execute asynchronously by adding the following bean to your #Configuration class.
#Bean(name = "applicationEventMulticaster")
public ApplicationEventMulticaster simpleApplicationEventMulticaster() {
SimpleApplicationEventMulticaster eventMulticaster =
new SimpleApplicationEventMulticaster();
eventMulticaster.setTaskExecutor(new SimpleAsyncTaskExecutor());
return eventMulticaster;
}
If you've defined a custom TaskExecutor then you should replace new SimpleAsyncTaskExecutor() with yourCustomTaskExecutorBeanMethod()
I had run into a similar issue where ApplicationReadyEvent was annoted for 3 function but in debugging we found it to fire for only 1 always. We added #Order and kept that function with the highest index(i.e. lowest priority), as in our case it was running for an indefinite while loop, and found that all the 3 functions were invoked seamlessly.
I've successfully implemented a custom post processor filter with the help of the wro4j documentation.
Its job is to generate and prepend SASS vars to a group of SASS files which are then handed off to the rubySassCss filter for transpiling, and it's doing this job well.
The problem is that I wanted to hand the job of determining the SASS vars off to a custom ThemeManager #Service managed by Spring. I hadn't considered that the filter wouldn't be able to see the autowired #Service but that seems to be the case.
When I #Autowire the #Service into a controller, it works fine, but when I try the same thing with the filter I get a NPE when attempting to use it.
Is there a way to make the #Service visible to the filters or am I approaching this the wrong way?
Thanks for any help.
UPDATE:
It's taken some doing and attacking from a lot of angles, but I seem to be having success with autowiring my themeManagerService into the app configuration where I have my WRO filterRegistrationBean bean. I then pass the themeManagerService bean as a second argument to my custom ConfigurableWroManagerFactory.
Living in the custom WroManagerFactory is a reference to a custom UriLocator, which takes that themeManagerService as an argument. The custom UriLocator is invoked by a CSS resource containing an arbitrary keyword within a group.
The new UriLocator is able to generate a ByteArrayInputStream from what the themeManagerService provides it and pass it into the pipeline.
Simple.
I'll follow up when this approach pans/fizzles out.
In the end, I was able to provide the spring managed ThemeManagerService directly to the custom post processor, rather than relying on a custom UriLocator. I had tried that early on, but forgot to call super() in the new constructor, so the processor registration system was breaking.
I pass the #Autowired ThemeManagerService to my CustomConfigurableWroManagerFactory when registering the WRO bean:
#Autowired
ThemeManagerService themeManagerService;
#Bean
FilterRegistrationBean webResourceOptimizer(Environment env) {
FilterRegistrationBean fr = new FilterRegistrationBean();
ConfigurableWroFilter filter = new ConfigurableWroFilter();
Properties props = buildWroProperties(env);
filter.setProperties(props);
//The overridden constructor passes ThemeManager along
filter.setWroManagerFactory(new CustomConfigurableWroManagerFactory(props,themeManagerService));
filter.setProperties(props);
fr.setFilter(filter);
fr.addUrlPatterns("/wro/*");
return fr;
}
The constructor injection of ThemeManagerService into CustomConfigurableWroManagerFactory means it can be passed along to the custom postprocessor as it's registered by contributePostProcessors:
public class CustomConfigurableWroManagerFactory extends Wro4jCustomXmlModelManagerFactory {
private ThemeManagerService themeManagerService;
public CustomConfigurableWroManagerFactory(Properties props,ThemeManagerService themeManagerService) {
//forgetting to call super derailed me early on
super(props);
this.themeManagerService = themeManagerService;
}
#Override
protected void contributePostProcessors(Map<String, ResourcePostProcessor> map) {
//ThemeManagerService is provided as the custom processor is registered
map.put("repoPostProcessor", new RepoPostProcessor(themeManagerService));
}
}
Now, the post processor has access to ThemeManagerService:
#SupportedResourceType(ResourceType.CSS)
public class RepoPostProcessor implements ResourcePostProcessor {
private ThemeManagerService themeManagerService;
public RepoPostProcessor(ThemeManagerService themeManagerService) {
super();
this.themeManagerService = themeManagerService;
}
public void process(final Reader reader, final Writer writer) throws IOException {
String resourceText = "/* The custom PostProcessor fetched the following SASS vars from the ThemeManagerService: */\n\n";
resourceText += themeManagerService.getFormattedProperties();
writer.append(resourceText);
//read in the merged SCSS and add it after the custom content
writer.append(IOUtils.toString(reader));
reader.close();
writer.close();
}
}
This approach is working as expected/intended so far. Hope it comes in handy for someone else.
Wro4j is a great tool and much appreciated.
I would like to build a Spring application, where new components can be added easily and without much configuration. For example: You have different kinds of documents. These documents should be able to get exported into different fileformats.
To make this functionality easy to maintain, it should (basically) work the following way:
Someone programs the file format exporter
He/ She writes a component, which checks if the file format exporter is licensed (based on Spring Conditions). If the exporter is licensed a specialized Bean is injected in the application context.
The "whole rest" works dynamically based on the injected beans. Nothing needs to be touched in order to display it on the GUI, etc.
I pictured it the following way:
#Component
public class ExcelExporter implements Condition {
#PostConstruct
public void init() {
excelExporter();
}
#Bean
public Exporter excelExporter(){
Exporter exporter= new ExcelExporter();
return exporter;
}
#Override
public boolean matches(ConditionContext context, AnnotatedTypeMetadata metadata) {
return true;
}
}
In order to work with those exporters (display them, etc.) I need to get all of them. I tried this:
Map<String, Exporter> exporter =BeanFactoryUtils.beansOfTypeIncludingAncestors(appContext, Exporter.class, true, true);
Unfortunate this does not work (0 beans returned). I am fairly new to this, would anyone mind to tell me how this is properly done in Spring? Maybe there is a better solution for my problem than my approach?
You can get all instances of a given type of bean in a Map effortlessly, since it's a built in Spring feature.
Simply autowire your map, and all those beans will be injected, using as a key the ID of the bean.
#Autowired
Map<String,Exporter> exportersMap;
If you need something more sophisticated, such as a specific Map implementation or a custom key. Consider defining your custom ExporterMap, as follows
#Component
class ExporterMap implements Map{
#Autowired
private Set<Exporter> availableExporters;
//your stuff here, including init if required with #PostConstruct
}