Handling Spring Batch flow job - spring

Situation :
There is a batch steps named A, B, C, D, E.
A is always executed first.
Depending on A's exitStatus, if A gives "SOMECONDITION", B is executed. On "OTHERCONDITION" C is executed. Job instances are not allowed to execute both B and C.
Regardless of executed Step is B or C, they should go to D.
After executing D, by using next(), E is executed.
Problem : Ugly code structure.
By redundant use of on(), to(), I managed to figure out the flow that I need.
As directly observable, there is really ugly code next() on each on-to, to direct them into common flow.
Adding more ugliness, I have to use from()-on("*") to direct each of on-to-next to some common flow.
Question :
At least this works, but i wonder, is this the only code that satisfies above condition?

Problem : Ugly code structure.
Adding more ugliness, I have to [...]
We, the Spring team, are always open to constructive feedback. So if you have ideas on how to improve the DSL of defining job flows, you are welcome to open a ticket on our issue tracker and we will do our best to improve things if needed.
Now to answer your question, the next(...) statements after each on("..").to("..") are not necessary. This is not what the documentation says, see example here: Conditional Flow.
According to your description, the flow to implement is the following:
This is actually a conditional flow (A -> decider -> B or C) followed by a sequential flow (D -> E).
The way I would define such flow is something like:
#Bean
public Job job() {
Flow conditionalFlow = new FlowBuilder<SimpleFlow>("conditionalFlow")
.start(stepA())
.next(decider()).on("SOMECONDITION").to(stepB())
.from(decider()).on("OTHERCONDITION").to(stepC())
.end();
return new JobBuilder("myJob", jobRepository)
.start(conditionalFlow)
.next(stepD())
.next(stepE())
.end()
.build();
}
This way, the flows are clearly defined. Here is a complete sample that you can run and check the results (make sure to use Spring Batch 5 and to have HSQLDB in the classpath):
package org.springframework.batch.sample;
import javax.sql.DataSource;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.job.builder.FlowBuilder;
import org.springframework.batch.core.job.builder.JobBuilder;
import org.springframework.batch.core.job.flow.Flow;
import org.springframework.batch.core.job.flow.FlowExecutionStatus;
import org.springframework.batch.core.job.flow.JobExecutionDecider;
import org.springframework.batch.core.job.flow.support.SimpleFlow;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.batch.core.repository.JobRepository;
import org.springframework.batch.core.step.builder.StepBuilder;
import org.springframework.batch.repeat.RepeatStatus;
import org.springframework.context.ApplicationContext;
import org.springframework.context.annotation.AnnotationConfigApplicationContext;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseBuilder;
import org.springframework.jdbc.datasource.embedded.EmbeddedDatabaseType;
import org.springframework.jdbc.support.JdbcTransactionManager;
import org.springframework.transaction.PlatformTransactionManager;
#Configuration
#EnableBatchProcessing
public class MyJobConfiguration {
private final JobRepository jobRepository;
private final PlatformTransactionManager transactionManager;
public MyJobConfiguration(JobRepository jobRepository, PlatformTransactionManager transactionManager) {
this.jobRepository = jobRepository;
this.transactionManager = transactionManager;
}
private Step createStep(String stepName) {
return new StepBuilder("step" + stepName, jobRepository)
.tasklet((contribution, chunkContext) -> {
System.out.println("running step " + stepName);
return RepeatStatus.FINISHED;
}, transactionManager)
.build();
}
#Bean
public Step stepA() {
return createStep("A");
}
#Bean
public Step stepB() {
return createStep("B");
}
#Bean
public Step stepC() {
return createStep("C");
}
#Bean
public Step stepD() {
return createStep("D");
}
#Bean
public Step stepE() {
return createStep("E");
}
#Bean
public JobExecutionDecider decider() {
return (jobExecution, stepExecution) -> {
return new FlowExecutionStatus("SOMECONDITION"); // SOMECONDITION, OTHERCONDITION
};
}
#Bean
public Job job() {
Flow conditionalFlow = new FlowBuilder<SimpleFlow>("conditionalFlow")
.start(stepA())
.next(decider()).on("SOMECONDITION").to(stepB())
.from(decider()).on("OTHERCONDITION").to(stepC())
.end();
return new JobBuilder("myJob", jobRepository)
.start(conditionalFlow)
.next(stepD())
.next(stepE())
.end()
.build();
}
public static void main(String[] args) throws Exception {
ApplicationContext context = new AnnotationConfigApplicationContext(MyJobConfiguration.class);
JobLauncher jobLauncher = context.getBean(JobLauncher.class);
Job job = context.getBean(Job.class);
jobLauncher.run(job, new JobParameters());
}
// infrastructure beans
static class DataSourceConfiguration {
#Bean
public DataSource dataSource() {
return new EmbeddedDatabaseBuilder()
.setType(EmbeddedDatabaseType.HSQL)
.addScript("/org/springframework/batch/core/schema-hsqldb.sql")
.build();
}
#Bean
public JdbcTransactionManager transactionManager(DataSource dataSource) {
return new JdbcTransactionManager(dataSource);
}
}
}
This sample prints:
running step A
running step B
running step D
running step E
If you make the decider return "OTHERCONDITION", the sample prints:
running step A
running step C
running step D
running step E
which is what is required from your description.

Related

Spring integration email pop3 inbound adapter not working/starting

With java spring integration written the below code to read the email from gmail.
As per logs seems the connection with gmail is formed, but on new email its not reading or not going into handle() method. Please help.
o.s.i.m.AbstractMailReceiver - attempting to receive mail from folder [INBOX]
import lombok.extern.log4j.Log4j2;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.integration.channel.QueueChannel;
import org.springframework.integration.config.EnableIntegration;
import org.springframework.integration.dsl.IntegrationFlow;
import org.springframework.integration.dsl.IntegrationFlows;
import org.springframework.integration.dsl.Pollers;
import org.springframework.integration.mail.MailReceiver;
import org.springframework.integration.mail.dsl.Mail;
import org.springframework.integration.mail.support.DefaultMailHeaderMapper;
import org.springframework.integration.mapping.HeaderMapper;
import org.springframework.messaging.Message;
import org.springframework.messaging.PollableChannel;
import javax.mail.internet.MimeMessage;
#Log4j2
#Configuration
#EnableIntegration
public class EmailReceiver {
#Autowired
private PollableChannel pop3Channel;
#Bean
public PollableChannel receivedChannel() {
return new QueueChannel();
}
#Bean
public IntegrationFlow pop3MailFlow() {
return IntegrationFlows
.from(Mail.pop3InboundAdapter("pop.gmail.com", 995, "userName", "password")
.javaMailProperties(p -> {
p.put("mail.debug", "true");
p.put("mail.pop3.socketFactory.fallback", "false");
p.put("mail.pop3.port", 995);
p.put("mail.pop3.socketFactory.class", "javax.net.ssl.SSLSocketFactory");
p.put("mail.pop3.socketFactory.port", 995);
})
.headerMapper(mailHeaderMapper()),
e -> e.poller(Pollers.fixedRate(5000).maxMessagesPerPoll(1)))
.handle((payload, header) -> logMail(payload))
.get();
}
public Message logMail(Object payload) {
Message message = (Message)payload;
log.info("*******Email[TEST]********* ", payload);
return message;
}
#Bean
public HeaderMapper<MimeMessage> mailHeaderMapper() {
return new DefaultMailHeaderMapper();
}
}
The problem is here:
.maxFetchSize(1)
By default the AbstractMailReceiver tries to fetch all the messages from the mail box. And looks like it takes a lot of time.
Another problem that with the .headerMapper(mailHeaderMapper() a Mail.pop3InboundAdapter does not produce a Message but rather byte[]. So, your .handle((payload, header) -> logMail(payload)) not only bad by the request-reply definition in the end of flow, but also fails with ClassCast like this, because you expect the type which is not produced for you:
Caused by: java.lang.ClassCastException: class [B cannot be cast to class org.springframework.messaging.Message ([B is in module java.base of loader 'bootstrap'; org.springframework.messaging.Message is in unnamed module of loader 'app')
at com.firm.demo.EmailReceiver.logMail(EmailReceiver.java:59)
at com.firm.demo.EmailReceiver.lambda$pop3MailFlow$2(EmailReceiver.java:53)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:568)
Well, no: better to say you are missing the fact that this signature .handle((payload, header) -> logMail(payload)) deals with the payload not the whole message. So, your expectations in the logMail() are wrong. It has to be byte[]. And consider to have a one-way handler over there in the end of flow anyway.
UPDATE
The working code is like this:
#Log4j2
#Configuration
#EnableIntegration
public class EmailReceiver {
#Autowired
private PollableChannel pop3Channel;
private MailReceiver receiver;
#Bean
public PollableChannel receivedChannel() {
return new QueueChannel();
}
#Bean
public IntegrationFlow pop3MailFlow() {
return IntegrationFlows
.from(Mail.pop3InboundAdapter("pop.gmail.com", 995, "userName", "password")
.javaMailProperties(p -> {
p.put("mail.debug", "false");
p.put("mail.pop3.socketFactory.fallback", "false");
p.put("mail.pop3.port", 995);
p.put("mail.pop3.socketFactory.class", "javax.net.ssl.SSLSocketFactory");
p.put("mail.pop3.socketFactory.port", 995);
})
.maxFetchSize(1)
.headerMapper(mailHeaderMapper()),
e -> e.poller(Pollers.fixedRate(5000).maxMessagesPerPoll(1)))
.handle(this, "logMail")
.get();
}
public void logMail(String payload) {
log.info("*******Email[TEST]********* \n" + payload);
}
#Bean
public HeaderMapper<MimeMessage> mailHeaderMapper() {
return new DefaultMailHeaderMapper();
}
}

Extending a exception aspect in order to have application specific implementation

in my spring boot application, I have been using one external commons library for handling exceptions. The external library has an aspect defined for the same something like below:
#Aspect
#Order(0)
public class InternalExceptionAspect {
public InternalExceptionAspect() {
}
#Pointcut("#within(org.springframework.stereotype.Service)")
public void applicationServicePointcut() {
}
#AfterThrowing(
pointcut = "applicationServicePointcut()",
throwing = "e"
)
public void translate(JoinPoint joinPoint, Throwable e) {
String resourceId = this.getResourceId();
if (e instanceof BadInputException) {
BadInputException inputException = (BadInputException)e;
throw new BadRequestAlertException(inputException.getErrorCode().getDefaultMessage(), inputException.getMessage(), inputException.getErrorCode().getHttpStatusCode(), resourceId, inputException.getErrorCode().getCode());
} else if (!(e instanceof BadServerStateException) && !(e instanceof InternalException)) {
String message;
if (e instanceof JDBCException) {
...
throw new BadServerStateException(message, resourceId, "20");
} else {
...
throw new BadServerStateException(message, resourceId, "10");
}
} else {
InternalException serverStateException = (InternalException)e;
throw new BadServerStateException(serverStateException.getErrorCode().getDefaultMessage(), serverStateException.getMessage(), resourceId, serverStateException.getErrorCode().getHttpStatusCode(), serverStateException.getErrorCode().getCode(), serverStateException.getErrorCode().getErrorType().name());
}
}
String getResourceId() {
RequestHeaders requestHeaders = RequestResponseContext.getRequestHeaders();
return requestHeaders.getResourceId();
}
}
Here I would like to introduce another else if block in order to handle DuplicateKeyException for my application.
The problem is, the above code, being part of the commons library is being used by multiple other applications. But, I would like to do the change to be applied only in my application.
I have been thinking to inherit the Aspect class something like below, inside my application:
#Aspect
#Order(0)
public class MyInternalExceptionAspect extends InternalExceptionAspect {
public MyInternalExceptionAspect() {
}
#Pointcut("#within(org.springframework.stereotype.Service)")
public void applicationServicePointcut() {
}
#AfterThrowing(
pointcut = "applicationServicePointcut()",
throwing = "e"
)
public void translate(JoinPoint joinPoint, Throwable e) {
if(e instanceof DuplicateKeyException) {
...
}
super.translate(joinpoint, e);
}
}
But, I am not sure, if this is the correct approach to do this. Could anyone please help here regarding what would be the best approach to achieve this? Thanks.
You cannot extend a concrete aspect using class MyInternalExceptionAspect extends InternalExceptionAspect. It will cause an exception in Spring AOP:
...AopConfigException:
[...MyInternalExceptionAspect] cannot extend concrete aspect
[...InternalExceptionAspect]
Only abstract aspects are meant to be extended.
But you can simply create a new aspect without inheritance and make sure that it has a lower priority than the original aspect.
Why lower priority?
Acording to the #Order javadoc, "lower values have higher priority".
You want your own aspect's #AfterReturning advice to kick in before the original aspect possibly transforms the exception of interest into something else, before you have a chance to handle it. But according to the Spring manual: "The highest precedence advice runs first "on the way in" (so, given two pieces of before advice, the one with highest precedence runs first). "On the way out" from a join point, the highest precedence advice runs last (so, given two pieces of after advice, the one with the highest precedence will run second).".
Therefore, your own aspect should have #Order(1), giving it lower priority than the original aspect, but making the #AfterThrowing advide run before the original one. Sorry for the reverse logic, even though it makes sense. You just need to be aware of it.
Here is an MCVE, simulating your situation in a simplified way:
package de.scrum_master.spring.q69862121;
import org.springframework.stereotype.Service;
#Service
public class MyService {
public void doSomething(Throwable throwable) throws Throwable {
if (throwable != null)
throw throwable;
}
}
package de.scrum_master.spring.q69862121;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.ConfigurableApplicationContext;
import org.springframework.context.annotation.Configuration;
#SpringBootApplication
#Configuration
public class DemoApplication {
private static final Logger log = LoggerFactory.getLogger(DemoApplication.class.getName());
public static void main(String[] args) throws Throwable {
try (ConfigurableApplicationContext appContext = SpringApplication.run(DemoApplication.class, args)) {
doStuff(appContext);
}
}
private static void doStuff(ConfigurableApplicationContext appContext) {
MyService myService = appContext.getBean(MyService.class);
List<Throwable> throwables = Arrays.asList(
null, // No exception -> no aspect should kick in
new Exception("oops"), // Not covered by any aspects -> no translation
new IllegalArgumentException("uh-oh"), // Original aspect translates to RuntimeException
new NullPointerException("null"), // Custom aspect translates to RuntimeException
new ArithmeticException("argh") // Custom aspect translates to IllegalArgumentException,
// then original aspect translates to RuntimeException
);
for (Throwable originalThrowable : throwables) {
try {
myService.doSomething(originalThrowable);
}
catch (Throwable translatedThrowable) {
log.info(translatedThrowable.toString());
}
}
}
}
As you can see, the application calls the service, the first time with null, not causing any exception, then with several types of exceptions the aspects are meant to either ignore or translate.
package de.scrum_master.spring.q69862121;
import org.aspectj.lang.JoinPoint;
import org.aspectj.lang.annotation.AfterThrowing;
import org.aspectj.lang.annotation.Aspect;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.core.annotation.Order;
import org.springframework.stereotype.Component;
#Aspect
#Component
#Order(0)
public class InternalExceptionAspect {
private static final Logger log = LoggerFactory.getLogger(InternalExceptionAspect.class.getName());
#AfterThrowing(pointcut = "#within(org.springframework.stereotype.Service)", throwing = "e")
public void translate(JoinPoint joinPoint, Throwable e) {
log.info(joinPoint + " -> " + e);
if (e instanceof IllegalArgumentException)
throw new RuntimeException("Transformed by InternalExceptionAspect", e);
}
}
package de.scrum_master.spring.q69862121;
import org.aspectj.lang.JoinPoint;
import org.aspectj.lang.annotation.AfterThrowing;
import org.aspectj.lang.annotation.Aspect;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.core.annotation.Order;
import org.springframework.stereotype.Component;
#Aspect
#Component
#Order(1)
public class MyInternalExceptionAspect {
private static final Logger log = LoggerFactory.getLogger(MyInternalExceptionAspect.class.getName());
#AfterThrowing(pointcut = "#within(org.springframework.stereotype.Service)", throwing = "e")
public void translate(JoinPoint joinPoint, Throwable e) {
log.info(joinPoint + " -> " + e);
if (e instanceof NullPointerException)
throw new RuntimeException("Transformed by MyInternalExceptionAspect", e);
if (e instanceof ArithmeticException)
throw new IllegalArgumentException("Transformed by MyInternalExceptionAspect", e);
}
}
The console log proves that everything works as expected, also with regard to invocation order:
d.s.s.q.MyInternalExceptionAspect : execution(void de.scrum_master.spring.q69862121.MyService.doSomething(Throwable)) -> java.lang.Exception: oops
d.s.s.q69862121.InternalExceptionAspect : execution(void de.scrum_master.spring.q69862121.MyService.doSomething(Throwable)) -> java.lang.Exception: oops
d.s.spring.q69862121.DemoApplication : java.lang.Exception: oops
d.s.s.q.MyInternalExceptionAspect : execution(void de.scrum_master.spring.q69862121.MyService.doSomething(Throwable)) -> java.lang.IllegalArgumentException: uh-oh
d.s.s.q69862121.InternalExceptionAspect : execution(void de.scrum_master.spring.q69862121.MyService.doSomething(Throwable)) -> java.lang.IllegalArgumentException: uh-oh
d.s.spring.q69862121.DemoApplication : java.lang.RuntimeException: Transformed by InternalExceptionAspect
d.s.s.q.MyInternalExceptionAspect : execution(void de.scrum_master.spring.q69862121.MyService.doSomething(Throwable)) -> java.lang.NullPointerException: null
d.s.s.q69862121.InternalExceptionAspect : execution(void de.scrum_master.spring.q69862121.MyService.doSomething(Throwable)) -> java.lang.RuntimeException: Transformed by MyInternalExceptionAspect
d.s.spring.q69862121.DemoApplication : java.lang.RuntimeException: Transformed by MyInternalExceptionAspect
d.s.s.q.MyInternalExceptionAspect : execution(void de.scrum_master.spring.q69862121.MyService.doSomething(Throwable)) -> java.lang.ArithmeticException: argh
d.s.s.q69862121.InternalExceptionAspect : execution(void de.scrum_master.spring.q69862121.MyService.doSomething(Throwable)) -> java.lang.IllegalArgumentException: Transformed by MyInternalExceptionAspect
d.s.spring.q69862121.DemoApplication : java.lang.RuntimeException: Transformed by InternalExceptionAspect

Configuring Spring WebFlux WebClient to use a custom thread pool

Is it possible to configure WebClient to use a custom thread pool other than the reactor-http-nio thread pool (When using Netty)? If it is possible , can we somehow restrict that custom thread pool to run only on a particular processor core?
Yes. You can.
Create some where your own Thread Pool and EventLoopGroup (or create NioEventLoopGroup bean). For example:
{
Intger THREADS = 10;
BasicThreadFactory THREADFACTORY = new BasicThreadFactory.Builder()
.namingPattern("HttpThread-%d")
.daemon(true)
.priority(Thread.MAX_PRIORITY)
.build();
EXECUTOR = new ThreadPoolExecutor(
THREADS,
THREADS,
0L,
TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<>(),
THREADFACTORY,
new ThreadPoolExecutor.AbortPolicy());
NioEventLoopGroup RESOURCE= new NioEventLoopGroup(THREADS,EXECUTOR);
}
Register your own ReactorResourceFactory. And provide your own EventLoopGrooup based on custom thread Executor
#Bean
public ReactorResourceFactory reactorResourceFactory(NioEventLoopGroup RESOURCE) {
ReactorResourceFactory f= new ReactorResourceFactory();
f.setLoopResources(new LoopResources() {
#Override
public EventLoopGroup onServer(boolean b) {
return RESOURCE;
}
});
f.setUseGlobalResources(false);
return f;
}
Then register ReactorClientHttpConnector. In example below it is used custom SSL Context
#Bean
public ReactorClientHttpConnector reactorClientHttpConnector(ReactorResourceFactory r) throws SSLException {
SslContext sslContext = SslContextBuilder
.forClient()
.trustManager(InsecureTrustManagerFactory.INSTANCE)
.build();
return new ReactorClientHttpConnector(r, m -> m.secure(t -> t.sslContext(sslContext)));
}
Finally build WebClient
#Bean
public WebClient webClient(ReactorClientHttpConnector r) {
return WebClient.builder().clientConnector(r).build();
}
If you want to use same for WebServer. Do same configuration for ReactiveWebServerFactory.
#Bean
public ReactiveWebServerFactory reactiveWebServerFactory(NioEventLoopGroup RESOURCE) {
NettyReactiveWebServerFactory factory = new NettyReactiveWebServerFactory();
factory.addServerCustomizers(hs->hs.tcpConfiguration(s->s.runOn(RESOURCE)));
return factory;
}
Imports:
import io.netty.channel.EventLoopGroup;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.handler.ssl.SslContext;
import io.netty.handler.ssl.SslContextBuilder;
import io.netty.handler.ssl.util.InsecureTrustManagerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.DependsOn;
import org.springframework.http.client.reactive.ReactorClientHttpConnector;
import org.springframework.http.client.reactive.ReactorResourceFactory;
import org.springframework.stereotype.Component;
import org.springframework.web.reactive.function.client.WebClient;
import reactor.netty.resources.LoopResources;
import org.apache.commons.lang3.concurrent.BasicThreadFactory;
import org.springframework.boot.web.embedded.netty.NettyReactiveWebServerFactory;
import org.springframework.boot.web.reactive.server.ReactiveWebServerFactory;
import java.util.concurrent.*;

Spring Integration Flow route based on file name

I have a requirement to unzip and file and process its contents. Inside the zip file, there could be two types of file individual or firm. That can be distinguished by file name. After processing of all files, it should call another program module, also archive the processed file in a different location. Would like use Spring integration for the same. I am trying to achieve this by the following code, but it is creating problem while routing based on file name. I am using JDK 8, Spring 5
.<File, Boolean>route(new Function<File, Boolean>() {
#Override
public Boolean apply(File f) {
return f.getName().contains("individual");
}
}, m -> m
.subFlowMapping(true, sf -> sf.gateway(individualProcessor()))
.subFlowMapping(false, sf -> sf.gateway(firmProcessor()))
)
Exception
Caused by: java.lang.IllegalArgumentException: Found ambiguous parameter type [interface java.util.function.Function] for method match: [public default <V> java.util.function.Function<V, R> java.util.function.Function.compose(java.util.function.Function<? super V, ? extends T>), public static <T> java.util.function.Function<T, T> java.util.function.Function.identity(), public java.lang.Boolean com.xxx.thirdpatysystem.config.IntegrationConfig$1.apply(java.io.File)]
at org.springframework.util.Assert.isNull(Assert.java:155) ~[spring-core-5.0.4.RELEASE.jar:5.0.4.RELEASE]
at org.springframework.integration.util.MessagingMethodInvokerHelper.findHandlerMethodsForTarget(MessagingMethodInvokerHelper.java:843) ~[spring-integration-core-5.0.3.RELEASE.jar:5.0.3.RELEASE]
at org.springframework.integration.util.MessagingMethodInvokerHelper.<init>(MessagingMethodInvokerHelper.java:362) ~[spring-integration-core-5.0.3.RELEASE.jar:5.0.3.RELEASE]
at org.springframework.integration.util.MessagingMethodInvokerHelper.<init>(MessagingMethodInvokerHelper.java:231) ~[spring-integration-core-5.0.3.RELEASE.jar:5.0.3.RELEASE]
at org.springframework.integration.util.MessagingMethodInvokerHelper.<init>(MessagingMethodInvokerHelper.java:225) ~[spring-integration-core-5.0.3.RELEASE.jar:5.0.3.RELEASE]
at org.springframework.integration.handler.MethodInvokingMessageProcessor.<init>(MethodInvokingMessageProcessor.java:60) ~[spring-integration-core-5.0.3.RELEASE.jar:5.0.3.RELEASE]
at org.springframework.integration.router.MethodInvokingRouter.<init>(MethodInvokingRouter.java:46) ~[spring-integration-core-5.0.3.RELEASE.jar:5.0.3.RELEASE]
at org.springframework.integration.dsl.IntegrationFlowDefinition.route(IntegrationFlowDefinition.java:1922) ~[spring-integration-core-5.0.3.RELEASE.jar:5.0.3.RELEASE]
at org.springframework.integration.dsl.IntegrationFlowDefinition.route(IntegrationFlowDefinition.java:1895) ~[spring-integration-core-5.0.3.RELEASE.jar:5.0.3.RELEASE]
even I have tried below
.<File, Boolean>route(f -> f.getName().contains("individual"), m -> m
.subFlowMapping(true, sf -> sf.gateway(individualProcessor()))
.subFlowMapping(false, sf -> sf.gateway(firmProcessor()))
)
Exception
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.integration.dsl.IntegrationFlow]: Factory method 'thirdpatysystemFlow' threw exception; nested exception is java.lang.UnsupportedOperationException
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:185) ~[spring-beans-5.0.4.RELEASE.jar:5.0.4.RELEASE]
at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:579) ~[spring-beans-5.0.4.RELEASE.jar:5.0.4.RELEASE]
... 17 common frames omitted
Caused by: java.lang.UnsupportedOperationException: null
at org.springframework.integration.dsl.StandardIntegrationFlow.configure(StandardIntegrationFlow.java:89) ~[spring-integration-core-5.0.3.RELEASE.jar:5.0.3.RELEASE]
at org.springframework.integration.dsl.IntegrationFlowDefinition.gateway(IntegrationFlowDefinition.java:2172) ~[spring-integration-core-5.0.3.RELEASE.jar:5.0.3.RELEASE]
at org.springframework.integration.dsl.IntegrationFlowDefinition.gateway(IntegrationFlowDefinition.java:2151) ~[spring-integration-core-5.0.3.RELEASE.jar:5.0.3.RELEASE]
Entire code snippet is below
import java.io.File;
import java.util.concurrent.TimeUnit;
import java.util.function.Function;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.task.TaskExecutor;
import org.springframework.integration.channel.DirectChannel;
import org.springframework.integration.config.EnableIntegration;
import org.springframework.integration.core.MessageSource;
import org.springframework.integration.dsl.IntegrationFlow;
import org.springframework.integration.dsl.IntegrationFlows;
import org.springframework.integration.dsl.Pollers;
import org.springframework.integration.dsl.channel.MessageChannels;
import org.springframework.integration.file.FileReadingMessageSource;
import org.springframework.integration.file.FileWritingMessageHandler;
import org.springframework.integration.file.filters.AcceptOnceFileListFilter;
import org.springframework.integration.file.filters.ChainFileListFilter;
import org.springframework.integration.file.filters.RegexPatternFileListFilter;
import org.springframework.integration.zip.splitter.UnZipResultSplitter;
import org.springframework.integration.zip.transformer.UnZipTransformer;
import org.springframework.integration.zip.transformer.ZipResultType;
import org.springframework.messaging.MessageHandler;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;
/**
* #author dpoddar
*
*/
#Configuration
#EnableIntegration
public class IntegrationConfig {
#Value("${input.directory}")
private String inputDir;
#Value("${outputDir.directory}")
private String outputDir;
#Value("${input.scan.frequency: 100}")
private long scanFrequency;
#Bean
public MessageSource<File> inputFileSource() {
FileReadingMessageSource src = new FileReadingMessageSource();
src.setDirectory(new File(inputDir));
src.setAutoCreateDirectory(true);
ChainFileListFilter<File> chainFileListFilter = new ChainFileListFilter<>();
chainFileListFilter.addFilter(new AcceptOnceFileListFilter<>() );
chainFileListFilter.addFilter(new RegexPatternFileListFilter("(?i)^.+\\.zip$"));
src.setFilter(chainFileListFilter);
return src;
}
#Bean
public UnZipTransformer unZipTransformer() {
UnZipTransformer unZipTransformer = new UnZipTransformer();
unZipTransformer.setExpectSingleResult(false);
unZipTransformer.setZipResultType(ZipResultType.FILE);
//unZipTransformer.setWorkDirectory(new File("/usr/tmp/uncompress"));
unZipTransformer.setDeleteFiles(true);
return unZipTransformer;
}
#Bean
public UnZipResultSplitter splitter() {
UnZipResultSplitter splitter = new UnZipResultSplitter();
return splitter;
}
#Bean
public DirectChannel outputChannel() {
return new DirectChannel();
}
#Bean
public MessageHandler fileOutboundChannelAdapter() {
FileWritingMessageHandler adapter = new FileWritingMessageHandler(new File(outputDir));
adapter.setDeleteSourceFiles(true);
adapter.setAutoCreateDirectory(true);
adapter.setExpectReply(false);
return adapter;
}
#Bean
public TaskExecutor taskExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(5);
executor.setMaxPoolSize(10);
executor.setQueueCapacity(25);
return executor;
}
#Autowired
DirectChannel outputChannel;
#Autowired
MessageHandler fileOutboundChannelAdapter;
#Bean
public IntegrationFlow individualProcessor() {
return flow -> flow.handle("thirdpatysystemprocessor","processfile").channel(outputChannel).handle(fileOutboundChannelAdapter);
}
#Bean
public IntegrationFlow firmProcessor() {
return flow -> flow.handle("thirdpatysystemprocessor","processfile").channel(outputChannel).handle(fileOutboundChannelAdapter);
}
#Bean
public IntegrationFlow thirdpatysystemAgentDemographicFlow() {
return IntegrationFlows
.from(inputFileSource(), spec -> spec.poller(Pollers.fixedDelay(scanFrequency,TimeUnit.SECONDS)))
.transform(unZipTransformer())
.split(splitter())
.channel(MessageChannels.executor(taskExecutor()))
.<File, Boolean>route(new Function<File, Boolean>() {
#Override
public Boolean apply(File f) {
return f.getName().contains("individual");
}
}, m -> m
.subFlowMapping(true, sf -> sf.gateway(individualProcessor()))
.subFlowMapping(false, sf -> sf.gateway(firmProcessor()))
)
.aggregate()
/*.handle("thirdpatysystemprocessor","processfile")
.channel(outputChannel())
.handle(fileOutboundChannelAdapter())*/
.get()
;
}
}
The java.lang.IllegalArgumentException: Found ambiguous parameter type [interface java.util.function.Function] has been fixed in the Spring Integration 5.0.5: https://jira.spring.io/browse/INT-4456. So, now with an explicit Function impl we do this:
MethodInvokingRouter methodInvokingRouter = isLambda(router)
? new MethodInvokingRouter(new LambdaMessageProcessor(router, payloadType))
: new MethodInvokingRouter(router, ClassUtils.FUNCTION_APPLY_METHOD);
We explicitly point to the apply() method.
The re-use of existing IntegrationFlow beans in the sub-flows (gateway()) has been fixed in version 5.0.4: https://jira.spring.io/browse/INT-4434
So, what you need is just to upgrade your project to the latest dependencies. In particular Spring Integration 5.0.7: https://spring.io/projects/spring-integration#learn

How to run DelegatingSecurityContextRunnable every time when tomcat creates new Thread

I have an spring app which is using tomcat with websockets. I would like to use the DelegatingSecurityContextRunnable to be executed every time when tomcat creates a new thread, i.e. warp the tomcat thread. Does anyone know how this is done. The reason for the question can be found.here
Maybe this can be done with using AOP and some advice?
In Spring boot you can configure a Wrapper by hooking into the Tomcat connector. See this as an example:
#Bean
public EmbeddedServletContainerFactory servletContainerFactory() {
TomcatEmbeddedServletContainerFactory factory = new TomcatEmbeddedServletContainerFactory();
factory.addConnectorCustomizers(new TomcatConnectorCustomizer() {
#Override
public void customize(Connector connector) {
AbstractProtocol protocolHandler = (AbstractProtocol) connector.getProtocolHandler();
TaskQueue taskqueue = new TaskQueue() {
#Override
public boolean offer(Runnable e, long timeout, TimeUnit unit) throws InterruptedException {
return super.offer(new MyRunnable(e), timeout, unit);
}
#Override
public boolean offer(Runnable o) {
return super.offer(new MyRunnable(o));
}
};
TaskThreadFactory tf = new TaskThreadFactory("artur-" + "-exec-", false, 0);
ThreadPoolExecutor e = new ThreadPoolExecutor(10, 10, 1000, TimeUnit.SECONDS, taskqueue);
taskqueue.setParent(e);
protocolHandler.setExecutor(e);
}
});
return factory;
}
And here is my custom Runable (this can be any wrapper, i did not bother implementing exactly yours):
static class MyRunnable implements Runnable {
private Runnable r;
public MyRunnable(Runnable r) {
this.r = r;
}
#Override
public void run() {
System.out.println("Custom runable");
runInner();
}
void runInner() {
r.run();
}
}
And here are my imports:
import java.util.concurrent.TimeUnit;
import org.apache.catalina.connector.Connector;
import org.apache.coyote.AbstractProtocol;
import org.apache.tomcat.util.threads.TaskQueue;
import org.apache.tomcat.util.threads.TaskThreadFactory;
import org.apache.tomcat.util.threads.ThreadPoolExecutor;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.context.embedded.EmbeddedServletContainerFactory;
import org.springframework.boot.context.embedded.tomcat.TomcatConnectorCustomizer;
import org.springframework.boot.context.embedded.tomcat.TomcatEmbeddedServletContainerFactory;
import org.springframework.boot.web.support.SpringBootServletInitializer;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.FilterType;
import org.springframework.context.annotation.PropertySource;
What this does:
The Tomcat connector initialises itself. You can set the executor to use, in which case Tomcat will stop creating its own configuration and instead use yours.
By overwriting the offer methods in the queue, you have the chance to wrap your Runnable in any custom Runnable. In my case, for testing, I simply added a Sysout to see that everything is working correctly.
The Threadpool implementation I used is an exact copy of the tomcat default (minus the properties). This way, behaviour stays the same, except that any Runnable is now your delegating wrapper.
When I test that, my console prints:
Custom runable
I hope this is what you were looking for.
I use spring boot, but this is essentially a tomcat issue not a spring issue. You can adapt the solution to your specific scenario.
-- Artur

Resources