How change default channel on Spring Integration Flow with Java DSL - spring

I do not understand very well, as I can change the error channel for all my integration flow. I need to handle exceptions like InvalidAccessTokenException that can be thrown in a subflow inside the router.
What I've tried is to handle exceptions from the default channel "errorChannel" by:
#Bean
public IntegrationFlow errorFlow() {
return IntegrationFlows.from("errorChannel")
.handle("errorService", "handleError")
.get();
}
This error is treated by a method with the following signature:
void handleError(Message<Exception> exception)
But the default behavior persists, that is to say it still shows the trace by console.
So my question is: in java DSL how can I configure an error channel? Is it possible to map a group of exceptions to a particular error channel to make the management service of that group more cohesive?.
The configuration of my integration flow I explain below:
#Configuration
#IntegrationComponentScan
public class InfrastructureConfiguration {
private static Logger logger = LoggerFactory.getLogger(InfrastructureConfiguration.class);
#Autowired
private IFacebookService facebookService;
#Autowired
private IInstagramService instagramService;
#Autowired
private IYoutubeService youtubeService;
/**
* The Pollers builder factory can be used to configure common bean definitions or
* those created from IntegrationFlowBuilder EIP-methods
*/
#Bean(name = PollerMetadata.DEFAULT_POLLER)
public PollerMetadata poller() {
return Pollers.fixedDelay(10, TimeUnit.SECONDS).get();
}
#Bean
public TaskExecutor taskExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(5);
executor.setMaxPoolSize(10);
executor.setQueueCapacity(25);
return executor;
}
/**
* MongoDbMessageSource is an instance of MessageSource which returns a Message with a payload
* which is the result of execution of a Query
*/
#Bean
#Autowired
public MessageSource<Object> mongoMessageSource(MongoDbFactory mongo) {
MongoDbMessageSource messageSource = new MongoDbMessageSource(mongo, new LiteralExpression("{}"));
messageSource.setExpectSingleResult(false);
messageSource.setEntityClass(UserEntity.class);
messageSource.setCollectionNameExpression(new LiteralExpression("users"));
return messageSource;
}
#Bean
#ServiceActivator(inputChannel = "storeChannel")
public MessageHandler mongodbAdapter(MongoDbFactory mongo) throws Exception {
MongoDbStoringMessageHandler adapter = new MongoDbStoringMessageHandler(mongo);
adapter.setCollectionNameExpression(new LiteralExpression("comments"));
return adapter;
}
#Bean
public IntegrationFlow errorFlow() {
return IntegrationFlows.from("errorChannel")
.handle("errorService", "handleError")
.get();
}
#Bean
#Autowired
public IntegrationFlow processUsers(MongoDbFactory mongo, PollerMetadata poller) {
return IntegrationFlows.from(mongoMessageSource(mongo), c -> c.poller(poller))
.<List<UserEntity>, Map<ObjectId, List<SocialMediaEntity>>>transform(userEntitiesList
-> userEntitiesList.stream().collect(Collectors.toMap(UserEntity::getId, UserEntity::getSocialMedia))
)
.split(new AbstractMessageSplitter() {
#Override
protected Object splitMessage(Message<?> msg) {
return ((Map<ObjectId, List<SocialMediaEntity>>) msg.getPayload()).entrySet();
}
})
.channel("directChannel_1")
.enrichHeaders(s -> s.headerExpressions(h -> h.put("user-id", "payload.key")))
.split(new AbstractMessageSplitter() {
#Override
protected Object splitMessage(Message<?> msg) {
return ((Entry<ObjectId, List<SocialMediaEntity>>) msg.getPayload()).getValue();
}
})
.channel(MessageChannels.executor("executorChannel", this.taskExecutor()))
.<SocialMediaEntity, SocialMediaTypeEnum>route(p -> p.getType(),
m
-> m.subFlowMapping(SocialMediaTypeEnum.FACEBOOK,
sf -> sf.handle(SocialMediaEntity.class, (p, h) -> facebookService.getComments(p.getAccessToken())))
.subFlowMapping(SocialMediaTypeEnum.YOUTUBE,
sf -> sf.handle(SocialMediaEntity.class, (p, h) -> youtubeService.getComments(p.getAccessToken())))
.subFlowMapping(SocialMediaTypeEnum.INSTAGRAM,
sf -> sf.handle(SocialMediaEntity.class, (p, h) -> instagramService.getComments(p.getAccessToken())))
)
.channel("directChannel_2")
.aggregate()
.channel("directChannel_3")
.<List<List<CommentEntity>>, List<CommentEntity>>transform(comments ->
comments.stream().flatMap(List::stream).collect(Collectors.toList()))
.aggregate()
.channel("directChannel_4")
.<List<List<CommentEntity>>, List<CommentEntity>>transform(comments ->
comments.stream().flatMap(List::stream).collect(Collectors.toList()))
.channel("storeChannel")
.get();
}
#PostConstruct
protected void init(){
Assert.notNull(facebookService, "The Facebook Service can not be null");
Assert.notNull(instagramService, "The Instagram Service can not be null");
Assert.notNull(youtubeService, "The Youtube Service can not be null");
}
}
An example of an exception that can be launched in any of the social networking services is this:
public class InvalidAccessTokenException extends RuntimeException {
private SocialMediaTypeEnum socialMediaType;
private String accessToken;
public InvalidAccessTokenException(SocialMediaTypeEnum socialMediaType, String accessToken) {
this.socialMediaType = socialMediaType;
this.accessToken = accessToken;
}
public SocialMediaTypeEnum getSocialMediaType() {
return socialMediaType;
}
public String getAccessToken() {
return accessToken;
}
}
Is it possible to bind this exception to a particular error channel?.
Thanks in advance.
I have tried changing the error channel using the PoolSpec as follows:
#Bean(name = PollerMetadata.DEFAULT_POLLER)
public PollerMetadata poller() {
return Pollers.fixedDelay(10, TimeUnit.SECONDS)
.errorChannel("customErrorChannel")
.get();
}
But messages still continue to go to the default channel 'errorChannel'.
Here is an excerpt of the log messages:
2017-07-25 20:20:51.922 DEBUG 3268 --- [ taskExecutor-4] o.s.i.channel.PublishSubscribeChannel : preSend on channel 'errorChannel', message: ErrorMessage [payload=org.springframework.messaging.MessageHandlingException: nested exception is sanchez.sanchez.sergio.exception.InvalidAccessTokenException, failedMessage=GenericMessage [payload=SocialMediaEntity{id=59778bf93681ac0cc4e20089, accessToken=maite_access_token_facebook, type=FACEBOOK, invalidToken=false}, headers={sequenceNumber=1, sequenceDetails=[[35dd7519-59cd-f0ea-69b2-c5fbb7c1c57f, 2, 5]], mongo_collectionName=users, sequenceSize=3, user-id=59778bf93681ac0cc4e20094, correlationId=6665f8ee-2bc9-e6c0-17de-35d63a0afaaa, id=6925b3ea-0b30-3304-b7f0-d235959d7db1, timestamp=1501006851466}], headers={id=6eb1789c-21d7-1814-48ee-77553abd99b9, timestamp=1501006851922}]
2017-07-25 20:20:51.923 DEBUG 3268 --- [ taskExecutor-5] o.s.integration.handler.LoggingHandler : _org.springframework.integration.errorLogger.handler received message: ErrorMessage [payload=org.springframework.messaging.MessageHandlingException: nested exception is sanchez.sanchez.sergio.exception.InvalidAccessTokenException, failedMessage=GenericMessage [payload=SocialMediaEntity{id=59778bf93681ac0cc4e2008c, accessToken=david_access_token_facebook, type=FACEBOOK, invalidToken=false}, headers={sequenceNumber=1, sequenceDetails=[[35dd7519-59cd-f0ea-69b2-c5fbb7c1c57f, 4, 5]], mongo_collectionName=users, sequenceSize=3, user-id=59778bf93681ac0cc4e20095, correlationId=254f918f-52d2-1cff-7ba5-7343a39a8941, id=3a5989df-2af7-56d0-2697-5fcaf75a2891, timestamp=1501006851527}], headers={id=c270d1ef-68ac-cb6b-245f-11e567b5b7e8, timestamp=1501006851922}]
2017-07-25 20:20:51.922 DEBUG 3268 --- [ taskExecutor-3] o.s.integration.handler.LoggingHandler : _org.springframework.integration.errorLogger.handler received message: ErrorMessage [payload=org.springframework.messaging.MessageHandlingException: nested exception is sanchez.sanchez.sergio.exception.InvalidAccessTokenException, failedMessage=GenericMessage [payload=SocialMediaEntity{id=59778bf93681ac0cc4e2008f, accessToken=elena_access_token_facebook, type=FACEBOOK, invalidToken=false}, headers={sequenceNumber=1, sequenceDetails=[[35dd7519-59cd-f0ea-69b2-c5fbb7c1c57f, 5, 5]], mongo_collectionName=users, sequenceSize=3, user-id=59778bf93681ac0cc4e20096, correlationId=1aba58bf-733e-f1ed-a82e-67f482ff3531, id=5eb9c517-f451-e2d6-938d-e436bb362aef, timestamp=1501006851549}], headers={id=30a965d3-f829-d6cf-c971-7b4ed2f15f88, timestamp=1501006851922}]
2017-07-25 20:20:51.923 DEBUG 3268 --- [ taskExecutor-4] o.s.integration.handler.LoggingHandler : _org.springframework.integration.errorLogger.handler received message: ErrorMessage [payload=org.springframework.messaging.MessageHandlingException: nested exception is sanchez.sanchez.sergio.exception.InvalidAccessTokenException, failedMessage=GenericMessage [payload=SocialMediaEntity{id=59778bf93681ac0cc4e20089, accessToken=maite_access_token_facebook, type=FACEBOOK, invalidToken=false}, headers={sequenceNumber=1, sequenceDetails=[[35dd7519-59cd-f0ea-69b2-c5fbb7c1c57f, 2, 5]], mongo_collectionName=users, sequenceSize=3, user-id=59778bf93681ac0cc4e20094, correlationId=6665f8ee-2bc9-e6c0-17de-35d63a0afaaa, id=6925b3ea-0b30-3304-b7f0-d235959d7db1, timestamp=1501006851466}], headers={id=6eb1789c-21d7-1814-48ee-77553abd99b9, timestamp=1501006851922}]
2017-07-25 20:20:51.927 ERROR 3268 --- [ taskExecutor-3] o.s.integration.handler.LoggingHandler : org.springframework.messaging.MessageHandlingException: nested exception is sanchez.sanchez.sergio.exception.InvalidAccessTokenException, failedMessage=GenericMessage [payload=SocialMediaEntity{id=59778bf93681ac0cc4e2008f, accessToken=elena_access_token_facebook, type=FACEBOOK, invalidToken=false}, headers={sequenceNumber=1, sequenceDetails=[[35dd7519-59cd-f0ea-69b2-c5fbb7c1c57f, 5, 5]], mongo_collectionName=users, sequenceSize=3, user-id=59778bf93681ac0cc4e20096, correlationId=1aba58bf-733e-f1ed-a82e-67f482ff3531, id=5eb9c517-f451-e2d6-938d-e436bb362aef, timestamp=1501006851549}]
at org.springframework.integration.dsl.LambdaMessageProcessor.processMessage(LambdaMessageProcessor.java:130)
at org.springframework.integration.handler.ServiceActivatingHandler.handleRequestMessage(ServiceActivatingHandler.java:89)
I have tried two new approaches to change the default error channel:
Create a custom ErrorHandler with the error channel and declare it in the PoolSpec:
#Bean
public MessageChannel customErrorChannel() {
return MessageChannels.direct("customErrorChannel").get();
}
#Bean
public ErrorHandler errorHandler() {
MessagePublishingErrorHandler messagePublishingErrorHandler = new MessagePublishingErrorHandler();
messagePublishingErrorHandler.setDefaultErrorChannel(customErrorChannel());
return messagePublishingErrorHandler;
}
#Bean(name = PollerMetadata.DEFAULT_POLLER)
public PollerMetadata poller() {
return Pollers.fixedDelay(10, TimeUnit.SECONDS)
.errorHandler(errorHandler())
.get();
}
The error messages are still going to the default channel 'errorChannel'.
Adding the MessageHeaders.ERROR_CHANNEL header to explicitly indicate the error channel:
.enrichHeaders(s ->
s.headerExpressions(h -> h.put("user-id", "payload.key"))
.header(MessageHeaders.ERROR_CHANNEL, "customErrorChannel")
)
If this approach works, error messages are directed to the "customErrorChannel".

Yes, you can do that. There is an ErrorMessageExceptionTypeRouter for similar task. So, you will be able to route your InvalidAccessTokenException to the specific channel.
Also be aware that PollerSpec can be supplied with the errorChannel(), so you don't need to worry that all your exceptions go to the default errorChannel.
UPDATE
OK. After some your code investigation I see this:
.channel(MessageChannels.executor("executorChannel", this.taskExecutor()))
That means you shift your message to different thread and, therefore, any exception there is already far away from the try...catch of the poller's algorithm to send to your custom customErrorChannel.
The ExecutorChannel has this logic:
if (!(this.executor instanceof ErrorHandlingTaskExecutor)) {
ErrorHandler errorHandler = new MessagePublishingErrorHandler(
new BeanFactoryChannelResolver(this.getBeanFactory()));
this.executor = new ErrorHandlingTaskExecutor(this.executor, errorHandler);
}
Where that MessagePublishingErrorHandler is really based on the errorChannel by default. What you can do here is something like declaration of similar bean for the taskExecutor() bean and injection your customErrorChannel into the MessagePublishingErrorHandler.
Another option which should work here with the MessagePublishingErrorHandler is errorChannel header population upstream the executorChannel definition.

Related

Messages cannot be routed to the error channel defined in header if an exception occurred in handler

Two IntegrationFlows are defined as follows:
#Bean
public IntegrationFlow myFlow() {
return IntegrationFlows.from("input.channel")
.handle("myService", "handle")
.get();
}
#Bean
public IntegrationFlow exceptionFlow() {
return IntegrationFlows.from("error.channel")
.handle(m -> System.out.println(m.getHeaders()))
.get();
}
and the handler of MyService's `handle1 method is just to print out the message and then throw an exception:
public class MyService {
public String handle(String s) {
System.out.println(s);
throw new RuntimeException("error");
}
}
In the test, a message with a defined error channel value which is exactly error.channel is put into the input.channelchannel, and it is expected to route to theerror.channel` channel.
#Test
public void myTest() {
Message<String> m = MessageBuilder.withPayload("foo").setHeader(MessageHeaders.ERROR_CHANNEL, "error.channel").build();
this.someInputChannel.send(m);
}
However, it throws the exception in the test and the message is not routed to the error channel.
That's correct behavior. The errorChannel header is consulted only when there is a thread executor. Any exception is thrown to a caller as a it is done in plain Java. In case of QueueChannel and an ExecutorChannel there is an MessagePublishingErrorHandler which wraps a task call to try..catch and sends an ErrorMessage into an errorChannel header.
In your case it is just plain Java call with that this.someInputChannel.send(m); , so you get an exception directly in this main thread.
See more info in the Reference Manual: https://docs.spring.io/spring-integration/docs/current/reference/html/#namespace-errorhandler

Spring Integration call another handler method after aggregation

I am developing a system which will read and process file from a directory. Once all the file has been processed it will call a method which in turn generates a file. Also, it should route/process the files based on file name, I have used spring integration router for the same. Below is the code snippet of the Integration. My question is, this is not working if I remove any of the line .channel(aggregatorOutputChannel()) or .channel(confirmChannel()), also I have to keep the same channel .channel(aggregatorOutputChannel()) before and after the aggregator. Why do I need all 3 channel declaration? if this is wrong how to correct it.
I am using JDK 8, Spring 5, Spring boot 2.0.4.
#Configuration
#EnableIntegration
public class IntegrationConfig {
#Value("${agent.demographic.input.directory}")
private String inputDir;
#Value("${agent.demographic.output.directory}")
private String outputDir;
#Value("${confirmationfile.directory}")
private String confirmDir;
#Value("${input.scan.frequency: 2}")
private long scanFrequency;
#Value("${processing.waittime: 6000}")
private long messageGroupWaiting;
#Value("${thread.corepoolsize: 10}")
private int corepoolsize;
#Value("${thread.maxpoolsize: 20}")
private int maxpoolsize;
#Value("${thread.queuecapacity: 1000}")
private int queuedepth;
#Bean
public MessageSource<File> inputFileSource() {
FileReadingMessageSource src = new FileReadingMessageSource();
src.setDirectory(new File(inputDir));
src.setAutoCreateDirectory(true);
ChainFileListFilter<File> chainFileListFilter = new ChainFileListFilter<>();
chainFileListFilter.addFilter(new AcceptOnceFileListFilter<>() );
chainFileListFilter.addFilter(new RegexPatternFileListFilter("(?i)^.+\\.xml$"));
src.setFilter(chainFileListFilter);
return src;
}
#Bean
public UnZipTransformer unZipTransformer() {
UnZipTransformer unZipTransformer = new UnZipTransformer();
unZipTransformer.setExpectSingleResult(false);
unZipTransformer.setZipResultType(ZipResultType.FILE);
unZipTransformer.setDeleteFiles(true);
return unZipTransformer;
}
#Bean("agentdemographicsplitter")
public UnZipResultSplitter splitter() {
UnZipResultSplitter splitter = new UnZipResultSplitter();
return splitter;
}
#Bean
public DirectChannel outputChannel() {
return new DirectChannel();
}
#Bean
public DirectChannel aggregatorOutputChannel() {
return new DirectChannel();
}
#Bean("confirmChannel")
public DirectChannel confirmChannel() {
return new DirectChannel();
}
#Bean
public MessageHandler fileOutboundChannelAdapter() {
FileWritingMessageHandler adapter = new FileWritingMessageHandler(new File(outputDir));
adapter.setDeleteSourceFiles(true);
adapter.setAutoCreateDirectory(true);
adapter.setExpectReply(true);
adapter.setLoggingEnabled(true);
return adapter;
}
#Bean
public MessageHandler confirmationfileOutboundChannelAdapter() {
FileWritingMessageHandler adapter = new FileWritingMessageHandler(new File(confirmDir));
adapter.setDeleteSourceFiles(true);
adapter.setAutoCreateDirectory(true);
adapter.setExpectReply(false);
adapter.setFileNameGenerator(defaultFileNameGenerator() );
return adapter;
}
#Bean
public TaskExecutor taskExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(corepoolsize);
executor.setMaxPoolSize(maxpoolsize);
executor.setQueueCapacity(queuedepth);
return executor;
}
#Bean
public DefaultFileNameGenerator defaultFileNameGenerator() {
DefaultFileNameGenerator defaultFileNameGenerator = new DefaultFileNameGenerator();
defaultFileNameGenerator.setExpression("payload.name");
return defaultFileNameGenerator;
}
#Bean
public IntegrationFlow confirmGeneration() {
return IntegrationFlows.
from("confirmChannel")
.handle(confirmationfileOutboundChannelAdapter())
.get();
}
#Bean
public IntegrationFlow individualProcessor() {
return flow -> flow.handle("thirdpartyIndividualAgentProcessor","processfile").channel(outputChannel()).handle(fileOutboundChannelAdapter());
}
#Bean
public IntegrationFlow firmProcessor() {
return flow -> flow.handle("thirdpartyFirmAgentProcessor","processfile").channel(outputChannel()).handle(fileOutboundChannelAdapter());
}
#Bean
public IntegrationFlow thirdpartyAgentDemographicFlow() {
return IntegrationFlows
.from(inputFileSource(), spec -> spec.poller(Pollers.fixedDelay(scanFrequency,TimeUnit.SECONDS)))
.channel(MessageChannels.executor(taskExecutor()))
.<File, Boolean>route(f -> f.getName().contains("individual"), m -> m
.subFlowMapping(true, sf -> sf.gateway(individualProcessor()))
.subFlowMapping(false, sf -> sf.gateway(firmProcessor()))
)
.channel(aggregatorOutputChannel())
.aggregate(aggregator -> aggregator.groupTimeout(messageGroupWaiting).correlationStrategy(new CorrelationStrategy() {
#Override
public Object getCorrelationKey(Message<?> message) {
return "xyz";
}
}))
.channel(aggregatorOutputChannel())
.handle("agentDemograpicOutput","generateAgentDemographicFile")
.channel(confirmChannel())
.get();
}
}
Below is the log
2018-09-07 17:29:20.003 DEBUG 10060 --- [ taskExecutor-2] o.s.integration.channel.DirectChannel : preSend on channel 'outputChannel', message: GenericMessage [payload=C:\thirdpartyintg\input\18237232_firm.xml, headers={replyChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#1a867ae7, errorChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#1a867ae7, file_name=18237232_firm.xml, file_originalFile=C:\thirdpartyintg\input\18237232_firm.xml, id=dd70999a-8b8d-93d2-1a43-a961ac2c339f, file_relativePath=18237232_firm.xml, timestamp=1536366560003}]
2018-09-07 17:29:20.003 DEBUG 10060 --- [ taskExecutor-2] o.s.i.file.FileWritingMessageHandler : fileOutboundChannelAdapter received message: GenericMessage [payload=C:\thirdpartyintg\input\18237232_firm.xml, headers={replyChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#1a867ae7, errorChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#1a867ae7, file_name=18237232_firm.xml, file_originalFile=C:\thirdpartyintg\input\18237232_firm.xml, id=dd70999a-8b8d-93d2-1a43-a961ac2c339f, file_relativePath=18237232_firm.xml, timestamp=1536366560003}]
2018-09-07 17:29:20.006 DEBUG 10060 --- [ taskExecutor-2] o.s.integration.channel.DirectChannel : postSend (sent=true) on channel 'outputChannel', message: GenericMessage [payload=C:\thirdpartyintg\input\18237232_firm.xml, headers={replyChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#1a867ae7, errorChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#1a867ae7, file_name=18237232_firm.xml, file_originalFile=C:\thirdpartyintg\input\18237232_firm.xml, id=dd70999a-8b8d-93d2-1a43-a961ac2c339f, file_relativePath=18237232_firm.xml, timestamp=1536366560003}]
2018-09-07 17:29:20.006 DEBUG 10060 --- [ taskExecutor-2] o.s.integration.channel.DirectChannel : postSend (sent=true) on channel 'firmProcessor.input', message: GenericMessage [payload=C:\thirdpartyintg\input\18237232_firm.xml, headers={replyChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#1a867ae7, errorChannel=org.springframework.messaging.core.GenericMessagingTemplate$TemporaryReplyChannel#1a867ae7, file_name=18237232_firm.xml, file_originalFile=C:\thirdpartyintg\input\18237232_firm.xml, id=0e6dcb75-db99-1740-7b58-e9b42bfbf603, file_relativePath=18237232_firm.xml, timestamp=1536366559761}]
2018-09-07 17:29:20.007 DEBUG 10060 --- [ taskExecutor-2] o.s.integration.channel.DirectChannel : preSend on channel 'thirdpartyintgAgentDemographicFlow.channel#2', message: GenericMessage [payload=C:\thirdpartyintg\output\18237232_firm.xml, headers={file_originalFile=C:\thirdpartyintg\input\18237232_firm.xml, id=e6e2a30a-60b9-7cdd-84cc-4977d4c21c97, file_name=18237232_firm.xml, file_relativePath=18237232_firm.xml, timestamp=1536366560007}]
2018-09-07 17:29:20.008 DEBUG 10060 --- [ taskExecutor-2] o.s.integration.channel.DirectChannel : postSend (sent=true) on channel 'thirdpartyintgAgentDemographicFlow.channel#2', message: GenericMessage [payload=C:\thirdpartyintg\output\18237232_firm.xml, headers={file_originalFile=C:\thirdpartyintg\input\18237232_firm.xml, id=e6e2a30a-60b9-7cdd-84cc-4977d4c21c97, file_name=18237232_firm.xml, file_relativePath=18237232_firm.xml, timestamp=1536366560007}]
2018-09-07 17:29:20.009 DEBUG 10060 --- [ taskExecutor-2] o.s.integration.channel.DirectChannel : postSend (sent=true) on channel 'thirdpartyintgAgentDemographicFlow.subFlow#1.channel#0', message: GenericMessage [payload=C:\thirdpartyintg\input\18237232_firm.xml, headers={file_originalFile=C:\thirdpartyintg\input\18237232_firm.xml, id=13713de8-91ce-b1fa-f52d-450d3038cf9c, file_name=18237232_firm.xml, file_relativePath=18237232_firm.xml, timestamp=1536366559757}]
2018-09-07 17:29:26.009 INFO 10060 --- [ask-scheduler-9] o.s.i.a.AggregatingMessageHandler : Expiring MessageGroup with correlationKey[processdate]
2018-09-07 17:29:26.011 DEBUG 10060 --- [ask-scheduler-9] o.s.integration.channel.NullChannel : message sent to null channel: GenericMessage [payload=C:\thirdpartyintg\output\17019222_individual.xml, headers={file_originalFile=C:\thirdpartyintg\input\17019222_individual.xml, id=c654076b-696f-25d4-bded-0a43d1a8ca97, file_name=17019222_individual.xml, file_relativePath=17019222_individual.xml, timestamp=1536366559927}]
2018-09-07 17:29:26.011 DEBUG 10060 --- [ask-scheduler-9] o.s.integration.channel.NullChannel : message sent to null channel: GenericMessage [payload=C:\thirdpartyintg\output\18237232_firm.xml, headers={file_originalFile=C:\thirdpartyintg\input\18237232_firm.xml, id=e6e2a30a-60b9-7cdd-84cc-4977d4c21c97, file_name=18237232_firm.xml, file_relativePath=18237232_firm.xml, timestamp=1536366560007}]
First of all the RegexPatternFileListFilter should be first in the ChainFileListFilter. This way you won't overhead a memory in the AcceptOnceFileListFilter for files which you are not interested in.
You need .channel(confirmChannel()) in the end of the thirdpartyAgentDemographicFlow because this one is an input to your confirmGeneration flow.
I don't think that you .channel(aggregatorOutputChannel()) at all it has to implicit.
You also don't need that .channel(outputChannel()) in the sub-flows.
this is not working
Please, elaborate more: what error you get, how then it works etc...
You also can share some DEBUG logs for the org.springframework.integration to determine how your messages travel.
Also it would help a lot if your share some simple Spring Boot project on GitHub to let us to play with and reproduce according your provided instructions.
UPDATE
Also I've noticed that your aggregator is based on the groupTimeout(). To make it to send aggregated message to downstream you also need to configure there this:
/**
* #param sendPartialResultOnExpiry the sendPartialResultOnExpiry.
* #return the handler spec.
* #see AbstractCorrelatingMessageHandler#setSendPartialResultOnExpiry(boolean)
*/
public S sendPartialResultOnExpiry(boolean sendPartialResultOnExpiry) {
It is false by default, so your messages indeed are sent to the NullChannel.
See more info in the Docs: https://docs.spring.io/spring-integration/docs/current/reference/html/messaging-routing-chapter.html#agg-and-group-to

#Transactional rabbit listener cause infinite loop

Hi I have problem with my Rabbit listener which cause infinite loop on exception (requeue message). My configuration looks:
#Bean(name = "defContainer")
public RabbitListenerContainerFactory containerFactory(ConnectionFactory connectionFactory, PlatformTransactionManager transactionManager){
SimpleRabbitListenerContainerFactory containerFactory = new SimpleRabbitListenerContainerFactory();
containerFactory.setConnectionFactory(connectionFactory);
containerFactory.setConcurrentConsumers(5);
containerFactory.setAcknowledgeMode(AcknowledgeMode.AUTO);
containerFactory.setTransactionManager(transactionManager);
containerFactory.setMessageConverter(messageConverterAmqp());
containerFactory.setDefaultRequeueRejected(false);
return new TxRabbitListenerContainerFactory(containerFactory);
}
where transactionManager is JpaTransactionManager for transaction on postgre db.
TxRabbitListenerContainerFactory is my factory which set setAlwaysRequeueWithTxManagerRollback to false:
public class TxRabbitListenerContainerFactory implements RabbitListenerContainerFactory {
private SimpleRabbitListenerContainerFactory factory;
public TxRabbitListenerContainerFactory(SimpleRabbitListenerContainerFactory factory) {
this.factory = factory;
}
#Override
public MessageListenerContainer createListenerContainer(RabbitListenerEndpoint endpoint) {
SimpleMessageListenerContainer container = factory.createListenerContainer(endpoint);
container.setAlwaysRequeueWithTxManagerRollback(false);
return container;
}
}
Now I have listner like:
#RabbitListener(bindings = #QueueBinding(
value = #Queue(value = "topic.two", durable = "true"),
exchange = #Exchange(value = "topic.def", type = "topic", durable = "true"),
key = "letter.*"
), errorHandler = "rabErrorHandler", containerFactory = "defContainer")
#Transactional
public Motorcycle topicLetters(Motorcycle motorcycle) throws Exception{
motorcycle.setId(UUID.randomUUID().toString());
Testing testing = new Testing();
testingRepository.save(testing);
throwEx();
return motorcycle;
}
where method throwEx(); throw unchecked exception.
Data from DB are properly rollbacked (not commited), but message are constantly requeued, see it in listener:
#Bean
public RabbitListenerErrorHandler rabErrorHandler(){
return new RabbitListenerErrorHandler() {
#Override
public Object handleError(Message message, org.springframework.messaging.Message<?> message1, ListenerExecutionFailedException e) throws Exception {
System.out.println("FFFFFFFFFFF");
return null;
}
};
}
How to prevvent infinite loope, and why is it happend ?
EDIT:
Logs: pasted logs
Set defaultRequeueRejected to false on the container factory.
To programmatically decide when to requeue or nor, leave that at true and throw an AmqpRejectAndDontRequeueException when you don't want it requeued.
EDIT
There's something not adding up...
protected void prepareHolderForRollback(RabbitResourceHolder resourceHolder, RuntimeException exception) {
if (resourceHolder != null) {
resourceHolder.setRequeueOnRollback(isAlwaysRequeueWithTxManagerRollback() ||
RabbitUtils.shouldRequeue(isDefaultRequeueRejected(), exception, logger));
}
}
If both booleans are false, we don't requeue.
Found issue:
Reason: caused by errorHandler handler which was specified on listener level. In some cases error handler return null - which was causing infinite loop (instead rethrowing exception and rollback transaction)

DestinationResolutionException: no output-channel or replyChannel header available

I am implementing a flow where using a MongoDbMessageSource I get a list of users and I want to process each document in parallel. For this I use the default behavior of Split.
But the following error occurs after the split:
o.s.i.channel.PublishSubscribeChannel : preSend on channel 'errorChannel', message: ErrorMessage [payload=org.springframework.messaging.MessagingException: Dispatcher failed to deliver Message; nested exception is org.springframework.messaging.core.DestinationResolutionException: no output-channel or replyChannel header available, failedMessage=GenericMessage [payload=UserEntity{id=5974dfe53681ac160c78dc0f, firstName=David, lastName=García, age=14, socialMedia=[]}, headers={sequenceNumber=4, correlationId=8f8f7b7a-832a-8942-1922-26b6a7529091, id=bb373e42-d59c-42e6-d221-68bf1f56fec3, mongo_collectionName=users, sequenceSize=5, timestamp=1500831727759}], headers={id=9187ffcd-8c79-eb1e-8791-1cfe558ab134, timestamp=1500831727762}]
The code is as follows:
#Configuration
#IntegrationComponentScan
public class InfrastructureConfiguration {
private static Logger logger = LoggerFactory.getLogger(InfrastructureConfiguration.class);
/**
* The Pollers builder factory can be used to configure common bean definitions or
* those created from IntegrationFlowBuilder EIP-methods
*/
#Bean(name = PollerMetadata.DEFAULT_POLLER)
public PollerMetadata poller() {
return Pollers.fixedDelay(10, TimeUnit.SECONDS).get();
}
#Bean
public TaskExecutor taskExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(5);
executor.setMaxPoolSize(10);
executor.setQueueCapacity(25);
return executor;
}
/**
*
* MongoDbMessageSource is an instance of MessageSource which returns a Message with a payload
* which is the result of execution of a Query
*/
#Bean
#Autowired
public MessageSource<Object> mongoMessageSource(MongoDbFactory mongo) {
MongoDbMessageSource messageSource = new MongoDbMessageSource(mongo, new LiteralExpression("{}"));
messageSource.setExpectSingleResult(false);
messageSource.setEntityClass(UserEntity.class);
messageSource.setCollectionNameExpression(new LiteralExpression("users"));
return messageSource;
}
#Bean
public DirectChannel inputChannel() {
return new DirectChannel();
}
#Bean
#Autowired
public IntegrationFlow processUsers(MongoDbFactory mongo, PollerMetadata poller) {
return IntegrationFlows.from(mongoMessageSource(mongo), c -> c.poller(poller))
.split()
.channel(MessageChannels.executor("executorChannel", this.taskExecutor()))
.handle((GenericHandler<UserEntity>) (payload, headers) -> {
logger.debug("user:" + payload + " on thread "
+ Thread.currentThread().getName());
return payload;
})
.aggregate()
.get();
}
}
Does anyone know I'm doing wrong? Thanks in advance.
Use MessageHandler as suggested Barath:
#Bean
#Autowired
public IntegrationFlow processUsers(MongoDbFactory mongo, PollerMetadata poller) {
return IntegrationFlows.from(mongoMessageSource(mongo), c -> c.poller(poller))
.split()
.channel(MessageChannels.executor("executorChannel", this.taskExecutor()))
.wireTap(sf -> sf.handle(user -> logger.debug("user:" + user.getPayload().toString() + " on thread " + Thread.currentThread().getName())))
.aggregate()
.get();
}
The error persists, the complete trace of the error I put below:
2017-07-23 21:46:36.785 DEBUG 15148 --- [ taskExecutor-4] o.s.integration.handler.LoggingHandler : _org.springframework.integration.errorLogger.handler received message: ErrorMessage [payload=org.springframework.messaging.MessagingException: Dispatcher failed to deliver Message; nested exception is org.springframework.messaging.core.DestinationResolutionException: no output-channel or replyChannel header available, failedMessage=GenericMessage [payload=UserEntity{id=5974fd123681ac3b2c5c343a, firstName=David, lastName=García, age=14, socialMedia=[]}, headers={sequenceNumber=4, correlationId=da7be297-992b-8f5a-d41c-58a89e654fcc, id=eb55d6bf-8108-4be5-8b32-6260d4ceea9b, mongo_collectionName=users, sequenceSize=5, timestamp=1500839196770}], headers={id=83a833cd-ac87-f6be-fb75-8d6907e3d194, timestamp=1500839196784}]
2017-07-23 21:46:36.788 ERROR 15148 --- [ taskExecutor-4] o.s.integration.handler.LoggingHandler : org.springframework.messaging.MessagingException: Dispatcher failed to deliver Message; nested exception is org.springframework.messaging.core.DestinationResolutionException: no output-channel or replyChannel header available, failedMessage=GenericMessage [payload=UserEntity{id=5974fd123681ac3b2c5c343a, firstName=David, lastName=García, age=14, socialMedia=[]}, headers={sequenceNumber=4, correlationId=da7be297-992b-8f5a-d41c-58a89e654fcc, id=eb55d6bf-8108-4be5-8b32-6260d4ceea9b, mongo_collectionName=users, sequenceSize=5, timestamp=1500839196770}]
at org.springframework.integration.dispatcher.AbstractDispatcher.wrapExceptionIfNecessary(AbstractDispatcher.java:133)
at org.springframework.integration.dispatcher.AbstractDispatcher.tryOptimizedDispatch(AbstractDispatcher.java:120)
at org.springframework.integration.dispatcher.UnicastingDispatcher.doDispatch(UnicastingDispatcher.java:148)
at org.springframework.integration.dispatcher.UnicastingDispatcher.access$000(UnicastingDispatcher.java:53)
at org.springframework.integration.dispatcher.UnicastingDispatcher$3.run(UnicastingDispatcher.java:129)
at org.springframework.integration.util.ErrorHandlingTaskExecutor$1.run(ErrorHandlingTaskExecutor.java:55)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.springframework.messaging.core.DestinationResolutionException: no output-channel or replyChannel header available
at org.springframework.integration.handler.AbstractMessageProducingHandler.sendOutput(AbstractMessageProducingHandler.java:353)
at org.springframework.integration.handler.AbstractMessageProducingHandler.produceOutput(AbstractMessageProducingHandler.java:269)
at org.springframework.integration.handler.AbstractMessageProducingHandler.sendOutputs(AbstractMessageProducingHandler.java:186)
at org.springframework.integration.aggregator.AbstractCorrelatingMessageHandler.completeGroup(AbstractCorrelatingMessageHandler.java:671)
at org.springframework.integration.aggregator.AbstractCorrelatingMessageHandler.handleMessageInternal(AbstractCorrelatingMessageHandler.java:418)
at org.springframework.integration.handler.AbstractMessageHandler.handleMessage(AbstractMessageHandler.java:127)
at org.springframework.integration.dispatcher.AbstractDispatcher.tryOptimizedDispatch(AbstractDispatcher.java:116)
... 7 more
2017-07-23 21:46:36.788 DEBUG 15148 --- [ taskExecutor-4] o.s.i.channel.PublishSubscribeChannel : postSend (sent=true) on channel 'errorChannel', message: ErrorMessage [payload=org.springframework.messaging.MessagingException: Dispatcher failed to deliver Message; nested exception is org.springframework.messaging.core.DestinationResolutionException: no output-channel or replyChannel header available, failedMessage=GenericMessage [payload=UserEntity{id=5974fd123681ac3b2c5c343a, firstName=David, lastName=García, age=14, socialMedia=[]}, headers={sequenceNumber=4, correlationId=da7be297-992b-8f5a-d41c-58a89e654fcc, id=eb55d6bf-8108-4be5-8b32-6260d4ceea9b, mongo_collectionName=users, sequenceSize=5, timestamp=1500839196770}], headers={id=83a833cd-ac87-f6be-fb75-8d6907e3d194, timestamp=1500839196784}]
Your problem that Aggregator is request-reply component, but you don't have anything after that in your flow. That's why you have that error. You have to decide what to do with the aggregator result.

Error when sending bytes through TCP: Unexpected message - no endpoint registered with connection interceptor

I'm trying to rewrite integration.xml in an application to Java Config using DSL. My integration flow goes like this:
Communication object comes to sendCommunication channel
sendCommunication channel is routed into two different channels
objects in each channel are transformed to byte[]
data is logged with custom logger (wire tapped)
bytes from each channel are sent through TCP using two different TcpSendingMessageHandlers
Here's a part of my Integration.java related to this flow (some beans like custom logger are skipped):
#Bean(name = "sendCommunicationRouter")
public IntegrationFlow routeRoundRobin() {
return IntegrationFlows.from(getSendCommunication())
.route(roundRobinRouter, "route",
s -> s.channelMapping("sendCommunication1",
"sendCommunication1")
.channelMapping("sendCommunication2",
"sendCommunication2"))
.get();
}
#Bean(name = "sendCommunication")
public MessageChannel getSendCommunication() {
return getDefaultMessageChannel();
}
#Bean(name = "sendCommunication1")
public MessageChannel getSendCommunication1() {
return getDefaultMessageChannel();
}
#Bean(name = "sendCommunication2")
public MessageChannel getSendCommunication2() {
return getDefaultMessageChannel();
}
#Bean(name = "tcpClientOutbound1")
public TcpSendingMessageHandler getTcpClientOutbound1() {
return getDefaultTcpClientOutbound(getOutboundConnectionFactory1());
}
#Bean(name = "tcpClientOutbound2")
public TcpSendingMessageHandler getTcpClientOutbound2() {
return getDefaultTcpClientOutbound(getOutboundConnectionFactory2());
}
private TcpSendingMessageHandler getDefaultTcpClientOutbound(TcpNetClientConnectionFactory connectionFactory) {
TcpSendingMessageHandler handler = new TcpSendingMessageHandler();
handler.setConnectionFactory(connectionFactory);
handler.setTaskScheduler(myScheduler);
handler.setClientMode(true);
handler.setRetryInterval(DEFAULT_CHANNEL_RETRY_INTERVAL);
handler.start();
return handler;
}
#Bean
public IntegrationFlow handleOutgoingCommunication1() {
return handleOutgoingCommunication(getSendCommunication1(), getTcpClientOutbound1());
}
#Bean
public IntegrationFlow handleOutgoingCommunication2() {
return handleOutgoingCommunication(getSendCommunication2(), getTcpClientOutbound2());
}
private IntegrationFlow handleOutgoingCommunication(MessageChannel inputChannel, TcpSendingMessageHandler handler) {
return IntegrationFlows.from(inputChannel)
.<Communication, byte[]>transform(communication -> communicationTransformer.toBytes(communication))
.wireTap(getLogger())
.handle(handler)
.get();
}
I'm getting this error when I'm trying to send data through sendCommunication channel (IPs hidden intentionally):
2016-10-09 19:52:45 WARN TcpNetConnection:186 - Unexpected message -
no endpoint registered with connection interceptor:
IP:PORT:37007:b2347dad-b65c-4686-b016-5ef5ee613bd5 - GenericMessage [payload=byte[267], headers={ip_tcp_remotePort=PORT,
ip_connectionId=IP:PORT:37007:b2347dad-b65c-4686-b016-5ef5ee613bd5,
ip_localInetAddress=/LOCAL IP, ip_address=IP,
id=c6fb70b1-6d06-a909-cfc4-4eac7c715de5, ip_hostname=IP,
timestamp=1476035565330}]
I'd kindly appreciate any help, this error is giving me a headache since yesterday. I couldn't find any explanation myself, google is giving me only this source code on github.
It's just a warning that the an inbound message (reply?) was received from the server to which you sent a message, and there's no inbound channel adapter configured to handle incoming messages.
Perhaps if you show the XML you are trying to replace with Java config, someone can help.

Resources