We have a Spring Java application using RabbitMQ, and here is the scenario:
There is a consumer receiving messages from a queue and sending them to another one. We are using "SimpleRabbitListenerContainerFactory" as the container factory, but when sending the messages to the other queue inside a "parallelStream" we've got an IllegalStateException "Cannot determine target ConnectionFactory for lookup key" Exception
When we remove the "parallelStream" it works flawlessly.
public void sendMessage(final StagingMessage stagingMessage, final Long timestamp, final String country) {
final List<TransformedMessage> messages = processMessageList(stagingMessage);
messages.parallelStream().forEach(message -> {
final TransformedMessage transformedMessage = buildMessage(timestamp, ApiConstants.POST_METHOD, country);
myMessageSender.sendQueue(country, transformedMessage);
});
}
Connectio Facotory, where the lookup key is set:
#Configuration
#EnableRabbit
public class RabbitBaseConfig {
#Autowired
private QueueProperties queueProperties;
#Bean
#Primary
public ConnectionFactory connectionFactory(final ConnectionFactory connectionFactoryA, final ConnectionFactory connectionFactoryB) {
final SimpleRoutingConnectionFactory simpleRoutingConnectionFactory = new SimpleRoutingConnectionFactory();
final Map<Object, ConnectionFactory> map = new HashMap<>();
for (final String queue : queueProperties.getAQueueMap().values()) {
map.put("[" + queue + "]", connectionFactoryA);
}
for (final String queue : queueProperties.getBQueueMap().values()) {
map.put("[" + queue + "]", connectionFactoryB);
}
simpleRoutingConnectionFactory.setTargetConnectionFactories(map);
return simpleRoutingConnectionFactory;
}
#Bean
public Jackson2JsonMessageConverter jackson2JsonMessageConverter() {
return new Jackson2JsonMessageConverter();
}
}
Welcome to stack overflow!
You should always show the pertinent code and configuration beans when asking questions like this.
I assume you are using the RoutingConnectionFactory.
It uses a ThreadLocal to store the lookup key so the send has to happen on the same thread that set the key.
You generally should never go asynchronous in a listener anyway; you risk message loss. To increase concurrency, use the concurrency properties on the container.
EDIT
One technique would be to convey the lookup key in a message header:
#Bean
public RabbitTemplate template(ConnectionFactory rcf) {
RabbitTemplate rabbitTemplate = new RabbitTemplate(rcf);
Expression expression = new SpelExpressionParser().parseExpression("messageProperties.headers['cfSelector']");
rabbitTemplate.setSendConnectionFactorySelectorExpression(expression);
return rabbitTemplate;
}
#RabbitListener(queues = "foo")
public void listen1(String in) {
IntStream.range(0, 10)
.parallel()
.mapToObj(i -> in + i)
.forEach(val -> {
this.template.convertAndSend("bar", val.toUpperCase(), msg -> {
msg.getMessageProperties().setHeader("cfSelector", "[bar]");
return msg;
});
});
}
I'm pretty new to spring-integration anyway I'm using it in order to receive mails and elaborate them.
I used this spring configuration class:
#Configuration
#EnableIntegration
#PropertySource(value = { "classpath:configuration.properties" }, encoding = "UTF-8", ignoreResourceNotFound = false)
public class MailReceiverConfiguration {
private static final Log logger = LogFactory.getLog(MailReceiverConfiguration.class);
#Autowired
private EmailTransformerService emailTransformerService;
// Configurazione AE
#Bean
public MessageChannel inboundChannelAE() {
return new DirectChannel();
}
#Bean(name= {"aeProps"})
public Properties aeProps() {
Properties javaMailPropertiesAE = new Properties();
javaMailPropertiesAE.put("mail.store.protocol", "imap");
javaMailPropertiesAE.put("mail.debug", Boolean.TRUE);
javaMailPropertiesAE.put("mail.auth.debug", Boolean.TRUE);
javaMailPropertiesAE.put("mail.smtp.socketFactory.fallback", "false");
javaMailPropertiesAE.put("mail.imap.socketFactory.class", "javax.net.ssl.SSLSocketFactory");
return javaMailPropertiesAE;
}
#Bean(name="mailReceiverAE")
public MailReceiver mailReceiverAE(#Autowired MailConfigurationBean mcb, #Autowired #Qualifier("aeProps") Properties javaMailPropertiesAE) throws Exception {
return ConfigurationUtil.getMailReceiver("imap://USERNAME:PASSWORD#MAILSERVER:PORT/INBOX", new BigDecimal(2), javaMailPropertiesAE);
}
#Bean
#InboundChannelAdapter( autoStartup = "true",
channel = "inboundChannelAE",
poller = {#Poller(fixedRate = "${fixed.rate.ae}",
maxMessagesPerPoll = "${max.messages.per.poll.ae}") })
public MailReceivingMessageSource pollForEmailAE(#Autowired MailReceiver mailReceiverAE) {
MailReceivingMessageSource mrms = new MailReceivingMessageSource(mailReceiverAE);
return mrms;
}
#Transformer(inputChannel = "inboundChannelAE", outputChannel = "transformerChannelAE")
public MessageBean transformitAE( MimeMessage mailMessage ) throws Exception {
// amministratore email inbox
MessageBean messageBean = emailTransformerService.transformit(mailMessage);
return messageBean;
}
#Splitter(inputChannel = "transformerChannelAE", outputChannel = "nullChannel")
public List<Message<?>> splitIntoMessagesAE(final MessageBean mb) {
final List<Message<?>> messages = new ArrayList<Message<?>>();
for (EmailFragment emailFragment : mb.getEmailFragments()) {
Message<?> message = MessageBuilder.withPayload(emailFragment.getData())
.setHeader(FileHeaders.FILENAME, emailFragment.getFilename())
.setHeader("directory", emailFragment.getDirectory()).build();
messages.add(message);
}
return messages;
}
}
So far so good.... I start my micro-service and there is this component listening on the specified mail server and mails are downloaded.
Now I have this requirement: mail server configuration (I mean the string "imap://USERNAME:PASSWORD#MAILSERVER:PORT/INBOX") must be taken from a database and it can be configurable. In any time a system administrator can change it and the mail receiver must use the new configuration.
As far as I understood I should create a new instance of MailReceiver when a new configuration is present and use it in the InboundChannelAdapter
Is there any best practice in order to do it? I found this solution: ImapMailReceiver NO STORE attempt on READ-ONLY folder (Failure) [THROTTLED];
In this solution I can inject the ThreadPoolTaskScheduler if I define it in my Configuration class; I can also inject the DirectChannel but every-time I should create a new MailReceiver and a new ImapIdleChannelAdapter without considering this WARN message I get when the
ImapIdleChannelAdapter starts:
java.lang.RuntimeException: No beanfactory at org.springframework.integration.expression.ExpressionUtils.createStandardEvaluationContext(ExpressionUtils.java:79) at org.springframework.integration.mail.AbstractMailReceiver.onInit(AbstractMailReceiver.java:403)
Is there a better way to satisfy my scenario?
Thank you
Angelo
The best way to do this is to use the Java DSL and dynamic flow registration.
Documentation here.
That way, you can unregister the old flow and register a new one, each time the configuration changes.
It will automatically handle injecting dependencies such as the bean factory.
I am trying to figure out the best way to handle errors that might have occurred in a service that is called after a aggregate's group timeout occurred that mimics the same flow as if the releaseExpression was met.
Here is my setup:
I have a AmqpInboundChannelAdapter that takes in messages and send them to my aggregator.
When the releaseExpression has been met and before the groupTimeout has expired, if an exception gets thrown in my ServiceActivator, the messages get sent to my dead letter queue for all the messages in that MessageGroup. (10 messages in my example below, which is only used for illustrative purposes) This is what I would expect.
If my releaseExpression hasn't been met but the groupTimeout has been met and the group times out, if an exception gets throw in my ServiceActivator, then the messages do not get sent to my dead letter queue and are acked.
After reading another blog post,
link1
it mentions that this happens because the processing happens in another thread by the MessageGroupStoreReaper and not the one that the SimpleMessageListenerContainer was on. Once processing moves away from the SimpleMessageListener's thread, the messages will be auto ack.
I added the configuration mentioned in the link above and see the error messages getting sent to my error handler. My main question, is what is considered the best way to handle this scenario to minimize message getting lost.
Here are the options I was exploring:
Use a BatchRabbitTemplate in my custom error handler to publish the failed messaged to the same dead letter queue that they would have gone to if the releaseExpression was met. (This is the approach I outlined below but I am worried about messages getting lost, if an error happens during publishing)
Investigate if there is away I could let the SimpleMessageListener know about the error that occurred and have it send the batch of messages that failed to a dead letter queue? I doubt this is possible since it seems the messages are already acked.
Don't set the SimpleMessageListenerContainer to AcknowledgeMode.AUTO and manually ack the messages when they get processed via the Service when the releaseExpression being met or the groupTimeOut happening. (This seems kinda of messy, since there can be 1..N message in the MessageGroup but wanted to see what others have done)
Ideally, I want to have a flow that will that will mimic the same flow when the releaseExpression has been met, so that the messages don't get lost.
Does anyone have recommendation on the best way to handle this scenario they have used in the past?
Thanks for any help and/or advice!
Here is my current configuration using Spring Integration DSL
#Bean
public SimpleMessageListenerContainer workListenerContainer() {
SimpleMessageListenerContainer container =
new SimpleMessageListenerContainer(rabbitConnectionFactory);
container.setQueues(worksQueue());
container.setConcurrentConsumers(4);
container.setDefaultRequeueRejected(false);
container.setTransactionManager(transactionManager);
container.setChannelTransacted(true);
container.setTxSize(10);
container.setAcknowledgeMode(AcknowledgeMode.AUTO);
return container;
}
#Bean
public AmqpInboundChannelAdapter inboundRabbitMessages() {
AmqpInboundChannelAdapter adapter = new AmqpInboundChannelAdapter(workListenerContainer());
return adapter;
}
I have defined a error channel and defined my own taskScheduler to use for the MessageStoreRepear
#Bean
public ThreadPoolTaskScheduler taskScheduler(){
ThreadPoolTaskScheduler ts = new ThreadPoolTaskScheduler();
MessagePublishingErrorHandler mpe = new MessagePublishingErrorHandler();
mpe.setDefaultErrorChannel(myErrorChannel());
ts.setErrorHandler(mpe);
return ts;
}
#Bean
public PollableChannel myErrorChannel() {
return new QueueChannel();
}
public IntegrationFlow aggregationFlow() {
return IntegrationFlows.from(inboundRabbitMessages())
.transform(Transformers.fromJson(SomeObject.class))
.aggregate(a->{
a.sendPartialResultOnExpiry(true);
a.groupTimeout(3000);
a.expireGroupsUponCompletion(true);
a.expireGroupsUponTimeout(true);
a.correlationExpression("T(Thread).currentThread().id");
a.releaseExpression("size() == 10");
a.transactional(true);
}
)
.handle("someService", "processMessages")
.get();
}
Here is my custom error flow
#Bean
public IntegrationFlow errorResponse() {
return IntegrationFlows.from("myErrorChannel")
.<MessagingException, Message<?>>transform(MessagingException::getFailedMessage,
e -> e.poller(p -> p.fixedDelay(100)))
.channel("myErrorChannelHandler")
.handle("myErrorHandler","handleFailedMessage")
.log()
.get();
}
Here is the custom error handler
#Component
public class MyErrorHandler {
#Autowired
BatchingRabbitTemplate batchingRabbitTemplate;
#ServiceActivator(inputChannel = "myErrorChannelHandler")
public void handleFailedMessage(Message<?> message) {
ArrayList<SomeObject> payload = (ArrayList<SomeObject>)message.getPayload();
payload.forEach(m->batchingRabbitTemplate.convertAndSend("some.dlq","#", m));
}
}
Here is the BatchingRabbitTemplate bean
#Bean
public BatchingRabbitTemplate batchingRabbitTemplate() {
ThreadPoolTaskScheduler scheduler = new ThreadPoolTaskScheduler();
scheduler.setPoolSize(5);
scheduler.initialize();
BatchingStrategy batchingStrategy = new SimpleBatchingStrategy(10, Integer.MAX_VALUE, 30000);
BatchingRabbitTemplate batchingRabbitTemplate = new BatchingRabbitTemplate(batchingStrategy, scheduler);
batchingRabbitTemplate.setConnectionFactory(rabbitConnectionFactory);
return batchingRabbitTemplate;
}
Update 1) to show custom MessageGroupProcessor:
public class CustomAggregtingMessageGroupProcessor extends AbstractAggregatingMessageGroupProcessor {
#Override
protected final Object aggregatePayloads(MessageGroup group, Map<String, Object> headers) {
return group;
}
}
Example Service:
#Slf4j
public class SomeService {
#ServiceActivator
public void processMessages(MessageGroup messageGroup) throws IOException {
Collection<Message<?>> messages = messageGroup.getMessages();
//Do business logic
//ack messages in the group
for (Message<?> m : messages) {
com.rabbitmq.client.Channel channel = (com.rabbitmq.client.Channel)
m.getHeaders().get("amqp_channel");
long deliveryTag = (long) m.getHeaders().get("amqp_deliveryTag");
log.debug(" deliveryTag = {}",deliveryTag);
log.debug("Channel = {}",channel);
channel.basicAck(deliveryTag, false);
}
}
}
Updated integrationFlow
public IntegrationFlow aggregationFlowWithCustomMessageProcessor() {
return IntegrationFlows.from(inboundRabbitMessages()).transform(Transformers.fromJson(SomeObject.class))
.aggregate(a -> {
a.sendPartialResultOnExpiry(true);
a.groupTimeout(3000);
a.expireGroupsUponCompletion(true);
a.expireGroupsUponTimeout(true);
a.correlationExpression("T(Thread).currentThread().id");
a.releaseExpression("size() == 10");
a.transactional(true);
a.outputProcessor(new CustomAggregtingMessageGroupProcessor());
}).handle("someService", "processMessages").get();
}
New ErrorHandler to do nack
public class MyErrorHandler {
#ServiceActivator(inputChannel = "myErrorChannelHandler")
public void handleFailedMessage(MessageGroup messageGroup) throws IOException {
if(messageGroup!=null) {
log.debug("Nack messages size = {}", messageGroup.getMessages().size());
Collection<Message<?>> messages = messageGroup.getMessages();
for (Message<?> m : messages) {
com.rabbitmq.client.Channel channel = (com.rabbitmq.client.Channel)
m.getHeaders().get("amqp_channel");
long deliveryTag = (long) m.getHeaders().get("amqp_deliveryTag");
log.debug("deliveryTag = {}",deliveryTag);
log.debug("channel = {}",channel);
channel.basicNack(deliveryTag, false, false);
}
}
}
}
Update 2 Added custom ReleaseStratgedy and change to aggegator
public class CustomMeasureGroupReleaseStratgedy implements ReleaseStrategy {
private static final int MAX_MESSAGE_COUNT = 10;
public boolean canRelease(MessageGroup messageGroup) {
return messageGroup.getMessages().size() >= MAX_MESSAGE_COUNT;
}
}
public IntegrationFlow aggregationFlowWithCustomMessageProcessorAndReleaseStratgedy() {
return IntegrationFlows.from(inboundRabbitMessages()).transform(Transformers.fromJson(SomeObject.class))
.aggregate(a -> {
a.sendPartialResultOnExpiry(true);
a.groupTimeout(3000);
a.expireGroupsUponCompletion(true);
a.expireGroupsUponTimeout(true);
a.correlationExpression("T(Thread).currentThread().id");
a.transactional(true);
a.releaseStrategy(new CustomMeasureGroupReleaseStratgedy());
a.outputProcessor(new CustomAggregtingMessageGroupProcessor());
}).handle("someService", "processMessages").get();
}
There are some flaws in your understanding.If you use AUTO, only the last message will be dead-lettered when an exception occurs. Messages successfully deposited in the group, before the release, will be ack'd immediately.
The only way to achieve what you want is to use MANUAL acks.
There is no way to "tell the listener container to send messages to the DLQ". The container never sends messages to the DLQ, it rejects a message and the broker sends it to the DLX/DLQ.
I have subscriber and publisher subscriber collect the messages and once it reaches specified number of message collected, it pass the control to publisher
I need to publish the message to different queue, Upon successful publish of each message I need to manually ack the subscribed queue to remove the message.
I have used return call back and confirm call back on rabbit template but callback functions are not called on successful publish of message.
I guess issue is both publisher and consumer use the same connection thats why callback is not called But I am not sure .
#SpringBootApplication
#ComponentScan(basePackages={"com.comp.dftp.scrubber.configuration","com.comp.dftp.scrubber.container",
"com.comp.dftp.scrubber.subscriber","com.comp.dftp.scrubber.publisher"})
#EnableAutoConfiguration(exclude=RabbitAutoConfiguration.class)
#EnableRabbit
public class DftpEppScrubberApplication {
private static final Logger LOGGER = LoggerFactory.getLogger(DftpEppScrubberApplication.class);
RabbitListenerEndpointRegistrar registrar;
AppConfig appConfig;
public static void main(String[] args) {
SpringApplication app = new SpringApplication(DftpEppScrubberApplication.class);
ConfigurableApplicationContext appContext = app.run(args);
}
#PreDestroy
public void graceFullExit() {
LOGGER.info("----------- Stopping Scrubber Container--------------");
if(registrar!= null && registrar.getEndpointRegistry() != null ) {
registrar.getEndpointRegistry().stop();
}
LOGGER.info("Container Stopped Sucessfully!");
/*
* If Application is not ready to exit safely we will weight for 30 sec
* and re-check again, This will be continued until the flag ReadyToGraccefullyExit
* is set true be process which has not completed Yet, Once publisher completed the process
* and reader ack back to publisher.
*/
while(!appConfig.isReadyToGraccefullyExit()) {
try{
Thread.sleep(30 * 1000);
appConfig.setReadyToGraccefullyExit(true); /* this is just for testing*/
} catch (InterruptedException e) {
LOGGER.error("Some error in gracefully exiting the application!", e);
}
}
LOGGER.info("###STOP FROM THE LIFECYCLE###");
}
}
```java
#Component
#Configuration
public class EPPQ2ListenerConfigurer implements RabbitListenerConfigurer{
public EPPQ2ListenerConfigurer(ConfigurableApplicationContext ctx) {
// TODO Auto-generated constructor stub
}
#Override
public void configureRabbitListeners(RabbitListenerEndpointRegistrar registrar) {
registrar.setMessageHandlerMethodFactory(messageHandlerMethodFactory());
}
#Bean
MessageHandlerMethodFactory messageHandlerMethodFactory() {
DefaultMessageHandlerMethodFactory messageHandlerMethodFactory = new DefaultMessageHandlerMethodFactory();
messageHandlerMethodFactory.setMessageConverter(consumerJackson2MessageConverter());
return messageHandlerMethodFactory;
}
#Bean
public MappingJackson2MessageConverter consumerJackson2MessageConverter() {
return new MappingJackson2MessageConverter();
}
}
```java
#Configuration
public class ListenerContainerFactory {
static final Logger logger = LoggerFactory.getLogger(ListenerContainerFactory.class);
#Autowired
RabbitMqConfig rabbitMqConfig;
#Autowired
EPPQ2Subscriber receiver;
public ListenerContainerFactory(ConfigurableApplicationContext ctx) {
printContainerStartMsg();
}
private void printContainerStartMsg() {
logger.info("----------- Scrubber Container Starts --------------");
}
#Bean
public CachingConnectionFactory subscriberConnectionFactory() {
CachingConnectionFactory subsCachingConnectionFactory = new CachingConnectionFactory(rabbitMqConfig.getSubscriberHosts(),
rabbitMqConfig.getSubscriberPort());
subsCachingConnectionFactory.setUsername(rabbitMqConfig.getSubscriberUsername());
subsCachingConnectionFactory.setPassword(rabbitMqConfig.getSubscriberPassword());
subsCachingConnectionFactory.setVirtualHost("hydra.services");
subsCachingConnectionFactory.setConnectionNameStrategy(f -> "subscriberConnection");
return subsCachingConnectionFactory;
}
#Bean
public SimpleRabbitListenerContainerFactory queueListenerContainer(
#Qualifier("subscriberConnectionFactory") CachingConnectionFactory subscriberConnectionFactory,
MessageListenerAdapter listenerAdapter) {
connectionFactory.setAddresses(rabbitMqConfig.getSubscriberHosts());
connectionFactory.setVirtualHost("hydra.services");
connectionFactory.setPort(rabbitMqConfig.getSubscriberPort());
connectionFactory.setUsername(rabbitMqConfig.getSubscriberUsername());
connectionFactory.setPassword(rabbitMqConfig.getSubscriberPassword());
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setAcknowledgeMode(AcknowledgeMode.MANUAL);
factory.setConnectionFactory(subscriberConnectionFactory);
factory.setErrorHandler(errorHandler());
return factory;
}
#Bean
MessageListenerAdapter listenerAdapter(EPPQ2Subscriber receiver) {
return new MessageListenerAdapter(receiver, "receiveMessage");
}
#Bean
public ErrorHandler errorHandler() {
return new ConditionalRejectingErrorHandler(fatalExceptionStrategy());
}
#Bean
public ScrubberFatalExceptionStrategy fatalExceptionStrategy() {
return new ScrubberFatalExceptionStrategy();
}
}
```java
#Component
public class EPPQ2Subscriber {
private static final Logger LOGGER = LoggerFactory.getLogger(EPPQ2Subscriber.class);
//#RabbitListener(queues = "#{queue.getName()}") #TODO I wann to use this in later point in time.. !
#Autowired
RabbitMqConfig rabbitMqConfig;
#Autowired
AppConfig appConfig;
#Autowired
EPPQ2PublisherImpl eppQ2Publisher;
List<Message> messageList = new ArrayList<Message>();
List<Long> deliveryTagList = new ArrayList<Long>();
/**
* Method is listener's receive message method , invoked when there is message ready to read
* #param message - Domain object of message encapsulated
* #param channel - rabitmq client channel
* #param messageId - #TODO Delete it later.
* #param messageProperties - amqp message properties contains message properties such as delivery tag etc..
*/
#RabbitListener(id="messageListener",queues = "#{rabbitMqConfig.getSubscriberQueueName()}",containerFactory="queueListenerContainer")
public void receiveMessage(Message message, Channel channel, #Header("id") String messageId,
MessageProperties messageProperties) {
LOGGER.info("Result:" + message.getClass() + ":" + message.toString());
if(messageList.size() < appConfig.getSubscriberChunkSize() ) {
messageList.add(message);
LOGGER.info("For Test Size:"+messageList.size()+ "chunk size : "+appConfig.getSubscriberChunkSize());
deliveryTagList.add(messageProperties.getDeliveryTag());
} else {
// call the service here to decrypt, read pan, call danger to scrub, encrypt pan and re-pack them in message again.
//after this branch messageList should have scrubbed and encrypted message objects ready to publish.
// Here is call for publish and ack messages..
eppQ2Publisher.sendMessages(messageList, channel, deliveryTagList);
}
}
}
````java
#Configuration
public class TopicConfiguration {
private static final Logger LOGGER = LoggerFactory.getLogger(TopicConfiguration.class);
#Autowired
RabbitMqConfig rabbitMqConfig;
#Autowired
EPPQ2Publisher eppQ2Publisher;
/**
* Caching connection factory
* #return CachingConnectionFactory
*/
#Bean
public CachingConnectionFactory cachingConnectionFactory() {
CachingConnectionFactory cachingConnectionFactory = new CachingConnectionFactory(rabbitMqConfig.getPublisherHosts(),
rabbitMqConfig.getPublisherPort());
cachingConnectionFactory.setUsername(rabbitMqConfig.getPublisherUsername());
cachingConnectionFactory.setPassword(rabbitMqConfig.getPublisherPassword());
cachingConnectionFactory.setVirtualHost("hydra.services");
cachingConnectionFactory.createConnection();
cachingConnectionFactory.setConnectionNameStrategy(f -> "publisherConnection");
return cachingConnectionFactory;
}
/**
* Bean RabbitTemplate
* #return RabbitTemplate
*/
#Bean
public RabbitTemplate template(
#Qualifier("cachingConnectionFactory") CachingConnectionFactory cachingConnectionFactory) {
final RabbitTemplate rabbitTemplate = new RabbitTemplate(cachingConnectionFactory);
rabbitTemplate.setMessageConverter(producerJackson2MessageConverter());
RetryTemplate retryTemplate = new RetryTemplate();
ExponentialBackOffPolicy backOffPolicy = new ExponentialBackOffPolicy();
backOffPolicy.setInitialInterval(500);
backOffPolicy.setMultiplier(10.0);
backOffPolicy.setMaxInterval(10000);
retryTemplate.setBackOffPolicy(backOffPolicy);
rabbitTemplate.setRetryTemplate(retryTemplate);
rabbitTemplate.setExchange(rabbitMqConfig.getPublisherTopic());
rabbitTemplate.setUsePublisherConnection(true);
rabbitTemplate.setMandatory(true);
rabbitTemplate.setConfirmCallback((correlation, ack, reason) -> {
if (correlation != null) {
LOGGER.info("Received " + (ack ? " ack " : " nack ") + "for correlation: " + correlation);
if (ack) {
// this is confirmation received..
// here is code to ack Q1. correlation.getId() and ack it !!
eppQ2Publisher.ackMessage(new Long(correlation.getId().toString()));
} else {
// no confirmation received and no need to do any thing for
// retry..
}
}
});
rabbitTemplate.setReturnCallback((message, replyCode, replyText, exchange, routingKey) -> {
LOGGER.error("Returned: " + message + "\nreplyCode: " + replyCode + "\nreplyText: " + replyText
+ "\nexchange/rk: " + exchange + "/" + routingKey);
});
return rabbitTemplate;
}
/**
* Bean Jackson2JsonMessageConverter
* #return Jackson2JsonMessageConverter
*/
#Bean
public Jackson2JsonMessageConverter producerJackson2MessageConverter() {
return new Jackson2JsonMessageConverter();
}
}
Here is some logs
----------
33morg.springframework.amqp.rabbit.listener.BlockingQueueConsumer[0;39m: Received message: (Body:'[B#1c63ef5(byte[2003])' MessageProperties [headers={}, contentLength=0, receivedDeliveryMode=NON_PERSISTENT, redelivered=true, receivedExchange=, receivedRoutingKey=hydra.Syphon.q1, deliveryTag=11, consumerTag=amq.ctag-VLCSea_a-FmED6I54TyM4w, consumerQueue=hydra.Syphon.q1])
SimpleAsyncTaskExecutor-1 org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter[0;39m: Processing [GenericMessage [payload=byte[2003], headers={amqp_receivedDeliveryMode=NON_PERSISTENT, amqp_receivedRoutingKey=hydra.Syphon.q1, amqp_deliveryTag=11, amqp_consumerQueue=hydra.Syphon.q1, amqp_redelivered=true, id=2a24ec32-2576-a208-4cbf-f36e6170ac83, amqp_consumerTag=amq.ctag-VLCSea_a-FmED6I54TyM4w, timestamp=1554262675707}]]
SimpleAsyncTaskExecutor-1 com.discover.dftp.scrubber.subscriber.EPPQ2Subscriber[0;39m: Result:class com.discover.dftp.scrubber.domain.Message:Header [header={RETRY_COUNT=0, PUBLISH_EVENT_TYPE=AUTH}, payLoad={MTI=400, MTI_REQUEST=400, PAN=6011000000000000, PROCCODE=00, PROCCODE_REQUEST=00, FROM_ACCOUNT=00, TO_ACCOUNT=00, TRANSACTION_AMOUNT=000000000100, TRANSMISSION_MMDDHHMMSS=0518202930, STAN=000001, LOCALTIME_HHMMSS=010054, LOCALDATE_YYMMDD=180522, MERCHANT_TYPE=5311, ACQUIRING_COUNTRY_CODE=840, POS_ENTRY_MODE=02, POS_PIN_ENTRY_CAPABILITIES=0, FUNCTION_CODE=400, MESSAGE_REASON_CODE=06, ACQUIRING_ID_CODE=000000, FORWARDING_ID_CODE=000000, RETRIEVAL_REFERENCE_NUMBER=1646N472D597, APPROVAL_CODE=12345R, RESPONSE_CODE=00, MERCHANT_NUMBER=601100000000596, CARD_ACCEPTOR_NAME=Discover Acq Simulator, CARD_ACCEPTOR_CITY=Riverwoods, CARD_ACCEPTOR_STATE=IL, CARD_ACCEPTOR_COUNTRY=840, CARD_ACCEPTOR_COUNTRY_3NUMERIC=840, NRID=123456789012345, TRANSACTION_CURRENCY_CODE=840, POS_TERMINAL_ATTENDANCE_INDICATOR=0, POS_PARTIAL_APPROVAL_INDICATOR=0, POS_TERMINAL_LOCATION_INDICATOR=0, POS_TRANSACTION_STATUS_INDICATOR=0, POS_ECOMMERCE_TRAN_INDICATOR=0, POS_TYPE_OF_TERMINAL_DEVICE=0, POS_CARD_PRESENCE_INDICATOR=0, POS_CARD_CAPTURE_CAPABILITIES_INDICATOR=1, POS_TRANSACTION_SECURITY_INDICATOR=0, POS_CARD_DATA_TERMINAL_INPUT_CAPABILITY_INDICATOR=2, POS_CARDHOLDER_PRESENCE_INDICATOR=0, DFS_POS_DATA=0000010000200, GEODATA_STREET_ADDRESS=2500 LAKE COOK ROAD , GEODATA_POSTAL_CODE=600150000, GEODATA_COUNTY_CODE=840, GEODATA_STORE_NUMBER=10001, GEODATA_MALL_NAME=DISCOVER FINANCIAL SR, VERSION_INDICATOR=03141, REVERSAL_FLAG=Y, ISS_REFERENCE_ID=72967956, ISS_PROCESSOR_REFERENCE_ID=123459875}]
SimpleAsyncTaskExecutor-1 org.springframework.retry.support.RetryTemplate[0;39m: Retry: count=0
SimpleAsyncTaskExecutor-1 org.springframework.amqp.rabbit.core.RabbitTemplate[0;39m: Executing callback RabbitTemplate$$Lambda$268/19900766 on RabbitMQ Channel: Cached Rabbit Channel: AMQChannel(amqp://dftp_publisher#10.15.190.18:5672/hydra.services,1), conn: Proxy#10fa93b Shared Rabbit Connection: SimpleConnection#6329d2 [delegate=amqp://dftp_publisher#10.15.190.18:5672/hydra.services, localPort= 62375]
SimpleAsyncTaskExecutor-1 org.springframework.amqp.rabbit.core.RabbitTemplate[0;39m: Publishing message (Body:'{"HEADER":{"RETRY_COUNT":0,"PUBLISH_EVENT_TYPE":"AUTH"},"PAYLOAD":{"MTI":"400","MTI_REQUEST":"400","PAN":"6011000000000000","PROCCODE":"00","PROCCODE_REQUEST":"00","FROM_ACCOUNT":"00","TO_ACCOUNT":"00","TRANSACTION_AMOUNT":"000000000100","TRANSMISSION_MMDDHHMMSS":"0518202930","STAN":"000001","LOCALTIME_HHMMSS":"010054","LOCALDATE_YYMMDD":"180522","MERCHANT_TYPE":"5311","ACQUIRING_COUNTRY_CODE":"840","POS_ENTRY_MODE":"02","POS_PIN_ENTRY_CAPABILITIES":"0","FUNCTION_CODE":"400","MESSAGE_REASON_CODE":"06","ACQUIRING_ID_CODE":"000000","FORWARDING_ID_CODE":"000000","RETRIEVAL_REFERENCE_NUMBER":"1646N472D597","APPROVAL_CODE":"12345R","RESPONSE_CODE":"00","MERCHANT_NUMBER":"601100000000596","CARD_ACCEPTOR_NAME":"Discover Acq Simulator","CARD_ACCEPTOR_CITY":"Riverwoods","CARD_ACCEPTOR_STATE":"IL","CARD_ACCEPTOR_COUNTRY":"840","CARD_ACCEPTOR_COUNTRY_3NUMERIC":"840","NRID":"123456789012345","TRANSACTION_CURRENCY_CODE":"840","POS_TERMINAL_ATTENDANCE_INDICATOR":"0","POS_PARTIAL_APPROVAL_INDICATOR":"0","POS_TERMINAL_LOCATION_INDICATOR":"0","POS_TRANSACTION_STATUS_INDICATOR":"0","POS_ECOMMERCE_TRAN_INDICATOR":"0","POS_TYPE_OF_TERMINAL_DEVICE":"0","POS_CARD_PRESENCE_INDICATOR":"0","POS_CARD_CAPTURE_CAPABILITIES_INDICATOR":"1","POS_TRANSACTION_SECURITY_INDICATOR":"0","POS_CARD_DATA_TERMINAL_INPUT_CAPABILITY_INDICATOR":"2","POS_CARDHOLDER_PRESENCE_INDICATOR":"0","DFS_POS_DATA":"0000010000200","GEODATA_STREET_ADDRESS":"2500 LAKE COOK ROAD ","GEODATA_POSTAL_CODE":"600150000","GEODATA_COUNTY_CODE":"840","GEODATA_STORE_NUMBER":"10001","GEODATA_MALL_NAME":"DISCOVER FINANCIAL SR","VERSION_INDICATOR":"03141","REVERSAL_FLAG":"Y","ISS_REFERENCE_ID":"72967956","ISS_PROCESSOR_REFERENCE_ID":"123459875"}}' MessageProperties [headers={__TypeId__=com.discover.dftp.scrubber.domain.Message}, contentType=application/json, contentEncoding=UTF-8, contentLength=1705, deliveryMode=PERSISTENT, priority=0, deliveryTag=0])on exchange [hydra.test.exc], routingKey = [800]
SimpleAsyncTaskExecutor-1 org.springframework.retry.support.RetryTemplate[0;39m: Retry: count=0
SimpleAsyncTaskExecutor-1 org.springframework.amqp.rabbit.core.RabbitTemplate[0;39m: Executing callback RabbitTemplate$$Lambda$268/19900766 on RabbitMQ Channel: Cached Rabbit Channel: AMQChannel(amqp://dftp_publisher#10.15.190.18:5672/hydra.services,1), conn: Proxy#10fa93b Shared Rabbit Connection: SimpleConnection#6329d2 [delegate=amqp://dftp_publisher#10.15.190.18:5672/hydra.services, localPort= 62375]
SimpleAsyncTaskExecutor-1 org.springframework.amqp.rabbit.core.RabbitTemplate[0;39m: Publishing message (Body:'{"HEADER":{"RETRY_COUNT":0,"PUBLISH_EVENT_TYPE":"AUTH"},"PAYLOAD":{"MTI":"400","MTI_REQUEST":"400","PAN":"6011000000000000","PROCCODE":"00","PROCCODE_REQUEST":"00","FROM_ACCOUNT":"00","TO_ACCOUNT":"00","TRANSACTION_AMOUNT":"000000000100","TRANSMISSION_MMDDHHMMSS":"0518202930","STAN":"000001","LOCALTIME_HHMMSS":"010054","LOCALDATE_YYMMDD":"180522","MERCHANT_TYPE":"5311","ACQUIRING_COUNTRY_CODE":"840","POS_ENTRY_MODE":"02","POS_PIN_ENTRY_CAPABILITIES":"0","FUNCTION_CODE":"400","MESSAGE_REASON_CODE":"06","ACQUIRING_ID_CODE":"000000","FORWARDING_ID_CODE":"000000","RETRIEVAL_REFERENCE_NUMBER":"1646N472D597","APPROVAL_CODE":"12345R","RESPONSE_CODE":"00","MERCHANT_NUMBER":"601100000000596","CARD_ACCEPTOR_NAME":"Discover Acq Simulator","CARD_ACCEPTOR_CITY":"Riverwoods","CARD_ACCEPTOR_STATE":"IL","CARD_ACCEPTOR_COUNTRY":"840","CARD_ACCEPTOR_COUNTRY_3NUMERIC":"840","NRID":"123456789012345","TRANSACTION_CURRENCY_CODE":"840","POS_TERMINAL_ATTENDANCE_INDICATOR":"0","POS_PARTIAL_APPROVAL_INDICATOR":"0","POS_TERMINAL_LOCATION_INDICATOR":"0","POS_TRANSACTION_STATUS_INDICATOR":"0","POS_ECOMMERCE_TRAN_INDICATOR":"0","POS_TYPE_OF_TERMINAL_DEVICE":"0","POS_CARD_PRESENCE_INDICATOR":"0","POS_CARD_CAPTURE_CAPABILITIES_INDICATOR":"1","POS_TRANSACTION_SECURITY_INDICATOR":"0","POS_CARD_DATA_TERMINAL_INPUT_CAPABILITY_INDICATOR":"2","POS_CARDHOLDER_PRESENCE_INDICATOR":"0","DFS_POS_DATA":"0000010000200","GEODATA_STREET_ADDRESS":"2500 LAKE COOK ROAD ","GEODATA_POSTAL_CODE":"600150000","GEODATA_COUNTY_CODE":"840","GEODATA_STORE_NUMBER":"10001","GEODATA_MALL_NAME":"DISCOVER FINANCIAL SR","VERSION_INDICATOR":"03141","REVERSAL_FLAG":"Y","ISS_REFERENCE_ID":"72967956","ISS_PROCESSOR_REFERENCE_ID":"123459875"}}' MessageProperties [headers={__TypeId__=com.discover.dftp.scrubber.domain.Message}, contentType=application/json, contentEncoding=UTF-8, contentLength=1705, deliveryMode=PERSISTENT, priority=0, deliveryTag=0])on exchange [hydra.test.exc], routingKey = [800]
You need to enable returns and confirms in the CachingConnectionFactory configuration. See the documentation.
This feature requires a CachingConnectionFactory that has its publisherReturns property set to true (see Publisher Confirms and Returns).
...
For publisher confirms (also known as publisher acknowledgements), the template requires a CachingConnectionFactory that has its publisherConfirms property set to true.
Follow #Gary Russell's answer, I added configurations below in my codes, then ConfirmCallback starts to work. I use SpringBoot 2.5.2 and RabbitMQ 3.8.18
cachingConnectionFactory.setPublisherReturns(true);
cachingConnectionFactory.setPublisherConfirmType(CachingConnectionFactory.ConfirmType.CORRELATED);
Rabbitmq-Batch-Rabbitmq-Publish-Subscribe