Spring Integration ( Retry Strategy) - spring

I want to create a simple IntegrationFlow with Spring integration, and I am having difficulties.
I want to create an integration flow that takes messages from a queue in Rabbit Mq and posts the messages to an endpoint Rest.
The problem I am dealing with is that when a request fails, it continues to retry endlessly, how can I implement a retry strategy in this code?
For example, I want 3 retrys ,the first retry after 1 second, the second after 5 seconds and the third after 1 minute.
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_JSON);
RestTemplate restTemplate = new RestTemplate();
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer(connectionFactory);
container.setQueueNames(BOUTIQUE_QUEUE_NAME);
container.setAcknowledgeMode(AcknowledgeMode.AUTO);
return IntegrationFlows.from(Amqp.inboundAdapter(container)) /* Get Message from RabbitMQ */
.handle(msg ->
{
String msgString = new String((byte[]) msg.getPayload(), StandardCharsets.UTF_8);
HttpEntity<String> requestBody = new HttpEntity<String>(msgString, headers);
restTemplate.postForObject(ENDPOINT_LOCAL_URL, requestBody, String.class);
System.out.println(msgString);
})
.get();
}

Add a retry interceptor to the listener container's advice chain. See https://docs.spring.io/spring-amqp/docs/2.2.10.RELEASE/reference/html/#retry and https://docs.spring.io/spring-amqp/docs/2.2.10.RELEASE/reference/html/#async-listeners
EDIT
#SpringBootApplication
public class So63596805Application {
private static final Logger LOG = LoggerFactory.getLogger(So63596805Application.class);
public static void main(String[] args) {
SpringApplication.run(So63596805Application.class, args);
}
#Bean
IntegrationFlow flow(SimpleRabbitListenerContainerFactory factory, RabbitTemplate template) {
SimpleMessageListenerContainer container = factory.createListenerContainer();
container.setQueueNames("foo");
container.setAdviceChain(RetryInterceptorBuilder.stateless()
.maxAttempts(5)
.backOffOptions(1000, 2.0, 10000)
.recoverer((msg, cause) -> LOG.info("Retries exhausted for " + msg))
.build());
return IntegrationFlows.from(Amqp.inboundAdapter(container))
.handle(msg -> {
LOG.info(msg.getPayload().toString());
throw new RuntimeException("test");
})
.get();
}
}
This uses an exponential back off policy.
If you use
.maxAttempts(4)
.backOffOptions(1000, 5.0, 60000)
You will get 3 retries after 1, 5 and 25 seconds.
1000, 8.0, 60000 will give you 1, 8 and 60 seconds.
If you must have your specs (1, 5, 60), you will need a custom BackOffPolicy.

Related

Spring Retry with RetryTemplate in Spring Boot, Java8

I am using Spring Boot 2.1.14.RELEASE, Java8, Spring Boot.
I have a client from which I have to access another rest service.
I need to retry an Http404 and HTTP500 2 times whereas not retry any other exceptions.
I am using RestTemplate to invoke the rest service like this:
restTemplate.postForEntity(restUrl, requestEntity, String.class);
I looked into using Retryable as well as RetryTemplate and implemented the retry functionality using RetryTemplate.
I have implemented this in 2 ways:
OPTION1:
The RetryTemplate bean is:
#Bean
public RetryTemplate retryTemplate() {
RetryTemplate retryTemplate = new RetryTemplate();
FixedBackOffPolicy fixedBackOffPolicy = new FixedBackOffPolicy();
fixedBackOffPolicy.setBackOffPeriod(retryProperties.getDelayForCall());
retryTemplate.setBackOffPolicy(fixedBackOffPolicy);
retryTemplate.setRetryPolicy(exceptionClassifierRetryPolicy);
return retryTemplate;
}
ClassifierRetryPolicy is:
#Component
public class ExceptionClassifierRetryPolicy1 extends ExceptionClassifierRetryPolicy {
#Inject
private RetryProperties retryProperties;
public ExceptionClassifierRetryPolicy1(){
final SimpleRetryPolicy simpleRetryPolicy = new SimpleRetryPolicy();
simpleRetryPolicy.setMaxAttempts(2);
this.setExceptionClassifier(new Classifier<Throwable, RetryPolicy>() {
#Override
public RetryPolicy classify(Throwable classifiable) {
if (classifiable instanceof HttpServerErrorException) {
// For specifically 500
if (((HttpServerErrorException) classifiable).getStatusCode() == HttpStatus.INTERNAL_SERVER_ERROR) {
return simpleRetryPolicy;
}
return new NeverRetryPolicy();
}
else if (classifiable instanceof HttpClientErrorException) {
// For specifically 404
if (((HttpClientErrorException) classifiable).getStatusCode() == HttpStatus.NOT_FOUND) {
return simpleRetryPolicy;
}
return new NeverRetryPolicy();
}
return new NeverRetryPolicy();
}
});
}
}
In my client class, I am using retryTemplate like this:
public void postToRestService(...,...){
...
retryTemplate.execute(context -> {
logger.info("Processing request...");
responseEntity[0] = restTemplate.postForEntity(restURL, requestEntity, String.class);
return null;
}, context -> recoveryCallback(context));
...
}
The rest service being invoked is throwing HTTP404 on every request.
My expectation is: The client should submit one request, receive HTTP404, and perform 2 retries. So a total of 3 requests submitted to rest service before invoking recovery callback method.
My observation is: The client is submitting 2 requests to rest service.
Above observation makes sense from what I have read about RetryTemplate.
So the questions are:
Is the above implementation of retryTemplate correct? If not, how to implement and invoke it? Another option that I tried implementing (but didn't get any far) was using a RetryListenerSupport on the client method and invoking the retryTemplate inside the onError method.
Are we supposed to bump up the retry count by 1 to achieve what is desired? I have tried this and it gets me what I need but the RetryTemplate isn't created with this purpose in mind.
OPTION2: Code implementing option mentioned in #1 above:
Client method:
#Retryable(listeners = "RestClientListener")
public void postToRestService(...,...){
...
responseEntity[0] = restTemplate.postForEntity(restURL, requestEntity, String.class);
...
}
Listener:
public class RestClientListener extends RetryListenerSupport {
private static final Logger logger = LoggerFactory.getLogger(RestClientListener.class);
#Inject
RestTemplate restTemplate;
#Inject
RetryTemplate retryTemplate;
public <T, E extends Throwable> void onError(RetryContext context, RetryCallback<T, E> callback, Throwable throwable) {
logger.info("Retrying count for RestClientListener "+context.getRetryCount());
...
final ResponseEntity<String>[] responseEntity = new ResponseEntity[]{null};
if( context.getLastThrowable().getCause() != null &&
(context.getLastThrowable().getCause() instanceof RestClientResponseException &&
((RestClientResponseException) context.getLastThrowable().getCause()).getRawStatusCode() == HttpStatus.NOT_FOUND.value()))
{
logger.info("Retrying now: ", context.getLastThrowable().toString());
retryTemplate.execute(context2 -> {
logger.info("Processing request...: {}", context2);
responseEntity[0] = restTemplate.postForEntity(restURL, requestEntity, String.class);
return responseEntity;
}, context2 -> recoveryCallback(context2));
}
else {
// Only retry for the above if condition
context.setExhaustedOnly();
}
}
}
The problem with this approach is that I cannot find a way to share objects between my client and clientListener classes. These objects are required in order to create requestEntity and header objects. How can this be achieved?
simpleRetryPolicy.setMaxAttempts(2);
Means 2 attempts total, not 2 retries.

spring integrationflow does not set the message listener on receivercontainer

We use spring-integration (4.3.12) together with spring-amqp(1.7.4) to send and receive messages between micro services.
To keep out the integration/amqp configuration stuff out of the micro services, we want to use a library containing integration/amqp factories for creation of the objects required.
What I expect:
I create an IntegrationFlow instance with a messageHandler/messageHandler method (see below for the code) and a SimpleMessageHandlerContainer. When I send a message to the bound exchange then IO expect the messageHandler gets called with the message.
What do I get:
An exception: "MessageDispatchingException: Dispatcher has no subscribers"
If I use the MessageListenerContainer directly (set the messageHandler direct at the container) then I get the message as expected.
I guess, the problem lies within the programmatically initialization of the integrationFlow, but I cant find any information what I´m doing wrong.
Can anybody give me a hint?
Thanx
Now the code used:
public IntegrationFlow createMessageNotifierIntegrationFlow(//
String brokerNameSpace, String messageHandlerNameSpace, //
Object messageHandler, String methodName) {
ConnectionFactory cf = createConnectionFactory(brokerNameSpace);
AmqpAdmin amqpAdmin = createAmqpAdmin(brokerNameSpace, cf);
Inbound inbound = createInbound(messageHandlerNameSpace);
Queue queue = createQueue(messageHandlerNameSpace, amqpAdmin, inbound);
MessageNotifierIntegrationFlowBuilder builder = MessageNotifierIntegrationFlowBuilder
.newBuilder(messageHandlerNameSpace, this);
IntegrationFlow integrationFlow = builder//
.withConnectionFactory(cf)//
.withMessageHandler(messageHandler)//
.withMessageHandlerMethod(methodName)//
.withFlowExceptionHandler(new FlowExceptionHandler())//
.withInbound(inbound)//
.withAmqpAdmin(amqpAdmin)//
.withInboundQueue(queue)//
.build();
String beanName = brokerNameSpace + "-" + messageHandlerNameSpace + "-" + inbound.getQueue();
return integrationFlow;
}
public IntegrationFlow build() {
LOGGER.info("creating IntegrationFlow for {}", inbound.getQueue());
validateBuilder();
SimpleMessageListenerContainer receiverContainer = amqpObjectFactory.createMessageListenerContainer(//
inbound, connectionFactory, //
flowExceptionHandler, inboundQueue, messageHandlerNameSpace);
final AmqpInboundChannelAdapterSpec adapter = (AmqpInboundChannelAdapterSpec) Amqp.inboundAdapter(receiverContainer);
StandardIntegrationFlow flow = IntegrationFlows //
.from(adapter) //
.log("receiveData")//
.transform(TO_STRING_TRANSFORMER) //
.handle(messageHandler, messageHandlerMethod) //
.log("to message handler").get();
// flow.start() maybe later?
flow.start();
return flow;
}
public SimpleMessageListenerContainer createMessageListenerContainer(//
final Inbound inbound, //
final ConnectionFactory connectionFactory, //
final FlowExceptionHandler flowExceptionHandler, //
final Queue inboundQueue, String messageHandlerNameSpace) {
final String beanName = messageHandlerNameSpace + "-container-" + inbound.getQueue();
SimpleMessageListenerContainer container = null;
container = new SimpleMessageListenerContainer(connectionFactory);
container.setMaxConcurrentConsumers(inbound.getMaxconsumers());
container.setConcurrentConsumers(inbound.getMinconsumers());
container.setStartConsumerMinInterval(inbound.getMininterval());
container.addQueues(inboundQueue);
container.setAcknowledgeMode(inbound.getAckmode());
container.setDefaultRequeueRejected(inbound.getRequeueRejected());
container.setErrorHandler(flowExceptionHandler);
container.setRecoveryInterval(RECOVERY_INTERVAL);
container.setAutoStartup(false);
return container;
}

EmbeddedKafka how to check received messages in unit test

I created a spring boot application that sends messages to a Kafka topic. I am using spring spring-integration-kafka:
A KafkaProducerMessageHandler<String,String> is subscribed to a channel (SubscribableChannel) and pushes all messages received to one topic.
The application works fine. I see messages arriving in Kafka via console consumer (local kafka).
I also create an Integrationtest that uses KafkaEmbedded. I am checking the expected messages by subscribing to the channel within the test - all is fine.
But i want the test to check also the messages put into kafka. Sadly Kafka's JavaDoc is not the best. What i tried so far is:
#ClassRule
public static KafkaEmbedded kafkaEmbedded = new KafkaEmbedded(1, true, "myTopic");
//...
#Before
public void init() throws Exception {
mockConsumer = new MockConsumer<>( OffsetResetStrategy.EARLIEST );
kafkaEmbedded.consumeFromAnEmbeddedTopic( mockConsumer,"sikom" );
}
//...
#Test
public void endToEnd() throws Exception {
// ...
ConsumerRecords<String, String> records = mockConsumer.poll( 10000 );
StreamSupport.stream(records.spliterator(), false).forEach( record -> log.debug( "record: " + record.value() ) );
}
The problem is that i don't see any records. I am not sure if my KafkaEmbedded setup is correct.
But messages are receive by the channel.
This works for me. Give it a try
#RunWith(SpringRunner.class)
#SpringBootTest
public class KafkaEmbeddedTest {
private static String SENDER_TOPIC = "testTopic";
#ClassRule
// By default it creates two partitions.
public static KafkaEmbedded embeddedKafka = new KafkaEmbedded(1, true, SENDER_TOPIC);
#Test
public void testSend() throws InterruptedException, ExecutionException {
Map<String, Object> senderProps = KafkaTestUtils.producerProps(embeddedKafka);
//If you wish to send it to partitions other than 0 and 1,
//then you need to specify number of paritions in the declaration
KafkaProducer<Integer, String> producer = new KafkaProducer<>(senderProps);
producer.send(new ProducerRecord<>(SENDER_TOPIC, 0, 0, "message00")).get();
producer.send(new ProducerRecord<>(SENDER_TOPIC, 0, 1, "message01")).get();
producer.send(new ProducerRecord<>(SENDER_TOPIC, 1, 0, "message10")).get();
Map<String, Object> consumerProps = KafkaTestUtils.consumerProps("sampleRawConsumer", "false", embeddedKafka);
// Make sure you set the offset as earliest, because by the
// time consumer starts, producer might have sent all messages
consumerProps.put("auto.offset.reset", "earliest");
final List<String> receivedMessages = Lists.newArrayList();
final CountDownLatch latch = new CountDownLatch(3);
ExecutorService executorService = Executors.newSingleThreadExecutor();
executorService.execute(() -> {
KafkaConsumer<Integer, String> kafkaConsumer = new KafkaConsumer<>(consumerProps);
kafkaConsumer.subscribe(Collections.singletonList(SENDER_TOPIC));
try {
while (true) {
ConsumerRecords<Integer, String> records = kafkaConsumer.poll(100);
records.iterator().forEachRemaining(record -> {
receivedMessages.add(record.value());
latch.countDown();
});
}
} finally {
kafkaConsumer.close();
}
});
latch.await(10, TimeUnit.SECONDS);
assertTrue(receivedMessages.containsAll(Arrays.asList("message00", "message01", "message10")));
}
}
I am using countdown latch because Producer.Send(..) is an async operation. So what i am doing here is waiting in an infinite loop polling kafka every 100 milliseconds, if there is new record and if so adding it to a List for future assertions and then reducing the countdown. And I will wait for 10 seconds in total just to be sure.
You can as well use a simple loop and then exit after a few minutes.(If you don't wish to use CountdownLatch and ExecutorService stuff)

rabbitmq + spring boot + consumers idle for 15 mins after processing a single message

We use spring boot (1.5) and integration to connect to Rabbit MQ as a service on PCF
Properties set
- concurrency = 15
- maxConcurrency = 25
- default prefetch and txSize.
How the message are processed.
The queue received 26 messages within less than 1 sec.
15 consumers started and completed with the next minute.
Average time taken by each consumer is 30 secs approx.
Few consumers took less than 15 secs.
As each consumer acknowledged, new messages(16th, 17th message) from ready becomes unacknowledged on rabbit MQ.
So far as expected, since i assume new messages are being proccessed as old messages are acknowledged.
But as new messages become unack'ed on rabbitMQ, they are not processed by any consumers. Consumers are stuck and idle.
This continues and all ready state messages become unacknowledged and stay there without any activity.
They all resume after approx 15mins.
I see this behavior always.
Any guidance ?
Here is the code that does wiring of rabbit MQ
The MessageHandler in between has code that connects to big data and does some querying.
#Configuration
#EnableRabbit
public class RabbitMQ {
protected final Logger logger = LoggerFactory.getLogger(this.getClass());
#Value("${spring.rabbitmq.listener.concurrency:3}")
private int concurrentConsumers;
#Value("${spring.rabbitmq.listener.maxConcurrency:5}")
private int maxConcurrentConsumers;
#Value("${spring.rabbitmq.template.retry.max-attempts:3}")
private int maxAttempts;
#Value("${spring.rabbitmq.template.retry.initial-interval:2000}")
private int initialInterval;
#Value("${spring.rabbitmq.template.retry.multiplier:3}")
private int multiplier;
#Value("${spring.rabbitmq.template.retry.max-interval:10000}")
private int maxInterval;
#Autowired
private BotQueues botQueues;
#Autowired
private RetryFailLogger retryRecoverer;
#Bean
public StatefulRetryOperationsInterceptor statefulRetryOperationsInterceptor() {
return RetryInterceptorBuilder.stateful()
.backOffOptions(initialInterval, multiplier, maxInterval) // initialInterval, multiplier, maxInterval
.maxAttempts(maxAttempts)
.messageKeyGenerator(message -> message.getMessageProperties().getMessageId())
.recoverer(retryRecoverer)
.build();
}
#Bean
public Jackson2JsonMessageConverter defaultJsonMessageConverter() {
Jackson2JsonMessageConverter jsonConverter = new Jackson2JsonMessageConverter();
DefaultClassMapper classMapper = new DefaultClassMapper();
classMapper.setDefaultType(JobDetailInfo.class);
jsonConverter.setClassMapper(classMapper);
return jsonConverter;
}
#Bean
public SimpleMessageListenerContainer container(ConnectionFactory connectionFactory)
throws BeansException, ClassNotFoundException {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer(connectionFactory);
container.setQueues(botQueues.inputQueues());
container.setConcurrentConsumers(concurrentConsumers);
container.setMaxConcurrentConsumers(maxConcurrentConsumers);
container.setChannelTransacted(true);
container.setAdviceChain(new Advice[] {statefulRetryOperationsInterceptor()});
return container;
}
#Bean
public AmqpInboundChannelAdapter inbound(SimpleMessageListenerContainer container,
#Qualifier("commonInputChannel") MessageChannel amqpInputChannel) {
AmqpInboundChannelAdapter adapter = new AmqpInboundChannelAdapter(container);
adapter.setMessageConverter(defaultJsonMessageConverter());
adapter.setOutputChannel(amqpInputChannel);
return adapter;
}
#Bean
#ServiceActivator(inputChannel = "commonInputChannel", outputChannel = "commonOutputChannel")
public MessageHandler messageHandler() {
return new MessageHandler();
}
#Router(inputChannel = "commonOutputChannel")
public String resolveJobChannel(JobDetailInfo jobDetailInfo) {
String returnChannel = "";
if (jobDetailInfo != null) {
switch (jobDetailInfo.getJobStatus()) {
case Scheduled:
returnChannel = "collectorChannel";
break;
default:
break;
}
}
return returnChannel;
}
}
Thread Dump:
{"threadName":"container-14","threadId":40,"blockedTime":-1,"blockedCount":45,"waitedTime":-1,"waitedCount":66654,"lockName":"java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject#1b4302a9","lockOwnerId":-1,"lockOwnerName":null,"inNative":false,"suspended":false,"threadState":"TIMED_WAITING","stackTrace":[{"methodName":"park","fileName":"Unsafe.java","lineNumber":-2,"className":"sun.misc.Unsafe","nativeMethod":true},{"methodName":"parkNanos","fileName":"LockSupport.java","lineNumber":215,"className":"java.util.concurrent.locks.LockSupport","nativeMethod":false},{"methodName":"awaitNanos","fileName":"AbstractQueuedSynchronizer.java","lineNumber":2078,"className":"java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject","nativeMethod":false},{"methodName":"poll","fileName":"LinkedBlockingQueue.java","lineNumber":467,"className":"java.util.concurrent.LinkedBlockingQueue","nativeMethod":false},{"methodName":"nextMessage","fileName":"BlockingQueueConsumer.java","lineNumber":461,"className":"org.springframework.amqp.rabbit.listener.BlockingQueueConsumer","nativeMethod":false},{"methodName":"doReceiveAndExecute","fileName":"SimpleMessageListenerContainer.java","lineNumber":1214,"className":"org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer","nativeMethod":false},{"methodName":"receiveAndExecute","fileName":"SimpleMessageListenerContainer.java","lineNumber":1189,"className":"org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer","nativeMethod":false},{"methodName":"access$1500","fileName":"SimpleMessageListenerContainer.java","lineNumber":97,"className":"org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer","nativeMethod":false},{"methodName":"run","fileName":"SimpleMessageListenerContainer.java","lineNumber":1421,"className":"org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer","nativeMethod":false},{"methodName":"run","fileName":"Thread.java","lineNumber":748,"className":"java.lang.Thread","nativeMethod":false}],"lockedMonitors":[],"lockedSynchronizers":[],"lockInfo":{"className":"java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject","identityHashCode":457376425}},
{"threadName":"container-13","threadId":39,"blockedTime":-1,"blockedCount":34,"waitedTime":-1,"waitedCount":66160,"lockName":null,"lockOwnerId":-1,"lockOwnerName":null,"inNative":true,"suspended":false,"threadState":"RUNNABLE","stackTrace":[{"methodName":"socketRead0","fileName":"SocketInputStream.java","lineNumber":-2,"className":"java.net.SocketInputStream","nativeMethod":true},{"methodName":"socketRead","fileName":"SocketInputStream.java","lineNumber":116,"className":"java.net.SocketInputStream","nativeMethod":false},{"methodName":"read","fileName":"SocketInputStream.java","lineNumber":171,"className":"java.net.SocketInputStream","nativeMethod":false},{"methodName":"read","fileName":"SocketInputStream.java","lineNumber":141,"className":"java.net.SocketInputStream","nativeMethod":false},{"methodName":"readMore","fileName":"VisibleBufferedInputStream.java","lineNumber":140,"className":"org.postgresql.core.VisibleBufferedInputStream","nativeMethod":false},{"methodName":"ensureBytes","fileName":"VisibleBufferedInputStream.java","lineNumber":109,"className":"org.postgresql.core.VisibleBufferedInputStream","nativeMethod":false},{"methodName":"read","fileName":"VisibleBufferedInputStream.java","lineNumber":67,"className":"org.postgresql.core.VisibleBufferedInputStream","nativeMethod":false},{"methodName":"receiveChar","fileName":"PGStream.java","lineNumber":280,"className":"org.postgresql.core.PGStream","nativeMethod":false},{"methodName":"processResults","fileName":"QueryExecutorImpl.java","lineNumber":1916,"className":"org.postgresql.core.v3.QueryExecutorImpl","nativeMethod":false},{"methodName":"execute","fileName":"QueryExecutorImpl.java","lineNumber":288,"className":"org.postgresql.core.v3.QueryExecutorImpl","nativeMethod":false},{"methodName":"executeInternal","fileName":"PgStatement.java","lineNumber":430,"className":"org.postgresql.jdbc.PgStatement","nativeMethod":false},{"methodName":"execute","fileName":"PgStatement.java","lineNumber":356,"className":"org.postgresql.jdbc.PgStatement","nativeMethod":false},{"methodName":"executeWithFlags","fileName":"PgStatement.java","lineNumber":303,"className":"org.postgresql.jdbc.PgStatement","nativeMethod":false},{"methodName":"executeCachedSql","fileName":"PgStatement.java","lineNumber":289,"className":"org.postgresql.jdbc.PgStatement","nativeMethod":false},{"methodName":"executeWithFlags","fileName":"PgStatement.java","lineNumber":266,"className":"org.postgresql.jdbc.PgStatement","nativeMethod":false},{"methodName":"execute","fileName":"PgStatement.java","lineNumber":262,"className":"org.postgresql.jdbc.PgStatement","nativeMethod":false},{"methodName":"validate","fileName":"PooledConnection.java","lineNumber":532,"className":"org.apache.tomcat.jdbc.pool.PooledConnection","nativeMethod":false},{"methodName":"validate","fileName":"PooledConnection.java","lineNumber":443,"className":"org.apache.tomcat.jdbc.pool.PooledConnection","nativeMethod":false},{"methodName":"borrowConnection","fileName":"ConnectionPool.java","lineNumber":802,"className":"org.apache.tomcat.jdbc.pool.ConnectionPool","nativeMethod":false},{"methodName":"borrowConnection","fileName":"ConnectionPool.java","lineNumber":651,"className":"org.apache.tomcat.jdbc.pool.ConnectionPool","nativeMethod":false},{"methodName":"getConnection","fileName":"ConnectionPool.java","lineNumber":198,"className":"org.apache.tomcat.jdbc.pool.ConnectionPool","nativeMethod":false},{"methodName":"getConnection","fileName":"DataSourceProxy.java","lineNumber":132,"className":"org.apache.tomcat.jdbc.pool.DataSourceProxy","nativeMethod":false},{"methodName":"getConnection","fileName":"DatasourceConnectionProviderImpl.java","lineNumber":122,"className":"org.hibernate.engine.jdbc.connections.internal.DatasourceConnectionProviderImpl","nativeMethod":false},{"methodName":"obtainConnection","fileName":"NonContextualJdbcConnectionAccess.java","lineNumber":35,"className":"org.hibernate.internal.NonContextualJdbcConnectionAccess","nativeMethod":false},{"methodName":"acquireConnectionIfNeeded","fileName":"LogicalConnectionManagedImpl.java","lineNumber":99,"className":"org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl","nativeMethod":false},{"methodName":"getPhysicalConnection","fileName":"LogicalConnectionManagedImpl.java","lineNumber":129,"className":"org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl","nativeMethod":false},{"methodName":"getConnectionForTransactionManagement","fileName":"LogicalConnectionManagedImpl.java","lineNumber":247,"className":"org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl","nativeMethod":false},{"methodName":"begin","fileName":"LogicalConnectionManagedImpl.java","lineNumber":254,"className":"org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl","nativeMethod":false},{"methodName":"begin","fileName":"JdbcResourceLocalTransactionCoordinatorImpl.java","lineNumber":203,"className":"org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl","nativeMethod":false},{"methodName":"begin","fileName":"TransactionImpl.java","lineNumber":56,"className":"org.hibernate.engine.transaction.internal.TransactionImpl","nativeMethod":false},{"methodName":"beginTransaction","fileName":"HibernateJpaDialect.java","lineNumber":189,"className":"org.springframework.orm.jpa.vendor.HibernateJpaDialect","nativeMethod":false},{"methodName":"doBegin","fileName":"JpaTransactionManager.java","lineNumber":380,"className":"org.springframework.orm.jpa.JpaTransactionManager","nativeMethod":false},{"methodName":"getTransaction","fileName":"AbstractPlatformTransactionManager.java","lineNumber":373,"className":"org.springframework.transaction.support.AbstractPlatformTransactionManager","nativeMethod":false},{"methodName":"createTransactionIfNecessary","fileName":"TransactionAspectSupport.java","lineNumber":447,"className":"org.springframework.transaction.interceptor.TransactionAspectSupport","nativeMethod":false},{"methodName":"invokeWithinTransaction","fileName":"TransactionAspectSupport.java","lineNumber":277,"className":"org.springframework.transaction.interceptor.TransactionAspectSupport","nativeMethod":false},{"methodName":"invoke","fileName":"TransactionInterceptor.java","lineNumber":96,"className":"org.springframework.transaction.interceptor.TransactionInterceptor","nativeMethod":false},{"methodName":"proceed","fileName":"ReflectiveMethodInvocation.java","lineNumber":179,"className":"org.springframework.aop.framework.ReflectiveMethodInvocation","nativeMethod":false},{"methodName":"intercept","fileName":"CglibAopProxy.java","lineNumber":673,"className":"org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor","nativeMethod":false},{"methodName":"handleMessage","fileName":"","lineNumber":-1,"className":"message.handlers.MessageHandler$$EnhancerBySpringCGLIB$$cf53d23e","nativeMethod":false},{"methodName":"invoke","fileName":null,"lineNumber":-1,"className":"sun.reflect.GeneratedMethodAccessor145","nativeMethod":false},{"methodName":"invoke","fileName":"DelegatingMethodAccessorImpl.java","lineNumber":43,"className":"sun.reflect.DelegatingMethodAccessorImpl","nativeMethod":false},{"methodName":"invoke","fileName":"Method.java","lineNumber":498,"className":"java.lang.reflect.Method","nativeMethod":false},{"methodName":"execute","fileName":"ReflectiveMethodExecutor.java","lineNumber":113,"className":"org.springframework.expression.spel.support.ReflectiveMethodExecutor","nativeMethod":false},{"methodName":"getValueInternal","fileName":"MethodReference.java","lineNumber":102,"className":"org.springframework.expression.spel.ast.MethodReference","nativeMethod":false},{"methodName":"access$000","fileName":"MethodReference.java","lineNumber":49,"className":"org.springframework.expression.spel.ast.MethodReference","nativeMethod":false},{"methodName":"getValue","fileName":"MethodReference.java","lineNumber":347,"className":"org.springframework.expression.spel.ast.MethodReference$MethodValueRef","nativeMethod":false},{"methodName":"getValueInternal","fileName":"CompoundExpression.java","lineNumber":88,"className":"org.springframework.expression.spel.ast.CompoundExpression","nativeMethod":false},{"methodName":"getTypedValue","fileName":"SpelNodeImpl.java","lineNumber":131,"className":"org.springframework.expression.spel.ast.SpelNodeImpl","nativeMethod":false},{"methodName":"getValue","fileName":"SpelExpression.java","lineNumber":330,"className":"org.springframework.expression.spel.standard.SpelExpression","nativeMethod":false},{"methodName":"evaluateExpression","fileName":"AbstractExpressionEvaluator.java","lineNumber":169,"className":"org.springframework.integration.util.AbstractExpressionEvaluator","nativeMethod":false},{"methodName":"processInternal","fileName":"MessagingMethodInvokerHelper.java","lineNumber":319,"className":"org.springframework.integration.util.MessagingMethodInvokerHelper","nativeMethod":false},{"methodName":"process","fileName":"MessagingMethodInvokerHelper.java","lineNumber":155,"className":"org.springframework.integration.util.MessagingMethodInvokerHelper","nativeMethod":false},{"methodName":"processMessage","fileName":"MethodInvokingMessageProcessor.java","lineNumber":93,"className":"org.springframework.integration.handler.MethodInvokingMessageProcessor","nativeMethod":false},{"methodName":"handleRequestMessage","fileName":"ServiceActivatingHandler.java","lineNumber":89,"className":"org.springframework.integration.handler.ServiceActivatingHandler","nativeMethod":false},{"methodName":"handleMessageInternal","fileName":"AbstractReplyProducingMessageHandler.java","lineNumber":109,"className":"org.springframework.integration.handler.AbstractReplyProducingMessageHandler","nativeMethod":false},{"methodName":"handleMessage","fileName":"AbstractMessageHandler.java","lineNumber":127,"className":"org.springframework.integration.handler.AbstractMessageHandler","nativeMethod":false},{"methodName":"doDispatch","fileName":"UnicastingDispatcher.java","lineNumber":160,"className":"org.springframework.integration.dispatcher.UnicastingDispatcher","nativeMethod":false},{"methodName":"dispatch","fileName":"UnicastingDispatcher.java","lineNumber":121,"className":"org.springframework.integration.dispatcher.UnicastingDispatcher","nativeMethod":false},{"methodName":"doSend","fileName":"AbstractSubscribableChannel.java","lineNumber":89,"className":"org.springframework.integration.channel.AbstractSubscribableChannel","nativeMethod":false},{"methodName":"send","fileName":"AbstractMessageChannel.java","lineNumber":423,"className":"org.springframework.integration.channel.AbstractMessageChannel","nativeMethod":false},{"methodName":"send","fileName":"AbstractMessageChannel.java","lineNumber":373,"className":"org.springframework.integration.channel.AbstractMessageChannel","nativeMethod":false},{"methodName":"doSend","fileName":"GenericMessagingTemplate.java","lineNumber":115,"className":"org.springframework.messaging.core.GenericMessagingTemplate","nativeMethod":false},{"methodName":"doSend","fileName":"GenericMessagingTemplate.java","lineNumber":45,"className":"org.springframework.messaging.core.GenericMessagingTemplate","nativeMethod":false},{"methodName":"send","fileName":"AbstractMessageSendingTemplate.java","lineNumber":105,"className":"org.springframework.messaging.core.AbstractMessageSendingTemplate","nativeMethod":false},{"methodName":"sendMessage","fileName":"MessageProducerSupport.java","lineNumber":188,"className":"org.springframework.integration.endpoint.MessageProducerSupport","nativeMethod":false},{"methodName":"access$1100","fileName":"AmqpInboundChannelAdapter.java","lineNumber":56,"className":"org.springframework.integration.amqp.inbound.AmqpInboundChannelAdapter","nativeMethod":false},{"methodName":"processMessage","fileName":"AmqpInboundChannelAdapter.java","lineNumber":246,"className":"org.springframework.integration.amqp.inbound.AmqpInboundChannelAdapter$Listener","nativeMethod":false},{"methodName":"onMessage","fileName":"AmqpInboundChannelAdapter.java","lineNumber":203,"className":"org.springframework.integration.amqp.inbound.AmqpInboundChannelAdapter$Listener","nativeMethod":false},{"methodName":"doInvokeListener","fileName":"AbstractMessageListenerContainer.java","lineNumber":822,"className":"org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer","nativeMethod":false},{"methodName":"invokeListener","fileName":"AbstractMessageListenerContainer.java","lineNumber":745,"className":"org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer","nativeMethod":false},{"methodName":"access$001","fileName":"SimpleMessageListenerContainer.java","lineNumber":97,"className":"org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer","nativeMethod":false},{"methodName":"invokeListener","fileName":"SimpleMessageListenerContainer.java","lineNumber":189,"className":"org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$1","nativeMethod":false},{"methodName":"invoke","fileName":null,"lineNumber":-1,"className":"sun.reflect.GeneratedMethodAccessor112","nativeMethod":false},{"methodName":"invoke","fileName":"DelegatingMethodAccessorImpl.java","lineNumber":43,"className":"sun.reflect.DelegatingMethodAccessorImpl","nativeMethod":false},{"methodName":"invoke","fileName":"Method.java","lineNumber":498,"className":"java.lang.reflect.Method","nativeMethod":false},{"methodName":"invokeJoinpointUsingReflection","fileName":"AopUtils.java","lineNumber":333,"className":"org.springframework.aop.support.AopUtils","nativeMethod":false},{"methodName":"invokeJoinpoint","fileName":"ReflectiveMethodInvocation.java","lineNumber":190,"className":"org.springframework.aop.framework.ReflectiveMethodInvocation","nativeMethod":false},{"methodName":"proceed","fileName":"ReflectiveMethodInvocation.java","lineNumber":157,"className":"org.springframework.aop.framework.ReflectiveMethodInvocation","nativeMethod":false},{"methodName":"doWithRetry","fileName":"StatefulRetryOperationsInterceptor.java","lineNumber":229,"className":"org.springframework.retry.interceptor.StatefulRetryOperationsInterceptor$MethodInvocationRetryCallback","nativeMethod":false},{"methodName":"doExecute","fileName":"RetryTemplate.java","lineNumber":286,"className":"org.springframework.retry.support.RetryTemplate","nativeMethod":false},{"methodName":"execute","fileName":"RetryTemplate.java","lineNumber":210,"className":"org.springframework.retry.support.RetryTemplate","nativeMethod":false},{"methodName":"invoke","fileName":"StatefulRetryOperationsInterceptor.java","lineNumber":173,"className":"org.springframework.retry.interceptor.StatefulRetryOperationsInterceptor","nativeMethod":false},{"methodName":"proceed","fileName":"ReflectiveMethodInvocation.java","lineNumber":179,"className":"org.springframework.aop.framework.ReflectiveMethodInvocation","nativeMethod":false},{"methodName":"invoke","fileName":"JdkDynamicAopProxy.java","lineNumber":213,"className":"org.springframework.aop.framework.JdkDynamicAopProxy","nativeMethod":false},{"methodName":"invokeListener","fileName":null,"lineNumber":-1,"className":"com.sun.proxy.$Proxy137","nativeMethod":false},{"methodName":"invokeListener","fileName":"SimpleMessageListenerContainer.java","lineNumber":1276,"className":"org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer","nativeMethod":false},{"methodName":"executeListener","fileName":"AbstractMessageListenerContainer.java","lineNumber":726,"className":"org.springframework.amqp.rabbit.listener.AbstractMessageListenerContainer","nativeMethod":false},{"methodName":"doReceiveAndExecute","fileName":"SimpleMessageListenerContainer.java","lineNumber":1219,"className":"org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer","nativeMethod":false},{"methodName":"receiveAndExecute","fileName":"SimpleMessageListenerContainer.java","lineNumber":1189,"className":"org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer","nativeMethod":false},{"methodName":"access$1500","fileName":"SimpleMessageListenerContainer.java","lineNumber":97,"className":"org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer","nativeMethod":false},{"methodName":"run","fileName":"SimpleMessageListenerContainer.java","lineNumber":1421,"className":"org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer","nativeMethod":false},{"methodName":"run","fileName":"Thread.java","lineNumber":748,"className":"java.lang.Thread","nativeMethod":false}],"lockedMonitors":[{"className":"org.postgresql.core.v3.QueryExecutorImpl","identityHashCode":1428002949,"lockedStackDepth":9,"lockedStackFrame":{"methodName":"execute","fileName":"QueryExecutorImpl.java","lineNumber":288,"className":"org.postgresql.core.v3.QueryExecutorImpl","nativeMethod":false}}],"lockedSynchronizers":[{"className":"java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync","identityHashCode":1110565932}],"lockInfo":null}
It's not clear why you would see that behavior; in most cases (in my experience) such issues are caused by the listener thread "stuck" in user code. It's not possible for a consumer thread to be "idle" in that way; and in any case, it wouldn't restart 15 minutes later.
Next step (for me) would be to take a thread dump to see what the threads are doing. Boot provides the /dump endpoint to do that if you can't use jstack or VisualVM.

Spring amqp delay messaging with rabbitMQ

I am struggling hard to find out the way for scheduled/Delaying messages in Spring AMQP/Rabbit MQ and found solution in here.But i still with a prolem
about Spring AMQP/Rabbit MQ which can not received any message.
My source as the following:
#Configuration
public class AmqpConfig {
#Bean
public ConnectionFactory connectionFactory() {
CachingConnectionFactory connectionFactory = new CachingConnectionFactory();
connectionFactory.setAddresses("172.16.101.14:5672");
connectionFactory.setUsername("admin");
connectionFactory.setPassword("admin");
connectionFactory.setPublisherConfirms(true);
return connectionFactory;
}
#Bean
#Scope("prototype")
public RabbitTemplate rabbitTemplate() {
RabbitTemplate template = new RabbitTemplate(connectionFactory());
return template;
}
#Bean
CustomExchange delayExchange() {
Map<String, Object> args = new HashMap<String, Object>();
args.put("x-delayed-type", "direct");
return new CustomExchange("my-exchange", "x-delayed-message", true, false, args);
}
#Bean
public Queue queue() {
return new Queue("spring-boot-queue", true);
}
#Bean
Binding binding(Queue queue, Exchange delayExchange) {
return BindingBuilder.bind(queue).to(delayExchange).with("spring-boot-queue").noargs();
}
#Bean
public SimpleMessageListenerContainer messageContainer() {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer(connectionFactory());
container.setQueues(queue());
container.setExposeListenerChannel(true);
container.setMaxConcurrentConsumers(1);
container.setConcurrentConsumers(1);
container.setAcknowledgeMode(AcknowledgeMode.MANUAL);
container.setMessageListener(new ChannelAwareMessageListener() {
public void onMessage(Message message, Channel channel) throws Exception {
byte[] body = message.getBody();
System.err.println("receive msg : " + new String(body));
channel.basicAck(message.getMessageProperties().getDeliveryTag(), false); //确认消息成功消费
}
});
return container;
}
}
#Component
public class Send implements RabbitTemplate.ConfirmCallback{
private RabbitTemplate rabbitTemplate;
#Autowired
public Send(RabbitTemplate rabbitTemplate) {
this.rabbitTemplate = rabbitTemplate;
this.rabbitTemplate.setConfirmCallback(this);
rabbitTemplate.setMandatory(true);
}
public void sendMsg(String content) {
CorrelationData correlationId = new CorrelationData(UUID.randomUUID().toString());
rabbitTemplate.convertAndSend("my-exchange", "", content, new MessagePostProcessor() {
#Override
public Message postProcessMessage(Message message) throws AmqpException {
message.getMessageProperties().setHeader("x-delay", 6000);
return message;
}
},correlationId);
System.err.println("delay message send ................");
}
/**
* 回调
*/
#Override
public void confirm(CorrelationData correlationData, boolean ack, String cause) {
System.err.println(" callback id :" + correlationData);
if (ack) {
System.err.println("ok");
} else {
System.err.println("fail:" + cause);
}
}
}
Is there someone could give a help.
Thanks all.
Delay messaging is nothing to do with Spring amqp, it's a library which will reside with your code, so the library can't hold any message as such. There are two approaches you can try:
Old Approach:
Set the TTL(time to live) header in each message/queue(policy) and then introduce a DLQ to handle it. once the ttl expired your messages will move from DLQ to main queue so that your listener can process it.
Latest Approach:
Recently RabbitMQ came up with RabbitMQ Delayed Message Plugin , using which you can achieve the same and this plugin support available since RabbitMQ-3.5.8.
You can declare an exchange with the type x-delayed-message and then publish messages with the custom header x-delay expressing in milliseconds a delay time for the message. The message will be delivered to the respective queues after x-delay milliseconds
Details:
To use the delayed-messaging feature, declare an exchange with the type x-delayed-message:
Map<String, Object> args = new HashMap<String, Object>();
args.put("x-delayed-type", "direct");
channel.exchangeDeclare("my-exchange", "x-delayed-message", true, false, args);
Note that we pass an extra header called x-delayed-type, more on it under the Routing section.
Once we have the exchange declared we can publish messages providing a header telling the plugin for how long to delay our messages:
byte[] messageBodyBytes = "delayed payload".getBytes("UTF-8");
Map<String, Object> headers = new HashMap<String, Object>();
headers.put("x-delay", 5000);
AMQP.BasicProperties.Builder props = new AMQP.BasicProperties.Builder().headers(headers);
channel.basicPublish("my-exchange", "", props.build(), messageBodyBytes);
byte[] messageBodyBytes2 = "more delayed payload".getBytes("UTF-8");
Map<String, Object> headers2 = new HashMap<String, Object>();
headers2.put("x-delay", 1000);
AMQP.BasicProperties.Builder props2 = new AMQP.BasicProperties.Builder().headers(headers2);
channel.basicPublish("my-exchange", "", props2.build(), messageBodyBytes2);
In the above example we publish two messages, specifying the delay time with the x-delay header. For this example, the plugin will deliver to our queues first the message with the body "more delayed payload" and then the one with the body "delayed payload".
If the x-delay header is not present, then the plugin will proceed to route the message without delay.
More here: git

Resources