The problem I am facing is very strange. My application is a Spring Boot application where I am using a Camel route to listen to AMQ 7 and then processing the message till it gets stored in a database or sent to another AMQ 7 instance. The issue is messages are consumed from AMQ via Camel but it's not maintaining the order in application logs. This means that before the flow for first message is getting completed I could see the logs start getting printed for another message dropping into AMQ which is making it difficult to track that the logs belong to which message from AMQ. Is it a prob with AMQ or Camel routes? Any suggestions is highly appreciated.
This issue could be related to the camel route if the asyncConsumer option is enabled, see the documentation for further details https://camel.apache.org/components/latest/activemq-component.html
Related
I have Apache-Camel with Spring application. Application acts as a bridge between two AMQP destinations. It consumes messages from one broker and publishes it on to the other broker. Communication is done both ways over AMQP1.0 protocol.
Problem
I am facing a IDLE connection issue. After few days of operations, the consumers stops receiving messages, unless restarted. Moreover, I am not able to get any ERROR logs. This issue goes away after restart of application.
My expectation is that similar to Spring-JMS, Apache Camel shall retry connecting the consumers. Kindly guide me if I need to configure something in Camel to perform reconnection tries and do proper logging.
Camel Route COnfiguration
cmlCntxt.addRoutes(new RouteBuilder() {
public void configure() {
from("incomingOne:queue:" + inQueueOne)
.to("outGoingBroker:queue:"outQueueOne).transform(body().append("\n\n"));
from("inQueueTwo:queue:" + inQueueTwo).to("outGoingBroker:"+outQueueTwo).transform(body().append("\n\n"));
}
});
Moreover I am not having control of the brokers at both ends and am unable to check why my consumers are not receiving messages. That is why I am expecting camel ERROR logs to be informative for me to debug the issue, whether connectivity or else.
Try configuring jms.requestTimeout property at your remoteURI. by default, the requestTimeout is indefinite . So incase of any issues, it might stuck forever.
Also try using failover to connect the broker and enable debugging in application.
if you are still facing the issue, kindly edit with broker details
So using JMS and ActiveMQ, I can be sure that my message sent from my Spring Boot application using JmsTemplate will reach it's destination application even if that destination application is down at the time I send the message to ActiveMQ. As when the destination application starts up, it grabs the message from the queue. Great!
However.
What happens if my Spring Boot application tries to send a JMS message to a queue on the ActiveMQ server, but the ActiveMQ server is down at that point or the network is down and I get a connection refused exception?
What is the recommended way to make sure my application keeps trying to re-sends the message to ActiveMQ until it is successful? Is this something I have to develop into my application myself? Are there any nifty Spring tools or annotations which do this for me? Any advice on best practice or how I should be handling this scenario?
You can try Spring-Retry. Has lots of fine grain controls for it:
http://www.baeldung.com/spring-retry
https://github.com/spring-projects/spring-retry
If it is critical that you don't lose this message, you will want to save it to some alternative persistent store (e.g. filesystem, local mq server) along with whatever retry code you come up with. But for those occasional network glitches or a very temporary mq shutdown/restart, Spring-Retry alone should do the trick.
Couple of approaches I can think of
1. You can set up another ActiveMq as fallback. In your code you don't have to do anything, just change your broker url from
activemq.broker.url=tcp://amq01.blah.blah.com:61616
to
activemq.broker.url=failover:(tcp://amq01.blah.blah.com:61616,tcp://amq02.blah.blah.com:61616)?randomize=false
The rest is automatically taken care of. i.e. when one of them is down, the messages are sent to other.
Another approach is to send to a internal queue (like seda, direct) when activemq is down and read from there.
Adding failover to the url is one appropriate way.
And another reasonable way is to making sure activemq always online , as activemq has the master-slave mode(http://activemq.apache.org/masterslave.html) to get high availability.
I am using Spring Integration in my project.
We have a requirement that in case where we will have stop Spring standalone service if database goes down.
In Message listener when I persist the data into database I check if I get CannotGetJdbcConnectionException then stop the Spring service using applicationContext.close() method.
Problem here is if I received any message on to the Queue and database goes down.
I tried to close Spring service then all resource goes down except DefaultMessageListenerContainer that holds that message.
If I terminate the process manually then message goes into inbound Queue which is correct.
Is there any way I could stop Spring service forcefully and put the message back to Inbound Queue?
I hope I am clear with my point here.
Thanks
Sachin
You should configure the DMLC with setSessionTransacted(true) (acknowledge="transacted" when using the namespace to define the endpoints).
Then any in-flight messages will be rolled-back onto the queue.
Application Data Flow:
JSon Messages--> Active MQ --> Spring XD-- Business Login(Transform JSon to Java Object)--> Save Data to Target DB--> DB.
Question:
Sprin-Xd is running in cluster mode, configured with Radis.
Spring XD picks up the message from the Active message queue(AMQ). So message is no longer in AMQ. Now while one of the containers where this message is being processed with some business logic suddenly goes down. In this scenarios-
Will Spring-XD framework automatically re-process that particular message ? what's mechanism behind that?
Thanks,
Abhi
Not with a Redis transport; Redis has no infrastructure to support such a requirement ("transactional" reads). You would need to use a rabbit or kafka transport.
EDIT:
See Application Configuration (scroll down to RabbitMQ) and Message Bus Configuration.
Specifically, the default ackMode is AUTO which means messages are acknowledged on success.
Is there a way catch that time-out and send it back to user?
Not sure if Apache Camel provides something out of the box.
I want to test and alert the user which queues are up and running. So that it will be clear to user that which functionality of the application will work or not.
I am using both Spring and Apache Camel in my project.
Ah! I found out. Camel has an out of the box solution.
Camel 2.1: Specifies whether to test the connection on startup. This ensures that when Camel starts that all the JMS consumers have a valid connection to the JMS broker. If a connection cannot be granted then Camel throws an exception on startup. This ensures that Camel is not started with failed connections. From Camel 2.8 onwards also the JMS producers is tested as well.