Is there a way catch that time-out and send it back to user?
Not sure if Apache Camel provides something out of the box.
I want to test and alert the user which queues are up and running. So that it will be clear to user that which functionality of the application will work or not.
I am using both Spring and Apache Camel in my project.
Ah! I found out. Camel has an out of the box solution.
Camel 2.1: Specifies whether to test the connection on startup. This ensures that when Camel starts that all the JMS consumers have a valid connection to the JMS broker. If a connection cannot be granted then Camel throws an exception on startup. This ensures that Camel is not started with failed connections. From Camel 2.8 onwards also the JMS producers is tested as well.
Related
While doing some load tests with the ActiveMQ Artemis broker and my Spring Boot application I am getting into performance issues.
What I am doing is, sending e.g. 12,000 messages per second to the broker with JMSeter and the application receives them and saves them to a DB. That works fine. But when I extend my application by a filter mechanism, which forwards events after saving to DB, back to the broker using jmsTemplate.send(destination, messageCreator) it goes very slow.
I first used ActiveMQ 5.x and there this mechanism works fine. There you could configure the ActiveMQConnectionFactory with setAsyncSend(true) to tune performance. For the ActiveMQ Artemis ConnectionFactory implementation there is no such a possibility. Is there another way to tune performance like in ActiveMQ 5.x?
I am using Apache ActiveMQ Artemis 2.16.0 (but also tried 2.15.0), artemis-jms-client 2.6.4, and Spring Boot 1.5.16.RELEASE.
The first thing to note is that you need to be very careful when using Spring's JmsTemplate to send messages as it employs a well-known anti-pattern that can really kill performance. It will actually create a new JMS connection, session, and producer for every message it sends. I recommend you use a connection pool like this one which is based on the ActiveMQ 5.x connection pool implementation but now supports JMS 2. For additional details about the danger of using JmsTemplate see the ActiveMQ documentation. This is also discussed in an article from Pivotal (i.e. the "owners" of Spring).
The second point here is that you can tune if persistent JMS messages are sent synchronously or not using the blockOnDurableSend URL property, e.g.:
tcp://localhost:61616?blockOnDurableSend=false
This will ensure that persistent JMS messages are sent asynchronously. This is discussed further in the ActiveMQ Artemis documentation.
I have Apache-Camel with Spring application. Application acts as a bridge between two AMQP destinations. It consumes messages from one broker and publishes it on to the other broker. Communication is done both ways over AMQP1.0 protocol.
Problem
I am facing a IDLE connection issue. After few days of operations, the consumers stops receiving messages, unless restarted. Moreover, I am not able to get any ERROR logs. This issue goes away after restart of application.
My expectation is that similar to Spring-JMS, Apache Camel shall retry connecting the consumers. Kindly guide me if I need to configure something in Camel to perform reconnection tries and do proper logging.
Camel Route COnfiguration
cmlCntxt.addRoutes(new RouteBuilder() {
public void configure() {
from("incomingOne:queue:" + inQueueOne)
.to("outGoingBroker:queue:"outQueueOne).transform(body().append("\n\n"));
from("inQueueTwo:queue:" + inQueueTwo).to("outGoingBroker:"+outQueueTwo).transform(body().append("\n\n"));
}
});
Moreover I am not having control of the brokers at both ends and am unable to check why my consumers are not receiving messages. That is why I am expecting camel ERROR logs to be informative for me to debug the issue, whether connectivity or else.
Try configuring jms.requestTimeout property at your remoteURI. by default, the requestTimeout is indefinite . So incase of any issues, it might stuck forever.
Also try using failover to connect the broker and enable debugging in application.
if you are still facing the issue, kindly edit with broker details
I am currently working on an Integration application that uses Camel with Spring Boot. There is a camel route in integration application that receive messages from source Artemis broker that is transformed and sent to another Artemis broker.
The camel route looks like this:
from(sourceQueue).process(transformProcessor).to(destinationQueue)
When the camel route starts, it recreates the queue names mentioned in the from and to and the previous messages are lost. We do not expect this to happen.
One way I found to do this is in the Artemis ActiveMQ broker.xml, disable the queue and topic auto creation and create the queue(s) using Artemis API.
My question is, can we configure camel JMS / AMQP component to create the queue only if it is not present and if present use the existing ones?
By default Camel will use DynamicDestinationResolver. You can create your own custom DestinationResolver and plug it in your endpoint (or into your component)
.to("jms:queue:myQueue?destinationResolver=MyCustomDestinationResolver");
You can also use JndiDestinationResolver, which by default does not fallback into creating a dynamic destination.
I don't know Artemis but it sounds weird for a broker to delete a queue with its messages. At least its "brother" ActiveMQ has by default the behavior you expect: queues are automatically created if they do not exist, but they just stay if they already exist.
Are you sure the queues are recreated on route start? Are these queues persistent? Could it be that a consumer just drains the queue? I also found a queue attribute of Artemis named auto-delete-queues that would delete the queue if it was drained by a consumer.
auto-delete-queues Whether or not to the broker should automatically delete auto-created JMS queues when they have both 0 consumers and 0 messages.
So using JMS and ActiveMQ, I can be sure that my message sent from my Spring Boot application using JmsTemplate will reach it's destination application even if that destination application is down at the time I send the message to ActiveMQ. As when the destination application starts up, it grabs the message from the queue. Great!
However.
What happens if my Spring Boot application tries to send a JMS message to a queue on the ActiveMQ server, but the ActiveMQ server is down at that point or the network is down and I get a connection refused exception?
What is the recommended way to make sure my application keeps trying to re-sends the message to ActiveMQ until it is successful? Is this something I have to develop into my application myself? Are there any nifty Spring tools or annotations which do this for me? Any advice on best practice or how I should be handling this scenario?
You can try Spring-Retry. Has lots of fine grain controls for it:
http://www.baeldung.com/spring-retry
https://github.com/spring-projects/spring-retry
If it is critical that you don't lose this message, you will want to save it to some alternative persistent store (e.g. filesystem, local mq server) along with whatever retry code you come up with. But for those occasional network glitches or a very temporary mq shutdown/restart, Spring-Retry alone should do the trick.
Couple of approaches I can think of
1. You can set up another ActiveMq as fallback. In your code you don't have to do anything, just change your broker url from
activemq.broker.url=tcp://amq01.blah.blah.com:61616
to
activemq.broker.url=failover:(tcp://amq01.blah.blah.com:61616,tcp://amq02.blah.blah.com:61616)?randomize=false
The rest is automatically taken care of. i.e. when one of them is down, the messages are sent to other.
Another approach is to send to a internal queue (like seda, direct) when activemq is down and read from there.
Adding failover to the url is one appropriate way.
And another reasonable way is to making sure activemq always online , as activemq has the master-slave mode(http://activemq.apache.org/masterslave.html) to get high availability.
Application Data Flow:
JSon Messages--> Active MQ --> Spring XD-- Business Login(Transform JSon to Java Object)--> Save Data to Target DB--> DB.
Question:
Sprin-Xd is running in cluster mode, configured with Radis.
Spring XD picks up the message from the Active message queue(AMQ). So message is no longer in AMQ. Now while one of the containers where this message is being processed with some business logic suddenly goes down. In this scenarios-
Will Spring-XD framework automatically re-process that particular message ? what's mechanism behind that?
Thanks,
Abhi
Not with a Redis transport; Redis has no infrastructure to support such a requirement ("transactional" reads). You would need to use a rabbit or kafka transport.
EDIT:
See Application Configuration (scroll down to RabbitMQ) and Message Bus Configuration.
Specifically, the default ackMode is AUTO which means messages are acknowledged on success.