I have a queue that messages has properties name orderno
how can I fetch message from queue sorted by orderno
What you want to do is really the resequencer enterprise integration pattern
You can achieve that using Apache Camel, which is bundled with ActiveMQ.
What you need to do is to add the following to the ´camel.xml´ config file in ActiveMQ.
<camelContext id="camel" xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="activemq:queue:unordered"/>
<resequence>
<simple>in.header.orderno</simple>
<to uri="activemq:queue:ordered" />
<stream-config capacity="5000" timeout="4000"/>
</resequence>
</route>
</camelContext>
Then just make sure that file is included inside your ActiveMQ config, such as activemq.xml:
<import resource="camel.xml"/>
Note that this setup will read messages in any order from the "unordered" queue, while your application should read the re-ordered from the "ordered" queue.
Related
Im trying to implement a load balancer using Apache Camel and Spring.
To do so, one has to configure the target servers in a spring.xml like this:
<camelContext xmlns="http://camel.apache.org/schema/spring">
<route>
<from uri="localhost:8000"/>
<loadBalance>
<roundRobin/>
<to uri="localhost:8080"/>
<to uri="localhost:8081"/>
<to uri="localhost:8082"/>
</loadBalance>
</route>
</camelContext>
Now, is there a way to read in all target servers from a properties file?
Just so one could change them without editing the spring.xml...
As your describe, I think the recipient-list(http://camel.apache.org/recipient-list.html) can solve your problem.
You can decide which uri to be sent in the java code, things like read servers from a properties file can be easily done.
There are multiple choices available to do that:
http://camel.apache.org/recipient-list.html
http://camel.apache.org/dynamic-router.html
Starting from Camel 2.16 you can also use "dynamic to" (http://camel.apache.org/message-endpoint.html)
I am trying to delay a message to be in queue for few seconds.
But when I use camel delay option, it is not delaying in queue, instead it is immediately consumed, and delaying in the route path.
How can we delay messages so that they will be there in queue for few seconds?
My spring with camel configuration looks like below.
<bean id="jms" class="org.apache.camel.component.jms.JmsComponent">
<property name="connectionFactory" ref="jmsConnectionFactory" />
</bean>
<camelContext id="camelContext" xmlns="http://camel.apache.org/schema/spring">
<route id="routeOne" delayer="10000">
<from uri="jms://queueone?concurrentConsumers=1"/>
<log message="routeOne incoming message ${body}"/>
<delay><constant>30000</constant></delay>
<process ref="loggerProcessor"/>
</route>
</camelContext>
<bean id="loggerProcessor" name="loggerProcessor" class="emh.LoggerProcessor"/>
Delay via Camel
Camel delay and throttle will remove (consume) the message from ActiveMQ queue and keep it (in-memory) in the route until processed. Transacted JMS might alleviate issues concerning losing the message, haven't tried that yet. Food for thought.
Delay via ActiveMQ
I got it to work in Spring Boot 2 with Camel and ActiveMQ using the AMQ_SCHEDULED_DELAY header combined with enabling schedulerSupport [1, 2, 3, 4, 5, 6]. This is per JMS 2.0 Delivery Delay.
Note that a message will only appear in the ActiveMQ queue (pending) when there are no active consumers - i.e. application down or Camel route disabled. Configuration below is for using an existing ActiveMQ broker.
application.properties (source)
spring.activemq.broker-url=tcp://localhost:61616
RouteBuilder
from("jms://incomingQueue").setHeader("AMQ_SCHEDULED_DELAY", constant("1000")).inOnly("jms://delayedQueue");
from("jms://delayedQueue").process(...);
activemq.xml (existing broker installation)
<broker xmlns="http://activemq.apache.org/schema/core" brokerName="localhost" dataDirectory="${activemq.data}" schedulerSupport="true">
For OSX Homebrew found under /usr/local/Cellar/activemq/<version>/libexec/conf.
If you want to use a broker embedded in your application you can use the BrokerService.
By default data is persisted in activemq-data and requires the activemq-kahadb-store dependency - also if you chose JDBC (source). Using brokerService.setPersistent( false ) the store dependency is no longer required, but then the JMS message is stored in-memory again.
#Bean
public BrokerService brokerService()
{
BrokerService brokerService = new BrokerService();
brokerService.setSchedulerSupport( true );
return brokerService;
}
Further examples can be found here and here.
Camel has built in support for the Throttler patter, there's a throttler component. Refer: http://camel.apache.org/throttler.html
Simply add the following to the route and that should delay the message.
<throttle timePeriodMillis="30000">
I am using Camel to read messages off an Amazon SQS queue, do some CPU-intensive processing, and then place them on another SQS queue. Camel is being invoked through the maven plugin, using mvn camel:run.
When run on a server with multiple cores, Camel only appears to be using a single core to process messages (as observed by monitoring CPU utilization of all cores). How can I take advantage of all available cores?
Things I have tried:
Running multiple instances of the JVM (e.g. by running several mvm camel:run jobs in parallel). This works, but does not seem ideal.
Setting the maxMessagesPerPoll=10 on the consumer. Did not appear to make a difference.
The route in question:
<route id="marctomods" errorHandlerRef="eh">
<from uri="aws-sqs://{{sqs.environment}}-normalize-marcxml?accessKey=${access.key}&secretKey=${secret.key}&amazonSQSClient=#sqsClient&maxMessagesPerPoll=10" />
<process ref="modsProcessor"/>
<to uri="aws-sqs://{{sqs.environment}}-enrich?accessKey=${access.key}&secretKey=${secret.key}&amazonSQSClient=#sqsClient" />
</route>
You generally only get a single message in transit through a route. http://camel.apache.org/parallel-processing-and-ordering.html explains the parallel processing options.
You could, for example, read messages from sqs onto a seda queue and then have multiple parallel consumers reading from the seda queue, eg
<route>
<from uri="aws-sqs://{{sqs.environment}}-normalize-marcxml />
<to uri="seda:myQueue">
</route>
<route>
<from uri="seda:myQueue?concurrentConsumers=10" />
<process ref="modsProcessor"/>
<to uri="aws-sqs://{{sqs.environment}}-enrich>
</route>
I have CXF endpoint. I have dedicated route defined for each operation and routing the request to appropriate routes using header.operation name with reciepient list.
For couple operations(routes), I have post request to JMS endpoint which gets processed by another application and response is recieved in another queue. I have to read response message and transform the message. My requesting thread(webservice call) would be waiting for the transformed message.
Currently after posting request, currently I have written my own processor which use spring jms template to read message. I came across few examples such as split routes but not sure how it will work for overall synchronous communication which has in between JMS communication.
You can find a JMX Request/Reply example here:
<bean id="jms" class="org.apache.activemq.camel.component.ActiveMQComponent">
<property name="brokerURL" value="vm://localhost?broker.persistent=false"/>
</bean>
<bean id="setResponseProcessor" class="org.apache.camel.itest.jetty.SetResponseProcessor"/>
<camelContext xmlns="http://camel.apache.org/schema/spring" trace="true">
<route>
<from uri="cxf://serverBean"/>
<to uri="jms:responseQueue"/>
</route>
<route>
<from uri="jms:responseQueue"/>
<process ref="setResponseProcessor"/>
</route>
</camelContext>
With the java JMS API, I got from a DB an array of bytes and then I'm sending it to an ActiveMQ as a javax.jms.BytesMessage. After that with camel I want to put the file on a location,
I have this route in camel:
<route>
<from uri="activemq:queue.fileOuput"/>
<convertBodyTo type="java.nio.ByteBuffer"/>
<to uri="file://C:/output/"/>
</route>
But my problem is that my file in c:\output\ directory, I got the file with the message id as the file name, like
queue-preVerificacion-fileoutput-ID-jmachine-57401-1347652410053-0-1-1-1-1
but I want to put the name I have in the database, like MyFile.xml.
I have tried to set a message property like fileName and file:name, and also I saw in the apache document that I need to put a header "org.apache.camel.file.name", but with jms I don't know how to do it.
So my question is how can I put a customized name in the camel route?
Thanks to all.
Just place the file name in the jms message (as a string property).
// Something like this if you send the message using plain java/jms:
msg.setStringProperty("filename","MyFile.xml");
..//Send msg
Then you can do something like this in camel
<to uri="file://C:/output/?fileName=${header.filename}"/>
you just need to set the "CamelFileName" header value (based on a message header, etc)
<route>
<from uri="activemq:queue.fileOuput"/>
<convertBodyTo type="java.nio.ByteBuffer"/>
<setHeader headerName="CamelFileName">
<constant>${header.fileName}</constant>
</setHeader>
<to uri="file://C:/output/"/>
</route>
I think "org.apache.camel.file.name" is for camel 1.x , in the 2.x version CamelFileName worked fine. But I wanted a more dynamic file name, name based on the content. This example using a processor worked well ( camel 2.18 )
<route>
<from uri="MQ:MY_Q_NAME" />
<process ref="MyMessageProcessor"/>
<to uri="file://E:\OUTPUT" />
</route>
Inside the Processor :
exchange.getIn().setHeader(Exchange.FILE_NAME, myFileName);