It looks like ATG's implementation of JMS is achieved through scheduler whereby it polls database in specific interval through SqlJmsProvider component. I do agree that ATG's DMS does provide with all JMS feature like Queue, Topic, Durable Subscribers, Retry etc... but isin't using a scheduler to poll the DB to send ATG Order to Fulfillment an overkill ? (too many queries fired)
From the ATG Documentation it explains that there are two JMS providers available within ATG:
SQL JMS
Local JMS
The difference between the two are:
Local JMS is Synchronous and extremely fast. It runs within a single transaction and is bound to a single process, thus when sending a message, it blocks while waiting for an acknowledgement. SQL JMS on the otherhand, is Asynchronous and can be used across processes (thus an order submission on Commerce can be processed on Fulfillment). SQL JMS is non-blocking so once the message is put on the queue, the requesting process can continue. This also means that Commerce can continue running, even if Fulfillment is down. The messages are also persisted in SQL JMS while they are stateless and lost during a restart for Local JMS.
Using the scheduler to poll the queue is an acceptable solution and most of the older asynchronous message queues implemented this solution. In IBM MQ Version 7 performance has been improved by reducing the amount of polling while using IBM WAS Version 6 for example the solution is also based on regular polling of the queue.
So no, polling the database on a scheduled interval is not an overkill.
Related
I have multiple instances of a worker connected to a queue and all requests will be distributed to worker instances in a load balanced way. When a new worker instance is connected to the queue, I should dump a small data from mainstream app to this new worker instance (one time job).
Currently I'm using REST endpoint from mainstream app for doing this at application start-up but can we leverage the messaging queue for this? Once a new worker instance connected to queue, it will ask the initial data dump to mainstream app through queue and then app will reply with initial data.
Is it possible using messaging queue/topic? Kindly share your views/suggestions to achieve this using activemq
If you're using ActiveMQ Artemis this kind of requirement is typically fulfilled with a queue that supports both non-destructive and last-value semantics. The last-value semantics allows the queue to stay up-to-date with the latest messages and the non-destructive semantics means that even when consumers acknowledge the messages they will remain on the queue for the next client which connects. When using this combination clients can first consume all the messages from this special "initialization" queue and then continue on with whatever other messaging work they need to do.
Unfortunately ActiveMQ "Classic" doesn't support either of these semantics and there is no straight-forward way to get equivalent behavior.
We have a requirement in where we need to send only one message at a time to a backend process. The call back of this process takes around an hour, only after the call back can we send another request to the process.
I am trying to achieve this by using a manager bpel process that will hold the messages first if there is already something being processed in the backend, and then send it once it realizes that the backend is free. This approach will work, but our architect wants a cleaner solution. He suggested using JMS queues. The idea is for the jms queue to messages to be read by a amanger one at a time, only moving on to the next one once we receive the callback from the backend and we know that the composite and bpel instance is finished. I've been scouring the internet for weeks, but I couldn't find a working jms based solution for my requirement.
I've tried the suggestions for this link but turning on unit of order and acknowledgement properties does nothing.
Try this approach!!
Use a event driven bpel process.
Use a database flag as your next trigger. (flag is TRUE)
jms Adapter receives first message from the queue. Here use a delay in the adapter since you are expecting the bpel to be long running. use below setting.
<binding.jca config="MyServiceInboundQueue_jms.jca">
<property name="minimumDelayBetweenMessages">10000</property>
<property name="singleton">true</property>
</binding.jca>
if flag == TRUE in the db causes the db adapter to proceed with the bpel process,
else skip the bpel.
mark flag==FALSE
call the backend system
callback is received after an hour.
set flag==TRUE
Hi Jonar,
At my company we always use JMS queues for Asynchronous messaging. You could do with a delay timer build in your composite set to 1 hour and 15 minutes for example, and it will work most of the time, but its hella messy. The whole idea is for any asynchronous process to kick off when a message is put upon its queue target (specified by the JMS queue). The JMS adapter in the composite of your project will pick up the message from the queue when it is free to process the queue. The goal for you would be to put the message on the queue and pick it up from it using the adapter. It will know which message to pick up because you specify which queues it listens to in the adapter.
The following blog post by John-Brown Evans eplains the whole process from step one. It might be a bit tedious, but I found it very helpful. Its using SOa Suite 11g instead of the nowadays more commonly used 12c, but its fundamentals remain the same.
Awesome JMS queue tutorial
I hope this works for you!
Cheers,
Jesper
I have an backlogged topic of ActiveMQ messages for customer data in a production environment, and I need to write a script that dequeues it and does whatever business logic is necessary.
If something goes wrong in the business logic, but I've already read the message (via JMS probably), that would presumably mean all the messages are gone - and I'd be in huge trouble.
Is there a way to read the messages without deleting them?
This is a common pattern in messaging. If you're using the JMS API you have a couple of options:
Use CLIENT_ACKNOWLEDGE mode when you create your session and acknowledge the message once the business logic is complete. If the business logic fails don't acknowledge the message and it won't be removed from the queue.
Use a transacted session. If the business logic completes successfully then commit the transaction. If the business fails then rollback the transaction.
Both of these are very common and you can find more information about all the proper API calls, etc. using your favorite search engine.
I have a TIBCO BusinessWorks process that is publishing to a JMS topic -- lets call it TOPIC.A -- there is one process that is subscribing to that topic with a name SUBSCRIBE.A.
The problem I have is that the first server that starts listening to SUBSCRIBE.A hooks in fine. The other 3 servers, running the exact same process get an error of "WebSphere MQ call failed with compcode '2' ('MQCC_FAILED') reason '2429' ('MQRC_SUBSCRIPTION_IN_USE')"
This can't be reasonable behavior for enterprise software, and I know WebsphereMQ, JMS and TIBCO Businessworks all scale well, so I must be missing something. I only want each event processed once, but a single box just isn't going to do it, both for fail-over reasons and shear volume reasons.
What do I have to do to let all 4 servers in the cluster to service the subscription SUBSCRIBE.A?
I agree it does not seem like reasonable behavior for enterprise software - but this restriction is imposed by the JMS specification. The JMS 1.1 spec, section 6.66.1 says "Only one session at a time can have a TopicSubscriber for a particular durable subscription".
That said, WebSphere MQ does provide a vendor-specific option that allows you to do what you want: see the CLONESUPP Connection Factory property. This is documented in the Infocenter at the CLONESUPP properties page.
While it's specific to MQ, if you specify this using administered objects you code will not need to use any vendor-specific methods.
The reason for MQRC 2429 is because all of your 4 subscribers are using the same client ID and all are trying to use the same durable subscription. When a subscriber is already actively listening on a durable subscription, no other subscribers can listen on the same. If you want all subscribers to listen concurrently, then have separate client ids for each of your subscribers.
But you must note that all subscribers will get a copy of the same message published by a publisher (TIBCO BusinessWorks in your case). So if all subscribers are active, all will process the event.
For fail-over you can look at using WebSphere MQ Multi-instance queue manager feature or any other HA solutions. For load sharing you can look at WebSphere MQ clustering feature.
in order to reach high performance production of messages with jms with transactions enabled, one needs to control the amount of messages being sent on each transaction, the larger the number the higher the performance are,
is it possible to control transactions in such a way using spring integration ?
one might suggest using an aggregator, but that defeats the purpose because i dont want to have one message containing X smaller messages on the queue, but actually X messages on my queue..
Thanks !
I'm not aware of your setup, but I'd bump up the concurrent consumers on the source than try to tweak the outbound adapter. What kind of data source is pumping in this volume of data ? From my experience, usually the producer lags behind the publisher - unless both are JMS / messaging resources - like in the case of a bridge. In which case you will mostly see a significant improvement by bumping up the concurrent consumers, because you are dedicating n threads to receive messages and process them in parallel, and each thread will be running in its own "transaction environment".
It's also worthwhile to note that JMS does not specify a transport mechanism, and its unto the broker to choose the transport. If you are using activemq you can try experimenting with open wire vs amqp and see if you get the desired throughput.