How can I view/browse the message on the queue programmatically? I don't want to consume the message.
Below is the code I am using to send the message:
#Autowired
private JmsTemplate producerJmsTemplate;
#Value("${.jms.host}")
private String jmsHost;
public void sendMessage(String message, String destQueue) {
this.producerJmsTemplate.convertAndSend(destQueue, message);
}
If you're using JMS then you can use a QueueBrowser. The Solace documentation has additional details:
Create a QueueBrowser object.
Call Session.createBrowser(...) and pass in the queue that you want to browse.
Optionally, you can also pass in a selector string for the Selector property. Using a selector enables the client to only browse messages that match a selector. Note that it could take longer for the message bus to evaluate spooled messages against a selector, especially if the selector used is complex. For more information on selectors, refer to Selectors.
Use the QueueBrowser to get an Enumeration object that can be used to browse the current queue messages in the order they were received (from oldest to newest) by the queue.
Call QueueBrowser.getEnumeration().
Iterate over the messages on the queue.
Call Enumeration.hasMoreElements(). This method returns true if there is at least one message available in the browser’s local message buffer; otherwise, it returns false.
If it returns false, it does not necessarily mean that the queue is empty, rather the local buffer does not currently contain any messages. Subsequent calls to Enumeration.hasMoreElements() or Enumeration.nextElement() could return true and return a message, respectively.
Related
The below shows a snippet for the function, please suggest how to send data to different topics based on if it has error or not
public Function<KStream<String,?>, KStream<String,?>> process(){
return input -> input.map(key, value) {
try{
// logic of function here
}catch(Exception e) {
// How do I send to different topic from here??
}
return new KeyValue<>(key,value);
}
}
Set the kafka consumer binding's enableDlq option to true; when the listener throws an exception the record is sent to the dead letter topic after retries are exhausted. If you want to fail immediately, set the consumer binding's maxAttempts property to 1 (default is 3).
See the documentation.
enableDlq
When set to true, it enables DLQ behavior for the consumer. By default, messages that result in errors are forwarded to a topic named error.<destination>.<group>. The DLQ topic name can be configurable by setting the dlqName property or by defining a #Bean of type DlqDestinationResolver. This provides an alternative option to the more common Kafka replay scenario for the case when the number of errors is relatively small and replaying the entire original topic may be too cumbersome. See Dead-Letter Topic Processing processing for more information. Starting with version 2.0, messages sent to the DLQ topic are enhanced with the following headers: x-original-topic, x-exception-message, and x-exception-stacktrace as byte[]. By default, a failed record is sent to the same partition number in the DLQ topic as the original record. See Dead-Letter Topic Partition Selection for how to change that behavior. Not allowed when destinationIsPattern is true.
Default: false.
In a spring boot application, I have a class with jms listener.
public class PaymentNotification{
#JmsListener(destination="payment")
public void receive(String payload) throws Exception{
//mapstring conversion
....
paymentEvent = billingService.insert(paymentEvent); //transactional method
//call rest...
billingService.save(paymentEvent);
//send info to jms
}
}
I saw then when a error happen, data is inserted in the database, that ok, but it's like receive method is called again and again... but queue is empty when I check on the server.
If there is an error, I don't want method is called again, Is there something for that.
The JMS Message Headers might contain additional information to help with your processing. In particular JMSRedelivered could be of some value. The Oracle doc states that "If a client receives a message with the JMSRedelivered field set, it is likely, but not guaranteed, that this message was delivered earlier but that its receipt was not acknowledged at that time."
I ran the following code to explore what was available in my configuration (Spring Boot with IBM MQ).
#JmsListener(destination="DEV.QUEUE.1")
public void receive(Message message) throws Exception{
for (Enumeration<String> e = message.getPropertyNames(); e.hasMoreElements();)
System.out.println(e.nextElement());
}
From here I could find JMSXDeliveryCount is available in JMS 2.0. If that property is not available, then you may well find something similar for your own configuration.
One strategy would be to use JMSXDeliveryCount, a vendor specific property or maybe JMSRedelivered (if suitable for your needs) as a way to check before you process the message. Typically, the message would be sent to a specific blackout queue where the redelivery count exceeds a set threshold.
Depending on the messaging provider you are using it might also be possible to configure back out queue processing as properties of the queue.
In a spring boot application using artemis we try to avoid queues containing too many messages. The intention is to only put in new messages if the number of messages currently in the queue falls below a certain limit, e.g. 100 messages. However, that seems not to work but we don't know why or what the "correct" method would be to implement that functionality. The number of messages as extracted by the code below is always 0 although in the gui there are messages.
To reproduce the problem I installed apache-artemis-2.13.0 locally.
We are doing something like the following
if (!jmsUtil.queueHasNotMoreElementsThan(QUEUE_ALMOST_EMPTY_MAX_AMOUNT, reprocessingMessagingProvider.getJmsTemplate())) {
log.info("Queue has too many messages. Will not send more...");
return;
}
jmsUtil is implemented like
public boolean queueHasNotMoreElementsThan(int max, JmsOperations jmsTemplate) {
return Boolean.TRUE.equals(
jmsTemplate.browse((session, queueBrowser) -> {
Enumeration enumeration = queueBrowser.getEnumeration();
return notMoreElemsThan(enumeration, max);
}));
}
private Boolean notMoreElemsThan(Enumeration enumeration, int max) {
for (int i = 0; i <= max; i++) {
if (!enumeration.hasMoreElements()) {
return true;
}
enumeration.nextElement();
}
return false;
}
As a check I used additionally the following method to give me the number of messages in the queue directly.
public int countPendingMessages(String destination, JmsOperations jmsTemplate) {
Integer totalPendingMessages = jmsTemplate.browse(destination,
(session, browser) -> Collections.list(browser.getEnumeration()).size());
int messageCount = totalPendingMessages == null ? 0 : totalPendingMessages;
log.info("Queue {} message count: {}", destination, messageCount);
return messageCount;
}
That method of extracting the queue size seems to be used as well by others and is based on the documentation of QueueBrowser: The getEnumeration method returns a java.util.Enumeration that is used to scan the queue's messages.
Would the above be the correct way on how to obtain the queue size? If so, what could be the cause of the problem? If not, how should the queue size be queried? Does spring offer any other possibility of accessing the queue?
Update: I read another post and the documentation but I wouldn't know on how to obtain the ClientSession.
There are some caveats to using a QueueBrowser to count the number of messages in the queue. The first is noted in the QueueBrowser JavaDoc:
Messages may be arriving and expiring while the scan is done. The JMS API does not require the content of an enumeration to be a static snapshot of queue content. Whether these changes are visible or not depends on the JMS provider.
So already the count may not be 100% accurate.
Then there is the fact that there may be messages still technically in the queue which have been dispatched to a consumer but have not yet been acknowledged. These messages will not be counted by the QueueBrowser even though they may be cancelled back to the queue at any point if the related consumer closes its connection.
Simply put the JMS API doesn't provide a truly reliable way to determine the number of messages in a queue. Furthermore, "Spring JMS" is tied to the JMS API. It doesn't have any other way to interact with a JMS broker. Given that, you'll need to use a provider-specific mechanism to determine the message count.
ActiveMQ Artemis has a rich management API that is accessible though, among other things, specially constructed JMS messages. You can see this in action in the "Management" example that ships with ActiveMQ Artemis in the examples/features/standard/management directory. It demonstrates how to use JMS resources and provider-specific helper classes to get the message count for a JMS queue. This is essentially the same solution as given in the other post you mentioned, but it uses the JMS API rather than the ActiveMQ Artemis "core" API.
I came across call back queue feature in RMQ. And its pretty fancy too. The whole idea is I have created One Message queue (queue1), its callback queue(queue1_cb) and its dlq(queue1_dlq). I am implementing HA feature with 2 nodes.
The problem comes when I am deploying 2 instances of my application(I have one sender and one receiver app in Spring boot). Both are listening to same HA cluster. The scenario is as below.
Sender publishes a message to RMQ.
Receiver app consumes message. Receiver app has to call third party API which is socket based API and its asynchronous so i do not get response in same connection. SO i store object of Channel & Message which i need to ack the message. (Please note i am delaying the ack till i receive response from third party API.
When i deploy 2 instances of receiver app, any instance will get response from third party API. And both will not have object of Channel and Message to ack message and send message to callback queue.
Can any one suggest me a solution on proiority?
Below is my code.
At Receiver side :
#Override
public void onMessage(Message arg0, Channel arg1) throws Exception {
String msg = new String (arg0.getBody());
AppObject obj = mapper.readValue(msg, AppObject.class);
Packet packet = new Packet();
packet.setChannel(arg1);
packet.setMessage(arg0);
packet.setAppObject(obj);
AppParam.objects.put(
String.valueOf(key , packet);
//Call third party API
}
At the time of acking and sending callback message:
public boolean pushMessageToCallBack(String key , AppObject packet, Channel channel, Message message){
RabbitTemplate replyRabbitTemplate = //Get the RabbitTemplate object. It is handled properly.
replyRabbitTemplate.convertAndSend(packet);
channel.basicAck(message.getMessageProperties().getDeliveryTag(), false);
}
You need a different callback queue for each instance or, more simply, just use Direct Reply-to where you don't need a queue at all.
I am building an integration between Salesforce and Twilio that sends/receives SMS using TwilioForce REST API. The main issue is getting around the 10-call API limit from Salesforce, as well as the prohibition on HTTP call outs from a trigger.
I am basing the design on Dan Appleman's Asynchronous Request processes, but in either Batch mode or RequestAsync(), ASync(), Sync(), repeat... I'm still hitting the limits.
I'd like to know how other developers have done this successfully; the integrations have been there for a while, but the examples are few and far between.
Are you sending unique messages for each record that has been updated? If not, then why not send one message to multiple recipients to save on your API limits?
Unfortunately, if you do actually need to send more than 10 unique messages there is no way to send messages in bulk with the Twilio API, you could instead write a simple application that runs on Heroku or some other application platform that you can call out to that will handle the SMS functionality for you.
I have it working now using the following structure (I apologize for the formatting - it's mostly pseudocode):
ASyncRequest object:
AsyncType (picklist: 'SMS to Twilio' is it for now),
Params (long text area: comma-separated list of Ids)
Message object:
To (phone), From (phone), Message (text), Sent (boolean), smsId (string), Error (text)
Message trigger: passes trigger details to CreateAsyncRequests() method.
CreateAsyncRequests: evaluate each new/updated Message__c; if Sent == false for any messages, we create an AsyncRequest, type=SMS to Twilio, Params += ',' + message.Id.
// Create a list to be inserted after all the Messages have been processed
List requests = new List();
Once we reach 5 message.Ids in a single AsyncRequest.Params list, add it to requests.
If all the messages have been processed and there's a request with < 5 Ids in Params, add it to requests as well.
If requests.size() > 0 {
insert requests;
AsyncProcessor.StartBatch();
}
AsyncProcessor implements .Batchable and .AllowsCallouts, and queries ASyncRequest__c for any requests that need to be processed, which in this case will be our Messages list.
The execute() method takes the list of ASyncRequests, splits each Params value into its component Message Ids, and then queries the Message object for those particular Messages.
StartBatch() calls execute() with 1 record at a time, so that each execute() process will still contain fewer than the maximum 10 callouts.
Each Message is processed in a try/catch block that calls SendMessage(), sets Message.smsId = Twilio.smsId and sets Message.Sent = true.
If no smsId is returned, then the message was not sent, and I set a boolean bSidIsNull = true indicating that (at least) one message was not sent.
** If any message failed, no smsIds are returned EVEN FOR MESSAGES THAT WERE SUCCESSFUL **
After each batch of messages is processed, I check bSidIsNull; if true, then I go back over the list of messages and put any that do not have an smsId into a map indexed by the Twilio number I'm trying to send them From.
Since I limited each ASyncRequest to 5 messages, I still have the use of a callout to retrieve all of the messages sent from that Twilio.From number for the current date, using
client.getAccount().getMessages('From' => fromNumber, 'DateSent' => currentDate)
Then I can update the Message.smsIds for all of the messages that were successful, and add an error message to Message.Error_on_Send__c for any that failed.