I am currently using Akka-Camel for integrating my Akka based application with various message queues.
Now that I want to upgrade to the latest version of Akka I see that this integration is now deprecated.
I have tried using alpakka-jsm integration but am unable to see whether I can perform explicit ack to the message queues once I have processed the message. Does this functionality exist in this new component?
Alpakka 0.15 makes the acknowledgement mode in the JMS connector configurable. An example that's adapted from the linked documentation:
val jmsSource: Source[Message, NotUsed] = JmsSource(
JmsSourceSettings(connectionFactory)
.withQueue("myqueue")
.withAcknowledgeMode(AcknowledgeMode.ClientAcknowledge)
)
val result = jmsSource
.map {
case textMessage: TextMessage =>
val text = textMessage.getText
textMessage.acknowledge()
text
}
.runWith(Sink.seq)
Related
In a spring boot application using artemis we try to avoid queues containing too many messages. The intention is to only put in new messages if the number of messages currently in the queue falls below a certain limit, e.g. 100 messages. However, that seems not to work but we don't know why or what the "correct" method would be to implement that functionality. The number of messages as extracted by the code below is always 0 although in the gui there are messages.
To reproduce the problem I installed apache-artemis-2.13.0 locally.
We are doing something like the following
if (!jmsUtil.queueHasNotMoreElementsThan(QUEUE_ALMOST_EMPTY_MAX_AMOUNT, reprocessingMessagingProvider.getJmsTemplate())) {
log.info("Queue has too many messages. Will not send more...");
return;
}
jmsUtil is implemented like
public boolean queueHasNotMoreElementsThan(int max, JmsOperations jmsTemplate) {
return Boolean.TRUE.equals(
jmsTemplate.browse((session, queueBrowser) -> {
Enumeration enumeration = queueBrowser.getEnumeration();
return notMoreElemsThan(enumeration, max);
}));
}
private Boolean notMoreElemsThan(Enumeration enumeration, int max) {
for (int i = 0; i <= max; i++) {
if (!enumeration.hasMoreElements()) {
return true;
}
enumeration.nextElement();
}
return false;
}
As a check I used additionally the following method to give me the number of messages in the queue directly.
public int countPendingMessages(String destination, JmsOperations jmsTemplate) {
Integer totalPendingMessages = jmsTemplate.browse(destination,
(session, browser) -> Collections.list(browser.getEnumeration()).size());
int messageCount = totalPendingMessages == null ? 0 : totalPendingMessages;
log.info("Queue {} message count: {}", destination, messageCount);
return messageCount;
}
That method of extracting the queue size seems to be used as well by others and is based on the documentation of QueueBrowser: The getEnumeration method returns a java.util.Enumeration that is used to scan the queue's messages.
Would the above be the correct way on how to obtain the queue size? If so, what could be the cause of the problem? If not, how should the queue size be queried? Does spring offer any other possibility of accessing the queue?
Update: I read another post and the documentation but I wouldn't know on how to obtain the ClientSession.
There are some caveats to using a QueueBrowser to count the number of messages in the queue. The first is noted in the QueueBrowser JavaDoc:
Messages may be arriving and expiring while the scan is done. The JMS API does not require the content of an enumeration to be a static snapshot of queue content. Whether these changes are visible or not depends on the JMS provider.
So already the count may not be 100% accurate.
Then there is the fact that there may be messages still technically in the queue which have been dispatched to a consumer but have not yet been acknowledged. These messages will not be counted by the QueueBrowser even though they may be cancelled back to the queue at any point if the related consumer closes its connection.
Simply put the JMS API doesn't provide a truly reliable way to determine the number of messages in a queue. Furthermore, "Spring JMS" is tied to the JMS API. It doesn't have any other way to interact with a JMS broker. Given that, you'll need to use a provider-specific mechanism to determine the message count.
ActiveMQ Artemis has a rich management API that is accessible though, among other things, specially constructed JMS messages. You can see this in action in the "Management" example that ships with ActiveMQ Artemis in the examples/features/standard/management directory. It demonstrates how to use JMS resources and provider-specific helper classes to get the message count for a JMS queue. This is essentially the same solution as given in the other post you mentioned, but it uses the JMS API rather than the ActiveMQ Artemis "core" API.
I am using ActiveMQ for messaging and there is one requirement that if message is duplicate then it should handled by AMQ automatically.
For that I generate unique message key and set to messageproccessor.
following is code :
jmsTemplate.convertAndSend(dataQueue, event, messagePostProccessor -> {
LocalDateTime dt = LocalDateTime.now();
long ms = dt.get(ChronoField.MILLI_OF_DAY) / 1000;
String messageUniqueId = event.getResource() + event.getEntityId() + ms;
System.out.println("messageUniqueId : " + messageUniqueId);
messagePostProccessor.setJMSMessageID(messageUniqueId);
messagePostProccessor.setJMSCorrelationID(messageUniqueId);
return messagePostProccessor;
});
As it can be seen code generates unique id and then set it to messagepostproccessor.
Can somehelp me on this, is there any other configuration that I need do.
A consumer can receive duplicate messages mainly for two reasons: a producer sent the same message more times or a consumer receive the same message more times.
Apache ActiveMQ Artemis includes powerful automatic duplicate message detection, filtering out messages sent by a producer more times.
To prevent a consumer from receiving the same message more times, an idempotent consumer must be implemented, ie Apache Camel provides an Idempotent consumer component that would work with any JMS provider, see: http://camel.apache.org/idempotent-consumer.html
I've been trying my hands on Spring AMQP. And I have a couple of questions:
I'd like to know what is Publisher returns and how is it different from Publisher Confirm. Of my understanding, we have a Publisher Confirm Callback that checks the status of acks. Now I looked at the documentation in Spring AMQP and Rabbit MQ. didn't really find or understand much on this.
And also why is it that if the message is tried to send to a non-existing queue, I don't get any sort of acknowledgement (ack/nack
) nor do I get any errors. Is there a way to setTimeouts for non-acknowledgements?
Short answer from the link https://www.rabbitmq.com/confirms.html :
"For unroutable messages, the broker will issue a confirm once the exchange verifies a message won't route to any queue (returns an empty list of queues). If the message is also published as mandatory, the basic.return is sent to the client before basic.ack."
In Spring AMQP if you set 'spring.rabbitmq.publisherReturns' to true this will mean messages will be 'mandatory' (unless you set mandatory to false) because of the following code:
private boolean determineMandatoryFlag() {
Boolean mandatory = this.properties.getTemplate().getMandatory();
return (mandatory != null ? mandatory : this.properties.isPublisherReturns());
}
I suggest you to read this article. There is a good description of all possible acknowledgments scenarios, including returns for the unrouted messages, like your non-existing queue.
From the Spring AMQP perspective you should bear in mind: https://docs.spring.io/spring-amqp/docs/2.0.3.RELEASE/reference/html/_reference.html#template-confirms
This feature requires a CachingConnectionFactory that has its publisherReturns property set to true.
I have a client app that connects to an Oracle AQ multi-consumer queue. I want to use OracleAQQueue.Listen to listen for new messages on the queue. API docs show that the Listen method can be used for multi-consumer queues. My code for listening to the queue is shown below.
string consumerName = "APPINST1";
using (OracleConnection con = new OracleConnection(connectionString))
{
con.Open();
OracleAQQueue queue = new OracleAQQueue("MY_Q");
queue.MessageType = OracleAQMessageType.Udt;
queue.UdtTypeName = "MY_Q_MSG";
queue.DequeueOptions.DeliveryMode = OracleAQMessageDeliveryMode.Persistent;
queue.Connection = con;
Console.WriteLine("Listening for messages...");
queue.Listen(new string[] { consumerName });
}
The problem that I'm having is that on the line of code where I call queue.Listen(), I get the Oracle exception:
ORA-25295: Subscriber is not allowed to dequeue buffered messages
Googling for advice on this particular error hasn't been too helpful. I've removed and re-added my subscriber to the queue several times to no avail. My guess is that I'm not setting some property correctly before I make the call to Listen, but I can't figure out the issue.
Any ideas?
I ran across the following note in the Streams Advanced Queuing User's Guide, in Chapter 10 - Oracle Streams AQ Operations Using PL/SQL:
Note: Listening to multiconsumer queues is not supported in the Java API.
Although I can't find it explicitly stated anywhere, I'm guessing the same rule applies to the ODP.NET API.
You must set the visibility attribute to IMMEDIATE to use buffered messaging.
My experimental application is quite simple, trying what can be done with Actors and Akka.
After JVM start, it creates actor system with couple of plain actors, JMS consumer (akka.camel.Consumer) and JMS producer (akka.camel.Producer). It sends couple of messages between actors and also JMS producer -> JMS server -> JMS consumer. It basically talks to itself via JMS service.
From time to time I was experiencing weird behaviour: it seemed that from time to time, first of messages which where supposed to be sent to JMS server was somehow lost. By looking at my application logs, I could see that applications is trying to send the message but it was never received by JMS server. (For each run I have to start JVM&Application again).
Akka Camel Documentation mentions that it's possible that some components may not be fully initialized at the begining: "Some Camel components can take a while to startup, and in some cases you might want to know when the endpoints are activated and ready to be used."
I tried to implement following to wait for Camel initialization
val system = ActorSystem("actor-system")
val camel = CamelExtension(system)
val jmsConsumer = system.actorOf(Props[JMSConsumer])
val activationFuture = camel.activationFutureFor(jmsConsumer)(timeout = 10 seconds, executor = system.dispatcher)
val result = Await.result(activationFuture,10 seconds)
which seems to help with this issue. (Although, when removing this step now, I'm not able to recreate this issue any more... :/).
My question is whether this is correct way to ensure all components are fully initialized?
Should I use
val future = camel.activationFutureFor(actor)(timeout = 10 seconds, executor = system.dispatcher)
Await.result(future, 10 seconds)
for each akka.camel.Producer and akka.camel.Consumer actor to be sure that everything is initialized properly?
Is that all I should to do, or something else should be done as well? Documentation is not clean on that and it's not easy to test as issue was happening only occasionaly...
You need to initialize the camel JMS component and also Producer before sending any messages.
import static java.util.concurrent.TimeUnit.SECONDS;
import scala.concurrent.Future;
import scala.concurrent.duration.Duration;
import akka.dispatch.OnComplete;
ActorRef producer = system.actorOf(new Props(SimpleProducer.class), "simpleproducer");
Timeout timeout = new Timeout(Duration.create(15, SECONDS));
Future<ActorRef> activationFuture = camel.activationFutureFor(producer,timeout, system.dispatcher());
activationFuture.onComplete(new OnComplete<ActorRef>() {
#Override
public void onComplete(Throwable arg0, ActorRef arg1)
throws Throwable {
producer.tell("First!!");
}
},system.dispatcher());