I have a PublishSubscribeChannel in my application, which should deliver messages to different MessageHandlers inside the same JVM. Handlers are subscribed to the channel using #StreamListener annotation. Channel uses Executors so delivery is asynchronous.
Now, I want to test that senders and handlers agree on the specific object type which send through channel (the type of Message body). AFAIU I have two ways to test this:
Find all subscribers of the given channel and verify their
signature.
Send a message to a channel and verify that no handlers have thrown an exception.
I have no idea how to do (1). And I think I could do (2) by listening to errorChannel (there should be no messages there), but I don't quite understand how long should I wait for error messages.
Any suggestions?
For 1, you can use reflection to look at the collection of handlers in the channel's dispatcher; then use reflection again to look at the hander's Method.
However, your design is flawed, unless you don't mind losing messages; the incoming message will be ack'd as soon as you hand off to the executor; if the server then crashes, the message will be lost.
If you get rid of the executor, it would be simpler to add an interceptor to the channel, which will be notified of any exceptions in its afterSendCompletion() method (satisfying your 2).
Related
I am using github.com/nsqio/go-nsq Go package to work with NSQ, and I've met following problem. When producer writes a message - it creates a topic, but not a channel, and it seems that NSQ server just discards a messages when there are no channels. I don't want to lose messages, also I don't want to rely on consumer and producer startup order. So a possible solution is to create a channel on producer side, so when it writes it will not discard messages. How can I create and destroy a consumer without consuming a message, just for sake of channel creation?
sequence of
nsq.NewConsumer()
consumer.AddConcurrentHandlers()
consumer.ConnectToNSQLookupd()
can consume
removing AddConcurrentHandlers leads to error
From protocol spec I cannot see how is it supposed to do, other that sending SUB followed by CLS, but because those are two commands and non "atomic" kind of op - something theoretically may happen in between..
So probably I am trying to do that wrong? RABBITMQ for example can pre-create it, is here something like that?
I have a workflow whose message payload (MasterObj) is being enriched several times. During the 2nd enrichment () an UnknownHostException was thrown by an outbound gateway. My error channel on the enricher is called but the message the error-channel receives is an exception, and the failed msg in that exception is no longer my MasterObj (original payload) but it is now the object gotten from request-payload-expression on the enricher.
The enricher calls an outbound-gateway and business-wise this is optional. I just want to continue my workflow with the payload that I've been enriching. The docs say that the error-channel on the enricher can be used to provide an alternate object (to what the enricher's request-channel would return) but even when I return an object from the enricher's error-channel, it still takes me to the workflow's overall error channel.
How do I trap errors from enricher's + outbound-gateways, and continue processing my workflow with the same payload I've been working on?
Is trying to maintain a single payload object for the entire workflow the right strategy? I need to be able to access it whenever I need.
I was thinking of using a bean scoped to the session where I store the payload but that seems to defeat the purpose of SI, no?
Thanks.
Well, if you worry about your MasterObj in the error-channel flow, don't use that request-payload-expression and let the original payload go to the enricher's sub-flow.
You always can use in that flow a simple <transformer expression="">.
On the other hand, you're right: it isn't good strategy to support single object through the flow. You carry messages via channel and it isn't good to be tied on each step. The Spring Integration purpose is to be able to switch from different MessageChannel types at any time with small effort for their producers and consumers. Also you can switch to the distributed mode when consumers and producers are on different machines.
If you still need to enrich the same object several times, consider to write some custom Java code. You can use a #MessagingGateway on the matter to still have a Spring Integration gain.
And right, scope is not good for integration flow, because you can simply switch there to a different channel type and lose a ThreadLocal context.
I want to validate an incoming message, using FluentValidation in my case, and if it fails it should return immediately. I looked into http://docs.masstransit-project.com/en/latest/usage/observers.html, and in my case, I like the idea of
public class ConsumeObserver : IConsumeObserver
{
Task IConsumeObserver.PreConsume<T>(ConsumeContext<T> context)
{
//1.Validate here
//2. If success go on to consumer
//3. If fails exit with the result of validation and don't go through consumer.
}
Task IConsumeObserver.PostConsume<T>(ConsumeContext<T> context)
{
}
Task IConsumeObserver.ConsumeFault<T>(ConsumeContext<T> context, Exception exception)
{
}
}
because I get the message already deserialized and so is easy to use the validator. The problem is that I don't know how to return without going through consumer and a the same time keep the validation errors.
Thank you.
Observers typically watch versus take action, and that's the approach with observers in MassTransit. While you could throw an exception from the PreConsume method, which would cause the message to either retry or be sent to the error queue, it's not the most obvious behavior to developers down the road who may not understand why the message is failing.
Another approach would be to create middleware component that can validate the message, and if it's not valid, perform a specific action on the message (such as moving it to an invalid queue, or dumping it to a log, or whatever) so that the message is removed from the queue. It's important to understand how this might impact the message producer.
For instance, if it was a request message, and the sender is waiting on a response, discarding the message means that no response will be received. The default behavior of a consumer that throws an exception is to propagate the fault back to the requestor, completing the cycle, so keep that in mind.
Another option is to just add the validation behavior to the consumer, using either an injected validation interface, or within the consumer itself. That way, the handling of the message is close to the consumer which improves code cohesion and makes it easy to see what is happening.
Ideally, validating at the message producer is the best option, to avoid flooding the queue with invalid messages. So that's another option.
So, several choices, your requirements will dictate which makes the most sense.
I'm trying to handle two different types of problems while processing a message.
The first problem is if the remote database is down. In that case, the message should stop processing, and try again later. This message should never go to a DLQ, and should keep trying until the remote database is up.
The second problem is when there is a problem with the message. In that case, it should go to the DLQ.
How should I be structuring the following code?
#Override
public void onMessage(Message message) {
try {
// Do some processing
messageProcessing(message); // Should DLQ if message is bad
// Save to the database
putNamedLocation(message); // <<--- Exception when external DB is down
} catch (Exception e) {
logger.error(e.getMessage());
mdc.setRollbackOnly();
}
}
Assuming you can detect bad messages definitively in the code body of the MDB, I would write the bad messages to the DLQ directly. This gives you a bit more freedom to perhaps categorize the error and optionally send different types of bad messages to different "DLQ-Like" queues, and/or apply a time-to-live to DLQ'ed messages so that no-hope-of-ever-being-processed type messages don't pile up in the queue for ever. You can add #Resource annotated instance variables to your MDB class referencing the ConnectionFactory and Queue references to support the sending of the messages to the target DLQ. The bottom line is, make sure you detect the error and DLQ the message yourself.
As for the DB being down, you can detect this by catching exceptions when acquiring a connection or writing your updates. In this case, clean up your resources and throw a RuntimeException. This will cause the message to be redelivered, but you will want to check the JMS configuration for two things:
Make sure the max-redelivery count is high enough, otherwise the count will tick over and the message will be DLQed eventually anyway.
If your JMS implementation supports it, add a redelivery delay to rejected messages to allow some time for the DB to come back up, otherwise your messages will endlessly spin in a deliver/reject loop.
To avoid #2 (which is tricky if your JMS implementation does not support redilvery delay, like WebSphereMQ), you can use the JBoss JMX management interface for the MDB to stop (and later restart) delivery on the MDB. However, you can't do this inside the MDB in the same thread that is processing the message because the MDB will wait for the message to complete processing, which it can't because it is waiting for the MDB to stop, which it can't because...[and so on] so... your best bet is to start some sort of sentry that polls the DB and when it finds it down, stops the MDB and when it finds it up again, restarts it. See this question for a snippet on how to do that.
That last part should help deal with any unexpected exceptions resulting from message validations. (i.e. the DB is fine, but for some reason the message is totally fubar resulting in uncaught exceptions which causes the message to be redelivered). Since down-DB messages should not be redelivered more than a few times (on account of your sentry), you can check a message's redelivery count and if it is ridiculously high then you know you have poison message and you can ditch it, or DLQ it.
Hope that's helpful.
I have a single AMQ queue that receives simple messages with string body. Consider I'm sending CLSIDs as message bodies. CLSIDs could be not unique, but I'd like to reject all messages with not unique bodies and keep only single instance of such messages in the queue. Is there any simple way to do it?
Currently I'm using a workaround. Messages from the queue are consumed by some processor that tries to insert bodies into a simple DB table with UNIQUE constraint applied to message_body field. If processor inserts the messages succesfuly - it's assigned to exchange.out.body and sent to other queue. If ConstraintViolationException is thrown - nothing is resent to other queue.
I would like to know does AMQ support something similar out of the box?
I believe you can write an interceptor for activemq where you can perform certain actions on messages. Check out: http://activemq.apache.org/interceptors.html
That being said, in my personal opinion this is bad practice. ActiveMQ is a messaging system which should only be responssible for transport of the message. All logic can beter be performed using your application ( either make sure the sender cannot send the same message more then once OR , create an intermediate consumer which indeed matches the received body with a database that contains already seen message bodies BEFORE, routing the message to the actual receiver queue)