What is the best practice for handling exceptions in MassTransit 3+ with regard to Request/Response pattern? The docs here mention that if a ResponseAddress exists on a message, the Fault message will be sent to that address, but how does one consumer/receive the messages at that address? The ResponseAddress for Bus.Request seems to be an auto-generated MassTransit address that I don't have control over, so I don't know how to access the exception thrown in the main consumer. What am I missing? Here's my code to register the consumer and its fault consumer using Unity container:
cfg.ReceiveEndpoint(host, "request_response_queue", e =>
{
e.Consumer<IConsumer<IRequestResponse>>(container);
e.Consumer(() => container.Resolve<IMessageFaultConsumer<IRequestResponse>>() as IConsumer<Fault<IRequestResponse>>);
});
And here's my attempt at a global message fault consumer:
public interface IMessageFaultConsumer<TMessage>
{
}
public class MessageFaultConsumer<TMessage> : IConsumer<Fault<TMessage>>, IMessageFaultConsumer<TMessage>
{
public Task Consume(ConsumeContext<Fault<TMessage>> context)
{
Console.WriteLine("MessageFaultConsumer");
return Task.FromResult(0);
}
}
This approach DOES work when I use Bus.Publish as opposed to Bus.Request. I also looked into creating an IConsumeObserver and putting my global exception logging code into the ConsumeFault method, but that has the downside of being invoked every exception prior to the re-tries giving up. What is the proper way to handle exceptions for request/response?
First of all, the request/response support in MassTransit is meant to be used with the .Request() method, or the request client (MessageRequestClient or PublishRequestClient). With these methods, if the consumer of the request message throws an exception, that exception is packaged into the Fault<T>, which is sent to the ResponseAddress. Since the .Request() method, and the request client are both asynchronous, using await will throw an exception with the exception data from the fault included. That's how it is designed, await the request and it will either complete, timeout, or fault (throw an exception upon await).
If you are trying to put in some global "exception handler" code for logging purposes, you really should log those at the service boundary, and an observer is the best way to handle it. This way, you can just implement the ConsumeFault method, and log to your event sink. However, this is synchronous within the consumer pipeline, so recognize the delay that could be introduced.
The other option is to of course just consume Fault<T>, but as you mentioned, it does not get published when the request client is used with the response address in the header. In this case, perhaps your requester should publish an event indicating that operation X faulted, and you can log that -- at the business context level versus the service level.
There are many options here, it's just choosing the one that fits your use case best.
Related
How can I cancel the execution of a Consume method of an IConsumer<>? I tried to create a cancellation token and use another thread to trigger the cancellation, and I can get a ConsumerCancelledException, but I could not handle the Fault<> message. E.g.
I have a MyDataConsumer: IConsumer<MyData> and MyDataFaultConsumer: IConsumer<Fault<MyData>>: IConsumer<Fault<MyData>>
The IConsumer<MyData> throws an ConsumerCancelledException
The failed message went into the MyData_error queue, but it did not go to the MyDataFault queue
However, if I directly throw an exception in my MyDataConsumer, my fault consumer can get the data. It seems the ConsumerCancelledException is handled differently than other exceptions.
We have implemented sqslistner as the documentation suggests, the best way to receive AWS SQS message Cloud Spring Doc.
There are two ways for receiving SQS messages, either use the receive
methods of the QueueMessagingTemplate or with annotation-driven
listener endpoints. The latter is by far the more convenient way to
receive messages.
Everything is working as expected. If business process failed, we throw a runtime exception. The particular message is sent back to the SQS queue for retry. When visibility timeout passed the message reappears to the worker for processing.
Sample Code is here:
#SqsListener(value="sample-standard-queue",deletionPolicy = SqsMessageDeletionPolicy.ON_SUCCESS)
public void receiveMessage(String message) {
log.info("Message Received **************************** "+message );
log.info("After Conversion"+new JSONObject(message).getString("payload"));
throw new RuntimeException("An exception was thrown during the execution of the SQS listener method and Message will be still available in Queue");
}
But there are some examples where "Acknowledgment" is used instead of throwing run time exception. Documentation doesn't suggest that.
Which one is the best way to deal with a business logic failure scenario?Is Acknowledgment necessary?
Thanks in advance.
One way is to keep a track of messages being processed in some RDS table. If any message gets retried then increase the retry count in the table for that particular message.
There should be some configured numbers of retries that you want to retry one particular message and then you may want to move that to a dead-letter-queue or you may log it and just simply discard it.
There can be multiple ways of handling it: One way can be:
#SqsListener(value="sample-standard-queue",deletionPolicy = SqsMessageDeletionPolicy.ON_SUCCESS)
public void receiveMessage(String message) {
try{
log.info("Message Received **************************** "+message );
log.info("After Conversion"+new JSONObject(message).getString("payload"));
}catch(Exception e){
// check if its retry count has exhausted or not
// if exhausted - then acknowledge it (push it into dead-letter-queue) and dont throw the exception
// If not exhausted - increase the retry count in the table before throwing exception
throw new RuntimeException("An exception was thrown during the execution of the SQS listener method and Message will be still available in Queue");
}
}
In a spring boot application, I have a class with jms listener.
public class PaymentNotification{
#JmsListener(destination="payment")
public void receive(String payload) throws Exception{
//mapstring conversion
....
paymentEvent = billingService.insert(paymentEvent); //transactional method
//call rest...
billingService.save(paymentEvent);
//send info to jms
}
}
I saw then when a error happen, data is inserted in the database, that ok, but it's like receive method is called again and again... but queue is empty when I check on the server.
If there is an error, I don't want method is called again, Is there something for that.
The JMS Message Headers might contain additional information to help with your processing. In particular JMSRedelivered could be of some value. The Oracle doc states that "If a client receives a message with the JMSRedelivered field set, it is likely, but not guaranteed, that this message was delivered earlier but that its receipt was not acknowledged at that time."
I ran the following code to explore what was available in my configuration (Spring Boot with IBM MQ).
#JmsListener(destination="DEV.QUEUE.1")
public void receive(Message message) throws Exception{
for (Enumeration<String> e = message.getPropertyNames(); e.hasMoreElements();)
System.out.println(e.nextElement());
}
From here I could find JMSXDeliveryCount is available in JMS 2.0. If that property is not available, then you may well find something similar for your own configuration.
One strategy would be to use JMSXDeliveryCount, a vendor specific property or maybe JMSRedelivered (if suitable for your needs) as a way to check before you process the message. Typically, the message would be sent to a specific blackout queue where the redelivery count exceeds a set threshold.
Depending on the messaging provider you are using it might also be possible to configure back out queue processing as properties of the queue.
I have an xamarin.android with xamarin.insights intergrated.
Right now every time I handle error manually (try/catch) I'm adding information about environment (staging/production):
try
{
ExceptionThrowingFunction();
}
catch (Exception exception)
{
exception.Data["Environment"] = "staging";
throw;
}
But this information is missing in case if error handled by xamarin.insights itself (in case of crash).
It is possible to add additional exception data in case of crash?
docs reference I used
From reading the docs page reference that you mentioned, I still get the impression that you have to call the .Report method as well as in:-
Insights.Report(exception, new Dictionary <string, string> {
{"Some additional info", "foobar"}
});
What I believe they are saying in this example:-
try {
ExceptionThrowingFunction();
}
catch (Exception exception) {
exception.Data["AccountType"] = "standard";
throw;
}
Is that you have the ability when any Exception is encountered, to package additional information that you can later send to the Insights server, as the Data property of the Exception is just a Key/Value Dictionary.
So if you had an Exception several layers deep, you can choose to re-throw the Exception with additional information contained within it that you will later send to the Insights server.
At a higher level, you can then take the Exception that was thrown deeper down the call-hierarchy and then call the Insights.Report, with:-
Insights.Report(
{the rethrown exception in your higher up try..catch block},
{rethrown exception}.Data
);
that will then send all the additional Key/Value information previously captured.
From seeing your last part of your question though it looks like you are interested in Insights handling and sending this additional .Data automatically should there be an unhandled exception.
If it is not currently being sent, then perhaps suggest to them that this can be sent also? As it sounds a feasible request for this to automatically be sent as well incase of an unhandled exception.
Update 1:-
Yes - I understand about the unhandled exception scenario now that you are referring to.
I have not dealt with this component directly, so there may be hooks / event handlers or something already defined where you can tap into this, and execute some custom code just prior to this being sent.
If this is not available, then perhaps suggest this to them to include as its a Beta product?
Alternatively, you could still achieve this yourself by capturing the unhandled exceptions just prior to them falling. You'd have to code this however on each platform.
For instance on Windows Phone in the App class there is Application_UnhandledException(object sender, ApplicationUnhandledExceptionEventArgs e) to which you could then supplement the Exception thrown with this extra .Data.
For Android you could take a look at this post that describes how to catch uncaughtException that will help you in capturing the unhandled exceptions.
Whether just supplementing the Exception in these handlers above is enough all depends on how they've written their hook into this, as to how well it behaves and whether it is executed first, prior to their implementation.
You will have to try and see if it does. If it doesn't behave well, allowing you to supplement extra data prior to the automatic call to Insights, you have another fallback solution, to just do the .Report call manually within these unhandled exception handlers yourself to make this work and supplement the extra .Data to achieve your aim.
I am performing some simple tests with ActiveMQ to see how it performs on a non stable network. The first test consists in a producer that sends messages to a remote queue. The message is of type ObjectMessage with serializable content inside (a list of Objects).
With a good network everything works correctly, but when I launch the same tests using netem to simulate packages losses, delays and corruptions I get the following error when consuming the messages when trying to extract the content of the Message:
2011-03-16 11:59:21,791 ERROR [com.my.MessageConsumer] Failed to build body from bytes. Reason: java.io.StreamCorruptedException: invalid handle value: 017E0007
javax.jms.JMSException: Failed to build body from bytes. Reason: java.io.StreamCorruptedException: invalid handle value: 017E0007
So it seems like the message was corrupted while sending to the remote Queue but anyway stored, and only when is consumed the consumer see that the message is corrupted.
After this I will use a local Queue and a Network Connector to forward the messages to the remote Queue, and that I hope it solve the problem, but I was surprised that there was not any kind of validation between the producer and the destination (at least a checksum or something like that) that guarantees a correct delivery, am I doing something wrong or is the normal behaviour?
I don't have the code here right now, but it was super simple, just a MessageListener:
public class myMessageConsumer implements MessageListener{
public void onMessage(Message message){
try
{
if (message instanceof ObjectMessage){
ObjectMessage myMessage = (ObjectMessage) message;
List dtoList = (List) myMessage.getObject();
}
} catch(Exception ex){
ex.printStackTrace();
}
}
}
If the exact code is needed I'll put it when I go back from holidays, but it was exactly like that.
The broker isn't going to validate the contents of each and every message that it processes, that would be a tremendous waste of time and slow down message dispatch significantly. The client received a bad message and threw a JMSException to indicate that the message contents were corrupted which should be sufficient for your app to respond correctly.
Where's your code?
If that exception comes from your code, seems like it's possible that you've got a bug. For example, getting some JMS error receiving the message but messing up error handling and trying to process the results anyway. For a test like you describe, you'd need a good focus on error handling in your clients.
I don't have experience w/ ActiveMQ, but it seems very surprising that it'd allow corrupt message delivery. Not that I'm wanting the JMS implementation to unpack the ObjectMessage to check. Just that it should deliver a byte-for-byte uncorrupted copy of what was sent. Or error out if it can't.