I have used the sample found at
https://github.com/MassTransit/MassTransit/tree/develop/src/Samples/Sample.AzureFunctions.ServiceBus. The function correctly executes when a message hits the Queue, however the consume code registered by the following never gets executed...
cfg.Consumer(() => new SubmitOrderConsumer());
the consume class
public class SubmitOrderConsumer :
IConsumer<SubmitOrder>
{
public Task Consume(ConsumeContext<SubmitOrder> context)
{
LogContext.Debug?.Log("Processing Order: {OrderNumber}", context.Message.OrderNumber);
context.Publish<OrderReceived>(new
{
context.Message.OrderNumber,
Timestamp = DateTime.UtcNow
});
return context.RespondAsync<OrderAccepted>(new {context.Message.OrderNumber});
}
}
Is there anything else I could be missing, is this supported in .net Core 3?
Thanks
Related
I Noticed a weird issue in one of our applications, from time to time, we get MessageNotConsumedException errors on API requests which we route via MT's Mediator.
As you will notice below, we have configured a customer LogFilter<T> which implements IFilter<ConsumeContext<T>> which ensure that we log each mediator message before and after consuming, or a 'ConsumeFailed' log in case an exception is thrown in any consumer.
When the error manifests itself, in the logs we see the following sequence of events:
T 0 : PreConsume logged
T +5ms: PostConsume logged
T +6ms: R-FAULT logged (I believe this logging is made by MT's internals?)
T +9ms: API Request 500 response logged, with `MessageNotConsumedException` as internal error
In the production environment, we see these errors with various timings, it happens in requests taking as 'little' as 9ms, over several seconds up to 30+ seconds.
I've trying to reproduce this problem in my local development environment, and did manage to produce the same sequence of events, but only by adding a delay of 35 seconds inside the consumer (see GetSomethingById class below for consumer body)
If I reduce the delay to 30s or less, the reponse will be fine.
Since the production errors are happening with very low handling times in the consumer, I suspect what I'm able to reproduce is not exactly the same.
However I'd still like to understand why I'm getting the MessageNotConsumedException, since while debugging I can easily step through my entire consumer (after the delay has elapsed) and happily reach the context.RespondAsync() call without any problems. Also while stepping through the consumer, the context.CancellationToken has not been cancelled.
I also came across this question, which sounds exactly like what I'm having, however I did add the HttpContext scope as documented. To be fair, I didn't try this change in production yet, but my local issue with the 35s delay remains unchanged.
I have MassTransit medatior configured as follows:
services.AddHttpContextAccessor();
services.AddMediator(x =>
{
x.AddConsumer<GetSomethingByIdHandler>();
x.ConfigureMediator((context, cfg) =>
{
//The order of using the middleware matters, so don't change this
cfg.UseHttpContextScopeFilter(context); // Extension method & friends copy/pasted from https://masstransit-project.com/usage/mediator.html#http-context-scope
cfg.UseConsumeFilter(typeof(LogFilter<>), context);
});
});
The LogFilter which is configured is the following class:
public class LogFilter<T> : IFilter<ConsumeContext<T>> where T : class
{
private readonly ILogger<LogFilter<T>> _logger;
public LogFilter(ILogger<LogFilter<T>> logger)
{
_logger = logger;
}
public void Probe(ProbeContext context) => context.CreateScope("log-filter");
public async Task Send(ConsumeContext<T> context, IPipe<ConsumeContext<T>> next)
{
LogPreConsume(context);
try
{
await next.Send(context);
}
catch (Exception exception)
{
LogConsumeException(context, exception);
throw;
}
LogPostConsume(context);
}
private void LogPreConsume(ConsumeContext context) => _logger.LogInformation(
"{MessageType}:{EventType} correlated by {CorrelationId} on {Address}"
+ " with send time {SentTime:dd/MM/yyyy HH:mm:ss:ffff}",
typeof(T).Name,
"PreConsume",
context.CorrelationId,
context.ReceiveContext.InputAddress,
context.SentTime?.ToUniversalTime());
private void LogPostConsume(ConsumeContext context) => _logger.LogInformation(
"{MessageType}:{EventType} correlated by {CorrelationId} on {Address}"
+ " with send time {SentTime:dd/MM/yyyy HH:mm:ss:ffff}"
+ " and elapsed time {ElapsedTime}",
typeof(T).Name,
"PostConsume",
context.CorrelationId,
context.ReceiveContext.InputAddress,
context.SentTime?.ToUniversalTime(),
context.ReceiveContext.ElapsedTime);
private void LogConsumeException(ConsumeContext<T> context, Exception exception) => _logger.LogError(exception,
"{MessageType}:{EventType} correlated by {CorrelationId} on {Address}"
+ " with sent time {SentTime:dd/MM/yyyy HH:mm:ss:ffff}"
+ " and elapsed time {ElapsedTime}"
+ " and message {#message}",
typeof(T).Name,
"ConsumeFailure",
context.CorrelationId,
context.ReceiveContext.InputAddress,
context.SentTime?.ToUniversalTime(),
context.ReceiveContext.ElapsedTime,
context.Message);
}
I then have a controller method which looks like this:
[Route("[controller]")]
[ApiController]
public class SomethingController : ControllerBase
{
private readonly IMediator _mediator;
public SomethingController(IMediator mediator)
{
_mediator = mediator;
}
[HttpGet("{somethingId}")]
public async Task<IActionResult> GetSomething([FromRoute] int somethingId, CancellationToken ct)
{
var query = new GetSomethingByIdQuery(somethingId);
var response = await _mediator
.CreateRequestClient<GetSomethingByIdQuery>()
.GetResponse<Something>(query, ct);
return Ok(response.Message);
}
}
The consumer which handles this request is as follows:
public record GetSomethingByIdQuery(int SomethingId);
public class GetSomethingByIdHandler : IConsumer<GetSomethingByIdQuery>
{
public async Task Consume(ConsumeContext<GetSomethingByIdQuery> context)
{
await Task.Delay(35000, context.CancellationToken);
await context.RespondAsync(new Something{Name = "Something cool"});
}
}
MessageNotConsumedException is thrown when a message is sent using mediator and that message is not consumed by a consumer. That wouldn't typically be a transient error since one would expect that the consumer remains configured/connected to the mediator for the lifetime of the application.
I'm switching some code which uses MassTransit (v7.2.2 on .NET 5) to use a more declarative format (and away from multiple calls to ReceiveEndpoint()) and ideally to using ConsumerDefinitions for the configuration (though not part of this example for simplicity), along with some Dependency Injection with Quartz.NET (yanked from this example, though it running 3.3.3), in doing so I find now that my Consumers are not consuming, despite messages being sent and examples referenced. Take the following standing up of the MassTransit service:
var hostBuilder = Host.CreateDefaultBuilder(args)
.ConfigureServices((hostContext, services) =>
{
services.AddMassTransit(mt =>
{
mt.AddBus(provider => Bus.Factory.CreateUsingInMemory(cfg =>
{
//cfg.AutoStart = true; //No change when on
cfg.UseInMemoryOutbox();
cfg.ConfigureEndpoints(provider);
}));
mt.AddConsumer<TheMessageConsumer>();
services.AddMediator(cfg =>
{
cfg.AddConsumer<TheMessageConsumer>();
});
});
services.AddMassTransitHostedService();
});
var host = hostBuilder.Build();
var busControl = host.Services.GetService<IBusControl>();
busControl.Start(); //Just in case
var message = new TheMessage() { Message = $"<Message-{DateTime.Now.ToLongTimeString()}>" };
Console.WriteLine($"Sending: {message.Message}");
await busControl.Publish(message);
host.Run();
For note, the breaking out of the message sending here is to simplify my repro, as in my full code base, it's being sent by a Quartz fired job.
For this example, the message & receiver are also quite simple:
public class TheMessage
{
public string Message { get; set; }
}
public class TheMessageConsumer : IConsumer<TheMessage>
{
public Task Consume(ConsumeContext<TheMessage> context)
{
Console.WriteLine($"Message received: {context.Message.Message}");
return Task.CompletedTask;
}
}
The bus is started, either in the case I explicitly start it, the AutoStart flag is set, or the MassTransitHostedService does it, yet the message doesn't get received. Ditto if I have the full example with Quartz firing off a job with messages much later.
Can someone suggest what I am missing?
The code you posted is seriously a hodgepodge of snippets, none of which make any sense when used together. For example, AddBus is deprecated, and mediator has no business being in that project at all.
I'd suggest using one of the MassTransit Templates to create a new project from scratch (you may need to up the NuGet versions to 7.2.2).
Watch this video which explains the templates and how to use them.
With your comment, and updated question, you really only need the following:
var hostBuilder = Host.CreateDefaultBuilder(args)
.ConfigureServices((hostContext, services) =>
{
services.AddMassTransit(mt =>
{
mt.AddConsumer<TheMessageConsumer>();
mt.UsingInMemory((context,cfg) =>
{
cfg.UseInMemoryOutbox();
cfg.ConfigureEndpoints(context);
}));
});
services.AddMassTransitHostedService();
});
var host = hostBuilder.Build();
await host.RunAsync();
If you want to test it, and send messages in the same process, simply add a BackgroundService (after AddMassTransitHostedService) to publish your messages.
You shouldn't publish until after the bus has been started.
I am using Masstransit with RabbitMQ. As part of some deployment procedure, At some point in time I need my service to disconnect and stop receiving any messages.
Assuming that I won't need the bus until the next restart of the service, will it be Ok to use bus.StopAsync()?
Is there a way to get list of end points and then remove them from listining ?
You should StopAsync the bus, and then when ready, call StartAsync to bring it back up (or start it at the next service restart).
To stop receiving messages without stopping the buss I needed a solution that will avoid the consume message pipeline from consuming any type of message. I tried with observers but unsuccessfully. My solution came up with custom consuming message filter.
The filter part looks like this
public class ComsumersBlockingFilter<T> :
IFilter<ConsumeContext<T>>
where T : class
{
public void Probe(ProbeContext context)
{
var scope = context.CreateFilterScope("messageFilter");
}
public async Task Send(ConsumeContext<T> context, IPipe<ConsumeContext<T>> next)
{
// Check if the service is degraded (true for this demo)
var isServiceDegraded = true;
if (isServiceDegraded)
{
//Suspend the message for 5 seconds
await Task.Delay(TimeSpan.FromMilliseconds(5000), context.CancellationToken);
if (!context.CancellationToken.IsCancellationRequested)
{
//republish the message
await context.Publish(context.Message);
Console.WriteLine($"Message {context.MessageId} has been republished");
}
// NotifyConsumed to avoid skipped message
await context.NotifyConsumed(TimeSpan.Zero, "messageFilter");
}
else
{
//Next filter in the pipe is called
await next.Send(context);
}
}
}
The main idea is to delay with cancellation token and the republish the message. After that call contect.NotifyConsumed to avoid the next pipeline filters and return normally.
I've discovered no Masstransit configuration that allows a service bus Topic to be created with Duplicate Detection enabled.
You can do it with Queues simply enough. But for Topics it seems a bit of a mystery.
Does anybody have a working sample?
Perhaps it is not possible.
I've been trying to use the IServiceBusBusFactoryConfigurator provided by the Bus.Factory.CreateUsingAzureServiceBus method.
I'd thought that some use of IServiceBusBusFactoryConfigurator.Publish method and IServiceBusBusFactoryConfigurator.SubscriptionEndpoint method would accomplish the task, but after a myriad of trials I've come up with no solution.
To configure your message type topic with duplicate detection, you must configure the publish topology in both the producer and the consumer (it only needs to be configured once per bus instance, but if your producer is a separate bus instance, it would also need the configuration). The topic must also not already exist as it would not be updated once created in Azure.
To configure the publish topology:
namespace DupeDetection
{
public interface DupeCommand
{
string Value { get; }
}
}
var busControl = Bus.Factory.CreateUsingAzureServiceBus(cfg =>
{
cfg.Publish<DupeCommand>(x => x.EnableDuplicateDetection(TimeSpan.FromMinutes(10)));
cfg.ReceiveEndpoint("dupe", e =>
{
e.Consumer<DupeConsumer>();
});
}
The consumer is normal (no special settings required).
class DupeConsumer :
IConsumer<DupeCommand>
{
public Task Consume(ConsumeContext<DupeCommand> context)
{
return Task.CompletedTask;
}
}
I've added a unit test to verify this behavior, and can confirm that when two messages with the same MessageId are published back-to-back, only a single message is delivered to the consumer.
Test log output:
10:53:15.641-D Create send transport: sb://masstransit-build.servicebus.windows.net/MassTransit.Azure.ServiceBus.Core.Tests.DupeDetection/DupeCommand
10:53:15.784-D Topic: MassTransit.Azure.ServiceBus.Core.Tests.DupeDetection/DupeCommand (dupe detect)
10:53:16.375-D SEND sb://masstransit-build.servicebus.windows.net/MassTransit.Azure.ServiceBus.Core.Tests.DupeDetection/DupeCommand dc3a0000-ebb8-e450-949c-08d8e8939c7f MassTransit.Azure.ServiceBus.Core.Tests.DupeDetection.DupeCommand
10:53:16.435-D SEND sb://masstransit-build.servicebus.windows.net/MassTransit.Azure.ServiceBus.Core.Tests.DupeDetection/DupeCommand dc3a0000-ebb8-e450-949c-08d8e8939c7f MassTransit.Azure.ServiceBus.Core.Tests.DupeDetection.DupeCommand
10:53:16.469-D RECEIVE sb://masstransit-build.servicebus.windows.net/MassTransit.Azure.ServiceBus.Core.Tests/input_queue dc3a0000-ebb8-e450-949c-08d8e8939c7f MassTransit.Azure.ServiceBus.Core.Tests.DupeDetection.DupeCommand MassTransit.IConsumer<MassTransit.Azure.ServiceBus.Core.Tests.DupeDetection.DupeCommand>(00:00:00.0017972)
You can see the (dupe detect) attribute shown on the topic declaration.
Here is the solution I finally found. It does not rely on trying any of the ReceiveEndpoint or SubscriptionEndpoint configuration methods which never seemed to give me what I wanted.
IBusControl bus = Bus.Factory.CreateUsingAzureServiceBus(cfg =>
{
cfg.Publish<MembershipNotifications.MembershipSignupMessage>(configure =>
{
configure.EnableDuplicateDetection(_DuplicateDetectionWindow);
configure.AutoDeleteOnIdle = _AutoDeleteOnIdle;
configure.DefaultMessageTimeToLive = _MessageTimeToLive;
});
}
await bus.Publish(new MessageTest());
I have set up a signalR website .net core. My function in my hub is:
public async Task Notify(int id) {
await Clients.All.InvokeAsync("Notified", id);
}
I have also tested this with the following js:
let connection = new signalR.HubConnection(myURL);
connection.on('Notified', data => {
console.log(4, data);
});
connection.start();
The js code seems to work fine and I see the log when I try connection.Invoke('Notify').
Now I have a console app that can needs to make the invoke. I am trying this in two ways and don't mind either solution:
1. A mvc controller within the signalR website that can take the id and invoke 'Notified'.
2. Use the client library Microsoft.AspNetCore.SignalR.Client in the console app.
The way 1 I have only done in classic asp.net like this:
GlobalHost.ConnectionManager.GetHubContext(hubName)
But couldn't find a way to do this in .net core.
Way 2 I have used the library and tried this so far:
var con = new HubConnectionBuilder();
con.WithUrl(myURL);
var connection = con.Build();
connection.InvokeAsync("Notify",args[0]).Wait();
This is the closest I have come to create a connection in the same way as the js code. However this code throws a null pointer when calling connection.InvokeAsync. The connection object is not null. It seems to be an internal object that is null. According to the stack trace the exception is thrown when a MoveNext() function is internally called.
Well looks like both are not currently possible. As of now I just used a forced way which is hopefully temporary.
I have created and used the following base class for hubs:
public abstract class MyHub : Hub
{
private static Dictionary<string, IHubClients> _clients = new Dictionary<string, IHubClients>();
public override Task OnConnectedAsync()
{
var c = base.OnConnectedAsync();
_clients.Remove(Name);
_clients.Add(Name, Clients);
return c;
}
public static IHubClients GetClients(string Name) {
return _clients.GetValueOrDefault(Name);
}
}
GlobalHost is gone. You need to inject IHubContext<THub> like in this sample.
This can be a bug in SignalR alpha1. Can you file an issue on https://github.com/aspnet/signalr and include a simplified repro?