Configuring a RetryPolicy inside a ReceiveEndpoint of a queue used for store messages from commands and events (as appears bellow) appears does not works when the queue is a saga endpoint queue.
This configuration works fine (note the endpoint RegisterOrderServiceQueue):
...
var bus = BusConfigurator.ConfigureBus((cfg, host) =>
{
cfg.ReceiveEndpoint(host, RabbitMqConstants.RegisterOrderServiceQueue, e =>
{
e.UseRetry(Retry.Except<ArgumentException>().Immediate(3));
...
...Bus RetryPolicy configuration on windows service to run the Saga state machine does not work (note the endpoint SagaQueue):
...
var bus = BusConfigurator.ConfigureBus((cfg, host) =>
{
cfg.ReceiveEndpoint(host, RabbitMqConstants.SagaQueue, e =>
{
e.UseRetry(Retry.Except<ArgumentException>().Immediate(3));
...
StateMachine class source code that throws an ArgumentException:
...
During(Registered,
When(ApproveOrder)
.Then(context =>
{
throw new ArgumentException("Test for monitoring sagas");
context.Instance.EstimatedTime = context.Data.EstimatedTime;
context.Instance.Status = context.Data.Status;
})
.TransitionTo(Approved),
...
But when ApproveOrder occurs the RetryPolicy rules are ignored, and connecting a ConsumeObserver in the bus that Sagas is connected, the ConsumeFault's method is executed 5 times (that is the default behavior of masstransit).
This should work? There is any missconception on my configurations?
Related
I am building a proof of concept with masstransit, rabbitmq and the jobconsumers. It is very nice and I am making progress. But there is one thing I cannot solve, that is changing the queue names of the queues Job, JobAttempt and JobType. I cannot find the right place to inject an endpointnameformatter. We are running multiple applications on a rabbitmq cluster and the jobs of the different solutions should not be mixed. So my question is where and how can a change the queue names?
I have tried:
services.AddMassTransit(mt =\>
mt.AddDelayedMessageScheduler();
mt.AddConsumer\<LongJobConsumer\>(cfg =\>
{
cfg.Options\<JobOptions\<LongJobConsumer\>\>(options =\> options
.SetJobTimeout(TimeSpan.FromMinutes(15))
.SetConcurrentJobLimit(1));
});
mt.SetEndpointNameFormatter(new DefaultEndpointNameFormatter("MyQueue", false));
mt.UsingRabbitMq((context, cfg) =>
{
cfg.UseDelayedMessageScheduler();
cfg.Host(rabbitMqConfig.Host, host =>
{
host.Username(rabbitMqConfig.Username);
host.Password(rabbitMqConfig.Password);
host.Heartbeat(rabbitMqConfig.Heartbeat);
});
cfg.UseGlobalRetryPolicy();
cfg.UseInMemoryOutbox();
ServiceInstanceOptions options = new ServiceInstanceOptions()
.EnableJobServiceEndpoints();
cfg.ServiceInstance(options, instance =>
{
instance.ConfigureJobServiceEndpoints();
instance.ConfigureEndpoints(context, new DefaultEndpointNameFormatter("MyQueue", false));
});
You can configure the endpoint names when configuring the job service endpoints:
instance.ConfigureJobServiceEndpoints(x =>
{
x.JobServiceStateEndpointName = "job-type";
x.JobServiceJobAttemptStateEndpointName = "job-attempt";
x.JobServiceJobStateEndpointName = "job";
});
I am tryng to update our existing consumer SQS queues with new settings. In this case we need to change all queue TTLs to the maximum of 14 days (for both normal queues, error and DLQ queues).
After much searching around I found a code that seems like it can do that:
cfg.QueueAttributes.Add(QueueAttributeName.MessageRetentionPeriod, 1209600);
cfg.SendTopology.ConfigureErrorSettings = settings => settings.QueueAttributes.Add(QueueAttributeName.MessageRetentionPeriod, 1209600);
cfg.SendTopology.ConfigureDeadLetterSettings = settings => settings.QueueAttributes.Add(QueueAttributeName.MessageRetentionPeriod, 1209600);
However when building against existing queues it doesnt seem to update the settings.
Here is my full MassTransit setup:
services.AddMassTransit(x =>
{
x.AddAmazonSqsMessageScheduler();
x.AddConsumers(assembliesWithConsumers.ToArray());
x.UsingAmazonSqs((context, cfg) =>
{
cfg.UseAmazonSqsMessageScheduler();
cfg.Host("aws", h =>
{
if (!String.IsNullOrWhiteSpace(mtSettings.AccessKey))
{
h.AccessKey(mtSettings.AccessKey);
}
if (!String.IsNullOrWhiteSpace(mtSettings.SecretKey))
{
h.SecretKey(mtSettings.SecretKey);
}
h.Scope($"{mtSettings.Prefix}-{mtSettings.Environment}", true);
var sqsConfig = !String.IsNullOrWhiteSpace(mtSettings.SqsServiceUrl)
? new AmazonSQSConfig() { ServiceURL = mtSettings.SqsServiceUrl }
: new AmazonSQSConfig()
{ RegionEndpoint = RegionEndpoint.GetBySystemName(mtSettings.Region) };
h.Config(sqsConfig);
var snsConfig = !String.IsNullOrWhiteSpace(mtSettings.SnsServiceUrl)
? new AmazonSimpleNotificationServiceConfig()
{ ServiceURL = mtSettings.SnsServiceUrl }
: new AmazonSimpleNotificationServiceConfig()
{ RegionEndpoint = RegionEndpoint.GetBySystemName(mtSettings.Region) };
h.Config(snsConfig);
});
cfg.QueueAttributes.Add(QueueAttributeName.MessageRetentionPeriod, 1209600);
cfg.SendTopology.ConfigureErrorSettings = settings => settings.QueueAttributes.Add(QueueAttributeName.MessageRetentionPeriod, 1209600);
cfg.SendTopology.ConfigureDeadLetterSettings = settings => settings.QueueAttributes.Add(QueueAttributeName.MessageRetentionPeriod, 1209600);
cfg.ConfigureEndpoints(context, new BusEnvironmentNameFormatter(mtSettings.Environment));
});
});
Does anyone know how to force SQS queues settings update with MassTransit?
MassTransit will not update settings on existing queues or topics. The only way to get settings applied is to delete the queue or topic, after which MassTransit will recreate it with the configured settings.
I have many apps (around 80) running in their own docker containers. They all create bus control.
I'm using rabbit transport.
When I start all containers including rabbit at the same time, some apps seems to have issues with initializing bus control properly. All messages produce the OperationCanceledException. Not seeing other warnings/errors logged during app start up. Restart the container fix the issue. I think this started to happen after upgrade from 5.5 to 6.2.
stack trace:
MassTransit.RequestCanceledException: The request was canceled, RequestId: 0b1c0000-0001-0c00-dec8-08d7daa945f5
---> System.OperationCanceledException: The agent is stopped or has been stopped, no additional provocateurs can be created.
at GreenPipes.Caching.Internals.NodeValueFactory1.CreateValue()
at GreenPipes.Caching.Internals.NodeTracker1.AddNode(INodeValueFactory1 nodeValueFactory)
--- End of inner exception stack trace ---
at MassTransit.Clients.ClientRequestHandle1.SendRequest()
at MassTransit.Clients.ResponseHandlerConnectHandle`1.GetTask()
I use autofac to inject the IBusControl instance, the creation of it looks like this:
bus = Bus.Factory.CreateUsingRabbitMq(rabbit =>
{
rabbit.Durable = true;
rabbit.Host(new Uri(serverUri), h =>
{
h.Username(settings.Username);
h.Password(settings.Password);
});
rabbit.PrefetchCount = settings.ReceiverPrefetchCount;
rabbit.UseMessageScheduler(SchedulerQueueUri);
rabbit.UseAppMetrics(metrics);
rabbit.BusObserver(observer);
foreach (var receiveEndpoint in receiveEndpoints)
{
rabbit.ReceiveEndpoint(receiveEndpoint.Queue, rec =>
{
ConfigureCommonFilters(componentContext, brokerSettings, rec);
rec.PrefetchCount = brokerSettings.ReceiverConcurrencyLimit;
foreach (var endpointAction in receiveEndpoint.EndpointActions)
{
endpointAction(rec, brokerSettings, componentContext);
}
});
});
}
My service is started using
IWebHostBuilder.UseStartup<Startup>().Build().RunAsync(_cancellationTokenSource.Token).ConfigureAwait(false);
In Startup
public virtual void Configure(
IWebHostEnvironment env, IApplicationBuilder app, IHostApplicationLifetime appLifetime, IServiceProvider serviceProvider
)
{
var busControl = serviceProvider.GetService<IBusControl>();
// other configuration
appLifetime?.ApplicationStarted.Register(() => retry.RunReliably(() => busControl?.Start()));
appLifetime?.ApplicationStopping.Register(() => retry.RunReliably(() => busControl?.Stop()));
}
The EndpointName property in a ConsumerDefinition file seems to be ignored by MassTransit. I know the ConsumerDefinition is being used because the retry logic works. How do I get different commands to go to a different queue? It seems that I can get them all to go through one central queue but I don't think this is best practice for commands.
Here is my app configuration that executes on startup when creating the MassTransit bus.
Bus.Factory.CreateUsingAzureServiceBus(cfg =>
{
cfg.Host(_config.ServiceBusUri, host => {
host.SharedAccessSignature(s =>
{
s.KeyName = _config.KeyName;
s.SharedAccessKey = _config.SharedAccessKey;
s.TokenTimeToLive = TimeSpan.FromDays(1);
s.TokenScope = TokenScope.Namespace;
});
});
cfg.ReceiveEndpoint("publish", ec =>
{
// this is done to register all consumers in the assembly and to use their definition files
ec.ConfigureConsumers(provider);
});
And my handler definition in the consumer (an azure worker service)
public class CreateAccessPointCommandHandlerDef : ConsumerDefinition<CreateAccessPointCommandHandler>
{
public CreateAccessPointCommandHandlerDef()
{
EndpointName = "specific";
ConcurrentMessageLimit = 4;
}
protected override void ConfigureConsumer(
IReceiveEndpointConfigurator endpointConfigurator,
IConsumerConfigurator<CreateAccessPointCommandHandler> consumerConfigurator
)
{
endpointConfigurator.UseMessageRetry(r =>
{
r.Immediate(2);
});
}
}
In my app that is sending the message I have to configure it to send to the "publish" queue, not "specific".
EndpointConvention.Map<CreateAccessPointsCommand>(new Uri($"queue:specific")); // does not work
EndpointConvention.Map<CreateAccessPointsCommand>(new Uri($"queue:publish")); // this does work
Because you are configuring the receive endpoint yourself, and giving it the name publish, that's the receive endpoint.
To configure the endpoints using the definitions, use:
cfg.ConfigureEndpoints(provider);
This will use the definitions that were registered in the container to configure the receive endpoints, using the consumer endpoint name defined.
This is also explained in the documentation.
I'm using the following code to send a request/response message between two different processes.
This is the process that "sends" the request:
// configure host
var hostUri = new Uri(configuration["RabbitMQ:Host"]);
services.AddSingleton(provider => Bus.Factory.CreateUsingRabbitMq(cfg =>
{
var host = cfg.Host(hostUri, h =>
{
h.Username(configuration["RabbitMQ:Username"]);
h.Password(configuration["RabbitMQ:Password"]);
});
}));
// add request client
services.AddScoped(provider => provider.GetRequiredService<IBus>().CreateRequestClient<QueryUserInRole, QueryUserInRoleResult>(new Uri(hostUri, "focus-authorization"), TimeSpan.FromSeconds(5)));
// add dependencies
services.AddSingleton<IPublishEndpoint>(provider => provider.GetRequiredService<IBusControl>());
services.AddSingleton<ISendEndpointProvider>(provider => provider.GetRequiredService<IBusControl>());
services.AddSingleton<IBus>(provider => provider.GetRequiredService<IBusControl>());
// add the service class so that the runtime can automatically handle the start and stop of our bus
services.AddSingleton<Microsoft.Extensions.Hosting.IHostedService, BusService>();
Here's the implementation of the BusService:
public class BusService : Microsoft.Extensions.Hosting.IHostedService
{
private readonly IBusControl _busControl;
public BusService(IBusControl busControl)
{
_busControl = busControl;
}
public Task StartAsync(CancellationToken cancellationToken)
{
return _busControl.StartAsync(cancellationToken);
}
public Task StopAsync(CancellationToken cancellationToken)
{
return _busControl.StopAsync(cancellationToken);
}
}
The problem is that when the CreateRequestClient code runs, the bus has not started up yet. Thus the response is never returned from the consumer.
If I replace the host configuration with the following code, I get the desired behavior:
var bus = Bus.Factory.CreateUsingRabbitMq(cfg =>
{
var host = cfg.Host(hostUri, h =>
{
h.Username(configuration["RabbitMQ:Username"]);
h.Password(configuration["RabbitMQ:Password"]);
});
});
bus.Start();
services.AddSingleton(bus);
For some reason, the BusService(IHostedService) executes AFTER the AddScoped delegates.
My question is: what is the correct way to start up the bus before using the CreateRequestClient method? Or is the latter approach to starting up the bus sufficient?