I am trying to implement a heartbeat feature for my application hence was trying to implement recurring message feature from masstransit rabbitmq. I was trying to implement it on the sample given on masstransit's website. Here's all of the code.
namespace MasstransitBasicSample
{
using System;
using System.Threading.Tasks;
using MassTransit;
using MassTransit.Scheduling;
class Program
{
static void Main(string[] args)
{
var bus = Bus.Factory.CreateUsingRabbitMq(sbc =>
{
var host = sbc.Host(new Uri("rabbitmq://localhost"), h =>
{
h.Username("guest");
h.Password("guest");
});
sbc.UseMessageScheduler(new Uri("rabbitmq://localhost/quartz"));
sbc.ReceiveEndpoint(host, "test_queue", ep =>
{
ep.Handler<YourMessage>(context =>
{
return Console.Out.WriteLineAsync($"Received: {context.Message.Text}");
});
ep.Handler<PollExternalSystem>(context =>
{
return Console.Out.WriteLineAsync($"Received: {context.Message}");
});
});
});
bus.Start();
SetRecurring(bus);
Console.WriteLine("Press any key to exit");
Console.ReadKey();
bus.Stop();
}
private static async Task SetRecurring(IBusControl bus)
{
var schedulerEndpoint = await bus.GetSendEndpoint(new Uri("rabbitmq://localhost/quartz"));
var scheduledRecurringMessage = await schedulerEndpoint.ScheduleRecurringSend(
new Uri("rabbitmq://localhost/test_queue"), new PollExternalSystemSchedule(), new PollExternalSystem());
}
}
public class YourMessage { public string Text { get; set; } }
public class PollExternalSystemSchedule : DefaultRecurringSchedule
{
public PollExternalSystemSchedule()
{
CronExpression = "* * * * *"; // this means every minute
}
}
public class PollExternalSystem { }
}
I have created a queue called quartz in my rabbitmq queue.
When i run the application it sends a message to the quartz queue and that message just stays there , it does not go to the test queue.
I was also expecting another message to be sent to the quartz queue after a minute based on the cron expression, that also does not happen.
Is my setup wrong?
Any help would be much appreciated.
You need to run the scheduling service that listens on rabbitmq://localhost/quartz, where your messages are being sent.
The documentation page says:
There is a standalone MassTransit service, MassTransit.QuartzService,
which can be installed and used on servers for this purpose. It is
configured via the App.config file and is a good example of how to
build a standalone MassTransit service.
Alternatively, you can host Quartz scheduling in the same process by using in-memory scheduling, described here, by configuring it like this:
var busControl = Bus.Factory.CreateUsingRabbitMq(cfg =>
{
var host = cfg.Host(new Uri("rabbitmq://localhost/"), h =>
{
h.Username("guest");
h.Password("guest");
});
cfg.UseInMemoryScheduler();
});
Related
I want to be able to execute multiple jobs concurrently on a Job Consumer. At the moment if I run one service instance and try to execute 2 jobs concurrently, 1 job waits for the other to complete (i.e. waits for the single job slot to become available).
However if I run 2 instances by using dotnet run twice to create 2 separate processes I am able to get the desired behavior where both jobs run at the same time.
Is it possible to run 2 (or more) jobs at the same time for a given consumer inside a single process? My application requires the ability to run several jobs concurrently but I don't have the ability to deploy many instances of my application.
Checking the application log I see this line which I feel may have something to do with it:
[04:13:43 DBG] Concurrent Job Limit: 1
I tried changing the SagaPartitionCount to something other than 1 on instance.ConfigureJobServiceEndpoints to no avail. I can't seem to get the Concurrent Job Limit to change.
My configuration looks like this:
services.AddMassTransit(x =>
{
x.AddDelayedMessageScheduler();
x.SetKebabCaseEndpointNameFormatter();
// registering the job consumer
x.AddConsumer<DeploymentConsumer>(typeof(DeploymentConsumerDefinition));
x.AddSagaRepository<JobSaga>()
.EntityFrameworkRepository(r =>
{
r.ExistingDbContext<JobServiceSagaDbContext>();
r.LockStatementProvider = new SqlServerLockStatementProvider();
});
// add other saga repositories here for JobTypeSaga and JobAttemptSaga here as well
x.UsingRabbitMq((context, cfg) =>
{
var rmq = configuration.GetSection("RabbitMq").Get<RabbitMq>();
cfg.Host(rmq.Host, rmq.Port, rmq.VirtualHost, h =>
{
h.Username(rmq.Username);
h.Password(rmq.Password);
});
cfg.UseDelayedMessageScheduler();
var options = new ServiceInstanceOptions()
.SetEndpointNameFormatter(context.GetService<IEndpointNameFormatter>() ?? KebabCaseEndpointNameFormatter.Instance);
cfg.ServiceInstance(options, instance =>
{
instance.ConfigureJobServiceEndpoints(js =>
{
js.SagaPartitionCount = 1;
js.FinalizeCompleted = true;
js.ConfigureSagaRepositories(context);
});
instance.ConfigureEndpoints(context);
});
});
}
Where DeploymentConsumerDefinition looks like
public class DeploymentConsumerDefinition : ConsumerDefinition<DeploymentConsumer>
{
protected override void ConfigureConsumer(IReceiveEndpointConfigurator endpointConfigurator,
IConsumerConfigurator<DeploymentConsumer> consumerConfigurator)
{
consumerConfigurator.Options<JobOptions<DeploymentConsumer>>(options =>
{
options.SetJobTimeout(TimeSpan.FromMinutes(20));
options.SetConcurrentJobLimit(10);
options.SetRetry(r =>
{
r.Ignore<InvalidOperationException>();
r.Interval(5, TimeSpan.FromSeconds(10));
});
});
}
}
Your definition should specify the job consumer message type, not the job consumer type:
public class DeploymentConsumerDefinition : ConsumerDefinition<DeploymentConsumer>
{
protected override void ConfigureConsumer(IReceiveEndpointConfigurator endpointConfigurator,
IConsumerConfigurator<DeploymentConsumer> consumerConfigurator)
{
// MESSAGE TYPE NOT CONSUMER TYPE
consumerConfigurator.Options<JobOptions<DeploymentCommand>>(options =>
{
options.SetJobTimeout(TimeSpan.FromMinutes(20));
options.SetConcurrentJobLimit(10);
options.SetRetry(r =>
{
r.Ignore<InvalidOperationException>();
r.Interval(5, TimeSpan.FromSeconds(10));
});
});
}
}
I believe I am missing/misunderstanding something fundamental about the way .net5 works. In setting up an integration test environment for my GraphQL API, I am missing the step on how to start the GraphQL server from said test environment.
When I run the main project, the server is started properly and I can navigate to localhost in the browser and successfully execute GraphQL queries/mutations. My goal here is to set up some automated integration tests.
I'm using NUnit as my test runner and am using WebApplicationFactory<Startup> to "start the server" as I understand it.
In my test project, I'm under the impression that WebApplicationFactory<Startup> is supposed to basically use the Startup.cs class from my main project in my test project so that I don't have to duplicate all the settings, configurations, and injected services. Please correct me if that assumption is not correct.
I've pasted the code I think is relevant.
ApiWebApplicationFactory<Startup>
public class ApiWebApplicationFactory<TStartup> : WebApplicationFactory<Startup> where TStartup : class
{
public static IConfigurationRoot Configuration { get; set; }
public ApiWebApplicationFactory()
{
var configBuilder = new ConfigurationBuilder()
.AddJsonFile("appsettings.json", optional: false, reloadOnChange: true);
Configuration = configBuilder.Build();
}
protected override void ConfigureClient(HttpClient client)
{
base.ConfigureClient(client);
client.BaseAddress = new Uri("https://localhost");
client.Timeout = new TimeSpan(0, 10, 0);
}
// Based on my assumption this class reuses everything in the Startup.cs class
// I don't actually think this is necessary, but thought it was worth trying
// the test with and without this code.
protected override void ConfigureWebHost(IWebHostBuilder builder)
{
builder.ConfigureServices(services =>
{
services
.AddGraphQLServer()
.AddQueryType<Query>()
.AddMutationType<Mutation>()
.AddType<GraphQLContentItem>()
.AddType<GraphQLFolder>();
});
}
}
OneTimesetUp
[OneTimeSetUp]
public void OneTimeSetUp()
{
_factory = new ApiWebApplicationFactory<Startup>();
_client = _factory.WithWebHostBuilder(builder =>
{
builder.ConfigureServices(services =>
{
services.AddScoped<ICacheRepository, MockCache>();
});
}).CreateClient();
var connString = ApiWebApplicationFactory<Startup>.Configuration.GetConnectionString("DefaultConnection");
var options = new DbContextOptionsBuilder<CmsContext>()
.UseMySql(connString, ServerVersion.AutoDetect(connString))
.Options;
_dbContext = new CmsContext(options);
_dbContext.Database.EnsureCreated();
}
Test
[Test]
public async Task Test()
{
// If I set a breakpoint here, I can't navigate to the URL like I'm expecting to
var graphQLHttpClient =
new GraphQLHttpClient(
new GraphQLHttpClientOptions { EndPoint = new Uri("https://localhost/graphql") },
new NewtonsoftJsonSerializer(),
_client);
var request = new GraphQLRequest
{
Query = #"
query GetCurrentSession() {
getCurrentSession() {
id
name
}
}",
OperationName = "GetCurrentSession"
};
// Error is thrown here with "Bad Request"
var response = await graphQLHttpClient.SendQueryAsync<Session>(request);
// Further code is omitted
}
Please let me know if you see what I am missing. Thanks in advance~
The EndpointName property in a ConsumerDefinition file seems to be ignored by MassTransit. I know the ConsumerDefinition is being used because the retry logic works. How do I get different commands to go to a different queue? It seems that I can get them all to go through one central queue but I don't think this is best practice for commands.
Here is my app configuration that executes on startup when creating the MassTransit bus.
Bus.Factory.CreateUsingAzureServiceBus(cfg =>
{
cfg.Host(_config.ServiceBusUri, host => {
host.SharedAccessSignature(s =>
{
s.KeyName = _config.KeyName;
s.SharedAccessKey = _config.SharedAccessKey;
s.TokenTimeToLive = TimeSpan.FromDays(1);
s.TokenScope = TokenScope.Namespace;
});
});
cfg.ReceiveEndpoint("publish", ec =>
{
// this is done to register all consumers in the assembly and to use their definition files
ec.ConfigureConsumers(provider);
});
And my handler definition in the consumer (an azure worker service)
public class CreateAccessPointCommandHandlerDef : ConsumerDefinition<CreateAccessPointCommandHandler>
{
public CreateAccessPointCommandHandlerDef()
{
EndpointName = "specific";
ConcurrentMessageLimit = 4;
}
protected override void ConfigureConsumer(
IReceiveEndpointConfigurator endpointConfigurator,
IConsumerConfigurator<CreateAccessPointCommandHandler> consumerConfigurator
)
{
endpointConfigurator.UseMessageRetry(r =>
{
r.Immediate(2);
});
}
}
In my app that is sending the message I have to configure it to send to the "publish" queue, not "specific".
EndpointConvention.Map<CreateAccessPointsCommand>(new Uri($"queue:specific")); // does not work
EndpointConvention.Map<CreateAccessPointsCommand>(new Uri($"queue:publish")); // this does work
Because you are configuring the receive endpoint yourself, and giving it the name publish, that's the receive endpoint.
To configure the endpoints using the definitions, use:
cfg.ConfigureEndpoints(provider);
This will use the definitions that were registered in the container to configure the receive endpoints, using the consumer endpoint name defined.
This is also explained in the documentation.
I'm using the following code to send a request/response message between two different processes.
This is the process that "sends" the request:
// configure host
var hostUri = new Uri(configuration["RabbitMQ:Host"]);
services.AddSingleton(provider => Bus.Factory.CreateUsingRabbitMq(cfg =>
{
var host = cfg.Host(hostUri, h =>
{
h.Username(configuration["RabbitMQ:Username"]);
h.Password(configuration["RabbitMQ:Password"]);
});
}));
// add request client
services.AddScoped(provider => provider.GetRequiredService<IBus>().CreateRequestClient<QueryUserInRole, QueryUserInRoleResult>(new Uri(hostUri, "focus-authorization"), TimeSpan.FromSeconds(5)));
// add dependencies
services.AddSingleton<IPublishEndpoint>(provider => provider.GetRequiredService<IBusControl>());
services.AddSingleton<ISendEndpointProvider>(provider => provider.GetRequiredService<IBusControl>());
services.AddSingleton<IBus>(provider => provider.GetRequiredService<IBusControl>());
// add the service class so that the runtime can automatically handle the start and stop of our bus
services.AddSingleton<Microsoft.Extensions.Hosting.IHostedService, BusService>();
Here's the implementation of the BusService:
public class BusService : Microsoft.Extensions.Hosting.IHostedService
{
private readonly IBusControl _busControl;
public BusService(IBusControl busControl)
{
_busControl = busControl;
}
public Task StartAsync(CancellationToken cancellationToken)
{
return _busControl.StartAsync(cancellationToken);
}
public Task StopAsync(CancellationToken cancellationToken)
{
return _busControl.StopAsync(cancellationToken);
}
}
The problem is that when the CreateRequestClient code runs, the bus has not started up yet. Thus the response is never returned from the consumer.
If I replace the host configuration with the following code, I get the desired behavior:
var bus = Bus.Factory.CreateUsingRabbitMq(cfg =>
{
var host = cfg.Host(hostUri, h =>
{
h.Username(configuration["RabbitMQ:Username"]);
h.Password(configuration["RabbitMQ:Password"]);
});
});
bus.Start();
services.AddSingleton(bus);
For some reason, the BusService(IHostedService) executes AFTER the AddScoped delegates.
My question is: what is the correct way to start up the bus before using the CreateRequestClient method? Or is the latter approach to starting up the bus sufficient?
Trying to run core api host. i have this in the EntityFrameworkModule
public override void PreInitialize()
{
Configuration.MultiTenancy.IsEnabled = true;`
// CONNECTION STRING RESOLVER
Configuration.ReplaceService<IConnectionStringResolver, DbPerTenantConnectionStringResolver>(DependencyLifeStyle.Transient);
Configuration.DefaultNameOrConnectionString = MyConsts.DefaultConnectionStringName;
if (!SkipDbContextRegistration)
{
//DEFAULT
Configuration.Modules.AbpEfCore().AddDbContext<MyContext>(options =>
{
if (options.ExistingConnection != null)
MyContextConfigurer.Configure(options.DbContextOptions, options.ExistingConnection);
else
MyContextConfigurer.Configure(options.DbContextOptions, options.ConnectionString);
});
}
}
ERROR
Mvc.ExceptionHandling.AbpExceptionFilter - Can't create component 'Ccre.EntityFrameworkCore.AbpZeroDbMigrator' as it has dependencies to be satisfied.
'Ccre.EntityFrameworkCore.AbpZeroDbMigrator' is waiting for the following dependencies:
- Service 'Abp.MultiTenancy.IDbPerTenantConnectionStringResolver' which was not registered.
How/where do I register the IDbPerTenantConnectionStringResolver?
I have this line in the PreInitialize of the Migrator.MigratorModule
Configuration.ReplaceService<IConnectionStringResolver, DbPerTenantConnectionStringResolver>(DependencyLifeStyle.Transient);
as well as in the EntityFrameworkModule