I have a producer, which send more than 1000 messages in a minute to a specific endpoint. I’m using Microsoft DI and I’ve configured the send Endpoint as described here https://masstransit-project.com/usage/producers.html#send .
// Masstransit setup
serviceCollection.AddMassTransit(mt =>
{
mt.UsingAzureServiceBus((ctx, cfg) =>
{
cfg.Host(massTransitSettings.TestServiceBusConnectionString);
cfg.ReceiveEndpoint("mytestmessage", e =>
{
e.MaxDeliveryCount = 3; //How many times the transport will redeliver the message on negative acknowledgment
});
});
});
serviceCollection.AddTransient<ITestMessageProducer, TestMessageProducer>();
// Producer setup
public class TestMessageProducer : ITestMessageProducer
{
private readonly ISendEndpointProvider _testEndpoint;
public TestMessageProducer(ISendEndpointProvider testEndpoint)
{
_testEndpoint = testEndpoint;
}
public async Task SendTestMessage(ITestMessage testmessage)
{
var endpoint = await _testEndpoint.GetSendEndpoint(new Uri("queue:mytestmessage"));
await endpoint.Send(testmessage);
}
}
Query:
The SendTestMessage function has been called very frequently as mention above. Will it be ok to call “GetSendEndpoint” everytime? I have read somewhere that GetSendEndpoint creates a new instance of ISendEndpoint everytime.
Will the MaxDeliveryCount still be worked on my sendendpoint?
Thank you.
Send endpoints are cached by address, only a single instance will be created.
MaxDeliveryCount is a receive endpoint concern, but you should not configure a receive endpoint without consumers as all messages will be moved to the _skipped queue.
Related
After much toil and trial and error I managed to issue a "request" from my saga and see it handle the response. My jubilation was cut short however by the appearance of a message in my states' skipped queue. (i'm using azure service bus)
It is of type "urn:message:MassTransit.Scheduling:CancelScheduledMessage".
I am a complete newbie at with mass transit and I'm just trying to get a contrived example going.
My saga calls TaxiToRunway/TaxiingComplete. My bit of saga code
Request(()=>TaxiToRunway, config =>
{
config.Timeout = TimeSpan.FromSeconds(30);
});
...
public Request<PlaneState, TaxiToRunway, TaxiingComplete> TaxiToRunway { get; private set; }
...
Initially(
When(ReadyToDepart)
.Then(context =>
{
context.Saga.Altitude = 0;
context.Saga.Speed = 0;
context.Saga.FlightNo = context.Message.FlightNo;
context.Saga.CorrelationId = context.Message.CorrelationId;
Console.WriteLine($"Flight {context.Message.FlightNo} is ready to depart.");
})
.TransitionTo(Taxiing)
.Request(TaxiToRunway,
(context) => context.Init<TaxiToRunway>(new {CorrelationId = context.Saga.CorrelationId}))
...
During(Taxiing,
Ignore(ReadyToDepart),
When(TaxiToRunway.Completed)
.Then(x =>
{
x.ToString();
})
.TransitionTo(TakingOff),
With a debugger attached I hit the x.ToString() line.
The consumer (in a different host):
public class TaxiToRunwayConsumer: IConsumer<TaxiToRunway>
{
public async Task Consume(ConsumeContext<TaxiToRunway> context)
{
await context.RespondAsync<TaxiingComplete>(new
{
context.Message.CorrelationId
});
}
}
Saga startup config:
cfg.AddSagaStateMachine<PlaneStateMachine, PlaneState>()
.MessageSessionRepository();
cfg.AddServiceBusMessageScheduler();
cfg.UsingAzureServiceBus((context, sbCfg) =>
{
var connectionString = appConfig.ServiceBus.ConnectionString;
sbCfg.Host(connectionString);
EndpointConvention.Map<TaxiToRunway>(new Uri("sb://xxx.servicebus.windows.net/taxi-to-runway"));
sbCfg.UseServiceBusMessageScheduler();
sbCfg.ReceiveEndpoint("plane-state", e =>
{
e.UseInMemoryOutbox();
e.RequiresSession = true;
e.PrefetchCount = 50;
e.MaxConcurrentCalls = 50;
e.ConfigureSaga<PlaneState>(context);
});
sbCfg.ConfigureEndpoints(context);
});
I can see this in the log output:
dbug: MassTransit.Messages[0]
SEND sb://dbpdf-us-dev-sam.servicebus.windows.net/plane-state 80d90000-5d7b-2cf0-7a6b-08da0fd3e7b7 MassTransit.Scheduling.CancelScheduledMessage
Am I supposed to be handling this as an event??
Learning curve on this sure is steep! My question is what do I need to do to not have these messages go to skipped?
So, the reason this doesn't work:
The message session saga repository can only correlate by the SessionId, since it's session-stored data.
The requestId, therefore, MUST equal the saga instance correlationId (aka, the SessionId)
The timeout message, sent by the request, gets a tokenId based upon the sequence number of the scheduled message
Which isn't saved anywhere
So the request timeout isn't canceled
The proper approach, in this scenario, is to use a Request/Response that doesn't have a timeout and use a separate Schedule to schedule the timeout yourself.
I'm running a MassTransit configuration with AmazonSQS. In my program I start by creating a receiveenpoint with the queue "input-queue1", I subscribe this SQS queue to an SNS topic named "topic1" and associate a consumer to this receiveendpoint that does some standard printing of the messages it receives. After starting the bus i want to subscribe the already created queue "input-queue1" to another SNS topic, named "topic2", but I couldn't find a way of doing this after starting the Bus (It's important to me that i can do this after the Bus is started). Is there a way of doing this and i'm just missing something, or is it not possible at all? (I tried with the commented portion of the code but it didn't work)
class Program
{
static async Task Main(string[] args)
{
var bus = Bus.Factory.CreateUsingAmazonSqs(x =>
{
x.Host(Constants.Region, h =>
{
h.AccessKey(Constants.AccesskeyId);
h.SecretKey(Constants.SecretAccessKey);
});
x.ReceiveEndpoint("input-queue1", e =>
{
e.Subscribe("topic1", callback => { });
e.Consumer(() => new Handler());
});
});
bus.StartAsync().Wait();
/*var handle = bus.ConnectReceiveEndpoint("input-queue1", e => {
e.Subscribe("topic2", callback => { });
});
var ready = await handle.Ready;*/
Console.WriteLine("Listening to messages...");
Console.WriteLine("Press enter to quit");
Console.ReadLine();
}
}
You can't change the topology of a receive endpoint once it has been created. This means that no new topic subscriptions can be created, and existing subscriptions cannot be removed.
If you need to change the configuration of the receive endpoint, you would need to do it yourself by using the SNS API to add the subscription yourself. I would question why you would want to do this though. If the consumer isn't able to consume the message forwarded to the queue, it would be moved to the skipped queue.
The requirement is like to process the messages from dead letter queue by exposed a REST service API(Spring Boot).
So that once REST service is called, one message will be consumed from the DL queue and will publish in the main queue again for processing.
#RabbitListener(queues = "QUEUE_NAME") consumes the message immediately which is not required as per the scenario. The message only has to be consumed by the REST service API.
Any suggestion or solution?
I do not think RabbitListener will help here.
However you could implement this behaviour manually.
Spring Boot automatically creates RabbitMq connection factory so you could use it. When http call is made just read single message from the queue manually, you could use basic.get to synchronously get just one message:
#Autowire
private ConnectionFactory factory
void readSingleMessage() {
Connection connection = null;
Channel channel = null;
try {
connection = factory.newConnection();
channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME, true, false, false, null);
GetResponse response = channel.basicGet(QUEUE_NAME, true);
if (response != null) {
//Do something with the message
}
} finally {
//Check if not null
channel.close();
connection.close();
}
}
If you are using Spring; you can avoid all the boilerplate in the other answer using RabbitTemplate.receive(...).
EDIT
To manually ack/reject the message, use the execute method instead.
template.execute(channel -> {
GetResponse got = channel.basicGet("foo", false);
// ...
channel.basicAck(got.getEnvelope().getDeliveryTag(), false);
return null;
});
It's a bit lower level, but again, most of the boilerplate is taken care of for you.
I've been trying to come up with a demo of a website that uses MassTransit with RabbitMQ to post messages to a service running on Service Fabric as a Stateful service.
Everything was going fine, my client would post a message:
IBusControl bus = BusConfigurator.ConfigureBus();
Uri sendToUri = new Uri($"{RabbitMqConstants.RabbitMqUri}" + $"{RabbitMqConstants.PeopleServiceQueue}");
ISendEndpoint endPoint = await bus.GetSendEndpoint(sendToUri);
await endPoint.Send<ICompanyRequest>(new {CompanyId = id });
My consumer in my service fabric service was defined like:
IBusControl busControl = Bus.Factory.CreateUsingRabbitMq(cfg =>
{
IRabbitMqHost host = cfg.Host(new Uri(RabbitMqConstants.RabbitMqUri), h =>
{
h.Username(RabbitMqConstants.UserName);
h.Password(RabbitMqConstants.Password);
});
cfg.ReceiveEndpoint(host, RabbitMqConstants.PeopleServiceQueue, e =>
{
e.Consumer<PersonInformationConsumer>();
});
});
busControl.Start();
This does allow me to consume the message in my class and I can process it fine. The problem comes when we want to use IReliableDictonary or IReliableQueue or anything that needs to reference the context that is run from the RunAsync function in the service fabric service.
So my question is, how can I configure (is it possible) MassTransit to work within a Stateful Service Fabric Service which knowledge of the service context itself?
Many thanks in advance.
Mike
Update
Ok, I've made some progress on this, if I point the register routines to my message consumer class (eg):
ServiceRuntime.RegisterServiceAsync("ServiceType", context => new PersonInformationConsumer(context)).GetAwaiter().GetResult();
ServiceEventSource.Current.ServiceTypeRegistered(Process.GetCurrentProcess().Id, typeof(PersonInformationConsumer).Name);
Then in my consumer class for my messages I can do the following:
internal sealed class PersonInformationConsumer : StatefulService, IConsumer<ICompanyRequest>
{
private static StatefulServiceContext _currentContext;
#region Constructors
public PersonInformationConsumer(StatefulServiceContext serviceContext) : base(serviceContext)
{
_currentContext = serviceContext;
}
public PersonInformationConsumer() : base(_currentContext)
{
}
I can now successfully call the service message:
ServiceEventSource.Current.ServiceMessage(this.Context, "Message has been consumed, request Id: {0}", context.Message.CompanyId);
The problem I have now is trying to store something on the IReliableDictionary, doing this causes as "Object reference not set to an instance of an object" error :( ... any ideas would be appreciated (although may not read until new year now!)
public async Task Consume(ConsumeContext<ICompanyRequest> context)
{
ServiceEventSource.Current.ServiceMessage(this.Context, "Message has been consumed, request Id: {0}", context.Message.CompanyId);
using (ITransaction tx = StateManager.CreateTransaction())
{
try
{
var myDictionary = await StateManager.GetOrAddAsync<IReliableDictionary<string, long>>("myDictionary");
This is causing the error.... HELP! :)
You'll need to do a bit more to get MassTransit and stateful services working together, there's a few issues to concern yourself here.
Only the master within a stateful partition (n masters within n partitions) will be able to write/update to the stateful service, all replicas will throw exceptions when trying to write back any state. So you'll need to deal with this issue, on the surface it sounds easy until you take in to consideration the master can move around the cluster due to re-balancing the cluster, the default for general service fabric applications is to just turn off the processing on the replicas and only run the work on the master. This is all done by the RunAsync method (try it out, run 3 stateful services with something noddy in the RunAsync method, then terminate the master).
There is also partitioning of your data to consider, due to stateful services scale with partitions, you'll need to create a way to distributing data to separate endpoint on your service bus, maybe have a separate queue that only listens to a given partition range? Say you have a UserCreated message, you might split this on country UK goes to partition 1, US goes to partition 2 etc...
If you just want to get something basic up and running, I'd limit it to one partition and just try putting your bus creation within the the RunAsync and shutdown the bus once a cancelation is requested on the cancelation token.
protected override async Task RunAsync(CancellationToken cancellationToken)
{
var busControl = Bus.Factory.CreateUsingRabbitMq(cfg =>
{
IRabbitMqHost host = cfg.Host(new Uri(RabbitMqConstants.RabbitMqUri), h =>
{
h.Username(RabbitMqConstants.UserName);
h.Password(RabbitMqConstants.Password);
});
cfg.ReceiveEndpoint(host, RabbitMqConstants.PeopleServiceQueue, e =>
{
// Pass in the stateful service context
e.Consumer(c => new PersonInformationConsumer(Context));
});
});
busControl.Start();
while (true)
{
if(cancellationToken.CancellationRequested)
{
//Service Fabric wants us to stop
busControl.Stop();
cancellationToken.ThrowIfCancellationRequested();
}
await Task.Delay(TimeSpan.FromSeconds(1));
}
}
Ok here i'm , i'm right now following the guides on spring site but i'm having problem on how to deliver a notification to only one user using WebSocket, i'm following this guide https://spring.io/guides/gs/messaging-stomp-websocket/ .
What I want is: i have 2 users, both of them subscribe to process1... User1 need to let the server process his pass...now i want that the server will deliver the notification only to User1...
#Controller
public class ProcessController {
#MessageMapping("/ProcessOwner/approve/{pass}")
#SendTo("")
public String notifica(#DestinationVariable String pass)throws Exception{
return "ok"+pass;
}
}
Now what should i write in the #SendTo field to deliver the answer only to user1 ? if ill write /Process/process1 both user1 and user2 will receive the message....
You ca use the sufix. It's send to every unique client.
When you subscribe in the js code:
client.connect('guest', 'guest', function(frame) {
var suffix = frame.headers['queue-suffix'];
client.subscribe("/queue/error" + suffix, function(msg) {
// handle error
});
client.subscribe("/queue/position-updates" + suffix, function(msg) {
// handle position update
});
});
On the server side you can use #ReplyToUser or the message template
String user = "fabrice";
String queue = "/queue/position-updates";
this.messagingTemplate.convertAndSendToUser(user, queue, position);
See more here: http://assets.spring.io/wp/WebSocketBlogPost.html (section: Sending Messages To a Single User)