Adding Observers to already running MassTransit system - masstransit

I am trying to add a microservice to a system that contains a MassTransit observer, which will observe request response or publish messages already being used in the system. I cannot redeploy the existing services easily so would prefer to avoid it if possible.
The following code only executes when the service starts, it does not execute when a message is sent.
BusControl = Bus.Factory.CreateUsingRabbitMq(cfg =>
{
var host = cfg.Host(new Uri($"{settings.Protocol}://{settings.RabbitMqHost}/"), h =>
{
h.Username(settings.RabbitMqConsumerUser);
h.Password(settings.RabbitMqConsumerPassword);
});
cfg.ReceiveEndpoint(host, "pub_sub_flo", ec => { });
host.ConnectSendObserver(new RequestObserver());
host.ConnectPublishObserver(new RequestObserver());
});
Observers:
public class RequestObserver : ISendObserver, IPublishObserver
{
public Task PreSend<T>(SendContext<T> context) where T : class
{
return Task.CompletedTask;
}
public Task PostSend<T>(SendContext<T> context) where T : class
{
var proxy = new StoreProxyFactory().CreateProxy("fabric:/MessagePatterns");
proxy.AddEvent(new ConsumerEvent()
{
Id = Guid.NewGuid(),
ConsumerId = Guid.NewGuid(),
Message = "AMQPRequestResponse",
Date = DateTimeOffset.Now,
Type = "Observer"
}).Wait();
return Task.CompletedTask;
}
public Task SendFault<T>(SendContext<T> context, Exception exception) where T : class
{
return Task.CompletedTask;
}
public Task PrePublish<T>(PublishContext<T> context) where T : class
{
return Task.CompletedTask;
}
public Task PostPublish<T>(PublishContext<T> context) where T : class
{
var proxy = new StoreProxyFactory().CreateProxy("fabric:/MessagePatterns");
proxy.AddEvent(new ConsumerEvent()
{
Id = Guid.NewGuid(),
ConsumerId = Guid.NewGuid(),
Message = "AMQPRequestResponse",
Date = DateTimeOffset.Now,
Type = "Observer"
}).Wait();
return Task.CompletedTask;
}
public Task PublishFault<T>(PublishContext<T> context, Exception exception) where T : class
{
return Task.CompletedTask;
}
}
Can anyone help?
Many thanks in advance.

The observers are only called for messages sent, published, etc. on the bus instance to which they are attached. They will not observe messages sent or received by other bus instances.
If you want to observe those messages, you could create an observer queue and bind that queue to your service exchanges so that copies of the request messages are sent to your service. The replies, however, would not be easy to get since they're sent directly to the client queues via temporary exchanges.
cfg.ReceiveEndpoint(host, "service-observer", e =>
{
e.Consumer<SomeConsumer>(...);
e.Bind("service-endpoint");
});
This will bind the service endpoint exchange to your receive endpoint queue, so that copies of the messages are sent to your consumer.
This is commonly referred to as a wire tap.

Related

Nest.js Websocket Gateway loosing socket connecition using redid broker

I have to implement websocket communication in my nest.js app. I've successfully setup the websocket gateway and I have tested it with postman. My code looks like this
export class SocketIOAdapter extends IoAdapter {
constructor(private app: INestApplicationContext, private configService: ConfigService) {
super(app);
}
createIOServer(port: number, options: ServerOptions) {
const clientPort = parseInt(this.configService.getOrThrow("PORT"));
const cors = {
origin: [
`http://localhost:${clientPort}`,
new RegExp(`/^http:\/\/192\.168\.1\.([1-9]|[1-9]\d):${clientPort}$/`),
],
};
const optionsWithCORS: ServerOptions = {
...options,
cors,
};
const server: Server = super.createIOServer(port, optionsWithCORS);
const orderRepository = this.app.get(OrderRepository);
server
.of("/orders")
.use(createTokenMiddleware(orderRepository));
return server;
}
}
const createTokenMiddleware =
(orderRepository: OrderRepository) =>
async (socket: Socket, next) => {
// here I run some logic using my order repository
next();
} catch {
next(new Error("FORBIDDEN"));
}
};
And
#WebSocketGateway({
namespace: "/orders",
})
#Injectable()
export class OrderGateway implements OnGatewayInit, OnGatewayConnection, OnGatewayDisconnect {
private readonly logger = new Logger(OrderGateway.name);
#WebSocketServer() io: Namespace;
afterInit(): void {
this.logger.log("Websocket Gateway initialized.");
}
async handleConnection(client: Socket) {
const sockets = this.io.sockets;
// here I run some logic to know which rooms to use for this client
const roomsToJoin = [...]
await client.join(roomsToJoin);
}
async handleDisconnect(client: Socket) {
this.logger.log(`Disconnected socket id: ${client.id}`);
}
public emitOrderStatusChangeNotification(order: OrderDTO) {
this.io
.to("Here I put some roomId that depends on order")
.emit("order_status_changed", JSON.stringify(order));
}
}
Now, whenever I want to send a notification, I inject the OrderGateway and call emitOrderStatusChangeNotification. This works fine, however, my app is deployed on several instances behind a load balancer. The latter breaks this approach as socket clients may be connected to a different server from the one I'm sending the notification. So, the next step to scale web sockets (as far as I understand) is to use a broker. I tried to use Redis pub/sub in the following way. I have this two classes:
#Injectable()
export class NotificationPublisherService {
constructor(#Inject("ORDER_NOTIFICATION_SERVICE") private client: ClientProxy) {}
async publishEvent(order: OrderDTO) {
console.log("will emit to redis");
this.client.emit(Constants.notificationEventName, order);
}
}
#Controller()
export class NotificationSuscriberController {
private readonly logger = new Logger(NotificationSuscriberController.name);
constructor(private readonly orderGateway: OrderGateway) {}
#EventPattern(Constants.notificationEventName)
async handleOrderStatusChangeEvent(order: OrderDTO) {
try {
this.orderGateway.emitOrderStatusChangeNotification(order);
} catch (err) {
this.logger.log("error sending notification");
}
}
As you can see, I'm injecting orderGateway in the class that have the method that handles the data from redis and in that handler I send the notification. Finally, I replaced all the invocations of emitOrderStatusChangeNotification to the publishEvent method of NotificationPublisherService. After doing this, the flow works well except from the last step. This means, the data is put on redis and read by the suscriber, which tries to send the websocket notification. However, when logging the connected clients for that room in emitOrderStatusChangeNotification method, I'm getting that there are no connected clients, even though I confirmed there where connected clients on that room (I did this by logging the list of connected clients after doing client.join in the handleConnection method of OrderGateway). My best guess is that an instance of OrderGateway handles the socket connection and a different instance of OrderGateway is processing the data from Redis broker. I tried to explicitly set the scope of the Gateway to Default to guarantee that my app has only one instance of OrderGateway (I also confirmed that it has not any request scoped dependency that could bubble up and make it not default scoped). It did not work and I'm out of ideas. Does anyone know what could be happening? Thanks in advance
EDIT
As Gregorio suggested in the answers, I had to extend my adapter as explained in the docs, the following code worked for me
export class SocketIOAdapter extends IoAdapter {
private adapterConstructor: ReturnType<typeof createAdapter>;
constructor(private app: INestApplicationContext, private configService: ConfigService) {
super(app);
}
async connectToRedis(): Promise<void> {
const pubClient = createClient({ url: "redis://localhost:6379" });
const subClient = pubClient.duplicate();
await Promise.all([pubClient.connect(), subClient.connect()]);
this.adapterConstructor = createAdapter(pubClient, subClient);
}
createIOServer(port: number, options: ServerOptions) {
const clientPort = parseInt(this.configService.getOrThrow("PORT"));
const cors = {
origin: [
`http://localhost:${clientPort}`,
new RegExp(`/^http:\/\/192\.168\.1\.([1-9]|[1-9]\d):${clientPort}$/`),
],
};
const optionsWithCORS: ServerOptions = {
...options,
cors,
};
const server: Server = super.createIOServer(port, optionsWithCORS);
const orderRepository = this.app.get(OrderRepository);
server
.adapter(this.adapterConstructor)
.of(`/orders`)
.use(createTokenMiddleware(orderRepository));
return server;
}
}
const createTokenMiddleware =
(orderRepository: OrderRepository) =>
async (socket: Socket, next) => {
// here I run some logic using my order repository
next();
} catch {
next(new Error("FORBIDDEN"));
}
};
}
and in my main.ts
const redisIoAdapter = new SocketIOAdapter(app, configService);
await redisIoAdapter.connectToRedis();
Have you tried following this page from the nest.js docs? I think it might help you in what you're looking for. You should write in your SocketIOAdapter what it says there in order to connect with Redis, it is not necessary to have the NotificationPublisherService or the NPController.
https://docs.nestjs.com/websockets/adapter

Configure DeadLetter queue for send endpoint

I am trying to configure a Producer to send a message to a Consumer that has a deadletter queue configured. The Producer is using a SendEndpoint (Or rather the request/response pattern), but I get an exception from RabbitMQ.
I have the following consumer:
public void ConfigureServices(IServiceCollection services)
{
services.AddControllers();
services.AddMassTransit(x =>
{
x.AddConsumer<SomeMessageRequestConsumer>();
x.AddBus(provider => Bus.Factory.CreateUsingRabbitMq(busConfig =>
{
busConfig.Host(new Uri("rabbitmq://rabbit#localhost"), "/", hostConfigurator =>
{
hostConfigurator.Password("Guest");
hostConfigurator.Username("Guest");
});
busConfig.ReceiveEndpoint(nameof(SomeMessage), x =>
{
x.ConfigureConsumer<SomeMessageRequestConsumer>(provider);
x.Durable = false;
x.ConfigureConsumeTopology = false;
x.BindDeadLetterQueue("SomeMessageDeadLetter", "SomeMessageDeadLetter", null);
});
}));
});
services.AddMassTransitHostedService();
}
I have the following Producer:
public void ConfigureServices(IServiceCollection services)
{
services.AddControllers();
services.AddSingleton<IReplyToClientFactory, ReplyToClientFactory>();
services.AddMassTransit(x =>
{
x.AddBus(provider => Bus.Factory.CreateUsingRabbitMq(busConfig =>
{
busConfig.Host(new Uri("rabbitmq://rabbit#localhost"), "/", hostConfigurator =>
{
hostConfigurator.Password("Guest");
hostConfigurator.Username("Guest");
});
}));
});
services.AddMassTransitHostedService();
}
In the Producer project I have a controller that send the message like so:
public ProducerController(IReplyToClientFactory clientFactory)
{
this.clientFactory = clientFactory;
}
[HttpPost]
public async Task<IActionResult> Post(CancellationToken cancellationToken)
{
var serviceAddress = new Uri($"queue:{nameof(SomeMessage)}?durable=false");
var client = this.clientFactory.GetFactory().CreateRequestClient<SomeMessage>(serviceAddress);
var (successResponse, failResponse) = await client.GetResponse<SomeMessageSuccessResponse, SomeMessageFailResponse>(new SomeMessage()
{
Text = "Hello",
}, cancellationToken, TimeSpan.FromSeconds(5));
return Ok();
}
I get the following error on RabbitMQ :
operation queue.declare caused a channel exception precondition_failed: inequivalent arg 'x-dead-letter-exchange' for queue 'SomeMessage' in vhost '/': received none but current is the value 'SomeMessageDeadLetter' of type 'longstr'
I have tried to configure the deadletter on the Publish, Send and Message Topologies but with no success. Is what I am trying to do possible or am I chasing the wind here?
You could change the destination address from a queue to an exchange, to decouple your producer from the consumer queue configuration. To send to the exchange, changed your address format to:
$"exchange:{nameof(SomeMessage)}"
That way, you don't need to know the queue configuration to send the request.

Masstransit scheduler with quartz.net object reference not set while reading message

I've been receiving the following error while setting up the quartz.net scheduler with MassTransit
After the message is scheduled on the RabbitMQ, when the quartz.net tries to read it, the error is thrown:
MT-Fault-Message: Object reference not set to an instance of an object.
MT-Fault-Timestamp: 2021-06-02T18:46:56.1335404Z
MT-Fault-StackTrace: at MassTransit.QuartzIntegration.ScheduleMessageConsumer.TranslateJsonBody(String body, String destination)
at MassTransit.QuartzIntegration.ScheduleMessageConsumer.CreateJobDetail(ConsumeContext context, Uri destination, JobKey jobKey, Nullable`1 tokenId)
at MassTransit.QuartzIntegration.ScheduleMessageConsumer.Consume(ConsumeContext`1 context)
at MassTransit.Pipeline.ConsumerFactories.DelegateConsumerFactory`1.Send[TMessage](ConsumeContext`1 context, IPipe`1 next)
at MassTransit.Pipeline.ConsumerFactories.DelegateConsumerFactory`1.Send[TMessage](ConsumeContext`1 context, IPipe`1 next)
at MassTransit.Pipeline.Filters.ConsumerMessageFilter`2.GreenPipes.IFilter<MassTransit.ConsumeContext<TMessage>>.Send(ConsumeContext`1 context, IPipe`1 next)
at MassTransit.Pipeline.Filters.ConsumerMessageFilter`2.GreenPipes.IFilter<MassTransit.ConsumeContext<TMessage>>.Send(ConsumeContext`1 context, IPipe`1 next)
at GreenPipes.Partitioning.Partition.Send[T](T context, IPipe`1 next)
at GreenPipes.Filters.TeeFilter`1.<>c__DisplayClass5_0.<<Send>g__SendAsync|1>d.MoveNext()
--- End of stack trace from previous location ---
at GreenPipes.Filters.OutputPipeFilter`2.SendToOutput(IPipe`1 next, TOutput pipeContext)
at GreenPipes.Filters.OutputPipeFilter`2.SendToOutput(IPipe`1 next, TOutput pipeContext)
at GreenPipes.Filters.DynamicFilter`1.<>c__DisplayClass10_0.<<Send>g__SendAsync|0>d.MoveNext()
--- End of stack trace from previous location ---
at MassTransit.Pipeline.Filters.DeserializeFilter.Send(ReceiveContext context, IPipe`1 next)
at GreenPipes.Filters.RescueFilter`2.GreenPipes.IFilter<TContext>.Send(TContext context, IPipe`1 next)
MT-Fault-ConsumerType: MassTransit.QuartzIntegration.ScheduleMessageConsumer
MT-Fault-MessageType: MassTransit.Scheduling.ScheduleMessage
This is how the ConfigureServices is setup:
.ConfigureServices((host, services) =>
{
services.Configure<OtherOptions>(host.Configuration);
services.Configure<QuartzOptions>(host.Configuration.GetSection("Quartz"));
services.AddSingleton<QuartzConfiguration>();
services.AddMassTransit(x =>
{
x.UsingRabbitMq((context, cfg) =>
{
var options = context.GetService<QuartzConfiguration>();
cfg.AddScheduling(s =>
{
s.SchedulerFactory = new StdSchedulerFactory(options.Configuration);
s.QueueName = options.Queue;
});
var vhost = host.Configuration.GetValue<string>("RabbitMQ:VirtualHost");
cfg.Host(string.Empty, vhost, h =>
{
h.Username( host.Configuration.GetValue<string>("RabbitMQ:User"));
h.Password(host.Configuration.GetValue<string>("RabbitMQ:Password"));
h.UseCluster(c =>
{
c.Node(host.Configuration.GetValue<string>("RabbitMQ:Node1"));
c.Node(host.Configuration.GetValue<string>("RabbitMQ:Node2"));
});
});
});
});
services.AddMassTransitHostedService();
});
And this is how the quartz configs are setup:
public NameValueCollection Configuration
{
get
{
var configuration = new NameValueCollection(13)
{
{"quartz.scheduler.instanceName", _options.Value.InstanceName},
{"quartz.scheduler.instanceId", "AUTO"},
{"quartz.plugin.timeZoneConverter.type","Quartz.Plugin.TimeZoneConverter.TimeZoneConverterPlugin, Quartz.Plugins.TimeZoneConverter"},
{"quartz.serializer.type", "json"},
{"quartz.threadPool.type", "Quartz.Simpl.SimpleThreadPool, Quartz"},
{"quartz.threadPool.threadCount", (_options.Value.ThreadCount ?? 10).ToString("F0")},
{"quartz.jobStore.misfireThreshold", "60000"},
{"quartz.jobStore.type", "Quartz.Impl.AdoJobStore.JobStoreTX, Quartz"},
{"quartz.jobStore.driverDelegateType", "Quartz.Impl.AdoJobStore.SqlServerDelegate, Quartz"},
{"quartz.jobStore.tablePrefix", _options.Value.TablePrefix},
{"quartz.jobStore.dataSource", "default"},
{"quartz.dataSource.default.provider", _options.Value.Provider},
{"quartz.dataSource.default.connectionString", _options.Value.ConnectionString},
{"quartz.jobStore.useProperties", "true"}
};
foreach (var key in configuration.AllKeys)
{
_logger.LogInformation("{Key} = {Value}", key, configuration[key]);
}
return configuration;
}
}
Also the extension method to setup MassTransit scheduling:
public static void AddScheduling(this IBusFactoryConfigurator configurator, Action<InMemorySchedulerOptions> configure)
{
if (configurator == null)
throw new ArgumentNullException(nameof(configurator));
var options = new InMemorySchedulerOptions();
configure?.Invoke(options);
if (options.SchedulerFactory == null)
throw new ArgumentNullException(nameof(options.SchedulerFactory));
var observer = new SchedulerBusObserver(options);
configurator.ReceiveEndpoint(options.QueueName, e =>
{
var partitioner = configurator.CreatePartitioner(Environment.ProcessorCount);
e.Consumer(() => new ScheduleMessageConsumer(observer.Scheduler), x =>
x.Message<ScheduleMessage>(m => m.UsePartitioner(partitioner, p => p.Message.CorrelationId)));
e.Consumer(() => new CancelScheduledMessageConsumer(observer.Scheduler), x =>
x.Message<CancelScheduledMessage>(m => m.UsePartitioner(partitioner, p => p.Message.TokenId)));
configurator.UseMessageScheduler(e.InputAddress);
configurator.ConnectBusObserver(observer);
});
}
I tried debugging and couldn't find a reason for the error I'm receiving, the database connections are being created successfully and also the message scheduling on the consumer is being completed successfully.
Is there a way to actually debug while the scheduler is reading the message or to know what's exactly missing that is throwing the Object reference error?
EDIT
Debugging the Masstransit.Quartz framework I noticed the issue is while trying to deserialize the message on method TranslateJsonBody.
var envelope = JObject.Parse(body);
envelope["destinationAddress"] = destination;
var message = envelope["message"];
var payload = message["payload"];
var payloadType = message["payloadType"];
As you can see, the method uses camelCase, and our broker is sending with PascalCase, and since there's no handler, it's returning null when the method tries to get envelope["message"] because we're sending it as "Message". Is there any configuration to force camelCase when deserializing?

spring boot DeferredResult onError how to invoke the callback?

Need to perform some asynchronous processing in a Rest service without holding up the server's Http threads .
I think DeferredResult would be a good option.
However when I am trying to ensure my callback on error gets called - am not able to do so .
Here is a naive attempt on my part:
#GetMapping("/getErrorResults")
public DeferredResult<ResponseEntity<?>> getDeferredResultsError(){
final String METHOD_NAME = "getDeferredResultsError";
logger.info("START : {}",METHOD_NAME);
DeferredResult<ResponseEntity<?>> deferredOutput = new DeferredResult<>();
ForkJoinPool.commonPool().submit(() -> {
logger.info("processing in separate thread");
int age = 0;
try {
age = age / 0;
}catch(Exception e) {
logger.error("we got some error");
logger.error(e);
throw e;
}
logger.info("after try catch block");
});
deferredOutput.onError((Throwable t) -> {
logger.error("<<< HERE !!! >>>");
deferredOutput.setErrorResult(
ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
.body(t.getMessage()));
});
logger.info("done");
return deferredOutput;
}
When I call this Rest endpoint from Postman - I can see in server logs the arithmetic exception by zero but dont see the 'onError' getting invoked.
After some time get a response in Postman as follows:
{
"timestamp": "2019-07-30T09:57:16.854+0000",
"status": 503,
"error": "Service Unavailable",
"message": "No message available",
"path": "/dfr/getErrorResults"
}
So my question is how does the 'onError' get invoked ?
You need to pass the DeferredResult object to the asynchronous operation so you could update it in case of success or failure:
#GetMapping(value = "/getErrorResults")
public DeferredResult<ResponseEntity<String>> getDeferredResultsError() {
DeferredResult<ResponseEntity<String>> deferredResult = new DeferredResult<>();
ForkJoinPool.commonPool().submit(() -> {
System.out.println("Processing...");
int age = 0;
try {
age = age / 0;
deferredResult.setResult(ResponseEntity.ok("completed"));
}catch(Exception e) {
System.out.println("Failed to process: " + e.getMessage());
deferredResult.setErrorResult(
ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR)
.body(e.getMessage()));
}
});
return deferredResult;
}
In the code you posted, you returned the DeferredResult object without passing it to the asynchronous operation. So after your return it, SpringMVC holds the client connection and wait until the DeferredResult object will be assigned with some kind of result. But in your case, the DeferredResult is not held by the asynchronous operation and will never updated so you get "Service Unavailable".
Here you can find working (light) project example.

Multiple subscribers to the OnSendingHeaders event in an owin middleware

I have two middlewares similar to the ones below which are both subscribing to the OnSendingHeaders event. One sets some headers and the other one removes some headers:
public class Middleware1 : OwinMiddleware
{
public Middleware1(OwinMiddleware next) : base(next)
{
}
public override async Task Invoke(IOwinContext context)
{
context.Response.OnSendingHeaders(state =>
{
var res = (OwinResponse)state;
// im removing some headers
}, context.Response);
await Next.Invoke(context);
}
}
public class Middleware2 : OwinMiddleware
{
public Middleware2(OwinMiddleware next) : base(next)
{
}
public override async Task Invoke(IOwinContext context)
{
context.Response.OnSendingHeaders(state =>
{
var res = (OwinResponse)state;
// im setting some headers
}, context.Response);
await Next.Invoke(context);
}
}
I also have web api controllers that creates excel and pdf-documents and returns an IHttpActionResult. That piece of code is working as intended.
However, when I issue a request to a controller that returns a pdf/excel-document, the response gets aborted in my browser after the headers are sent. The response is sent back as a http status code 200 and the correct Content-Length header is sent, but the file to be downloaded gets aborted.
This error seems to be tied to the subscribing of the OnSendingHeaders event. Because if I remove the middlewares or set/remove the headers without subscribing to the event (calling context.Response.Headers directly), everything works fine.
Is this event not supposed to be subscribed to multiple times or what could otherwise be causing this issue?

Resources