Create custom exchange-to-exchange binding using MassTransit - masstransit

I have microservice-based system which works with documents. Service publishes DocflowErrorMq, ImportedDocflowMq events, and other services are subscribed to these events. Critical service DocflowRegistry should process messages quickly, so we have to introduce multiple consumers. On the other hand message order shouldn't be broken and competing consumer doesn't suite. Consistent hash exchange distributes messages by routing key equals to document id, messages related to one document goes to one queue. So, we have simple manual scaling. I can't create binding between MqModels.Docflows:ImportedDocflowMq and docflow-process-dr exchanges (marked red on Diagram). Is it possible to create it with MassTransit?
DocflowRegistry service config:
services.AddMassTransit(x =>
{
x.AddConsumer<DocflowSendingErrorTestConsumer>();
x.AddConsumer<DocflowImportTestConsumer>();
x.UsingRabbitMq((context, cfg) =>
{
var virtualHost = configuration["RabbitMq:Settings:VirtualHost"] ?? "/";
cfg.Host(configuration["RabbitMqHost"], virtualHost, h =>
{
h.Username(configuration["RabbitMqUserName"]);
h.Password(configuration["RabbitMqPassword"]);
});
cfg.ReceiveEndpoint("docflow.process-1.docflowregistry", e =>
{
e.ConfigureConsumer<DocflowSendingErrorTestConsumer>(context);
e.ConfigureConsumer<DocflowImportTestConsumer>(context);
e.Bind("docflow-process-dr", x =>
{
x.Durable = true;
x.AutoDelete = false;
x.ExchangeType = "x-consistent-hash";
x.RoutingKey = "1";
});
e.ConfigureConsumeTopology = false;
e.SingleActiveConsumer = true;
});
cfg.ReceiveEndpoint("docflow.process-2.docflowregistry", e =>
{
e.ConfigureConsumer<DocflowSendingErrorTestConsumer>(context);
e.ConfigureConsumer<DocflowImportTestConsumer>(context);
e.Bind("docflow-process-dr", x =>
{
x.Durable = true;
x.AutoDelete = false;
x.ExchangeType = "x-consistent-hash";
x.RoutingKey = "1";
});
e.ConfigureConsumeTopology = false;
e.ConcurrentMessageLimit = 1;
e.SingleActiveConsumer = true;
});
});
});
Config of TodoList service:
services.AddMassTransit(x =>
{
x.AddConsumer<DocflowSendingErrorTestConsumer>();
x.AddConsumer<DocflowImportTestConsumer>();
x.UsingRabbitMq((context, cfg) =>
{
var virtualHost = configuration["RabbitMq:Settings:VirtualHost"] ?? "/";
cfg.Host(configuration["RabbitMqHost"], virtualHost, h =>
{
h.Username(configuration["RabbitMqUserName"]);
h.Password(configuration["RabbitMqPassword"]);
});
cfg.ReceiveEndpoint("docflow-process-todolist", e =>
{
e.ConfigureConsumer<DocflowSendingErrorTestConsumer>(context);
e.ConfigureConsumer<DocflowImportTestConsumer>(context);
e.SingleActiveConsumer = true;
});
});
});
Publish code:
var endPoint = await _massTransitBus.GetPublishSendEndpoint<DocflowErrorMq>();
var docflowGuid = Guid.NewGuid();
await endPoint.Send(new DocflowErrorMq
{
DocflowId = docflowGuid,
AbonentId = Guid.NewGuid()
},
context =>
{
context.SetRoutingKey(docflowGuid.ToString());
});

Create an interface, DocflowProcessDr, and make each of those message contracts published implement it. Then, you can configure the publish topology for that interface in the bus:
cfg.Message<DocflowProcessDr>(x => x.SetEntityName("docflow-process-dr"));
cfg.Publish<DocflowProcessDr>(x =>
{
x.ExchangeType = "x-consistent-hash";
});
Since MassTransit will create a polymorphic topology on the broker, you'll have an exchange-to-exchange binding between the published type and the interface.
Then, just publish the message:
var docflowGuid = Guid.NewGuid();
var endPoint = await _massTransitBus.Publish<DocflowErrorMq>(new DocflowErrorMq
{
DocflowId = docflowGuid,
AbonentId = Guid.NewGuid()
},
context =>
{
context.SetRoutingKey(docflowGuid.ToString());
});
Calling GetPublishSendEndpoint<T>() is weird, don't encourage it.

Related

Masstransit channels

I configured MassTransit on my .NET core application as follows:
public void ConfigureServices(IServiceCollection services)
{
[...]
// producer
services.AddMassTransit(x =>
{
x.AddBus(provider => Bus.Factory.CreateUsingRabbitMq(cfg =>
{
cfg.Host(new Uri(_configuration["RabbitMQ:URI"] + _configuration["RabbitMQ:VirtualHost"]), $"ENG {_configuration["SiteID"]} Producer", h =>
{
h.Username(_configuration["RabbitMQ:UserName"]);
h.Password(_configuration["RabbitMQ:Password"]);
});
cfg.Publish<NormUpdate>(x =>
{
x.Durable = true;
x.AutoDelete = false;
x.ExchangeType = "fanout"; // default, allows any valid exchange type
});
cfg.ConfigurePublish(x => x.UseExecute(x =>
{
x.Headers.Set("SiteID", _configuration["SiteID"]);
}));
}));
});
services.AddMassTransit<ISecondBus>(x =>
{
x.AddConsumer<NormConsumer>();
x.AddBus(context => Bus.Factory.CreateUsingRabbitMq(cfg =>
{
cfg.UseMessageRetry(r => r.Intervals(100, 200, 500, 800, 1000));
cfg.Host(new Uri(_configuration["RabbitMQ:URI"] + _configuration["RabbitMQ:VirtualHost"]), $"ENG {_configuration["SiteID"]} Consumer", h =>
{
h.Username(_configuration["RabbitMQ:UserName"]);
h.Password(_configuration["RabbitMQ:Password"]);
});
cfg.ReceiveEndpoint($"norm-queue-{_configuration["SiteID"]}", e =>
{
e.Durable = true;
e.AutoDelete = false;
e.Consumer<NormConsumer>(context);
e.UseConcurrencyLimit(1);
e.ExchangeType = "fanout";
e.PrefetchCount = 1;
});
}));
});
services.AddOptions<MassTransitHostOptions>().Configure(options =>
{
options.WaitUntilStarted = false;
options.StopTimeout = TimeSpan.FromSeconds(30);
});
[...]
}
public interface ISecondBus : IBus
{
}
I noticed that when connections are created, the consumer connection has 2 channels. Channel (1) with no attached consumers, channel (2) with one consumer.
I expected to have only one channel on receiver.
Is this a normal behavior or am I doing something wrong?
You should have three channels:
One for the first bus you configured
One for the second bus you configured (MultiBus configures completely separate bus instances, nothing is shared)
One for the receive endpoint on the second bus you configured.
1 + 1 + 1 = 3
Q.E.D.
Yes, I was in Math club back in primary school.

Cannot Register saga using Masstransit.Integration.AspNetCore

I cannot register saga using Masstransit.AspNetCore package, it results on this error. I'm using Default Container with error "No service for type 'Automatonymous.Registration.ISagaStateMachineFactory' has been registered"
e.AddSagaStateMachine<CityAvailabilityStateMachine, CityAvailabilityState>(new NullSagaStateMachineRegistrar());
e.AddBus(provider =>{
var credentials = provider.GetService<Credential>();
return Bus.Factory.CreateUsingRabbitMq(cfg =>
{
var host = cfg.Host(new Uri(credentials.Uri), h =>
{
h.Username(credentials.UserName);
h.Password(credentials.Password);
h.Heartbeat(60);
});
cfg.ReceiveEndpoint(host,{credentials.BoundedContext}-sagas", configurator =>
{
configurator.PrefetchCount = 16;
configurator.UseRetry(r => r.Interval(2, 100));
configurator.ConfigureSaga<CityAvailabilityState>(provider);
});
});

How do i allow socket.io users to join a room with multiple namespaces?

I have a socket.js file on the server side, and I want my user communication to be separated into different namespaces, namely
this.mainRoom = io.of('/main_room')
this.privateRoom = io.of('/private_room')
When users join the room specific to '/main_room' namespace, it works fine.
But I want the user to be able to join the '/private_room' namespace.
Here's the code
const socks = function (io) {
let self = this
self.privateRoom = null
// where all the messaging and the events that are not part of the
// user-to-user communication
this.mainRoom = io.of('/main_room')
this.mainRoom.on('connection', (socket) => {
socket.on('set_name', (data) => {
const nickname = data.name
socket.nickname = nickname
socket.emit('name_set', data)
socket.send(JSON.stringify({
type: 'serverMessage',
nickname: socket.nickname,
message: "Welcome! "
})
)
socket.broadcast.emit('user_entered', {
nickname: socket.nickname,
})
})
socket.on('join_room', (room) => {
// concept roomname !== namespace
// concept Rooms are subchannels of the namespaces.
//create another socket, since socketio doesnt allow a single socket
//to be connected to multiple namespaces
socket.join(room.name, () => {
---------->> how do i allow user to join the room in the private_room namespace? <<---------
---------->> how should i create socket2 to join the same room.name? <<----------
// let socket2 = self.privateRoom.sockets[socket.id]
// socket2.join(room.name)
})
})
})
// split the socket code into 2 separate parts for namespaces
// this is so that the messaging and events that are related to users
// are in a namespace
// while messaging and events that are not part of user-to-user
// communication is separated
this.privateRoom = io.of('/private_room')
this.privateRoom.on('connection', (socket) => {
socket.on('message', (message) => {
message = JSON.parse(message)
if (message.type === "userMessage") {
socket.in(socket.room).broadcast.send(JSON.stringify(message))
message.type = "myMessage"
socket.send(JSON.stringify(message))
}
})
})
}
ah! silly error
Just had to rename this.privateRoom's socket as socket2 and make it a global variable
self.socket2 = null
this.privateRoom = io.of('/private_room')
this.privateRoom.on('connection', (socket2) => {
self.socket2 = socket2
self.socket2.on('message', (message) => {
message = JSON.parse(message)
if (message.type === "userMessage") {
self.socket2.in(self.socket2.room).send(JSON.stringify(message))
message.type = "myMessage"
self.socket2.send(JSON.stringify(message))
}
})
})
And give it back to mainRoom
let socket2 = self.socket2.join(room.name)
socket2.room = room.name
socket.in(room.name).emit('user_entered', {'name': socket.nickname})

Create an Rx.Subject using Subject.create that allows onNext without subscription

When creating an Rx.Subject using Subject.create(observer, observable), the Subject is so lazy. When I try to use subject.onNext without having a subscription, it doesn't pass messages on. If I subject.subscribe() first, I can use onNext immediately after.
Let's say I have an Observer, created like so:
function createObserver(socket) {
return Observer.create(msg => {
socket.send(msg);
}, err => {
console.error(err);
}, () => {
socket.removeAllListeners();
socket.close();
});
}
Then, I create an Observable that accepts messages:
function createObservable(socket) {
return Observable.fromEvent(socket, 'message')
.map(msg => {
// Trim out unnecessary data for subscribers
delete msg.blobs;
// Deep freeze the message
Object.freeze(msg);
return msg;
})
.publish()
.refCount();
}
The subject is created using these two functions.
observer = createObserver(socket);
observable = createObservable(socket);
subject = Subject.create(observer, observable);
With this setup, I'm not able to subject.onNext immediately (even if I don't care about subscribing). Is this by design? What's a good workaround?
These are actually TCP sockets, which is why I haven't relied on the super slick websocket subjects.
The basic solution, caching nexts before subscription with ReplaySubject:
I think all you wanted to do is use a ReplaySubject as your observer.
const { Observable, Subject, ReplaySubject } = Rx;
const replay = new ReplaySubject();
const observable = Observable.create(observer => {
replay.subscribe(observer);
});
const mySubject = Subject.create(replay, observable);
mySubject.onNext(1);
mySubject.onNext(2);
mySubject.onNext(3);
mySubject.subscribe(x => console.log(x));
mySubject.onNext(4);
mySubject.onNext(5);
Results in:
1
2
3
4
5
A socket implementation (example, don't use)
... but if you're looking at doing a Socket implementation, it gets a lot more complicated. Here is a working socket implementation, but I don't recommend you use it. Rather, I'd suggest that you use one of the community supported implementations either in rxjs-dom (if you're an RxJS 4 or lower) or as part of RxJS 5, both of which I've helped work on.
function createSocketSubject(url) {
let replay = new ReplaySubject();
let socket;
const observable = Observable.create(observer => {
socket = new WebSocket(url);
socket.onmessage = (e) => {
observer.onNext(e);
};
socket.onerror = (e) => {
observer.onError(e);
};
socket.onclose = (e) => {
if (e.wasClean) {
observer.onCompleted();
} else {
observer.onError(e);
}
}
let sub;
socket.onopen = () => {
sub = replay.subscribe(x => socket.send(x));
};
return () => {
socket && socket.readyState === 1 && socket.close();
sub && sub.dispose();
}
});
return Subject.create(replay, observable);
}
const socket = createSocketSubject('ws://echo.websocket.org');
socket.onNext('one');
socket.onNext('two');
socket.subscribe(x => console.log('response: ' + x.data));
socket.onNext('three');
socket.onNext('four');
Here's the obligatory JsBin

ReactiveUI Testing

I am attempting to see if the results of a view model are performing the correct actions.
My observables are setup as follows:
public FilterBoxViewModel()
{
var asyncFilterResults = this.filterItemsCommand.RegisterAsyncTask(x =>
this.PerformFilter(x as string));
this.filteredItems = new ObservableAsPropertyHelper<IEnumerable<IFilterBoxItem>>(
asyncFilterResults, _ => this.RaisePropertyChanged("FilteredItems"));
this.WhenAnyValue(x => x.SearchTerm)
.Throttle(TimeSpan.FromMilliseconds(50))
.Skip(1)
.Subscribe(this.filterItemsCommand.Execute);
}
Then further down I have
private async Task<IEnumerable<IFilterBoxItem>> PerformFilter(string searchTerm)
{
if (string.IsNullOrWhiteSpace(searchTerm))
{
return Enumerable.Empty<IFilterBoxItem>();
}
// Perform getting the items on the main thread and async await the results.
// This is provide a immutable version of the results so we don't cause
// threading issues.
var items = await Observable.Start(
() => this.FilterBoxManager.RootElements.GetAllItemsEnumerable()
.ToList().Select(x => new { Name = x.Name, Item = x }),
RxApp.MainThreadScheduler);
return
items.Where(x =>
x.Name.IndexOf(searchTerm, StringComparison.OrdinalIgnoreCase) >= 0)
.Select(x => x.Item);
}
In my test, I am running the test schedular and advancing it, however, I am getting the PerformFilter performing at different times than I expect
eg my test is:
(new TestScheduler()).With(scheduler =>
{
var viewModel = new FilterBoxViewModel();
var testManager = new TestManager { RootElements = this.sampleItems };
viewModel.FilterBoxManager = testManager;
viewModel.SearchTerm = "folder";
scheduler.AdvanceBy(TimeSpan.FromMilliseconds(51).Ticks);
Assert.AreEqual(viewModel.FilteredItems.Select(x => x.Name), folderSearchResults);
viewModel.SearchTerm = "apple";
Assert.AreEqual(viewModel.FilteredItems.Select(x => x.Name), appleSearchResults);
});
How do I make the tester more predictable?
I am running ReactiveUI 5.5.1 and in a XAML application.
Your Throttle doesn't set a scheduler, this is a classic TestScheduler mistake

Resources