I am trying to create an exchange of type amq.fanout using rabbitmq-c, an amqp client for RabbitMQ . However, I'm getting tripped up on what exactly I have to declare for it. According to this, I have to declare the exchange name and type, and optionally durability and lifetime semantics for the exchange.
When I declare an exchange, it looks like I can supply the 'exchange', which I presume is the name of the exchange, and well as the exchange type, e.g. amq.fanout. From examples/amqp_exchange_declare.c
amqp_exchange_declare(conn,
1,
amqp_cstring_bytes(exchange),
amqp_cstring_bytes(exchangetype),
0, 0, 0, 0,
amqp_empty_table);
But then if I look at an example of publishing, there is a place to supply the exchange type, in the second parameter which is named exchange. For example, examples/amqp_producer.c
amqp_basic_publish(conn,
1,
amqp_cstring_bytes("amq.direct"),
amqp_cstring_bytes(queue_name),
0,
0,
NULL,
message_bytes)
But there's no place to supply the name of the exchange. So how can I publish to a named exchange?
And why the heck am I supplying a queue_name to publish? I should only be publishing to exchanges - queues are supposed to be hidden from the publisher. Is this all just poorly named variables in the examples or am I missing something fundamental?
looking at the docs I see the 3rd parameter is the exchange to use.
int amqp_basic_publish ( amqp_connection_state_t state,
amqp_channel_t channel,
amqp_bytes_t exchange,
amqp_bytes_t routing_key,
amqp_boolean_t mandatory,
amqp_boolean_t immediate,
struct amqp_basic_properties_t_ const * properties,
amqp_bytes_t body
)
what's probably confusing is the use of the "amq.direct" exchange in many of the examples... this is not the exchange type. rather, this is a built-in default exchange that is named "amq.direct". it happens to be a "direct" exchange type, but "amq.direct" is the name of the exchange, not the type.
The naming is a bit confusing, at first... it took me nearly a year to figure this out, honestly. :)
It's common for simple examples to do a "publish to queue" where you push the message through the amq.direct exchange with the queue name as the routing key. this will publish the message directly to that queue. most of the time, this is only useful for demos or in an RPC (request/response) scenario where you need to publish a response to a specific reply-to queue.
i wrote a blog post on the relationships between exchanges, queues and bindings a while back. it may help clear up some of the confusion.
hope that helps!
Related
New to Masstransit here and I wanted to correlate my message with an int value (let's call it OrderId of type INT. I will use the same order management example from Masstransit).
I wanted to see if anyone was able to use an int to correlate events and messages in Masstransit.
I used this code in my events (both the events)
x.CorrelateById<int>(state => state.OrderId, context => context.Message.OrderId)
This compiles fine but then throws and exception
NotImplementedByDesignException. Masstransit doc says : "Redis only supports event correlation by CorrelationId, it does not support queries. If a saga uses expression-based correlation, a NotImplementedByDesignException will be thrown."
I am using CorrelateById() so not sure why I am seeing this exception. I don't see any query here (or is it this func that returns the OrderId? The correlationId has a similar expression although it takes only one argument of type: Func<ConsumeContext<TMessage>, Guid> selector)
All I want is to be able to correlate message and event by an Int prop coming to my first event of the state machine. (That I have no control over by the way).
I feel creating another GUID just for Masstransit and link it to that OrderId is not the best option here.
My other question: is there any out of the box way in MassTransit to get the all event data for a specific CorrelationId (event sourcing style). I saw there was a MassTransit.EventStoreIntegration repo on github but since we can get the last version of the state instance I thought there would be a way to see all state instances/ or messages being persisted then pulled.
Sure I can see that in the log but I would like to see only the state changes that were pushed.
Thanks
Correlation by anything other than the Guid CorrelationId property is a query. And, as you've found out, Redis doesn't support queries. If you need to correlate events using a type other than a Guid, Redis is not an option.
As per #Chris answer, and because I wanted to use an INT property and still use redis, I used Guid CorrelationId that was generated from that INT value with padded zeros.
CorrelationId = new Guid(OrderIdIntValue.ToString().PadLeft(32, '0'))
I am aware of the Guid vs int discussion in distributed systems.
For my case this works with no issues since that Int value is generated by a unique source of truth (that I consume and don't have control over).
What is the best practice to handle seen/unseen messages in a chat room application based on Nodejs/SocketIO/React.
Consider User1 sends a message to a room. If another user has seen that message, notify all users that the state of message has been seen.
In my opinion using message brokers can be the better solution instead socket. I actually think that socket should only handle chat messages that are synchronously. but for seen/unseen status I prefer message brokers that are asynchronous. Are there any solutions or best practice in large scale applications?
It's unclear what you have currently tried, meaning that I can only advise solutions in order to achieve your aim.
To firstly identify that a message was seen, IntersectionObserver is an inbuilt API that detects when an element has entered the viewport, meaning that it is visible, therefore; obviously seen. I have added comments in the code below where you should add a function to call to the server that the message was seen, however, that's up to you to implement.
const observer = new window.IntersectionObserver(([entry]) => {
if (entry.isIntersecting) {
// Send a message to the server that the user has viewed the message.
// Eg. socket.emit('read-message', message.id)
return
}
}, {
root: null,
threshold: 0.1,
})
observer.observe(document.getElementById(message.id));
Additionally, there's no need to use message broker, as socket.io can handle simple interactions such as this.
You then need to send a message to the server that denotes the specified message ID was seen, then broadcast to every other client that the state was changed, and update it to read - if that's needed.
I'm wondering if I'm doing something wrong, I expected MassTransit would automatically register ReceiveEndpoints in the EndpointConvention.
Sample code:
services.AddMassTransit(x =>
{
x.AddServiceBusMessageScheduler();
x.AddConsumersFromNamespaceContaining<MyNamespace.MyRequestConsumer>();
x.UsingAzureServiceBus((context, cfg) =>
{
// Load the connection string from the configuration.
cfg.Host(context.GetRequiredService<IConfiguration>().GetValue<string>("ServiceBus:ConnectionString"));
cfg.UseServiceBusMessageScheduler();
// Without this line I'm getting an error complaining about no endpoint convention for x could be found.
EndpointConvention.Map<MyRequest>(new Uri("queue:queue-name"));
cfg.ReceiveEndpoint("queue-name", e =>
{
e.MaxConcurrentCalls = 1;
e.ConfigureConsumer<MyRequestConsumer>(context);
});
cfg.ConfigureEndpoints(context);
});
});
I thought this line EndpointConvention.Map<MyRequest>(new Uri("queue:queue-name")); wouldn't be necessary to allow sending to the bus without specifing the queue name, or am I missing something?
await bus.Send<MyRequest>(new { ...});
The EndpointConvention is a convenience method that allows the use of Send without specifying the endpoint address. There is nothing in MassTransit that will automatically configured this because, frankly, I don't use it. And I don't think anyone else should either. That stated, people do use it for whatever reason.
First, think about the ramifications - if every message type was registered as an endpoint convention, what about messages that are published and consumed on multiple endpoints? That wouldn't work.
So, if you want to route messages by message type, MassTransit has a feature for that. It's called Publish and it works great.
But wait, it's a command, and commands should be Sent.
That is true, however, if you are in control of the application and you know that there is only one consumer in your code base that consumes the KickTheTiresAndLightTheFires message contract, publish is as good as send and you don't need to know the address!
No, seriously dude, I want to use Send!
Okay, fine, here are the details. When using ConfigureEndpoints(), MassTransit uses the IEndpointNameFormatter to generate the receive endpoint queue names based upon the types registered via AddConsumer, AddSagaStateMachine, etc. and that same interface can be used to register your own endpoint conventions if you want to use Send without specifying a destination address.
You are, of course, coupling the knowledge of your consumer and message types, but that's your call. You're already dealing with magic (by using Send without an explicit destination) so why not right?
string queueName = formatter.Consumer<T>()
Use that string for the message types in that consumer as a $"queue:{queueName}" address and register it on the EndpointConvention.
Or, you know, just use Publish.
I've been trying my hands on Spring AMQP. And I have a couple of questions:
I'd like to know what is Publisher returns and how is it different from Publisher Confirm. Of my understanding, we have a Publisher Confirm Callback that checks the status of acks. Now I looked at the documentation in Spring AMQP and Rabbit MQ. didn't really find or understand much on this.
And also why is it that if the message is tried to send to a non-existing queue, I don't get any sort of acknowledgement (ack/nack
) nor do I get any errors. Is there a way to setTimeouts for non-acknowledgements?
Short answer from the link https://www.rabbitmq.com/confirms.html :
"For unroutable messages, the broker will issue a confirm once the exchange verifies a message won't route to any queue (returns an empty list of queues). If the message is also published as mandatory, the basic.return is sent to the client before basic.ack."
In Spring AMQP if you set 'spring.rabbitmq.publisherReturns' to true this will mean messages will be 'mandatory' (unless you set mandatory to false) because of the following code:
private boolean determineMandatoryFlag() {
Boolean mandatory = this.properties.getTemplate().getMandatory();
return (mandatory != null ? mandatory : this.properties.isPublisherReturns());
}
I suggest you to read this article. There is a good description of all possible acknowledgments scenarios, including returns for the unrouted messages, like your non-existing queue.
From the Spring AMQP perspective you should bear in mind: https://docs.spring.io/spring-amqp/docs/2.0.3.RELEASE/reference/html/_reference.html#template-confirms
This feature requires a CachingConnectionFactory that has its publisherReturns property set to true.
In my RabbitMQ, I have an topic exchanger called room-topic-exchange and the bindings are like this
When I send a message to an specific queue, using the exchanger, everything works fine. I'm sending as follow:
template.convertAndSend(ROOM_TOPIC_EXCHANGE, roomId, message);
but when I try to send to ALL queues, nothing happens. I'm trying as this
template.convertAndSend(ROOM_TOPIC_EXCHANGE, "room*", message);
I declared the exchanger and the bind as follow
TopicExchange allRooms = new TopicExchange(ROOM_TOPIC_EXCHANGE, false, true);
admin.declareExchange(allRooms);
admin.declareBinding(BindingBuilder.bind(q).to(allRooms).with(roomId));
I can't see what I'm doing wrong. I read the documentantion, and tried with routing key room# too and nothing happened.
The topic exchange doesn't work that way; you bind with wildcards, you don't use a wildcard in the routing key.
A queue bound with room.* will get messages sent to room.123 or room.124.
You can achieve what you want by adding a second binding to each room, say room.splat; then sending to room.splat will go to both queues.
Or, you can add a second fanout exchange. Bind both queues to both exchanges (no routing key needed for the fanout) and send broadcasts to the fanout exchange and directed messages to the topic.