I was wondering if there is a way to raise events on new subscription to a publisher socket
For example:
PublisherSocket publisher = new PublisherSocket();
publisher.Bind("tcp://*:5555");
NetMQPoller poller = new NetMQPoller { publisher };
poller.RunAsync();
poller.OnSubscription += topic =>
{
AddToPool(topic);
};
Task.Factory.StartNew(() =>
{
SubscriberSocket sub = new SubscriberSocket();
sub.Connect("tcp://127.0.0.1:5555");
Thread.Sleep(1000);
sub.Subscribe("A");
}, TaskCreationOptions.LongRunning);
publisher.ReceiveReady += Publisher_ReceiveReady;
Of course, OnSubscription doesn't really exist, but I was wondering if there is any way around it.
I need my server to be aware of all the subscriptions.
I though about two ways to implement it:
Create an additional router socket in the server, all subscriptions will be sent both to the publisher socket and to the router socket (unsubscriptions as well). This will allow me to poll the subscriptions from the router.
Not use publisher/subscriber at all, create all the pubsub mechanism with router/dealer.
What would you suggest me do?
If you use an XPUB rather than PUB socket you can receive the subscription messages as you would regular messages on any other socket type.
Related
How to connect an endpoint(exchange-exchange-queue) in masstransit, accumulate data in the queue, and then, after some time, connect a consumer to this endpoint?
I wanted to do something like:
Task.Run(async () =>
{
for (var i = 0;; i++)
{
await _bus.Publish(new Event(i), stoppingToken);
await Task.Delay(1_000, stoppingToken);
}
});
// a command comes to connect the consumer
var endpoint = _bus.ConnectReceiveEndpoint();
await endpoint.Ready;
// I prepare the consumer, as soon as it is ready, I connect it,
then I read the data that has accumulated during the preparation
(my consumer needs to load the state before reading the data,
and also cannot skip the data during its preparation)
endpoint.ReceiveEndpoint.ConnectConsumer(() =>
_serviceProvider.GetRequiredService<EventConsumer>());
but this code will not create an exchange-exchange relationship, so the queue will be empty
If the exchange bindings (wired to the receive endpoint, which ultimately is a queue) do not exist when messages are published, they are discarded by RabbitMQ.
You would need to connect the receive endpoint in advance, so that the messages end up in the queue.
Can some one help how to configure NATS jet stream subscription in spring boot asynchronously example: looking for an equivalent annotation like #kafkalistener for Nats jetstream
I am able to pull the messages using endpoint but however when tried to pull messages using pushSubscription dispatcherhandler is not invoked. Need to know how to make the listener to be active and consume messages immediately once the messages are published to the subject.
Any insights /examples regarding this will be helpful, thanks in advance.
I don't know what is your JetStream retention policy, neither the way you want to subscribe. But I have sample code for WorkQueuePolicy push subscription, wish this will help you.
public static void subscribe(String streamName, String subjectKey,
String queueName, IMessageHandler iMessageHandler) throws IOException,
InterruptedException, JetStreamApiException {
long s = System.currentTimeMillis();
Connection nc = Nats.connect(options);
long e = System.currentTimeMillis();
logger.info("Nats Connect in " + (e - s) + " ms");
JetStream js = nc.jetStream();
Dispatcher disp = nc.createDispatcher();
MessageHandler handler = (msg) -> {
try {
iMessageHandler.onMessageReceived(msg);
} catch (Exception exc) {
msg.nak();
}
};
ConsumerConfiguration cc = ConsumerConfiguration.builder()
.durable(queueName)
.deliverGroup(queueName)
.maxDeliver(3)
.ackWait(Duration.ofMinutes(2))
.build();
PushSubscribeOptions so = PushSubscribeOptions.builder()
.stream(streamName)
.configuration(cc)
.build();
js.subscribe(subjectKey, disp, handler, false, so);
System.out.println("NatsUtil: " + durableName + "subscribe");
}
IMessageHandler is my custom interface to handle nats.io received messages.
First, configure the NATS connection. Here you will specify all your connection details like server address(es), authentication options, connection-level callbacks etc.
Connection natsConnection = Nats.connect(
new Options.Builder()
.server("nats://localhost:4222")
.connectionListener((connection, eventType) -> {})
.errorListener(new ErrorListener(){})
.build());
Then construct a JetStream instance
JetStream jetStream = natsConnection.jetStream();
Now you can subscribe to subjects. Note that JetStream consumers can be durable or ephemeral, can work according to push or pull logic. Please refer to NATS documentation (https://docs.nats.io/nats-concepts/jetstream/consumers) to make the appropriate choice for your specific use case. The following example constructs a durable push consumer:
//Subscribe to a subject.
String subject = "my-subject";
//queues are analogous to Kafka consumer groups, i.e. consumers belonging
//to the same queue (or, better to say, reading the same queue) will get
//only one instance of each message from the corresponding subject
//and only one of those consumers will be chosen to process the message
String queueName = "my-queue";
//Choosing delivery policy is analogous to setting the current offset
//in a partition for a consumer or consumer group in Kafka.
DeliverPolicy deliverPolicy = DeliverPolicy.New;
PushSubscribeOptions subscribeOptions = ConsumerConfiguration.builder()
.durable(queueName)
.deliverGroup(queueName)
.deliverPolicy(deliverPolicy)
.buildPushSubscribeOptions();
Subscription subscription = jetStream.subscribe(
subject,
queueName,
natsConnection.createDispatcher(),
natsMessage -> {
//This callback will be called for incoming messages
//asynchronously. Every subscription configured this
//way will be backed by its own thread, that will be
//used to call this callback.
},
true, //true if you want received messages to be acknowledged
//automatically, otherwise you will have to call
//natsMessage.ack() manually in the above callback function
subscribeOptions);
As for the declarative API (i.e. some form of #NatsListener annotation analogous to #KafkaListener from Spring for Apache Kafka project), there is none available out of the box in Spring. If you feel like you absolutely need it, you can write one yourself, if you are familiar with Spring BeanPostProcessor-s or other extension mechanism that can help to do that. Alternatively you can refer to 3rd party libs, it looks like a bunch of people (including myself) felt a bit uncomfortable when switching from Kafka to NATS, so they tried to bring the usual way of doing things with them from the Kafka world. Some examples can be found on github:
https://github.com/linux-china/nats-spring-boot-starter,
https://github.com/dstrelec/nats
https://github.com/amalnev/declarative-nats-listeners
There may be others.
I am working with Spring WebSocket and Stomp.
Note: the broker is ActiveMQ
I have two #Schedule methods that sends messages to Queue and Topic respectively
For a subscription for a Queue, the code is as follows:
$('#ws_connect').click(function(){
console.log('Connect clicked');
var socket = new SockJS('/project-app/ws/notification');
stompClient = Stomp.over(socket);
stompClient.connect({}, function (frame) {
console.log('Connected: ' + frame);
$('#notification').append('<p>Connected</p>');
stompClient.subscribe('/queue/somedestination', function (notification) {
... append the content to the html page, it works
});
});
});
If there is no user the Queue is getting message by message. Once the user arrives and do the connection, automatically it gets all the messages enqueued and the Queue pass to be empty. If a new message arrives it appears automatically, it because the user remains connected yet. If the user disconnects and later do the connection, it can see again all the messages enqueued. Until here all is Ok
As follows for a subscription for a Topic
$('#ws_connect').click(function(){
console.log('Connect clicked');
var socket = new SockJS('/project-app/ws/notification');
stompClient = Stomp.over(socket);
stompClient.connect({}, function (frame) {
console.log('Connected: ' + frame);
$('#notification').append('<p>Connected</p>');
stompClient.subscribe('/topic/somedestination', function (notification) {
... append the content to the html page, it works
});
});
});
Practically the js code is the same than the Queue version. Just the destination is different.
Here the problem is that if exists messages enqueued in the Topic and if the user does the connection the messages do not appear automatically. Of course meanwhile the user remains connected he can see each new message, same case if multiple users are connected to the same Topic all can see the same new message.
But again for the first user to connect to the Topic destination. He is not able to get the messages enqueued from that Topic when he does the connection.
Is it the normal behaviour?
How can be retrieved the messages enqueued?
Perhaps a suggested approach to handle this scenario?
In some way the "worst" scenario would be create multiple Queues for each potential user. But just curious if through Topic is possible get this requeriment
The Topic destination does not store messages when there are no subscribers so the code is working as expected. The only time a Topic would retain messages for subscribers is if the subscriber had created a durable topic subscription and then gone offline at which point the broker would store any message sent to the Topic with the persistent flag enabled. The caveat here being that any message sent to that Topic before the subscription is made would be dropped.
See the documentation for more help.
I'm new to ZeroMQ (and to networking in general), and have a question about using ZeroMQ in a setup where multiple clients connect to a single server. My situation is as follows:
--1 server
--multiple clients
--Clients send messages to server: I've already figured out how to do this part.
--Server sends messages to a specific client: This is the part I'm having trouble with. When certain events get handled on the server, the server will need to send a message to a specific client -- not all clients. In other words, the server will need to be able to choose which client to send a given message to.
Right now, this is my server code:
using (NetMQContext ctx = NetMQContext.Create())
{
using (var server = ctx.CreateResponseSocket())
{
server.Bind(#"tcp://127.0.0.1:5555");
while (true)
{
string fromClientMessage = server.ReceiveString();
Console.WriteLine("From Client: {0}", fromClientMessage);
server.Send("ack"); // There is no overload for the 'Send'
method that takes an IP address as an argument!
}
}
}
I have a feeling that the problem is that my design is wrong, and that the ResponseSocket type isn't meant to be used in the way that I want to use it. Since I'm new to this, any advice is very much appreciated!
when using the Response socket you always replying to the client that sent you the message. So the Request-Response socket types together are just simple request response.
To more complicated scenarios you probably want to use Dealer-Router.
With router the first frame of each message is the routing id (the identity of the client that sent you the message)
so your example with router will look like:
using (NetMQContext ctx = NetMQContext.Create())
{
using (var server = ctx.CreateRouterSocket())
{
server.Bind(#"tcp://127.0.0.1:5555");
while (true)
{
byte[] routingId = server.Receive();
string fromClientMessage = server.ReceiveString();
Console.WriteLine("From Client: {0}", fromClientMessage);
server.SendMore(routingId).Send("ack");
}
}
}
I also suggest to read the zeromq guide, it will probably answer most of your questions.
I'm trying to create a listener in tibco rendezvous. I want to listen in on a particular subject. I'm aware that its supposed to look something like this:
TibrvListener(
TibrvQueue queue,
TibrvMsgCallback callback,
TibrvTransport transport,
java.lang.String subject,
java.lang.Object closure)
throws TibrvException
I have this code. However, I don't know a couple of things. How do I create a TibrvMsgCallback object? How do I pass in the transport? I have a publisher that sends the message as a seperate program. Do I recreate an identical transport in my subscribe program?
queue = new TibrvQueue();
dispatcher = new TibrvDispatcher(queue);
queue.setName(key);
this.listener = new TibrvListener(queue, null, null, subject, null);
TibrvTransport message = this.listener.getTransport();
You first open the Tibrv
Tibrv.open(Tibrv.IMPL_NATIVE);
Create transport
TibrvTransport transport = new TibrvRvdTransport(service, network, daemon);
Create Listener
new TibrvListener(Tibrv.defaultQueue(), this, transport, subject, null);
If your listener is "this", your class needs to implement TibrvMsgCallback
Messages can be processed on arrival in the onMsg(TibrvListener listener, TibrvMsg msg) method.