From what I've read so-far, a JMS Topic is 1-to-Many and I wonder if its possible to support Many-to-Many using a topic. Consider a topic called "Reports" with multiple services spread out across an enterprise needing to publish scheduled reports. Having multiple publishers would alleviate the need to subscribe interested applications to a topic for each of the reporting services.
Note:
I'm going to use Spring and ActiveMQ in my solution.
#Mondain: yes, very much possible. A practical example would be live stock market price feed provided by multiple sources and those feed consumed by multiple channels.
Yes, you can create many TopicPublisher from your TopicSession, and many applications can connect the same Topic using TopicPublisher or TopicSubscriber.
You can do something like this, and call CreateMessageProducer to create a new instance of producer anywhere in your application.
public ActiveMqProducer(string activeMqServiceUrl)
{
_activeMqServiceUrl = activeMqServiceUrl;
IConnectionFactory factory = new ConnectionFactory(new Uri(_activeMqServiceUrl));
_activeMqConnection = factory.CreateConnection();
_activeMqSession = _activeMqConnection.CreateSession(AcknowledgementMode.Transactional);
_activeMqConnection.Start();
}
private IMessageProducer CreateMessageProducer(string mqTopicName)
{
ITopic destination = SessionUtil.GetTopic(_activeMqSession, mqTopicName);
var producer = _activeMqSession.CreateProducer(destination);
return producer;
}
Related
We have 5 topics and we want to have a service that scales for example to 5 instances of the same app.
This would mean that i would want to dynamically (via for example Redis locking or similar mechanism) determine which instance should listen to what topic.
I know that we could have 1 topic that has 5 partitions - and each node in the same consumer group would pick up a partition. Also if we have a separately deployed service we can set the topic via properties.
The issue is that those two are not suitable for our situation and we want to see if it is possible to do that via what i explained above.
#PostConstruct
private void postConstruct() {
// Do logic via redis locking or something do determine topic
dynamicallyDeterminedVariable = // SOME LOGIC
}
#KafkaListener(topics = "{dynamicallyDeterminedVariable")
void listener(String data) {
LOG.info(data);
}
Yes, you can use SpEL for the topic name.
#{#someOtherBean.whichTopicToUse()}.
We have UserCreated event that gets published from UserManagement.Api. I have two other Apis, Payments.Api and Notification.Api that should react to that event.
In both Apis I have public class UserCreatedConsumer : IConsumer<UserCreated> (so different namespaces) but only one queue (on SQS) gets created for both consumers.
What is the best way to deal with this situation?
You didn't share your configuration, but if you're using:
x.AddConsumer<UserCreatedConsumer>();
As part of your MassTransit configuration, you can specify an InstanceId for that consumer to generate a unique endpoint address.
x.AddConsumer<UserCreatedConsumer>()
.Endpoint(x => x.InstanceId = "unique-value");
Every separate service (not an instance of the same service) needs to have a different queue name of the receiving endpoint, as described in the docs:
cfg.ReceiveEndpoint("queue-name-per-service-type", e =>
{
// rest of the configuration
});
It's also mentioned in the common mistakes article.
I have a client app that connects to an Oracle AQ multi-consumer queue. I want to use OracleAQQueue.Listen to listen for new messages on the queue. API docs show that the Listen method can be used for multi-consumer queues. My code for listening to the queue is shown below.
string consumerName = "APPINST1";
using (OracleConnection con = new OracleConnection(connectionString))
{
con.Open();
OracleAQQueue queue = new OracleAQQueue("MY_Q");
queue.MessageType = OracleAQMessageType.Udt;
queue.UdtTypeName = "MY_Q_MSG";
queue.DequeueOptions.DeliveryMode = OracleAQMessageDeliveryMode.Persistent;
queue.Connection = con;
Console.WriteLine("Listening for messages...");
queue.Listen(new string[] { consumerName });
}
The problem that I'm having is that on the line of code where I call queue.Listen(), I get the Oracle exception:
ORA-25295: Subscriber is not allowed to dequeue buffered messages
Googling for advice on this particular error hasn't been too helpful. I've removed and re-added my subscriber to the queue several times to no avail. My guess is that I'm not setting some property correctly before I make the call to Listen, but I can't figure out the issue.
Any ideas?
I ran across the following note in the Streams Advanced Queuing User's Guide, in Chapter 10 - Oracle Streams AQ Operations Using PL/SQL:
Note: Listening to multiconsumer queues is not supported in the Java API.
Although I can't find it explicitly stated anywhere, I'm guessing the same rule applies to the ODP.NET API.
You must set the visibility attribute to IMMEDIATE to use buffered messaging.
I'm sending lots of data to my app through JMeter.
My subscribe block and the publisher look like this:
BunnyStarter.start_bunny_components
cons = BunnyStarter.queue.subscribe do |delivery_info, metadata, payload|
method_calling ( payload )
cons.cancel
end
BunnyStarter.exchange.publish(body.to_json, routing_key: BunnyStarter.queue.name)
And my BunnyStarter class:
def self.start_bunny_components
if ##conn.nil?
##conn = Bunny.new
##conn.start
##ch = ##conn.create_channel
##queue = ##ch.queue("dump_probe_queue")
##exchange = ##ch.default_exchange
end
end
The problem is, although I call consumer.cancel after method_calling, in my Rabbit MQ admin I still see that I get like one thousand consumers created in about 6 minutes.
Is that because of the rate and the amount of data I'm sending?
How can I improve this?
I have seen this issue before. The reason its creating 1000 of consumers is because you are creating channel per connection. Eventually your consumer will shut down after a while because of this.
The number of consumers getting created are not because of the data but its because in the consumer its creating one connection per subscription.
Solution:
Instead of creating multiple channels, create only one channel and use number of connections using the same channel.
I mean one instance of Connection and multiple instances of Model so you can share the same connection for multiple model.
I have to implement a chat application using websocket, users will chat via groups, there can be thousands of groups and a user can be in multiple groups. I'm thinking about 2 solutions:
[1] for each group chat, I create a websocket endpoint (using camel-atmosphere-websocket), users in the same group can subscribe to the group endpoint and send/receive message over that endpoint. it means there can be thousands of websocket endpoints. Client side (let's say iPhone) has to subscribes to multiple wbesocket endpoints. is this a good practice?
[2] I just create one websocket endpoint for all groups. Client side just subscribes to this endpoint and I manage the messages distribution myself on server: get group members, pick the websocket of each member from list of connected websockets then write the message to each member via websocket.
Which solution is better in term of performance and easy to implement on both client and server?
Thanks.
EDIT 2015-10-06
I chose the second approach and did a test with jetty websocket client, I use camel atmosphere websocket on server side. On client side, I create websocket connections to server in threads. There was a problem with jetty that I can just create around 160 websocket connections (it means around 160 threads). The result is that I almost see no difference when the number of clients increases from 1 to 160.
Yes, 160 is not a big number, but I think I will do more test when I actually see the performance problem, for now, I'm ok with second approach.
If you are interested in the test code, here it is:
http://www.eclipse.org/jetty/documentation/current/jetty-websocket-client-api.html#d0e22545
I think second approach will be better to use for performance. I am using the same for my application, but it is still in testing phase so can't comment about the real time performance. Now its running for 10-15 groups and working fine. In my app, there is similar condition like you in which user can chat based on group. I am handling the the group creation on server side using node.js. Here is the code to create group, but it is for my app specific condition. Just pasting here for the reference. Getting homeState and userId from front-end. Creating group based on the homeState. This code is only for example, it won't work for you. To improve performance you can use clustering.
this.ConnectionObject = function(homeState, userId, ws) {
this.homeState = homeState;
this.userId = userId;
this.wsConnection = ws;
},
this.createConnectionEntry = function(homeState, userId,
ws) {
var connObject = new ws.thisRefer.ConnectionObject(homeState, userId,
ws);
var connectionEntryList = null;
if (ws.thisRefer.connectionMap[homeState] != undefined) {
connectionEntryList = ws.thisRefer.connectionMap[homeState];
} else {
connectionEntryList = new Array();
}
connectionEntryList.push(connObject);
console.log(connectionEntryList.length);
ws.thisRefer.connectionMap[homeState] = connectionEntryList;
ws.thisRefer.connecteduserIdMap[userId] = "";
}
Browsers implement a restriction on the numbers of websocket that can be opened by the same tab. You can't rely on being able to create as many connection as possible. Go for solution #2