Here is the following scenario. I have a Event Producer which publishes events. I referred the mircosoft document https://learn.microsoft.com/en-us/azure/event-hubs/event-hubs-java-get-started-send
According to my usecase, I have a bean which would create the eventhubProducerClient connection at start of my application. However , the producer.close() in my send method (from above documentation) is called after each event is sent. So, this leads to close of my producer and when I would like to send the next event, there is already an exception that the producer is terminated.
What is the best way to handle the producer.close (). can I leave the producer open ? Wouldnt that cause a memory leak ? Is there a strategy on how I can handle this ?
any lead would be helpful
thank you
Each of the Event Hubs client types is safe to cache and use as a singleton for the lifetime of the application, which is best practice when events are being published or read regularly. The clients are responsible for efficient management of network, CPU, and memory use, working to keep usage low during periods of inactivity. Calling close on a client is required to ensure that network resources and other unmanaged objects are properly cleaned up.
Related
What are the best practices regarding sessions in an application that is designed to fetch messages from a MQ server every 5 seconds?
Should I keep one session open for the whole time (could be weeks or longer), or better open a session, fetch the messages, and then close the session again?
I am using the .net IBM XMS v8 client library.
Adding to what #Attila Repasi's response, I would go for a consumer with message listener attached. The message listener would get called whenever a message needs to be delivered to application. This avoids application explicitly calling receive() to retrieve messages from queue and waste CPU cycles if there are no messages on the queue.
Check the XMS.NET best practices
Keep the connection and session open for a longer period if your application sends or receive message continuously. Creation of connection or session is a time consuming operation and consumes lot of resources and involves network flow (for client connections).
I'm not sure what you are calling a session, but typically applications connect to the queue manager serving them once at start, and keep that connection up while running.
I don't see a reason to disconnect just to reconnect 5 seconds later.
As for keeping the queues open, it depends on your environment.
If there are no special circumstances, I would keep the queue open.
I think the most worth thinking about is how you issue the GETs to read the messages.
I'm designing a quite complicated system and was wondering what the best way is to put a jms consumer (activemq, vm protocol, non persitent) inside a netty handler.
Let me explain, i have several clients connecting to my netty server using websockets. For every client connection i create a jms consumer that listens for interesting messages on one or more topics. If a interesting message arrives i need to do a extra step (additional filtering) before sending the message to the client using the websocket.
Is the following a good way to do this:
inside a SimpleChannelInboundHandler i declare a private non static consumer
the consumer is initialized in channelActive
the consumer is destroyed in channelInactive
when a message is received by consumer i do the extra filter a send it using ctx.channel().write()
In this setup i'm a bit worried that the consumer might turn into slow consumer and slow everything down, cause the websocket goes over the internet.
I came up with a more complex one to decouple the "receiving of message by consumer" and "sending of message through a websocket".
inside a SimpleChannelInboundHandler i declare a private non static consumer
the consumer is initialized in channelActive
the consumer is destroyed in channelInactive
when a message is received by consumer i put it in a blockedqueue
every minute i let a thread (created for every client) look in the queue and send the found messages to the client using ctx.channel().write().
At this point i'm a bit worried about the extra thread per client.
Or is there maybe a better way to accomplish this task?
This is a classic slow consumer problem and the first step to resolving it is to determine what the appropriate action is when a slow consumer is detected. If it is acceptable that the slow consumer misses messages then the solution is some variation on dropping messages or unsubscribing them from the feed. For example, if it's acceptable that the client misses messages then, when one is received from JMS, check if the channel is writable. If it isn't, drop the message. If you want to give yourself a bit more of a buffer (although OS buffers are quite large) you can track the number of write completion future's that haven't completed (ie the messages haven't been written to the OS send buffer) and drop messages if there are too many outstanding write requests.
If the client may not miss messages, and is consistently slow, then the problem is more difficult. One option might be to divert messages to a JMS queue with a specific header value, then open a new consumer that reads messages from that queue using a JMS selector. This will put more load on the JMS server but might be appropriate for temporary slowness and hopefully it won't interfere with you main topic feeds. Alternatively you might want to stash the messages in a different store, such as a database, so you can poll for messages when they can be sent. If you do this right a single polling thread can cope with many clients (query for clients which have outstanding messages, then for each client, load a bunch of messages). However this isn't as convenient as using JMS.
I wouldn't go with option 2 because the blocking queue is only going to solve the problem temporarily, and you can achieve the same thing by tracking how many write operations are waiting to complete.
Sorry, if it is a duplicate question.
I have a legacy web application which uses Queues (yes. normal Java Queue) and custom polling (every 500ms). A REST web service (/message) will be called, which will return the message if any otherwise empty string.
My need: If any message is available in Queue, in Real-Time, the client should get the message. So I can save 500ms.
Is there any advantage to moving to JMS from current approach? From this link JMS MessageConsumer's messageListener makes push or pull? it seems, MessageListener (process is asynchronous) uses polling which is no different from current approach.
If it is vendor based, how HornetQ/ActiveMQ supports MessageListener?
EDIT:
The queue is used for integration of two systems. A web app & standlone java program.
Either receive or a MessageListener will be asynchronous and will be called as soon as you receive a message.
you could control the pre-fetch size of your client.
Now, if all you need is to avoid the delay of poling every 500 ms, using a Queue system may be an overkill? It's perfect fine to use java.util.Queue (or any other subclass).
If all you need is to block until an element of a java.util.Queue is available, and you don't need distributed messaging, persistence or anything like you could simply using BlockingDequeue and your thread would unblock as soon as you have a message..
Look at this:
http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/BlockingDeque.html
The Async MessageListener is implemented using a push based model. In ActiveMQ the broker sends a number of messages to the client based in it's set prefetch value so that messages are ready for consumption. Whether or not this helps with your particular use case is a question you need to answer for yourself.
I'm building a simple message delegation application. Messages are being send on both ends via JMS. I'm using a MDB to process incoming messages, transform them and send them to a target queue. Unfortunately the same messages can be send to the incoming queue more than once but it is not allowed to forward duplicates.
So what is the best way to accomplish that?
Since there can be multiple MDBs listening on the incoming queue a need a single cache where I can store the unique message uuids of the incoming messages for at least an hour. How should this cache be accessed? Via a singleton/ static class (I'm running Java EE 5 and thus don't have the singleton annotation)?
In addition I think all operations must be synchronized, right? Does that harm performance too much?
#Ingo: are you OK with database solution. You can full fledged DB server or simple apache derby solution for this..
If so, you can have a simple table where you can store message unique UId and can check against it for uniqueness....this solution will have following benefits:
Simple code
No need of time bound cache(1 hour). You can check for uniqueness of a message forever.
Persistent record of what messages came in.
No need of expensive synchronized, you can rely on DB isolation level to have consistency.
centralized solution for your possibly many deployments of application.
I'm using a message listener to process some messages from MQ based on Spring's DefaultMessageListenerContainer. After I receive a message, I have to make a Web Service (WS) call. However, I don't want to do this in the onMessage method because it would block the onMessage method until the invocation of WS is successful and this introduces latency in dequeuing of messages from the queue. How can I decouple the invocation of the Web Service by calling it outside of the onMesage method or without impacting the dequeuing of messages?
Thanks,
I think you might actually want to invoke the web service from your onMessage. Why do you want to dequeue messages quickly, then delay further processing? If you do what you're saying, you'd probably have to introduce another level of queueing, or some sort of temporary "holding" collection, which is redundant. The point of the queue is to hold messages, and your message listener will pull them off and process them as quickly as possible.
If you are looking for a way to maximize throughput on the queue, you might think about making it multi-threaded, so that you have multiple threads pulling messages off the queue to invoke the web service. You can easily do this by setting the "concurrentConsumers" configuration on the DefaultMessageListenerContainer. If you set concurrentConsumers to 5, you'll have 5 threads pulling messages off the queue to process. It does get tricky if you have to maintain ordering on the messages, but there may be solutions to that problem if that's the case.
I agree with answer provided before me , however I can see a usecase similar to this very common in practice. I'm adding my two cents It might be valid in some cases that you don't want to do time consuming work in your onMessage Thread (which is pulling message from Q)
We have something similar in one workflow, where if user selects some XYZ option on GUI that means at server we need to connect to another external webservice to get ABCD in this case we do not make call to webservice in onMessage Thread and use ThreadPool to dispatch and handle that call.
If something wrong happens during webservice call we broadcast that to GUI as separate Message , there is concept of request id which is preserved across messages so that GUI can relate error messages. You can use ExecutorService implementation to submit task.
hope it helps.