I've found references in a few places to some internal logging capabilities of ZMQ. The functionality that I think might exist is the ability to connect to either or both of a inproc or ipc SUB socket and listen to messages that give information about the internal state of ZMQ. This would be quite useful when debugging a distributed application. For instance, if messages are missing/being dropped, it might shed some light on why they're being dropped.
The most obvious mention of this is here: http://lists.zeromq.org/pipermail/zeromq-dev/2010-September/005724.html, but it's also referred to here: http://lists.zeromq.org/pipermail/zeromq-dev/2011-April/010830.html. However, I haven't found any documentation of this feature.
Is some sort of logging functionality truly available? If so, how is it used?
Some grepping through the git history eventually answered my question. The short answer is that a way for ZMQ to transmit logging messages to the outside world was implemented, but it was never used to actually send logging messages by the rest of the code base. After a while it was removed since nothing used it.
The commit that originally added it making use of an inproc socket:
https://github.com/zeromq/libzmq/commit/ce0972dca3982538fd123b61fbae3928fad6d1e7
The commit that added a new "sys" socket type specifically to support the logging:
https://github.com/zeromq/libzmq/commit/651c1adc80ddc724877f2ebedf07d18e21e363f6
JIRA issue, pull request, and commit to remove the functionality:
https://zeromq.jira.com/browse/LIBZMQ-336
https://github.com/zeromq/libzmq/pull/277
https://github.com/zeromq/libzmq/commit/5973da486696aca389dab0f558c5ef514470bcd2
Related
Is there a relatively simple, documented way in a Phoenix application to read how many active sockets and channels are currently open at any given time? And more specifically, is it possible to filter this data by topic and other channel connection metadata?
My use case is for analytics on active connections to my backend.
Thanks for any suggestions!
You are looking for Phoenix.Presence. From the documentation:
Provides Presence tracking to processes and channels.
This behaviour provides presence features such as fetching presences for a given topic, as well as handling diffs of join and leave events as they occur in real-time. Using this module defines a supervisor and allows the calling module to implement the Phoenix.Tracker behaviour which starts a tracker process to handle presence information.
Basically, you are supposed to implement Phoenix.Presence behaviour (the almost ready-to-go example is there in docs,) and Phoenix.Tracker according to your needs.
I am building a new application that receives data from a number of external devices and needs to make it available to a number of different components. ZeroMQ seems purpose-built for the "data bus" aspect of my architecture.
I recently became aware that zmq STREAM sockets can connect to native TCP sockets and send/received messages. Using zmq throughout has a lot of appeal, but I have one problem that I don't know how to get around.
One of my devices needs to be set up. That is, I connect a socket to it, send it some configuration information, then sit back and wait for it to send me data. The device also has a "reset" capability (useful in some contexts), that requires re-sending the configuration information. Doing this depends upon having visibility to the setup/tear-down stage of the socket interface. I need to know when a new connection is established, so I can send the necessary configuration messages.
It seems that zmq is purposely designed to shield me from that knowledge. Is there a way to do what I want? Or should I just use regular sockets for this interface?
Well, it turns out that reading (the right version of) the fine manual can be instructive.
When a connection is made, a zero-length message will be received by the application. Similarly, when the peer disconnects (or the connection is lost), a zero-length message will be received by the application.
I guess all that remains is to disambiguate between connect and disconnect. Still looking for advice from the community, if others have dealt with this situation before.
Following up on your own answer, I would hesitate to rely on that zero length connect/disconnect message as your whole strategy - that seems needlessly fragile. It's not clear to me from your question which end is persistent and which end needs configuration information, but I expect that one end knows it's resetting and reconnecting, and that end needs configuration information from the peer, so it should ask for it with a message when it needs it, to which the peer responds with the requested information.
If the peer does not yet have the required configuration information before it receives some other message, it could either queue up that work or it could respond back with the need for the config, and then have the rest of the network handle that need appropriately.
You shouldn't need stream/tcp sockets to make that work, it should work with more standard ZMQ socket types, you just need to build the robustness into your application rather than trying to get it for free from TCP/socket actions.
If I've missed your point, and what I'm suggesting won't work for some reason, you will have to give more specific information about your network topology for anyone else to understand what a suitable solution might be.
Recently I attended an interview, he asked this question
I am putting messages in Q. Manager, but client unable to get that messages, what is the problem can you explain it?
(All permission are ok, and put and get are enable state).
There are a 101 possible reasons. That is why MQ provides an MQRC back to the application, and further information in the AMQERR01.LOG. Without either of those you cannot even begin to guess. (P.S. I suspect that would have been a suitable reply in an interview!!)
But, since you ask for us to guess, here's a few more different from those Valerie suggested.
Perhaps the client channel max message length is shorter than the messages on the queue.
The codepage between client and queue manager may be such that data cannot be converted.
Client application get buffer isn't big enough
Hasn't specified accept truncated and the message was bigger than the buffer
AMS is in use and he's not the intended recipient (different from permissions)
This is a very broad question, would need to check error code received by client. Could be programming situation where client is getting based on a specific message or correl ID that does not exist. Could be that channel auth is blocking client. Also, it could be that the putting application did not commit the messages so they are not really available for the get.
I am relatively new to JMS and I have been reading a lot on it lately.
I am planning to design a web app which would do the following:
User logs into the system and publishes a message/question to a topic.
All the users who have subscribed to the topic read the message/question and reply to it.
The originator reviews all the answers and picks the best answer.
The originator now replies to only the user whose answer he/she picked and asks for further clarification.
The responder gets the message and replies.
So, once the originator has picked the answer, the JMS now becomes a request/reply design.
My questions are:
Is it possible to publish to a topic with setJmsReplyTo(tempQueue)?
Can request/reply approach be async?
Is it a good idea to have per user queue?
These questions might some dumb to some of the experts here, but please bare in mind that I am still learning.
Thanks.
Is it possible to publish to a topic with setJmsReplyTo(tempQueue)?
You should be able but I'm not 100% sure about it. By the way, I searched in my bookmarks and found this link that should explain what you have to do to build up a Request/Response system using JMS
http://activemq.apache.org/how-should-i-implement-request-response-with-jms.html
Can request/reply approach be async?
A message listener is an object that acts as an asynchronous event handler for messages. So you approach about request/reply, if using JMS, is by default async.
http://docs.oracle.com/javaee/1.3/jms/tutorial/1_3_1-fcs/doc/prog_model.html#1023398
Is it a good idea to have per user queue?
I don't know how many user you expect to have but having one queue for each user is not a good way to handle the messages. I had a problem similar to yours but we used a single queue for each of the macro area and we structured the message to hold the information of the user that sent it in order to store the information later and use it to further analysis.
JMSReplyTo is just a message header, nothing else. So It is possible to publish a message withing a topic with specific value in this header.
Sure! If you would like to create a scalable system you should design event driven system using async instead of blocking aproach. MessageListener can help you.
It is specific to JMS broker implementation. If queue creation is quite cheap there is no problems with such a solution.
Messaging middleware solutions (JMS, Tibco, etc.) allow publish/subscribe with "topic" filtering using wildcards to subscribe to all messages of a certain "topic", e.g. SUBSCRIBE("ACCOUNT.*") topic allows you to subscribe to both "ACCOUNT.WITHDRAW" message and "ACCOUNT.CHECKBALANCE" message.
The problem is that such subscription also receives my own published messages.
I'm looking for a mechanism, similar to, say, UDP multicast loopback which can be turned ON or OFF by the transport layer without messing with the data being sent.
Is there a common, declarative (no custom code, configuration only) way to configure the middleware not to receive messages which that very same service instance has published? Ideally, this should also be able to filter out everything published by ALL servers (nodes) of the same "kind".
Thanks in advance.
The JMS API contains this option for TopicSubscribers, e.g. TIBCO EMS let's you create a consumer with the "noLocal" property. That means no messages published over the same connection, get consumed by clients on the same connection.
e.g. take a look here how to create a topic subscriber with the "noLocal" option:
https://docs.tibco.com/pub/enterprise_message_service/7.0.1-march-2013/doc/html/tib_ems_api_reference/api/javadoc/javax/jms/TopicSession.html
No one is answering, so I'll chime in (in a hand-wavey way).
I believe there's nothing in the JMS spec around controlling whether you get your own sent messages back on a topic receiver. So any capability like this would be a non-portable vendor feature. Especially for your second requirement (based on "kind" of JMS client versus some control based on the same connection doing the sending/receiving).
If you've got no flexibility to modify code or message content (properties), I think you've got no portable solutions. And likely no solution at all for that second "kind" requirement.
If you want to investigate vendor-specific options, you'll need to tell us which vendor you're interested in. You may get nothing, but there's no way to know without asking.