I need to setup a client which will send sqs to a server:
client side:
...
sqs = AWS::SQS.new
q = sqs.queues.create("q_name")
m = q.send_message("meta")
...
but how the server could read the message of the client?
Thank you in advance.
First you need to have your server connect to SQS then you can get your queue.
Do a get_messages on your queue. Go to boto docs to get more information on the attributes. This will give you 1 to 10 message objects based on your parameters. Then on each of those objects do a get_body() then you'll have the string of the message.
Here's a simple example in python. Sorry don't know ruby.
sqsConn = connect_to_region("us-west-1", # this is the region you created the queue in
aws_access_key_id=AWS_ACCESS_KEY_ID,
aws_secret_access_key=AWS_SECRET_ACCESS_KEY)
QUEUE = sqsConn.get_queue("my-queue") # the name of your queue
msgs = QUEUE.get_messages(num_messages=10, # try and get 10 messages
wait_time_seconds=1, # wait 1 second for these messages
visibility_timeout=10) # keep them visible for 10 seconds
body = msgs[0].get_body() # get the string from the first object
Hope this helps.
Related
I have read all salient posts and the zeromq guide on the topic of pub/sub late joiner, and hopefully I simply missed the answer, but just in case I haven't, I have two questions about the zeromq proxy in context to the late joiner:
Does the zeromq proxy with the suggested XSUB/XPUB sockets also suffer from the late joiner problem, ie. are the first few pub messages of a new publisher dropped ?
If so, what is the accepted solution to ensure that even the first published message is received by subscribers with matching topic (my latest info is to sleep...)
I don't believe it is pertinent to the question, but just in case: here are
My proxy's run method, which runs in its own thread; it starts a capture thread if DEBUG is true to log all messages; if no matching subscription exists, nothing is captured.
The proxy works fine including the capture socket. However, I am now adding functionality where a new publisher is started in a thread and will immediately start to publish messages. Hence my question, if a message is published straight away, will it be dropped ?
def run(self):
msg = debug_log(self, f"{self.name} thread running...")
debug_log(self, msg + "binding sockets...")
xpub = self.zmq_ctx.socket(zmq.XPUB)
xpub.bind(sys_conf.system__zmq_broker_xpub_addr)
xsub = self.zmq_ctx.socket(zmq.XSUB)
xsub.bind(sys_conf.system__zmq_broker_xsub_addr)
if self.loglevel == DEBUG and sys_conf.system__zmq_broker_capt_addr:
debug_log(self, msg + " done, now starting broker with message "
"capture and logging.")
capt = self.zmq_ctx.socket(zmq.PAIR)
capt.bind(sys_conf.system__zmq_broker_capt_addr)
capt_th = Thread(
name=self.name + "_capture_thread", target=self.capture_run,
args=(self.zmq_ctx,), daemon=True
)
capt_th.start()
capt.recv() # wait for peer thread to start and check in
debug_log(self, msg + "capture peer synchronised, proceeding.")
zmq.proxy(xpub, xsub, capt) # blocks until thread terminates
else:
debug_log(self, msg + " starting broker.")
zmq.proxy(xpub, xsub) # blocks until thread terminates
def capture_run(self, ctx):
""" capture socket's thread's run method.
debug_log(self, f"{self.name} capture thread running.")
sock = ctx.socket(zmq.PAIR)
sock.connect(sys_conf.system__zmq_broker_capt_addr)
sock.send(b"") # ack message to calling thread
while True:
try: # assume we're getting topic string followed by python object
topic = sock.recv_string()
obj = sock.recv_pyobj()
except Exception: # if not simply log message in bytes
topic = None
obj = sock.recv()
debug_log(self, f"{self.name} capture_run received topic {topic}, "
f"obj {obj}.")
My users of the proxy (they are all both subscribers and publishers) in different threads and/or processes from the proxy:
...
# establish zmq subscriber socket and connect to broker
self._evm_subsock = self._zmq_ctx.socket(zmq.SUB)
self.subscribe_topics()
self._evm_subsock.connect(sys_conf.system__zmq_broker_xpub_addr)
# establish pub socket and connect to broker
self._evm_pub_sock = self._zmq_ctx.socket(zmq.PUB)
self._evm_pub_sock.connect(sys_conf.system__zmq_broker_xsub_addr)
debug_log(self, msg + " Connected to pub/sub broker.")
...
Regarding: “Sending response back to the out/reply queue.”
There is a requirement to send the response back to a different queue (reply queue).
While sending the response, we have to use the correlation and message id from the request message and pass it to the reply queue as header. I suspect the format of correlation/message id is wrong.
While reading the message, the correlation id and message id format are as below:
MessageId = “ID:616365323063633033343361313165646139306638346264”
CorrelationId = “ID:36626161303030305f322020202020202020202020202020”
While sending the back to out/reply queue, we are passing these ids as below:
ITextMessage txtReplyMessage = sessionOut.CreateTextMessage();
txtReplyMessage.JMSMessageID = “616365323063633033343361313165646139306638346264”;
txtReplyMessage.JMSCorrelationID = “36626161303030305f322020202020202020202020202020”;
txtReplyMessage.Text = sentMessage.Contents;
txtReplyMessage.JMSDeliveryMode = DeliveryMode.NonPersistent;
txtReplyMessage.JMSPriority = sentMessage.Priority;
messagePoducerOut.Send(txtReplyMessage);
Please note:
With the XMS.NET library, we need to pass the correlation and message id in string format as per shown above
With MQ API’s (which we were using earlier) passing the correlation and message ids we use to send in bytes format like below:
MQMessage queueMessage = new MQMessage();
string[] parms = document.name.Split('-');
queueMessage.MessageId = StringToByte(parms[1]);
queueMessage.CorrelationId = StringToByte(parms[2]);
queueMessage.CharacterSet = 1208;
queueMessage.Encoding = MQC.MQENC_NATIVE;
queueMessage.Persistence = 0; // Do not persist the replay message.
queueMessage.Format = "MQSTR ";
queueMessage.WriteString(document.contents);
queueOut.Put(queueMessage);
queueManagerOut.Commit();
Please help to troubleshoot the problem.
Troubleshooting is a bit difficult because you haven’t clearly specified the trouble (is there an exception, or is the message just not be correlated successfully?).
In your code you have missed to add the “ID:” prefix. However, to address the requirements, you should not need to bother too much about what is in this field, because you simply need to copy one value to the other:
txtReplyMessage.JMSCorrelationID = txtRequestMessage.JMSMessageID
A bit unclear what the issue is. Are you able to run the provided examples in the MQ tools/examples? This approach uses tmp queues(AMQ.*) as JMSReplyTo
Start the "server" application first.
Request/Response Client: "SimpleRequestor"
Request/Response Server: "SimpleRequestorServer"
You can find the exmaples at the default install location(win):
"C:\Program Files\IBM\MQ\tools\dotnet\samples\cs\xms\simple\wmq"
The "SimpleMessageSelector" will show how to use the selector pattern.
Note the format on the selector: "JMSCorrelationID = '00010203040506070809'"
IBM MQ SELECTOR
I have a simple scenario of publish a message to a SNS topic, which is subscribed by a SQS queue, but somehow the queue never receives the messages (i.e., not show up in the SQS console). Here are the codes in Ruby:
sns = Aws::SNS::Client.new;
sqs = Aws::SQS::Client.new;
q1 = sqs.create_queue({queue_name: "queue1”});
t1 = sns.create_topic({name: "topic1"});
q1_attr = sqs.get_queue_attributes({queue_url: q1.queue_url,attribute_names: ["All"]});
s1 = sns.subscribe({topic_arn: t1.topic_arn, protocol: "sqs", endpoint: q1_attr.attributes['QueueArn']});
resp = sns.publish({topic_arn: t1.topic_arn, message: "Test message"});
Is there anything missing?
The second quote in this line:
q1 = sqs.create_queue({queue_name: "queue1”});
is a 'fancy' quote ” instead of ". Change it to this:
q1 = sqs.create_queue({queue_name: "queue1"});
I'm trying to read messages from a queue using Bunny. I only have read permissions on the RabbitMQ server but it seems the code I'm using tries to create the queue - though I can see the queue already exists with queue_exists?().
There must be a process in Bunny whereby one can simply read messages off an existing queue? Here's the code I'm using
require 'bunny'
class ExampleConsumer < Bunny::Consumer
def cancelled?
#cancelled
end
def handle_cancellation(_)
#cancelled = true
end
end
conn = Bunny.new("amqp://xxx:xxx#xxx", automatic_recovery: false)
conn.start
ch = conn.channel
q = ch.queue("a_queue")
consumer = ExampleConsumer.new(ch, q)
When I execute the above I receive:
/Users/jamessmith/.rvm/gems/ruby-1.9.3-p392/gems/bunny-1.7.1/lib/bunny/channel.rb:1915:in `raise_if_continuation_resulted_in_a_channel_error!': ACCESS_REFUSED - access to queue 'a_queue' in vhost '/' refused for user 'xxx' (Bunny::AccessRefused)
in most RMQ configurations I've seen, the consumer will have permissions to create the queue that they need.
If you must have your permissions set up so that you can't create the queue from your consumer, I'd suggest opening an issue ticket with the Bunny gem. it doesn't look like that is supported
I am using IBM Websphere MQ. I have the queue manager and queue name. Now, I want to check whether the queue has any messages in it?
I did not work on this before. Pleas help
Please let me know if you need further information!
Thanks
The below code is .NET / amqmdnet - but you might try and convert this in the meantime until a Java dev sees your post.
To see if there is a message on the queue, without actually taking it off the queue, use MQC.MQOO_BROWSE on the Queue and IBM.WMQ.MQC.MQGMO_BROWSE_FIRST as the option
You'll get MQRC_NO_MSG_AVAILABLE if the queue is empty.
MQMessage queueMessage = new MQMessage();
MQQueueManager queueManager = new MQQueueManager(qmName, channelName, connName);
MQQueuequeue = queueManager.AccessQueue(qName,
MQC.MQOO_BROWSE + MQC.MQOO_FAIL_IF_QUIESCING);
MQGetMessageOptions opt = new MQGetMessageOptions();
opt.Options = IBM.WMQ.MQC.MQGMO_BROWSE_FIRST;
queueMessage.CorrelationId = IBM.WMQ.MQC.MQMI_NONE;
queueMessage.MessageId = IBM.WMQ.MQC.MQMI_NONE;
queue.Get(queueMessage, opt);
String sMessage = queueMessage.ReadString(queueMessage.DataLength);
To peek the next message use IBM.WMQ.MQC.MQGMO_BROWSE_NEXT;
To actually read the message OFF the queue, use MQC.MQOO_INPUT_SHARED on the AccessQueue.
The answer didn't show how to check for MQRC_NO_MSG_AVAILABLE. Here is my solution. If there are better ones please let me know.
try
{
queue.Get(queueMessage, opt);
String sMessage = queueMessage.ReadString(queueMessage.DataLength);
}
catch (MQException err)
{
if (err.ReasonCode.CompareTo(MQC.MQRC_NO_MSG_AVAILABLE) == 0)
return true;
}
For Windows machine
It depends on where your queue manager is.
You could use MQUtilities - ih03 pack - which has rfhUtil.exe (Local Qm) and rfhUtilC.exe (for remote qm)
For Local QM , it is straight forward you need to place appropriate values and hit browse, it will show you Queue Depth.
For Remote QM, Place /TCP/(PortNo) for queue manager name and queue for queue name. Hit browse and you will get to know the queue depth.
For Unix/Ubuntu/Linux versions - There is a product called MQVisualEdit which is similar to this one.