I am working in the cool but very constrained environment of Tropo (cloud telephony) Ruby scripting. The entire app is a single JRuby file. No gems, no requires.
I need to send simple messages to a single SQS queue. I don't need to do any other SQS operations. Before I start pulling code out of existing gems to do this, I wanted to see if anyone has standalone code for sending SQS messages or code that does the HTTP request signing that SQS requires.
We ended up going with the following super-simple code to post a message to an SQS queue that already exists. No gems required.
super_simple_sqs.rb
Related
I'm looking for a method to programmatically identify the triggers associated with an SQS queue. Looking through the SQS sdk docs, it doesn't seem this is possible. I had thought instead to try from the other end, and it appears the Lambda ListEventSourceMappings function would likely do what I want, since I'm able to provide it with the queue ARN. However, this requires the ListSourceMappings permission on all lambdas (*), which isn't really ideal - though it shouldn't really hurt, just not what I want. Is there another mechanism for this that I'm missing, or another approach?
Lambda polls SQS queues. It doesn't appear that way in the console, because they hide some of the details from you, but behind the scenes there is a process running within the AWS Lambda system that is polling your SQS queue and invoking your Lambda function when a message is available.
SQS doesn't push messages to Lambda (or anywhere else). SQS just holds messages and hands them out to anything that asks for them. So from an SQS perspective, there is no knowledge of who the message consumers are.
Given the above, the only way to find what you want is to use the Lambda ListEventSourceMappings API.
Is there a way to mock s3 bucket, sqs queues, and sns topics in the Ruby aws-sdk (similar to Moto in the Python sdk) to where assertions can be made on those uploaded objects and sent messages after the application code is run?
I know I am able to stub the response of the aws sdk so that no real requests are sent using the following code:
Aws.config[:stub_responses] = true
But that doesn't get me the ability to make assertions on uploaded s3 objects and sent sqs/sns messages.
All of the unit tests in the following folder use stubs for Amazon S3 for Ruby. Hope this helps you: https://github.com/awsdocs/aws-doc-sdk-examples/blob/master/ruby/example_code/s3/tests
You can extend this pattern for Amazon SQS and Amazon SNS as well. I hope to add some myself by the end of this year...
What is the most efficient way to receive messages from a Amazon SQS queue?
I've been using the Peddler Gem to create, register and subscribe to an Amazon SQS queue that captures Amazon Marketplace order changes. All good there, the SQS queue is receiving the messages fine. The next step I'm a bit fuzzy on and need some help before I go down a rabbit hole.
It seems like the SQS queue should just be like a webhook that I can subscribe to, too receive notices. But I'm not seeing that option anywhere.
But then it looks like I can use the Shoryuken Gem or maybe Amazon's own AWS SDK for Ruby to create workers to poll the queue in order to get notified of new messages.
Is the Shoryuken gem the most efficient way to pull messages from SQS? Or is there a better way?
IMO Shoryuken is currently the most efficient way for polling SQS messages in Ruby.
You can go ahead and use only the aws-sdk, that would work - with certain limitations. If you go on that path, you will ended implementing a lot of stuff around the aws-sdk, which Shoryuken already does. With the sdk you can receive messages in a loop, call a Ruby class to consume them etc. Shoryuken is a process for polling messages, which uses multithread for performance wise. Besides that, a single process can receive messages from multiple queues.
It seems like the SQS queue should just be like a webhook that I can subscribe to, too receive notices. But I'm not seeing that option anywhere.
That is not SQS, the service that's like that is AWS SNS. If Amazon Marketplace can also integrate with SNS, you can implement a pub/sub calling webhooks.
PS: Shoryuken author here :)
Scenario:
In a web-application some parts are realized in PHP and some other in node.js. Communication between PHP and node.js should be realized via an asynchronous queue/worker system.
In the PHP part of the application API requests should be queued. In the node.js part queued API requests should be processed (worker). Results should be saved back to the queue. Later the results should be retrieved using PHP. The queue should support retry strategies and support notification (to the client) on completed requests.
Question:
I do not want to realize the queue on my own. The work queue itself should not run in PHP, because i do not want long running PHP processes.
I found the work queues
beanstalkd
resque
celery
rabbitmq
Are they suitable for this scenario? Resque looks great. However can a PHP client work together with a Ruby queue? Has anybody experience with something similar? Can worker write back results to the working queue? Can clients be notified on results?
after doing a lot of research i am using rabbitmq.
there are "official" client libraries for multiple plattforms out there. thus subsystems running on different plattforms can work together quite simple.
there are php forks of resque out there. but i do like it the rabbitmq way. one message broker, good documentation, "official" client libraries.
So our team has recently implemented torquebox into our jruby on rails applications. The purpose of this was to be able to receive queue/topic messages from an outside source which is streaming live data.
We have setup our queues/topics and they are receiving the messages without an issue. The next step we want to take is to get these messages on the browser.
So we started to look into leveraging the power of stomp. But we have come across some issues with this. It seems from the documentation that the purpose of using stomp + websockets is to receive messages from the client-side and push those messages to other clients. But we want to receive messages on our queues, and then push these messages to the client-side using websockets. Is this possible? Or would we have to implement a different technology such as Pusher or socket.io to get the queue/topic messages to the browser?
Thanks.
I think stomplets is good solution for this task. In rails application you should use ruby base stomp client, in browser javascript base stomp client. In rails just send data, and in browser just receive.
More detail how do it you can find in torquebox documentation
http://torquebox.org/documentation/2.0.0/stomp.html
It is indeed possible to push messages straight from the server to clients. It took me quite a bit of digging to find it as it is not listed in the documentation directly. Their blog lists it in their example of how to build a chat client using websockets.
http://torquebox.org/news/2011/08/23/stomp-chat-demo-part3/
Basically you use the inject method to choose which channel you're publishing to, and then use the publish method on the returned object to actually send the message. This code excerpt from the article should get you pointed in the right direction.
inject( '/topics/chat' ).publish( message,
:properties=>{
:recipient=>username,
:sender=>'system'
} )
It looks like :properties is the same thing as message headers. I'll be giving this a go over the next couple of days to see how well this works in Rails.