In my rails app, I need to make a http request to 3rd party service, since http request is synchronous, sometimes it takes more than 20 seconds to get response back from them.
I just push some data to that service, I don't care about what the response will be, so I want to make the request asynchronous, so my code will continue execution and not been blocked.
How could I do it in ruby?
I need to make a http request...so my code will continue execution and not been blocked.
def some_action
Thread.new do
uri = URI('http://localhost:4567/do_stuff')
Net::HTTP.post_form(uri, 'x' => '1', 'y' => '2')
end
puts "****Execution continues here--before the response is received."
end
Here's a sinatra app you can use to test it:
1) $ gem install sinatra
2)
#my_sinatra_app.rb
require 'sinatra'
post '/do_stuff' do
puts "received: #{params}" #Check the sinatra server window for the output.
sleep 20 #Do some time consuming task.
puts "Sending response..."
"The result was: 30" #The response(which the rails app ignores).
end
output:
$ ruby my_sinatra_app.rb
== Sinatra (v1.4.6) has taken the stage on 4567 for development with backup from Thin
Thin web server (v1.6.4 codename Gob Bluth)
Maximum connections set to 1024
Listening on localhost:4567, CTRL+C to stop
received: {"x"=>"1", "y"=>"2"}
<20 second delay>
Sending response...
127.0.0.1 - - [11/Nov/2015:12:54:53 -0400] "POST /do_stuff HTTP/1.1" 200 18 20.0032
When you navigate to some_action() in your rails app, the rails server window will immediately output ****Execution continues here--before the response is received., and the sinatra server window will immediately output the params hash, which contains the data sent in the post request. Then after a 20 second delay the sinatra server window will output Sending response....
You need to use Event Machine and Fibers in ruby.
Github: em-http-request
Related
I'm pretty new to Ruby and Sinatra and I'm trying to set up a basic sinatra server to listen for HTTP post requests, and then process the data.
I need to send the response within 5 seconds or the server (shopify) which sends the POST thinks that the request has failed and sends it again. To avoid that, Shopify advises to defer processing until after the response has been sent.
I'm not sure how to trigger my processing once Sinatra has sent the response.
Will this work ?
require 'sinatra'
require 'json'
webhook_data = Order.new
post '/' do
request.body.rewind
data = request.body.read
webhook_data.parsed_json = JSON.parse(data)
puts "My response gets sent here, right ?"
end
after do
#DO MY PROCESSING HERE
end
Is there any better way to do this ?
You can use any solution for background jobs processing. Here is example for Sidekiq usage with Sinatra.
You can try to use Threads as well:
set :threaded, true
post '/' do
request.body.rewind
data = request.body.read
Thread.new do
# data processing staff goes here
end
# here goes response
end
I am working on an assignment where I have to develop a web server in Ruby using the socket library. I was able to get a simple web server up and running as seen in this thread here .
I am currently working on getting and storing the body of an HTTP request into a variable in my web server. The problem I am running into is trying to define a while loop that gets the entire body of a HTTP request.
I am attempting to get the body of a HTTP request by using the gets method. I could not find any documentation on this method (I saw it being used here)
and was wondering if there were more documentation online.
In my first post here, someone suggested that I use the Content-Length header to determine the size of the body and how much data to read from the socket. I don't really understand how I would go about implementing this because I am unsure how the gets method functions.
Since this is for an assignment, I don't think posting code would be a good idea. I am looking for more information on the gets method and any tips to point me towards the right direction.
You shouldn't be using gets. gets tries to read complete lines (ie it reads up to a line separator), but there is no guarantee that an http request body ends with a line separator.
Instead you should be using read - this allows you to read an arbitrary amount of data (as you mentioned you can use the content length header to know how much to read)
Your ultimate problem isn't related to gets, or even really anything in your code. But before we get to that, let's answer this question & explore sockets a little bit.
If you follow the chain up, you find that Ruby's TCPSocket class inherits from its IO class. It's IO that provides gets. gets will read, line-by-line, until there's nothing more to read. Let's create a simple client that connects to a port, spits out 4 lines of poetry, and then quits:
# poetry_sender.rb
require 'socket'
poem = ["'God save thee, ancient Mariner!",
"From the fiends, that plague thee thus!—",
"Why look'st thou so?'—With my cross-bow",
"I shot the ALBATROSS."]
puts "Client establishing connection..."
s = TCPSocket.new 'localhost', 2000
puts "Client sending poetry..."
poem.each { |line| s.puts line } # Print each line out on the socket
s.close # Close our socket
puts "All done."
And a simple server, that displays what the client sends us:
# poetry_receiver.rb
require 'socket'
server = TCPServer.new 2000 # Server bind to port 2000
loop do
puts "Server now awaiting some poetry..."
socket = server.accept # Wait for a client to connect
while line = socket.gets
puts "A client sent us this beautiful line: #{line}"
end
puts "They had nothing more to say; let's disconnect them."
socket.close
end
If you run the server (poetry_receiver.rb) first, and then the client, you'll see some output like this:
Server now awaiting a connection...
A client sent us this beautiful line: 'God save thee, ancient Mariner!
A client sent us this beautiful line: From the fiends, that plague thee thus!—
A client sent us this beautiful line: Why look'st thou so?'—With my cross-bow
A client sent us this beautiful line: I shot the ALBATROSS.
They had nothing more to say; let's disconnect them.
Server now awaiting a connection...
The last two lines are the important ones; they indicate that socket.gets returned nil and we exited the while loop.
So, how can we modify our poetry_sender.rb so the server doesn't detect the end of the poem? You might think it's got something to do with blank lines, but if you set poem = [] or poem = ["", "", ""] then you'll find that it still gets disconnected OK. But what if we added a delay before closing the socket in poetry_sender.rb?
sleep 60
s.close # Close our socket
puts "All done."
Now you'll see a big delay in the server output. The TCP server doesn't break out of its while loop until the TCP client closes its socket.
Now we can turn to your broader problem: you're trying to implement a simple HTTP server, but your server is getting hung up in a while loop when you try to connect via your web browser. It's because your web browser is keeping that socket open; but it has to, otherwise it has no way to send you back a response. So, how do we know when a client has finished sending us a response? The HTTP 1.1 spec says:
A client sends an HTTP request to a server in the form of a request message... followed by header fields... an empty line to indicate the end of the header section, and finally a message body containing the payload body (if any).
Let's not worry about the message body; how could we write a while loop that terminates if it has no more impact, or if it receives a blank line? Here's one way, in a simple HTTP server that just sends back "Hello world" no matter what request it receives:
require 'socket'
server = TCPServer.new('localhost', 2345)
http_request = [] # We'll store the lines of our incoming request here.
loop do
socket = server.accept
while (line = socket.gets) && line.chomp != '' # While the client is connected, and hasn't sent us a blank line yet...
http_request << line
end
# Send response headers
socket.print "HTTP/1.1 200 OK\r\n" +
"Content-Type: text/plain\r\n" +
"Connection: close\r\n" +
"\r\n"
# Send response body
socket.print "Hello world!"
socket.close
end
Quite late to the party, but I'm currently implementing my own rack app server (for fun).
Here you can see how I do it: https://github.com/tak1n/reifier/blob/master/lib/reifier/request.rb
The first line of a HTTP request is always the request line, which is basically something like GET /test HTTP/1.1
After the request line until \r\n you get the headers.
After that you are able to read the body (if PUT / POST request) with just using the CONTENT_LENGTH you parsed from the headers.
I have working code that gets data over https (below). In fact it runs some test through php. I used standard timeout that works fine. Now while "waiting" for server response I need to implement timer. Because in some cases the test won't finish - the php code will not finish - the ruby time out works ok. So I need to kill some process to capture the error in the existing https session.
How can I implement my own time out for https request on top of existing time out?
The existing timeout will be always greater than custom timeout. eg existing timeout is 10mins and the custom will be 5 mins.
uri = URI.parse(url)
start = Time.new
http_read_timeout=60*10
connection = Net::HTTP.new(uri.host, 443)
connection.use_ssl = true
begin
response = connection.start() do |http|
http.open_timeout = 50
http.read_timeout = http_read_timeout
http.request_get(uri.request_uri)
# here I need to place a code that is triggered
# in case of custom timeout is reached
end
rescue Timeout::Error
# "Connection failed
time_out_message ="security time out - after #{http_read_timeout} sec"
return time_out_message
end
puts "finished"
I don't get it. What does your custom timeout do? You are making an HTTP request...it either returns or times out.
You're already setting the timeout value. Your code can't reach into the future & tell you what the external code would eventually return, if it did...so what do you want it to do, exactly?
But if you really just need an external Timeout wrapper, you can use Timeout::Timeout. Like this:
require 'timeout'
Timeout::timeout(your_timeout_period) do
run_some_code
rescue => err
do_something_with err
# and maybe the below?
raise
end
I try to read URLs from a Redis store and simply fetch the HTTP status of the URLs. All within EventMachine. I don't know what's wrong with my code, but it's not asynchronous like expected.
All requests are fired from the first one to the last one and curiously I only get the first response (the HTTP header I want to check) after the last request. Does anyone have a hint what's going wrong there?
require 'eventmachine'
require 'em-hiredis'
require 'em-http'
EM.run do
#redis = EM::Hiredis.connect
#redis.errback do |code|
puts "Error code: #{code}"
end
#redis.keys("domain:*") do |domains|
domains.each do |domain|
if domain
http = EM::HttpRequest.new("http://www.#{domain}", :connect_timeout => 1).get
http.callback do
puts http.response_header.http_status
end
else
EM.stop
end
end
end
end
I'm running this script for a few thousand domains so I would expect to get the first responses before sending the last request.
While EventMachine is async, the reactor itself is single threaded. So, while your loop is running and firing off those thousands of requests, none of them are being executed until the loop exits. Then, if you call EM.stop, you'll stop the reactor before they execute.
You can use something like EM::iterator to break up the processing of domains into chunks that let the reactor execute. Then you'll need to do some magic if you really want to EM.stop by keeping a counter of the dispatched requests and the received responses before you stop the reactor.
I have a very basic TCP server implemented in Ruby. In general it does what it's supposed to, but every once in a while I get "The connection to the server was reset while the page was loading" error. I have a feeling that it has something to do with close terminating the connection too soon. If so, how do I wait for all the data to be sent? Or is it something else?
require 'socket'
server = TCPServer.new('', 80)
loop do
session = server.accept
begin
session.print Time.now
ensure
session.close
end
end
I'm not an expert in this area, but here is what I believe is happening....
The browser sends a GET request with the header field "Connection: keep-alive". So the browser is expecting to keep the connection alive at least until it receives a complete chunk of the response. Under this protocol, the server response must include a header specifying the length of the response, so that the browser knows when it has received the complete response. After this point, the connection can be closed without the browser caring.
The original example closes the connection too quickly, before the browser can validate that a complete response was received. Curiously, if I run that example and refresh my browser several times, it will load about every 1 in 10 tries. Maybe this erratic behavior is due to the browser occasionally executing fast enough to beat my server closing the connection.
Below is a code example that executes consistently in my browser:
require 'socket'
response = %{HTTP/1.1 200 OK
Content-Type: text;charset=utf-8
Content-Length: 12
Hello World!
}
server = TCPServer.open(80)
loop do
client = server.accept
client.puts response
sleep 1
client.close
end
I suspect it's because the browser is expecting an HTTP response with headers &c. Curiously, you can make the "reset" error happen every time if you put before the "ensure" a sleep of, say, one second.
How to fix it depends upon what you are after. If this is not to be an HTTP server, then don't use the browser to test it. Instead, use telnet or write a little program. If it is to be an HTTP server, then take a look at webrick, which is built into Ruby MRI >= 1.8. Here's how:
#!/usr/bin/ruby1.8
require 'webrick'
# This class handles time requests
class TimeServer < WEBrick::HTTPServlet::AbstractServlet
def do_GET(request, response)
response.status = 200
response['Content-Type'] = 'text/plain'
response.body = Time.now.to_s
end
end
# Create the server. There are many other options, if you need them.
server = WEBrick::HTTPServer.new(:Port=>8080)
# Whenever a request comes in for the root page, use TimeServer to handle it
server.mount('/', TimeServer)
# Finally, start the server. Does not normally return.
server.start
Also, should note that including Connection: close in the response header doesn't seem to help me at all with this connection reset error in my browser (FFv3.6). I have to include both the content-length header field, and include the sleep method to put some delay in the connection closing in order to get a consistent response in my browser.