Serving files & handling requests with Webrick on same port - ruby

Hi I have a need to be able to receive requests from GitLab (body => JSON) as well as serve files on same port. I am trying to use Webrick for this purpose. I can do these separately.
To serve files I do:
server = WEBrick::HTTPServer.new(:Port => 3030, :DocumentRoot => '/')
server.start
To receive and process jSON I do:
server = WEBrick::HTTPServer.new(:Port => 3030, :DocumentRoot => '/')
server.mount_proc '/' do | req, res |
Queue.new(req.body)
end
But I need this functionality combined, is there a way to do this with Webrick?

Yes, this is certainly possible with Webrick or any HTTP server. There will be two different HTTP actions depending on what the users wants to do, 1.) a GET request to serve the files or 2.) a POST request to process some JSON.
Here's a simple example to show you how to do both:
class Server < WEBrick::HTTPServlet::AbstractServlet
def do_GET (request, response)
puts "this is a get request"
end
def do_POST (request, response)
puts "this is a post request who received #{request.body}"
end
end
server = WEBrick::HTTPServer.new(:Port => 3030)
server.mount "/", Server
trap("INT") {
server.shutdown
}
server.start
Once that is running you can test this by doing the following in a separate terminal window:
curl localhost:3030
output:
this is a get request
localhost - - [23/Apr/2015:06:39:20 EDT] "GET / HTTP/1.1" 200 0
- -> /
To test the POST request:
curl -d "{\"json\":\"payload\"}" localhost:3030
output:
this is a post request who received {"json":"payload"}
localhost - - [23/Apr/2015:06:40:07 EDT] "POST / HTTP/1.1" 200 0
- -> /

Since you mentioned the purpose was a light code-base, here's a light, fast script using the Plezi framework...
This will allow for easier testing, I think (but I'm biased). Also, Plezi is faster on my machine then Webrick (although it's a pure ruby framework, no rack or 'c' extensions involved).
require 'plezi'
class MyController
def index
# parsed JSON is acceible via the params Hash i.e. params[:foo]
# raw JSON request is acceible via request[:body]
# returned response can be set by returning a string...
"The request's params (parsed):\n#{params}\n\nThe raw body:\n#{request[:body]}"
end
end
# start to listen and set the root path for serving files.
listen root: './'
# set a catch-all route so that MyController#index is always called.
route '*', MyController
(if you're running the script from the terminal, remember to exit irb using the exit command - this will activate the web server)

Related

Faraday (Ruby) Timeout Errors

I'm attempting to put a small payload generated in route A (Sinatra app) to Route B using Faraday. So the code basically looks like:
post "/routeA" do
foo.save
foo_id = foo.id
conn = Faraday.new(:url => "http://localhost:3001/routeB" ) do |builder|
builder.request :url_encoded
builder.response :logger
builder.adapter :net_http
end
resp = conn.put do |req|
req.url '/routeB'
req.headers['Content-Type'] = 'application/json'
req.body = {:id => foo_id }.to_json
req.options = {
#:timeout => 5, # see below, these aren't the problem
#:open_timeout => 2
}
end
# never gets here b/c Timeout error always thrown
STDERR.puts resp.body
end
put "/routeB" do
# for test purposes just log output
STDERR.puts request.body.read.to_s.inspect
status 202
body '{"Ok"}'
end
Problem is that it always throws a timeout error (I've run without the timeout options, and with the ones shown above -> same results). However, the logs show the request is going through:
I, [2012-03-24T16:56:13.241329 #17673] INFO -- : put http://localhost:3001/routeB
D, [2012-03-24T16:56:13.241427 #17673] DEBUG -- request: Content-Type: "application/json"
#<Faraday::Error::TimeoutError>
DEBUG - POST (60.7987ms) /routeA - 500 Internal Server Error
"{\"id\":7}"
DEBUG - PUT (0.0117ms) /routeB - 202 Accepted
Not sure how to get past the timeout error? Any insight would be appreciated. Thanks.
The problem is that the application cannot respond to another request until it's finished with the current one. That is, when you make a PUT request on the /routeB, the application got that, and it's waiting for the current request (/routeA) to finish. But the request won't finish because it's waiting to get the response from the /routeB. I think this is what causes the timeout error.

Ruby Net::HTTP - following 301 redirects

My users submit urls (to mixes on mixcloud.com) and my app uses them to perform web requests.
A good url returns a 200 status code:
uri = URI.parse("http://www.mixcloud.com/ErolAlkan/hard-summer-mix/")
request = Net::HTTP.get_response(uri)(
#<Net::HTTPOK 200 OK readbody=true>
But if you forget the trailing slash then our otherwise good url returns a 301:
uri = "http://www.mixcloud.com/ErolAlkan/hard-summer-mix"
#<Net::HTTPMovedPermanently 301 MOVED PERMANENTLY readbody=true>
The same thing happens with 404's:
# bad path returns a 404
"http://www.mixcloud.com/bad/path/"
# bad path minus trailing slash returns a 301
"http://www.mixcloud.com/bad/path"
How can I 'drill down' into the 301 to see if it takes us on to a valid resource or an error page?
Is there a tool that provides a comprehensive overview of the rules that a particular domain might apply to their urls?
301 redirects are fairly common if you do not type the URL exactly as the web server expects it. They happen much more frequently than you'd think, you just don't normally ever notice them while browsing because the browser does all that automatically for you.
Two alternatives come to mind:
1: Use open-uri
open-uri handles redirects automatically. So all you'd need to do is:
require 'open-uri'
...
response = open('http://xyz...').read
If you have trouble redirecting between HTTP and HTTPS, then have a look here for a solution:
Ruby open-uri redirect forbidden
2: Handle redirects with Net::HTTP
def get_response_with_redirect(uri)
r = Net::HTTP.get_response(uri)
if r.code == "301"
r = Net::HTTP.get_response(URI.parse(r['location']))
end
r
end
If you want to be even smarter you could try to add or remove missing backslashes to the URL when you get a 404 response. You could do that by creating a method like get_response_smart which handles this URL fiddling in addition to the redirects.
I can't figure out how to comment on the accepted answer (this question might be closed), but I should note that r.header is now obsolete, so r.header['location'] should be replaced by r['location'] (per https://stackoverflow.com/a/6934503/1084675 )
rest-client follows the redirections for GET and HEAD requests without any additional configuration. It works very nice.
for result codes between 200 and 207, a RestClient::Response will be returned
for result codes 301, 302 or 307, the redirection will be followed if the request is a GET or a HEAD
for result code 303, the redirection will be followed and the request transformed into a GET
example of usage:
require 'rest-client'
RestClient.get 'http://example.com/resource'
The rest-client README also gives an example of following redirects with POST requests:
begin
RestClient.post('http://example.com/redirect', 'body')
rescue RestClient::MovedPermanently,
RestClient::Found,
RestClient::TemporaryRedirect => err
err.response.follow_redirection
end
Here is the code I came up with (derived from different examples) which will bail out if there are too many redirects (note that ensure_success is optional):
require "net/http"
require "uri"
class Net::HTTPResponse
def ensure_success
unless kind_of? Net::HTTPSuccess
warn "Request failed with HTTP #{#code}"
each_header do |h,v|
warn "#{h} => #{v}"
end
abort
end
end
end
def do_request(uri_string)
response = nil
tries = 0
loop do
uri = URI.parse(uri_string)
http = Net::HTTP.new(uri.host, uri.port)
request = Net::HTTP::Get.new(uri.request_uri)
response = http.request(request)
uri_string = response['location'] if response['location']
unless response.kind_of? Net::HTTPRedirection
response.ensure_success
break
end
if tries == 10
puts "Timing out after 10 tries"
break
end
tries += 1
end
response
end
Not sure if anyone is looking for this exact solution, but if you are trying to download an image http/https and store it to a variable
require 'open_uri_redirections'
require 'net/https'
web_contents = open('file_url_goes_here', :ssl_verify_mode => OpenSSL::SSL::VERIFY_NONE, :allow_redirections => :all) {|f| f.read }
puts web_contents

Web Server Flow in the Rack OAuth-2 Server

I'm trying to integrate the Rack OAuth-2 server into my sinatra application, to use it in a web-server flow implementation and I can't make it work :(. I the following code in the oauth controller
require "rack/oauth2/sinatra"
module RestKit
module Network
class OAuth2 < Sinatra::Base
use Rack::Logger
set :sessions, true
set :show_exceptions, true
ENV["DB"] = "test"
DATABASE = Mongo::Connection.new[ENV["DB"]]
register Rack::OAuth2::Sinatra
oauth.authenticator = lambda do |username, password|
"Batman" if username == "cowbell" && password == "more" end
oauth.host = "localhost"
oauth.database = DATABASE
# 3. Obtaining End-User Authorization
before "/oauth/*" do
halt oauth.deny! if oauth.scope.include?("time-travel") # Only Superman can do that
end
get "/oauth/authorize" do
"client: #{oauth.client.display_name}\nscope: #{oauth.scope.join(", ")}\nauthorization: #{oauth.authorization}"
end
post "/oauth/grant" do
oauth.grant! "Batman"
end
post "/oauth/deny" do
oauth.deny!
end
# 5. Accessing a Protected Resource
before { #user = oauth.identity if oauth.authenticated? }
oauth_required "/user"
get "/user" do
#user
end
get "/list_tokens" do
oauth.list_access_tokens("Batman").map(&:token).join(" ")
end
end
end
end
Then I try to obtain an authorization code using curl from terminal with:
curl -i http://localhost:4567/oauth/authorize -F response_type=code -F client_id=[the ID] -F client_secret=[the secret] -F redirect_uri=http://localhost:4567/oauth/showcode
and Just I got as a response:
HTTP/1.1 400 Bad Request
Content-Type: text/plain
Content-Length: 20
Connection: keep-alive
Server: thin 1.2.11 codename Bat-Shit Crazy
Missing redirect URL
Do you have any ideas what I'm doing wrong? Thanks!
The end of your curl request is:
-F redirect_uri=http://localhost:4567/oauth/showcode
but you haven't defined that route in the code above, i.e. where is:
get "/oauth/showcode" do
? That's why the error is "Missing redirect URL".

Passing data between blocks using sinatra

I'm trying to pass data between blocks using sinatra. For example:
#data = Hash.new
post "/" do
#data[:test] = params.fetch("test").to_s
redirect "/tmp"
end
get "/tmp" do
puts #data[:test]
end
However whenever i get to the tmp block #data is nil and throws an error. Why is that?
The reason is because the browser actually performs two separate HTTP requests.
Request: POST /
Response: 301 -> Location: /tmp
Request: GET /tmp
Response: ...
Two requests means two separate processes thus the #data instance variable is cleared once the first response is sent.
If you want to preserve the information, you need to use cookies or sessions, otherwise pass the data in querystring
post "/" do
test = params[:test]
redirect "/tmp?test=#{test}"
end
get "/tmp" do
puts params[:test]
end

How to make an HTTP GET with modified headers?

What is the best way to make an HTTP GET request in Ruby with modified headers?
I want to get a range of bytes from the end of a log file and have been toying with the following code, but the server is throwing back a response saying that "it is a request that the server could not understand" (the server is Apache).
require 'net/http'
require 'uri'
#with #address, #port, #path all defined elsewhere
httpcall = Net::HTTP.new(#address, #port)
headers = {
'Range' => 'bytes=1000-'
}
resp, data = httpcall.get2(#path, headers)
Is there a better way to define headers in Ruby?
Does anyone know why this would be failing against Apache? If I do a get in a browser to http://[address]:[port]/[path] I get the data I am seeking without issue.
Created a solution that worked for me (worked very well) - this example getting a range offset:
require 'uri'
require 'net/http'
size = 1000 #the last offset (for the range header)
uri = URI("http://localhost:80/index.html")
http = Net::HTTP.new(uri.host, uri.port)
headers = {
'Range' => "bytes=#{size}-"
}
path = uri.path.empty? ? "/" : uri.path
#test to ensure that the request will be valid - first get the head
code = http.head(path, headers).code.to_i
if (code >= 200 && code < 300) then
#the data is available...
http.get(uri.path, headers) do |chunk|
#provided the data is good, print it...
print chunk unless chunk =~ />416.+Range/
end
end
If you have access to the server logs, try comparing the request from the browser with the one from Ruby and see if that tells you anything. If this isn't practical, fire up Webrick as a mock of the file server. Don't worry about the results, just compare the requests to see what they are doing differently.
As for Ruby style, you could move the headers inline, like so:
httpcall = Net::HTTP.new(#address, #port)
resp, data = httpcall.get2(#path, 'Range' => 'bytes=1000-')
Also, note that in Ruby 1.8+, what you are almost certainly running, Net::HTTP#get2 returns a single HTTPResponse object, not a resp, data pair.

Resources