I'm building some kind of proxy.
When I call some url in a rack application, I forward that request to an other url.
The request I forward is a POST with a file and some parameters.
I want to add more parameters.
But the file can be quite big. So I send it with Net::HTTP#body_stream instead of Net::HTTP#body.
I get my request as a Rack::Request object and I create my Net::HTTP object with that.
req = Net::HTTP::Post.new(request.path_info)
req.body_stream = request.body
req.content_type = request.content_type
req.content_length = request.content_length
http = Net::HTTP.new(#host, #port)
res = http.request(req)
I've tried several ways to add the proxy's parameters. But it seems nothing in Net::HTTP allows to add parameters to a body_stream request, only to a body one.
Is there a simpler way to proxy a rack request like that ? Or a clean way to add my parameters to my request ?
Well.. as i see it, this is a normal behaviour. I'll explain why. If you only have access to a Rack::Request,(i guess that) your middleware does not parse the response (you do not include something like ActionController::ParamsParser), so you don't have access to a hash of parameters, but to a StringIo. This StringIO corresponds to a stream like:
Content-Type: multipart/form-data; boundary=AaB03x
--AaB03x
Content-Disposition: form-data; name="param1"
value1
--AaB03x
Content-Disposition: form-data; name="files"; filename="file1.txt"
Content-Type: text/plain
... contents of file1.txt ...
--AaB03x--
What you are trying to do with the Net::HTTP class is to: (1). parse the request into a hash of parameters; (2). merge the parameters hash with your own parameters; (3). recreate the request. The problem is that Net::HTTP library can't do (1), since it is a client library, not a server one.
Therefore, you can not escape parsing some how your request before adding the new parameters.
Possible solutions:
Insert ActionController::ParamsParser before your middleware. After that, you may use the excellent rest-client lib to do something like:
RestClient.post ('http://your_server' + request.path_info), :params => params.merge(your_params)
You can attempt to make a wrapper on the StringIO object, and add, at the end of stream,your own parameters. However, this is not trivial nor advisable.
Might be one year too late, but I had the same issue verifying Paypal IPNs. I wanted to forward back the IPN request to Paypal for verification but needed to add :cmd => '_notify-validate'.
Instead of modifying the body stream, or body, I appended it as part of the URL path, like so:
reply_request = Net::HTTP::Post.new(url.path + '?cmd=_notify-validate')
It seems a bit of a hack, but I think it's worth it if you aren't going to use it for anything else.
Related
I have this Sinatra::Base code:
class Crush < Sinatra::Base
post '/upload' do
erb params.inspect
end
end
I am using Postman and its interface for uploading a file. So I send a POST request with form-data, where in the body of the request the name is hello and the value is a file test.txt which contains just a simple string hey there.
When I do params.inspect I get this long string
{"------WebKitFormBoundaryocOEEr26iZGSe75n\r\nContent-Disposition: form-data; name"=>"\"hello\"; filename=\"test.txt\"\r\nContent-Type: text/plain\r\n\r\nhey there\r\n------WebKitFormBoundaryocOEEr26iZGSe75n--\r\n"}
So basically a long has with a single key and a single value. Reading most Sinatra tutorials (where the file is accepted from a form), there's a nice way Sinatra handles this using params[:file], but this doesn't seem to be the case when the file is coming straight from the body of an HTTP request.
I tried a non-modular approach too withou Sinatra::Base, thinking it's some parsing middle-ware missing, but got the same result.
Is there something I'm missing here? Must I go and make my own custom parser to get the content of this long hash? Or is there an easier way?
I figured it's Postman issue. When I switch from 'x-www-form-urlencoded' to 'form-data' in Postman, in the Header section, the field: Content-Type => application/x-www-form-urlencoded is NOT removed. So for those who encounter this problem, make sure you remove it manually.
I'm using Goliath and Grape. On my goliath server it calls the grape api like so:
when '/posts' then FrameworksAPI::API.call(env)
On my grape api class, my method is as simple as this:
get '/:id' do
Post.find(params[:id])
end
I'd like to modify the headers - specifically the 'Content-Length' but unsure how to.
Also i'd like to ask an additional question. How do i create callback/filters specifically before the method GET returns the result i'd like to modify the result.
Grape has a header helper for a few versions now.
header 'Content-Length`, 42.to_s
For your second question on modifying the body, try using after do ... at API level.
The return from your FrameworksAPI::API.call(env) method will be a triplet [status_code, headers, body]. So, instead of just returning that from your case you'd do something like:
when '/posts' then
status, headers, body = FrameworksAPI::API.call(env)
headers['whatever'] = blah
[status, headers, body]
You can also change the body, just be careful as the body maybe an array.
There is also a Content-Length middleware that is provided by Goliath. Content-Length is loaded by default although if you set a custom Content-Length it will take precedence. Just be carefull that other middlewares like the formattings don't change the body after you set your content-length.
I'm trying to create a route with Sinatra that only accepts POST with an Content-type: application/json without success.
My approach is as follows:
post '/dogs', :provides => :json do
# returns here a json response
end
Testing with curl, I have seen that :provides => :json configures the route to respond with an Content-Type: application/json.
That's right because I want also to respond with a JSON message to the POST request but I really need that this route only respond to POST requests with a Content-Type: application/json and not, for example, to others (e.g. Content-Type: application/xml).
Is there any way in Sinatra to restrict the route to only accept requests with a certain Content-Type?
Requests do not contain "Content-Type" header, but rather have "Accept". Sinatra should basically only respond to requests with "Accept" containing "application/json". Just to make sure:
post '/gods', :provides => :json do
pass unless request.accept? 'application/json'
...
end
Read this
http://rack.rubyforge.org/doc/classes/Rack/Request.html
request.content_type will tell you
Phil might be right regarding RFC but in reality many things put a content-type in a POST request, therefore it is useful to know what it is.
i would think it is something like:
pass unless request.accept? == 'application/json'
I am writing mock Ruby servers to test components of API. I send a POST request with a body, and I'd like my mock server to return a body of that POST request. Currently i have this code:
require 'socket'
webserver = TCPServer.new('127.0.0.1', 7125)
loop do
session = webserver.accept
session.print "HTTP/1.1 200/OK\r\nContent-type:text/html\r\n\r\n"
request = session.gets
session.puts request
session.close
end
A POST request with body FOO returns a response with body that contains only POST / HTTP/1.1 How to fix it?
Writing your own HTTP server is really going to get you into trouble because the specification, while superficially simple, has a number of subtle nuances that can trip you up. In this case, you're reading one line with gets and ignoring the bulk of the submission. You're going to have to address that by reading in and properly decoding the posted data.
For something with a familiar interface you might start with Net::HTTP::Server instead.
I would like to store the cookies from one open-uri call and pass them to the next one. I can't seem to find the right docs for doing this. I'd appreciate it if you could tell me the right way to do this.
NOTES: w3.org is not the actual url, but it's shorter; pretend cookies matter here.
h1 = open("http://www.w3.org/")
h2 = open("http://www.w3.org/People/Berners-Lee/", "Cookie" => h1.FixThisSpot)
Update after 2 nays: While this wasn't intended as rhetorical question I guarantee that it's possible.
Update after tumbleweeds: See (the answer), it's possible. Took me a good while, but it works.
I thought someone would just know, but I guess it's not commonly done with open-uri.
Here's the ugly version that neither checks for privacy, expiration, the correct domain, nor the correct path:
h1 = open("http://www.w3.org/")
h2 = open("http://www.w3.org/People/Berners-Lee/",
"Cookie" => h1.meta['set-cookie'].split('; ',2)[0])
Yes, it works. No it's not pretty, nor fully compliant with recommendations, nor does it handle multiple cookies (as is).
Clearly, HTTP is a very straight-forward protocol, and open-uri lets you at most of it. I guess what I really needed to know was how to get the cookie from the h1 request so that it could be passed to the h2 request (that part I already knew and showed). The surprising thing here is how many people basically felt like answering by telling me not to use open-uri, and only one of those showed how to get a cookie set in one request passed to the next request.
You need to add a "Cookie" header.
I'm not sure if open-uri can do this or not, but it can be done using Net::HTTP.
# Create a new connection object.
conn = Net::HTTP.new(site, port)
# Get the response when we login, to set the cookie.
# body is the encoded arguments to log in.
resp, data = conn.post(login_path, body, {})
cookie = resp.response['set-cookie']
# Headers need to be in a hash.
headers = { "Cookie" => cookie }
# On a get, we don't need a body.
resp, data = conn.get(path, headers)
Thanks Matthew Schinckel your answer was really useful. Using Net::HTTP I was successful
# Create a new connection object.
site = "google.com"
port = 80
conn = Net::HTTP.new(site, port)
# Get the response when we login, to set the cookie.
# body is the encoded arguments to log in.
resp, data = conn.post(login_path, body, {})
cookie = resp.response['set-cookie']
# Headers need to be in a hash.
headers = { "Cookie" => cookie }
# On a get, we don't need a body.
resp, data = conn.get(path, headers)
puts resp.body
Depending on what you are trying to accomplish, check out webrat. I know it is usually used for testing, but it can also hit live sites, and it does a lot of the stuff that your web browser would do for you, like store cookies between requests and follow redirects.
you would have to roll your own cookie support by parsing the meta headers when reading and adding a cookie header when submitting a request if you are using open-uri. Consider using httpclient http://raa.ruby-lang.org/project/httpclient/ or something like mechanize instead http://mechanize.rubyforge.org/ as they have cookie support built in.
There is a RFC 2109 and RFC 2965 cookie jar implementation to be found here for does that want standard compliant cookie handling.
https://github.com/dwaite/cookiejar