I'm using net/http in Ruby to send a GET request to a web service. The important bit of code is the following:
Net::HTTP.start("www.thehost.com") do |http|
req = Net::HTTP::Get.new("/api" + ROUTES[sym] + "?" + params)
resp = http.request req
end
params is a string which contains key-value pairs in the form key=val&blag=blorg. When the response comes in, it turns out to be an error page from the server, which quotes the request URI; instead of key=val&blag=blorg, it has key=val&blag=blorg. Since when I enter the same address into a web browser with & instead of &, I get the expected response, I suspect that the escaping of & is what's causing the problem. If anyone with more experience disagrees with that, however, feel free to rename my question!
I should note that when I use http://www.thehost.com/api#{ROUTES[sym]}?#{params} with Net::HTTP.get_response, I get the expected response. What can I do to fix this?
Just a wild guess: are you doing this in Rails, and could its XSS protection/HTML escaping features be the culprit? What happens if you change it to params.html_safe?
Since using #{...} in one case works, what happens if you do "/api#{ROUTES[sym]}?#{params}" instead of concatenating strings with +?
Related
I am currently developing a Ruby API based on Sinatra. This API mostly receives GET requests from an existing social platform which supports external API integration.
The social platform fires off GET requests in the following format (only relevant parameters shown):
GET /{command}
Parameters: command and text
Where text is a string that the user has entered.
In my case, params[:text] is in fact a series of commands, delimited by a space. What I want to achieve is, for example: If params[:text]="corporate finance"
Then I want my API to interpret the request as a GET request to
/{command}/corporate/finance
instead of requesting /{command} with a string as a parameter containing the rest of the request.
Can this be achieved on my side? Nothing can be changed in terms of the initial request from the social platform.
EDIT: I think a better way of explaining what I am trying to achieve is the following:
GET /list?text=corporate finance
Should hit the same endpoint/route as
GET /list/corporate/finance
This must not affect the initial GET request from the social platform as it expects a response containing text to display to the user. Is there a neat, best practice way of doing this?
get "/" do {
text = params[:text].split.join "/"
redirect "#{params[:command]}/#{text}"
end
might do the trick. Didn't check though.
EDIT: ok, the before filter was stupid. Basically you could also route to "/" and then redirect. Or, even better:
get "/:command" do {
text = params[:text].split.join "/"
redirect "#{params[:command]}/#{text}"
}
There a many possible ways of achieving this. You should check the routes section of the sinatra docs (https://github.com/sinatra/sinatra)
The answer by three should do the trick, and to get around the fact that the filter will be invoked with every request, a conditional like this should do:
before do
if params[:text]
sub_commands = params[:text].split.join "/"
redirect "#{params[:command]}/#{sub_commands}"
end
end
I have tested it in a demo application and it seems to work fine.
The solution was to use the call! method.
I used a regular expression to intercept calls which match /something with no further parameters (i.e. /something/something else). I think this step can be done more elegantly.
From there, I split up my commands:
get %r{^\/\w+$} do
params[:text] ? sub_commands="/"+params[:text].split.join("/") : sub_commands=""
status, headers, body = call! env.merge("PATH_INFO" => "/#{params[:command]}#{sub_commands}")
[status, headers, body]
end
This achieves exactly what I needed, as it activates the correct endpoint, as if the URL was typed it the usual format i.e. /command/subcommand1/subcommand2 etc.
Sorry, I completely misunderstood your question, so I replace my answer with this:
require 'sinatra'
get '/list/?*' do
"yep"
end
like this, the following routes all lead to the same
You need to add a routine for each command or replace the command with a * and depend your output based on a case when.
The params entered by the user can be referred by the params hash.
http://localhost:4567/list
http://localhost:4567/list/corporate/finance
http://localhost:4567/list?text=corporate/finance
I am completely new to ruby. I have the following code:
body = "hello"
site = "api.mysite.net"
port = 80
conn = Net::HTTP.new(site, port)
resp, data = conn.post("/v1/profile", body, {})
puts body
my questions are:
Where should I go for a library on how NET::HTTP.new() , conn.post() etc... works?
What does the comma between resp and data mean?
How come puts body gives me nothing even though I have hello defined initially? And when passed through the post(), I figure it would assign it a value? but instead puts resp.body actually gives me the http response.
This is all so new to me, just trying to get a handle on things.
Read the docs I guess, but you will need background knowledge on HTTP to really understand it.
That's shorthand for assigning two variables at the same time, assuming the right-hand side returns an array of 2 (or more) items.
You've posted the body in your request, resp.body is the body in the response. I don't know why body should be empty though. I would double-check that, but it sounds like a side effect of conn.post if anything.
BTW there are several nice 3rd-party gems which make HTTP client development much easier than dealing with Net::HTTP, e.g. RESTClient, Excon, HTTparty. Check these out. Or if you want to use the standard Ruby library, also look at Open URI as a higher-level API.
I'm using open-uri to get content from a page on the web to be used with nokogiri.
I'm trying something like:
url = "http://pesquisa.bvsalud.org/portal/?output=site&lang=pt&from=0&sort=&format=summary&count=20&fb=&page=1&q=\"qualidade+de+vida\"&index=tw"
response = open(url)
Then I get the error: URI::InvalidURIError: bad URI(is not URI?)
The catch is: I know I can use URI.encode(url) to prevent some special characters in the url, but the website I'm requesting doesn't give me the same response when I sanitize the url, it doesn't answer properly when using '%22' instead of double quotes..
How can I make such request using double quotes? Any other library that can do it? Open-uri doesn't accept that. I tryed to use the gems addressable-uri and eat, but I get the same error on both. :/
URI.encode('http://pesquisa.bvsalud.org/portal/?output=site&lang=pt&from=0&sort=&format=summary&count=20&fb=&page=1&q=\"qualidade+de+vida\"&index=tw')
=> "pesquisa.bvsalud.org/portal/?output=site&lang=pt&from=0&sort=&format=summary&count=20&fb=&page=1&q=%5C%22qualidade+de+vida%5C&index=tw"
I was trying to send a GET request to Twitter (user ID replaced for privacy reasons) using Net::HTTP:
url = URI.parse("http://api.twitter.com/1/friends/ids.json?user_id=12345")
resp = Net::HTTP.get_response(url)
this throws an exception in Net::HTTP:
NoMethodError: undefined method empty?' for #<URI::HTTP:0x59f5c04>
from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/net/http.rb:1470:ininitialize'
just by coincidence, I stumbled upon a similar code snippet, which used URI.encode prior to URI.parse, so I copied that and tried again:
url = URI.parse(URI.encode("http://api.twitter.com/1/friends/ids.json?user_id=12345"))
resp = Net::HTTP.get_response(url)
now it works fine, but why? There are no reserved characters that need escaping in the URL I mentioned, so why do I have to call URI.encode for get_response to succeed?
Ruby 1.8 get seems to require a forward slash at the end of the uri.
"http://www.google.com/"
worked, while
"http://www.google.com"
did not.
I would like to store the cookies from one open-uri call and pass them to the next one. I can't seem to find the right docs for doing this. I'd appreciate it if you could tell me the right way to do this.
NOTES: w3.org is not the actual url, but it's shorter; pretend cookies matter here.
h1 = open("http://www.w3.org/")
h2 = open("http://www.w3.org/People/Berners-Lee/", "Cookie" => h1.FixThisSpot)
Update after 2 nays: While this wasn't intended as rhetorical question I guarantee that it's possible.
Update after tumbleweeds: See (the answer), it's possible. Took me a good while, but it works.
I thought someone would just know, but I guess it's not commonly done with open-uri.
Here's the ugly version that neither checks for privacy, expiration, the correct domain, nor the correct path:
h1 = open("http://www.w3.org/")
h2 = open("http://www.w3.org/People/Berners-Lee/",
"Cookie" => h1.meta['set-cookie'].split('; ',2)[0])
Yes, it works. No it's not pretty, nor fully compliant with recommendations, nor does it handle multiple cookies (as is).
Clearly, HTTP is a very straight-forward protocol, and open-uri lets you at most of it. I guess what I really needed to know was how to get the cookie from the h1 request so that it could be passed to the h2 request (that part I already knew and showed). The surprising thing here is how many people basically felt like answering by telling me not to use open-uri, and only one of those showed how to get a cookie set in one request passed to the next request.
You need to add a "Cookie" header.
I'm not sure if open-uri can do this or not, but it can be done using Net::HTTP.
# Create a new connection object.
conn = Net::HTTP.new(site, port)
# Get the response when we login, to set the cookie.
# body is the encoded arguments to log in.
resp, data = conn.post(login_path, body, {})
cookie = resp.response['set-cookie']
# Headers need to be in a hash.
headers = { "Cookie" => cookie }
# On a get, we don't need a body.
resp, data = conn.get(path, headers)
Thanks Matthew Schinckel your answer was really useful. Using Net::HTTP I was successful
# Create a new connection object.
site = "google.com"
port = 80
conn = Net::HTTP.new(site, port)
# Get the response when we login, to set the cookie.
# body is the encoded arguments to log in.
resp, data = conn.post(login_path, body, {})
cookie = resp.response['set-cookie']
# Headers need to be in a hash.
headers = { "Cookie" => cookie }
# On a get, we don't need a body.
resp, data = conn.get(path, headers)
puts resp.body
Depending on what you are trying to accomplish, check out webrat. I know it is usually used for testing, but it can also hit live sites, and it does a lot of the stuff that your web browser would do for you, like store cookies between requests and follow redirects.
you would have to roll your own cookie support by parsing the meta headers when reading and adding a cookie header when submitting a request if you are using open-uri. Consider using httpclient http://raa.ruby-lang.org/project/httpclient/ or something like mechanize instead http://mechanize.rubyforge.org/ as they have cookie support built in.
There is a RFC 2109 and RFC 2965 cookie jar implementation to be found here for does that want standard compliant cookie handling.
https://github.com/dwaite/cookiejar