I'm using open-uri to get content from a page on the web to be used with nokogiri.
I'm trying something like:
url = "http://pesquisa.bvsalud.org/portal/?output=site&lang=pt&from=0&sort=&format=summary&count=20&fb=&page=1&q=\"qualidade+de+vida\"&index=tw"
response = open(url)
Then I get the error: URI::InvalidURIError: bad URI(is not URI?)
The catch is: I know I can use URI.encode(url) to prevent some special characters in the url, but the website I'm requesting doesn't give me the same response when I sanitize the url, it doesn't answer properly when using '%22' instead of double quotes..
How can I make such request using double quotes? Any other library that can do it? Open-uri doesn't accept that. I tryed to use the gems addressable-uri and eat, but I get the same error on both. :/
URI.encode('http://pesquisa.bvsalud.org/portal/?output=site&lang=pt&from=0&sort=&format=summary&count=20&fb=&page=1&q=\"qualidade+de+vida\"&index=tw')
=> "pesquisa.bvsalud.org/portal/?output=site&lang=pt&from=0&sort=&format=summary&count=20&fb=&page=1&q=%5C%22qualidade+de+vida%5C&index=tw"
Related
I have this Sinatra::Base code:
class Crush < Sinatra::Base
post '/upload' do
erb params.inspect
end
end
I am using Postman and its interface for uploading a file. So I send a POST request with form-data, where in the body of the request the name is hello and the value is a file test.txt which contains just a simple string hey there.
When I do params.inspect I get this long string
{"------WebKitFormBoundaryocOEEr26iZGSe75n\r\nContent-Disposition: form-data; name"=>"\"hello\"; filename=\"test.txt\"\r\nContent-Type: text/plain\r\n\r\nhey there\r\n------WebKitFormBoundaryocOEEr26iZGSe75n--\r\n"}
So basically a long has with a single key and a single value. Reading most Sinatra tutorials (where the file is accepted from a form), there's a nice way Sinatra handles this using params[:file], but this doesn't seem to be the case when the file is coming straight from the body of an HTTP request.
I tried a non-modular approach too withou Sinatra::Base, thinking it's some parsing middle-ware missing, but got the same result.
Is there something I'm missing here? Must I go and make my own custom parser to get the content of this long hash? Or is there an easier way?
I figured it's Postman issue. When I switch from 'x-www-form-urlencoded' to 'form-data' in Postman, in the Header section, the field: Content-Type => application/x-www-form-urlencoded is NOT removed. So for those who encounter this problem, make sure you remove it manually.
I'm working on a fork of a library that implements Faraday to build URLs.
site = "https://example.io/#/"
path = "oauth/authorize"
connection = Faraday.new(site)
resource = Faraday::Utils.URI(path)
URL = connection.build_url(resource)
Notice that my site URL ends with a hashbang. But when the above code is executed, Faraday strips out the hashbang entirely:
https://example.io/oauth/authorize
But my application requires it to build this URL (with the hashbang):
https://example.io/#/oauth/authorize
Now before I go ripping out Faraday and monkey-patching something terrible.. can I do this by setting an option on Faraday?
I think the answer here would be to quit trying to preserve the hash portion of the URL in Faraday since that portion is ignored for HTTP requests.
The hash part of the URL (also known as URI "fragment identifier") is never sent to the server. It can only have a meaning in the client. Typically, when the HTTP client is a web browser, the fragment identifier holds the name of the element to scroll to. Or, hashbang tricks can be employed with some JavaScript interaction.
But to use such URLs in Faraday doesn't make sense because the hash portion will never get sent to the server anyway.
Having '#' in path variable instead of site variable i am getting the output as you require.
site = "https://example.io/"
path = "#/oauth/authorize"
connection = Faraday.new(site)
resource = Faraday::Utils.URI(path)
URL = connection.build_url(resource)
Please try the above code and let me know the result.
I have a url, "localhost/test/http://myimage.com/" (I'm passing myimage.com, because it's hosted on another site and I'm accessing it via an api) my question is how do I go about encoding the image portion of the URL? I thought about doing a gsub on the '.' and '/' and then gsubing them back, but I'm wondering if there's an easier way. Thanks for your help.
You could use URI::encode_www_form_component(str) and URI::decode_www_form_component
Check: http://www.ruby-doc.org/stdlib-1.9.3/libdoc/uri/rdoc/URI.html
You can use the uri library to escape and unescape a url
require 'uri'
escaped = URI.escape(data, Regexp.new("[^#{URI::PATTERN::UNRESERVED}]"))
and you can get the data back with
original = URI.unescape(escaped)
I have some special characters on some of my url's. For example:
http://blabla.com/title/?t=burası
When I am giving links to that pages from other pages, I use:
URI.encode(s)
which produces this link:
/title/?t=buras%C4%B1
While everything is normal until this point, also I have another form on the page which has the encoded url. But browsers tend to decode my encoded url when users visits /title/?t=buras%C4%B1 and turns the url into /title/?t=burası . The problem begins here because when the user tries to send something from the form on 'burası' page, sinatra gives an error:
ERROR URI::InvalidURIError: bad URI(is not URI?): http://localhost:3000/title/?t=burası
I think it is because of HTTP request's 'referrer' string, but I couldn't find any workarounds as long as browsers tend to decode the string automatically.
You should encode the URI in the FORM method.
For example, make your HTML FORM action look something like this:
<form action="/title/?t=buras%C4%B1" method="post">
Does that solve it for you?
(Edit: thanks x1a4 for the correction-- of course you're right! :)
I'm using net/http in Ruby to send a GET request to a web service. The important bit of code is the following:
Net::HTTP.start("www.thehost.com") do |http|
req = Net::HTTP::Get.new("/api" + ROUTES[sym] + "?" + params)
resp = http.request req
end
params is a string which contains key-value pairs in the form key=val&blag=blorg. When the response comes in, it turns out to be an error page from the server, which quotes the request URI; instead of key=val&blag=blorg, it has key=val&blag=blorg. Since when I enter the same address into a web browser with & instead of &, I get the expected response, I suspect that the escaping of & is what's causing the problem. If anyone with more experience disagrees with that, however, feel free to rename my question!
I should note that when I use http://www.thehost.com/api#{ROUTES[sym]}?#{params} with Net::HTTP.get_response, I get the expected response. What can I do to fix this?
Just a wild guess: are you doing this in Rails, and could its XSS protection/HTML escaping features be the culprit? What happens if you change it to params.html_safe?
Since using #{...} in one case works, what happens if you do "/api#{ROUTES[sym]}?#{params}" instead of concatenating strings with +?