Trying to parse through all pages in category of products. I want to check every page using original_url+'&p='+value, where value is number of pages.
How to write this properly full_source = url + "&p=" + "#{number_of_page}" ``
I think you want
full_source = "#{url}?p=#{number_of_page}"
Your question had an ampersand:
full_source = "#{url}&p=#{number_of_page}"
A better way to do it would be:
uri = URI.parse(url)
uri.query = URI.encode_www_form({ p: number_of_page })
puts uri.to_s # URL with query params
Here is an example:
url = 'http://google.com/search'
uri = URI.parse(url)
uri.query = URI.encode_www_form({ q: 'ruby' })
puts uri.to_s
# => "http://google.com/search?q=ruby""
Related
I'm simply trying to get a response from the API that includes certain fields that I'm specifying in my uri string but I keep receiving an InvalidURIError. I've come here as a last resort, having spent hours trying to debug this.
I've already tried using the URI.encode() method on it as well, but only get the same error.
Here's my code:
url = params[:url]
uri = URI('https://graph.facebook.com/v2.3/?id=' + url + '&fields=share,og_object{id,url,engagement}&access_token=' + CONFIG['fb_access_token'])
req = Net::HTTP::Post.new(uri.path)
req.set_form_data('fields' => 'og_object[engagement]','access_token' => CONFIG['fb_access_token'])
res = Net::HTTP.new(uri.host, uri.port)
res.verify_mode = OpenSSL::SSL::VERIFY_NONE
res.use_ssl = true
response = nil
res.start do |http|
response = http.request(req)
end
response = http.request(req)
output = ""
output << "#{response.body} <br />"
return output
And the error I'm receiving:
URI::InvalidURIError - bad URI(is not URI?): https://graph.facebook.com/v2.3/?id=http://www.wikipedia.org&fields=share,og_object{id,url,engagement}&access_token=960606020650536|eJC0PoCARFaqKZWZHdwN5ogkhfs
I'm just exhausted at this point so if I left out any important information just let me know and I'll respond with it as soon as I can. Thank you!
The problem is you're just dumping strings into your URI without escaping them first.
Since you're using Sinatra you can use Rack::Utils.build_query to construct your URI's query component with the values correctly escaped:
uri = URI('https://graph.facebook.com/v2.3/')
uri.query = Rack::Utils.build_query(
id: url,
fields: 'share,og_object{id,url,engagement}',
access_token: CONFIG['fb_access_token']
)
I have to add a new param to an indeterminate URL, let's say param=value.
In case the actual URL has already params like this
http://url.com?p1=v1&p2=v2
I should transform the URL to this other:
http://url.com?p1=v1&p2=v2¶m=value
But if the URL has not any param yet like this:
http://url.com
I should transform the URL to this other:
http://url.com?param=value
I feel worry to solve this with Regex because I'm not sure that looking for the presence of & could be enough. I'm thinking that maybe I should transform the URL to an URI object, and then add the param and transform it to String again.
Looking for any suggestion from someone who has been already in this situation.
Update
To help with the participation I'm sharing a basic test suite:
require "minitest"
require "minitest/autorun"
def add_param(url, param_name, param_value)
# the code here
"not implemented"
end
class AddParamTest < Minitest::Test
def test_add_param
assert_equal("http://url.com?param=value", add_param("http://url.com", "param", "value"))
assert_equal("http://url.com?p1=v1&p2=v2¶m=value", add_param("http://url.com?p1=v1&p2=v2", "param", "value"))
assert_equal("http://url.com?param=value#&tro&lo&lo", add_param("http://url.com#&tro&lo&lo", "param", "value"))
assert_equal("http://url.com?p1=v1&p2=v2¶m=value#&tro&lo&lo", add_param("http://url.com?p1=v1&p2=v2#&tro&lo&lo", "param", "value"))
end
end
require 'uri'
uri = URI("http://url.com?p1=v1&p2=2")
ar = URI.decode_www_form(uri.query) << ["param","value"]
uri.query = URI.encode_www_form(ar)
p uri #=> #<URI::HTTP:0xa0c44c8 URL:http://url.com?p1=v1&p2=2¶m=value>
uri = URI("http://url.com")
uri.query = "param=value" if uri.query.nil?
p uri #=> #<URI::HTTP:0xa0eaee8 URL:http://url.com?param=value>
EDIT:(by fguillen, to merge all the good propositions and also to make it compatible with his question test suite.)
require 'uri'
def add_param(url, param_name, param_value)
uri = URI(url)
params = URI.decode_www_form(uri.query || "") << [param_name, param_value]
uri.query = URI.encode_www_form(params)
uri.to_s
end
More elegant solution:
url = 'http://example.com?exiting=0'
params = {new_param: 1}
uri = URI.parse url
uri.query = URI.encode_www_form URI.decode_www_form(uri.query || '').concat(params.to_a)
uri.to_s #=> http://example.com?exiting=0&new_param=1
Well, you may also not know if this parameter already exists in url. If you want to replace it with new value in this case, you can do this:
url = 'http://example.com?exists=0&other=3'
params = {'exists' => 1, "not_exists" => 2}
uri = URI.parse url
uri.query = URI.encode_www_form(URI.decode_www_form(uri.query || '').to_h.merge(params))
uri.to_s
You can try to use my gem iri:
Iri.new('http://url.com?p1=v1&p2=v2').add(:param, 'value').to_s
I need to post using three parameters and a body which consists of 512 bytes. I can get the body right but I can't seem to get the parameters to take:
require 'net/http'
#ip_address = Array['cueserver.dnsalias.com']
#cueserver = 0
#playback = 'p1'
def send_cuescript(data)
params = {'id' => '1', 'type' => "20",'dst' => 'RES' }
begin
url = URI.parse('http://'+ #ip_address[#cueserver] + '/set.cgi')
http = Net::HTTP.new(url.host, url.port)
response, body = http.post(url.path, params, data)
rescue Timeout::Error, Errno::EINVAL, Errno::ECONNRESET, EOFError,
Net::HTTPBadResponse, Net::HTTPHeaderSyntaxError, Net::ProtocolError => e
end
response_array = []
puts 'got this value: ' + response.to_s
response.body.each_byte { |e| response_array.push(e.to_s(16))}
end
data_array = Array.new(512, "\x80")
send_cuescript(data_array.join)
I am getting an error from the initialize_http_header. I know there must be a way to set the parameters and the body separately but I can't seem to find any reference to this.
Why do you have to send part of the params in the url and part of it in the body?
If you have to do this, try
url = URI.parse('http://'+ #ip_address[#cueserver] + '/set.cgi?' + params.to_param)
PS: to_param is from active support. You need to write your own if you are not using active support.
In the documentation for URI.parse is the following code:
require 'uri'
uri = URI.parse("http://www.ruby-lang.org/")
p uri
# => #<URI::HTTP:0x202281be URL:http://www.ruby-lang.org/>
I'm wondering why, in the example, there is the letter p, e.g. p uri.
Also, how is using .parse different from doing uri = URI("http://..."), as in the example at the top of the page?
It seems URI(url) and URI.parse(url) do exactly the same:
u1 = URI("http://stackoverflow.com/")
u2 = URI.parse("http://stackoverflow.com/")
u1 == u2 # => true
How can I send HTTP GET request with parameters via ruby?
I have tried a lot of examples but all of those failed.
I know this post is old but for the sake of those brought here by google, there is an easier way to encode your parameters in a URL safe manner. I'm not sure why I haven't seen this elsewhere as the method is documented on the Net::HTTP page. I have seen the method described by Arsen7 as the accepted answer on several other questions also.
Mentioned in the Net::HTTP documentation is URI.encode_www_form(params):
# Lets say we have a path and params that look like this:
path = "/search"
params = {q: => "answer"}
# Example 1: Replacing the #path_with_params method from Arsen7
def path_with_params(path, params)
encoded_params = URI.encode_www_form(params)
[path, encoded_params].join("?")
end
# Example 2: A shortcut for the entire example by Arsen7
uri = URI.parse("http://localhost.com" + path)
uri.query = URI.encode_www_form(params)
response = Net::HTTP.get_response(uri)
Which example you choose is very much dependent on your use case. In my current project I am using a method similar to the one recommended by Arsen7 along with the simpler #path_with_params method and without the block format.
# Simplified example implementation without response
# decoding or error handling.
require "net/http"
require "uri"
class Connection
VERB_MAP = {
:get => Net::HTTP::Get,
:post => Net::HTTP::Post,
:put => Net::HTTP::Put,
:delete => Net::HTTP::Delete
}
API_ENDPOINT = "http://dev.random.com"
attr_reader :http
def initialize(endpoint = API_ENDPOINT)
uri = URI.parse(endpoint)
#http = Net::HTTP.new(uri.host, uri.port)
end
def request(method, path, params)
case method
when :get
full_path = path_with_params(path, params)
request = VERB_MAP[method].new(full_path)
else
request = VERB_MAP[method].new(path)
request.set_form_data(params)
end
http.request(request)
end
private
def path_with_params(path, params)
encoded_params = URI.encode_www_form(params)
[path, encoded_params].join("?")
end
end
con = Connection.new
con.request(:post, "/account", {:email => "test#test.com"})
=> #<Net::HTTPCreated 201 Created readbody=true>
I assume that you understand the examples on the Net::HTTP documentation page but you do not know how to pass parameters to the GET request.
You just append the parameters to the requested address, in exactly the same way you type such address in the browser:
require 'net/http'
res = Net::HTTP.start('localhost', 3000) do |http|
http.get('/users?id=1')
end
puts res.body
If you need some generic way to build the parameters string from a hash, you may create a helper like this:
require 'cgi'
def path_with_params(page, params)
return page if params.empty?
page + "?" + params.map {|k,v| CGI.escape(k.to_s)+'='+CGI.escape(v.to_s) }.join("&")
end
path_with_params("/users", :id => 1, :name => "John&Sons")
# => "/users?name=John%26Sons&id=1"