I have this method in my class:
def self.get(url)
#TODO We could test with https too
if url.match(/^http/)
correct_url = url
else
correct_url = "http://#{url}"
end
uri = URI.parse(correct_url)
if uri.respond_to? 'request_uri'
http = Net::HTTP.new(uri.host, uri.port)
request = Net::HTTP::Get.new(uri.request_uri)
http.request(request)
else
puts "Incorrect URI"
end
end
Unfortunately it's not following the redirects.
Can someone tell me how to make this method allow a maximum of three redirects?
Try this:
def self.get(url)
# TODO: test with https too
url = "http://#{url}" unless url.match(/^http/)
3.times do
uri = URI.parse(url)
if uri.respond_to?(:request_uri)
response = Net::HTTP.get_response(uri)
case response.code
when '301', '302'
url = response.header['location']
else
return response
end
end
end
end
Related
I want to call an API using only slightly different URIs. This code snippet does what I want, but I want to know if there is a more efficient way to do it.
require 'net/http'
require 'json'
# These URIs differ only in the path
orders = URI('https://hft-api.lykke.com/api/Orders?orderType=Unknown')
asset_pairs = URI('https://hft-api.lykke.com/api/AssetPairs')
lykke_req = Net::HTTP::Get.new(orders)
lykke_req['User-Agent'] = 'curl/7.67.0'
lykke_req['api-key'] = 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx'
lykke_req['Accept'] = 'application/json'
response = Net::HTTP.start(orders.hostname,
orders.port,
:use_ssl => true) {|http| http.request(lykke_req)}
puts JSON.parse(response.body)
lykke_req = Net::HTTP::Get.new(asset_pairs)
lykke_req['User-Agent'] = 'curl/7.67.0'
lykke_req['Accept'] = 'application/json'
response = Net::HTTP.start(asset_pairs.hostname,
asset_pairs.port,
:use_ssl => true) {|http| http.request(lykke_req)}
puts JSON.parse(response.body)
All I do is to reuse the same code but with a slightly different URI.
For my lykke_req objects, I can write
puts lykke_req
puts lykke_req.path
which gives me
#<Net::HTTP::Get:0x00007f947f1fdce8>
/api/Orders?orderType=Unknown
So it seems to me all i have to do is change the value of lykke_req.path. But I can't work out how to do it. I am looking for something like this
lykke_req.path = "/api/AssetPairs"
which fails with
undefined method `path=' for #<Net::HTTP::Get GET> (NoMethodError)
I found this on the official documentation page, but I can't find out what [R] means. Does it mean read only? Do I really have to go through the hassle of creating a new URI object, then creating a new Net::HTTP::Get object each time?
path [R]
The problem here is that you're trying to alter the net request object instead of the uri object:
irb(main):001:0> uri = URI('https://hft-api.lykke.com/api/Orders?orderType=Unknown')
=> #<URI::HTTPS https://hft-api.lykke.com/api/Orders?orderType=Unknown>
irb(main):002:0> uri.path = '/foo'
=> "/foo"
irb(main):003:0> uri.to_s
=> "https://hft-api.lykke.com/foo?orderType=Unknown"
But I would really just wrap this in a class so that you can encapsulate and structure your code and avoid duplication:
class LykkeAPIClient
BASE_URI = URI('https://hft-api.lykke.com/api')
def initalize(api_key:)
#api_key = api_key
end
def get_orders
get '/Orders?orderType=Unknown'
end
def get_asset_pairs
get '/AssetPairs'
end
def get(path)
req = Net::HTTP::Get.new(BASE_URI.join(path))
req['User-Agent'] = 'curl/7.67.0'
req['Accept'] = 'application/json'
req['api-key'] = #api_key
response = Net::HTTP.start(req.hostname, req.port, use_ssl: true) do |http|
http.request(uri)
end
# #todo check response status!
JSON.parse(response.body)
end
end
#client = LykkeAPIClient.new(api_key: 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx')
#orders = #client.get_orders
Insstead of lykke_req.path= do lykke_req.uri.path=
https://ruby-doc.org/stdlib-2.6.5/libdoc/net/http/rdoc/Net/HTTPGenericRequest.html
What can cause while making an HTTP connection to return EBADF (Bad file descriptor).
Here is my following code wherein the HTTP connection is made. Although the error is very less now(happening very less) but before I put those error on rescue I need to understand what is the reason for the EBADF
def make_http_request(url, headers={})
uri = URI(url)
Net::HTTP.start(uri.host, uri.port) do |http|
req = Net::HTTP::Get.new(uri, headers)
resp = http.request(req)
if resp.code.to_i != 200
logger.error "Retrieve #{resp.code} with #{url} and #{headers}"
return false
end
return resp.body
end
rescue SocketError, Net::ReadTimeout, Errno::ECONNREFUSED => e
logger.error "make_http_request #{url} with #{headers} resulted in #{e.message} \n #{e.backtrace}"
return false
end
I have a feeling that connect syscall is receiving an FD which ain't valid at that given point in time. But still unable to understand how can that happens.
If it helps the code is used in an application that operates with multiple threads.
In a nutshell, the definition of the above method looks like this...
module Eval
def make_http_request(url, headers={})
...
...
..
end
def request_local_endpoint(url, headers)
response = make_http_request(url, headers)
response && response.fetch('bravo',nil)
end
def request_external_endpoint(url, headers)
response = make_http_request(url, headers)
response && response.fetch('token',nil)
end
end
class RequestBuilder
include Eval
attr_reader :data
def initialize(data)
#data = data
end
def start
token = request_external_endpoint('http://external.com/endpoint1',{'Content-Type'.freeze => 'application/json', 'Authorization' => 'abcdef'})
return unless token
result = request_local_endpoint('http://internal.com/endpoint1',{'Content-Type'.freeze => 'application/json'})
return result
end
end
10.times {
Thread.new { RequestBuilder.new('sample data').start }
}
Hi this is my very first Ruby program.
I'm trying to write a simple ruby app to make a request to a URL and see if it's available. If it is, it'll print OK and else it'll print false.
This is what I've got so far, can you please assist, do I need to import any libs?
class WebRequest
def initialize(name)
#name = name.capitalize
end
def makeRequest
puts "Hello #{#name}!"
#uri = URI.parse("https://example.com/some/path")
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE # read into this
#data = http.get(uri.request_uri)
end
end
req = WebRequest.new("Archie")
req.makeRequest
Here is sample code to do any request:
require 'net/http'
require 'uri'
url = URI.parse('http://www.example.com/index.html')
req = Net::HTTP::Get.new(url.path)
res = Net::HTTP.start(url.host, url.port) do |http|
http.request(req)
end
puts res.body
gem install httparty
then
require 'httparty'
response = HTTParty.get('https://example.com/some/pathm')
puts response.body
Something simpler:
[1] pry(main)> require 'open-uri'
=> true
[2] pry(main)> payload = open('http://www.google.com')
=> #<File:/var/folders/2p/24pztc5s63d69hhx81002bq80000gn/T/open-uri20131217-84948-ttwnho>
[3] pry(main)> payload.inspect
=> "#<Tempfile:/var/folders/2p/24pztc5s63d69hhx81002bq80000gn/T/open-uri20131217-84948-ttwnho>"
[4] pry(main)> payload.read
payload.read would return the response body and you can easy use payload as File object since it is an instance of Tempfile
This is what I've ended up with
require 'net/http'
class WebRequest
def initialize()
#url_addr = 'http://www.google.com/'
end
def makeRequest
puts ""
begin
url = URI.parse(#url_addr)
req = Net::HTTP::Get.new(url.path)
res = Net::HTTP.start(url.host, url.port) {|http|
http.request(req)
}
puts "OK Connected to #{#url_addr} with status code #{res.code}"
rescue
puts "Failed to connect to #{#url_addr}"
end
end
end
req = WebRequest.new()
req.makeRequest
I am following Jimm Stout's suggestion for websites that don't set content-type.
agent = Mechanize.new do |a|
a.post_connect_hooks << ->(_,_,response,_) do
if response.content_type.empty?
response.content_type = 'text/html'
end
end
end
How do I avoid setting the Content-Type if I get a redirect, a 40x or a 50x.
You can make sure the response class is Net::HTTPOK.
agent = Mechanize.new do |a|
a.post_connect_hooks << ->(_,_,response,_) do
break unless response.class == Net::HTTPOK
if response.content_type.empty?
response.content_type = 'text/html'
end
end
end
Suppose that after primitive validation of user submitted URL I have the string that looks like URL:
url = 'http://www.thisdomaindoesntexist.com/dont_even_ask_about/this/uri'
How can I check if its available or not?
I tried this in my is_valid_link function:
require "net/http"
url = URI.parse(url)
req = Net::HTTP.new(url.host, url.port)
res = req.request_head(url.path)
It works if the server exists giving me back the HTTP response, but the problem is that in case of bad url I get an error like this:
SocketError in PostsController#create
getaddrinfo: nodename nor servname provided, or not known
How should I do this kind of validation properly?
Thanks in advance.
You can use rescue to catch errors and do some error handling
begin
require "net/http"
url = URI.parse(url)
req = Net::HTTP.new(url.host, url.port)
res = req.request_head(url.path)
rescue
# error occured, return false
false
else
# valid site
true
end
Use rescue inline:
require "net/http"
url = URI.parse(url)
req = Net::HTTP.new(url.host, url.port)
res = req.request_head(url.path) rescue false
I've looked for a way to check if URL existed for 5 hours and this thread actually helped me.
I'm a newbie in rails and wanted to find something easy.
Here how I integrated the code into controller:
require "net/http"
def url
url = URI.parse('http://www.url that you want to check.com/' + "/")
end
def req
#req = Net::HTTP.new(url.host, url.port)
end
def res #res
#res = req.request_head(url.path)
rescue
false
end
def test
if res == false
"something"
else
"another thing"
end
you have to make sure that URL ends with "/", else the code won't work.