I'm still learning ruby, and there seems to be a problem with my code that i can't figure out
require 'net/http'
File.open("html.txt", "r") do |file_handle|
file_handle.each_line do |server|
uri = URI( server )
res = Net::HTTP.get_response(uri)
if res.code != 200
puts " #{uri} => #{res.code}"
end
end
end
html.txt
http://stackoverflow.com
http://google.com
http://facebook.com
http://serverfault.com
http://twitter.com
I don't want to print out the URI's which has a response '200'
But right now this is what gets printed:
http://stackoverflow.com => 200
http://google.com => 302
http://facebook.com => 302
http://serverfault.com => 200
http://twitter.com => 301
This is my expected output:
http://google.com => 302
http://facebook.com => 302
http://twitter.com => 301
Can anyone help me with this..? thanks in advance.
The response code is a string.
You should be using the condition res.code != '200', note the quotes.
Your code should look like this:
require 'net/http'
File.open("html.txt", "r") do |file_handle|
file_handle.each_line do |server|
uri = URI( server )
res = Net::HTTP.get_response(uri)
if res.code != '200'
puts " #{uri} => #{res.code}"
end
end
end
Related
I would like to limit searching images using
"filter query parameters ( https://msdn.microsoft.com/en-us/library/dn760791.aspx )". But I always get photos whose size is around 250 - 300 pixel( both of width and height), although I want them, which is bigger than 500 x 500 pixel.
I know there is already similar question(Bing Image Search API filter by image size), but I couldn't solve the problem.
I'm using Ruby and the code is following.
What is the problem?
require "open-uri"
require "FileUtils"
require 'net/http'
require 'json'
#dirName = "/Users/hoge/img"
FileUtils.mkdir_p(#dirName) unless FileTest.exist?(#dirName)
def save_image(url, num)
filePath = "#{#dirName}/christ#{num.to_s}.jpg"
open(filePath, 'wb') do |output|
open(url) do |data|
output.write(data.read)
end
end
end
search_word = 'christ painting'
count = 5
size = 'Large'
uri = URI('https://api.cognitive.microsoft.com/bing/v5.0/images/search')
uri.query = URI.encode_www_form({
'q' => search_word,
'count' => count,
'size' => size
})
request = Net::HTTP::Post.new(uri.request_uri)
request['Content-Type'] = 'multipart/form-data'
request['Ocp-Apim-Subscription-Key'] = 'mykey' # Fix Me
request.body = "{body}"
response = Net::HTTP.start(uri.host, uri.port, :use_ssl => uri.scheme == 'https') do |http|
http.request(request)
end
count.times do |i|
begin
image_url = JSON.parse(response.body)["value"][i]["thumbnailUrl"]
save_image(image_url, i)
rescue => e
puts "image#{i} is error!"
puts e
end
end
I am using net/http to send a bunch of request to some internal IP addresses.
Here's a snippet of the code:
File.open("internalcorpIPs", "r") do |f|
f.each_line do |line|
puts line
res = Net::HTTP.get_response(URI.parse(line))
getCode = res.code
end
end
I'm strictly just making a request to http://IP and https://IP but it seems like this method only works if every single IP/line address is live. How do I skip IP addresses with no webserver (or 80/443 ports)?
Is it possible to make it read the line, and move on to the next if no response code was returned?
Thanks!
You could simply wrap your request in begin/rescue block like this:
File.open("internalcorpIPs", "r") do |f|
f.each_line do |line|
puts line
begin
# strip and encode uri from the file
uri = URI.parse(URI.encode(line.strip))
res = Net::HTTP.get_response(uri)
getCode = res.code
rescue Timeout::Error => e
puts e
false
end
end
end
But you will wait for 60 seconds at least before going in timeout, so I suggest to decrease the timeout. Furthermore, you could introduce an additional guard clause to check if the uri contains the scheme http:// or https://, otherwise raise an exception (or something else).
require 'net/http'
File.open("internalcorpIPs", "r") do |f|
f.each do |line|
puts line
begin
# strip and encode uri from the file
uri = URI.parse(URI.encode(line.strip))
# if uri misses the schema (http:// or https://) -> raise error
raise URI::Error, "uri #{uri} miss the scheme" unless uri.scheme
http = Net::HTTP.new(uri.host, uri.port)
http.open_timeout = 2 # seconds
http.read_timeout = 2 # seconds
http.start do |conn|
response = conn.request_get(path = '/')
puts response.code
end
rescue Timeout::Error, URI::Error, SocketError => e
puts e
false
end
end
end
Additional notes:
Open Timeout
Number of seconds to wait for the connection to open. Any number may be used, including Floats for fractional seconds. If the HTTP object cannot open a connection in this many seconds, it raises a Net::OpenTimeout exception. The default value is 60 seconds.
Read Timeout
Number of seconds to wait for one block to be read (via one read(2) call). Any number may be used, including Floats for fractional seconds. If the HTTP object cannot read data in this many seconds, it raises a Net::ReadTimeout exception. The default value is 60 seconds.
URI Scheme
Difference between generic uri (URI::Generic) and http uri (URI::HTTP).
uri = URI.parse('1.1.1.1')
=> #<URI::Generic 1.1.1.1>
uri.scheme
=> nil
uri.host
=> nil
uri.port
=> nil
uri.path
=> "1.1.1.1"
uri = URI.parse('http://1.1.1.1')
=> #<URI::HTTP http://1.1.1.1>
uri.scheme
=> "http"
uri.host
=> "1.1.1.1"
uri.port
=> 80
uri.path
=> ""
references:
Net::HTTP Api
URI Module
hope it helps!
UPDATE
URI.parse accepts a string as argument and automatically set the port if not specified:
❯ irb
2.2.0 :001 > require 'net/http'
=> true
2.2.0 :002 > uri = URI.parse('http://1.1.1.1')
=> #<URI::HTTP http://1.1.1.1>
2.2.0 :003 > uri.host
=> "1.1.1.1"
2.2.0 :004 > uri.port
=> 80
2.2.0 :005 > uri2 = URI.parse('http://mydomain')
=> #<URI::HTTP http://mydomain>
2.2.0 :006 > uri2.host
=> "mydomain"
2.2.0 :007 > uri2.port
=> 80
2.2.0 :008 > uri3 = URI.parse('https://mydomain')
=> #<URI::HTTPS https://mydomain>
2.2.0 :009 > uri3.host
=> "mydomain"
2.2.0 :010 > uri3.port
=> 443
I am trying to pull campaign stats from Clickbank API in ruby. When I run the sample code Clickbank provided. I get the following error:
uninitialized constant Net::HTTPS (NameError). What am I missing?
Example Code.
require 'net/http'
require 'net/https'
http = Net::HTTPS.new('api.clickbank.com')
http.use_ssl = false
path = '/rest/1.3/orders/list'
headers = {
'Authorization' => '<< DEVKEY >>:<< APIKEY>>',
'Accept' => 'application/json'
}
resp, data = http.get(path, nil, headers)
puts 'Code = ' + resp.code
puts 'Message = ' + resp.message
resp.each {|key, val| puts key + ' = ' + val}
puts data
Yes I put my dev and api key into
In Ruby 2.4.1 enable ssl as a parameter of Net::HTTP.start
Net::HTTP.start(uri.host, uri.port, use_ssl: true)
https://ruby-doc.org/stdlib-2.4.1/libdoc/net/http/rdoc/Net/HTTP.html#class-Net::HTTP-label-HTTPS
Use Net:HTTP and enable SSL instead of using Net::HTTPS and disabling SSL.
Example:
http = Net::HTTP.new('api.clickbank.com')
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
You actually don't want to disable ssl as that API requires it. I was able to get it working like so based on the documentation for http found here: http://ruby-doc.org/stdlib-2.1.1/libdoc/net/http/rdoc/Net/HTTP.html
require 'net/http'
uri = URI('https://api.clickbank.com/rest/1.3/orders/list')
req = Net::HTTP::Get.new(uri)
# set headers on the request
req['Authorization'] = '<< DEVKEY >>:<< APIKEY>>'
req['Accept'] = 'application/json'
# perform the request
resp, data = Net::HTTP.start(uri.hostname, uri.port) {|http|
http.request(req)
}
puts 'Code = ' + resp.code
puts 'Message = ' + resp.message
resp.each {|key, val| puts key + ' = ' + val}
puts data
For example, to get response 200 OK from "example.com", necessary:
require 'net/http'
uri = URI('http://example.com/index.html')
res = Net::HTTP.get_response(uri)
puts res.code # => '200'
puts res.message # => 'OK'
How to make similar for pop.gmail.com?
Try this:
require "net/pop"
Net::POP3.enable_ssl(OpenSSL::SSL::VERIFY_NONE)
conn = Net::POP3.new("pop.gmail.com", 995)
conn.start(user_name, password)
conn.started?
I have four arguments taken from user input in the script. All other string interpolation arguments work fine except for URI.parse.
Snippets from the code are:
require 'net/http'
#ARGVs:
prompt = 'test: '
url = ARGV[0]
user = ARGV[1]
pass = ARGV[2]
xml_user = ARGV[3]
# User supplied input:
puts "Whats the URL?"
print prompt
url = STDIN.gets.chomp()
# HTTP connection
uri = URI.parse('#{url}')
req = Net::HTTP.new(uri.hostname, uri.port)
# Header: Creds to get a session
user_and_pass = "#{user}" + ':' + "#{pass}"
base64user_and_pass = Base64.encode64(user_and_pass)
# POST method
res = req.post(uri.path, xml_data, {'Content-Type' => 'text/xml', 'Content-Length' => xml_data.length.to_s,
'Authorization' => "Basic #{base64user_and_pass}", "Connection" => "keep-alive" })
puts res.body
Error:
Ruby200-x64/lib/ruby/2.0.0/uri/common.rb:176:in `split': bad URI(is not URI?): #{url} (URI::InvalidURIError)
A point in the right direction would be appreciated.
uri = URI.parse('#{url}')
should be:
uri = URI.parse(url)
Here's a bit more idiomatic-Ruby version of the code:
require 'net/http'
PROMPT = 'test: '
# ARGVs:
url, user, pass, xml_user = ARGV[0, 4]
# User supplied input:
puts "Whats the URL?"
print PROMPT
url = STDIN.gets.chomp()
# HTTP connection
uri = URI.parse(url)
req = Net::HTTP.new(uri.hostname, uri.port)
# Header: Creds to get a session
base64user_and_pass = Base64.encode64("#{ user }:#{ pass }")
# POST method
res = req.post(
uri.path,
xml_data,
{
'Content-Type' => 'text/xml',
'Content-Length' => xml_data.length.to_s,
'Authorization' => "Basic #{ base64user_and_pass }",
"Connection" => "keep-alive"
}
)
puts res.body
Don't do things like:
user_and_pass = "#{user}" + ':' + "#{pass}"
and:
uri = URI.parse('#{url}')
user, pass and url are already strings, so sticking them inside a string and interpolating their values is a waste of CPU. As developers we need to be aware of our data-types.
It could be written as one of these:
user_and_pass = user + ':' + pass
user_and_pass = '%s:%s' % [user, pass]
user_and_pass = [user, pass].join(':')
But it's more idiomatic to see it how I wrote it above.