Is there a way to attach Ruby Net::HTTP request to a specific IP address / network interface? - ruby

Im looking a way to use different IP addresses for each GET request with standard Net::HTTP library. Server has 5 ip addresses and assuming that some API`s are blocking access when request limit per IP is reached. So, only way to do it - use another server. I cant find anything about it in ruby docs.
For example, curl allows you to attach it to specific ip address (in PHP):
$req = curl_init($url)
curl_setopt($req, CURLOPT_INTERFACE, 'ip.address.goes.here';
$result = curl_exec($req);
Is there any way to do it with Net::HTTP library? As alternative - CURB (ruby curl binding). But it will be the last thing i`ll try.
Suggestions / Ideas?
P.S. The solution with CURB (with dirty tests, ip`s being replaced):
require 'rubygems'
require 'curb'
ip_addresses = [
'1.1.1.1',
'2.2.2.2',
'3.3.3.3',
'4.4.4.4',
'5.5.5.5'
]
ip_addresses.each do |address|
url = 'http://www.ip-adress.com/'
c = Curl::Easy.new(url)
c.interface = address
c.perform
ip = c.body_str.scan(/<h2>My IP address is: ([\d\.]{1,})<\/h2>/).first
puts "for #{address} got response: #{ip}"
end

I know this is old, but hopefully someone else finds this useful, as I needed this today. You can do the following:
http = Net::HTTP.new(uri.host, uri.port)
http.local_host = ip
response = http.request(request)
Note that you I don't believe you can use Net::HTTP.start, as it doesn't accept local_host as an option.

There is in fact a way to do this if you monkey patch TCPSocket:
https://gist.github.com/800214
Curb is awesome but won't work with Jruby so I've been looking into alternatives...

Doesn't look like you can do it with Net:HTTP. Here's the source
http://github.com/ruby/ruby/blob/trunk/lib/net/http.rb
Line 644 is where the connection is opened
s = timeout(#open_timeout) { TCPSocket.open(conn_address(), conn_port()) }
The third and fourth arguments to TCPSocket.open are local_address and local_port, and since they're not specified, it's not possible. Looks like you'll have to go with curb.

Of course you can. I did as below:
# remote_host can be IP or hostname
uri = URI.parse( "http://" + remote_host )
http = Net::HTTP.new( uri.host, uri.port )
request = Net::HTTP::Get.new(uri.request_uri)
request.initialize_http_header( { "Host" => domain })
response = http.request( request )

Related

Ruby 1.8 hangs making http request

I have the next configuration:
Net::HTTP.ssl_context_accessor 'ssl_version'
#http = Net::HTTP.new(#url.host, 443)
#http.ssl_version = :SSLv2
#http.use_ssl = true
#http.verify_mode = OpenSSL::SSL::VERIFY_NONE
#http.set_debug_output $stderr
#http.open_timeout = 10
#http.read_timeout = 10
And then I use the #http object to make a request_get this way:
path = "/login.cgi?username=#{#url.user}&password=#{#url.password}"
debug("Making request #{#http.address}")
response = #http.request_get(path)
debug("#{response.body}")
#cookie = response.get_fields('set-cookie').split('; ')[0]
Puppet.debug('Cookie got!')
The server is supposed to return me a cookie, but the only output I get from the debug is
Debug: Making request server.com
opening connection to server.com...
opened
And it hangs there forever (not even raising timeout).
I'm very new to ruby, and this code has been retrieved from other stackoverflow questions, and was suppose to work.
I've been searching for google, but haven't found anything similar, any idea?
Changing the SSL version to SSLv3 and the request_get method by post solved the problem.

ruby and net/http request without content-type

I'm trying to make a call to a Tika server using Net::HTTP::Put. The issue is that the call always passes the Content-Type, which keeps Tika from running the detectors (which I want) and then chokes due to the default Content-Type of application/x-www-form-urlencoded. Tika docs suggest to not use that.
So, I have the following:
require 'net/http'
port = 9998
host = "localhost"
path = "/meta"
req = Net::HTTP::Put.new(path)
req.body_stream = File.open(file_name)
req['Transfer-Encoding'] = 'chunked'
req['Accept'] = 'application/json'
response = Net::HTTP.new(host, port).start { |http|
http.request(req)
}
I tried adding req.delete('content-type') and setting initheaders = {} in various ways, but the default content-type keeps getting sent.
Any insights would be greatly appreciated, since I would rather avoid having to make multiple curl calls ... is there any way to suppress the sending of that default header?
If you set req['Content-Type'] = nil then Net::HTTP will set it to the default of 'application/x-www-form-urlencoded', but if you set it to a blank string Net::HTTP leaves it alone:
req['Content-Type'] = ''
Tika should see that as an invalid type and enable the detectors.
It seems that Tika will run the detectors if the Content-Type is application/octet-stream. Adding
req.content_type = "application/octet-stream"
is now allowing me to get results.

Getting assignees for a set of tasks

I'm developing a tool to assign a bunch of tasks to a guy according to some criteria.
I fetch tasks for a given tag.
I only assign a task to a guy if the task has no assignee.
My problem comes with the last statement. Fetching a list of tasks do not provide enough information. Going through the documentation, I remember I can format the response with the fields I need using opt_fields but I don't succeed to implement it.
I have this piece of code:
# set up HTTPS connection
uri = URI.parse("https://app.asana.com/api/1.0/tags/8232053093879/tasks?opt_fields=name,assignee")
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_PEER
# set up the request
header = { "Content-Type" => "application/json" }
request = Net::HTTP::Get.new(uri.path, header)
request.basic_auth(AppConfig[:api_key], '')
# issue the request
response = http.start { |h| h.request(request) }
# output
body = JSON.parse(response.body)
And it keeps responding with:
{"id"=>8232053093904, "name"=>"Implement open VPN"}
{"id"=>8232053093899, "name"=>"Implement a #emi tool for random task affectation."}
{"id"=>8232053093893, "name"=>"List possibilities for internal server hosting ?"}
{"id"=>8232053093890, "name"=>"Create a server FAQ (how to access, how to restart an app, how to set up a new server)"}
{"id"=>8232053093883, "name"=>"Help alban debug munin configuration (server monitoring tool)"}
{"id"=>8232053093876, "name"=>"Think how to improve nanoc deployment"}
While, using curl:
curl -u 8NYknPS.aMxj55LsWwwujpZgNqQ078xf: "https://app.asana.com/api/1.0/tags/8232053093879/tasks?opt_fields=name,assignee"
I get:
{"data":[
{"id":8232053093904,"name":"Implement open VPN","assignee":null},
{"id":8232053093899,"name":"Implement a #emi tool for random task affectation.","assignee":null},
{"id":8232053093893,"name":"List possibilities for internal server hosting ?","assignee":{"id":1069528343983}},
{"id":8232053093890,"name":"Create a server FAQ (how to access, how to restart an app, how to set up a new server)","assignee":null},
{"id":8232053093883,"name":"Help alban debug munin configuration (server monitoring tool)","assignee":{"id":1069528343983}},
{"id":8232053093876,"name":"Think how to improve nanoc deployment","assignee":{"id":753180655981}}
]}
What am I missing?
You may need to give the nested fields, like ?opt_fields=name,assignee,assignee.id - it's a bit clunky, unfortunately. But if you just want the whole assignee, you can use ?opt_expand=assignee.
Hope that helps!

Connect to Microsoft Push Notification Service for Windows Phone 8 from Ruby

We are developing a WP8 app that requires push notifications.
To test it we have run the push notification POST request with CURL command line, making sure that it actually connects, authenticates with the client SSL certificate and sends the correct data. We know for a fact that this work as we are receiving pushes to the devices.
This is the CURL command we have been using for testing purposes:
curl --cert client_cert.pem -v -H "Content-Type:text/xml" -H "X-WindowsPhone-Target:Toast" -H "X-NotificationClass:2" -X POST -d "<?xml version='1.0' encoding='utf-8'?><wp:Notification xmlns:wp='WPNotification'><wp:Toast><wp:Text1>My title</wp:Text1><wp:Text2>My subtitle</wp:Text2></wp:Toast></wp:Notification>" https://db3.notify.live.net/unthrottledthirdparty/01.00/AAF9MBULkDV0Tpyj24I3bzE3AgAAAAADCQAAAAQUZm52OkE1OUZCRDkzM0MyREY1RkE
Of course our SSL cert is needed to actually use the URL, but I was hoping someone else has done this and can see what we are doing wrong.
Now, our problem is that we need to make this work with Ruby instead, something we have been unable to get to work so far.
We have tried using HTTParty with no luck, and also net/http directly without any luck.
Here is a very simple HTTParty test script I have used to test with:
require "httparty"
payload = "<?xml version='1.0' encoding='utf-8'?><wp:Notification xmlns:wp='WPNotification'><wp:Toast><wp:Text1>My title</wp:Text1><wp:Text2>My subtitle</wp:Text2></wp:Toast></wp:Notification>"
uri = "https://db3.notify.live.net/unthrottledthirdparty/01.00/AAF9MBULkDV0Tpyj24I3bzE3AgAAAAADCQAAAAQUZm52OkE1OUZCRDkzM0MyREY1RkE"
opts = {
body: payload,
headers: {
"Content-Type" => "text/xml",
"X-WindowsPhone-Target" => "Toast",
"X-NotificationClass" => "2"
},
debug_output: $stderr,
pem: File.read("/Users/kenny/Desktop/client_cert.pem"),
ca_file: File.read('/usr/local/opt/curl-ca-bundle/share/ca-bundle.crt')
}
resp = HTTParty.post uri, opts
puts resp.code
This seems to connect with SSL properly, but then the MS IIS server returns 403 to us for some reason we don't get.
Here is essentially the same thing I've tried using net/http:
require "net/http"
url = URI.parse "https://db3.notify.live.net/unthrottledthirdparty/01.00/AAF9MBULkDV0Tpyj24I3bzE3AgAAAAADCQAAAAQUZm52OkE1OUZCRDkzM0MyREY1RkE"
payload = "<?xml version='1.0' encoding='utf-8'?><wp:Notification xmlns:wp='WPNotification'><wp:Toast><wp:Text1>My title</wp:Text1><wp:Text2>My subtitle</wp:Text2></wp:Toast></wp:Notification>"
pem_path = "./client_cert.pem"
cert = File.read pem_path
http = Net::HTTP.new url.host, url.port
http.use_ssl = true
http.cert = OpenSSL::X509::Certificate.new cert
http.key = OpenSSL::PKey::RSA.new cert
http.ca_path = '/etc/ssl/certs' if File.exists?('/etc/ssl/certs') # Ubuntu
http.ca_file = '/usr/local/opt/curl-ca-bundle/share/ca-bundle.crt' if File.exists?('/usr/local/opt/curl-ca-bundle/share/ca-bundle.crt') # Mac OS X
http.verify_mode = OpenSSL::SSL::VERIFY_PEER
r = Net::HTTP::Post.new url.path
r.body = payload
r.content_type = "text/xml"
r["X-WindowsPhone-Target"] = "toast"
r["X-NotificationClass"] = "2"
http.start do
resp = http.request r
puts resp.code, resp.body
end
Like the HTTParty version, this also returns 403..
I'm starting to get the feeling that this won't actually work with net/http, but I've also seen a few examples of code claiming to work, but I can't see any difference compared to what we have tested with here.
Does anyone know how to fix this? Is it possible? Should I use libcurl instead perhaps? Or even do a system call to curl? (I may have to do the last one as an interim solution if we can't get this to work soon).
Any input is greatly appreciated!
Thanks,
Kenny
Try using some tool like http://mitmproxy.org to compare requests from your code and curl.
For example curl in addition to specified headers does send User-Agent and Accept-headers, microsoft servers may be checking for these for some reason.
If this does not help - then it's ssl-related

Making a URL in a string usable by Ruby's Net::HTTP

Ruby's Net:HTTP needs to be given a full URL in order for it to connect to the server and get the file properly. By "full URL" I mean a URL including the http:// part and the trailing slash if it needs it. For instance, Net:HTTP won't connect to a URL looking like this: example.com, but will connect just fine to http://example.com/. Is there any way to make sure a URL is a full URL, and add the required parts if it isn't?
EDIT: Here is the code I am using:
parsed_url = URI.parse(url)
req = Net::HTTP::Get.new(parsed_url.path)
res = Net::HTTP.start(parsed_url.host, parsed_url.port) {|http|
http.request(req)
}
If this is only doing what the sample code shows, Open-URI would be an easier approach.
require 'open-uri'
res = open(url).read
This would do a simple check for http/https:
if !(url =~ /^https?:/i)
url = "http://" + url
end
This could be a more general one to handle multiple protocols (ftp, etc.)
if !(url =~ /^\w:/i)
url = "http://" + url
end
In order to make sure parsed_url.path gives you a proper value (it should be / when no specific path was provided), you could do something like this:
req = Net::HTTP::Get.new(parsed_url.path.empty? ? '/' : parsed_url.path)

Resources