Net::HTTP doesn't start downloading file after post request - ruby

I am trying to make POST request with Net::HTTP to specific website, and start downloading file after response is returned.
I tested this first with wget, and it worked fine.I also tried with httpi and unirest gems and it went fine . However, because Net::HTTP is low-level API and I need more control(for example, tracking downloading progress), I'll stick with it.
The problem is, downloading just won't start.So here's the code:
require 'net/http'
require 'tempfile'
uri = URI.parse("http://mywebsite.com")
req = Net::HTTP::Post.new(uri.path)
req.set_form_data({'filed' => 'data'})
response = Net::HTTP.start(uri.hostname, uri.port) {|http|
http.request(req) do |response|
temp_file = Tempfile.new("new_file")
response.read_body do |chunk|
temp_file << chunk
end
end
}
I figured that the problem must be in response, what I get is not Net::HTTPOK
but HTTPSeeOther and here's more info about it
#<Net::HTTPSeeOther:0x91f2670>
{"server"=>["nginx"], "date"=>["Sat, 21 Dec 2013 22:32:14 GMT"],
"content-type"=>["text/html"], "transfer-encoding"=>["chunked"],
"connection"=>["keep-alive"], "pragma"=>["no-cache"],
"cache-control"=>["no-cache, must-revalidate"],
"expires"=>["Sat, 26 Jul 1997 05:00:00 GMT"],
"location"=>["http://mywebsite.com/path/to/file.rar"]}
The other 2 gems I mentioned knew how to start download automatically and I don't know what to do with HTTPSeeOther .

It appears that you're getting a redirect response from the server, which wget, curl, and high-level libraries will follow automatically. With Net::HTTP, you'll need to start a new GET request to that location.

Related

Html-parsing with Oga and httpclient in ruby

I'm trying to download a page with httpclient and parse it useing oga (https://github.com/YorickPeterse/oga)
My program looks like this:
require 'httpclient'
require 'oga'
url = 'http://stackoverflow.com/questions/1496096/is-there-a-limit-to-the-length-of-html-attributes'
c = HTTPClient.new
content = c.get_content(url)
document = Oga.parse_html(content)
I get this error:
LL::ParserError: Unexpected end of input, expected element closing tag instead on line 431
parser_error at /home/binaryplease/.rvm/gems/jruby-1.7.19/gems/oga-0.3.1-java/lib/oga/xml/parser.rb:255
each_token at /home/binaryplease/.rvm/gems/jruby-1.7.19/gems/oga-0.3.1-java/lib/oga/xml/parser.rb:231
parse at org/libll/Driver.java:303
parse at /home/binaryplease/.rvm/gems/jruby-1.7.19/gems/oga-0.3.1-java/lib/oga/xml/parser.rb:262
parse_html at /home/binaryplease/.rvm/gems/jruby-1.7.19/gems/oga-0.3.1-java/lib/oga/oga.rb:25
(root) at test.rb:12
I verified that httpclient is downloading correctly and the file doesnt end there. I also tryed other links, some work but most of them give me this error.
In general smaller pages seem to work just fine
Is there a problem with the library or am I making an error?

SSL Error in net::HTTP.get

I'm trying to use ruby to do a search based on account 1234, but I keep getting an SSL_connect returned=1 errno=0 certificate error. The site is signed, but for some reason Ruby doesn't like it. The certificate is properly signed, so I'm not sure why it's rejecting it.
require 'net/http'
uri = URI("https://secure.domain.com/api/account/1234?api_key"+key)
response = Net::HTTP.get(uri)
puts response.body #this must show the JSON contents
Based on another suggestion I also tried, but received the same error.
Net::HTTP.start(uri.host, uri.port,
:use_ssl => uri.scheme == 'https') do |http|
request = Net::HTTP::Get.new uri
response = http.request request # Net::HTTPResponse object
end
Turns out the error was due to Windows not having support for something like OpenSSL built in. After installing the appropriate binaries onto the system it worked without error.

Send request and response get from server - Need working code for this WSDL file in RUBY

Please help me, I need code for my WSDL file for below wsdl file url using RUBY language parameter can be passed as pcEmployeecode...
Parameter can passed as STRING ie.,pcEmployeeCode="02385"
url ="http://online.mccolls.com.au:8080/wsa/wsa1/wsdl?targetURI=urn:OHWebServiceFMGT"
Please Refer this WSDL file and post me if you have sucess fully got the response result...
Any idea's....
I used like this, Still error hitting.
require 'rubygems'
#require 'soap4r'
require 'http-access2'
require 'soap/rpc/driver'
require 'soap/rpc/driver'
require 'net/http'
Net::HTTP.version_1_2
url =
"http://online.mccolls.com.au:8080/wsa/wsa1/wsdl?targetURI=urn:OHWebServiceFMGT"
soap = SOAP::RPC::Driver.new(url,"urn:OHWebServiceFMGT")
soap.wiredump_dev = STDERR
soap.options["protocol.http.ssl_config.verify_mode"] =
OpenSSL::SSL::VERIFY_NONE
soap.add_method('getEmployeeFmgtDetails','pcEmployeeCode')
puts soap.getEmployeeFmgtDetails('02385')

sslv3 alert illegal parameter using Net::HTTP in Ruby on Linux

I am attempting to use a remote system for user authentication. This chunk of code gets a response when I run it on MacOSX, but fails on my machine:
def create
uri = URI.parse('https://ourclient.example.com/')
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
request = Net::HTTP::Post.new('/login.jsp')
request.set_form_data({'login_name' => params[:login], 'password' => params[:password]})
response = http.request(request)
puts "Response BODY: #{response.body.inspect}"
end
Turning verify off gets rid of a warning on the Macs. On my machine, the http.request raises this exception:
OpenSSL::SSL::SSLError (SSL_connect returned=1 errno=0 state=SSLv2/v3 read server hello A: sslv3 alert illegal parameter):
app/controllers/sessions_controller.rb:16:in `create'
I get the same behavior using IRB without Rails. I did a clean install of Fedora 14 yesterday, installed the required development tools and libraries. I'm using Ruby 1.9.2-p180, and Rails 3.0.4. I thought I might have had my libraries misconfigured (I had Fedora 12 that had been upgraded a few times), but this is now a new install.
The remote system is probably Microsoft's IIS, but I'm not certain of that. Perhaps I can use an older SSL protocol, but my Google-fu can't find the incantation.
I would appreciate any tips on resolving this issue. Thanks,
Chris
how was created the ssl cert on the server? self-signed or ca-signed?
which is the error you receive if you remove the VERIFY_NONE?
use Mechanize gem for it
gem install mechanize
and try
require 'mechanize'
page = Mechanize.new{|a| a.ssl_version, a.verify_mode = 'SSLv3', OpenSSL::SSL::VERIFY_NONE}.get "**YOUR HTTPS LINK HERE**"

How do I get an HTTPS request with SSL client cert to work with Ruby EventMachine?

I am trying to access an HTTPS web service that uses SSL cert authentication using Ruby EventMachine but I am not getting it to work.
I have written the following simple code block to test it end-to-end:
require 'rubygems'
require 'em-http'
EventMachine.run do
url = 'https://foobar.com/'
ssl_opts = {:private_key_file => '/tmp/private.key',
:cert_chain_file => '/tmp/ca.pem',
:verify_peer => false}
http = EventMachine::HttpRequest.new(url).get :ssl => ssl_opts
http.callback do
p http.response_header.status
p http.response_header
p http.response
EventMachine.stop
end
http.errback do
EventMachine.stop
fail "Request failed"
end
end
Running the above outputs <SSL_incomp> followed by the raised RuntimeError message. I have tried running with :verify_peer set to both true and false and it gives me the same error. Running EventMachine::HttpRequest#get without the :ssl option does the same.
I have also tried sending the request to GMail (https://mail.google.com) without the :ssl option (i.e. plain HTTPS without cert) and that works, outputting status code 200, the headers and the body.
I have tried doing the same request to the web service with curl and that works:
curl --silent --cert /tmp/private.key --cacert /tmp/ca.pem https://foobar.com/
I am thinking that I am either using the em-http-request gem or EventMachine incorrectly or that the SSL files are in a format that works with curl but not EventMachine.
I someone knows how to solve the example above or provide a similar example using EventMachine directly would be much appreciated!
The file passed to curl's --cert contains both the cert and the key (unless you pass in a --key separately). Just use /tmp/private.key as the argument to both :private_key_file and :cert_chain_file
See http://github.com/eventmachine/eventmachine/issues/#issue/115 for more details about the issue and a patch that exposes the underlying error (instead of just printing out SSL_incomp).

Resources