Download multiple XML files - ruby

I'm writing an application to parse XML. I have to obtain data from one XML file, and then in a loop I have to open another XML file.
The code looks like this:
$doc = Nokogiri::XML(open('myxmladress'))
$doc.xpath('//job').each do |job|
if job.xpath('name').text.include?('joe')
$doc2 = Nokogiri::XML(open('myxmladress_for_joe'))
end
end
I believe that I cannot have multiple HTTP connections open.
Can I simply download the whole file instead of using
$doc Nokogiri::XML(open('myxmladress'))
or is there any way to close the Nokogiri HTTP connection?
What is more I'm downloading it by https.
My error:
in `open_http': 500 Server Error (OpenURI::HTTPError)
from /home/nagios/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/open-uri.rb:737:in `buffer_open'
from /home/nagios/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/open-uri.rb:212:in `block in open_loop'
from /home/nagios/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/open-uri.rb:210:in `catch'
from /home/nagios/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/open-uri.rb:210:in `open_loop'
from /home/nagios/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/open-uri.rb:151:in `open_uri'
from /home/nagios/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/open-uri.rb:717:in `open'
from /home/nagios/.rvm/rubies/ruby-2.3.0/lib/ruby/2.3.0/open-uri.rb:35:in `open'
from jenkins_auth.rb:97:in `block (2 levels) in combine_partial_results'
from /home/nagios/.rvm/gems/ruby-2.3.0/gems/nokogiri-1.6.7.2/lib/nokogiri/xml/node_set.rb:187:in `block in each'
from /home/nagios/.rvm/gems/ruby-2.3.0/gems/nokogiri-1.6.7.2/lib/nokogiri/xml/node_set.rb:186:in `upto'
from /home/nagios/.rvm/gems/ruby-2.3.0/gems/nokogiri-1.6.7.2/lib/nokogiri/xml/node_set.rb:186:in `each'
from jenkins_auth.rb:89:in `block in combine_partial_results'
from /home/nagios/.rvm/gems/ruby-2.3.0/gems/nokogiri-1.6.7.2/lib/nokogiri/xml/node_set.rb:187:in `block in each'
from /home/nagios/.rvm/gems/ruby-2.3.0/gems/nokogiri-1.6.7.2/lib/nokogiri/xml/node_set.rb:186:in `upto'
from /home/nagios/.rvm/gems/ruby-2.3.0/gems/nokogiri-1.6.7.2/lib/nokogiri/xml/node_set.rb:186:in `each'
from jenkins_auth.rb:86:in `combine_partial_results'
from jenkins_auth.rb:130:in `get_tests_for_job'
from jenkins_auth.rb:137:in `<main>'

You are using OpenUri to fetch a document from a URL. I am pretty sure that this doesn't leave any open connection, but reads the document into an IO like object, kind of like a file.
Your problem seems to be that the server has had an internal error.

Related

connection closed by server error when getting accounts using coinbase-exchange websocket

I'm using the ruby coinbase-exchange gem, and I'm executing the code by copying and pasting it into irb.
This only happens when I'm running it as a websocket – when I just execute the code on its own, without the socket, it works fine.
When the following code executes,
accounts = Array.new(2, Array.new)
rest_api.accounts do |resp|
accounts = resp
end
this error is produced:
Coinbase::Exchange::APIError: GET https://api.exchange.coinbase.com/accounts?: connection closed by server
from /usr/local/lib/ruby/gems/2.3.0/gems/coinbase-exchange-0.1.2/lib/coinbase/exchange/adapters/em_http.rb:59:in `block in http_verb'
from /usr/local/lib/ruby/gems/2.3.0/gems/eventmachine-1.2.0.1/lib/em/deferrable.rb:158:in `set_deferred_status'
from /usr/local/lib/ruby/gems/2.3.0/gems/eventmachine-1.2.0.1/lib/em/deferrable.rb:198:in `fail'
from /usr/local/lib/ruby/gems/2.3.0/gems/em-http-request-1.1.5/lib/em-http/client.rb:123:in `on_error'
from /usr/local/lib/ruby/gems/2.3.0/gems/em-http-request-1.1.5/lib/em-http/client.rb:117:in `unbind'
from /usr/local/lib/ruby/gems/2.3.0/gems/em-http-request-1.1.5/lib/em-http/http_connection.rb:205:in `block in unbind'
from /usr/local/lib/ruby/gems/2.3.0/gems/em-http-request-1.1.5/lib/em-http/http_connection.rb:205:in `map'
from /usr/local/lib/ruby/gems/2.3.0/gems/em-http-request-1.1.5/lib/em-http/http_connection.rb:205:in `unbind'
from /usr/local/lib/ruby/gems/2.3.0/gems/em-http-request-1.1.5/lib/em-http/http_connection.rb:31:in `unbind'
from /usr/local/lib/ruby/gems/2.3.0/gems/eventmachine-1.2.0.1/lib/eventmachine.rb:1481:in `event_callback'
from /usr/local/lib/ruby/gems/2.3.0/gems/eventmachine-1.2.0.1/lib/eventmachine.rb:202:in `release_machine'
from /usr/local/lib/ruby/gems/2.3.0/gems/eventmachine-1.2.0.1/lib/eventmachine.rb:202:in `ensure in run'
from /usr/local/lib/ruby/gems/2.3.0/gems/eventmachine-1.2.0.1/lib/eventmachine.rb:205:in `run'
from (irb):79
from /usr/local/bin/irb:11:in `<main>'
What's the problem here? How do I get around it?

Openshift rhc port forward exception

I am trying to open ports so i access my database remotely from my pc
I am running on Ruby200-x64 and when i try to access port forward
rhc port-forward -a appname
i get the following response/error:
Checking available ports ... C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/net-ssh-2.9.2/lib/net/ssh/authentication/pageant.rb:176:in `malloc': bignum too big to convert into `long' (RangeError)
from C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/net-ssh-2.9.2/lib/net/ssh/authentication/pageant.rb:176:in `malloc_ptr'
from C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/net-ssh-2.9.2/lib/net/ssh/authentication/pageant.rb:240:in `get_token_information'
from C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/net-ssh-2.9.2/lib/net/ssh/authentication/pageant.rb:213:in `get_current_user'
from C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/net-ssh-2.9.2/lib/net/ssh/authentication/pageant.rb:189:in `get_security_attributes_for_user'
from C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/net-ssh-2.9.2/lib/net/ssh/authentication/pageant.rb:332:in `send_query'
from C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/net-ssh-2.9.2/lib/net/ssh/authentication/pageant.rb:309:in `send'
from C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/net-ssh-2.9.2/lib/net/ssh/authentication/agent/socket.rb:148:in `send_packet'
from C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/net-ssh-2.9.2/lib/net/ssh/authentication/agent/socket.rb:165:in `send_and_wait'
from C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/net-ssh-2.9.2/lib/net/ssh/authentication/agent/socket.rb:76:in `negotiate!'
from C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/net-ssh-2.9.2/lib/net/ssh/authentication/agent/socket.rb:48:in `connect'
from C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/net-ssh-2.9.2/lib/net/ssh/authentication/key_manager.rb:179:in `agent'
from C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/net-ssh-2.9.2/lib/net/ssh/authentication/key_manager.rb:103:in `each_identity'
from C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/net-ssh-2.9.2/lib/net/ssh/authentication/methods/publickey.rb:19:in `authenticate'
from C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/net-ssh-2.9.2/lib/net/ssh/authentication/session.rb:79:in `block in authenticate'
from C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/net-ssh-2.9.2/lib/net/ssh/authentication/session.rb:66:in `each'
from C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/net-ssh-2.9.2/lib/net/ssh/authentication/session.rb:66:in `authenticate'
from C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/net-ssh-2.9.2/lib/net/ssh.rb:211:in `start'
from C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/rhc-1.38.4/lib/rhc/commands/port_forward.rb:86:in `run'
from C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/rhc-1.38.4/lib/rhc/commands.rb:294:in `execute'
from C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/rhc-1.38.4/lib/rhc/commands.rb:285:in `block (3 levels) in to_commander'
from C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/commander-4.2.1/lib/commander/command.rb:180:in `call'
from C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/commander-4.2.1/lib/commander/command.rb:180:in `call'
from C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/commander-4.2.1/lib/commander/command.rb:155:in `run'
from C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/commander-4.2.1/lib/commander/runner.rb:421:in `run_active_command'
from C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/rhc-1.38.4/lib/rhc/command_runner.rb:72:in `run!'
from C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/commander-4.2.1/lib/commander/delegates.rb:8:in `run!'
from C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/rhc-1.38.4/lib/rhc/cli.rb:37:in `start'
from C:/Ruby200-x64/lib/ruby/gems/2.0.0/gems/rhc-1.38.4/bin/rhc:20:in `<top (required)>'
from C:/Ruby200-x64/bin/rhc:23:in `load'
from C:/Ruby200-x64/bin/rhc:23:in `<main>'
It's a bit late but maybe it can help to newcomer.
The solution is, or better say- workaround, to exit the pageant while you port-forward the app. And when done turn it on again if you need to ssh to it.
I found a sort of workaround too, whenever i recieve this error i do the oldest trick in the books, a restart and it works all the times(for now)

Sinatra error - continuation called across threads

I am completely new to Ruby so my question may have quite a simple answer. However, I couldn't find an answer on stackoverflow.
I have the following very simple Sinatra app:
# myapp.rb
require 'rubygems'
require 'sinatra'
require 'json'
range=(199..2000).step(1)
set :port, 8888
get '/hostname' do
content_type :json
return range.next.to_json
end
Sinatra is starting:
ruby testsinatra.rb
== Sinatra/1.0 has taken the stage on 8888 for development with backup from WEBrick
[2014-09-11 08:43:18] INFO WEBrick 1.3.1
[2014-09-11 08:43:18] INFO ruby 1.8.7 (2011-06-30) [x86_64-linux]
[2014-09-11 08:43:18] INFO WEBrick::HTTPServer#start: pid=8215 port=8888
and serving first request:
curl -ks http://localhost:8888/hostname
199
but failing with an error at the second request:
RuntimeError - continuation called across threads:
/usr/lib/ruby/1.8/generator.rb:131:in `call'
/usr/lib/ruby/1.8/generator.rb:131:in `next'
/usr/lib/ruby/1.8/generator.rb:189:in `next'
testsinatra.rb:30:in `GET /hostname'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:863:in `call'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:863:in `route'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:521:in `instance_eval'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:521:in `route_eval'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:500:in `route!'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:497:in `catch'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:497:in `route!'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:476:in `each'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:476:in `route!'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:601:in `dispatch!'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:411:in `call!'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:566:in `instance_eval'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:566:in `invoke'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:566:in `catch'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:566:in `invoke'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:411:in `call!'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:399:in `call'
/usr/lib/ruby/gems/1.8/gems/rack-1.5.2/lib/rack/showexceptions.rb:24:in `call'
/usr/lib/ruby/gems/1.8/gems/rack-1.5.2/lib/rack/methodoverride.rb:21:in `call'
/usr/lib/ruby/gems/1.8/gems/rack-1.5.2/lib/rack/commonlogger.rb:33:in `call'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:979:in `call'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:1003:in `synchronize'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:1003:in `synchronize'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:979:in `call'
/usr/lib/ruby/gems/1.8/gems/rack-1.5.2/lib/rack/handler/webrick.rb:60:in `service'
/usr/lib/ruby/1.8/webrick/httpserver.rb:104:in `service'
/usr/lib/ruby/1.8/webrick/httpserver.rb:65:in `run'
/usr/lib/ruby/1.8/webrick/server.rb:173:in `start_thread'
/usr/lib/ruby/1.8/webrick/server.rb:162:in `start'
/usr/lib/ruby/1.8/webrick/server.rb:162:in `start_thread'
/usr/lib/ruby/1.8/webrick/server.rb:95:in `start'
/usr/lib/ruby/1.8/webrick/server.rb:92:in `each'
/usr/lib/ruby/1.8/webrick/server.rb:92:in `start'
/usr/lib/ruby/1.8/webrick/server.rb:23:in `start'
/usr/lib/ruby/1.8/webrick/server.rb:82:in `start'
/usr/lib/ruby/gems/1.8/gems/rack-1.5.2/lib/rack/handler/webrick.rb:14:in `run'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:946:in `run!'
/usr/lib/ruby/gems/1.8/gems/sinatra-1.0/lib/sinatra/main.rb:25
testsinatra.rb:27
Clearly, I'm missing something basic. Please advice.
Testing the code gives:
FiberError at /hostname
fiber called across threads
You can find a related question here: Sharing an enumerator across threads . It seems that the Fiber code stores the ID of the first thread that made an access to the object, and immediately fails when another thread tries to do so. You apparently just can't share enumerators between threads, and must resort to different means.
Please also note that global variables might be accessed simultaneously by different threads and you should always use thread-safe objects or explicit locking.

HTTP request error: name or service not known

I am trying to make HTTP requests via Ruby. When running the following code:
require "net/http"
require "uri"
uri = URI.parse("http://google.com/")
# Will print response.body
Net::HTTP.get_print(uri)
I get the following error:
SocketError: initialize: name or service not known
from org/jruby/ext/socket/RubyTCPSocket.java:129:in `initialize'
from org/jruby/RubyIO.java:1179:in `open'
from c:/jruby-1.7.3/lib/ruby/1.9/net/http.rb:762:in `connect'
from org/jruby/ext/timeout/Timeout.java:105:in `timeout'
from c:/jruby-1.7.3/lib/ruby/1.9/net/http.rb:762:in `connect'
from c:/jruby-1.7.3/lib/ruby/1.9/net/http.rb:755:in `do_start'
from c:/jruby-1.7.3/lib/ruby/1.9/net/http.rb:744:in `start'
from c:/jruby-1.7.3/lib/ruby/1.9/net/http.rb:454:in `get_response'
from c:/jruby-1.7.3/lib/ruby/1.9/net/http.rb:412:in `get_print'
from (irb):29:in `evaluate'
from org/jruby/RubyKernel.java:1066:in `eval'
from org/jruby/RubyKernel.java:1409:in `loop'
from org/jruby/RubyKernel.java:1174:in `catch'
from org/jruby/RubyKernel.java:1174:in `catch'
from c:/jruby-1.7.3/bin/irb:13:in `(root)'
This is probably proxy related, but I am not sure. How can I fix this problem?
Ruby will automatically use the HTTP proxy as specified in your environment variables.
EXPORT http_proxy=http://foo.bar:8080/
ruby your_http_script.rb
Same thing happened to me. No proxy was in use.
The reason was that I was not running as root and /etc/resolv.conf didn't have read permissions to anyone other than root. Adding read permissions solved the host resolving problem.

With Nokogiri i am getting error "initialize': getaddrinfo: No such host is known. (SocketError)"

I worte the below code, just to start the Nokogiri to mee the requirement and when I ran it for testing got the error:
D:\WIPData\Ruby\Scripts>Nokogiri.rb
C:/Ruby193/lib/ruby/1.9.1/net/http.rb:762:in `initialize': getaddrinfo: No such
host is known. (SocketError)
from C:/Ruby193/lib/ruby/1.9.1/net/http.rb:762:in `open'
from C:/Ruby193/lib/ruby/1.9.1/net/http.rb:762:in `block in connect'
from C:/Ruby193/lib/ruby/1.9.1/timeout.rb:54:in `timeout'
from C:/Ruby193/lib/ruby/1.9.1/timeout.rb:99:in `timeout'
from C:/Ruby193/lib/ruby/1.9.1/net/http.rb:762:in `connect'
from C:/Ruby193/lib/ruby/1.9.1/net/http.rb:755:in `do_start'
from C:/Ruby193/lib/ruby/1.9.1/net/http.rb:744:in `start'
from C:/Ruby193/lib/ruby/1.9.1/open-uri.rb:306:in `open_http'
from C:/Ruby193/lib/ruby/1.9.1/open-uri.rb:775:in `buffer_open'
from C:/Ruby193/lib/ruby/1.9.1/open-uri.rb:203:in `block in open_loop'
from C:/Ruby193/lib/ruby/1.9.1/open-uri.rb:201:in `catch'
from C:/Ruby193/lib/ruby/1.9.1/open-uri.rb:201:in `open_loop'
from C:/Ruby193/lib/ruby/1.9.1/open-uri.rb:146:in `open_uri'
from C:/Ruby193/lib/ruby/1.9.1/open-uri.rb:677:in `open'
from C:/Ruby193/lib/ruby/1.9.1/open-uri.rb:33:in `open'
from D:/WIPData/Ruby/Scripts/Nokogiri.rb:6:in `<main>'
D:\WIPData\Ruby\Scripts>
CODE
require 'nokogiri'
require 'open-uri'
# Get a Nokogiri::HTML::Document for the page we’re interested in...
doc = Nokogiri::HTML(open('http://www.google.co.in'))
Why so and how to fix the same,please advice me.
It appears you need to configure a proxy. Find out what the proxy URL/Port is for your organization (and whether there needs to be authentication). You may be able to view this information from your browser configuration. In order to use it with your Ruby code, you need to set the HTTP_PROXY environment variable.
You can set it in Ruby code:
ENV['HTTP_PROXY'] = 'http://hostname:port'
or if you need authentication:
ENV['HTTP_PROXY'] = 'http://username:password#hostname:port'
A more permanent solution is to set HTTP_PROXY in your system environment variables.

Resources