How to parse XML with Mechanize and XMLSimple in ruby? - ruby

I'm trying to fetch a remote XML file with Mechanize to get icecast status information. But I'm having problems to pass the XML file from Mechanize::File format to string or some XML format which XMLSimple can work with.
The XML document looks like that:
<icestats>
<admin>donschoe#stackoverflow.com</admin>
<!-- ... -->
</icestats>
My code looks like that right now:
require 'mechanize'
require 'xmlsimple'
server = 'example.net'
port = 8000
user = 'stackoverflow'
password = 'hackme'
agent = Mechanize.new
agent.user_agent_alias = 'Linux Firefox'
agent.add_auth("http://#{server}:#{port}/admin/status.xml", user, password)
agent.get("http://#{server}:#{port}/admin/status.xml")
xml = agent.current_page
status = XmlSimple.xml_in(xml)
puts status['admin']
This should output: donschoe#stackoverflow.com
But it throws:
/home/user/.gem/ruby/1.9.1/gems/xml-simple-1.1.2/lib/xmlsimple.rb:191:in 'xml_in': Could not parse object of type: <Mechanize::File>. (ArgumentError)
Now, I understand the XMLSimple needs a string and therefore I tried to convert the Mechanize::File format to string, replacing the second last line with:
status = XmlSimple.xml_in(xml.to_s)
But this throws an even more weird exception:
/usr/lib64/ruby/1.9.1/rexml/parsers/baseparser.rb:406:in `block in pull_event': Undefined prefix Mechanize: found (REXML::UndefinedNamespaceException)
from /usr/lib64/ruby/1.9.1/set.rb:222:in `block in each'
from /usr/lib64/ruby/1.9.1/set.rb:222:in `each_key'
from /usr/lib64/ruby/1.9.1/set.rb:222:in `each'
from /usr/lib64/ruby/1.9.1/rexml/parsers/baseparser.rb:404:in `pull_event'
from /usr/lib64/ruby/1.9.1/rexml/parsers/baseparser.rb:183:in `pull'
from /usr/lib64/ruby/1.9.1/rexml/parsers/treeparser.rb:22:in `parse'
from /usr/lib64/ruby/1.9.1/rexml/document.rb:231:in `build'
from /usr/lib64/ruby/1.9.1/rexml/document.rb:43:in `initialize'
from /home/user/.gem/ruby/1.9.1/gems/xml-simple-1.1.2/lib/xmlsimple.rb:965:in `new'
from /home/user/.gem/ruby/1.9.1/gems/xml-simple-1.1.2/lib/xmlsimple.rb:965:in `parse'
from /home/user/.gem/ruby/1.9.1/gems/xml-simple-1.1.2/lib/xmlsimple.rb:164:in `xml_in'
from /home/user/.gem/ruby/1.9.1/gems/xml-simple-1.1.2/lib/xmlsimple.rb:203:in `xml_in'
from debugging.rb:16:in `<main>'
What's wrong with my approach? When I download the XML file and use the local XML file the code above works as desired.
I'm especially looking for solutions with Mechanize rather than Nokogiri.

Try changing:
xml = agent.current_page
to:
xml = agent.current_page.body

Related

How to use Icescrum with Dashing (widget)?

I am trying to create a widget on Dashing that will extract data from Icescrum and update a specific widget.
The thing is I found info on how to program jira, so i've been trying to mess around with it to make it work with icescrum, but with no luck.
I tried this one:
require "uri"
require "net/http"
require "json"
SCHEDULER.every '2s', :first_in => 0 do |job|
uri = URI.parse("http://website.ca:8080/icescrum")
http = Net::HTTP.new(uri.host, uri.port)
req = Net::HTTP::Get.new("uri.request_uri")
req.basic_auth username, password
response = http.request(req)
issuesinProgress = JSON.parse(response.body)["total"]
send_event('synergy', value: issuesinProgress)
end
but I got this error:
scheduler caught exception:
undefined local variable or method `username' for main:Object
/home/administrator/dashboard/jobs/sample.rb:27:in `block in <top (required)>'
/var/lib/gems/1.9.1/gems/rufus-scheduler-2.0.24/lib/rufus/sc/jobs.rb:230:in `call'
/var/lib/gems/1.9.1/gems/rufus-scheduler-2.0.24/lib/rufus/sc/jobs.rb:230:in `trigger_block'
/var/lib/gems/1.9.1/gems/rufus-scheduler-2.0.24/lib/rufus/sc/jobs.rb:204:in `block in trigger'
/var/lib/gems/1.9.1/gems/rufus-scheduler-2.0.24/lib/rufus/sc/scheduler.rb:430:in `call'
/var/lib/gems/1.9.1/gems/rufus-scheduler-2.0.24/lib/rufus/sc/scheduler.rb:430:in `block in trigger_job'
Your code has a line:
req.basic_auth username, password
It's expecting two variables, "username" and "password", but neither look to be defined in your code, here. If they aren't defined you'll get an error that looks an awful lot like what you're getting.
So finally I just used a cURL function to get the information. I'll leave it here if anyone else needs it. It also sends the results to an xml file (if you don't want that then just remove the last part)
.rb file:
%x{curl -v -H "Content-Type: application/xml" -X POST --data "#filename.xml" -u username:password http://website.ca:8080/icescrum/ --output result.xml}
Remember to change these dummy names with your links:
filename.xml
username
password
website.ca:8080/icescrum/
result.xml
a complete set up of the widget is also available here :
https://github.com/Flash-Dash/Icescrum-widget

LimeSurvey RPC-JSON API using ruby

I've been trying to test the Limesurvey API and I was trying to make a simple ruby script for this. I tried the following gems:
https://github.com/chriskite/jimson
https://github.com/jjeffus/rpcjson/issues
I followed the examples on the documentation. For rpcjson:
require 'rubygems'
require 'rpcjson'
bc = RPC::JSON::Client.new 'http://localhost/limesurvey/index.php/admin/remotecontrol', 2
bc.get_session_key(username: "xxx", password: "xxx")
But I get the following error:
JSON::ParserError: A JSON text must at least contain two octets!
from C:/Ruby193/lib/ruby/gems/1.9.1/gems/json-1.8.1/lib/json/common.rb:1
55:in initialize'
from C:/Ruby193/lib/ruby/gems/1.9.1/gems/json-1.8.1/lib/json/common.rb:1
55:innew'
from C:/Ruby193/lib/ruby/gems/1.9.1/gems/json-1.8.1/lib/json/common.rb:1
55:in parse'
from C:/Ruby193/lib/ruby/gems/1.9.1/gems/json-1.8.1/lib/json/common.rb:4
68:inJSON'
from C:/Ruby193/lib/ruby/gems/1.9.1/gems/rpcjson-0.0.2/lib/rpcjson.rb:55
:in method_missing'
from (irb):3
from C:/Ruby193/bin/irb:12:in'
Using Jimson:
require 'jimson'
client = Jimson::Client.new('http://localhost/limesurvey/index.php/admin/remotecontrol')
client.get_session_key(username: "xxx", password: "xxx")
I get:
Jimson::Client::Error::InvalidResponse: Invalid or empty response from server.
from C:/Ruby193/lib/ruby/gems/1.9.1/gems/jimson-0.10.0/lib/jimson/client
.rb:75:in process_single_response'
from C:/Ruby193/lib/ruby/gems/1.9.1/gems/jimson-0.10.0/lib/jimson/client
.rb:33:inprocess_call'
from C:/Ruby193/lib/ruby/gems/1.9.1/gems/jimson-0.10.0/lib/jimson/client
.rb:165:in method_missing'
from (irb):4
from C:/Ruby193/bin/irb:12:in'
Has anyone ever tried this? Thanks in advance!

How can I use string interpolation to pass a url into Nokogiri?

So I have a method and when I pass in a straight URL (see code below) the method returns just fine. When I pass in the URL using #{} for the possibility of using different location in Craigslist, it throws the error shown at the bottom. I suppose my question is twofold:
Why doesn't Nokogiri allow me to open this?
Can I change this to accept the URL?
Code:
def get_post_date(listing_url)
# This method takes in a page and returns a date hopefully in a date format
# but right now text
listing = Nokogiri::HTML(open(listing_url)).css("p")
setter = ""
for element in listing
if element.css('time').text!=""&&setter==""
post_time = "poop" # Time.parse(element.css('time').text)
return "poop"
end
end
end
location = "sfbay"
# THIS throws an error
p get_post_date("#{location}.craigslist.org/sfc/vac/4248712420.html")
# THIS works
p get_post_date("sfbay.craigslist.org/sfc/vac/4248712420.html")
Error:
c:\>ruby cljobs.rb C:/Ruby193/lib/ruby/1.9.1/open-uri.rb:35:in
`initialize': No such file or direct ory -
sfbay.craigslist.org/sfc/vac/4248712420.html (Errno::ENOENT)
from C:/Ruby193/lib/ruby/1.9.1/open-uri.rb:35:in `open'
from C:/Ruby193/lib/ruby/1.9.1/open-uri.rb:35:in `open'
from cljobs.rb:7:in `get_post_date'
from cljobs.rb:40:in `'
In order to open a URL you need to require OpenURI. Otherwise nokogiri will try to open a file.
require 'open-uri'
listing = Nokogiri::HTML(open(listing_url))

Ruby Spreadsheet: Bad file descriptor - test.xls (Errno::EBADF)

I have problem with script that makes simple .xls file and writes data to one cell. Here is simple code:
require 'spreadsheet'
class Filter
def filter
#excel = Spreadsheet::Workbook.new
#sheet = #excel.create_worksheet
#sheet[0, 0] = "test"
#excel.write 'test.xls'
end
end
f = Filter.new
f.filter
But it raises error:
C:/Ruby193/lib/ruby/gems/1.9.1/gems/ruby-ole-1.2.11.5/lib/ole/storage/base.rb:62:in
write_nonblock': Bad file descriptor - test.xls (Errno::EBADF)
from C:/Ruby193/lib/ruby/gems/1.9.1/gems/ruby-ole-1.2.11.5/lib/ole/storage/base.rb:62:in
initialize'
from C:/Ruby193/lib/ruby/gems/1.9.1/gems/ruby-ole-1.2.11.5/lib/ole/storage/base.rb:78:in
new'
from C:/Ruby193/lib/ruby/gems/1.9.1/gems/ruby-ole-1.2.11.5/lib/ole/storage/base.rb:78:in
open'
from C:/Ruby193/lib/ruby/gems/1.9.1/gems/spreadsheet-0.7.4/lib/spreadsheet/excel/writer/workbook.rb:4
53:in write_from_scratch'
from C:/Ruby193/lib/ruby/gems/1.9.1/gems/spreadsheet-0.7.4/lib/spreadsheet/excel/writer/workbook.rb:6
31:inwrite_workbook'
from C:/Ruby193/lib/ruby/gems/1.9.1/gems/spreadsheet-0.7.4/lib/spreadsheet/writer.rb:15:in
block in write'
from C:/Ruby193/lib/ruby/gems/1.9.1/gems/spreadsheet-0.7.4/lib/spreadsheet/writer.rb:14:in
open'
from C:/Ruby193/lib/ruby/gems/1.9.1/gems/spreadsheet-0.7.4/lib/spreadsheet/writer.rb:14:in
write'
from C:/Ruby193/lib/ruby/gems/1.9.1/gems/spreadsheet-0.7.4/lib/spreadsheet/workbook.rb:116:in
write'
from filter.rb:10:in `filter'
from filter.rb:15:in `<main>'
because ruby-ole 1.2.11.5 doesn't support windows platform,
more detail: ruby-ole issue
you can use ruby-ole 1.2.11.4 to avoid this problem.
require 'rubygems'
gem 'ruby-ole','1.2.11.4'
require 'spreadsheet'
I've seen these before. First verify that you can write to that file's location.
My guess is either the file is already open in Excel or your antivirus is blocking the 'threat'.

How do I open local image file, encode it, and post via URI? (posting via Tumblr API)

I'm trying to read local image file, properly encode it and post to Tumbrl. According to the Tumblr API I can pass a parameter data which is Array (URL-encoded binary contents) Limit: 5 MB
I've tested my code with http://api.tumblr.com/v2/blog/#{BLOG}/info request. It is working. But I can't post a photo. Here is my code:
require 'oauth'
require 'oauth/consumer'
require 'open-uri'
require 'active_support'
CONSUMER = 'foo'
SECRET = 'foo'
TOKEN = 'foo'
TOKEN_SECRET = 'foo'
BLOG = 'foo'
consumer=OAuth::Consumer.new(CONSUMER, SECRET, {:site=>"http://tumblr.com"})
access_token = OAuth::AccessToken.new(consumer, TOKEN, TOKEN_SECRET)
# Here I tried one of two lines:
# data = Base64.encode64(IO.binread('./resized')) #first try
data = URI::encode(IO.binread('./resized')) #second try
# response = access_token.get "http://api.tumblr.com/v2/blog/#{BLOG}/info?api_key=#{CONSUMER}"
# puts response
response=access_token.post "http://api.tumblr.com/v2/blog/#{BLOG}/post?api_key=#{CONSUMER}&type=photo&data=#{data}&link=http://ya.ru&"
puts response
1st try:
% ruby ./w_oauth.rb
/usr/lib/ruby/1.9.1/uri/common.rb:176:in `split': bad URI(is not URI?): http://api.tumblr.com/v2/blog/foo/post?api_key=foo&type=photo&data=/9j/4AAQSkZJRgABAQEASABIAAD//gA7Q1JFQVRPUjogZ2QtanBlZyB2MS4w (URI::InvalidURIError)
ICh1c2luZyBJSkcgSlBFRyB2NjIpLCBxdWFsaXR5ID0gODAK/9sAQwAGBAUG
BQQGBgUGBwcGCAoQCgoJCQoUDg8MEBcUGBgXFBYWGh0lHxobIxwWFiAsICMm
(!!!long piece of image data skipped!!!)
FI/16HfTbyHPWurqdE+TGH4wx2js5SKQb+6b4bIj3aurqCrEtcXrf/4yf/dS
DLet/wCzEB6sa6uoomxJN2eaQj5mkYuerQj611dQM7Fx/wDLF/8AbXV1dTA/
/9k=
&link=http://ya.ru&
from /usr/lib/ruby/1.9.1/uri/common.rb:211:in `parse'
from /usr/lib/ruby/1.9.1/uri/common.rb:747:in `parse'
from /var/lib/gems/1.9.1/gems/oauth-0.4.6/lib/oauth/tokens/access_token.rb:7:in `request'
from /var/lib/gems/1.9.1/gems/oauth-0.4.6/lib/oauth/tokens/access_token.rb:47:in `post'
from ./w_oauth.rb:23:in `<main>'
2nd try:
% ruby ./w_oauth.rb
/var/lib/gems/1.9.1/gems/oauth-0.4.6/lib/oauth/helper.rb:14:in `force_encoding': can't modify frozen String (RuntimeError)
from /var/lib/gems/1.9.1/gems/oauth-0.4.6/lib/oauth/helper.rb:14:in `rescue in escape'
from /var/lib/gems/1.9.1/gems/oauth-0.4.6/lib/oauth/helper.rb:12:in `escape'
from /var/lib/gems/1.9.1/gems/oauth-0.4.6/lib/oauth/helper.rb:43:in `block (2 levels) in normalize'
from /var/lib/gems/1.9.1/gems/oauth-0.4.6/lib/oauth/helper.rb:42:in `collect'
from /var/lib/gems/1.9.1/gems/oauth-0.4.6/lib/oauth/helper.rb:42:in `block in normalize'
from /var/lib/gems/1.9.1/gems/oauth-0.4.6/lib/oauth/helper.rb:37:in `map'
from /var/lib/gems/1.9.1/gems/oauth-0.4.6/lib/oauth/helper.rb:37:in `normalize'
from /var/lib/gems/1.9.1/gems/oauth-0.4.6/lib/oauth/request_proxy/base.rb:98:in `normalized_parameters'
from /var/lib/gems/1.9.1/gems/oauth-0.4.6/lib/oauth/request_proxy/base.rb:113:in `signature_base_string'
from /var/lib/gems/1.9.1/gems/oauth-0.4.6/lib/oauth/signature/base.rb:77:in `signature_base_string'
from /var/lib/gems/1.9.1/gems/oauth-0.4.6/lib/oauth/signature/hmac/base.rb:12:in `digest'
from /var/lib/gems/1.9.1/gems/oauth-0.4.6/lib/oauth/signature/base.rb:65:in `signature'
from /var/lib/gems/1.9.1/gems/oauth-0.4.6/lib/oauth/signature.rb:23:in `sign'
from /var/lib/gems/1.9.1/gems/oauth-0.4.6/lib/oauth/client/helper.rb:45:in `signature'
from /var/lib/gems/1.9.1/gems/oauth-0.4.6/lib/oauth/client/helper.rb:75:in `header'
from /var/lib/gems/1.9.1/gems/oauth-0.4.6/lib/oauth/client/net_http.rb:91:in `set_oauth_header'
from /var/lib/gems/1.9.1/gems/oauth-0.4.6/lib/oauth/client/net_http.rb:30:in `oauth!'
from /var/lib/gems/1.9.1/gems/oauth-0.4.6/lib/oauth/consumer.rb:224:in `sign!'
from /var/lib/gems/1.9.1/gems/oauth-0.4.6/lib/oauth/consumer.rb:188:in `create_signed_request'
from /var/lib/gems/1.9.1/gems/oauth-0.4.6/lib/oauth/consumer.rb:159:in `request'
from /var/lib/gems/1.9.1/gems/oauth-0.4.6/lib/oauth/tokens/consumer_token.rb:25:in `request'
from /var/lib/gems/1.9.1/gems/oauth-0.4.6/lib/oauth/tokens/access_token.rb:12:in `request'
from /var/lib/gems/1.9.1/gems/oauth-0.4.6/lib/oauth/tokens/access_token.rb:47:in `post'
from ./w_oauth.rb:23:in `<main>'
UPD: ./resized is a proper JPEG file:
% file ./resized
./resized: JPEG image data, JFIF standard 1.01, comment: "CREATOR: gd-jpeg v1.0 (using IJG JPEG v62), quality = 80"
URI encoding is not enough. You also need to encode: , / ? : # & = + $ #.
Try:
URI.escape(IO.binread('./resized'), Regexp.new("[^#{URI::PATTERN::UNRESERVED}]"))

Resources