Sorry for the newbie question but still very new to Ruby and Mongo.
Not 100% sure what I am missing, but how would I get just the value back
from "url" in the following json result?
[{"url":"www.google.com"}]
So I want something like if I do a
puts url_var
it should just display www.google.com
MongoMapper code to get URl from Mongo
redirect_url = Surl.all(:url_key => "#{urlkey}")
myjson = redirect_url.to_json(:only => [:url])
puts redirect_url
gives me
[{"url":"www.google.com"}]
if I do something like
redirect_url.each do |key,value|
puts key
puts value
end
key is still the full json string and value is empty.
Surely I must be missing something really basic here
Okay, I assume Surl is a model of yours, which has at least these fields url_key and url, so to access the url you don't have to convert the result to json and then fetch to key, but you can access it directly.
Surl.all returns an array (check the mongo mapper docs), if you want a single result use Surl.find_by_url_key(urlkey). Now you can access the url directly by using puts redirect_url.url
redirect_url = Surl.find_by_url_key(urlkey)
puts redirect_url.url # => "www.google.com"
Be sure to read up on http://mongomapper.com/documentation/plugins/dynamic-querying.html and http://mongomapper.com/documentation/plugins/querying.html, to understand the results and differences.
Related
In my text file are a list of URLs. Using Mechanize I'm using that list to parse out the title and meta description. However, some of those URL pages don't have a meta description which stops my script with a nil error:
undefined method `[]' for nil:NilClass (NoMethodError)
I've read up and seen solutions if I were using Rails, but for Ruby I've only seen reject and compact as possible solutions to ignore nil values. I added compact at the end of the loop, but that doesn't seem to do anything.
require 'rubygems'
require 'mechanize'
File.readlines('parsethis.txt').each do |line|
page = Mechanize.new.get(line)
title = page.title
metadesc = page.at("head meta[name='description']")[:content]
puts "%s, %s, %s" % [line.chomp, title, metadesc]
end.compact!
It's just a list of urls in a text like this:
http://www.a.com
http://www.b.com
This is what will output in the console for example:
http://www.a.com, Title, This is a description.
If within the list of URLs there is no description or title on that particular page, it throws up the nil error. I don't want it to skip any urls, I want it to go through the whole list.
Here is one way to do it:
Edit( for added requirement to not skip any url's):
metadesc = page.at("head meta[name='description']")
puts "%s, %s, %s" % [line.chomp, title, metadesc ? metadesc[:content] : "N/A"]
This is untested but I'd do something like this:
require 'open-uri'
require 'nokogiri'
page_info = {}
File.foreach('parsethis.txt') { |url|
page = Nokogiri::HTML(open(url))
title = page.title
meta_desc = page.at("head meta[name='description']")
meta_desc_content = meta_desc ? meta_desc[:content] : nil
page_info[url] = {:title => title, :meta_desc => meta_desc_content}
}
page_info.each do |url, info|
puts [
url,
info[:title],
info[:meta_desc]
].join(', ')
end
File.foreach iteratively reads a file, returning each line individually.
page.title could return a nil if a page doesn't have a title; titles are optional in pages.
I break down accessing the meta-description into two steps. Meta tags are optional in HTML so they might not exist, at which point a nil would be returned. Trying to access a content= parameter would result in an exception. I think that's what you're seeing.
Instead, in my code, meta_desc_content is conditionally assigned a value if the meta-description tag was found, or nil.
The code populates the page_info hash with key/value pairs of the URL and its associated title and meta-description. I did it this way because a hash-of-hashes, or possibly an array-of-hashes, is a very convenient structure for all sorts of secondary manipulations, such as returning the information as JSON or inserting into a database.
As a second step the code iterates over that hash, retrieving each key/value pair. It then joins the values into a string and prints them.
There are lots of things in your code that are either wrong, or not how I'd do them:
File.readlines('parsethis.txt').each returns an array which you then have to iterate over. That isn't scalable, nor is it efficient. File.foreach is faster than File.readlines(...).each so get in the habit of using it unless you are absolutely sure you know why you should use readlines.
You use Mechanize for something that Nokogiri and OpenURI can do faster. Mechanize is a great tool if you are working with forms and need to navigate a site, but you're not doing that, so instead you're dragging around additional code-weight that isn't necessary. Don't do that; It leads to slow programs among other things.
page.at("head meta[name='description']")[:content] is an exception in waiting. As I said above, meta-descriptions are not necessarily going to exist in a page. If it doesn't then you're trying to do nil[:content] which will definitely raise an exception. Instead, work your way down to the data you want so you can make sure that the meta-description exists before you try to get at its content.
You can't use compact or compact! the way you were. An each block doesn't return an array, which is the class you need for compact or compact!. You could have used map but the logic would have been messy and puts inside map is rarely used. (Probably shouldn't be used is more likely but that's a different subject.)
How can I access Twitter::Cursor hash values returned by the Twitter API?
I am following the Jumpstartlab Microblogger tutorial for using the Twitter gem via the jumpstart_auth gem.
I am on iteration 4 step 1. I can return a friends object with the following code:
def friends_last_tweets
friends = client.friends
puts friends
end
=> Twitter::Cursor:0x00000104051928
However, the example account, 'client' in this case, has two 'friends' not just one so why does it only return one object? I thought maybe that object is the array or arrays with all of the friends accordingly in hash values within, thus use [] to access, but this returns "undefined method for Twitter::Cursor". I run each on the Twitter::Cursor object and it returns two fixnums:
def friends_last_tweets
friends = client.friends
friends.each { |f| puts f }
end
=> 18908095
108528349
So surely these numbers must represent each 'friend object' within the Twitter::Cursor object me thinks. I need to access the key/value pairs within that object, yet my attempted hash accessing results in undefined method or variable.
In case it's version issue related, I'm using Twitter5.11.0 and Jumpstart_auth 0.6.0.
those answers didn't helped me to get the last message (maybe the API changed in the meantime), that's how I finally did it:
def everyones_last_tweet
puts "\n\n here are the latest tweets of your friends:"
friends = #client.friends.collect { |f| #client.user(f) }
friends.each do |friend|
puts "\n\n#{friend.screen_name} wrote: \n\t #{friend.status.text}"
end
return ""
end
I'm not happy with that return string though
Access the 'friends' object in the same way you accessed the 'followers' object earlier in the tutorial in order to get a list of your followers' screen names.
To get an array of followers' screen names:
screen_names = #client.followers.collect {|f| #client.user(f).screen_name }
To get an array of friends' screen names:
screen_names = #client.friends.collect {|f| #client.user(f).screen_name }
To get the last tweet of a friend, you can use the object_id's you posted above, as:
last_tweet = #client.user(object_id).status.tweet
I hope this helps. I was caught on this issue for a while too.
I am pretty new to working with Ruby, especially with APIs but I've been trying to get the Darksky API to work, but I'm afraid I'm missing something obvious with how I'm using it.
Here is what I have
require 'darksky'
darksky = Darksky::API.new('my api key')
forecast = darksky.forecast('34.0500', '118.2500')
forecast
When I run this from the command line nothing happens. What am I doing wrong here?
Simply using forecast isn't going to do anything. You need to use puts at a minimum:
puts forecast
Or, see if Ruby's object pretty-printer can return something more interesting:
require 'pp'
pp forecast
Digging in further, I think their API doesn't work. Based on their examples, using a valid key and their location samples, plus the locations from their source site Forecast.io, also returns nil.
Using the REST interface directly from Forecast.io's site does return JSON. JSON is very easy to work with in Ruby, so it's a good way to go.
Here's some code to test the API, and Forecast.io's REST interface:
API_KEY = 'xxxxxxxxxxxxxxxxxxx'
LOCATION = %w[37.8267 -122.423]
require 'darksky'
darksky = Darksky::API.new(API_KEY)
forecast = darksky.forecast(*LOCATION)
forecast # => nil
brief_forecast = darksky.brief_forecast(*LOCATION)
brief_forecast # => nil
require 'json'
require 'httparty'
URL = "https://api.forecast.io/forecast/#{ API_KEY }/37.8267,-122.423"
puts URL
# >> https://api.forecast.io/forecast/xxxxxxxxxxxxxxxxxxx/37.8267,-122.423
puts HTTParty.get(URL).body[0, 80]
# >> {"latitude":37.8267,"longitude":-122.423,"timezone":"America/Los_Angeles","offse
Notice that LOCATION is 37.8267,-122.423 in both cases, which is Alcatraz according to the Forecast.io site. Also notice that the body output displayed is a JSON string.
Pass the returned JSON to the Ruby's JSON class like:
JSON[returned_json]
to get it parsed back into a Ruby Hash. Using OpenURI (because it comes with Ruby) instead of HTTParty, and passing it to JSON for parsing looks like:
body = open(URL).read
puts JSON[body]
I am trying to parse a SOAP response using Savon. The response is XML but is being returned as one long string. If I use #to_hash the entire XML object is still a string, now stored in
hash[:response][:return]
which means it is still a huge unusable mess.
My code looks like
response = soapClient.request(:get_sites_user_can_access) do
soap.body = { :sessionid => session[:login_response][:login_return],
:eid => user }
end
rep = response.to_hash
pp rep[:get_sites_user_can_access_response][:get_sites_user_can_access_return]
What step am I missing to get useful information out of the response? Note: Unfortunately I can't post the XML response because of the info it contains, but it looks like an entire XML document stored as a string. It's class is Nori::StringWithAttributes
I was able to get the desired results but parsing the Nori string(?) using this documentation. This seems like a less than ideal method, but I realized the last element is an array of hashes. So it's hash, of hashes, with an array of hashes. Anyway, here is what worked for me. Advice on how to make this less ugly and clunky would be appreciated.
response = soapClient.request(:get_sites_user_can_access) do
soap.body = { :sessionid => session[:login_response][:login_return],
:eid => user }
end
rep = response.to_hash[:get_sites_user_can_access_response][:get_sites_user_can_access_return]
hrep = Nori.parse(rep)
hrep[:list][:item].each { |item| pp item[:site_id] }
Here is my YELP client using signet but once I get response, I'm not able to convert to ruby hash to inspect response element.
require 'rubygems'
require 'json'
require 'net/http'
client = Signet::OAuth1::Client.new(
:client_credential_key =>
'xxxxxxxxxxxxxxxxxxx',
:client_credential_secret =>
'xxxxxxxxxxxxxxxxxxx',
:token_credential_key =>
'xxxxxxxxxxxxxxxxxxx',
:token_credential_secret =>
'xxxxxxxxxxxxxxxxxxx'
)
response = client.fetch_protected_resource(
:uri => 'http://api.yelp.com/v2/search?term=food&location=san+francisco'
)
# The Rack response format is used here
status, headers, body = response
puts body["businesses"]
Error:
`[]': can't convert String into Integer (TypeError)
Body prints fine in nice JSON format but I can do body["businesses"] for instance
JSON.parse(body).inspect is also not working.
Btw body outputs itself appears as JSON format but JSON.parse(body) doesn't produce hash
puts body
{"region":{"span":{"latitude_delta":0.0,"longitude_delta":0.0},"center":{"latitude":37.660418999999997,"longitude":-121.876508}},"total":853,"businesses":[{"rating":4.0,"mobile_url":"http://m.yelp.com/biz/TT1t4oHeZmqkoiuwgCN4bQ","rating_img_url":"http://media2.ak.yelpcdn.com/static/201012164084228337/img/ico/stars/stars_4.png","review_count":150,"name":"India Garden","rating_img_url_small":"http://media2.ak.yelpcdn.com/static/20101216418129184/img/ico/stars/stars_small_4.png","url":"http://www.yelp.com/biz/india-garden-pleasanton-2","phone":"9254854800","snippet_text":"We went to this place without seeing any reviews while we returning to San Jose from Cache Creek in Brooks. This place looks like a house which was...","image_url":"http://s3-media4.ak.yelpcdn.com/bphoto/8iFj1S9YaU5IdUazwZOG8A/ms.jpg","snippet_image_url":"http://s3-media3.ak.yelpcdn.com/photo/d2TovvsTn2eUw4xqTB4jyw/ms.jpg","display_phone":"+1-925-485-4800","rating_img_url_large":"http://media4.ak.yelpcdn.com/static/20101216169592178/img/ico/stars/stars_large_4.png","id":"india-garden-pleasanton-2","categories":[["Indian","indpak"],["Pakistani","pakistani"]],"location":{"cross_streets":"Main St & Neal St","city":"Pleasanton","display_address":["210 Rose Ave","(b/t Main St & Neal St)","Pleasanton, CA 94566"],"geo_accuracy":8,"postal_code":"94566","country_code":"US","address":["210 Rose Ave"],"coordinate":{"latitude":37.660418999999997,"longitude":-121.876508},"state_code":"CA"}}]}
Actually i'm pretty sure, that body is an Array at this point, since response contains four parts and not three, so the last two parts (an array) are put into the body-object.
Also Array is the only core-object i know, which complains about an [] parameter being not an integer. If it was a string, it would try a regex/contain match.
So to sum up, body is an Array with only one Value containing a String. So to get your Hash (from JSON) you have to real_body = JSON.parse body[0]. Then you should get your hash and
real_body["businesses"] puts your businesses (the output is rather long so i will not be posting it here)
body is a string at this point, not a hash. The [] operator is complaining because the string [] operator only takes an integer, so it's trying to turn your string into an integer and failing.
edit: you can test this by just printing out body.class