I was using the following code snippet, which was working, to fetch the JSON response from the Spotify oEmbed API.
I was doing this:
response = JSON.parse(open("https://embed.spotify.com/oembed/?url=http://open.spotify.com/album/5Mxj65mCzxUjDkoqz7JXPJ").read)
But now, this stopped working because:
2.0.0p247 :017 > open("https://embed.spotify.com/oembed/?url=http://open.spotify.com/album/5Mxj65mCzxUjDkoqz7JXPJ").read
=> ""
But, using cURL, I get:
$curl https://embed.spotify.com/oembed/\?url\=http://open.spotify.com/album/5Mxj65mCzxUjDkoqz7JXPJ
{"provider_url":"https:\/\/www.spotify.com","version":"1.0","thumbnail_width":300,"height":380,"thumbnail_height":300,"title":"Illum Sphere - 13. Embryonic","width":300,"thumbnail_url":"https:\/\/d3rt1990lpmkn.cloudfront.net\/cover\/d4f5d8624752eeae3dc8bc3d58e2155d095181c1","provider_name":"Spotify","type":"rich","html":"<iframe src=\"https:\/\/embed.spotify.com\/?uri=spotify:album:5Mxj65mCzxUjDkoqz7JXPJ\" width=\"300\" height=\"380\" frameborder=\"0\" allowtransparency=\"true\"><\/iframe>"}
Any advice?
Try passing the User-Agent:
require 'open-uri'
puts open("https://embed.spotify.com/oembed/?url=http://open.spotify.com/album/5Mxj65mCzxUjDkoqz7JXPJ", "User-Agent" => "Ruby/#{RUBY_VERSION}").read
Result:
{"provider_url":"https:\/\/www.spotify.com","version":"1.0","thumbnail_width":300,"height":380,"thumbnail_height":300,"title":"
Illum Sphere - 13. Embryonic","width":300,"thumbnail_url":"https:\/\/d3rt1990lpmkn.cloudfront.net\/cover\/d4f5d8624752eeae3dc8b
c3d58e2155d095181c1","provider_name":"Spotify","type":"rich","html":"<iframe src=\"https:\/\/embed.spotify.com\/?uri=spotify:al
bum:5Mxj65mCzxUjDkoqz7JXPJ\" width=\"300\" height=\"380\" frameborder=\"0\" allowtransparency=\"true\"><\/iframe>"}
Related
What is wrong with this?
HTTParty.get("https://www.instagram.com/#{username}/?__a=1", :headers => {"x-instagram-gis" => Digest::MD5.hexdigest("#{rhx_gis}:/#{username}/")})
If I do the same thing using curl passing the headers via --header, it works.
curl "https://www.instagram.com/hossamhossny/" | grep "rhx_gis"
We can then grab rhx_gis value then MD5 "<rhx_gis>:/username/" as "1ba367317d3c842eb4e940f1d62b29f2:/hossamhossny/" to produce the value of x-instagram-gis as "2830c5bff8e05b755724df2c6286f2b4".
Now with curl I can do that..
curl --header "x-instagram-gis:2830c5bff8e05b755724df2c6286f2b4" "https://www.instagram.com/hossamhossny/?__a=1"
Which produces the desired output..
{"logging_page_id":"profilePage_3669759838","show_suggested_profiles":false,"graphql":{"user":{"biography":"","blocked_by_viewer":false,"country_block":false,"external_url":null,"external_url_linkshimmed":null,"edge_followed_by":{"count":222},"followed_by_viewer":false,"edge_follow":{"count":55},"follows_viewer":false,"full_name":"Hossam SAld HossNy","has_blocked_viewer":false,"highlight_reel_count":0,"has_requested_viewer":false,"id":"3669759838","is_private":false,"is_verified":false,"mutual_followers":null,"profile_pic_url":"https://scontent-dfw5-1.cdninstagram.com/vp/6d079f6b5b60323cbfc4442c460d0e52/5BA22630/t51.2885-19/s150x150/13628116_1250649874979851_160235967_a.jpg","profile_pic_url_hd":"https://scontent-dfw5-1.cdninstagram.com/vp/6d079f6b5b60323cbfc4442c460d0e52/5BA22630/t51.2885-19/s150x150/13628116_1250649874979851_160235967_a.jpg","requested_by_viewer":false,"username":"hossamhossny","connected_fb_page":null,"edge_felix_video_timeline":{"count":0,"page_info":{"has_next_page":false,"end_cursor":null},"edges":[]},"edge_owner_to_timeline_media":{"count":0,"page_info":{"has_next_page":false,"end_cursor":null},"edges":[]},"edge_saved_media":{"count":0,"page_info":{"has_next_page":false,"end_cursor":null},"edges":[]},"edge_media_collections":{"count":0,"page_info":{"has_next_page":false,"end_cursor":null},"edges":[]}}}}
Switching back to HTTParty I tried this:
HTTParty.get("http://www.instagram.com/#{username}/?__a=1", :headers => {"x-instagram-gis" => "2830c5bff8e05b755724df2c6286f2b4"})
but that didn't work either; 403 unauthorized error. My problem is that I am not able to pass the headers properly using HTTParty.
The url with SSL returned error but below mentioned worked for me.
Try using it with http (without https)
HTTParty.get("http://www.instagram.com/#{username}/?__a=1", :headers => {"x-instagram-gis" => Digest::MD5.hexdigest("#{rhx_gis}:/#{username}/")})
I'm building out a YahooFinance Api and keep hitting a brick wall when trying to use open URI.
Code:
uri = ("http://ichart.finance.yahoo.com/table.csv?s=#{URI.escape(code)}&a=#{start_month}&b=#{start_day}&c=#{start_year}&d=#{end_month}&e=#{end_day}&f=#{end_year}&g=d&ignore=.csv")
puts "#{uri}"
conn = open(uri)
Error:
`split': bad URI(is not URI?): http://ichart.finance.yahoo.com/table.csv?s=%255EIXIC&a=00&b=1&c=1994&d=09&e=14&f=2014&g=d&ignore=.csv} (URI::InvalidURIError)
I have tried URI.unescape(code) which outputs code as ^IXIC, as well as leaving any URI methods out and code will come through as %5EIXIC.
After reading around on stack overflow, I've tried both of these methods to no avail:
uri = URI.parse(URI.encode(url.strip))
safeurl = URI.encode(url.strip)
Even after looking through the code for another ruby yahoo-finance gem, here, I can't seem to find a solution. Any help is greatly appreciated. Thanks
EDIT: I am able to use open(uri) when I manually enter in the url in single quotes. Do double quotes, (used for inserting ruby objects), play a role here?
Don't try to inject variables into URLs. If they contain characters that need to be encoded per the spec, they won't be by interpolation. Instead, take advantage of the right tools for the job, like Ruby's URI class or the Addressable::URI gem.
See "How to post a URL containting curly braces and colons" for how to do this using well tested wheels.
In your situation, something like this will work:
require 'uri'
code = 'qwer3456*&^%'
start_month = 1
start_day = 1
start_year = 2014
end_month = 12
end_day = 31
end_year = 2015
uri = URI.parse("http://ichart.finance.yahoo.com/table.csv")
uri.query = URI.encode_www_form(
{
'g' => 'd',
'ignore' => '.csv',
's' => code,
'a' => start_month,
'b' => start_day,
'c' => start_year,
'd' => end_month,
'e' => end_day,
'f' => end_year
}
)
uri.to_s # => "http://ichart.finance.yahoo.com/table.csv?g=d&ignore=.csv&s=qwer3456*%26%5E%25&a=1&b=1&c=2014&d=12&e=31&f=2015"
The code works for me though I don't think the API endpoint is correct:
[1] pry(main)> uri = URI("http://ichart.finance.yahoo.com/table.csv?s=%255EIXIC&a=00&b=1&c=1994&d=09&e=14&f=2014&g=d&ignore=.csv")
=> #<URI::HTTP:0x007fd63a2fff40 URL:http://ichart.finance.yahoo.com/table.csv?s=%255EIXIC&a=00&b=1&c=1994&d=09&e=14&f=2014&g=d&ignore=.csv>
[3] pry(main)> Net::HTTP.get(uri)
=> "<!doctype html public \"-//W3C//DTD HTML 4.01//EN\" \"http://www.w3.org/TR/html4/strict.dtd\">\n<html><head><title>Yahoo! - 404 Not Found</title><style>\n/* nn4 hide */ \n/*/*/\nbody {font:small/1.2em arial,helvetica,clean,sans-serif;font:x-small;text-align:center;}table {font-size:inherit;font:x-small;}\nhtml>body {font:83%/1.2em arial,helvetica,clean,sans-serif;}input {font-size:100%;vertical-align:middle;}p, form {margin:0;padding:0;}\np {padding-bottom:6px;margin-bottom:10px;}#doc {width:48.5em;margin:0 auto;border:1px solid #fff;text-align:center;}#ygma {text-align:right;margin-bottom:53px}\n#ygma img {float:left;}#ygma div {border-bottom:1px solid #ccc;padding-bottom:8px;margin-left:152px;}#bd {clear:both;text-align:left;width:75%;margin:0 auto 20px;}\nh1 {font-size:135%;text-align:center;margin:0 0 15px;}legend {display:none;}fieldset {border:0 solid #fff;padding:.8em 0 .8em 4.5em;}\nform {position:relative;background:#eee;margin-bottom:15px;border:1px solid #ccc;border-width:1px 0;}\n#s1p {width:15em;margin-right:.1em;}\nform span {position:absolute;left:70%;top:.8em;}form a {font:78%/1.2em arial;display:block;padding-left:.8em;white-space:nowrap;background: url(http://l.yimg.com/a/i/s/bullet.gif) no-repeat left center;} \nform .sep {display:none;}.more {text-align:center;}#ft {padding-top:10px;border-top:1px solid #999;}#ft p {text-align:center;font:78% arial;}\n/* end nn4 hide */\n</style></head>\n<body><div id=\"doc\">\n<div id=\"ygma\"><img\nsrc=http://l.yimg.com/a/i/yahoo.gif\nwidth=147 height=31 border=0 alt=\"Yahoo!\"><div><a\nhref=\"http://us.rd.yahoo.com/404/*http://www.yahoo.com\">Yahoo!</a>\n - Help</div></div>\n<div id=\"bd\"><h1>Sorry, the page you requested was not found.</h1>\n<p>Please check the URL for proper spelling and capitalization. If\nyou're having trouble locating a destination on Yahoo!, try visiting the\n<strong><a\nhref=\"http://us.rd.yahoo.com/404/*http://www.yahoo.com\">Yahoo! home\npage</a></strong> or look through a list of <strong><a\nhref=\"http://us.rd.yahoo.com/404/*http://docs.yahoo.com/docs/family/more/\">Yahoo!'s\nonline services</a></strong>. Also, you may find what you're looking for\nif you try searching below.</p>\n<form name=\"s1\" action=\"http://us.rd.yahoo.com/404/*-http://search.yahoo.com/search\"><fieldset>\n<legend><label for=\"s1p\">Search the Web</label></legend>\n<input type=\"text\" size=30 name=\"p\" id=\"s1p\" title=\"enter search terms here\">\n<input type=\"submit\" value=\"Search\">\n<span>advanced search <span class=sep>|</span> most popular</span>\n</fieldset></form>\n<p class=\"more\">Please try <strong><a\nhref=\"http://us.rd.yahoo.com/404/*http://help.yahoo.com\">Yahoo!\nHelp Central</a></strong> if you need more assistance.</p>\n</div><div id=\"ft\"><p>Copyright © 2014 Yahoo! Inc.\nAll rights reserved. <a\nhref=\"http://us.rd.yahoo.com/404/*http://privacy.yahoo.com\">Privacy\nPolicy</a> - <a\nhref=\"http://us.rd.yahoo.com/404/*http://docs.yahoo.com/info/terms/\">Terms\nof Service</a></p></div>\n</div></body></html>\n"
Looks like your problem is the ignore=.csv part.
I mean this is probably trying to encode it as a domain extension. Probably you should remove the dot to solve the problem.
I'm trying to upload a local video file to my youtube account using youtube data api.
Example: https://github.com/youtube/api-samples/blob/master/ruby/upload_video.rb
Call:
$ ruby upload_video.rb --file video.mp4
I'm getting an error message:
/Users/user/.rvm/rubies/ruby-2.1.1/lib/ruby/2.1.0/openssl/buffering.rb:326:in `syswrite': Broken pipe (Errno::EPIPE)
from /Users/user/.rvm/rubies/ruby-2.1.1/lib/ruby/2.1.0/openssl/buffering.rb:326:in `do_write'
from /Users/user/.rvm/rubies/ruby-2.1.1/lib/ruby/2.1.0/openssl/buffering.rb:344:in `write'
from /Users/user/.rvm/rubies/ruby-2.1.1/lib/ruby/2.1.0/net/http/generic_request.rb:202:in `copy_stream'
from /Users/user/.rvm/rubies/ruby-2.1.1/lib/ruby/2.1.0/net/http/generic_request.rb:202:in `send_request_with_body_stream'
from /Users/user/.rvm/rubies/ruby-2.1.1/lib/ruby/2.1.0/net/http/generic_request.rb:132:in `exec'
...
from upload_video.rb:73:in `main'
from upload_video.rb:91:in `<main>'
upload_video.rb (line 73):
videos_insert_response = client.execute!(
:api_method => youtube.videos.insert,
:body_object => body,
:media => Google::APIClient::UploadIO.new(opts[:file], 'video/*'),
:parameters => {
:uploadType => 'resumable',
:part => body.keys.join(',')
}
)
google-api-client uses faraday gem which default http adapter is Net::HTTP. And it seems it's not working properly. So we need to switch it to something else. I changed it to httpclient and now it works fine.
Just add this line somewhere in the beginning of your whatever.rb file (right after requiring google's libs):
Faraday.default_adapter = :httpclient
Full list of supported http adapters could be found here: https://github.com/lostisland/faraday#faraday
I'm facing a problem while trying to extract fields of foursquare2 gem response. Here's my code:
require 'foursquare2'
client = Foursquare2::Client.new(:client_id => 'XYZ', :client_secret => 'XYZ')
tips = client.venue_tips('4d169843b15cb1f7f4c4ae21')
And then I got the result:
#<Hashie::Mash count=8 items=[#<Hashie::Mash canonicalUrl="https://foursquare.com/item/4f689747e4b04ab624cce6d3" createdAt=1332254535 done=#<Hashie::Mash count=1> id="4f689747e4b04ab624cce6d3" lang="pt" likes=#<Hashie::Mash count=2 groups=[#<Hashie::Mash count=2 items=[#<Hashie::Mash bio="" contact=#<Hashie::Mash> firstName="Pedro" gender="male" homeCity="Recife, Brasil" id="19761425" lastName="T." photo="https://is1.4sqi.net/userpix_thumbs/G5R14BT031K2B2AX.jpg" tips=#<Hashie::Mash count=25>>, #<Hashie::Mash bio="" contact=#<Hashie::Mash> firstName="Nat\u00E1lia" gender="female" homeCity="Recife, Brasil" id="12706383" lastName="S."
photo="https://is0.4sqi.net/userpix_thumbs/UF0Q0OZFWNQLXRFG.jpg" tips=#<Hashie::Mash count=1>>] type="others">] summary="2 likes"> text="Nova padaria com bons produtos, destaque para as formadas de p\u00E3o \u00E0 tarde."
I'd like to have something as tips.firstName to get these values, but it doesn't work. Should I prefer the REST API?!
Thank you in advance, guys.
Edited after comments
The solution I did following the suggestion provided:
tips.groups[0].items.each{|i| puts i.firstName}
But it's not working. I just got the error: data_scraping_fq.rb:16:in <main>': undefined method[]' for nil:NilClass (NoMethodError)
tips.firstName won't work when you're using the venues/tips endpoint. Take a look at the JSON response when you use the Foursquare API explorer and you'll see that to get the tip submitter's first name, you need to look at `tips.items
In ruby how to convert the follwing into valid json
"{\"transferType\"=>\"D\", \"accountNumber\"=>\"123\", \"employeeId\"=>\"12\", \"districtNumber\"=>\"15\", \"orderedBy\"=>\"vbcb\", \"department\"=>\"vghghj\", \"po\"=>\"23434\", \"Phone\"=>\"4542433435\", \"Instructions\"=>\"vbvcnvbnb\"}"
The class of above is string. Because Without proper Json conversion i am problem like
Javascript replace for equal symbol
First of all you should make this in to proper JSON format. So replace the => in to :
and then try JSON.load or JSON.parse will work successfully.
2.2.3 :015 > data
=> "{\"transferType\"=>\"D\", \"accountNumber\"=>\"123\", \"employeeId\"=>\"12\", \"districtNumber\"=>\"15\", \"orderedBy\"=>\"vbcb\", \"department\"=>\"vghghj\", \"po\"=>\"23434\", \"Phone\"=>\"4542433435\", \"Instructions\"=>\"vbvcnvbnb\"}"
2.2.3 :016 > JSON.parse(data.gsub("=>", ":"))
=> {"transferType"=>"D", "accountNumber"=>"123", "employeeId"=>"12", "districtNumber"=>"15", "orderedBy"=>"vbcb", "department"=>"vghghj", "po"=>"23434", "Phone"=>"4542433435", "Instructions"=>"vbvcnvbnb"}
from the result you can do anything what do you want :)