I'm building out a YahooFinance Api and keep hitting a brick wall when trying to use open URI.
Code:
uri = ("http://ichart.finance.yahoo.com/table.csv?s=#{URI.escape(code)}&a=#{start_month}&b=#{start_day}&c=#{start_year}&d=#{end_month}&e=#{end_day}&f=#{end_year}&g=d&ignore=.csv")
puts "#{uri}"
conn = open(uri)
Error:
`split': bad URI(is not URI?): http://ichart.finance.yahoo.com/table.csv?s=%255EIXIC&a=00&b=1&c=1994&d=09&e=14&f=2014&g=d&ignore=.csv} (URI::InvalidURIError)
I have tried URI.unescape(code) which outputs code as ^IXIC, as well as leaving any URI methods out and code will come through as %5EIXIC.
After reading around on stack overflow, I've tried both of these methods to no avail:
uri = URI.parse(URI.encode(url.strip))
safeurl = URI.encode(url.strip)
Even after looking through the code for another ruby yahoo-finance gem, here, I can't seem to find a solution. Any help is greatly appreciated. Thanks
EDIT: I am able to use open(uri) when I manually enter in the url in single quotes. Do double quotes, (used for inserting ruby objects), play a role here?
Don't try to inject variables into URLs. If they contain characters that need to be encoded per the spec, they won't be by interpolation. Instead, take advantage of the right tools for the job, like Ruby's URI class or the Addressable::URI gem.
See "How to post a URL containting curly braces and colons" for how to do this using well tested wheels.
In your situation, something like this will work:
require 'uri'
code = 'qwer3456*&^%'
start_month = 1
start_day = 1
start_year = 2014
end_month = 12
end_day = 31
end_year = 2015
uri = URI.parse("http://ichart.finance.yahoo.com/table.csv")
uri.query = URI.encode_www_form(
{
'g' => 'd',
'ignore' => '.csv',
's' => code,
'a' => start_month,
'b' => start_day,
'c' => start_year,
'd' => end_month,
'e' => end_day,
'f' => end_year
}
)
uri.to_s # => "http://ichart.finance.yahoo.com/table.csv?g=d&ignore=.csv&s=qwer3456*%26%5E%25&a=1&b=1&c=2014&d=12&e=31&f=2015"
The code works for me though I don't think the API endpoint is correct:
[1] pry(main)> uri = URI("http://ichart.finance.yahoo.com/table.csv?s=%255EIXIC&a=00&b=1&c=1994&d=09&e=14&f=2014&g=d&ignore=.csv")
=> #<URI::HTTP:0x007fd63a2fff40 URL:http://ichart.finance.yahoo.com/table.csv?s=%255EIXIC&a=00&b=1&c=1994&d=09&e=14&f=2014&g=d&ignore=.csv>
[3] pry(main)> Net::HTTP.get(uri)
=> "<!doctype html public \"-//W3C//DTD HTML 4.01//EN\" \"http://www.w3.org/TR/html4/strict.dtd\">\n<html><head><title>Yahoo! - 404 Not Found</title><style>\n/* nn4 hide */ \n/*/*/\nbody {font:small/1.2em arial,helvetica,clean,sans-serif;font:x-small;text-align:center;}table {font-size:inherit;font:x-small;}\nhtml>body {font:83%/1.2em arial,helvetica,clean,sans-serif;}input {font-size:100%;vertical-align:middle;}p, form {margin:0;padding:0;}\np {padding-bottom:6px;margin-bottom:10px;}#doc {width:48.5em;margin:0 auto;border:1px solid #fff;text-align:center;}#ygma {text-align:right;margin-bottom:53px}\n#ygma img {float:left;}#ygma div {border-bottom:1px solid #ccc;padding-bottom:8px;margin-left:152px;}#bd {clear:both;text-align:left;width:75%;margin:0 auto 20px;}\nh1 {font-size:135%;text-align:center;margin:0 0 15px;}legend {display:none;}fieldset {border:0 solid #fff;padding:.8em 0 .8em 4.5em;}\nform {position:relative;background:#eee;margin-bottom:15px;border:1px solid #ccc;border-width:1px 0;}\n#s1p {width:15em;margin-right:.1em;}\nform span {position:absolute;left:70%;top:.8em;}form a {font:78%/1.2em arial;display:block;padding-left:.8em;white-space:nowrap;background: url(http://l.yimg.com/a/i/s/bullet.gif) no-repeat left center;} \nform .sep {display:none;}.more {text-align:center;}#ft {padding-top:10px;border-top:1px solid #999;}#ft p {text-align:center;font:78% arial;}\n/* end nn4 hide */\n</style></head>\n<body><div id=\"doc\">\n<div id=\"ygma\"><img\nsrc=http://l.yimg.com/a/i/yahoo.gif\nwidth=147 height=31 border=0 alt=\"Yahoo!\"><div><a\nhref=\"http://us.rd.yahoo.com/404/*http://www.yahoo.com\">Yahoo!</a>\n - Help</div></div>\n<div id=\"bd\"><h1>Sorry, the page you requested was not found.</h1>\n<p>Please check the URL for proper spelling and capitalization. If\nyou're having trouble locating a destination on Yahoo!, try visiting the\n<strong><a\nhref=\"http://us.rd.yahoo.com/404/*http://www.yahoo.com\">Yahoo! home\npage</a></strong> or look through a list of <strong><a\nhref=\"http://us.rd.yahoo.com/404/*http://docs.yahoo.com/docs/family/more/\">Yahoo!'s\nonline services</a></strong>. Also, you may find what you're looking for\nif you try searching below.</p>\n<form name=\"s1\" action=\"http://us.rd.yahoo.com/404/*-http://search.yahoo.com/search\"><fieldset>\n<legend><label for=\"s1p\">Search the Web</label></legend>\n<input type=\"text\" size=30 name=\"p\" id=\"s1p\" title=\"enter search terms here\">\n<input type=\"submit\" value=\"Search\">\n<span>advanced search <span class=sep>|</span> most popular</span>\n</fieldset></form>\n<p class=\"more\">Please try <strong><a\nhref=\"http://us.rd.yahoo.com/404/*http://help.yahoo.com\">Yahoo!\nHelp Central</a></strong> if you need more assistance.</p>\n</div><div id=\"ft\"><p>Copyright © 2014 Yahoo! Inc.\nAll rights reserved. <a\nhref=\"http://us.rd.yahoo.com/404/*http://privacy.yahoo.com\">Privacy\nPolicy</a> - <a\nhref=\"http://us.rd.yahoo.com/404/*http://docs.yahoo.com/info/terms/\">Terms\nof Service</a></p></div>\n</div></body></html>\n"
Looks like your problem is the ignore=.csv part.
I mean this is probably trying to encode it as a domain extension. Probably you should remove the dot to solve the problem.
Related
I'm now trying to upload a mp3 file to Soundcloud. Here I'm bogged down to the use of File.new command in Ruby.
I send a request and a passing parameter looks like the below.
Parameters: {..."mp3_1"=>#<ActionDispatch::Http::UploadedFile:0x007ff24d5e3ea8 #tempfile=#<Tempfile:/var/folders/kk/y_wprlln2qv6mzylj03g14x00000gn/T/RackMultipart20160316-21426-14vu8x1.mp3>, #original_filename="datasecurity.mp3", #content_type="audio/mp3", #headers="Content-Disposition: form-data; name=\"mp3_1\"; filename=\"datasecurity.mp3\"\r\nContent-Type: audio/mp3\r\n">}
Then, I write File.new command with the potentail file name and params[:mp3_1] like the below.
client = Soundcloud.new(:access_token => 'XXX')
track = client.post('/tracks', :track => {
:title => 'This is my sound',
:asset_data => File.new("file name",params[:mp3_1])
})
Now I get an error saying:
no implicit conversion of ActionDispatch::Http::UploadedFile into String
The paperclip function works ( storing file to the storage directly has been what I've done ) but this file.new doesn't allow me to move forward. If I can get any help, I really appreciate that (:
Best
you already have a file, no need to create a new one with File.new
have a closer look to your dump :
#tempfile=#<Tempfile:/var/folders/...../RackMultipart20160316-21426-14vu8x1.mp3
this is a file, you may use it directly in your call
client.post('/tracks', :track => {
:title => 'This is my sound',
:asset_data => params[:mp3_1].tempfile)
})
I don't know if this is just a goof syntax error, or if this is something I should legitimately be concerned about. I've searched online for this error and have found very little concerning it. So my guess is that this is either something so "no duh" that hardly anybody ends up with this error, or that it's so obscure that - again - hardly anybody ends up with it. Nonetheless, here we go:
This is what I'm using:
Ruby 2.1.1p76
Rails 4.0.5
SafeYAML 1.0.3
RubyMine 6.3
[should you need anymore information, please ask]
The error I'm getting exactly is: "Rails i18n locale file should have single root"
Here is a shortened version of my en.yml file (I'll include what I believe to be the problem areas)
<%
object_names = **{**
:administrator => 'Administrator',
:activity_log => 'Site Activity',
:answer => 'Answer',
:approval => 'Approval',
:user => 'Member',
:video => 'Video',
:vote => 'Like'
}
section_names = **{**
:approvals => 'Items Awaiting Review',
:advertisements => 'Advertisements',
:ad_placements => 'Ad Placements',
:awarded_badges => 'Badges',
:badges => 'Badges',
:videos => 'Videos'
}
anonymous = 'Anonymous**'
%>
en:
homepage:
mine: "My %{site_name}"
site_name: "%{site_name}"
site_condition_name: "%{site_condition_name}"
titles:
main: "%{site_name}"
delimiters:
minor:** ' **-** '
major: ' | '
scopes:
popular: Popular
newest: Newest
active: Active
my_feed: My Friends
my_activity: What I've Done
tracked_items: "Only <%= object_names[:tracked_item].pluralize %>"
everyone: Everyone
user: "By %{name}"
view: "By %{view}"
sent: "Sent <%= object_names[:message].pluralize %>"
page: "Page %{page}"
letter: "Starting With %{letter}"
query: "%{query}"
category: "%{category}"
**meta_description:**
main: "%{site_name} is a social network that connects people"
footer:
about: "About %{site_name}"
about_alliance_health: About Alliance Health
community_advocates: Community Advocates
terms_of_use: Terms of Use
oh - wow. Ok, it looks like bold isn't going to work inside the code markers. But I'm sure you guys can decipher what's going on. Basically anything that's bold (or delimited with a double asterisk '**') is where the IDE is marking the code with this particular error. So, as you can see, it seems rather chaotic and nonsensical. But I often find that when such things happen, it's usually one tiny little character somewhere that's throwing everything else off.
Now, I'm no YAML expert - in fact, I hardly even know the stuff (which is something I'm planning on changing here in the near future) so this may be something along the "no duh" lines. However, it is interesting to note that the bulk of the error-marked syntax starts with the last single-quote of the word: " 'anonymous' ", all the way down to: " minor: ", skipping the first following single quote, and picking up again on the hyphen, afterwhich there are no more errors for the rest of the nearly 5,500 lines of this file.
Thanks to anybody who helps out. I've been dorking around with this for far too long, and with very little online information on this particular issue. So any help is much appreciated :)
Thanks!
So - I'm not sure this will ever help anybody ... but in the case that it may, this is how I re-engineered the script to avoid the errors. I simply removed the usual delimiters that were causing the parsing confusion. And #mu is too short, you were correct. This is an erb.yml file, though I'm not sure why the original creator left the dual extension off. At any rate - thank you to everybody who offered any suggestions and ideas :)
<%
object_names = Hash.new
object_names[:administrator] = %q<Administrator>
object_names[:activity_log] = %q<Site Activity>
object_names[:answer] = %q<Answer>
object_names[:approval] = %q<Approval>
object_names[:user] = %q<Member>
object_names[:video] = %q<Video>
object_names[:vote] = %q<Like>
section_names = Hash.new
section_names[:approvals] = %q<Items Awaiting Review>
section_names[:advertisements] = %q<Advertisements>
section_names[:ad_placements] = %q<Ad Placements>
section_names[:awarded_badges] = %q<Badges>
section_names[:badges] = %q<Badges>
section_names[:videos] = %q<Videos>
%>
en:
homepage:
mine: "My %{site_name}"
site_name: "%{site_name}"
site_condition_name: "%{site_condition_name}"
titles:
main: "%{site_name}"
delimiters:
minor: ' - '
major: ' | '
scopes:
popular: Popular
newest: Newest
active: Active
my_feed: My Friends
my_activity: What I've Done
tracked_items: "Only <%= object_names[:tracked_item].pluralize %>"
everyone: Everyone
user: "By %{name}"
view: "By %{view}"
sent: "Sent <%= object_names[:message].pluralize %>"
page: "Page %{page}"
letter: "Starting With %{letter}"
query: "%{query}"
category: "%{category}"
meta_description:
main: "%{site_name} is a social network that connects people"
footer:
about: "About %{site_name}"
about_alliance_health: About Alliance Health
community_advocates: Community Advocates
terms_of_use: Terms of Use
I am using the ruby gem 'octokit' which implements the Github API v3. Mostly works great but I cannot seem to filter by date. I believe I have the syntax and time format correct, but it appears my option is ignored and the API returns the past 35 entries regardless of the since or until dates.
Here's a minimal reproducible example (after installing the octokit gem).
require 'octokit'
require 'time'
#day = "2012-09-27"
#until = DateTime.parse(#date).iso8601
#since = (DateTime.parse(#day) - 60*60*48).iso8601
a = Octokit.commits({:username => "cboettig", :repo => "labnotebook", :since => #since, :until => #until})
see the date of the output of last entry
a.last.commit.author.date
explicit day doesn't work either
b = Octokit.commits({:username => "cboettig", :repo => "labnotebook", :since => "2012-09-27T00:00:00+00:00"})
b.last.commit.author.date
The date I get in both examples is from August, outside the specified range given. What did I miss?
Background: I'm trying to write a little Jekyll plugin that uses the API to return commits made to a specified repo on the day of the post.
joeyw gives a great answer to this question here.
The second argument should be the sha or branch, and options should be the third argument, e.g.
Octokit.commits("cboettig/labnotebook", "master", :since => "2012-09-28T00:00:00+00:00").length
or
Octokit.commits("cboettig/labnotebook", nil, :since => "2012-09-28T00:00:00+00:00").length
works just fine. Here's my corresponding jekyll plugin
I am setting my own MediaWiki website locally, and am not able to get the InstantCommons feature to work (used to directly embed files from commons.wikimedia.org).
I get no error message, the files I try to load from Commons using the following syntax:
[[File:Cervus elaphus Luc Viatour 1.jpg|Cervus elaphus Luc Viatour 1]]
are just not loaded, and I end up with a red link on my page, referring to a non-existing file. It has been 2 days now that I am looking for a solution, but so far without any success.
I am running:
MediaWiki v.1.19.1
Fedora 16 (with SElinux)
PHP 5.3.15
MySQL Ver 14.14 Distrib 5.5.25a, for Linux (x86_64)
I have tried the following two configurations in my LocalSettings.php, without success:
$wgUseInstantCommons = true;
AND
$wgForeignFileRepos[] = array(
'class' => 'ForeignAPIRepo',
'name' => 'shared',
'apibase' => 'http://commons.wikimedia.org/w/api.php',
'fetchDescription' => true, // Optional
'descriptionCacheExpiry' => 43200, // 12 hours, optional (values are seconds)
'apiThumbCacheExpiry' => 43200, // 12 hours, optional, but required for local thumb caching
);
Any suggestion is most welcome.
OK, this is not (yet) an answer, but a debugging suggestion. It looks to me like the HTTP request from your server to Commons is failing for some reason, but unfortunately ForeignAPIRepo doesn't indicate the cause of the error in any way.
This is really a bug in MediaWiki, and should be fixed, but in the mean time, could you please try applying the following diff (or just manually adding the line marked with the + sign) to your includes/filerepo/ForeignAPIRepo.php file:
Index: includes/filerepo/ForeignAPIRepo.php
===================================================================
--- includes/filerepo/ForeignAPIRepo.php (revision 97048)
+++ includes/filerepo/ForeignAPIRepo.php (working copy)
## -385,6 +385,7 ##
if ( $status->isOK() ) {
return $req->getContent();
} else {
+ wfDebug( "ForeignAPIRepo: HTTP GET failed: " . $status->getXML() );
return false;
}
}
After applying it, try loading the file description page for a Commons image and look at the MediaWiki debug log. There should now be a line starting with ForeignAPIRepo: HTTP GET failed: followed by a few lines of XML error dump. That error data should hopefully indicate what's going wrong; please copy and paste it here.
Mine is not a definitive answer either. Referring to Ilmari Karonen's post, I was unable to find or get the getXML() method to execute for my version of Mediawiki v1.23.0. I was looking at the reference documentation found here to try and find any other method calls on the Status class that would give me good troubleshooting info. I ended up finding the following and editing the same file as mentioned in Ilmari Karonen's post includes/filerepo/ForeignAPIRepo.php beginning at line #521:
if ( $status->isOK() ) {
return $req->getContent();
} else {
$error = $status->getErrorsArray();
$dump = print_r($error, true);
wfDebug("ForeignAPIRepo: HTTP GET failed: $dump\n");
return false;
}
The default InstantCommons configuration of older MediaWikis is a bit silly. Due to T114098 I recommend one of the following, which will hopefully fix your problems:
upgrade to MediaWiki 1.27 (when it's released), or
set your LocalSettings.php to hotlink images to save on server-side requests and processing.
$wgUseInstantCommons = false;
$wgForeignFileRepos[] = array(
'class' => 'ForeignAPIRepo',
'name' => 'commonshotlink',
'apibase' => 'https://commons.wikimedia.org/w/api.php',
'hashLevels' => 2,
'url' => 'https://upload.wikimedia.org/wikipedia/commons',
'thumbUrl' => 'https://upload.wikimedia.org/wikipedia/commons/thumb',
'transformVia404' => true,
'fetchDescription' => true,
'descriptionCacheExpiry' => 43200,
'apiThumbCacheExpiry' => 24 * 3600,
);
I am aware of a number of posts on this topic - the most comprehensive of which is here.
My particular gremlin looks like this: (without further logging)
2012-04-17T00:28:50+00:00 DEBUG (7): HEADERS ALREADY SENT: <pre>[0] /domains/bodhi.uk.com/subdomains/dev/app/code/core/Mage/Core/Controller/Response/Http.php:51
[1] /domains/bodhi.uk.com/subdomains/dev/lib/Zend/Controller/Response/Abstract.php:766
[2] /domains/bodhi.uk.com/subdomains/dev/app/code/core/Mage/Core/Controller/Response/Http.php:89
[3] /domains/bodhi.uk.com/subdomains/dev/app/code/core/Mage/Core/Controller/Varien/Front.php:188
[4] /domains/bodhi.uk.com/subdomains/dev/app/code/core/Mage/Core/Model/App.php:349
[5] /domains/bodhi.uk.com/subdomains/dev/app/Mage.php:640
[6] /domains/bodhi.uk.com/subdomains/dev/index.php:80
This can occur loading the homepage on dev.bodhi.uk.com
Following the accepted answer in the linked post, creates a list of around 500 files, that are being included - not a practical method for debugging (especially when I may not recognise the issue when looking at it - I am currently assuming that I am looking for whitespace prior to a
For reference the output of the above is here on pastebin
Using the advice in the last comment (that appeared to work for the OP), and adding:
$headers = array();
foreach ($this->_headers as $i => $header) {
$headers[$header['name']] = $header;
}
$this->_headers = array_values($headers);
Mage::Log(array_values($headers));
Gives the unexpected responce:
2012-04-17T00:28:50+00:00 DEBUG (7): Array
(
[0] => Array
(
[name] => Content-Type
[value] => text/html; charset=UTF-8
[replace] =>
)
)
Ultimately I need to prevent this error, and I am struggling for a next step!
PS: I have also tried Alan's responce in comment 1, however the format of Abstract.php has changed (I am using version 1.6.2), and as I get no further output, I suspect I am not logging things correctly.
Thanks in advance!
Try ob_start() it may be helpful to you just put this code at the top of your file