I have some nested strings in a complex hash that triggers "ArgumentError" exceptions. What's the best practiced way in dealing with this?
require 'yaml'
{
a: 'hello',
b: [{f:'hello',g:Hash.new,i:{a:'hello'}}],
c: {e:"+."}
}.to_yaml #=> `Float': invalid value for Float(): "+" (ArgumentError)
Full error dump:
/Users/XXX/.rvm/rubies/ruby-2.0.0-p0/lib/ruby/2.0.0/psych/scalar_scanner.rb:99:in `Float': invalid value for Float(): "+" (ArgumentError)
from /Users/XXX/.rvm/rubies/ruby-2.0.0-p0/lib/ruby/2.0.0/psych/scalar_scanner.rb:99:in `tokenize'
from /Users/XXX/.rvm/rubies/ruby-2.0.0-p0/lib/ruby/2.0.0/psych/visitors/yaml_tree.rb:272:in `visit_String'
from /Users/XXX/.rvm/rubies/ruby-2.0.0-p0/lib/ruby/2.0.0/psych/visitors/yaml_tree.rb:128:in `accept'
from /Users/XXX/.rvm/rubies/ruby-2.0.0-p0/lib/ruby/2.0.0/psych/visitors/yaml_tree.rb:324:in `block in visit_Hash'
from /Users/XXX/.rvm/rubies/ruby-2.0.0-p0/lib/ruby/2.0.0/psych/visitors/yaml_tree.rb:322:in `each'
from /Users/XXX/.rvm/rubies/ruby-2.0.0-p0/lib/ruby/2.0.0/psych/visitors/yaml_tree.rb:322:in `visit_Hash'
from /Users/XXX/.rvm/rubies/ruby-2.0.0-p0/lib/ruby/2.0.0/psych/visitors/yaml_tree.rb:128:in `accept'
from /Users/XXX/.rvm/rubies/ruby-2.0.0-p0/lib/ruby/2.0.0/psych/visitors/yaml_tree.rb:324:in `block in visit_Hash'
from /Users/XXX/.rvm/rubies/ruby-2.0.0-p0/lib/ruby/2.0.0/psych/visitors/yaml_tree.rb:322:in `each'
from /Users/XXX/.rvm/rubies/ruby-2.0.0-p0/lib/ruby/2.0.0/psych/visitors/yaml_tree.rb:322:in `visit_Hash'
from /Users/XXX/.rvm/rubies/ruby-2.0.0-p0/lib/ruby/2.0.0/psych/visitors/yaml_tree.rb:128:in `accept'
from /Users/XXX/.rvm/rubies/ruby-2.0.0-p0/lib/ruby/2.0.0/psych/visitors/yaml_tree.rb:92:in `push'
from /Users/XXX/.rvm/rubies/ruby-2.0.0-p0/lib/ruby/2.0.0/psych.rb:244:in `dump'
from /Users/XXX/.rvm/rubies/ruby-2.0.0-p0/lib/ruby/2.0.0/psych/core_ext.rb:14:in `psych_to_yaml'
This appears to be a bug in the bundled psych. Patching ~/.rvm/rubies/ruby-2.0.0-p0/lib/ruby/2.0.0/psych/scalar_scanner.rb at line 99 from:
Float(string.gsub(/[,_]|\.$/, ''))
to:
Float(string.gsub(/[,_]|\.$/, '')) rescue ArgumentError
fixes the issue. This is essentially what's in the psych gem as well as the Ruby 1.9 bundled version.
If you'd rather not patch your Ruby, using the psych-1.3.4 gem is another option; just be sure to require 'psych' rather than 'yaml':
gem 'psych', '=1.3.4'
require 'psych'
{a: 'hello', b: [{f:'hello',g:Hash.new,i:{a:'hello'}}], c: {e:"0+."}}.to_yaml
# => "---\n:a: hello\n:b:\n- :f: hello\n :g: {}\n :i:\n :a: hello\n:c:\n :e: 0+.\n"
This can be reproduced with a simpler example:
"+.".to_yaml
This appears to be a bug in the version of psych bundled with ruby 2.0.0 (and other versions, I'm sure):
when FLOAT
if string == '.'
#string_cache[string] = true
string
else
Float(string.gsub(/[,_]|\.$/, ''))
end
The problem is that that "+." looks like a valid floating point number like +.5.
This is fixed in Ruby 2.2.1 (or probably an earlier version), which specifically checks for the case where there may be a leading sign (+ or -):
when FLOAT
if string =~ /\A[-+]?\.\Z/
#string_cache[string] = true
string
else
Float(string.gsub(/[,_]|\.$/, ''))
end
Related
I'm using Ruby 2.4. How do I parse a tab-delimited line that contains a quote character? This is what's happening to me now ...
2.4.0 :003 > line = "11\tDave\tO\"malley"
=> "11\tDave\tO\"malley"
2.4.0 :004 > CSV.parse(line, col_sep: "\t")
CSV::MalformedCSVError: Illegal quoting in line 1.
from /Users/davea/.rvm/rubies/ruby-2.4.0/lib/ruby/2.4.0/csv.rb:1912:in `block (2 levels) in shift'
from /Users/davea/.rvm/rubies/ruby-2.4.0/lib/ruby/2.4.0/csv.rb:1868:in `each'
from /Users/davea/.rvm/rubies/ruby-2.4.0/lib/ruby/2.4.0/csv.rb:1868:in `block in shift'
from /Users/davea/.rvm/rubies/ruby-2.4.0/lib/ruby/2.4.0/csv.rb:1828:in `loop'
from /Users/davea/.rvm/rubies/ruby-2.4.0/lib/ruby/2.4.0/csv.rb:1828:in `shift'
from /Users/davea/.rvm/rubies/ruby-2.4.0/lib/ruby/2.4.0/csv.rb:1770:in `each'
from /Users/davea/.rvm/rubies/ruby-2.4.0/lib/ruby/2.4.0/csv.rb:1784:in `to_a'
from /Users/davea/.rvm/rubies/ruby-2.4.0/lib/ruby/2.4.0/csv.rb:1784:in `read'
from /Users/davea/.rvm/rubies/ruby-2.4.0/lib/ruby/2.4.0/csv.rb:1324:in `parse'
from (irb):4
from /Users/davea/.rvm/gems/ruby-2.4.0#global/gems/railties-5.0.1/lib/rails/commands/console.rb:65:in `start'
from /Users/davea/.rvm/gems/ruby-2.4.0#global/gems/railties-5.0.1/lib/rails/commands/console_helper.rb:9:in `start'
from /Users/davea/.rvm/gems/ruby-2.4.0#global/gems/railties-5.0.1/lib/rails/commands/commands_tasks.rb:78:in `console'
from /Users/davea/.rvm/gems/ruby-2.4.0#global/gems/railties-5.0.1/lib/rails/commands/commands_tasks.rb:49:in `run_command!'
from /Users/davea/.rvm/gems/ruby-2.4.0#global/gems/railties-5.0.1/lib/rails/commands.rb:18:in `<top (required)>'
from bin/rails:4:in `require'
from bin/rails:4:in `<main>'
Although teh example illustrates my point, I can't easily control the input coming in. So, although an answer coudl be< "Remove all quotes from teh string before parsing," I want to preserve the data as closely as possible.
That's a malformed document if you're trying to adhere to the CSV standard. Instad you might just brute-force it and pray there's no tabs in the data itself:
line.split(/\t/)
The CSV parsing library comes in handy when you're dealing with data like this:
"1\t2\t\"3a\t3b\"\t4"
Update: If you're prepared to abuse the CSV library a little then you can do this:
CSV.parse("11\tDave\tO\"malley", col_sep: "\t", quote_char: "\0")
That basically kills quote detection, so if there is other data that depends on that being processed correctly this may not work out.
"11\tDave\tO\"malley" is not valid CSV data. Strangely enough, the answer is to use two double-quotes, and to double quote each element
2.3.1 :001 > require 'csv'
=> true
2.3.1 :002 > line = "\"11\"\t\"Dave\"\t\"O\"\"malley\""
=> "\"11\"\t\"Dave\"\t\"O\"\"malley\""
2.3.1 :003 > puts line # for clarity
"11" "Dave" "O""malley"
=> nil
2.3.1 :004 > CSV.parse(line, col_sep: "\t")
=> [["11", "Dave", "O\"malley"]]
I have this YAML string:
---
lease: 2014-09-26 05:20:39.616606000 Z
createtime: 2014-09-20 05:20:39.616606000 Z
leased_at: 2014-09-23 00:22:37.052173269 +08:00
My ruby version is pretty old, but I cannot upgrade it, my version is:
ruby 1.9.1p376 (2009-12-07 revision 26041) [i686-linux]
When I try to use YAML.load to parse the above string:
YAML.load(str) # where str is above string
I got the following error:
irb(main):028:0> YAML.load(str)
RangeError: bignum too big to convert into `long'
from /build/toolchain/lin32/ruby-1.9.1-p376/lib/ruby/1.9.1/yaml.rb:133:in `utc'
from /build/toolchain/lin32/ruby-1.9.1-p376/lib/ruby/1.9.1/yaml.rb:133:in `node_import'
from /build/toolchain/lin32/ruby-1.9.1-p376/lib/ruby/1.9.1/yaml.rb:133:in `load'
from /build/toolchain/lin32/ruby-1.9.1-p376/lib/ruby/1.9.1/yaml.rb:133:in `load'
from (irb):28
I believe the problem is those times contain too long millisecond part. Given I cannot upgrade my ruby, how can I successfully parse about YAML string?
You can use regexp directly on your YAML string for match with your time pattern and truncate milliseconds.
More about regexp in ruby: http://ruby-doc.org/core-2.1.1/Regexp.html
I am creating a script to insert the files from folders to the Excel columns, but it seems I am doing wrong. Can any one help me for the same?
updated Ruby Code:
require 'fileutils'
require 'win32ole'
#Excel Application will be started from here.
#--------------------------------------------
excel = WIN32OLE.new('Excel.Application')
excel.visible = true
wb=excel.workbooks.open("E:\\WIPData\\Ruby\\Scripts\\Copy of GSL_File_DownLoad1.xlsx")
wbs= wb.Worksheets(1)
rows=2
column=2
until wbs.cells(rows,1).value == nil do
Dir.entries("E:\\WIPData\\Ruby").each do |f|
if f == wbs.cells(rows,1).value then
files_dir = File.expand_path("..", Dir.pwd)
column=2
Dir.foreach(files_dir.concat("/" + f)) do |x|
full_path=files_dir.concat("/" + x)
wbs.cells(rows,column).Select
wbs.oleobjects.add({
'Filename' => full_path,
'Link' => true,
'DisplayAsIcon' => false,
})
column = column + 1
end
break
end
end
end
wb.Save
wb.Close(0)
excel.Quit()
#Excel Application will be finished here.
#------------
Error:
E:/WIPData/Ruby/Scripts/test.rb:27:in `method_missing': (in OLE method `add': )
(WIN32OLERuntimeError)
OLE error code:800A03EC in Microsoft Excel
Cannot insert object.
HRESULT error code:0x80020009
Exception occurred.
from E:/WIPData/Ruby/Scripts/test.rb:27:in `block (2 levels) in <main>'
from E:/WIPData/Ruby/Scripts/test.rb:23:in `foreach'
from E:/WIPData/Ruby/Scripts/test.rb:23:in `block in <main>'
from E:/WIPData/Ruby/Scripts/test.rb:17:in `each'
from E:/WIPData/Ruby/Scripts/test.rb:17:in `<main>'
Your problem is on line 25 in your code. It is the method wbs.OLEObjects.Add(,full_path,False,True,,,f) that is causing the problem.
In VBA, it is perfectly fine to leave parameters to a method blank if they are not required. However, this is not available in Ruby.
In your original Macro, you passed keyword arguments to the method. One way of doing this in Ruby is with a Hash. An article on the Ruby on Windows blog suggests doing it like so:
wbs.oleobjects.add({
'Filename' => full_path,
'Link' => false,
'DisplayAsIcon' => true,
'IconIndex' => 0,
'IconLabel' => f,
'IconFileName' => icon_path
})
I did not see you provide an icon path in your Ruby code so I'm making an assumption on that final variable.
Also note that true and false are lowercase. Uppercase versions would be read by Ruby as either a constant or a class.
If you are doing work in Microsoft Office with Ruby, I would highly recommend frequenting Ruby on Windows. The author doesn't appear to post anymore but it is still a relevant source.
EDIT:
Your new error is probably due to Dir.entries. This method will grab . and .. when it pulls entries. I'd imagine Excel is tripping up on trying to add those two to the Worksheet.
There are two ways to remove this.
1) Skip them in your each block.
Dir.entries("E:\\WIPData\\Ruby").each do |f|
next if ['.', '..'].include? f
# The rest of your block code
end
2) Use Dir#glob which will not return . and ..
Dir.chdir("E:\\WIPData\\Ruby")
Dir.glob('*').each do |f|
# Your block code
end
EDIT:
For documentation's sake, this topic is also discussed on Ruby Forums.
[sagar#BL-53 RcTools]$ irb
1.9.3p0 :001 > require 'csv'
=> true
1.9.3p0 :002 > master = CSV.read("./public/jobs/in/Appexchange_Applications_Companies_487.csv")
ArgumentError: invalid byte sequence in UTF-8
from /home/sagar/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/csv.rb:1855:in `sub!'
from /home/sagar/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/csv.rb:1855:in `block in shift'
from /home/sagar/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/csv.rb:1849:in `loop'
from /home/sagar/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/csv.rb:1849:in `shift'
from /home/sagar/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/csv.rb:1791:in `each'
from /home/sagar/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/csv.rb:1805:in `to_a'
from /home/sagar/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/csv.rb:1805:in `read'
from /home/sagar/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/csv.rb:1411:in `block in read'
from /home/sagar/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/csv.rb:1354:in `open'
from /home/sagar/.rvm/rubies/ruby-1.9.3-p0/lib/ruby/1.9.1/csv.rb:1411:in `read'
from (irb):2
from /home/sagar/.rvm/rubies/ruby-1.9.3-p0/bin/irb:16:in `<main>'
1.9.3p0 :003 >
But when i do
1.9.3p0 :003 > master = CSV.open("./public/jobs/in/Appexchange_Applications_Companies_487.csv","r")
=> <#CSV io_type:File io_path:"./public/jobs/in/Appexchange_Applications_Companies_487.csv" encoding:UTF-8 lineno:0 col_sep:"," row_sep:"\r\n" quote_char:"\"">
1.9.3p0 :004 >
I just want to know why this is happening and what is solution. And i want to read the csv because it returns an array of that csv.
So if i read file in first way like
master = CSV.read("./public/jobs/in/Appexchange_Applications_Companies_487.csv")
It returns me an array
1.9.3p0 :008 > master.class
=> Array
But in second case, class is CSV.
What is solution to read csv in first way.
Regarding the error: First, make sure that you are using the correct character encodings. If you do, then you have probably invalid data in your csv file. You can probably fix it using iconv (see link posted by Chetan Muneshwar).
Regarding the second part of your question: CSV.open just opens the file for reading but does no reading yet. CSV.read will open the file, read its contents, and close it again. So to just get the data out of the file use CSV.read.
ruby-1.9.1-p243 :008 > a,b = IO.pipe
=> [#<IO:0x2010670>, #<IO:0x2010654>]
ruby-1.9.1-p243 :009 > a.ioctl(0x80000000, "\x00\x00")
Errno::ENOTTY: Inappropriate ioctl for device
from (irb):9:in `ioctl'
from (irb):9
from /Users/dlampa/.rvm/rubies/ruby-1.9.1-p243/bin/irb:16:in `<main>'
This would work if the IO object were an appropriate device and the command was valid.
ruby-1.9.3-p0 :022 > a,b = IO.pipe
=> [#<IO:fd 9>, #<IO:fd 10>]
ruby-1.9.3-p0 :023 > a.ioctl(0x80000000, "\x00\x00")
RangeError: bignum too big to convert into `long'
from (irb):23:in `ioctl'
from (irb):23
from /Users/dlampa/.rvm/rubies/ruby-1.9.3-p0/bin/irb:16:in `<main>'
On MacOSX 10.6.8
I'm assuming that this has something to do with bit signing (maybe), and have packed and unpacked things as signed and unsigned, but nothing seems to prevent 1.9.3 from throwing that range error. Of course, there's no problem in 64-bit implementations. I'm totally confused. Can someone explain why this is happening and perhaps provide a workaround?
Edit:
The command also works in 1.9.2-p290, so there was definitely a change from 1.9.2 to 1.9.3
ruby-1.9.2-p290 :001 > a,b = IO.pipe
=> [#<IO:fd 3>, #<IO:fd 4>]
ruby-1.9.2-p290 :002 > a.ioctl(0x80000000, "\x00\x00")
Errno::ENOTTY: Inappropriate ioctl for device
from (irb):2:in `ioctl'
from (irb):2
from /Users/dlampa/.rvm/rubies/ruby-1.9.2-p290/bin/irb:16:in `<main>'