Ruby Net::FTP gettextfile not able to save files locally - ruby

I am trying to retrieve files (.csv) from an ftp site and save them all locally in the same folder. My code looks like this:
#! /usr/bin/ruby
require 'logger'
require 'fileutils'
require 'net/ftp'
require 'rubygems'
require 'mysql2'
require 'roo'
require 'date'
# logging setup
log = Logger.new("/path_to_logs/ftp_log.log", 10, 1024000)
log.level = Logger::INFO
export_ftp_path = '/Receive/results/'
export_work_path ='/Users/pierce/results_exports/'
Net::FTP.open('host', 'username', 'password') do |ftp|
log.info("Logged into FTP")
ftp.passive = true
ftp.chdir("#{export_ftp_path}")
ftp.list.each do |file|
log.info("Found file #{file}")
new_file = file[56..115] #take part of the file name and remove spaces and periods
new_file = new_file.gsub(/[.]+/, "")
new_file = new_file.gsub(/\s/, "0")
ftp.gettextfile(file,"#{new_file}")
log.info("Downloaded file #{new_file}")
end
end
And here is the error I receive:
/Users/pierce/.rbenv/versions/1.9.2-p290/lib/ruby/1.9.1/net/ftp.rb:560:in `initialize': No such file or directory - (Errno::ENOENT)
from /Users/pierce/.rbenv/versions/1.9.2-p290/lib/ruby/1.9.1/net/ftp.rb:560:in `open'
from /Users/pierce/.rbenv/versions/1.9.2-p290/lib/ruby/1.9.1/net/ftp.rb:560:in `gettextfile'
from ftp_test.rb:44:in `block (2 levels) in <main>'
from ftp_test.rb:33:in `each'
from ftp_test.rb:33:in `block in <main>'
from /Users/pierce/.rbenv/versions/1.9.2-p290/lib/ruby/1.9.1/net/ftp.rb:116:in `open'
As suggested, here are the values I have for puts file and puts new_file.
file = -rwxr-xr-x 1 1130419 114727 9546 May 17 08:11 results_Wed. 16 May 2012.csv
new_file = results_Wed0230May02012csv
Any suggestions on what to change in gettextfile or within my script to get the files saved correctly?

You should use nlst instead of list when you just need a list of files in a directory. The output of list needs to be properly parsed otherwise.
When you request the file it has to be the original filename, including all spaces. When you save the file it can be anything you want (including spaces or not). The error was because you were requesting the wrong file. Use nlst in your case instead. It will make it much easier (no conversion or parsing needed).

Related

Ruby script can't find CSV file

I have a simple crawler written in Ruby that should crawl specific sites and save data into a CSV file, but, when running it from the Windows command line I get this error:
C:/Ruby24-x64/lib/ruby/2.4.0/csv.rb:1282:in `initialize': No such file or directory # rb_sysopen - csv/boxers.csv (Errno::ENOENT)
from C:/Ruby24-x64/lib/ruby/2.4.0/csv.rb:1282:in `open'
from C:/Ruby24-x64/lib/ruby/2.4.0/csv.rb:1282:in `open'
from boxers.rb:18:in `<main>'
This is the script:
#!/usr/bin/env ruby
require 'csv'
require 'mechanize'
agent = Mechanize.new{ |agent| agent.history.max_size=0 }
agent.user_agent = 'Mozilla/5.0'
base = "http://baseurl.com/"
division = ARGV[0]
search_url = "http://baseurl.com/ratings.php?sex=M&division=#{division}&pageID="
path='//*[#id="mainContent"]/table/tr[position()>2]'
boxers = CSV.open("csv/file.csv","w")
url = search_url+"1"
begin
page = agent.get(url)
rescue
print " -> error, retrying\n"
retry
end
end
boxers.close

Ruby Filename error?

I forked this gist from https://gist.github.com/mattdipasquale/571405
I am getting the following error:
deDUPER.rb:14:in `read': Invalid argument - /Volumes/Drobo #1 2009-2012/AMNH Video/2012/2012-01-17 Creatures of Light/Capture Scratch/Art 3:9/Capture Scratch/2012-03-09_microraptor livestream/A Cam_Microraptor livestream.mov (Errno::EINVAL)
from deDUPER.rb:14:in `block in <main>'
from deDUPER.rb:10:in `each'
from deDUPER.rb:10:in `<main>'
I think it is caused from illegal characters in the file or folder names, but i'm not sure. I don't want to change the file or folder names because they are linked to old Final Cut Pro project files that rely on referenced filepaths to keep the project intact. Does anyone have experience with this? Is there a way I can get this script to work without having to change the file or folder names?
# Define the unique method that removes duplicates
#!/usr/bin/ruby
require 'digest/md5'
library_path = ARGV[0]
hash = {}
Dir.glob(library_path + "/**/*", File::FNM_DOTMATCH).each do |filename|
next if File.directory?(filename)
puts 'Checking ' + filename
key = Digest::MD5.hexdigest(IO.read(filename)).to_sym
if hash.has_key? key
# puts "same file #{filename}"
hash[key].push filename
else
hash[key] = [filename]
end
end
hash.each_value do |filename_array|
if filename_array.length > 1
puts "=== Identical Files ===\n"
filename_array.each { |filename| puts ' '+filename }
end
end

File.rename not working on Windows

Here's the code:
files = Dir.glob("*")
files.each do |file|
if File.extname(file) == ".pdf"
format = file.split(".pdf")
format = format.join("").split(" ")
format[0] = format[0].gsub(".","/")
format[0] << "_"
format[0].prepend("_")
format[-1] << ".pdf"
format = format.join("")
puts "Changed #{file} to #{format}"
File.rename(file,format)
end
end
It is running from the same directory as the files. I have tried giving the File.rename parameters absolute paths as well, by appending the result of Dir.pwd to them. Code isn't very clean as it was something I whipped up real quick, please excuse that.
I get the following error:
Changed 05.01.14 Mid-Day 1.pdf to _05/01/14_Mid-Day1.pdf
script.rb:12:in `rename': No such file or directory - (05.01.14 Mid-Day 1.pdf, _05/01/14_Mid-Day1.pdf) (Errno::ENOENT)
from script.rb:12:in `block in <main>'
from script.rb:2:in `each'
from script.rb:2:in `<main>'
If I call File.exists?(file) it comes back as true. I'm thoroughly confused on why this doesn't work.
I am on running this on Windows.
The directory structure _05/01 does not seem to exist. You need to create the directories first with FileUtils.mkdir_p before being able to move the file there.

ruby - zlib header error when uncompressing tar.gz

I wanted to write a ruby program to unpackage tar.gz files, and I ran into some issues. After reading the documentation on the ruby-doc site, Zlib::GzipReader and Zlib::Inflate. Then I found this Module someone wrote on GitHub, and that didn't work either. So, using the examples from the Ruby page for Zlib::GzipReader, I these were able to run successfully.
irb(main):027:0* File.open('zlib_inflate.tar.gz') do |f|
irb(main):028:1* gz = Zlib::GzipReader.new(f)
irb(main):029:1> print gz.read
irb(main):030:1> gz.close
irb(main):031:1> end
#
#
#
irb(main):023:0* Zlib::GzipReader.open('zlib_inflate.tar.gz') { |gz|
irb(main):024:1* print gz.read
irb(main):025:1> }
Then, when trying to use the Zlib::Inflate options, I kept running into incorrect header check errors.
irb(main):047:0* zstream = Zlib::Inflate.new
=> #<Zlib::Inflate:0x00000002b15790 #dictionaries={}>
irb(main):048:0> buf = zstream.inflate('zlib_inflate.tar.gz')
Zlib::DataError: incorrect header check
from (irb):48:in `inflate'
from (irb):48
from C:/ruby/Ruby200p353/Ruby200-x64/bin/irb:12:in `<main>'
#
#
#
irb(main):049:0> cf = File.open("zlib_inflate.tar.gz")
=> #<File:zlib_inflate.tar.gz>
irb(main):050:0> ucf = File.open("ruby_inflate.rb", "w+")
=> #<File:ruby_inflate.rb>
irb(main):051:0> zi = Zlib::Inflate.new(Zlib::MAX_WBITS)
=> #<Zlib::Inflate:0x00000002c097f0 #dictionaries={}>
irb(main):052:0> ucf << zi.inflate(cf.read)
Zlib::DataError: incorrect header check
from (irb):52:in `inflate'
from (irb):52
from C:/ruby/Ruby200p353/Ruby200-x64/bin/irb:12:in `<main>'
#
#
#
irb(main):057:0* File.open("zlib_inflate.tar.gz") {|cf|
irb(main):058:1* zi = Zlib::Inflate.new
irb(main):059:1> File.open("zlib_inflate.rb", "w+") {|ucf|
irb(main):060:2* ucf << zi.inflate(cf.read)
irb(main):061:2> }
irb(main):062:1> zi.close
irb(main):063:1> }
Zlib::DataError: incorrect header check
from (irb):60:in `inflate'
from (irb):60:in `block (2 levels) in irb_binding'
from (irb):59:in `open'
from (irb):59:in `block in irb_binding'
from (irb):57:in `open'
from (irb):57
from C:/ruby/Ruby200p353/Ruby200-x64/bin/irb:12:in `<main>'
How would I go about taking a tar.gz file, and extract the contents so the files are not null? I did read this other post about Ruby Zlib from here, but that did not work for me as well. What am I doing wrong?

Ruby Spreadsheet: Bad file descriptor - test.xls (Errno::EBADF)

I have problem with script that makes simple .xls file and writes data to one cell. Here is simple code:
require 'spreadsheet'
class Filter
def filter
#excel = Spreadsheet::Workbook.new
#sheet = #excel.create_worksheet
#sheet[0, 0] = "test"
#excel.write 'test.xls'
end
end
f = Filter.new
f.filter
But it raises error:
C:/Ruby193/lib/ruby/gems/1.9.1/gems/ruby-ole-1.2.11.5/lib/ole/storage/base.rb:62:in
write_nonblock': Bad file descriptor - test.xls (Errno::EBADF)
from C:/Ruby193/lib/ruby/gems/1.9.1/gems/ruby-ole-1.2.11.5/lib/ole/storage/base.rb:62:in
initialize'
from C:/Ruby193/lib/ruby/gems/1.9.1/gems/ruby-ole-1.2.11.5/lib/ole/storage/base.rb:78:in
new'
from C:/Ruby193/lib/ruby/gems/1.9.1/gems/ruby-ole-1.2.11.5/lib/ole/storage/base.rb:78:in
open'
from C:/Ruby193/lib/ruby/gems/1.9.1/gems/spreadsheet-0.7.4/lib/spreadsheet/excel/writer/workbook.rb:4
53:in write_from_scratch'
from C:/Ruby193/lib/ruby/gems/1.9.1/gems/spreadsheet-0.7.4/lib/spreadsheet/excel/writer/workbook.rb:6
31:inwrite_workbook'
from C:/Ruby193/lib/ruby/gems/1.9.1/gems/spreadsheet-0.7.4/lib/spreadsheet/writer.rb:15:in
block in write'
from C:/Ruby193/lib/ruby/gems/1.9.1/gems/spreadsheet-0.7.4/lib/spreadsheet/writer.rb:14:in
open'
from C:/Ruby193/lib/ruby/gems/1.9.1/gems/spreadsheet-0.7.4/lib/spreadsheet/writer.rb:14:in
write'
from C:/Ruby193/lib/ruby/gems/1.9.1/gems/spreadsheet-0.7.4/lib/spreadsheet/workbook.rb:116:in
write'
from filter.rb:10:in `filter'
from filter.rb:15:in `<main>'
because ruby-ole 1.2.11.5 doesn't support windows platform,
more detail: ruby-ole issue
you can use ruby-ole 1.2.11.4 to avoid this problem.
require 'rubygems'
gem 'ruby-ole','1.2.11.4'
require 'spreadsheet'
I've seen these before. First verify that you can write to that file's location.
My guess is either the file is already open in Excel or your antivirus is blocking the 'threat'.

Resources