How to copy files and sub-folders - ruby

I'm trying to copy files including sub-folders, but I'm getting a RunTimeError copy: unknown file type.
I also created folders and files to test myself.
Folders and files for test:
- D:
- backup_target
- target1
- file1.txt
- sub1
- sub_file1.txt
- sub_file2.txt
- sub2
- sub1.txt
- sub.txt
- target2
- file1.txt
- file.txt
And I made a list as a text file in "backup_target_list2.txt":
D:\backup_target\target1
D:\backup_target\target2
My code is:
require 'fileutils'
LIST_FILE = "backup_target_list2.txt"
def copy_files(src, dst)
FileUtils.mkdir_p(File.dirname(dst))
FileUtils.copy_entry(src, dst)
end
def dst_naming(src, src_head, dst_head)
cur_date = Time.now.to_s[0..9]
dst = src.gsub(src_head, dst_head + "/backup_#{cur_date}/")
return dst
end
def get_backup_list(list_file)
if !File.exist?(list_file) then
return nil
end
path_arr = []
File.open(list_file, "r") do |f|
f.each_line { |line|
path_arr.push(replace_delim(line).gsub("\n", ""))
}
end
return path_arr
end
def replace_delim(str_obj, delim_org= "\\", delim_new = "/")
if str_obj.class == Array
str_arr = []
str_obj.each do |str|
str_arr.push(str.gsub! delim_org, delim_new)
end
return str_arr
else
return str_obj.gsub! delim_org, delim_new
end
end
get_backup_list(LIST_FILE).each do |path|
src = path
dst = dst_naming(src, /D:\//, "C:/Users/MyName/Desktop/")
print "src: #{src}\n=> dst: #{dst}\n"
copy_files(src, dst)
end
It works fine with the test folders.
The problem is, I'm getting a RunTimeError when I run the code with my real folders:
C:/Ruby200/lib/ruby/2.0.0/fileutils.rb:1375:in `copy': unknown file type: D:/MyRealFolder (RuntimeError)
from C:/Ruby200/lib/ruby/2.0.0/fileutils.rb:463:in `block in copy_entry'
from C:/Ruby200/lib/ruby/2.0.0/fileutils.rb:1485:in `call'
from C:/Ruby200/lib/ruby/2.0.0/fileutils.rb:1485:in `wrap_traverse'
from C:/Ruby200/lib/ruby/2.0.0/fileutils.rb:460:in `copy_entry'
from copy_test.rb:8:in `copy_files'
from copy_test.rb:52:in `block in <main>'
from copy_test.rb:48:in `each'
from copy_test.rb:48:in `<main>'
The difference between "real" and "test" is that "real" folders have more sub-folders and more files.
I also tried FileUtils.cp_r and FileUtils.cp but I'm still getting an error.
How can I fix it?

Related

Method to call other methods multiple times using hash length as iterator

I have 2 methods that I want to call multiple times. The number of times that I want to call them is based on the count of a hash I'm using. I'm trying to create a new method that calls the other 2 and repeats for the length of the hash count. My issue is that I'm getting an error
"findfiles2.rb:61:in` `chdir': no implicit conversion of Enumerator into String (TypeError)
from findfiles2.rb:61:in `store_directories'
from findfiles2.rb:138:in `block in repeat'
from findfiles2.rb:134:in `loop'
from findfiles2.rb:134:in `repeat'
from findfiles2.rb:153:in `<main>'"
Here's my code:
require 'date'
require "mail"
options = { :address => "smtp.gmail.com",
:port => 587,
:domain => 'gmail.com',
:user_name => 'username',
:password => 'password/',
:authentication => 'plain',
:enable_starttls_auto => true }
mail_sender = "somename#gmail.com"
mail_recipient = "somename#yahoo.com"
directories = {
"directory1" => "/path/to/folder1/",
"directory2" => "/path/to/folder2/",
"directory3" => "/path/to/folder3/",
"directory4" => "/path/to/folder4/",
"directory5" => "/path/to/folder5/"
}
directory_count = directories.count.to_i
file_output = "/path/to/output/"
exclude_folder = 'sample'
output_file_name = "directory_list"
output_file_extension = ".csv"
date_today = Date.today.to_s
log_file_path = "/path/to/output/"
log_name = "script_log_" + date_today + ".txt"
log_file_name = log_file_path + log_name
# starts log file
def start_log(file_output, log_name)
Dir.chdir(file_output)
log_output = File.open(log_name, 'a+')
$stdout = log_output
puts Time.now.to_s + " > " + "Starting Script..."
puts "_______________________________________________"
end
# stores subdirectory contents into an array
def store_directories(directory, folder_to_exclude)
# changes working directory to the directory variable
puts Time.now.to_s + " > " + "Updating search directory..."
Dir.chdir(directory)
# outputs only subdirectories with a creation date of older than 24 hours, except for folders names 'test'
Dir.glob("*.*").map(&File.method(:realpath))
puts Time.now.to_s + " > " + "Gathering subdirectories..."
subdir_list=Dir.glob("*").map(&File.method(:realpath)).reject{|files| (not File.directory?(files) && (File.mtime(files) < (Time.now - (60*1440))) && (not files == directory + folder_to_exclude)) }
return subdir_list
end
# checks to see if there are any directories in the array
def directory_check(directory_list, save_to_file, today_date, output_file, output_extension)
if directory_list.empty? == false
# changes the working directory to the file output directory for the file
Dir.chdir(save_to_file)
# writes the array contents into a new file
file_name = output_file + "_" + today_date + output_extension
puts Time.now.to_s + " > " + "Saving contents to: " + file_name
File.open(file_name, "a+") do |f|
directory_list.each { |element| f.puts(element) }
end
else
puts Time.now.to_s + " > " + "This directory does not contain any subdirectories that are older than 24 hours"
exit
end
end
# sends an email containing today's report if a file was created today
def send_email(today_date, output_file_path, output_file_name, output_file_extension, mail_options, email_sender, email_recipient)
backlog_file = output_file_path + output_file_name + "_" + today_date + output_file_extension
if File.exist?(backlog_file) == true
puts Time.now.to_s + " > " + "Sending email report to: " + email_recipient + "..."
Mail.defaults do
delivery_method :smtp, mail_options
end
Mail.deliver do
to email_recipient
from email_sender
subject 'Backlog for ' + today_date
body 'Attached is a report showing any batches that have not been processed within the last 24 hours.'
add_file backlog_file
end
else
puts Time.now.to_s + " > " + "No batches older than 24 hours to report"
exit
end
end
This is the method that is giving me trouble
def repeat(directory, times, exclude_folder)
# fail "times must be 1 or more" if times < 1
counter = 1
# counter_string = counter.to_s
# puts counter_string
# directory_counter = directory + counter_string
loop do
if counter != times
subdir_list_contents = store_directories(directory, exclude_folder)
directory_check(subdir_list_contents, file_output, date_today, output_file_name, output_file_extension)
counter = counter + 1
else
break
end
end
end
This is where I'm starting to run everything.
# Starting log file...
start_log(file_output, log_name)
repeat(directories.each, directory_count, exclude_folder)
# # outputs contents of directory 1 to the file (I want to perform this for the amount of times equal to the hash length, which is what I'm creating the repeat method for)
subdir_list_contents = store_directories(directory1, exclude_folder)
directory_check(subdir_list_contents, file_output, date_today, output_file_name, output_file_extension)
# # # If there is a new file from today, sends an email with file as attachment
send_email(date_today, file_output, output_file_name, output_file_extension, options, mail_sender, mail_recipient)
Your code is much too long. As you can see, nobody helps you.
$ ruby -w t.rb
t.rb:125: warning: mismatched indentations at 'end' with 'def' at 104
t.rb:37: warning: assigned but unused variable - log_file_name
/Users/b/.rvm/rubies/ruby-2.4.0-rc1/lib/ruby/site_ruby/2.4.0/rubygems/core_ext/kernel_require.rb:55:
in `require': cannot load such file -- mail (LoadError)
Post code that we can run. I'm not going to install a whole mail server. So I comment out
#require "mail"
then
t.rb:44:in `chdir': No such file or directory # dir_chdir - /path/to/output/ (Errno::ENOENT)
etc, etc.
Follow the error message :
t.rb:62:in `chdir': no implicit conversion of Enumerator into String (TypeError)
from t.rb:62:in `store_directories'
from t.rb:139:in `block in repeat'
from t.rb:136:in `loop'
from t.rb:136:in `repeat'
from t.rb:149:in `<main>'
All what you need to trace is repeat and store_directories, so you could have reduced the posted code to this strict minimum to reproduce the error :
directories = {
"directory1" => "/path/to/folder1/",
"directory2" => "/path/to/folder2/",
}
directory_count = directories.count.to_i
exclude_folder = 'sample'
# stores subdirectory contents into an array
def store_directories(directory, folder_to_exclude)
puts "directory=#{directory.inspect} folder_to_exclude=#{folder_to_exclude}"
Dir.chdir(directory)
end
def repeat(directory, times, exclude_folder)
store_directories(directory, exclude_folder)
end
repeat(directories.each, directory_count, exclude_folder)
Execution :
$ ruby -w t.rb
directory=#<Enumerator: {"directory1"=>"/path/to/folder1/", "directory2"=>"/path/to/folder2/"}:each> folder_to_exclude=sample
t.rb:12:in `chdir': no implicit conversion of Enumerator into String (TypeError)
from t.rb:12:in `store_directories'
from t.rb:16:in `repeat'
from t.rb:19:in `<main>'
Had you done this, you don't even had to post a question, because the cause of the error is obvious. In
repeat(directories.each, directory_count, exclude_folder)
directories.each returns an Enumerator. Remove each :
$ ruby -w t.rb
directory={"directory1"=>"/path/to/folder1/", "directory2"=>"/path/to/folder2/"} folder_to_exclude=sample
t.rb:12:in `chdir': no implicit conversion of Hash into String (TypeError)
from t.rb:12:in `store_directories'
from t.rb:16:in `repeat'
from t.rb:20:in `<main>'
I suppose that what you wanted to do is call repeat once for each directory :
def repeat(directory, exclude_folder)
puts "in repeat directory=#{directory} exclude_folder=#{exclude_folder}"
store_directories(directory, exclude_folder)
end
directories.each { | _key, directory | repeat(directory, exclude_folder) }
Execution :
$ ruby -w t.rb
in repeat directory=dir1 exclude_folder=sample
in store_directories directory="dir1" folder_to_exclude=sample
in repeat directory=dir2 exclude_folder=sample
in store_directories directory="dir2" folder_to_exclude=sample
t.rb:18:in `chdir': No such file or directory # dir_chdir - dir2 (Errno::ENOENT)
from t.rb:18:in `store_directories'
from t.rb:53:in `repeat'
chdir has a side effect : it changes the current directory, and the next time, it will start searching in the new current directory. To avoid this, you need to restore the previous state :
def store_directories(directory, folder_to_exclude)
puts "in store_directories directory=#{directory.inspect} folder_to_exclude=#{folder_to_exclude}"
current_directory = Dir.getwd
Dir.chdir(directory)
# ...
# Restore the directory that was current when entering the method.
# Without it, the next chdir will start from the directory left by the previous chdir.
Dir.chdir(current_directory)
end
Now it works :
$ ruby -w t.rb
in repeat directory=dir1 exclude_folder=sample
in store_directories directory="dir1" folder_to_exclude=sample
in repeat directory=dir2 exclude_folder=sample
in store_directories directory="dir2" folder_to_exclude=sample
After a few more changes, I end up with this :
require 'date'
directories = {
"directory1" => 'dir1',
"directory2" => 'dir2'
}
exclude_folder = 'sample'
#file_output = '.'
#date_today = Date.today.to_s
#output_file_name = 'directory_list'
#output_file_extension = '.csv'
# stores subdirectory contents into an array
def store_directories(directory, folder_to_exclude)
puts "in store_directories directory=#{directory.inspect} folder_to_exclude=#{folder_to_exclude}"
current_directory = Dir.getwd
puts Time.now.to_s + " > " + "Updating search directory..."
# changes working directory to the directory variable
Dir.chdir(directory)
# outputs only subdirectories with a creation date of older than 24 hours, except for folders names 'test'
puts Time.now.to_s + " > " + "Gathering subdirectories..."
subdir_list = Dir.glob("*").map { | file | File.realpath(file) }
puts "all files : subdir_list=#{subdir_list}"
puts "directory + folder_to_exclude=#{directory + folder_to_exclude}" # nonsense
subdir_list = subdir_list.reject do | file |
not File.directory?(file) \
&& File.mtime(file) < Time.now - 86400 \
&& (not file == folder_to_exclude)
end
puts "after reject : subdir_list=#{subdir_list}"
# Restore the directory that was current when entering the method.
# Without it, the next chdir will start from the directory left by the previous chdir.
Dir.chdir(current_directory)
puts "subdir_list=#{subdir_list.inspect}"
subdir_list
end
# checks to see if there are any directories in the array
def directory_check(directory_list, save_to_file, today_date, output_file, output_extension)
if directory_list.empty? == false
# changes the working directory to the file output directory for the file
Dir.chdir(save_to_file) # <----------------- problem !!!!
# writes the array contents into a new file
file_name = output_file + "_" + today_date + output_extension
puts Time.now.to_s + " > " + "Saving contents to: " + file_name
File.open(file_name, "a+") do |f|
directory_list.each { |element| f.puts(element) }
end
else
puts Time.now.to_s + " > " + "This directory does not contain any subdirectories that are older than 24 hours"
end
end
def repeat(directory, exclude_folder)
puts "in repeat directory=#{directory} exclude_folder=#{exclude_folder}"
subdir_list_contents = store_directories(directory, exclude_folder)
directory_check(subdir_list_contents, #file_output, #date_today, #output_file_name, #output_file_extension)
end
directories.each { | _key, directory | repeat(directory, exclude_folder) }
Execution :
$ ruby -w t.rb
in repeat directory=dir1 exclude_folder=sample
in store_directories directory="dir1" folder_to_exclude=sample
2017-10-27 08:05:24 +0200 > Updating search directory...
2017-10-27 08:05:24 +0200 > Gathering subdirectories...
all files : subdir_list=["/userdata/devl/ruby/zintlist/directories/dir1/x1.txt", "/userdata/devl/ruby/zintlist/directories/dir1/x2.txt"]
directory + folder_to_exclude=dir1sample
after reject : subdir_list=[]
subdir_list=[]
2017-10-27 08:05:24 +0200 > This directory does not contain any subdirectories that are older than 24 hours
in repeat directory=dir2 exclude_folder=sample
in store_directories directory="dir2" folder_to_exclude=sample
2017-10-27 08:05:24 +0200 > Updating search directory...
2017-10-27 08:05:24 +0200 > Gathering subdirectories...
all files : subdir_list=["/userdata/devl/ruby/zintlist/directories/dir2/x3.txt"]
directory + folder_to_exclude=dir2sample
after reject : subdir_list=[]
subdir_list=[]
2017-10-27 08:05:24 +0200 > This directory does not contain any subdirectories that are older than 24 hours

encoding error when calling FileUtils.copy_entry in Ruby

I'm writing code for copy and paste recursively.
But I got an encoding error when calling FileUtils.copy_entry
Error Messages :
C:/Ruby200/lib/ruby/2.0.0/fileutils.rb:1535:in `join': incompatible character encodings: CP949 and UTF-8 (Encoding::CompatibilityError)
from C:/Ruby200/lib/ruby/2.0.0/fileutils.rb:1535:in `join'
from C:/Ruby200/lib/ruby/2.0.0/fileutils.rb:1218:in `path'
from C:/Ruby200/lib/ruby/2.0.0/fileutils.rb:463:in `block in copy_entry'
from C:/Ruby200/lib/ruby/2.0.0/fileutils.rb:1485:in `call'
from C:/Ruby200/lib/ruby/2.0.0/fileutils.rb:1485:in `wrap_traverse'
from C:/Ruby200/lib/ruby/2.0.0/fileutils.rb:1488:in `block in wrap_traverse'
from C:/Ruby200/lib/ruby/2.0.0/fileutils.rb:1487:in `each'
from C:/Ruby200/lib/ruby/2.0.0/fileutils.rb:1487:in `wrap_traverse'
from C:/Ruby200/lib/ruby/2.0.0/fileutils.rb:1488:in `block in wrap_traverse'
from C:/Ruby200/lib/ruby/2.0.0/fileutils.rb:1487:in `each'
from C:/Ruby200/lib/ruby/2.0.0/fileutils.rb:1487:in `wrap_traverse'
from C:/Ruby200/lib/ruby/2.0.0/fileutils.rb:460:in `copy_entry'
from C:/Ruby200/lib/ruby/2.0.0/fileutils.rb:435:in `block in cp_r'
from C:/Ruby200/lib/ruby/2.0.0/fileutils.rb:1558:in `block in fu_each_src_dest'
from C:/Ruby200/lib/ruby/2.0.0/fileutils.rb:1574:in `fu_each_src_dest0'
from C:/Ruby200/lib/ruby/2.0.0/fileutils.rb:1556:in `fu_each_src_dest'
from C:/Ruby200/lib/ruby/2.0.0/fileutils.rb:434:in `cp_r'
from copy_test.rb:11:in `copy_files'
from copy_test.rb:81:in `block in <main>'
from copy_test.rb:76:in `each'
from copy_test.rb:76:in `<main>'
I'm calling copy_entry like this.
def copy_files(src, dst)
FileUtils.mkdir_p(File.dirname(dst))
FileUtils.copy_entry(src, dst)
end
There are some sub-folders and files named with my local language in src.
So I think (Encoding::CompatibilityError) occurs because of these sub-folders and files with local language (not English).
When I tested with only Enlgish folders and files, It worked.
But, I need non-English folders and files too.
How can I solve this problem?
Should I define a new method replacing copy_entry?
My Codes ADDED:
# -*- encoding: cp949 -*-
require 'fileutils'
LIST_FILE = "backup_target_list.txt"
DEST_FILE = "backup_dest.txt"
def copy_files(src, dst)
FileUtils.mkdir_p(File.dirname(dst))
FileUtils.copy_entry(src, dst)
end
def get_dst_base_name()
cur_date = Time.now.to_s[0..9]
return "backup_#{cur_date}"
end
def get_backup_list(list_file)
if !File.exist?(list_file) then
return nil
end
path_arr = []
File.open(list_file, "r") do |f|
f.each_line { |line|
path_arr.push(make_path(line.gsub("\n", "")))
}
end
return path_arr
end
def get_dest(dest_file)
if !File.exist?(dest_file) then
return nil
end
return File.open(dest_file, "r").readline
end
def make_path(*str)
path_new = nil
str.each do |item|
if item.class == Array then
path_new = (path_new == nil ? File.join(item) : File.join(path_new, item))
else
if item.include?(File::ALT_SEPARATOR) then
path_new = (path_new == nil ? File.join(item.split(File::ALT_SEPARATOR)) : File.join(path_new, item.split(File::ALT_SEPARATOR)))
else
path_new = (path_new == nil ? File.join(item) : File.join(path_new, item))
end
end
end
return path_new
end
get_backup_list(LIST_FILE).each do |path|
src = path
tmp = src.split(File::SEPARATOR)
dst = make_path(get_dest(DEST_FILE), get_dst_base_name, tmp[1..tmp.length])
print "src: #{src}\n=> dst: #{dst}\n"
copy_files(src, dst)
end
I saw you are using cp949 encoding, and the raised error tells CP949 and UTF-8 are incompatible, so why you just using UTF-8 encoding? So replace the shebang with
# coding: UTF-8
And to ensure all the characters read from LIST_FILE are utf-8 encoded, add the following:
line.force_encoding('utf-8')

Getting error while opening file , i have created folder with book_name > chapter_name in code and then creating file

In child folder(chapter_number), i am creating file
#!/usr/bin/env ruby
require 'roo'
Dir.glob("**/*.xlsx") do |file|
xlsx = Roo::Spreadsheet.open(file)
bookname = xlsx.column(1)
cahpter_number_array = xlsx.column(2).uniq
cahpter_number_array.each do |chapter|
book_name = bookname[1] if bookname
chapter_number = chapter if (cahpter_number_array && (chapter != "Chapter"))
Dir.mkdir(book_name) unless File.exists?(book_name)
Dir.mkdir("#{book_name}/#{chapter_number}") unless File.exists?("#{book_name}/#{chapter_number}")
xlsx.column(3).each do |md|
output_name = "#{book_name}/#{chapter_number}/#{File.basename(md.partition('-').first, '.*')}.md" if (md != "Verse")
output = File.open("#{output_name}", 'w')
output << "hello"
end
end
end
Error:
`initialize': Is a directory # rb_sysopen - . (Errno::EISDIR)
Below link is my source file:
source file link
Come on now, that isn't really your code. You can't call partition on a number:
file_name = [1,2,3,4,5,6,7]
file_name.each do |md|
... md.partition('-')
so you would have gotten an error for that before getting the error you posted.
In any case, the error message is saying that outputname is set equal to "." and when ruby tries to execute File.open(".", 'w') ruby finds that "." is the name of a directory on your system, and you can't write a directory. You can witness the same error doing this:
~/ruby_programs$ mkdir my_dir
~/ruby_programs$ irb
2.3.0 :001 > File.open('my_dir', 'w')
Errno::EISDIR: Is a directory # rb_sysopen - my_dir
from (irb):1:in `initialize'
from (irb):1:in `open'
from (irb):1
from /Users/7stud/.rvm/rubies/ruby-2.3.0/bin/irb:11:in `<main>'

Ruby Filename error?

I forked this gist from https://gist.github.com/mattdipasquale/571405
I am getting the following error:
deDUPER.rb:14:in `read': Invalid argument - /Volumes/Drobo #1 2009-2012/AMNH Video/2012/2012-01-17 Creatures of Light/Capture Scratch/Art 3:9/Capture Scratch/2012-03-09_microraptor livestream/A Cam_Microraptor livestream.mov (Errno::EINVAL)
from deDUPER.rb:14:in `block in <main>'
from deDUPER.rb:10:in `each'
from deDUPER.rb:10:in `<main>'
I think it is caused from illegal characters in the file or folder names, but i'm not sure. I don't want to change the file or folder names because they are linked to old Final Cut Pro project files that rely on referenced filepaths to keep the project intact. Does anyone have experience with this? Is there a way I can get this script to work without having to change the file or folder names?
# Define the unique method that removes duplicates
#!/usr/bin/ruby
require 'digest/md5'
library_path = ARGV[0]
hash = {}
Dir.glob(library_path + "/**/*", File::FNM_DOTMATCH).each do |filename|
next if File.directory?(filename)
puts 'Checking ' + filename
key = Digest::MD5.hexdigest(IO.read(filename)).to_sym
if hash.has_key? key
# puts "same file #{filename}"
hash[key].push filename
else
hash[key] = [filename]
end
end
hash.each_value do |filename_array|
if filename_array.length > 1
puts "=== Identical Files ===\n"
filename_array.each { |filename| puts ' '+filename }
end
end

Errno::EISDIR in 'initialize' : Is a directory # rb_sysopen - persistent.bak (Errno::EISDIR)

I am iterating through files in a folder to search for specific string.
There is a folder name as persistent.bak. While going through this folder, it is giving error... in 'initialize' : Is a directory # rb_sysopen - persistent.bak (Errno::EISDIR).
Dir.glob("**/*.*") do |file_name|
fileSdfInput = File.open(file_name)
fileSdfInput.each_line do |line|
if ((line.include?"DATE")
#count = #count + 1
end
end
end
your glob Dir.glob("**/*.*") matches the pattern persistent.bak
So inside your loop, you're actually trying to open the folder named persistent.bak as a file, which ruby doesn't appreciate.
Just to convince yourself, try to output the file name, you'll see it.
Simplest workaround :
Dir.glob("**/*.*") do |file|
next if File.directory? file
fileSdfInput = File.open(file)
fileSdfInput.each_line do |line|
if (line.include?"DATE")
#count = #count + 1
end
end
end

Resources