I am using code like the following to unzip files in Ruby:
def unzip_file (file)
Zip::ZipFile.open(file) do |zip_file|
zip_file.each do |f|
puts f.name if f.file?
end
end
end
I would like to ignore all files generated by compress zip in Mac such as: .DS_Store, etc. How can I best do it?
I believe that this does what you want:
Zip::ZipFile.open(file) do |zip_file|
names = zip_file.select(&:file?).map(&:name)
names.reject!{|n| n=~ /\.DS_Store|__MACOSX|(^|\/)\._/ }
puts names
end
That regular expression says,
Throw away files
that have .DS_Store in the name,
that have __MACOSX in the name,
or that have ._ at the beginning of the name (^) or right after a /.
That should cover all the 'junk' files and hopefully not hit any others.
If you want more than just the names—if you want to process the non-junk files—then instead you might do the following:
Zip::ZipFile.open(file) do |zip_file|
files = zip_file.select(&:file?)
files.reject!{|f| f.name =~ /\.DS_Store|__MACOSX|(^|\/)\._/ }
puts files.map(&:names) # or do whatever else you want with the array of files
end
Related
I need to open each file inside a directory. My attempt at this looks like:
Dir.foreach('path/to/directory') do |filename|
next if filename == '.' || filename == '..'
puts "working on #{filename}"
# this is where it crashes
file = File.open(filename, 'r')
#some code
file.close
# more code
end
My code keeps crashing at File.open(filename, 'r'). I'm not sure what filename should be.
The filename should include the path to the file when the file is not in the same directory than the Ruby file itself:
path = 'path/to/directory'
Dir.foreach(path) do |filename|
next if filename == '.' || filename == '..'
puts "working on #{filename}"
file = File.open("#{path}/#{filename}", 'r')
#some code
file.close
# more code
end
I recommend using Find.find.
While we can use various methods from the Dir class, it will look and retrieve the list of files before returning, which can be costly if we're recursively searching multiple directories or have a huge number of files embedded in the directories.
Instead, Find.find will walk the directories, returning both the directories and files as each is found. A simple check lets us decide which we want to continue processing or whether we want to skip it. The documentation has this example which should be easy to understand:
The Find module supports the top-down traversal of a set of file paths.
For example, to total the size of all files under your home directory, ignoring anything in a “dot” directory (e.g. $HOME/.ssh):
require 'find'
total_size = 0
Find.find(ENV["HOME"]) do |path|
if FileTest.directory?(path)
if File.basename(path)[0] == ?.
Find.prune # Don't look any further into this directory.
else
next
end
else
total_size += FileTest.size(path)
end
end
I'd go for Dir.glob or File.find. But not Dir.foreach as it returns . and .. which you don't want.
Dir.glob('something/*').each do |filename|
next if File.directory?(filename)
do_something_with_the_file(filename)
end
2 small questions to create the effect I'm looking for.
How do I check if a file exists within a directory with the extension of .zip?
If it does exist I need to make a folder with the same name as the .zip without the .zip extension for the folder.
Then I need to extract the files into the folder.
Secondly, what do I do if there are more than one .zip files in the folder?
I'm doing something like this and trying to put it into ruby
`mkdir fileNameisRandom`
`unzip fileNameisRandom.zip -d fileNameisRandom`
On a similar post I found something like
Dir.entries("#{Dir.pwd}").select {|f| File.file? f}
which I know checks all files within a directory and makes sure they are a file.
The problem is I don't know how to make sure that it is only an extension of .zip
Also, I found the Glob function which checks the extension of a filename from: http://ruby-doc.org/core-1.9.3/Dir.html
How do I ensure the file exists in that case, and if it doesn't I can print out an error then.
From the comment I now have
if Dir['*.zip'].first == nil #check to see if any exist
puts "A .zip file was not found"
elsif Dir['*.zip'].select {|f| File.file? f} then #ensure each of them are a file
#use a foreach loop to go through each one
Dir['*.zip'].select.each do |file|
puts "#{file}"
end ## end for each loop
end
Here's a way of doing this with less branching:
# prepare the data
zips= Dir['*.zip'].select{ |f| File.file? }
# check if data is sane
if zips.empty?
puts "No zips"
exit 0 # or return
end
# process data
zips.each do |z|
end
This pattern is easier to follow for fellow programmers.
You can also do it using a ruby gem called rubyzip
Gemfile:
source 'https://rubygems.org'
gem 'rubyzip'
run bundle
unzip.rb:
require 'zip'
zips= Dir['*.zip'].select{ |f| File.file? }
if zips.empty?
puts "No zips"
exit 0 # or return
end
zips.each do |zip|
Zip::File.open(zip) do |files|
files.each do |file|
# write file somewhere
# see here https://github.com/rubyzip/rubyzip
end
end
end
I finally pieced together different information from tutorials and used #rogerdpack and his comment for help.
require 'rubygems/package'
#require 'zlib'
require 'fileutils'
#move to the unprocessed directory to unpack the files
#if a .tgz file exists
#take all .tgz files
#make a folder with the same name
#put all contained folders from .tgz file inside of similarly named folder
#Dir.chdir("awaitingApproval/")
if Dir['*.zip'].first == nil #check to see if any exist, I use .first because Dir[] returns an array
puts "A .zip file was not found"
elsif Dir['*.zip'].select {|f| File.file? f} then #ensure each of them are a file
#use a foreach loop to go through each one
Dir['*.zip'].select.each do |file|
puts "" #newlie for each file
puts "#{file}" #print out file name
#next line based on `mkdir fileNameisRandom`
`mkdir #{Dir.pwd}/awaitingValidation/#{ File.basename(file, File.extname(file)) }`
#next line based on `unzip fileNameisRandom.zip -d fileNameisRandom`
placement = "awaitingValidation/" + File.basename(file, File.extname(file))
puts "#{placement}"
`sudo unzip #{file} -d #{placement}`
puts "Unzip complete"
end ## end for each loop
end
I am trying to write a script to do the following:
There are two directories A and B. In directory A, there are files called "today" and "today1". In directory B, there are three files called "today", "today1" and "otherfile".
I want to loop over the files in directory A and append the files that have similar names in directory B to the files in Directory A.
I wrote the method below to handle this but I am not sure if this is on track or if there is a more straightforward way to handle such a case?
Please note I am running the script from directory B.
def append_data_to_daily_files
directory = "B"
Dir.entries('B').each do |file|
fileName = file
next if file == '.' or file == '..'
File.open(File.join(directory, file), 'a') {|file|
Dir.entries('.').each do |item|
next if !(item.match(/fileName/))
File.open(item, "r")
file<<item
item.close
end
#file.puts "hello"
file.close
}
end
end
In my opinion, your append_data_to_daily_files() method is trying to do too many things -- which makes it difficult to reason about. Break down the logic into very small steps, and write a simple method for each step. Here's a start along that path.
require 'set'
def dir_entries(dir)
Dir.chdir(dir) {
return Dir.glob('*').to_set
}
end
def append_file_content(target, source)
File.open(target, 'a') { |fh|
fh.write(IO.read(source))
}
end
def append_common_files(target_dir, source_dir)
ts = dir_entries(target_dir)
ss = dir_entries(source_dir)
common_files = ts.intersection(ss)
common_files.each do |file_name|
t = File.join(target_dir, file_name)
s = File.join(source_dir, file_name)
append_file_content(t, s)
end
end
# Run script like this:
# ruby my_script.rb A B
append_common_files(*ARGV)
By using a Set, you can easily figure out the common files. By using glob you can avoid the hassle of filtering out the dot-directories. By designing the code to take its directory names from the command line (rather than hard-coding the names in the script), you end up with a potentially re-usable tool.
My solution....
def append_old_logs_to_daily_files
directory = "B"
#For each file in the folder "B"
Dir.entries('B').each do |file|
fileName = file
#skip dot directories
next if file == '.' or file == '..'
#Open each file
File.open(File.join(directory, file), 'a') {|file|
#Get each log file from the current directory in turn
Dir.entries('.').each do |item|
next if item == '.' or item == '..'
#that matches the day we are looking for
next if !(item.match(fileName))
#Read the log file
logFilesToBeCopied = File.open(item, "r")
contents = logFilesToBeCopied.read
file<<contents
end
file.close
}
end
end
These are in a folder:
This_is_a_very_good_movie-y08iPnx_ktA.mp4
myMovie2-lKESbDzUwUg.mp4
his_is_another_movie-lKESbDzUwUg.mp4
How do I fetch the first part of the string mymovie1 from the file by giving the last part, y08iPnx_ktA? Something like:
get_first_part("y08iPnx_kTA") #=> "This_is_a_very_good_movie"
Break the problem into into parts. The method get_first_part should go something like:
Use Dir to get a listing of files.
Iterate over each file and;
Extract the "name" ('This_is_a_very_good_movie') and the "tag" ('y08iPnx_ktA'). The same regex should be used for each file.
If the "tag" matches what is being looked for, return "name".
Happy coding.
Play around in the REPL and have fun :-)
def get_first_part(path, suffix)
Dir.entries(path).find do |fname|
File.basename(fname, File.extname(fname)).end_with?(suffix)
end.split(suffix).first
end
Kind of expands on the answer from #Steve Wilhelm -- except doesn't use glob (there's no need for it when we're only working with filenames), avoids Regexp and uses File.exname(fname) to the File.basename call so you don't have to include the file extension. Also returns the string "This_is_a_very_good_movie" instead of an array of files.
This will of course raise if no file could be found.. in which case if you just want to return nil if a match couldn't be found:
def get_first_part(path, suffix)
file = Dir.entries(path).find do |fname|
File.basename(fname, File.extname(fname)).end_with?(suffix)
end
file.split(suffix).first if file
end
Can it be done cleaner than this? REVISED based on #Tin Man's suggestion
def get_first_part(path, suffix)
Dir.glob(path + "*" + suffix + "*").map { |x| File.basename(x).gsub(Regexp.new("#{suffix}\.*$"),'') }
end
puts get_first_part("/path/to/files/", "-y08iPnx_kTA")
If the filenames only have a single hyphen:
path = '/Users/greg/Desktop/test'
target = 'rb'
def get_files(path, target)
Dir.chdir(path) do
return Dir["*#{ target }*"].map{ |f| f.split('-').first }
end
end
puts get_files(path, 'y08iPnx_ktA')
# >> This_is_a_very_good_movie
If there are multiple hyphens:
def get_files(path, target)
Dir.chdir(path) do
return Dir["*#{ target }*"].map{ |f| f.split(target).first.chop }
end
end
puts get_files(path, 'y08iPnx_ktA')
# >> This_is_a_very_good_movie
If the code is assumed to be running from inside the directory containing the files, then Dir.chdir can be removed, simplifying things to either:
puts Dir["*#{ target }*"].map{ |f| f.split('-').first }
# >> This_is_a_very_good_movie
or
puts Dir["*#{ target }*"].map{ |f| f.split(target).first.chop }
# >> This_is_a_very_good_movie
I want to search my filesystem for any files with the extension .template.
The below works fine for everything except .htaccess.template
FileList.new(File.join(root, '**', '*.template')).each do |file|
# do stuff with file
end
because windows doesn't like nameless files, grrrr
How do I make this work on Windows? This code works fine on Linux....
How about
Dir.glob([".*.template", "*.template"])
Assuming that FileList here is the FileList class from rake then the problem is in Ruby's underlying Dir class (which is used by FileList) not matching files starting with . for the * wildcard. The relevant portion of rake.rb is
# Add matching glob patterns.
def add_matching(pattern)
Dir[pattern].each do |fn|
self << fn unless exclude?(fn)
end
end
Below is an ugly hack that overrides add_matching to also include files starting with . Hopefully someone else will be along to suggest a more elegant solution.
class Rake::FileList
def add_matching(pattern)
files = Dir[pattern]
# ugly hack to include files starting with . on Windows
if RUBY_PLATFORM =~ /mswin/
parts = File.split(pattern)
# if filename portion of the pattern starts with * also
# include the files matching '.' + the same pattern
if parts.last[0] == ?*
files += Dir[File.join(parts[0...-1] << '.' + parts.last)]
end
end
files.each do |fn|
self << fn unless exclude?(fn)
end
end
end
Update: I have just tested this on Linux here and the files starting with . are not included either. e.g. If I have a directory /home/mikej/root with 2 subdirectories a and b where each contains first.template and .other.template then
Rake::FileList.new('home/mikej/root/**/*.template')
=> ["/home/mikej/root/a/first.template", "/home/mikej/root/b/first.template"]
so I would double check the behaviour on Linux and verify that there isn't something else causing the difference in behaviour.