To determine the file type of an attached file, I used the OS "file" utility:
class AttachedFileTypeValidator < ActiveModel::Validator
def validate(record)
file = record.resource.uploaded_file
attached_file = Rails.root + file.path
file_type = `file #{attached_file}`
Rails.logger.info "Attached file type determined to be: #{file_type}"
unless file_type.split(',').first =~ /ASCII|UTF/
record.errors[:resource_content_type] << "Attachment does not appear to be a text CSV file, please ensure it was saved correctly."
end
end
end
Unfortunately brakeman suggests its a command line injection opportunity. I'm assuming this means someone figures out a clever name for a file like:
; rm -rf /;
And away we go. Whats a good way to sanitize filenames?
Use IO#popen to call the external command:
file_type = IO.popen(['file', attached_file]).read
This will handle proper escaping of funny characters in the filename for you.
Related
Venturing into Ruby lands (learning Ruby). I like it, fun programming language.
Anyhow, I'm trying to build a simple program to delete suffixes from a folder, where user provides the path to the folder in the Mac terminal.
The scenario goes like this:
User runs my program
The program ask user to enter the folder path
User drags and drop the folder into the Mac terminal
Program receives path such as "/Users/zhang/Desktop/test\ folder"
Program goes and renames all files in that folder with suffix such as "image_mdpi.png" to "image.png"
I'm encountering a problem though.
Right now, I'm trying to list the contents of the directory using:
Dir.entries(#directoryPath)
However, it seems Dir.entries doesn't like backslashes '\' in the path. If I use Dir.entries() for a path with backslash, I get an exception saying folder or file doesn't exist.
So my next thought would be to use :
Pathname.new(rawPath)
To let Ruby create a proper path. Unfortunately, even Pathname.new() doesn't like backslash either. My terminal is spitting out
#directoryPath is not dir
This is my source code so far:
# ------------------------------------------------------------------------------
# Renamer.rb
# ------------------------------------------------------------------------------
#
# Program to strip out Android suffixes like _xhdpi, _hpdi, _mdpi and _ldpi
# but only for Mac at the moment.
#
# --------------------------------------------------
# Usage:
# --------------------------------------------------
# 1. User enters a the path to the drawable folder to clean
# 2. program outputs list of files and folder it detects to clean
# 3. program ask user to confirm cleaning
require "Pathname"
#directoryPath = ''
#isCorrectPath = false
# --------------------------------------------------
# Method definitions
# --------------------------------------------------
def ask_for_directory_path
puts "What is the path to the drawable folder you need cleaning?:"
rawPath = gets.chomp.strip
path = Pathname.new("#{rawPath}")
puts "Stored dir path = '#{path}'"
if path.directory?
puts "#directoryPath is dir"
else
puts "#directoryPath is not dir"
end
#directoryPath = path.to_path
end
def confirm_input_correct
print "\n\nIs this correct? [y/N]: "
#isCorrectPath = gets.chomp.strip
end
def reconfirm_input_correct
print "please enter 'y' or 'N': "
#isCorrectPath = gets.strip
end
def output_folder_path
puts "The folder '#{#directoryPath}' contains the following files and folders:"
# Dir.entries doesn't like \
# #directoryPath = #directoryPath.gsub("\\", "")
puts "cleaned path is '#{#directoryPath}'"
begin
puts Dir.entries(#directoryPath)
rescue
puts "\n\nLooks like the path is incorrect:"
puts #directoryPath
end
end
def clean_directory
puts "Cleaning directory now..."
end
puts "Hello, welcome to Renamer commander.\n\n"
ask_for_directory_path
output_folder_path
confirm_input_correct
while #isCorrectPath != 'y' && #isCorrectPath != 'N' do
reconfirm_input_correct
end
if #isCorrectPath == 'y'
clean_directory
else
ask_for_directory_path
end
I went through this learning resource for Ruby two three days ago:
http://rubylearning.com/satishtalim/tutorial.html
I'm also using these resource to figure out what I'm doing wrong:
http://ruby-doc.org/core-2.3.0/Dir.html
https://robm.me.uk/ruby/2014/01/18/pathname.html
Any ideas?
Edit
Well, the current work around(?) is to clean my raw string and delete any backslashes, using new method:
def cleanBackslash(originalString)
return originalString.gsub("\\", "")
end
Then
def ask_for_directory_path
puts "\nWhat is the path to the drawable folder you need cleaning?:"
rawPath = gets.chomp.strip
rawPath = cleanBackslash(rawPath)
...
end
Not the prettiest I guess.
A sample run of the program:
Zhang-computer:$ ruby Renamer.rb
Hello, welcome to Renamer commander.
What is the path to the drawable folder you need cleaning?:
/Users/zhang/Desktop/test\ folder
Stored dir path = '/Users/zhang/Desktop/test folder'
#directoryPath is dir
The folder '/Users/zhang/Desktop/test folder' contains the following files and folders:
cleaned path is '/Users/zhang/Desktop/test folder'
.
..
.DS_Store
file1.txt
file2.txt
file3.txt
Is this correct? [y/N]:
:]
I don't think the problem is with the backslash, but with the whitespace. You don't need to escape it:
Dir.pwd
# => "/home/lbrito/work/snippets/test folder"
Dir.entries(Dir.pwd)
# => ["..", "."]
Try calling Dir.entries without escaping the whitespace.
There is no backslash in the path. The backslash is an escape character displayed by the shell to prevent the space from being interpreted as a separator.
Just like Ruby displays strings containing double quotes by escaping those double quotes.
Okay, first of all, using gets.chomp.strip is probably not a good idea :P
The better and closer solution to what your normally see in a real bash program is to use the Readline library:
i.e.
require "Readline"
...
def ask_for_directory_path
rawPath = String.new
rawPath = Readline.readline("\nWhat is the path to the drawable folder you need cleaning?\n> ", true)
rawPath = rawPath.chomp.strip
rawPath = cleanBackslash(rawPath)
#directoryPath = Pathname.new(rawPath)
end
Using Readline lets you tab complete the folder path. I also needed to clean my backslash from the readline using my own defined:
def cleanBackslash(originalString)
return originalString.gsub("\\", "")
end
After that, the Dir.entries(#directorPath) is able to list all the files and folders in the path, whether the user typed it in manually or drag and drop the folder into the Mac terminal:
Zhang-iMac:Renamer zhang$ ruby Renamer.rb
Hello, welcome to Renamer commander.
What is the path to the drawable folder you need cleaning?
> /Users/zhang/Ruby\ Learning/Renamer/test_folder
The folder '/Users/zhang/Ruby Learning/Renamer/test_folder' contains the following files and folders:
.
..
.DS_Store
drawable
drawable-hdpi
drawable-mdpi
drawable-xhdpi
drawable-xxhdpi
drawable-xxxhdpi
Is this correct? [y/N]: y
Cleaning directory now...
The program is not finish but I think that fixes my problem of the backslash getting in the way.
I don't know how real bash programs are made, but consider this my poor man's bash program lol.
Final program
Check it out:
I feel like a boss now! :D
I'm following a tutorial and am trying to change the names of three files in a folder that's located under 'drive/users/myname/test'. I'm getting the error:
'chdir': No such file or directory - test'.
The starting path is already 'drive/users/myname', which is why I thought that I only had to enter 'test' for Dir.chdir.
How do I correctly input the paths on Mac OS?
Dir.chdir('test')
pic_names = Dir['test.{JPG,jpg}']
puts "What do you want to call this batch"
batch_name = gets.chomp
puts
print "Downloading #{pic_names.length} files: "
pic_number = 1
pic_names.each do |p|
print '.'
new_name = "batch_name#{pic_number}.jpg"
File.rename(name, new_name)
pic_number += 1
end
I think you have to provide the absolute path. So, your first line should be:
Dir.chdir("/drive/users/myname/test")
According to the documentation:
Dir.chdir("/var/spool/mail")
puts Dir.pwd
should output/var/spool/mail.
You can look at the documentation for more examples.
In:
File.rename(name, new_name)
name is never defined prior to its attempted use.
Perhaps p is supposed to be name, or name should be p?
With that assumption I'd write the loop something like:
pic_names.each_with_index do |name, pic_number|
print '.'
new_name = "#{ batch_name }#{ 1 + pic_number }.jpg"
File.rename(name, File.join(File.dirname(name), new_name))
end
File.join(File.dirname(name), new_name) is important. You have to refer to the same path in both the original and new filenames, otherwise the file will be moved to a new location, which would be wherever the current-working-directory points to. That's currently masked by your use of chdir at the start, but, without that, you'd wonder where your files went.
I want to check if a directory has a ".ogg" or ".m4a" file. In every case the dir is empty before starting a download session. So it just can have one "ogg" or one "m4a" file.
I tried out this code to fetch the filename:
def self.get_filename
if File.exists?('*.ogg')
file = Dir.glob('*.ogg')
#testfile = file[0]
#filename = File.basename(#testfile,File.extname(#testfile))
end
if File.exists?('*.m4a')
file = Dir.glob('*.m4a')
#testfile = file[0]
#filename = File.basename(#testfile,File.extname(#testfile))
end
end
Sadly the filename is actual empty. Maybe anyone knows why?
I think that you need Dir.glob instead.
Dir.glob('/path/to/dir/*.ogg') do |ogg_file|
#testfile = ogg_file
#filename = File.basename(#testfile,File.extname(#testfile))
end
File#exists? does not support regular expressions.
You can do this instead:
if Dir["*.rb"].any?
#....
I have a gem that has code like this inside:
def read(file)
#file = File.new file, "r"
end
Now the problem is, say you have a directory structure like so:
app/main.rb
app/templates/example.txt
and main.rb has the following code:
require 'mygem'
example = MyGem.read('templates/example.txt')
It comes up with File Not Found: templates/example.txt. It would work if example.txt was in the same directory as main.rb but not if it's in a directory. To solve this problem I've added an optional argument called relative_to in read(). This takes an absolute path so the above could would need to be:
require 'mygem'
example = MyGem.read('templates/example.txt', File.dirname(__FILE__))
That works fine, but I think it's a bit ugly. Is there anyway to make it so the class knows what file read() is being called in and works out the path based on that?
There is an interesting library - i told you it was private. One can protect their methods with it from being called from outside. The code finds the caller method's file and removes it. The offender is found using this line:
offender = caller[0].split(':')[0]
I guess you can use it in your MyGem.read code:
def read( file )
fpath = Pathname.new(file)
if fpath.relative?
offender = caller[0].split(':')[0]
fpath = File.join( File.dirname( offender ), file )
end
#file = File.new( fpath, "r" )
end
This way you can use paths, relative to your Mygem caller and not pwd. Exactly the way you tried in your app/main.rb
Well, you can use caller, and a lot more reliably than what the other people said too.
In your gem file, outside of any class or module, put this:
c = caller
req_file = nil
c.each do |s|
if(s =~ /(require|require_relative)/)
req_file = File.dirname(File.expand_path(s.split(':')[0])) #Does not work for filepaths with colons!
break
end
end
REQUIRING_FILE_PATH = req_file
This will work 90% of the time, unless the requiring script executed a Dir.chdir. The File.expand_path depends on that. I'm afraid that unless your requirer passes their __FILE__, there's nothing you can do if they change the working dir.
Also you may check for caller:
def read(file)
if /^(?<file>.+?):.*?/ =~ caller(1).first
caller_dir, caller_file = Pathname.new(Regexp.last_match[:file]).split
file_with_path = File.join caller_dir, file
#file = File.new "#{file_with_path}", "r"
end
end
I would not suggest you to do so (the code above will break being called indirectly, because of caller(1), see reference to documentation on caller). Furthermore, the regex above should be tuned more accurately if the caller path is intended to contain colons.
This should work for typical uses (I'm not sure how resistant it is to indirect use, as mentioned by madusobwa above):
def read_relative(file)
#file = File.new File.join(File.dirname(caller.first), file)
end
On a side note, consider adding a block form of your method that closes the file after yielding. In the current form you're forcing clients to wrap their use of your gem with an ensure.
Accept a file path String as an argument. Convert to a Pathname object. Check if the path is relative. If yes, then convert to absolute.
def read(file)
fpath = Pathname.new(file)
if fpath.relative?
fpath = File.expand_path(File.join(File.dirname(__FILE__),file))
end
#file = File.new(fpath,"r")
end
You can make this code more succinct (less verbose).
post '/upload' do
unless params[:file] && (tmpfile = params[:file][:tempfile]) && (name = params[:file][:filename])
return haml(:upload)
end
time = Time.now.to_s
time.gsub!(/\s/, '')
name = time + name
while blk = tmpfile.read(65536)
File.open(File.join(Dir.pwd,"public/uploads", name), "wb") { |f| f.write(tmpfile.read) }
end
'success'
end
Everything goes where expected the files just end up being corrupted.
This bit looks really funky:
while blk = tmpfile.read(65536)
File.open(File.join(Dir.pwd,"public/uploads", name), "wb") { |f| f.write(tmpfile.read) }
end
I'm guessing you're trying to read your tempfile a 65536-byte block at a time, and then write those blocks successively to your destination file. But you never write blk, which is the first block you read; you write the rest of the file (tempfile.read) instead. And even if this loop did write blocks like it should, it opens the file anew for each block, overwriting the old contents! Anyway, I suspect you meant something like this:
File.open(File.join(Dir.pwd,"public/uploads", name), "wb") do |f|
while(blk = tempfile.read(65536))
f.write(blk)
end
end
That said, if you've got the file as a temp file (presumably already on your local file system), maybe all you need to do is move that file? It'll go way faster if that's the case - if the source and destination are on the same disk, it's just a matter of swapping some file system pointers, rather than copying all that data.
Hope that helps!
The code opens and replaces the file during every iteration of the loop, which causes part of the problem. The code also reads the tmpfile into blk then throws that data away. Time.now.to_s contains colons, which is the path separator on Mac OS X, and could cause a problem on OS X. The user-supplied filename could contain some bad stuff like .. which may allow users to overwrite files. Try this instead:
require 'pathname'
require 'zaru'
post '/upload' do
unless tmpfile = params[:file].try(:[], :tempfile)
return haml(:upload)
end
name = Zaru.sanitize!("#{Time.now.to_i}#{params[:file][:filename]}")
Pathname.pwd.join("public/uploads", name).open("wb") do |f|
while blk = tmpfile.read(65536)
f.write(blk)
end
end
'success'
end
You should also make sure that the filename doesn't end in something nefarious, like .js or .css, which could be exploited.