Delete a file using Ruby SFTP `remove` and `remove!` - ruby

I am trying to old delete files from an FTP using Ruby net/sftp, but I keep getting an error saying the file does not exist.
:error=>"Net::SFTP::StatusException (2, \"no such file\")"
I can manually delete files when logging in using the same creds, so I know I have permission.
require 'net/sftp'
ftp = Net::SFTP.start(#ftp_url, #ftp_user, :password => #ftp_pwd)
ftp.dir.entries('somePath').each do |entry|
begin
age_days = (Time.now.to_i - entry.attributes.atime) / 86400
if(age_days > ftp_max_file_age_days)
ftp.remove!(entry.name)
end
rescue Exception => e
# log error here
end
end
I prefer remove! so everything happens synchronously in this case, but I have also tried remove.
I also tried giving it the full path of the file instead of just the entry name (like 'somePath' + entry.name instead of just entry.name). I was thinking perhaps it was because I needed to change the working directory, which apparently net/sftp does not allow.
Thanks in advance!

Check if entry is a directory if yes then use ftp.rmdir. like below -
require 'net/sftp'
ftp = Net::SFTP.start(#ftp_url, #ftp_user, :password => #ftp_pwd)
ftp.dir.entries('somePath').each do |entry|
begin
age_days = (Time.now.to_i - entry.attributes.atime) / 86400
if(age_days > ftp_max_file_age_days)
if File.directory?(entry.name)
ftp.rmdir(entry.name)
else
ftp.remove!(entry.name)
end
end
rescue Exception => e
# log error here
end
end

We were eventually able to delete files using the remove method (instead of remove!.) We made a small change to the way we provide the password.
We confirmed that permissions on the FTP did not change, so I think using non_interactive: true may have been the trick.
require 'net/sftp'
def self.delete_report(endpoint, username, password, report_filename)
SSH_OPTIONS = { non_interactive: true }.freeze
report_filename_base = File.basename(report_filename, '.*')
Net::SFTP.start(endpoint, username, SSH_OPTIONS.merge(password: password)) do |sftp|
sftp.remove(report_filename)
sftp.remove("#{report_filename_base}.fin")
sftp.remove("processed/#{report_filename}")
sftp.remove("processed/#{report_filename_base}.fin")
sftp.remove("failed/#{report_filename}")
sftp.remove("failed/#{report_filename_base}.fin")
sftp.remove("failed/#{report_filename_base}.info")
end
I still don't fully understand why the same method did not work before, but we're able to delete files in subfolders too, as shown in this example.

Related

Passing jpg file directly on from backend server

I'm uploading an image file from my app via JSON to my ruby backend hosted by Heroku. It is then supposed to be passed to stripe via their gem via a path to my uploaded image. Is it possible to just pass it the tempfile? The stripe docs say to do:
Stripe::FileUpload.create(
:purpose => 'dispute_evidence',
:file => File.new('/path/to/a/file.jpg')
)
my backend post looks like:
post '/account/id' do
tempfile = params[:file][:tempfile]
filename = params[:file][:filename]
path = "#{tempfile.path}/#{filename}"
p path --> "/tmp/RackMultipart20170112-4-fgtv2n/photoID.jpg"
begin
file = Stripe::FileUpload.create({
:purpose => params[:purpose],
:file => File.new(path)
},
{
:stripe_account => params[:stripe_account]
}
)
rescue Stripe::StripeError => e
status 402
return "Error saving verification id to account: #{e.message}"
end
status 200
return file.to_json
end
but when running i get:
Errno::ENOENT - No such file or directory # rb_sysopen - /tmp/RackMultipart20170112-4-1qmr03y/photoID.jpg:
i cant figure out what im doing wrong. can any one help or suggest a better option?
You are mistaking tempfile for a directory. tempfile is actually your temporary file, so pass it instead of path (that you don't need any more).

Getting the modification time of a file on a FTP server

I need to get the modification time of a group of files on a server. I know how to get this on a local computer, but File.mtime doesn't work via FTP. How would I convert this code to work on a server?
files_sorted_by_time = Dir['*'].select { |f|
((Time.now - File.mtime(f)).to_i / 604800) < 7
}
You want Net::FTP#mtime.
Example from documentation:
Net::FTP.open('ftp.netlab.co.jp') do |ftp|
ftp.login
files = ftp.chdir('pub/lang/ruby/contrib')
files = ftp.list('n*')
ftp.getbinaryfile('nif.rb-0.91.gz', 'nif.gz', 1024)
ftp.mtime('file.pdf')
end
You can use #mtime with #nlst to filter through the list of remote files.
Net::FTP.open('ftp.netlab.co.jp') do |ftp|
ftp.login
ftp.nlst do |file|
if ftp.mtime(file) # ...
end
end

send_file for a tempfile in Sinatra

I'm trying to use Sinatra's built-in send_file command but it doesn't seem to be working for tempfiles.
I basically do the following to zip an album of mp3s:
get '/example' do
songs = ...
file_name = "zip_test.zip"
t = Tempfile.new(['temp_zip', '.zip'])
# t = File.new("testfile.zip", "w")
Zip::ZipOutputStream.open(t.path) do |z|
songs.each do |song|
name = song.name
name += ".mp3" unless name.end_with?(".mp3")
z.put_next_entry(name)
z.print(open(song.url) {|f| f.read })
p song.name + ' added to file'
end
end
p t.path
p t.size
send_file t.path, :type => 'application/zip',
:disposition => 'attachment',
:filename => file_name,
:stream => false
t.close
t.unlink
end
When I use t = File.new(...) things work as expected, but I don't want to use File as it will have concurrency problems.
When I use t = Tempfile.new(...), I get:
!! Unexpected error while processing request: The file identified by body.to_path does not exist`
Edit: It looks like part of the problem is that I'm sending multiple files. If I just send one song, the Tempfile system works as well.
My guess is that you have a typo in one of your song-names, or maybe a slash in one of the last parts of song.url? I adopted your code and if all the songs exist, sending the zip as a tempfile works perfectly fine.

Ruby / Watir-Webdriver - Cannot access the file after downloading it

I am working on a tests scenario that downloads a file from a website and adds it to folder.
For the download part, I am using the code described on the browser-downloads page within the Watir documentation.
The main problem was encountered in my tests when I am waiting for the file to be downloaded:
def verify_csv_file_exists
path = Dir.getwd + "/downloads/"
until File.exist?("#{path}*.csv") == true
sleep 1
end
end
When running the tests, the procedure above never stops, because it cannot see the file in the directory, although the file is downloaded.
Does anyone know a way how I can handle this situation?
Thank you.
You simply check the directory contents before you download the file, then wait until there's a new file added to the directory (by comparing the current content with the previous content). This is how you get the new file name:
This should do the job:
require 'watir-webdriver'
file_name = nil
download_directory = "#{Dir.pwd}/downloads"
download_directory.gsub!("/", "\\") if Selenium::WebDriver::Platform.windows?
downloads_before = Dir.entries download_directory
profile = Selenium::WebDriver::Firefox::Profile.new
profile['browser.download.folderList'] = 2 # custom location
profile['browser.download.dir'] = download_directory
profile['browser.helperApps.neverAsk.saveToDisk'] = "text/csv,application/pdf"
b = Watir::Browser.new :firefox, :profile => profile
b.goto 'https://dl.dropbox.com/u/18859962/hello.csv'
30.times do
difference = Dir.entries(download_directory) - downloads_before
if difference.size == 1
file_name = difference.first
break
end
sleep 1
end
raise "Could not locate a new file in the directory '#{download_directory}' within 30 seconds" if not file_name
puts file_name
You can't use "glob" with File.exists? like File.exists?("*.csv"). It checks whether the file named *.csv exists, not any file with name ends with .csv. You should use exact file name to check if a file exists.
Try it like this instead:
Dir.glob('downloads/*.csv').any?
Also how is sleeping for 1 second supposed to change anything? Is this a multithreaded app?

Using Open-URI to fetch XML and the best practice in case of problems with a remote url not returning/timing out?

Current code works as long as there is no remote error:
def get_name_from_remote_url
cstr = "http://someurl.com"
getresult = open(cstr, "UserAgent" => "Ruby-OpenURI").read
doc = Nokogiri::XML(getresult)
my_data = doc.xpath("/session/name").text
# => 'Fred' or 'Sam' etc
return my_data
end
But, what if the remote URL times out or returns nothing? How I detect that and return nil, for example?
And, does Open-URI give a way to define how long to wait before giving up? This method is called while a user is waiting for a response, so how do we set a max timeoput time before we give up and tell the user "sorry the remote server we tried to access is not available right now"?
Open-URI is convenient, but that ease of use means they're removing the access to a lot of the configuration details the other HTTP clients like Net::HTTP allow.
It depends on what version of Ruby you're using. For 1.8.7 you can use the Timeout module. From the docs:
require 'timeout'
begin
status = Timeout::timeout(5) {
getresult = open(cstr, "UserAgent" => "Ruby-OpenURI").read
}
rescue Timeout::Error => e
puts e.to_s
end
Then check the length of getresult to see if you got any content:
if (getresult.empty?)
puts "got nothing from url"
end
If you are using Ruby 1.9.2 you can add a :read_timeout => 10 option to the open() method.
Also, your code could be tightened up and made a bit more flexible. This will let you pass in a URL or default to the currently used URL. Also read Nokogiri's NodeSet docs to understand the difference between xpath, /, css and at, %, at_css, at_xpath:
def get_name_from_remote_url(cstr = 'http://someurl.com')
doc = Nokogiri::XML(open(cstr, 'UserAgent' => 'Ruby-OpenURI'))
# xpath returns a nodeset which has to be iterated over
# my_data = doc.xpath('/session/name').text # => 'Fred' or 'Sam' etc
# at returns a single node
doc.at('/session/name').text
end

Resources