I have a simple sinatra application that needs to generate a file (via an external process), send that file to the browser, and finally, delete the file from the filesystem. Something along these lines:
class MyApp < Sinatra::Base
get '/generate-file' do
# calls out to an external process,
# and returns the path to the generated file
file_path = generate_the_file()
# send the file to the browser
send_file(file_path)
# remove the generated file, so we don't
# completely fill up the filesystem.
File.delete(file_path)
# File.delete is never called.
end
end
It seems, however, that the send_file call completes the request, and any code after it does not get run.
Is there some way to ensure that the generated file is cleaned up after it has been successfully sent to the browser? Or will I need to resort to a cron job running a cleanup script on some interval?
Unfortunately there is no any callbacks when you use send_file. Common solution here is to use cron tasks to clean temp files
It could be a solution to temporarily store the contents of the file in a variable, like:
contents = file.read
After this, delete the file:
File.delete(file_path)
Finally, return the contents:
contents
This has the same effect as your send_file().
send_file is streaming the file, it is not a synchronous call, so you may not be able to catch the end of it to the cleanup the file. I suggest using it for static files or really big files. For the big files, you'll need a cron job or some other solution to cleanup later. You can't do it in the same method because send_file will not terminate while the execution is still in the get method. If you don't really care about the streaming part, you may use the synchronous option.
begin
file_path = generate_the_file()
result File.read(file_path)
#...
result # This is the return
ensure
File.delete(file_path) # This will be called..
end
Of course, if you're not doing anything fancy with the file, you may stick with Jochem's answer which eliminate begin-ensure-end altogether.
Related
I will probably kick myself after this but I have created a method within my cucumber application that will delete all files in a directory but it doesn't seem to execute as the files still exist after running it, though if i jump into my console and run the commands step by step it works.
Any reason why this would not work when calling the method itself?
def clean_reports
js_error_path = File.join(File.dirname(__FILE__), '/report/js_errors')
screenshot_path = File.join(File.dirname(__FILE__), '/report/screenshots')
FileUtils.rm_rf(Dir.glob(js_error_path + '/*'))
FileUtils.rm_rf(Dir.glob(screenshot_path + '/*'))
end
What would be handy is if I could just call FileUtils.rm_rf just once? is this possible?
My directory structure is like so
-cucumber_tests
-report
-js_errors
(files)
-screenshots
(files)
Thanks
Following line will help,
FileUtils.rm_rf(Dir['./report/js_errors', './report/screenshots'])
I have a file called backup.lua, which the program should write to every so often in order to backup its status, in case of a failure.
The problem is that the program writes the backup.lua file completely fine first-time round, but any other times it refuses to write to the file.
I tried removing the file while the program was still open but Windows told me that the file was in use by 'CrysisWarsDedicatedServer.exe', which is the program. I have told the host Lua function to close the backup.lua file, so why isn't it letting me modify the file at will after it has been closed?
I can't find anything on the internet (Google actually tried to correct my search) and the secondary programmer on the project doesn't know either.
So I'm wondering if any of you folks know what we are doing wrong here?
Host function code:
function ServerBackup(todo)
local write, read;
if todo=="write" then
write = true;
else
read = true;
end
if (write) then
local source = io.open(Root().."Mods/Infinity/System/Read/backup.lua", "w");
System.Log(TeamInstantAction:GetTeamScore(2).." for 2, and for 1: "..TeamInstantAction:GetTeamScore(1))
System.LogAlways("[System] Backing up serverdata to file 'backup.lua'");
source:write("--[[ The server is dependent on this file; editing it will lead to serious problems.If there is a problem with this file, please re-write it by accessing the backup system ingame.--]]");
source:write("Backup = {};Backup.Time = '"..os.date("%H:%M").."';Backup.Date = '"..os.date("%d/%m/%Y").."';");
source:write(XFormat("TeamInstantAction:SetTeamScore(2, %d);TeamInstantAction:SetTeamScore(1, %d);TeamInstantAction:UpdateScores();",TeamInstantAction:GetTeamScore(2), TeamInstantAction:GetTeamScore(1) ));
source:close();
for i,player in pairs(g_gameRules.game:GetPlayers() or {}) do
if (IsModerator(player)) then
CMPlayer(player, "[!backup] Completed server backup.");
end
end
end
--local source = io.open(Root().."Mods/Infinity/System/Read/backup.lua", "r"); Can the file be open here and by the Lua scriptloader too?
if (read) then
System.LogAlways("[System] Restoring serverdata from file 'backup.lua'");
--source:close();
Backup = {};
Script.LoadScript(Root().."Mods/Infinity/System/Read/backup.lua");
if not Backup or #Backup < 1 then
System.LogAlways("[System] Error restoring serverdata from file 'backup.lua'");
end
end
end
Thanks all :).
Edit:
Although the file is now written to the disk fine, the system fails to read the dumped file.
So, now the problem is that the "LoadScript" function isn't doing what you expect:
Because I'm psychic, i have divined that you're writing a Crysis plugin, and are attempting to use it's LoadScript API call.
(Please don't assume everyone here would guess this, or be bothered to look for it. It's vital information that must form part of your questions)
The script you're writing attempts to set Backup - but your script, as written - does not separate lines with newline characters. As the first line is a comment, the entire script will be ignored.
Basicallty the script you've written looks like this, which is all treated as a comment.
--[[ comment ]]--Backup="Hello!"
You need to write a "\n" after the comment (and, I'd recommend in other places too) to make it like this. In fact, you don't really need block comments at all.
-- comment
Backup="Hello!"
I am using sinatra,ruby and MongoDB to export a CSV file from the MongoDB. I am able to create the CSV file and export it.
I delete the file after exporting it.
But it gets deleted only after I exit sinatra.
Can anybody explain why is this?
Suppose a file abc****.csv is created.
I am deleting this file using
file_path = '/home/Test_app';
file = Tempfile.new([##uname,'.csv'],file_path);
file_name = file.path();
puts file_name # gives /home/Test_app/xyz****.csv
send_file(file_name, :disposition => 'attachment', :filename =>File.basename(file_name));
File.delete(file_name);
File.unlink(file_name);
But it gets deleted only after I exit sinatra server. Can anyone explain please?
Your never call file.close, meaning the file will be kept open and therefore not deleted until your application exits.
Try following the suggestion given in the Tempfile documentation:
file = Tempfile.new('foo')
begin
...do something with file...
ensure
file.close
file.unlink # deletes the temp file
end
This will make sure the file is properly closed and deleted even in the event of exceptions being raised in the code between begin and ensure.
Perhaps this is a large file; since the HTTP connection does not close until the streaming is complete, the code after send_file is not getting executed. That might be a valid reason. Have you checked if the entire file is being downloaded to the client? If that is not the case, try it out with a smaller file. I'm assuming you've implemented (but haven't written it here) the code for the data getting written into the file_name from MongoDB.
I am trying to write a ruby fcgi script which compresses files in a directory on the fly and sends the output blockwise as an http response. It is very important that this compression is done as a stream operation, otherwise the client will get a timeout for huge directories.
I have the following code:
d="/tmp/delivery/"
# send zip header
header(MimeTypes::ZIP)
# pseudocode from here on
IO.open(d) { |fh|
block=fh.readblock(1024)
#send zipped block as http response
print zip_it(block)
}
How do I achieve what I've written as pseudo-ruby in the above listing?
Tokland's idea of using the external zip command works pretty well. Here's a quick snippet that should work with Ruby 1.9 on Linux or similar environments. It uses an array parameter to popen() to avoid any shell quoting issues and sysread/syswrite to avoid buffering. You could display a status message in the empty rescue block if you like -- or you could use read and write, though I haven't tested those.
#! usr/bin/env ruby
d = '/tmp/delivery'
output = $stdout
IO.popen(['/usr/bin/zip', '-', d]) do |zip_output|
begin
while buf = zip_output.sysread(1024)
output.syswrite(buf)
end
rescue EOFError
end
end
AFAYK Zip format is not streamable, at end of compression it writes something in the file header.
gz or tar.gz is better option.
solved:
https://github.com/fringd/zipline.git
I'm building a specialized pipeline, and basically, every step in the pipeline involves taking one file as input and creating a different file as output. Not all files are in the same directory, all output files are of a different format, and because I'm using several different programs, different actions have to be taken to appease the different programs.
This has led to some complicated file management in my code, and the more I try to organize the file directories, the more ugly it's getting. Just about every class involves some sort of code like the following:
#fileName = File.basename(file)
#dataPath = "#{$path}/../data/"
MzmlToOther.new("mgf", "#{#dataPath}/spectra/#{#fileName}.mzML", 1, false).convert
system("wine readw.exe --mzXML #{#file}.raw #{$path}../data/spectra/#{File.basename(#file + ".raw", ".raw")}.mzXML 2>/dev/null")
fileName = "#{$path}../data/" + parts[0] + parts[1][6..parts[1].length-1].chomp(".pep.xml")
Is there some sort of design pattern, or ruby gem, or something to clean this up? I like writing clean code, so this is really starting to bother me.
You could use a Makefile.
Make is essential a DSL designed for handling converting one type of file to another type via running an external program. As an added bonus, it will handle only performing the steps necessary to incrementally update your output if some set of source files change.
If you really want to use Ruby, try a rakefile. Rake will do this, and it's still Ruby.
You can make this as sophisticated as you want but this basic script will match a file suffix to a method which you can then call with the file path.
# a conversion method can be used for each file type if you want to
# make the code more readable or if you need to rearrange filenames.
def htm_convert file
"HTML #{file}"
end
# file suffix as key, lambda as value, the last uses an external method
routines = {
:log => lambda {|file| puts "LOG #{file}"},
:rb => lambda {|file| puts "RUBY #{file}"},
:haml => lambda {|file| puts "HAML #{file}"},
:htm => lambda {|file| puts htm_convert(file) }
}
# this loops recursively through the directory and sub folders
Dir['**/*.*'].each do |f|
suffix = f.split(".")[-1]
if routine = routines[suffix.to_sym]
routine.call(f)
else
puts "UNPROCESSED -- #{f}"
end
end