How can I download multiple .xlsx files using axlsx gem? - download

Hi I'm having trouble downloading multiple files with axlsx. The problem is I'm sending an array of Id's to the controller and asking it to download the report using the render command. It raises an AbstractController::DoubleRenderError. I was thinking of overriding the error but realized it's a bad idea, I don't know what else to do... Any suggestions? Thanks.
My controller code looks like this:
def download_report
params[:user_id].each do |user_id|
#report = Report.find_by(:user_id => user_id)
render :xlsx => "download_report", :filename => "#{#report.user.last_name}.xlsx"
end
end
My axlsx template:
wb = xlsx_package.workbook
wb.add_worksheet(name: "Reports") do |sheet|
wb.styles do |s|
# template code
end
end

It is the built in expectation of Rails that you would call render once per request. And, the browser is going to expect one response per request. So, you are going to have to do something else!
You can use render_to_string, and combine the results into a zip file, serving that. See the bottom of this response.
Or, you could create a single spreadsheet and have each user's report show up on their own worksheet.
Or, on the client side, you could use javascript to request each spreadsheet and download each one separately.
The zip one would be something like this code, which uses render_to_string, rubyzip, and send_data:
def download_report
compressed_filestream = Zip::ZipOutputStream.write_buffer do |zos|
params[:user_id].each do |user_id|
#report = Report.find_by(:user_id => user_id)
content = render_to_string :xlsx => "download_report", :filename => "#{#report.user.last_name}.xlsx"
zos.put_next_entry("user_#{user_id}.xlsx")
zos.print content
end
end
compressed_filestream.rewind
send_data compressed_filestream.read, :filename => 'download_report.zip', :type => "application/zip"
end
Axlsx requires rubyzip, so you should have it already. And you probably want to lookup each user and use their name for the spreadsheet, unless you have it otherwise.

Related

Scraping a webpage with Mechanize and Nokogiri and storing data in XML doc

I am trying to scrape a website and store data in XML using Mechanize and Nokogiri. I didn't set up a Rails project and I am only using Ruby and IRB.
I wrote this method:
def mechanize_club
agent = Mechanize.new
agent.get("http://www.rechercheclub.applipub-fft.fr/rechercheclub/")
form = agent.page.forms.first
form.field_with(:name => 'codeLigue').options[0].select
form.submit
page2 = agent.get('http://www.rechercheclub.applipub-fft.fr/rechercheclub/club.do?codeClub=01670001&millesime=2015')
body = page2.body
html_body = Nokogiri::HTML(body)
codeclub = html_body.search('.form').children("tr:first").children("th:first").to_i
#codeclubs << codeclub
filepath = '/davidgeismar/Documents/codeclubs.xml'
builder = Nokogiri::XML::Builder.new(encoding: 'UTF-8') do |xml|
xml.root {
xml.codeclubs {
#codeclubss.each do |c|
xml.codeclub {
xml.code_ c.code
}
end
}
}
end
puts builder.to_xml
end
My first problem is that I don't know how to test my code.
I call ruby webscraper.rb in my console, the file is treated I think, but it doesn't create an XML file in the specified path.
Then, more specifically I am quite sure this code is wrong as I didn't get a chance to test it.
Basically what I am trying to do is to submit a form several times:
agent = Mechanize.new
agent.get("http://www.rechercheclub.applipub-fft.fr/rechercheclub/")
form = agent.page.forms.first
form.field_with(:name => 'codeLigue').options[0].select
form.submit
I think this code is ok, but I dont want it to only select options[0], I want it to select an option, then scrape all the data I need, then go back to page, then select options[1]... until there are no more options (an iteration I guess).
the file is treated I think, but it doesnt create an xml file in the specified path.
There is nothing in your code that creates a file. You print some output, but don't do anything to open or write a file.
Perhaps you should read the IO and File documentation and review how you are using your filepath variable?
The second problem is that you don't call your method anywhere. Though it's defined and Ruby will see it and parse the method, it has no idea what you want to do with it unless you invoke the method:
def mechanize_club
...
end
mechanize_club()

Is this the right way to submit files over a JSON api using ruby?

I posted yesterday a question about how to post files through JSON APIs here:
Posting JSON with file content on Ruby / Rails
However I couldn't really find exactly what I was looking for, so I tried by doing the following:
1) I wrote a rake task to do the upload:
desc "Tests JSON uploads with attached files on multipart formats"
task :picture => :environment do
file = File.open(Rails.root.join('lib', 'assets', 'photo.jpg'))
data = {title: "Something", description: "Else", file_content: Base64.encode64(file.read)}.to_json
req = Net::HTTP::Post.new("/users.json", {"Content-Type" => "application/json", 'Accept' => '*/*'})
req.body = data
response = Net::HTTP.new("localhost", "3000").start {|http| http.request(req) }
puts response.body
end
And then got this on the controller/model of my rails app, like this:
params[:user] = JSON.parse(request.body.read)
...
class User < ActiveRecord::Base
...
has_attached_file :picture, formats: {medium: "300x300#", thumb: "100#100"}
def file_content=(c)
filename = "#{Time.now.to_f.to_s.gsub('.', '_')}.jpg"
File.open("/tmp/#{filename}", 'wb') {|f| f.write(Base64.decode64(c).strip) }
self.picture = File.open("/tmp/#{filename}", 'r')
end
end
So, question is: Am I reinventing the wheel or is this the right way to do it?
BTW: It works, I just need to know if this is a convention for uploading files through json.
JSON is a data serializing format. There is no standard pattern for uploading data or files as data in the serialized object. JSON has expectations that the data fields will be basic objects so you probably want to use Base64 encoding of the file to turn it into a string.
You are free to define your structure however you want, and processing it is your responsibility.

Serve HTML files stored on S3 on a Rack app

Say I have some HTML documents stored on S3 likes this:
http://alan.aws-s3-bla-bla.com/posts/1.html
http://alan.aws-s3-bla-bla.com/posts/2.html
http://alan.aws-s3-bla-bla.com/posts/3.html
http://alan.aws-s3-bla-bla.com/posts/1/comments/1.html
http://alan.aws-s3-bla-bla.com/posts/1/comments/2.html
http://alan.aws-s3-bla-bla.com/posts/1/comments/3.html
etc, etc
I'd like to serve these with a Rack (preferably Sinatra) application, mapping the following routes:
get "/posts/:id" do
render "http://alan.aws-s3-bla-bla.com/posts/#{params[:id]}.html"
end
get "/posts/:posts_id/comments/:comments_id" do
render "http://alan.aws-s3-bla-bla.com/posts/#{params[:posts_id]}/comments/#{params[:comments_id}.html"
end
Is this a good idea? How would I do it?
There would obviously be a wait while you grabbed the file, so you could cache it or set etags etc to help with that. I suppose it depends on how long you want to wait and how often it is accessed, its size etc as to whether it's worth storing the HTML locally or remotely. Only you can work that bit out.
If the last expression in the block is a string that will automatically be rendered, so there's no need to call render as long as you've opened the file as a string.
Here's how to grab an external file and put it into a tempfile:
require 'faraday'
require 'faraday_middleware'
#require 'faraday/adapter/typhoeus' # see https://github.com/typhoeus/typhoeus/issues/226#issuecomment-9919517 if you get a problem with the requiring
require 'typhoeus/adapters/faraday'
configure do
Faraday.default_connection = Faraday::Connection.new(
:headers => { :accept => 'text/plain', # maybe this is wrong
:user_agent => "Sinatra via Faraday"}
) do |conn|
conn.use Faraday::Adapter::Typhoeus
end
end
helpers do
def grab_external_html( url )
response = Faraday.get url # you'll need to supply this variable somehow, your choice
filename = url # perhaps change this a bit
tempfile = Tempfile.open(filename, 'wb') { |fp| fp.write(response.body) }
end
end
get "/posts/:whatever/" do
tempfile = grab_external_html whatever # surely you'd do a bit more hereā€¦
tempfile.read
end
This might work. You may also want to think about closing that tempfile, but the garbage collector and the OS should take care of it.

Why is Net::HTTP timing out when I try to access a Prawn Generated PDF?

I am using Prawn to generate a PDF from my controller, and when accessed directly at the url, it works flawlessly, I.E. localhost:3000/responses/1.pdf
However, when I try to generate this file on the fly for inclusion in a Mailer, everything freezes up and it times out.
I have tried various methods for generating / attaching the file and none have changed the outcome.
I also tried modifying the timeout for Net::HTTP to no avail, it just takes LONGER to time out.
If I run this command on the Rails Console, I receive a PDF data stream.
Net::HTTP.get('127.0.0.1',"/responses/1.pdf", 3000)
But if I include this code in my controller, it times out.
I have tried two different methods, and both fail repeatedly.
Method 1
Controller:
http = Net::HTTP.new('localhost', 3000)
http.read_timeout = 6000
file = http.get(response_path(#response, :format => 'pdf')) #timeout here
ResponseMailer.confirmComplete(#response,file).deliver #deliver the mail!
Method 1 Mailer:
def confirmComplete(response,file)
email_address = response.supervisor_id
attachments["test.pdf"] = {:mime_type => "application/pdf", :content=> file}
mail to: email_address, subject: 'Thank you for your feedback!'
end
The above code times out.
Method 2 Controller:
ResponseMailer.confirmComplete(#response).deliver #deliver the mail!
Method 2 Mailer:
def confirmComplete(response)
email_address = response.supervisor_id
attachment "application/pdf" do |a|
a.body = Net::HTTP.get('127.0.0.1',"/responses/1.pdf", 3000) #timeout here
a.filename = "test.pdf"
end
mail to: email_address, subject: 'Thank you for your feedback!'
end
If I switch the a.body and a.filename, it errors out first with
undefined method `filename=' for #<Mail::Part:0x007ff620e05678>
Every example I find has a different syntax or suggestion but none fix the problem that Net::HTTP times out. Rails 3.1, Ruby 1.9.2
The problem is that, in development, you're only running one server process, which is busy generating the email. That process is sending another request (to itself) to generate a PDF and waiting for a response. The request for the PDF is basically standing in line at the server so that it can get it's PDF, but the server is busy generating the email and waiting to get the PDF before it can finish. And thus, you're waiting forever.
What you need to do is start up a second server process...
script/rails server -p 3001
and then get your PDF with something like ...
args = ['127.0.0.1','/responses/1.pdf']
args << 3001 unless Rails.env == 'production'
file = Net::HTTP.get(*args)
As an aside, depending on what server you're running on your production machine, you might run into issues with pointing at 127.0.0.1. You might need to make that dynamic and point to the full domain when in production, but that should be easy.
I agree with https://stackoverflow.com/users/811172/jon-garvin's analysis that you're only running one server process, but I would mention another solution. Refactor your PDF generation so you don't depend on your controller.
If you're using Prawnto, I'm guessing you have a view like
# app/views/response.pdf.prawn
pdf.text "Hello world"
Move this to your Response model: (or somewhere else more appropriate, like a presenter)
# app/models/response.rb
require 'tmpdir'
class Response < ActiveRecord::Base
def pdf_path
return #pdf_path if #pdf_generated == true
#pdf_path = File.join(Dir.tmpdir, rand(1e11).to_s)
Prawn::Document.generate(#pdf_path) do |pdf|
pdf.text "Hello world"
end
#pdf_generated = true
#pdf_path
end
def pdf_cleanup
if #pdf_generated and File.exist?(#pdf_path.to_s)
File.unlink #pdf_path
end
end
end
Then in your ResponsesController you can do:
# app/controllers/responses_controller.rb
def show
#response = Response.find params[:id]
respond_to do |format|
# this sends the PDF to the browser (doesn't email it)
format.pdf { send_file #response.pdf_path, :type => 'application/pdf', :disposition => 'attachment', :filename => 'test.pdf' }
end
end
And in your mailer you can do:
# this sends an email with the PDF attached
def confirm_complete(response)
email_address = response.supervisor_id
attachments['test.pdf'] = {:mime_type => "application/pdf", :content => File.read(response.pdf_path, :binmode => true) }
mail to: email_address, subject: 'Thank you for your feedback!'
end
Since you created it in the tmpdir, it will be automatically deleted when your server restarts. You can also call the cleanup function.
One final note: you might want to use a different model name like SupervisorReport or something - Response might get you in namespacing trouble later)

Sinatra, progress bar in upload form

I'm developing a Sinatra app that consists of an upload form, with a progress bar indicating how much of the upload has completed.
The process, as described by ryan dahl, is the following:
HTTP upload progress bars are rather obfuscated- they typically involve a process running on the server keeping track of the size of the tempfile that the HTTP server is writing to, then on the client side an AJAX call is made every couple seconds to the server during the upload to ask for the progress of the upload.
Every upload has a random session-id, and to keep track of the association i employ a class variable in my app (i know, that's horrible -- if you've got better ideas, please tell me)
configure do
##assoc = {}
end
I have a POST route for the upload, and a GET one for the AJAX polling.
Inside the POST route i save the association of session-id, Tempfile, and total size.
post '/files' do
tmp = params[:file][:tempfile]
# from here on, ##assoc[#sid] should have a value, even in other routes
##assoc[#sid] = { :file => tmp, :size => env['CONTENT_LENGTH'] }
File.open("#{options.filesdir}/#{filename}", 'w+') do |file|
file << tmp.read
end
end
In the GET route, i calculate the percentage based on the Tempfile's current size:
get '/status/:sid' do
h = ##assoc[params[:sid]]
unless h.nil?
percentage = (h[:file].size / h[:size].to_f) * 100
"#{percentage}%"
else
"0%"
end
end
The problem is that until the POST request hasn't completed (i.e., after it has read all of the Tempfile) the h.nil? returns true, which doesn't really make sense as I've just assigned ##assoc[#sid] a value in the other route.
So, what am I missing here?
EDIT: I've tried
set :reload, false
set :environment, :production
config { ##assoc ||= {} }
I also tried throwing a relational db at it (SQLite with DataMapper)
Neither worked.
I think i got what the problem is:
tmp = params[:file][:tempfile] doesn't return until the file has been fully received.
##assoc[#sid] = { :file => tmp, :size => env['CONTENT_LENGTH'] }
should be
##assoc[params[:sid]] = { :file => tmp, :size => env['CONTENT_LENGTH'] }

Resources