Forward a file download in sinatra using streaming - ruby

Question

I have made a ruby / sinatra website and I need to let the user to download a file.
This file is not local hosted, it is hosted on a remote API. end user must not see the true origin of the file.
get "/files/:elementKey/masterfile" do
content_type "application/octet-stream"
loadMasterfile(params[:elementKey])
end
With loadMasterfile:
http = Net::HTTP.new(plainURI,443)
http.use_ssl = true;
http.start do |http|
req = Net::HTTP::Get.new(resource, {"User-Agent" =>"API downloader"})
req.basic_auth(user.keytechUserName, user.keytechPassword)
response = http.request(req)
# return this as a file attachment
attachment( response["X-Filename"]) #Use the sinatra helper to set this as filename
response.body << This lets sinatra download the file and then forward the whole content to the browser
end
This code works, but:
The file is downloaded first to the ruby/sinatra and then forwarded to the browser.
User must wait until download starts - browser seems to freeze.
Is there a solution to start a download form a remote API and forward the contents in one flow?
I found nothing about that or just found solutions for local file downloads, but I must download a file from a remote API.
I also can not cache the file locally or on Amazon AWS.
Any Ideas?

Answer 1

To achieve this in a streaming fashion in which your app is the proxy, you'll need to send the client chunks as you are downloading chunks. This is not the default behavior of ruby / Net::HTTP, but it is possible.
From the ruby docs:
By default Net::HTTP reads an entire response into memory. If you are handling large files or wish to implement a progress bar you can instead stream the body directly to an IO.
Streaming is possible through read_body, though.
Net::HTTP Streaming Response Bodies
Example usage from the docs:
uri = URI('http://example.com/large_file')
Net::HTTP.start(uri.host, uri.port) do |http|
request = Net::HTTP::Get.new uri
http.request request do |response|
open 'large_file', 'w' do |io|
response.read_body do |chunk|
io.write chunk
end
end
end
end
This example from the docs writes the streaming data to a file, but you could replace it with writes to your response stream. In combination with Sinatra's streaming api, the code might look like this:
get "/files/:elementKey/masterfile" do
content_type "application/octet-stream"
stream do |out|
loadMasterfile(params[:elementKey]) do |chunk|
out << chunk
end
end
end
def loadMasterfile(resource, &block)
http = Net::HTTP.new(plainURI, 443)
http.use_ssl = true;
http.start do |http|
req = Net::HTTP::Get.new(resource, {"User-Agent" =>"API downloader"})
req.basic_auth(user.keytechUserName, user.keytechPassword)
http.request(req) do |origin_repsonse|
origin_repsonse.read_body(&block)
end
end
end
I'm not sure how you'd set the filename. You'd also want to handle errors appropriately in the net calls and stream close. Also note that a front-end like nginx can affect the buffering / chunking of streaming responses.