I am building an application which will allow users to upload images. Mostly, it will work with mobile browsers with slow internet connections. I was wondering if there are best practices for this. Does doing some encryption and than doing the transfer and decoding on server is a trick to try ? OR something else?
You would want something preferably with resumable uploads. Since your connections is slow you'd need something that can be resumed where you left off. A library i've come across over the many years is Nginx upload module:
http://www.grid.net.ru/nginx/upload.en.html
According to the site:
The module parses request body storing all files being uploaded to a directory specified by upload_store directive. The files are then being stripped from body and altered request is then passed to a location specified by upload_pass directive, thus allowing arbitrary handling of uploaded files. Each of file fields are being replaced by a set of fields specified by upload_set_form_field directive. The content of each uploaded file then could be read from a file specified by $upload_tmp_path variable or the file could be simply moved to ultimate destination. Removal of output files is controlled by directive upload_cleanup. If a request has a method other than POST, the module returns error 405 (Method not allowed). Requests with such methods could be processed in alternative location via error_page directive.
Related
I am trying to program a simple Firefox (66.0 Quantum) Extension intended to download complete HLS video streams. The basic approach is to start video-playback using the page's regular UI functionality and to intercept the loading of respective HLS .m3u8-playlist files from within the extension's background-script (i.e. using "browser.webRequest.onBeforeRequest"). The extension similarily also intercepts the first "video-chunk" request triggered by the playlist (using "browser.webRequest.onSendHeaders") - so it knowns what the correct http-headers for respective "video-chunk" requests would need to look like.
permission-wise I am currently using these:
"permissions": [
"downloads", "activeTab", "webRequest", "webRequestBlocking", "<all_urls>"
],
Based on the above collected information, ideally the extension should be able to create correct XMLHttpRequest requests for all the "movie-chucks" found in the playlist and to ultimately download all those files. Obviously a respective long running "download" task should be performed in the background-script and once the above information has been collected there should be no need to still keep the original browser tab open.
Problem: Some of the servers that serve the movie-chunks apparently rely on the "Referer" (and maybe "Origin") header for some kind of access-control and the XMLHttpRequest instances created from within the background-script DO NOT allow to create the respective correct header (since it is a restricted field it cannot be manually set to the correct value).
The only workaround that I found so far leaves a lot to be desired: A content.XMLHttpRequest created within a content-script uses the correct headers and my background-script can delegate (via messaging) the respective file loading to the content-script (which is rather silly and means that the background-script process will fail if the original browser tab is closed during download).
Is there any way that allows to properly do/complete all the download logic on the background-script side (even after the originating browser tab has been closed)?
Our application is entirely built on websockets. We don't do any HTTP request-reply. However, we are stuck with file download. If i receive file content via websockets can I wrote to local folder on user computer ?
If it makes a difference, we are only supporting Chrome so not issue if it doesn't work on other browsers.
Also, I know i can do this via HTTP. Trying to avoid it and stick to websockets since thats how the entire app is.
Thanks a lot in advance!
The solution depends on size of your file.
If size is less than about 50 MB, I would encode file's content to base64 string on the server and send this string to the client. Client should receive parts of the string, concat them to single result, and store. After receiving whole string, add link (tag <a>) to your page with attribute href set to "data:<data_type>;base64,<base64_encoded_file_content>". <data_type> is a mime type of your file, for example "text/html" or "image/png". Suggest file name by adding download attribute set to name of file (doesn't work for Chrome on OS X).
Unfortunately I have no solution for large files. Currently there is only FileEntry API that allows to write files with JS, but according to documentation it is supported only by Chrome v13+, learn more here https://developer.mozilla.org/en-US/docs/Web/API/FileEntry.
Im currently writing a small Docpad plugin to output a documents contentRenderedWithoutLayouts into a separate .json file next to the .html version for loading it via an ajax request later.
The plugin works by overriding Baseplugin's render: (opts) -> method and doing a few checks whether we're rendering a document and to html.
I now noticed that this method gets called multiple times for some documents, which seems to be render pass related. So how can I detect the final render pass per document to avoid writing the .json multiple times per render?
Many Thanks
--
Edit:
found the answer after another look at Docpads events list: http://docpad.org/docs/events
The writeAfter event is the right place to get the final data and have the output directory tree set up so I can put my .json files next to the .html.
In case you're interested find the plugin here: https://github.com/field/docpad-plugin-jsonfragment
Another approach to this would be to use the serverExtend event, and write a router that detects if it is an ajax request (existance of the IS_XHR header) and then sends the necessary json data from that. This would require your hosting platform to support node.js as you'll be using the docpad server.
I'm using the upload module to write the uploaded file to disk as soon as it arrives in nginx. In addition, I'd like to create 2 subrequests:
POST to a URL containing the uploaded file
POST to another URL without the uploaded file
The second request is easy to do because the upload module has already stripped out the upload. My problem is with the first request: How do I get the uploaded file back into the the subrequest.
A solution for my question has been committed to the echo module.
The module you linked to has the upload_set_form_field directive and a few special variables (listed in that directive), which you can use to pass the file details to the backend as a POST variables. The example given appears to put the upload back in the POST data. Can you adapt your backend script to make that work?
What is the best practice when serving files from the Zend Framework MVC? These files have to be served from the MVC as they are protected.
I know you can read in the file and place it into the Response object but this seems like a bad practice as you would be reading the entire file into memory then serving it. Right now I usually do:
header('Content-type: image/jpeg');
fpassthru(fopen($path, 'rb'));
exit;
But this also doesn't seem right as I'm stopping the execution of the script. Any suggestions?
I see nothing wrong with just exit(); What you will need to be careful of is any output buffering layers you may have on (gzip compression, etc). Large files could blow up those buffers pretty quick, so you'll want to close them out and potentially 'chunk' your output with a fopen/fread loop.
I would suggest building a super-simple script for retrieving files based on ticket system like in CMS you generate ticket to DB - filename, unique-hash and than redirect to the super-simple file-retieving script (file.php?hash=asd52ad3as1g5). It will get the hash from query and based on it fetch the real filename and push that to output as you have written using fpassthru. The hash need to be unique and hard to guess...
You could try using the X-Sendfile header. It is supported by lighttpd and newer versions of apache. Basically the webserver will replace the output of the script with the file you specified. The downside being that it is specific to the configuration of the webserver, so you may be on a host that doesn't support it.