Ok, I'm trying to upload a video, and validate the file type.
According to the documentation:
mimes:foo,bar,...
The file under validation must have a MIME type corresponding to one of the listed extensions.
Basic Usage Of MIME Rule
'photo' => 'mimes:jpeg,bmp,png'
I'm uploading a wmv video, and my rules are so:
return [
'file' => ['required', 'mimes:video/x-ms-wmv']
]
I've done a print_r() on Request::file('file') and I get the following data:
Symfony\Component\HttpFoundation\File\UploadedFile Object
(
[test:Symfony\Component\HttpFoundation\File\UploadedFile:private] =>
[originalName:Symfony\Component\HttpFoundation\File\UploadedFile:private] => SampleVideo.wmv
[mimeType:Symfony\Component\HttpFoundation\File\UploadedFile:private] => video/x-ms-wmv
[size:Symfony\Component\HttpFoundation\File\UploadedFile:private] => 70982901
[error:Symfony\Component\HttpFoundation\File\UploadedFile:private] => 0
[pathName:SplFileInfo:private] => C:\wamp\tmp\php6428.tmp
[fileName:SplFileInfo:private] => php6428.tmp
)
However I'm getting the error:
{"file":["The file must be a file of type: video\/x-ms-wmv."]}
I've tried changing the "mime type" to video/*, wmv (as per the docs) and also video/x-ms-wmv yet none of them validate the file correctly.
As you can see from the print_r() the mime type Symfony is getting is video/x-ms-wmv.
Am I doing something wrong? Or can Laravel/Symfony just not validate files well?
I appreciate the help
Edit
Ok, I opened validator.php and added echo $value->guessExtension(); to the ValidateMimes() method, and it outputs asf.
Why is Symfony outputting video\x-ms-wmv, the file extension is wmv, I'm validating both of them, but Laravel is guessing asf?!
It's too unreliable for video validation for me.
This is expected behaviour.
Laravel is calling guessExtension on Symphony's UploadedFile object, which will return the expected extension of the file, not the mimetype.
This is why the documenatation states that for an uploaded image you should use:
'photo' => 'mimes:jpeg,bmp,png'
Symfony's guessExtension calls getMimeType, which uses PHP's Fileinfo Functions to go and guess the mimetype of a given file.
Once getMimeType guesses the mimetype for the file, Symfony's MimeTypeExtensionGuesser kicks in to get the extension from the mime type retrieved from a file.
// ... cut from MimeTypeExtensionGuesser
'video/x-ms-asf' => 'asf',
'video/x-ms-wmv' => 'wmv',
'video/x-ms-wmx' => 'wmx',
'video/x-ms-wvx' => 'wvx',
'video/x-msvideo' => 'avi',
Therefore, your rules should be:
return [
'file' => ['required', 'mimes:wmv,asf']
]
The reason that asf should be included is mainly historical. To quote Wikipedia:
The most common media contained within an ASF file are Windows Media Audio (WMA) and Windows Media Video (WMV). The most common file extensions for ASF files are extension .WMA (audio-only files using Windows Media Audio, with MIME-type 'audio/x-ms-wma') and .WMV (files containing video, using the Windows Media Audio and Video codecs, with MIME-type 'video/x-ms-asf'). These files are identical to the old .ASF files but for their extension and MIME-type.
Microsoft's documentation about the difference between ASF and WMV/WMA files states:
The only difference between ASF files and WMV or WMA files are the file extensions and the MIME types [...] The basic internal structure of the files is identical.
Because the internal structure of the file is identical (including the magic numbers for the file format), wmv, wma and asf are one and the same. The only difference between the three extensions is the icon that is shown inside Explorer.
It's not just Windows Media files that will have this issue, Wikipedia lists many different video container formats that will have the same problem. If you want to find the video codec that is being used in a container, you are going to need to look at more then just the "magic patterns" that are used by the fileinfo functions.
That being said, expected behaviour != correct behaviour.
I submitted a pull request to add a new validator, called mimetypes. This does as you would expect and uses the guessed mimetype to validate an uploaded file, instead of the extension that is guessed from the mimetype.
Related
I need to validate uploaded files against a whitelist of allowed extensions (e.g. png, .pdf etc) and also check whether the extension corresponds to the real file content in order to avoid malicious files.
For now file upload is done via fileUpload() directive that parses file metadata and provides file content as a Source[ByteString, Any] which then gets streamed directly to S3 storage:
def uploadFileRoute: Route =
path("files") {
post {
(withSizeLimit(documentSizeLimit) & fileUpload("file")) {
case (fileInfo: FileInfo, contentSource: Source[ByteString, Any]) =>
onSuccess(service.uploadToS3(fileInfo, contentSource)) {
completeDefault(_)
}
}
}
}
Is there any good solution on how to perform such validation and to keep using the streaming upload (e.g. not to store the whole file locally ) if possible? Any code examples are appreciated!
Yesterday I set it up so I can serve MP3 files stored in my Dropbox using https://github.com/spatie/dropbox-api and Laravel. However this only works for small'ish files as the way it's working now, it has to load the entire file first and then serve it from Laravel. This doesn't work at all for movies or for long tracks as it takes forever and runs out of memory.
Here's the code I'm currently using
$authorizationToken = 'my-api-token';
$client = new \Spatie\Dropbox\Client($authorizationToken);
$path = "/offline/a-very-long-song.mp3"; // path in dropbox
$stream = $client->download($path);
$file = stream_get_contents($stream);
fclose($stream);
unset($stream);
$file_info = new \finfo(FILEINFO_MIME_TYPE);
return response($file, 200)->withHeaders([
'Content-Type' => $file_info->buffer($file),
'Content-Disposition' => 'inline; filename="' . basename($path) . '"',
]);
I was wondering if there's a way to stream it so it doesn't have to load the entire file first. I guess this happens naturally when you load a media file in the browser, but since there are no direct links to the physical file with Dropbox, I'm not sure if it's possible.
The Dropbox API does offer the ability to retrieve temporary direct links that can be used for streaming files like this, via the /2/files/get_temporary_link endpoint:
https://www.dropbox.com/developers/documentation/http/documentation#files-get_temporary_link
In the library you're using, that appears to be available as the getTemporaryLink method, as shown in the example here:
https://github.com/spatie/dropbox-api#a-minimal-implementation-of-dropbox-api-v2
i'm using fineuploader s3 and i've just discovered a unexpected ( from my pov ) behaviour.
I give to our customers the possibility to upload files for their works.
Every work has a code for example 12345_1298681 and 12345_84792782.
let's assume that customer started to upload files for both orders ( starting the upload sets the work as interrupted so at next uploader init them are set as _isResumable = true)
User uploads file a.mov to work 12345_1298681 ( s3key is 12345_1298681/a.mov ) and b.mov to 12345_84792782 ( s3Key is 12345_84792782/b.mov)
If for some reason user tries to upload a.mov to work 12345_84792782 fineuploader recognizes the file as resumables and in the console i can see
Identified file with ID 0 and name of a.mov as resumable.
Server requested UUID change from '60aecf65-67ca-4811-aa3c-6425620cc3f1' to 'a2bfc111-0c82-4e48-8512-a08c3e24cbd8'
But a.mov is resumable if added to work 12345_1298681 not if added to work 12345_84792782.
i've seen in the doc that if i return false to the onResume callback the file is restarted from the beginning, but the problem whit this approach is that resume data is deleted for the file and not for the s3Key and in this way i'll lose the ability to resume upload in 12345_1298681?
To be clearer ( hopefully )
i have
order 12345_123456 => s3 key => 12345_123456/a.mov
order 12345_987654 => s3 key => 12345_987654/a.mov
if i start to upload file for order 12345_123456 than i stop it, if i want start the upload for order 12345_987654 using the same file in my filesystem, fineuploader recognizes that the file is the same ( that's correct but it differs in the final key ) and uploads it to 12345_123456/a.mov instead of 12345_987654/a.mov
and this can lead to a problem:
i think i'm uploading file for one order instead i'm not.
Digging in the source of fineuploader i can see that the cookie id is generated by function
_getLocalStorageId from qq.XhrUploadHandler
instead of
_getLocalStorageId from qq.s3.XhrUploadHandler but in none of those methods there's something that consider key to differentiate file from another
I'm trying to use Mechanize::Download so I can work with FLV video streams on the fly without buffering the whole video into memory, however Mechanize isn't following the pluggable parser for 'video/flv' and keeps returning a Mechanize::File instead. Is there a way to use Mechanize to return an IO-like stream of a file? The file only needs to be read once so seeking isn't necessary. I need mechanize to submit the right Cookies and referrers to get the file.
irb(main):072:0> nn.agent.pluggable_parser
=> #<Mechanize::PluggableParser:0x0000000192eea0 #parsers={"text/html"=>Mechanize::Page, "application/xhtml+xml"=>Mechanize::Page, "application/vnd.wap.xhtml+xml"=>Mechanize::Page, "image"=>Mechanize::Image, "text/xml"=>Mechanize::XmlFile, "application/xml"=>Mechanize::XmlFile, "video/flv"=>Mechanize::Download}, #default=Mechanize::Download>
irb(main):073:0> nn.agent.get(vid.video_url).response['content-type']
=> "video/flv"
irb(main):074:0> nn.agent.get(vid.video_url).class
=> Mechanize::File
irb(main):075:0>
I need to add metadata to a PDF which I am creating using prawn. That meta-data will be extracted later by, probably, pdf-reader. This metadata will contain internal document numbers and other information needed by downstream tools.
It would be convenient to associate meta-data with each page of the PDF. The PDF specification claims that I can store per-page private data in a "Page-Piece Dictionary". Section 14.5 states:
A page-piece dictionary (PDF 1.3) may be used to hold private
conforming product data. The data may be associated with a page or
form XObject by means of the optional PieceInfo entry in the page
object (see Table 30) or form dictionary (see Table 95). Beginning
with PDF 1.4, private data may also be associated with the PDF
document by means of the PieceInfo entry in the document catalogue
(see Table 28).
How can I set a "page-piece dictionary" with prawn? I'm using prawn 0.12.0.
If that's not possible, how else can I achieve my goal of storing metadata about each page, either at the page level, or at the document level?
you can look at the source of prawn
https://github.com/prawnpdf/prawn/commit/131082af5abb71d83de0e2005ecceaa829224904
info = { :Title => "Sample METADATA",
:Author => "Me",
:Subject => "Not Working",
:CreationDate => Time.now }
#pdf = Prawn::Document.new(:template => filename, :info => info)
One way is to do none of the above; that is, don't attach the metadata as a page-piece dictionary, and don't attach it with prawn. Instead, attach the metadata as a file attachment using the pdftk command-line tool.
To do it this way, create a file with the metadata. For example, the file metadata.yaml might contain:
---
- :document_id: '12345'
:account_id: 10
:page_numbers:
- 1
- 2
- 3
- :document_id: '12346'
:account_id: 24
:page_numbers:
- 4
After you are done creating the pdf file with prawn, then use pdftk to attach the metadata file to the pdf file:
$ pdftk foo.pdf attach_files metadata.yaml output foo-with-attachment.pdf
Since pdftk will not modify a file in place, the output file must be different than the input file.
You may be able to extract the metadata file using pdf-reader, but you can certainly do it with pdftk. This command unpacks metadata.yaml into the unpacked-attachments directory.
$ pdftk foo-with-attachment.pdf unpack_files output unpacked-attachments