I want to be able to read/stream a video with Ruby, on Mac, and be able to get access to the pixel data for each frame.
What I've tried
https://github.com/streamio/streamio-ffmpeg
It's good at splitting the video into frames, but I don't know how to get access to the pixel data without saving each frame as an image first (or if it's possible at all).
require 'streamio-ffmpeg'
movie = FFMPEG::Movie.new("flakes.mp4")
movie.screenshot("screenshots/screenshot_%d.jpg", { custom: %w(-vf crop=60:60:10:10), vframes: (movie.duration).to_i, frame_rate: movie.frame_rate/24 }, { validate: false })`
https://github.com/wedesoft/hornetseye-ffmpeg
This seemed to have so much potential, but I don't think it's maintained anymore, and it's not meant to be used on MacOS really, so I'm having issues installing it there (headers not found and such, and no way to configure it afaik).
Any idea what tool or method I could use for this use case?
If you have ffmpeg available (streamio-ffmpeg just wraps command line calls to ffmpeg), you can create a named pipe with File.mkfifo and have ffmpeg write its screenshots to that pipe.
Then open the pipe in Ruby like you would any normal file and you can read the screenshot images directly from ffmpeg without them being transferred to/from disk. Parse the pixel data with the Ruby gem of your choice.
Related
I'm developing a system using ffmpeg to store some ip camera videos.
i'm using the segmentation command for store each 5 minutes a video for camera.
I have a wpf view where i can search historycal videos by dates. In this case i use the ffmpeg command concat to generate a video with the desire duration.
All this work excelent, my question is: it's possible concatenate the current file of the segmentation? i need for example, make a serch from the X date to the current time, but the last file is not generated yet by the ffmpeg. when i concatenate the files, the last one is not showing because is not finish the segment.
I hope someone can give me some guidance on what I can do.
Some video formats can always be playable during the build process. That is, you can make a copy of the unfinished segmentation directly and use it to merge.
I suggest you use flv or ts format to do this. mp4 is not supported. Also note that there is a delay from encoding to actually writing to the disk.
I'm not sure if direct copy will cause some data problems at the end of the segmentation file, but ffmpeg will ignore this part of the data during the merge process, so the merged video should be fine.
I want to create a user video which should take a photo album as input and play exactly like Facbook Look back video.
I have looked at couple of option including imagemagick and ffmpeg. Are there any good alternatives available for doing this.
If you want to create a video dynamically through the browser you cannot do this on client side (not in a convenient way anyways). There is no functionality in browsers today that allows you to create video files (only streams) and the option is to write JavaScript code to do all the low-level encoding etc. which will take ages (to write but also in processing) and be prone to errors etc.
Your best option is to send the individual frames to server as for example jpeg (or png if you need high quality) and process it there using jobs where the processing can be done with f.ex. FFMpeg (which is great for these things).
Track the job id using some sort of user id and have a database or file updated with current status so the user can come back and check.
I have a C# app. I have 100 JPEGs (for an example).
I can easily encode this into a video file.
When it has finished encoding on the client app I FTP upload to my server.
It would be 'neater' if I could not write the video file to a disc but instead write it to a memory stream (or array of bytes) and upload via web service perhaps?
I have checked out the ffmpeg documentation but as a C# developer I do not find it natural to understand the C examples.
I guess my first question is it possible, and if so can anyone post an example code in C#? At the moment I am using the process class in C# to execute ffmpeg.
Your process method is already good, just needs adjustments:
Set StartupInfo.RedirectStandardOutput = true and StartupInfo.UseShellExecute = false.
Instead of an output file name, call ffmpeg with pipe:, which will make it write to the standard output. Also, since the format cannot be determined from the file name anymore, make sure you use the -f <format> switch as well.
Start the process.
Read from Process.StandardOutput.BaseStream (.BaseStream, so the StreamReader that is .StandardOutput doesn't mess anything up) while the process is running into your memory stream.
Read anything still remaining buffered in Process.StandardOutput.BaseStream.
Profit.
I coded a thumbnailer a while back (BSD 2-clause), that has actual code that demonstrates this. Doesn't matter if it is an image or a video coming out of ffmpeg in the end.
As you may know, when you record a video on a windows phone, it is saved as a .mp4. I want to be able to access the video file (even if it's only stored in isolated storage for the app), and manipulate the pixel values for each frame.
I can't find anything that allows me to load a .mp4 into an app, then access the frames. I want to be able to save the manipulated video as .mp4 file as well, or be able to share it.
Has anyone figured out a good set of steps to do this?
My guess was to first load the .mp4 file into a Stream object. From here I don't know what exactly I can do, but I want to get it into a form where I can iterate through the frames, manipulate the pixels, then create a .mp4 with the audio again once the manipulation is completed.
I tried doing the exact same thing once. Unfortunately, there are no publicly available libraries that will help you with this. You will have to write your own code to do this.
The way to go about this would be to first read up on the storage format of mp4 and figure out how the frames are stored there. You can then read the mp4, extract the frames, modify them and stitch them back in the original format.
My biggest concern is that the hardware might not be powerful enough to accomplish this in a sufficiently small amount of time.
I've been playing with QTKit for a couple of days and I'm successfully able to record video data to a file from the camera using a QTCaptureSession and QTCaptureDeviceInput etc.
However what I want to do is send the data to another location, either over the network or to a different object within the same app (it doesn't matter) and then play the video data as if it were a stream.
I have a QTCaptureMovieFileOutput and I am passing nil as the file URL so that it doesn't actually record the data to the file (I'm only interested in the data contained in the QTSampleBuffer that is available via the delegate callback).
I have set a QTCompressionOptions object on the output specifying H264 Video and High Quality AAC Audio compression.
Each time I receive a call back, I append the data from the sample buffer into an NSMutableData object I have as an instance variable.
The problem I have is that no 'player' object in the QTKit framework seems capable of playing a 'stream' of video data. Am I correct in this assumption?
I tried creating a QTMovie object (to play in a QTMovieView) using my data instance variable but I get the error that the data is not a movie.
Am I approaching this issue from the wrong angle?
Previously I was using a QTCapturePreviewOutput which passes CVImageBufferRefs for each video frame. I was converting these frames into NSImages to display on a view.
While this gave the impression of streaming, it was slow and processor hungry.
How have other people conquered the streaming video problem?
Seems to me like you'd need to make a GL texture and then load the data into it on a per-frame basis. Everything about QTMovie seems to be based on pre-existing files, as far as my little mind can tell.