I'm trying to extract keyframes from a large video I have. The problem I'm seeing is that it is extracting far too many, leaving me with many very similar images.
Below is what I am currently using (from terminal)
ffmpeg -i video.mov -vf "select=eq(pict_type\,I)" -vsync vfr thumb%04d.png -hide_banner
What would be great is if there was a way I can either make it only output 1 in 5 keyframes. Or what would be even better is if there is a way I can make it only output if the frame is over x% different from the previous one.
1 in 5 keyframes:
ffmpeg -i video.mov -vf "select=eq(pict_type\,I),select='not(mod(n\,5))'" -vsync vfr thumb%04d.png
frame is over x% different from the previous one:
ffmpeg -i video.mov -vf "select=eq(pict_type\,I),select='gt(scene\,x/100)'" -vsync vfr thumb%04d.png
Related
I am trying to extract frames between a certain number as images
Currently I am doing it like this
ffmpeg
-i input.mp4
-vf select='eq(n\,34885)+eq(n\,34886)+eq(n\,34887)+eq(n\,34888)+eq(n\,34889)+eq(n\,34890)+eq(n\,34891)'
-vsync 0 output_frames_%d.png
Not only this the command above very cumbersome but also takes a lot of time to run, is there an easier and faster way to do this?
You may try this
ffmpeg -i input.mp4 -vf "select='between(n\,34885\,34891)'" -vsync 0 -start_number 34885 out-%02d.png
You could try this:
ffmpeg -i input.mp4 -vf select='between(n\,34885\,34891)' -vsync passthrough -frames 7 frame_%d.png
It's less cumbersome and stops once it has hit the target number of frames (7).
Machine learning algorithms for video processing typically work on frames (images) rather than video.
In my work, I use ffmpeg to dump a specific scene as a sequence of .png files, process them in some way (denoise, deblur, colorize, annotate, inpainting, etc), output the results into an equal number of .png files, and then update the original video with the new frames.
This works well with constant frame-rate (CFR) video. I dump the images as so (eg, 50-frame sequence starting at 1:47):
ffmpeg -i input.mp4 -vf "select='gte(t,107)*lt(selected_n,50)'" -vsync passthrough '107+%06d.png'
And then after editing the images, I replace the originals as so (for a 12.5fps CFR video):
ffmpeg -i input.mp4 -itsoffset 107 -framerate 25/2 -i '107+%06d.png' -filter_complex "[0]overlay=eof_action=pass" -vsync passthrough -c:a copy output.mp4
However, many of the videos I work with are variable frame-rate (VFR), and this has created some challenges.
A simple solution is to convert VFR video to CFR, which ffmpeg wants to do anyway, but I'm wondering if it's possible to avoid this. The reason is that CFR requires either dropping frames - since the purpose of ML video processing is usually to improve the output, I'd like to avoid this - or duplicating frames - but an upscaling algorithm that I'm working with right now uses the previous and next frame for data - if the previous or next frame is a duplicate, then ... no data for upscaling.
With -vsync passthrough, I had hoped that I could simply remove the -framerate option, and preserve the original frames as-is, but the resulting command:
ffmpeg -i input.mp4 -itsoffset 107 -i '107+%06d.png' -filter_complex "[0]overlay=eof_action=pass" -vsync passthrough -c:a copy output.mp4
uses ffmpeg's default of 25fps, and drops a lot of frames. Is there a reliable way to replace frames in VFR video?
Yes, it can be done, but it's complicated. It is crucial that the overlay video have exactly the same frame timestamps as the underlay video for this process to work reliably. Generating such a VFR video segment overlay requires capturing the frame timestamps from the source video to generate a precisely timed replacement segment.
The short version of the process is to replace the above commands with the following to extract the images:
ffmpeg -i input.mp4 -vf "select='gte(t,107)*lt(selected_n,50)',showinfo" -vsync passthrough '107+%06d.png' 2>&1 | 'sed s/\r/\n/g' | showinfo2concat.py --prefix="107+" >concat.txt
This requires a script that can be downloaded here. After editing the images, update the source video with:
ffmpeg -i input.mp4 -f concat -safe 0 -i concat.txt -filter_complex"[1]settb=1/90000,setpts=9644455+PTS*25/90000[o];[0:v:0][o]overlay=eof_action=pass" -vsync passthrough -r 90000 output.mp4
Where 90000 is the timescale (inverse of timebase), and 9644455 is the PTS of the first frame to replace.
See the source for more details about what these commands actually do.
I'm trying to use ffmpeg to output all key-frames from a video file and scale them down to 320px wide while maintaining aspect ratio.
I know I could do this with two separate commands but I am trying to find a way to do it tidily in one.
I've already succeeded in each of the steps individually using the following commands.
Output the keyframes:
.\ffmpeg -i input.mp4 -q:v 2 -vf select="eq(pict_type\,PICT_TYPE_I)" -vsync 0 thumb%07d.png
Scale images:
.\ffmpeg -i input.mp4 -vf scale=320:-1 thumb%07d.png
I won't share everything i've tried, but here's three failures at combining them.
//fail, not just keyframes, scaled
.\ffmpeg -i input.mp4 -q:v 2 -vf select="eq(pict_type\,PICT_TYPE_I)" -vsync 0 -vf scale=320:-1 thumb%07d.png -hide_banner
//fail, can't find suitable output format for scale command, invalid argument
.\ffmpeg -i input.mp4 -q:v 2 -vf select="eq(pict_type\,PICT_TYPE_I)" -vsync 0, scale=320:-1 thumb%07d.png -hide_banner
//fail
.\ffmpeg -i input.mp4 -q:v 2 -vf scale=320:-1, -vf select="eq(pict_type\,PICT_TYPE_I)" -vsync 0 thumb%07d.png -hide_banner
I've tried many different things, moving commands, combining using commas etc... But I have not had any success at combining the get key-frames and scale commands.
So how would I go about combining the get key-frames and scale commands so that it works?
thanks.
The select and scale filters here make for a linear sequence of filters, so the are to be specified one after the other. See http://ffmpeg.org/ffmpeg-filters.html#Filtering-Introduction
So, you can use
ffmpeg -i in -vf "select='eq(pict_type\,PICT_TYPE_I)',scale=320:-1" -vsync 0 -q:v 2 out%07d.png
but the below command will be quicker, as it drops non-keyframes at the decoding stage.
ffmpeg -skip_frame nokey -i in -vf "scale=320:-1" -vsync 0 -q:v 2 out%07d.png
I have the following command line:
ffmpeg -hide_banner -ss 5 -i test.mp4 -y -vf
"select='eq(pict_type\,PICT_TYPE_I)',
mpdecimate,showinfo,scale=320:240,tile=12x25" -vsync 2 out%%03d.png
As you can see, I make a mosaic of 12x25 (=300) tiles per output image. But I'd like to cap the output to a single image.
Is there a way to have ffmpeg stop processing the video after it found 300 frames?
Additionally, when grabbin the I-frames, is there a way to just keep 1/x for example
After playing with different options, I couldn't find any way to do this.
Use
ffmpeg -hide_banner -ss 5 -skip_frame nokey -i test.mp4 -y -vf "framestep=7,mpdecimate,showinfo,scale=320:240,tile=12x25" -vsync 0 -vframes 1 out.png
framestep value sets x in 1/x. You probably don't need mpdecimate if you're skipping x-1 keyframes. I've added -skip_frame nokey to avoid using the select filter. This method is much faster.
I would like to extract, say, 10 video frames from a given video file using ffmpeg, ideally uniformly distributed throughout the video. I know this can be done in a few ways, for example
ffmpeg -i input.mp4 -vf fps=1/10 out%03d.jpg
will output one image every 10 seconds. However, this is too slow for my liking and scales proportionally with the length of the video. I have read a bit about ffmpeg's seeking capability, for example
ffmpeg -ss 00:00:05 -i input.mp4 -frames:v 1 out001.jpg
will very quickly seek to the 5th second of the video and extract one frame. However I haven't come across a way to seek to multiple locations in the video without calling the above command repeatedly at various times.
Is there a quicker way of accomplishing this?
Using a long command, this can be done
e.g.
ffmpeg -ss 00:00:05 -i input.mp4
-ss 00:01:05 -i input.mp4
-ss 00:03:05 -i input.mp4
-ss 00:40:05 -i input.mp4
-map 0:v -frames:v 1 out001.jpg
-map 1:v -frames:v 1 out002.jpg
-map 2:v -frames:v 1 out003.jpg
-map 3:v -frames:v 1 out004.jpg