Remove sequentially duplicate frames when using FFmpeg - ffmpeg

Is there any way to detect duplicate frames within the video using ffmpeg?
I tried -vf flag with select=gt(scene\,0.xxx) for scene change. But, it did not work for my case.

Use the mpdecimate filter, whose purpose is to "Drop frames that do not differ greatly from the previous frame in order to reduce frame rate."
This will generate a console readout showing which frames the filter thinks are duplicates.
ffmpeg -i input.mp4 -vf mpdecimate -loglevel debug -f null -
To generate a video with the duplicates removed
ffmpeg -i input.mp4 -vf mpdecimate,setpts=N/FRAME_RATE/TB out.mp4
The setpts filter expression generates smooth timestamps for a video at FRAME_RATE FPS. See an explanation for timestamps at What is video timescale, timebase, or timestamp in ffmpeg?

I also had this problem and Gyan's excellent answer above got me started but the result of it was desynchronized audio so I had to explore more options:
mpdecimate vs decimate filters
mpdecimate is the standard recommendation I found all over SO and the internet, but I don't think it should be the first pick
it uses heuristics so it may and will skip some duplicate frames
you can tweak the detection with frac parameter, but that's extra work you may want to avoid if you can
it is not really supposed to work with mp4 container (source), but I was using mkv so this limitation didn't apply on my case, but good to be aware of it
decimate removes frames precisely, but it is useful only for periodically occurring duplicates
detected vs actual frame rate
so you have multimedia file with duplicate frames, it is good idea to make sure that the detected frame rate matches the actual one
ffprobe in.mkv will output the detected FPS; it may look like this
Stream #0:0: Video: h264 (Main), yuvj420p(pc, bt709, progressive), 1920x1080, SAR 1:1 DAR 16:9, 25 fps, 25 tbr, 1k tbn, 50 tbc (default)
the actual frame rate can be found out if you open the media in.mkv in a media player that lets you step one frame at the time; then count the steps needed to advance the playback time for 1 second, in my case it was 30 fps
not a big surprise for me, because every 6th frame was duplicate (5 good frames and 1 duplicate), so after 25 good frames there was also 5 duplicates
what is N/FRAME_RATE/TB
except the use of FRAME_RATE variable the N/FRAME_RATE/TB is equal to the example below from ffmpeg documentation (source)
Set fixed rate of 25 frames per second:
setpts=N/(25*TB)
the math behind it perfectly explained in What is video timescale, timebase, or timestamp in ffmpeg?
it basically calculates timestamp for each frame and multiplies it with timebase TB to enhance precision
FRAME_RATE variable vs literal FPS value (e.g. 25)
this is why it is important to know your detected and actual FPS
if the detected FPS matches your actual FPS (e.g. both are 30 fps) you can happily use FRAME_RATE variable in N/FRAME_RATE/TB
but if the detected FPS differs than you have to calculate the FRAME_RATE on your own
in my case my actual FPS was 30 frames per second and I removed every 6th frame, so the target FPS is 25 which leads to N/25/TB
if I used FRAME_RATE (and I actually tried that) it would take the wrong detected fps of 25 frames i.e. FRAME_RATE=25, run it through mpdecimate filter which would remove every 6th frame and it would update to FRAME_RATE=20.833 so N/FRAME_RATE/TB would actually be N/20.833/TB which is completely wrong
to use or not to use setpts
so the setpts filter already got pretty complicated especially because of the FPS mess that duplicate frames may create
the good news is you actually may not need the setpts filter at all
here is what I used with good results
ffmpeg -i in.mkv -vf mpdecimate out.mkv
ffmpeg -i in.mkv -vf decimate=cycle=6,setpts=N/25/TB out.mkv
but the following gave me desynchronized audio
ffmpeg -i in.mkv -vf mpdecimate,setpts=N/FRAME_RATE/TB out.mkv
ffmpeg -i in.mkv -vf mpdecimate,setpts=N/25/TB out.mkv
ffmpeg -i in.mkv -vf decimate=cycle=6 out.mkv
as you see
mpdecimate and decimate does not work the same way
mpdecimate worked better for me without setpts filter
while decimate needed setpts filter and furthermore I need to avoid FRAME_RATE variable and use N/25/TB instead because the actual FPS was not detected properly
note on asetpts
it does the same job as setpts does but for audio
it didn't really fix desync audio for me but you want to use it something like this -af asetpts=N/SAMPLE_RATE/TB
maybe you are supposed to adjust the SAMPLE_RATE according to the ratio of duplicate frames removed, but it seems to me like extra unnecessary work especially when my video had the audio in sync at the beginning, so it is better to use commands that will keep it that way instead of fixing it later
tl;dr
If the usually recommended command ffmpeg -i in.mkv -vf mpdecimate,setpts=N/FRAME_RATE/TB out.mkv does not work for you try this:
ffmpeg -i in.mkv -vf mpdecimate out.mkv
or
ffmpeg -i in.mkv -vf decimate=cycle=6,setpts=N/25/TB out.mkv
(cycle=6 because every 6th frame is duplicate and N/25/TB because after removing the duplicates the video will have 25 fps (avoid the FRAME_RATE variable); adjust for your use case)

I tried the solution here and none of them seem to work when you need to trim the video and keep the audio. There is a mpdecimate_trim repo that does a good job. It basically list all the frames to be dropped (using mpdecimate) and then creates a complex filter to trim all those frames (and the corresponding audio) from the video by splitting the video and only including the portion without duplicate frames.
I did have to tweak a few options in the code though. For instance, in mpdecimate_trim.py, I had to change this line:
dframes2 = get_dframes(ffmpeg(True, "-vf", "mpdecimate=hi=576", "-loglevel", "debug", "-f", "null", "-").stderr)
I had to detect duplicates a bit more aggressively, so I changed the mpdecimate option to mpdecimate=hi=64*32:lo=64*24:frac=0.05

If your getting duplicate frozen frames after cropping you could be entering your cmd line incorrectly. I was entering the orders of parameters incorrectly causing the first few seconds of my video to freeze. Here is the fix if that is the case. Hopefully this helps you avoid getting frozen frames altogether.
ffmpeg -ss {start_time} -i {input_path} -vcodec copy -acodec copy -to {end_time} {output_path}

Related

Scene detection and concat makes my video longer (FFMPEG)

I'm encoding videos by scenes. At this moment I got two solutions in order to do so. The first one is using a Python application which gives me a list of frames that represent scenes. Like this:
285
378
553
1145
...
The first scene begins from the frame 1 to 285, the second from 285 to 378 and so on. So, I made a bash script which encodes all this scenes. Basically what it does is to take the current and previous frames, then convert them to time and finally run the ffmpeg command:
begin=$(awk 'BEGIN{ print "'$previous'"/"'24'" }')
end=$(awk 'BEGIN{ print "'$current'"/"'24'" }')
time=$(awk 'BEGIN{ print "'$end'"-"'$begin'" }')
ffmpeg -i $video -r 24 -c:v libx265 -f mp4 -c:a aac -strict experimental -b:v 1.5M -ss $begin -t $time "output$count.mp4" -nostdin
This works perfect. The second method is using ffmpeg itself. I run this commands and gives me a list of times. Like this:
15.75
23.0417
56.0833
71.2917
...
Again I made a bash script that encodes all these times. In this case I don't have to convert to times because what I got are times:
time=$(awk 'BEGIN{ print "'$current'"-"'$previous'" }')
ffmpeg -i $video -r 24 -c:v libx265 -f mp4 -c:a aac -strict experimental -b:v 1.5M -ss $previous -t $time "output$count.mp4" -nostdin
After all this explained it comes the problem. Once all the scenes are encoded I need to concat them and for that what I do is to create a list with the video names and then run the ffmpeg command.
list.txt
file 'output1.mp4'
file 'output2.mp4'
file 'output3.mp4'
file 'output4.mp4'
command:
ffmpeg -f concat -i list.txt -c copy big_buck_bunny.mp4
The problem is that the "concated" video is longer than the original by 2.11 seconds. The original one lasts 596.45 seconds and the encoded lasts 598.56. I added up every video duration and I got 598.56. So, I think the problem is in the encoding process. Both videos have the same frames number. My goal is to get metrics about the encoding process, when I run VQMT to get the PSNR and SSIM I get weird results, I think is for this problem.
By the way, I'm using the big_buck_bunny video.
The probable difference is due to the copy codec. In the latter case, you tell ffmpeg to copy the segments, but it can't do that based on your input times.
It has to find first the previous I frames (a frame that can be decoded without any reference to any previous frame) and starts from here.
To get what you need, you need to either re-encode the video (like you did in the 2 former examples) or change the times to stop at I frames.
To assert I getting your issue correctly:
You have a source video (that's encoded at variable frame rate, close to 18fps)
You want to split the source video via ffmpeg, by forcing the frame rate to 24 fps.
Then you want to concat each segment.
I think the issue is mainly that you have some discrepancy in the timing (if I divide the frame index by the time you've given, I getting between 16fps to 18fps). When you are converting them in step 2, the output video segment time will be 24fps. ffmpeg does not resample in the time axis, so if you force a video rate, the video will accelerate or slow down.
There is also the issue of consistency for the stream:
Typically, a video stream must start with a I frame, so when splitting, FFMPEG has to locate the previous I frame (when using copy codec, and this changes the duration of the segment).
When you are concatenating, you could also have the issue of consistency (that is, if the segment you are concatenating does end with a I frame, and the next one starts with a I frame, it's possible FFMPEG drops either one, although I don't remember what is the current behavior now)
So, to solve your issue, if I were you, I would avoid step 2 (it's bad for quality anyway). That is, I would use ffmpeg to split the segments of interest based on the frame number (that's the only value that's not approximate in your scheme) in png or ppm frames (or to a pipe if you don't care about keeping them) and then concat all the frames by encoding them at the last step with the expected rate set to totalVideoTime / totalFrameCount.
You'll get a smaller and higher quality final video.
If you can't do what I said for whatever reason, at least for the concat input, you should use the ffconcat format:
ffconcat version 1.0
file segment1
duration 12.2
file segment2
duration 10.3
This will give you the expected duration by cutting each segment if it's longer
For selecting by frame number (instead of time as time is hard to get right on variable frame rate video), you should use the select filter like this:
-vf select=“between(n\,start_frame_num\,end_frame_num),setpts=STARTPTS"
I suggest checking the input and output frame rate and make sure they match. That could be a source of the discrepancy.

How I can convert any mp4 to adv8dvbt23.ts file?

I can download http://www.w6rz.net/adv8dvbt23.ts.
And there are many samples for dvbt sample ts files.
But, I want to convert my video file to TS file for dvbt.
First, I checked on google, but I cannot find any answer.
I think, this does not make sense, or, the way of thinking may have been wrong.
FFmpeg can used for this?
but, there is no any parmameter for Transmit mode, QAM / 64QAB, guard interval.
FFmpeg can used for this? but, there is no any parmameter for Transmit mode, QAM / 64QAB, guard interval.
As I explained already:
ffmpeg doesn't know anything about RF things like Constellation type; it is just a tool to transcode between different video formats. .ts is for "transport stream", and it's the video container format that DVB uses. The GNU Radio transmit flowgraphs on the other hand know nothing about video things – all they do is take the bits from a file. So that file needs to be in a format that the receiver would understand, and that's why I instructed you to use FFMPEG with the parameters you need. Since I don't know which bitrate you're planning on transmitting, I can't help you with how to use ffmpeg
So, you need to generate video data that your DVB-T receiver understands, but more importantly even, you need to put them in a container that ensures constant bitrate.
As pointed out in a different comment to your ham.stackexchange.com question about the topic, your prime source of examples would be GNU Radio's own gr-dtv module; when you look into gnuradio/gr-dtv/examples/README.dvbt, you'll find a link to https://github.com/drmpeg/dtv-utils , W6RZ's own tooling :)
There you'll find the tools necessary to calculate the exact stream bitrate you need your MPEG transport stream to have. Remember, a DVB-T transmitter has to transmit at a constant bits per second, so your video container must be constant-bitrate. That's why a transport stream pads the video data to achieve constant rate.
Then, you'll use ffmpeg to transcode your video and put into the transport stream container:
ffmpeg -re -i inputvideo.mpeg \
-vcodec mpeg2video \
-s 720x576 #resolution; this is a good choice, since most TVs will deal with it \
-r 25 #frames per second, use 25\
-flags cgop+ilme -sc_threshold 1000000000 #MPEG codec options\
-b:v 2M #Video *codec data* bit rate (defines video quality). Must be lower than stream bit rate, so < muxrate-(audio bitrate)\
-minrate:v 2M -maxrate:v 2M #enforce constant video bit rate\
-acodec mp2 -ac 2 -b:a 192k #audio codec, quality and bitrate\
-muxrate ${RATE FROM TOOL}
-f mpegts #specify you want a MPEG Transport Stream container as output\
outputfile.ts

FFMPEG frame extraction - stuck

trying to extract specific frames from a video with the following command (with specific names of files removed!:
ffmpeg -i video.mp4 -vf "select-gte(n\,6956)" -vframes 10262 folder/frame%d.jpg
However, in many cases, this results in the same frame (the first one) extracted repeatedly, rather than a progression of frames extracted.
The image sequence muxer, by default, is set to assume a constant frame rate output, so it will fill in missing timestamp gaps with duplicates.
The select filter does not reset timestamps, so, in your command, there's a "gap" from 0 to the timestamp of the first selected frame.
Use instead
ffmpeg -i video.mp4 -vf "select-gte(n\,6956)" -vsync 0 -vframes 10262 folder/frame%d.jpg
This changes video sync method to prevent frame duplication.

Framerate vs r vs Filter fps

I'm trying to better understand FFmpeg framerate.
Example: If I wanted to convert a 30 fps video to 23.976 fps.
What are the differences between:
Option
-framerate 24000/1001
Option
-r 24000/1001
Filter
-vf "fps=24000/1001"
x265 params
-x265-params "fps=24000/1001"
What I've read is:
-framerate is image sequence fps (input video fps?)
-vf "fps=" is encoding fps
-r is output fps
However I don't know if that is correct, or if it changes depending on what order you place them in the options.
Questions
-x265-params "fps=" Is it required use it's own fps param? Can it not use default options?
Should multiple Options, Filters ,and Params be combined, or should you only use one?
Input/Output Framerate
https://ffmpeg.org/ffmpeg.html#toc-Video-Options
-r[:stream_specifier] fps (input/output,per-stream)
If in doubt use -framerate instead of the input option -r.
Is -r input or output? How do you specify, by putting before or after the -i?
-framerate is an input per-file option. It is meant for input formats which don't have a framerate or PTS defined, image sequences being an example.
-r can be either an input or output option. As an input option, it retimes input frames at that rate. As an output option, it will duplicate or drop frames to achieve the given rate (note that it won't duplicate frames if output format accepts variable frame rate). Output r is the 'encoding rate' as well. If it is not specified, it is inherited from the input rate, whether that's manually set or altered or an inherent property of the stream.
fps filter allows one to alter a stream's framerate while filtering by dropping or duplicating frames to achieve the given rate. It overrides the input stream rate. Its main use is to manipulate a stream before combining it with other streams, or before filtering it further.
-x265-params fps is a private property of the x265 encoder. Its main purpose is to signal a duration for each frame for the purposes of rate-control. Encoders like x264/5 devote more bits to frames that are shown for longer. It does not actually alter framerate or number of frames or frame duration.

ffmpeg convert without loss quality

I need convert all videos to my video player (in website) when file type is other than flv/mp4/webm.
When I use: ffmpeg -i filename.mkv -sameq -ar 22050 filename.mp4 :
[h264 # 0x645ee0] error while decoding MB 22 1, bytestream (8786)
My point is, what I should do, when I need convert file type: .mkv and other(not supported by jwplayer) to flv/mp4 without quality loss.
Instead of -sameq (removed by FFMpeg), use -qscale 0 : the file size will increase but it will preserve the quality.
Do not use -sameq, it does not mean "same quality"
This option has been removed from FFmpeg a while ago. This means you are using an outdated build.
Use the -crf option instead when encoding with libx264. This is the H.264 video encoder used by ffmepg and, if available, is the default encoder for MP4 output. See the FFmpeg H.264 Video Encoding Guide for more info on that.
Get a recent ffmpeg
Go to the FFmpeg Download page and get a build there. There are options for Linux, OS X, and Windows. Or you can follow one of the FFmpeg Compile Guides. Because FFmpeg development is so active it is always recommended that you use the newest version that is practical for you to use.
You're going to have to accept some quality loss
You can produce a lossless output with libx264, but that will likely create absolutely huge files and may not be decodeable by the browser and/or be supported by JW Player (I've never tried).
The good news is that you can create a video that is roughly visually lossless. Again, the files may be somewhat large, but you need to make a choice between quality and file size.
With -crf choose a value between 18 to around 29. Choose the highest number that still gives an acceptable quality. Use that value for your videos.
Other things
Add -movflags +faststart. This will relocate the moov atom from the end of the file to the beginning. This will allow the video to begin playback while it is still being downloaded. Otherwise the whole video must be completely downloaded before it can begin playing.
Add -pix_fmt yuv420p. This will ensure a chroma subsampling that is compatible for all players. Otherwise, ffmpeg, by default and depending on several factors, will attempt to minimize or avoid chroma subsampling and the result is often not playable by non-FFmpeg based players.
convert all mkv to mp4 without quality loss (actually it is only re-packaging):
for %a in ("*.mkv") do ffmpeg.exe -i "%a" -vcodec copy -acodec copy -scodec mov_text "%~na.mp4"
For me that was the best way to convert it.
ffmpeg -i {input} -vcodec copy {output}
I am writing a script in python that appends multiple .webm files to one .mp4. It was taking me 10 to 20 seconds to convert one chunk of 5 seconds using:
ffmpeg -i {input} -qscale 0 copy {output}
There's some folders with more than 500 chunks.
Now it takes less than a second per chunk. It took me 5 minutes to convert a 1:20:00 long video.
For MP3, the best is to use -q:a 0 (same as '-qscale 0'), but MP3 has always loss quality.
To have less loss quality, use FLAC
See this documentation link

Resources