Is it possible to CLI ffmpeg to replace a specific frame at a specified interval with another image? I know how to extract all frames from a video, and re-stitch as another video, but I am looking to avoid this process, if possible.
My goal:
Given a video file input.mp4
Given a PNG file, image.png and given its known to occur at exactly a specific timestamp within input.mp4
create out.mp4 with image.png replacing that position of input.mp4
The basic command is
ffmpeg -i video -i image \
-filter_complex \
"[1]setpts=4.40/TB[im];[0][im]overlay=eof_action=pass" -c:a copy out.mp4
where 4.40 is the timestamp of the frame to be replaced.
Note that images default to a framerate of 25fps. Thus, for timestamps that are not multiples of (1/25=)0.04s, the framerate must be specified (eg, to replace frame at timestamp 3.5035 in a 29.97fps video):
ffmpeg -i input.vid -itsoffset 3.5035 -framerate 30000/1001 -i frame.png -filter_complex "[0:v:0][1]overlay=eof_action=pass" output.vid
This technique works just as well for replacing multiple sequential frames (eg, to replace frames starting at 107s in a 12.5fps video):
ffmpeg -i input.mp4 -itsoffset 107 -framerate 25/2 -i '107+%06d.png' -filter_complex "[0:v:0][1]overlay=eof_action=pass" output.mp4
This only works for videos with constant framerates (CFR). For VFR video, I have a separate question.
Related
I have a video that is interlaced and has a frame rate of 30000/1001.
If I deinterlace it using bwdif, and keeping one output frame per field, I get a fps of 60000/1001, as expected.
I want to convert this video to 50 fps keeping all frames, and thus slowing it a bit down.
If I do it with two ffmpeg commands, like the following, it works:
ffmpeg -i input.mp4 -filter_complex bwdif test.mp4
ffmpeg -i test.mp4 -filter_complex "[v1]setpts=60000/1001/50*PTS[v2];[v2]fps=50" test2.mp4
I would like to do it in one command. I tried the following command:
ffmpeg -i input.mp4 -filter_complex "[0]bwdif[v1];[v1]setpts=60000/1001/50*PTS[v2];[v2]fps=50" test.mp4
However, the result does not look as expected: some frames are duplicated and others are dropped.
Does anybody know why this is? Is there a correct way to do it in one command?
Machine learning algorithms for video processing typically work on frames (images) rather than video.
In my work, I use ffmpeg to dump a specific scene as a sequence of .png files, process them in some way (denoise, deblur, colorize, annotate, inpainting, etc), output the results into an equal number of .png files, and then update the original video with the new frames.
This works well with constant frame-rate (CFR) video. I dump the images as so (eg, 50-frame sequence starting at 1:47):
ffmpeg -i input.mp4 -vf "select='gte(t,107)*lt(selected_n,50)'" -vsync passthrough '107+%06d.png'
And then after editing the images, I replace the originals as so (for a 12.5fps CFR video):
ffmpeg -i input.mp4 -itsoffset 107 -framerate 25/2 -i '107+%06d.png' -filter_complex "[0]overlay=eof_action=pass" -vsync passthrough -c:a copy output.mp4
However, many of the videos I work with are variable frame-rate (VFR), and this has created some challenges.
A simple solution is to convert VFR video to CFR, which ffmpeg wants to do anyway, but I'm wondering if it's possible to avoid this. The reason is that CFR requires either dropping frames - since the purpose of ML video processing is usually to improve the output, I'd like to avoid this - or duplicating frames - but an upscaling algorithm that I'm working with right now uses the previous and next frame for data - if the previous or next frame is a duplicate, then ... no data for upscaling.
With -vsync passthrough, I had hoped that I could simply remove the -framerate option, and preserve the original frames as-is, but the resulting command:
ffmpeg -i input.mp4 -itsoffset 107 -i '107+%06d.png' -filter_complex "[0]overlay=eof_action=pass" -vsync passthrough -c:a copy output.mp4
uses ffmpeg's default of 25fps, and drops a lot of frames. Is there a reliable way to replace frames in VFR video?
Yes, it can be done, but it's complicated. It is crucial that the overlay video have exactly the same frame timestamps as the underlay video for this process to work reliably. Generating such a VFR video segment overlay requires capturing the frame timestamps from the source video to generate a precisely timed replacement segment.
The short version of the process is to replace the above commands with the following to extract the images:
ffmpeg -i input.mp4 -vf "select='gte(t,107)*lt(selected_n,50)',showinfo" -vsync passthrough '107+%06d.png' 2>&1 | 'sed s/\r/\n/g' | showinfo2concat.py --prefix="107+" >concat.txt
This requires a script that can be downloaded here. After editing the images, update the source video with:
ffmpeg -i input.mp4 -f concat -safe 0 -i concat.txt -filter_complex"[1]settb=1/90000,setpts=9644455+PTS*25/90000[o];[0:v:0][o]overlay=eof_action=pass" -vsync passthrough -r 90000 output.mp4
Where 90000 is the timescale (inverse of timebase), and 9644455 is the PTS of the first frame to replace.
See the source for more details about what these commands actually do.
Im using ffmpeg to change resolution of video file and after conversion to another location the video lasts 0 seconds, but originally it lasts 2mins
My ffmepg code:
ffmpeg -i input.mp4 -filter:v scale=480:320 -t 5 output.mp4
Why are you using the -t modifier? You want to cut the video up? Otherwise, this will convert the whole length to the new quality.
ffmpeg -i input.mp4 -filter:v scale480:320 output.mp4
I want asymmetrical side by side video with resolution 1920x1080. The first video has bitrate 1mb/s and the second video has bitrate 500kb/s. Both videos have the same resolution 1920x1080 and encoded h.265, container mp4.
I used ffmpeg code:
ffmpeg -i leftvideo.mp4 -i rightvideo.mp4 -filter_complex "[0:v] scale=iw/2:ih, pad=2*iw:ih [left]; [1:v] scale=iw/2:ih [right]; [left][right] overlay=main_w/2:0 [out]" -map [out] -c:v libx265 output.mp4
It works well but I want the resulting video quality while keeping. I don't want re-encoded.
Is it possible the two videos change resolution (960x1080) and together packed into container mp4?
EDIT: or another method?
Using ffmpeg
You are required to re-encode if you want to use filters in ffmpeg, but if you want to "keep the quality" you can use a lossless output:
ffmpeg -i left.mp4 -i right.mp4 -filter_complex \
"[0:v]scale=iw/2:ih[l];[1:v]scale=iw/2:ih[r];[l][r]hstack" \
-c:v libx264 -qp 0 output.mp4
The resulting file size may be huge. If this is not acceptable you can try a "visually lossless" output by changing -qp 0 to -crf 18.
You did not provide full details about your inputs, and did not mention audio, so I assumed you are not concerned with the audio.
You did not provide the complete console output from your command so I assumed your ffmpeg is new enough to use the hstack filter.
Using ffplay
Another option is to just use your player to play side-by-side and not even deal with re-encoding. Example using ffplay.
ffplay -f lavfi "movie=left.mp4,scale=iw/2:ih[v0];movie=right.mp4,scale=iw/2:ih[v1];[v0][v1]hstack"
With FFmpeg you have the option -t which will set the duration of the output. However I do not see a way to limit the duration of the input. Take this command
ffmpeg -i video.mp4 -c copy -t 60 out.mp4
This simply creates a 60 second clip of the original video. However if I wanted to clip the audio while keeping the full video stream, FFmpeg does not seem to have an option for this.
I have tried simply clipping the audio first, then combining the clipped audio with the video file, but this causes video/audio sync issues for me.
‘-aframes number (output)’
Set the number of audio frames to record. This is an alias for -frames:a.
§ Audio Options
ffmpeg -i video.mp4 -c copy -aframes 100 out.mp4
Use the "-itsoffset" option.
This makes the first 10 seconds mute.
ffmpeg -i video.mp4 -vn -acodec copy -ss 10.0 out_audio.mp4
ffmpeg -itsoffset 10.0 -i out_audio.mp4 -i video.mp4 -vcodec copy -acodec copy out.mp4