How do I compose three overlapping videos w/audio in ffmpeg? - ffmpeg

I have three videos: let's call them intro, recording and outro. My ultimate goal is to stitch them together like so:
Both intro and outro have alpha (prores 4444) and a "wipe" to transition, so when overlaying, they must be on top of the recording. The recording is h264, and ultimately I'm encoding out for youtube with these recommended settings.
I've figured out how to make the thing work correctly for intro + recording:
$ ffmpeg \
-i intro.mov \
-i recording.mp4 \
-filter_complex \
"[1:v]tpad=start_duration=10:start_mode=add:color=black[rv]; \
[1:a]adelay=delays=10s:all=1[ra]; \
[rv][0:v]overlay[v];[0:a][ra]amix[a]" \
-map "[a]" -map "[v]" \
-movflags faststart -c:v libx264 -profile:v high -bf 2 -g 30 -crf 18 -pix_fmt yuv420p \
out.mp4 -y
However I can't use the tpad trick for the outro because it would render black frames over everything.
I've tried various iterations with setpts/asetpts as well as passing -itsoffset for the input, but haven't come up with a solution that works correctly for both video and audio. This tries to start the outro at 16 seconds into the recording (10s start + 16s of recording is how I got to setpts=PTS+26/TB). del, but doesn't work correctly, I get both intro and outro audio from the first frame, and the recording audio cuts out when the outro overlay begins:
$ ffmpeg \
-i intro.mov \
-i recording.mp4 \
-i outro.mov \
-filter_complex \
"[1:v]tpad=start_duration=10:start_mode=add:color=black[rv]; \
[1:a]adelay=delays=10s:all=1[ra]; \
[2:v]setpts=PTS+26/TB[outv]; \
[2:a]asetpts=PTS+26/TB[outa]; \
[rv][0:v]overlay[v4]; \
[0:a][ra]amix[a4]; \
[v4][outv]overlay[v]; \
[a4][outa]amix[a]" \
-map "[a]" -map "[v]" \
-movflags faststart -c:v libx264 -profile:v high -bf 2 -g 30 -crf 18 -pix_fmt yuv420p \
out.mp4 -y
I think the right solution lies in the direction of using setpts correctly but I haven't been able to wrap my brain fully around it. Or, maybe I'm making life complicated and there's an easier approach?
In the nice-to-have realm, I'd love to be able to specify the start of the outro relative to the end of the recording. I will be doing this to a bunch of recordings of varying lengths. It would be nice to have one command to invoke on everything rather than figuring out a specific timestamp for each one.
Thank you!

Use adelay for all audio adjustments. Perform all mixing in a single amix.
Set the outro overlay to start only at the correct timestamps.
Use
$ ffmpeg \
-i intro.mov \
-i recording.mp4 \
-i outro.mov \
-filter_complex \
"[1:v]tpad=start_duration=10:start_mode=add:color=black[mainv]; \
[1:a]adelay=delays=10s:all=1[maina]; \
[2:v]setpts=PTS+26/TB[outv]; \
[2:a]adelay=delays=26s:all=1[outa]; \
[mainv][0:v]overlay=eof_action=pass[previd]; \
[previd][outv]overlay=enable='gte(t,26)'[v]; \
[maina][0:a][outa]amix=inputs=3[a]; \
-map "[v]" -map "[a]" \
-c:v libx264 -profile:v high -bf 2 -g 30 -crf 18 -pix_fmt yuv420p \
-movflags +faststart \
out.mp4 -y

Related

Why does the mp4 from ffmpeg freeze during the last 3 seconds?

I'm trying to generate a perfectly looping mp4 from three inputs:
A background png image
An image sequence of transparent png images with the number of particles increasing
Another image sequence of transparent png images with the number of particles decreasing
I'm currently trying to achieve this with two commands (I have to use 'overlay' twice). The problem is that after the second command the video (test2.mp4) freezes for the last 3 seconds. Why does it happen? ARe there any other commands I could try to use?
First command:
ffmpeg -framerate 30 \
-pattern_type glob -i 'images/increase/*.png' \
-framerate 30 \
-i screens/Background.png \
-i audio/50-White-Noise-10min.mp3 \
-filter_complex "[1:v][0:v] overlay" \
-preset slow -c:a copy -shortest -c:v libx264 -pix_fmt yuv420p test.mp4
Second command:
ffmpeg -framerate 30 \
-pattern_type glob -i 'images/decrease/*.png' \
-i test.mp4 \
-filter_complex "[1:v][0:v] overlay" \
-preset slow -c:a copy -shortest -c:v libx264 -pix_fmt yuv420p test2.mp4
The solution was to do what Rajib commented: chain the overlay filters and do it all in one command:
ffmpeg -framerate 30 \
-i screens/Background.png \
-framerate 30 \
-pattern_type glob -i 'images/increase/*.png' \
-framerate 30 \
-pattern_type glob -i 'images/decrease/*.png' \
-i audio/50-White-Noise-10min.mp3 \
-filter_complex "[0][1] overlay[out],[out][2] overlay" \
-c:a copy -shortest -c:v libx264 -pix_fmt yuvj420p loop.mp4

ffmpeg combine audio mix code into complex concate script

I got currently 2 different ffmpeg scripts which I want to combine. I do not have good ffmpeg experience and those codes are mostly googel code so please be patient with me
The first code is concating 3 videos:
ffmpeg -y -i "$vid1" -i "$fp" -i "$vid1" -filter_complex \
"[0:v]scale=$cResolution:force_original_aspect_ratio=decrease,pad=$cResolution:(ow-iw)/2:(oh-ih)/2,setsar=1,fps=30,format=yuv420p[v0]; \
[1:v]scale=$cResolution:force_original_aspect_ratio=decrease,pad=$cResolution:(ow-iw)/2:(oh-ih)/2,setsar=1,fps=30,format=yuv420p[v1]; \
[2:v]scale=$cResolution:force_original_aspect_ratio=decrease,pad=$cResolution:(ow-iw)/2:(oh-ih)/2,setsar=1,fps=30,format=yuv420p[v2]; \
[0:a]aformat=sample_rates=48000:channel_layouts=stereo[a0]; \
[1:a]aformat=sample_rates=48000:channel_layouts=stereo[a1]; \
[2:a]aformat=sample_rates=48000:channel_layouts=stereo[a2]; \
[v0][a0][v1][a1][v2][a2]concat=n=3:v=1:a=1[v][a]; \
[v]drawtext=text='example..':y=h-line_h-$h3:x=w/30*mod(t\,20):enable='gt(mod(t,$dr2),$Introdr_rounded)'[v]; \
[v]drawtext=text='example..':y=h-line_h-$hcentral:x=w/20*mod(t\,100):enable='gt(mod(t,$dr2),$Introdr_rounded)'[v]; \
[v]drawtext=text='example..':y=h-line_h-23:x=w/30*mod(t\,20):enable='gt(mod(t,$dr2),$Introdr_rounded)'[v]" \
-map "[v]" -map "[a]" -c:v libx264 -crf 22 -preset veryfast -c:a aac -movflags +faststart "$fp_dest"
The second code is overlay a background mp3 in endless loop to the created video from above. Its important to know that this code does overlap the audio of the video and does not replace it. In future I will lower the volume of the mp3 files to work as background music
ffmpeg -y -i "$fp_dest" -filter_complex "amovie=$audio:loop=0,asetpts=N/SR/TB[aud];[0:a][aud]amix[a]" -map 0:v -map '[a]' -c:v copy -c:a aac -b:a 256k -shortest ./test.mp4
So currently I got 2 steps which I want to combine into 1 step. Can you please help me to include the second code into the first one without change any logic of the code?
Use amix to mix the music and the concated audio. stream_loop is applied to the music to loop it.
ffmpeg -y -i "$vid1" -i "$fp" -i "$vid1" -stream_loop -1 -i "$audio" -filter_complex \
"[0:v]scale=$cResolution:force_original_aspect_ratio=decrease,pad=$cResolution:(ow-iw)/2:(oh-ih)/2,setsar=1,fps=30,format=yuv420p[v0]; \
[1:v]scale=$cResolution:force_original_aspect_ratio=decrease,pad=$cResolution:(ow-iw)/2:(oh-ih)/2,setsar=1,fps=30,format=yuv420p[v1]; \
[2:v]scale=$cResolution:force_original_aspect_ratio=decrease,pad=$cResolution:(ow-iw)/2:(oh-ih)/2,setsar=1,fps=30,format=yuv420p[v2]; \
[0:a]aformat=sample_rates=48000:channel_layouts=stereo[a0]; \
[1:a]aformat=sample_rates=48000:channel_layouts=stereo[a1]; \
[2:a]aformat=sample_rates=48000:channel_layouts=stereo[a2]; \
[v0][a0][v1][a1][v2][a2]concat=n=3:v=1:a=1[v][a]; \
[a][3]amix=duration=first[a]; \
[v]drawtext=text='example..':y=h-line_h-$h3:x=w/30*mod(t\,20):enable='gt(mod(t,$dr2),$Introdr_rounded)'[v]; \
[v]drawtext=text='example..':y=h-line_h-$hcentral:x=w/20*mod(t\,100):enable='gt(mod(t,$dr2),$Introdr_rounded)'[v]; \
[v]drawtext=text='example..':y=h-line_h-23:x=w/30*mod(t\,20):enable='gt(mod(t,$dr2),$Introdr_rounded)'[v]" \
-map "[v]" -map "[a]" -c:v libx264 -crf 22 -preset veryfast -c:a aac -b:a 256k -movflags +faststart "$fp_dest"

FFMPEG zoom-pan multiple images

I am trying to stitch multiple images with some zoom-pan happening on the images to create a video.
Command:-
ffmpeg -f lavfi -r 30 -t 10 -i \
color=#000000:1920x1080 \
-f lavfi \
-r 30 -t 10 \
-i aevalsrc=0 \
-i "image-1.png" \
-i "image-2.png" \
-y -filter_complex \
"[0:v]fifo[bg];\
[2:v]setpts=PTS-STARTPTS+0/TB,scale=4455:2506:force_original_aspect_ratio=decrease,zoompan=z='min(zoom+0.0015,2.5)':x='iw/2-(iw/zoom/2)':y='ih/2-(ih/zoom/2)':d=150:fps='30':s='1920x1080'[v2];\
[bg][v2]overlay=0:0:enable='between(t,0, 5)'[bg];\
[3:v]setpts=PTS-STARTPTS+5.07/TB,scale=3840:2160:force_original_aspect_ratio=decrease,zoompan=z='min(zoom+0.0015,2.5)':x='iw/2-(iw/zoom/2)':y='ih/2-(ih/zoom/2)':d=150:fps='30':s='1920x1080'[v3];\
[bg][v3]overlay=0:0:enable='between(t,5, 10)'[bg];\
[1:a]amix=inputs=1:duration=first:dropout_transition=0" \
-map "[bg]" -vcodec "libx264" -preset "veryfast" -crf "15" "output.mp4"
The output is not as expected, it only zooms only on the first image, the second image is just static.
FFMPEG version - 4.1
Use
ffmpeg -f lavfi -i color=#000000:1920x1080:r=30:d=10 \
-f lavfi -t 10 -i anullsrc \
-i "image-1.png" \
-i "image-2.png" \
-filter_complex \
"[2:v]scale=4455:2506:force_original_aspect_ratio=decrease,zoompan=z='min(zoom+0.0015,2.5)':x='iw/2-(iw/zoom/2)':y='ih/2-(ih/zoom/2)':d=150:fps=30:s='1920x1080'[v2];\
[bg][v2]overlay=0:0:enable='between(t,0,5)'[bg];\
[3:v]scale=3840:2160:force_original_aspect_ratio=decrease,zoompan=z='min(zoom+0.0015,2.5)':x='iw/2-(iw/zoom/2)':y='ih/2-(ih/zoom/2)':d=150:fps=30:s='1920x1080',setpts=PTS+5/TB[v3];\
[bg][v3]overlay=0:0:enable='between(t,5,10)'[bg];\
-map "[bg]" -map 1:a -vcodec libx264 -preset veryfast -crf 15 -y "output.mp4"
For lavfi sources, it's best to set frame rate and duration where applicable within the filter.
Since you're not looping the images, -t won't have any effect. Since zoompan will set fps in its output, you can skip input rate setting. And since it's a single image, setpts before zoompan has no relevance. It should be set only on the zoompan whose timestamps need to be shifted.
Since you've only one audio, no point sending it to amix - there's nothing to mix with! Just map it directly.

Encode HEVC/H.265/HDR Video for YouTube from 10bit Pro-Res using FFmpeg

I want to have an HDR YouTube video published, my source file is either an Apple ProRes or DNxHR using a chroma subsamplig 4:4:4 or full RGB, both 10bit, so the original source file has all what is needed in order to be encoded into a 10bit 4:2:0 H.265/HEVC (HDR).
I have followed some answers listed here, reviewed lots of different approaches, tried out many different commands without success, colors aren't right when using only FFmpeg, to much red, when using only Adobe to encode into H.264 with the recommended settings on their support page, the results is darker, here are the commands I've using:
I have tried this:
ffmpeg \
-i input.mov \
-c:v libx265 \
-tag:v hvc1 \
-crf 21 \
-preset fast \
-pix_fmt yuv420p10le \
-x265-params "colorprim=bt2020:transfer=smpte2084:colormatrix=bt2020nc:master-display=G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,10):max-cll=1000,400" \
-c:a libfdk_aac \
-b:a 128k \
-ac 2 \
-ar 44100 \
-movflags +faststart \
output.mp4
And this:
ffmpeg \
-y \
-hide_banner \
-i input.mov \
-pix_fmt yuv420p10le \
-vf "scale=out_color_matrix=bt2020:out_h_chr_pos=0:out_v_chr_pos=0,format=yuv420p10" \
-c:v libx265 \
-tag:v hvc1 \
-crf 21 \
-preset fast \
-x265-params 'crf=12:colorprim=bt2020:transfer=smpte-st-2084:colormatrix=bt2020nc:master-display="G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1)":max-cll="1000,400"' \
-c:a libfdk_aac \
-b:a 128k \
-ac 2 \
-ar 44100 \
-movflags +faststart \
output.mp4
I have also tried using MKVToolNix in order to insert the metadata into the encoded HEVC/H.265 file with the following command:
/Applications/MKVToolNix-9.7.1.app/Contents/MacOS/mkvmerge \
-o output.mkv \
--colour-matrix 0:9 \
--colour-range 0:1 \
--colour-transfer-characteristics 0:16 \
--colour-primaries 0:9 \
--max-content-light 0:1000 \
--max-frame-light 0:300 \
--max-luminance 0:1000 \
--min-luminance 0:0.01 \
--chromaticity-coordinates 0:0.68,0.32,0.265,0.690,0.15,0.06 \
--white-colour-coordinates 0:0.3127,0.3290 \
input.mp4
But the result is the same and YouTube don't recognize the file as an HDR file, it does only with the first FFmpeg command and with the file encoded with Adobe Premiere, but the colors don't look well, so, maybe I'm getting some concept wrong, thanks for your help.

How to assemble three videos with cross fade using FFMPEG

I am trying to assemble 3 videos (static title) (main feature) (static trailer). The title and trailer are encoded text with the main feature being h264 encoded (at 6Mbs). The title and trailer have nul audio encoded. The specific goal is a crossfade between the three segments. I have concat working fine, but adding crossfade is causing me issues.
How does setpts=PTS-STARTPTS+(4/TB)[v2]; work?
This code puts it together, but the bit rate and errors are wrong.
ffmpeg -y -i title.mp4 -i vid.mp4 -i trailer.mp4 -f lavfi -i color=black:s=1920x1080 -filter_complex \
"[0:v]format=pix_fmts=yuva420p,fade=t=out:st=04:d=2:alpha=1,setpts=PTS-STARTPTS[v0]; \
[1:v]format=pix_fmts=yuva420p,fade=t=in:st=0:d=2:alpha=1,fade=t=out:st=6:d=1:alpha=1,setpts=PTS-STARTPTS+10/TB[v1]; \
[2:v]format=pix_fmts=yuva420p,fade=t=in:st=0:d=2:alpha=1,fade=t=out:st=2:d=1:alpha=1,setpts=PTS-STARTPTS+20/TB[v2]; \
[3:v]trim=duration=30[over]; \
[over][v0]overlay[over1]; \
[over1][v1]overlay[over2]; \
[over2][v2]overlay=format=yuv420[outv]" \
-vcodec h264_videotoolbox -b:v 6000k -maxrate 6000k -bufsize 6000000 -map [outv] merge.mp4

Resources