FFmpeg concat and then add soundtrack results in a soundtrack that stutters - ffmpeg

I am trying to create a video composed of clips of images and videos. For the clips of images, I use ffmpeg to create a video file and then I add a silent audio stream through these two steps:
ffmpeg.exe -loop 1 -i MyImage.png -codec:v libx264 -t 4.0 -profile:v high -preset slow -r 25 -b:v 500k -maxrate 500k -pix_fmt yuv420p -vf scale=1280:720 MyImageMovie.mp4
ffmpeg.exe -f lavfi -i anullsrc=r=48000 -i MyImageMovie.mp4 -shortest -c:v copy -c:a aac -strict experimental -y MyImageMovieWithSilentAudioStream.mp4
Then I combine my video clips and image clips with
ffmpeg.exe -f concat -i videoList.txt -c copy -y concatVideo.mp4
At this point, the video looks good, any video clips that have audio streams seemed well synced to the video.
Now I add a soundtrack:
ffmpeg.exe -i concatVideo.mp4 -i soundtrack.mp3 -ar 48000 -filter_complex "[1:a]apad [b] ; [0:a][b]amerge=inputs=2[a]" -map 0:v -map "[a]" -c:v copy -ac 2 -shortest -y FinalVideo.mp4
The problem is that the soundtrack on FinalVideo.mp4 stutters at some (not all) of the concatenation joints.
I suspect it has to do with the audio stream and the video stream of the Image clips not being perfectly aligned. The aac has .0231s resolution and the video has 0.04s resolution. When I ffprobe the MyImageMovieWithSilentAudioStream.mp4 the duration is 4.00s but the start is 0.0213.
If my concatenated video has several of these image clips, the error can start to accumulate.
What can I do to keep the video and audio in sync and add a soundtrack that doesn't stutter?
Also, this is a little interesting, I don't hear the stutter when the final video is played on Windows Media Player, but it is there if I play it on VLC or via the html native video element.

Try adding the soundtrack in the same step as the concat.
ffmpeg -f concat -i videoList.txt -i soundtrack.mp3 \
-filter_complex "[1:a]apad[b];[0:a][b]amerge=inputs=2[a]" \
-map 0:v -map "[a]"
-c:v copy -c:a aac -ac 2 -ar 48000 -shortest -y FinalVideo.mp4
As an aside, you can also combine the image and silent stream generation,
ffmpeg -loop 1 -i MyImage.png -f lavfi -i anullsrc=r=48000 \
-vf scale=1280:720 \
-c:v libx264 -profile:v high -preset slow -r 25 -b:v 500k -maxrate 500k -pix_fmt yuv420p \
-c:a aac -strict experimental -t 4 -y MyImageMovieWithSilentAudioStream.mp4

Related

FFMPEG: Combine "Create video from images" + scale to x + add audio + overlay logo

I´m working on a webcam-project. It is for generating timelapse videos of sunset/sundown.
I´m using a raspberrypi to generate them with gphoto2 + DSLR.
At the end of the day the images should get to an video, with audio and an overlay logo.
And it should be scaled to 1920 pixel.
I got a nice solution an it worked.
Producing the timelapse video an scale it:
ffmpeg -y -framerate 25 -start_number 0000001 -i /var/www/html/webcam/2020-01-05_bilder/%7d.jpg -vf scale=1920:-1 -pix_fmt yuv420p /var/www/html/webcam/2020-01-05-tag-output-1920.mp4
Taking the output of (1) and add an overlay-logo, add audio
ffmpeg -y -i '/var/www/html/webcam/2020-01-05-tag-output-1920.mp4'
-i '/var/www/html/webcam-scripts/graphics/logo.png'
-i '/var/www/html/webcam-scripts/sounds/chill_time_5.mp3'
-shortest -filter_complex '[1][0]scale2ref=h=ow/mdar:w=iw/6[#A logo][liebfrauen]; [#A logo]format=argb,colorchannelmixer=aa=0.95[#B logo transparent]; [liebfrauen][#B logo transparent] overlay=(main_w-w)-(main_w*0.05):(main_h-h)-(main_h*0.01)'
-c:v libx264 -crf 18 -preset slow -pix_fmt yuv420p -c:a aac -strict -2
'/var/www/html/webcam/2020-01-05-tag-1920.mp4
I tried to combine both actions, but I get an error:
ffmpeg -y -framerate 25 -start_number 0000001 -i '/var/www/html/webcam/2020-01-05_bilder/%7d.jpg' -vf scale=1920:-1 -pix_fmt yuv420p -i '/var/www/html/webcam-scripts/graphics/logo.png' -i '/var/www/html/webcam-scripts/sounds/chill_time_5.mp3' -shortest -filter_complex '[1][0]scale2ref=h=ow/mdar:w=iw/6[#A logo][liebfrauen]; [#A logo]format=argb,colorchannelmixer=aa=0.95[#B logo transparent]; [liebfrauen][#B logo transparent] overlay=(main_w-w)-(main_w*0.05):(main_h-h)-(main_h*0.01)' -c:v libx264 -crf 18 -preset slow -pix_fmt yuv420p -c:a aac -strict -2 '/var/www/html/webcam/2020-01-05-tag-1920.mp4'
Error: Filtergraph 'scale=720:-1' was specified through the -vf/-af/-filter option for output stream 0:0, which is fed from a complex filtergraph.
-vf/-af/-filter and -filter_complex cannot be used together for the same stream.
Isn`t it possible to combine these inputs and scale it? Or ... Where is my misunderstanding?
Don't mix -vf and -filter_complex. Do all filtering in one filtergraph.
ffmpeg -y -framerate 25 -i '/var/www/html/webcam/2020-01-05_bilder/%7d.jpg' -i '/var/www/html/webcam-scripts/graphics/logo.png' -i '/var/www/html/webcam-scripts/sounds/chill_time_5.mp3' -filter_complex '[0]scale=1920:-2[v0];[1][v0]scale2ref=h=ow/mdar:w=iw/6[#A logo][liebfrauen]; [#A logo]format=argb,colorchannelmixer=aa=0.95[#B logo transparent]; [liebfrauen][#B logo transparent] overlay=(main_w-w)-(main_w*0.05):(main_h-h)-(main_h*0.01),format=yuv420p' -c:v libx264 -crf 18 -preset slow -c:a aac -shortest '/var/www/html/webcam/2020-01-05-tag-1920.mp4'
No need for -strict -2. It does nothing for modern ffmpeg.
I replaced -pix_fmt yuv420p with format=yuv420p so it is more organized.
-start_number 0000001 is not needed because 1 is the default.

Combine Two Commands (Get Video from Images)

I have 300 images and i wants to generate video from these images.
i am new to FFMPEG so now i am using two commands to generate video from images.
Command to generate video from images which also add Logo on video
ffmpeg -framerate 24 -i img_%d.jpg -i logo.png -filter_complex \
"[0:v][1:v] overlay=25:25:enable='between(t,0,20)'" \
-vcodec libx264 -crf 25 -pix_fmt yuv420p test_video.mp4
After using above command i am getting the video to add audio to this video i am using below command
ffmpeg -i test_video.mp4 -i inputfile.mp3 -c:v libx264 -c:a libvorbis -shortest final_video.mp4
which generates video and i am getting below message
MPEG-4 AAC decoder is required to play the file
Help to combine this both command. if possible can we add sound without any decoder required
Log for command 1 https://drive.google.com/file/d/1zS7gvrPy69VK_MkyE4127FpX2kEziJHq/view?usp=sharing
and Log command 2 https://drive.google.com/file/d/1rHqVGzj7f003aWP6eISiyUjsES8_EWuw/view?usp=sharing
Try next command:
ffmpeg -framerate 24 -i img_%d.jpg -i logo.png -i inputfile.mp3 -filter_complex \
"[0:v][1:v] overlay=25:25:enable='between(t,0,20)'" \
-vcodec libx264 -crf 25 -map 2:a -c:a copy -pix_fmt yuv420p -shortest test_video.mp4
-map 2:a is needed to skip image in case if there is cover image in track.
With -c:a copy track will not be re-encoded, so you will have mp3 inside of your video file.

FFMPEG image not updating

THE INPUT FILES
An overlay image that has is being updated every 5 seconds by a Python script
A small MP4 file that will be looped by a concat input
An MP3 file as audio source
THE COMMAND (UPDATED)
This is the command I'm currently using to combine and stream the inputs.
ffmpeg -re -i music.mp3 -f concat -i videoincludes.txt
-r 1 -loop 1 -f image2 -i overlay.png
-c:v libx264 -c:a aac -shortest -crf 23 -pix_fmt yuv420p
-maxrate 2500k -bufsize 2500k -preset ultrafast -r 30 -g 60 -b:v 2000k -b:a 192k -ar 44100
-filter_complex "[1:v][2:v] overlay=0:0" -map 0:a -strict -2
-f flv rtmp://a.rtmp.youtube.com/live2/{key}
Als tried using -framerate 1 instead of -r 1
THE ISSUE
So the issue is that the image doesn't always update. Sometimes it does update every couple seconds at the start but it stops updating after 10-20 seconds without any difference in log output and sometimes it just doesn't update.
I can however confirm that the image is being updated by the Python script but FFmpeg is just not picking this up.
I read setting the input format of the image to image2 should allow it to update so I am not sure what is wrong or what I can do to improve it.
I'm working on the same task, and finally, I think, I found the answer.
Because streams different from each other we must reset their timestamps with setpts=PTS-STARTPTS to have them begin in the same zero timestamp . And, also, try to use image2pipe instead of image2.
This is your code with timestamp reset:
ffmpeg -re -i music.mp3 -f concat -i videoincludes.txt
-r 1 -loop 1 -f image2pipe -i overlay.png
-c:v libx264 -c:a aac -shortest -crf 23 -pix_fmt yuv420p
-maxrate 2500k -bufsize 2500k -preset ultrafast -r 30 -g 60 -b:v 2000k -b:a 192k -ar 44100
-filter_complex "[1:v]setpts=PTS-STARTPTS[out_main]; [2:v]setpts=PTS-STARTPTS[out_overlay]; [out_main][out_overlay]overlay=0:0" -map 0:a -strict -2
-f flv rtmp://a.rtmp.youtube.com/live2/{key}
p.s and I think, there is no need in -r or -framerate anymore

ffmpeg audio watermark at specific time

I'm looking for a way to add an audio watermark, on specific time, to a video file (with existing audio) . something like: ffmpeg -i mainAVfile.mov -i audioWM.wav -filter_complex "[0:a][1:a] amix=inputs=2:enable='between(t,9,10)' [aud]; [0:v][aud]" -c:v libx264 -vf "scale=1280:720:sws_dither=ed:flags=lanczos, setdar=16:9" -c:a libfdk_aac -ac 2 -ab 96k -ar 48000 -af "aformat=channel_layouts=stereo, aresample=async=1000" -threads 0 -y output.mp4
The above command gives me this error Timeline ('enable' option) not supported with filter 'amix'. amerge didn't work as well. I kind of get lost with filter_complex syntax, specifically with the following conditions
On the main AV file, both audio and video tracks are filtered
Watermark should be between the 9th and 10th second (I already
generated a 1 second, 10k tone file)
The watermark need to survive the proceeding audio transcode
Use
ffmpeg -i mainAVfile.mov -i audioWM.wav
-filter_complex
"[0:a]aformat=channel_layouts=stereo,aresample=async=1000[main];
[1:a]atrim=0:1,adelay=9000|9000[wm];[main][wm]amix=inputs=2"
-vf "scale=1280:720:sws_dither=ed:flags=lanczos,setdar=16:9" -c:v libx264
-c:a libfdk_aac -ac 2 -ar 48000 -b:a 96k
-threads 0 -y output.mp4
It's preferable to perform all filtering in a single filtergraph. But I've kept the video filter as-is.

FFmpeg First 2 Seconds of Video Not Showing

This code works fine for some audio files (makes a slideshow of JPG pictures with a PNG watermark and MP3 audio, while maintaining aspect ratio) but for this audio file, the pictures are not showing for the first two seconds or so of the video:
ffmpeg -y -framerate 1/12 -i "media/%03d.jpg" -i "media/audio.mp3" -loop 1 -i "media/watermark.png" -filter_complex "[0:v]scale=iw*min(3840/iw\,2160/ih):ih*min(3840/iw\,2160/ih), pad=3840:2160:(3840-iw)/2:(2160-ih)/2[ss]; [ss][2:v] overlay=main_w-overlay_w-10:main_h-overlay_h-10:shortest=1[out]" -map "[out]" -map 1:a -c:v libx264 -r 24 -preset veryfast -tune stillimage -pix_fmt yuv420p -c:a copy -map_metadata -1 "media/video.mkv" -report
I tried converting the audio into different formats of MP3, tried changing bitrates, changed audio to stereo, and even tried converting it to a WAV. None of these things worked.
Here are the report results for when I run this command.
If it makes a difference, I'm using Ubuntu 14.04 and FFmpeg version N-77455-g4707497 (latest version).
This command should work, but I consider this bizarre behaviour as FFmpeg should be automatically padding frames as per output spec
ffmpeg -y -framerate 1/12 -i "media/%03d.jpg" -i "media/audio.mp3" -loop 1 -i "media/watermark.png" -filter_complex "[0:v]scale=iw*min(3840/iw\,2160/ih):ih*min(3840/iw\,2160/ih), pad=3840:2160:(3840-iw)/2:(2160-ih)/2,fps=24[ss]; [ss][2:v] overlay=main_w-overlay_w-10:main_h-overlay_h-10:shortest=1[out]" -map "[out]" -map 1:a -c:v libx264 -r 24 -preset veryfast -tune stillimage -pix_fmt yuv420p -c:a copy -map_metadata -1 "media/video.mkv"

Resources