Bash loop over files in directory outputting non existent files - bash

Question

I am making mp4 files from a series of images. The images should run about 1 hour but I cannot get a full video because FFMPEG looks for a file it should not have.
[image2 # 0x55fe14bef700] Could not open file : /mnt/run/image-005437.jpeg
What is confusing is that the list that I pass to ffmpeg should not include that file. If my script does not like the result it sends the file to a sub directory in the target folder called failed
the location of the file with that number
$ pwd
run/failed
]$ ls
failed-005437.jpeg
the command I use to launch ffmpeg is the following
image_folder=$1
singularity exec --bind $work_dir:/mnt $work_dir/longleaf2.sif ffmpeg -f concat -safe 0 -y -i <(for f in /mnt/processed_images/$image_folder/image-%06d.jpeg; do echo "file '$f'";done) -vf "crop=trunc(iw/3)*2:trunc(ih/2)*2" -movflags +faststart /mnt/run/summary_files/${image_folder}.mp4
I have checked processed images and it is not there so why is ffmpeg looking for it?
pastebin of the failed run
https://pastebin.com/pF5ZefLf
My check that the file is not in the folder reference by the for loop so that it should never cause an error
$ ls image-005437.*
ls: cannot access image-005437.*: No such file or directory

Answer 1

Problem
When you run:
for f in /mnt/processed_images/$image_folder/image-%06d.jpeg; do echo "file '$f'";done
It will output:
file '/mnt/processed_images/foo/image-%06d.jpeg'
So then ffmpeg will use the sequence pattern type for the image demuxer. This expects a contiguous sequence.
Solution 1: Glob
Use a glob:
for f in /mnt/processed_images/$image_folder/*.jpeg; do echo "file '$f'";done
Now it will output each file. In this example image-000003.jpeg does not exist so it does not get listed:
file '/mnt/processed_images/foo/image-000001.jpeg'
file '/mnt/processed_images/foo/image-000002.jpeg'
file '/mnt/processed_images/foo/image-000004.jpeg'
Solution 2: Simplify and skip concat
Even better is to simplify your command by using the glob pattern type for the image demuxer within ffmpeg itself, and then you can avoid the concat demuxer:
image_folder=$1
singularity exec --bind "$work_dir":/mnt "$work_dir"/longleaf2.sif ffmpeg -pattern_type glob -framerate 25 -i "/mnt/processed_images/$image_folder/*.jpeg" -vf "crop=trunc(iw/3)*2:trunc(ih/2)*2,format=yuv420p" -movflags +faststart /mnt/run/summary_files/${image_folder}.mp4
The image demuxer glob pattern is not available for Windows users.
Added the -framerate input option for the image demuxer.
Added the format filter to make YUV 4:2:0 chroma subsampling for compatibility.
The variables have been quoted. See shellcheck.net.
FFmpeg 4.1 release branch is old. Download or compile a modern version before doing anything else.