FFmpeg script skips files - bash

I wrote a shell script to convert many video files and save them with something appended to the file name. The script works, but it seems to randomly skip files, and a lot of them.
When I re-run the script, it will convert files it skipped before. How can I get it to stop skipping files?
workingDir=/home/user/Videos
# get list of files to convert
find /video/folder -iname "*.mp4" > $workingDir/file_list
# convert files
cat $workingDir/file_list | while read LINE; do
# FFmpeg often cuts off the beginning of this line
echo "$(dirname "$LINE")/$(basename "$LINE")"
if /usr/bin/ffmpeg -n -loglevel panic -v quiet -stats -i "$LINE" \
-c:v libx264 -vf scale="trunc(oh*a/2)*2:320" \
-pix_fmt yuv420p -preset:v slow -profile:v main -tune:v animation -crf 23 \
"$(dirname "$LINE")/$(basename "$LINE" \.mp4)"_reencoded.mp4 2>/dev/null; then
echo "Success: $(dirname "$LINE")/$(basename "$LINE")" >> $workingDir/results
else
echo "Failed: $(dirname "$LINE")/$(basename "$LINE")" >> $workingDir/results
fi
done
One problem seems to be that FFmpeg interferes with the script. The FFmpeg output often cuts off the beginning of the next command, even if the output is not shown. This is demonstrated by the echo line before the if statement, which is often cut off. But even for lines that aren't cut off, most of them will be skipped for no apparent reason.

ffmpeg reads from stdin, thereby consuming input meant for while read. Just redirect stdin for ffmpeg by adding < /dev/null

Related

handbrakecli/ffmpeg command to encode and auto filename(Ubuntu)

I have 2 preset .json files(from the GUI version on windows) to convert mkv to mp4.
converts to h264 and adds subtitle 1
converts to h264
I'm only trying to get no.2 to work at this stage.
for i in `*.mkv`; do HandBrakeCLI --preset-import-file HPRESET.json -Z "MYPRESET" --output *.mp4; done
no output name
HandBrakeCLI -i $1 --preset-import-gui HPRESET.json -Z "MYPRESET" --output $1.mp4
errors on output name
for i in `*.mkv`; do HandBrakeCLI --title $i --preset "Very Fast 1080p30" --output *.mp4; done
errors on output name AND not valid preset.
$ for i in `seq 4`; do HandBrakeCLI --input /dev/dvd --title $i --preset Normal --output NameOfDisc_Title$i.mp4; done
copied this from another stackoverflow question, but outputs as 1.mp4 and then 2.mp4 etc.
You can extract the filename without extension with something like that:
noext=${i%.*}
Example:
╰─$ for i in *.mkv; do echo "$i"; noext=${i%.*}; echo "$noext"; done
asdf.mkv
asdf
test.mkv
test
Same loop, different notation:
for i in *.mkv
do
#put the commands you want to run after "do" and before "done"
echo "$i"
noext=${i%.*}
echo "$noext"
done
Note that the for command will search any file in the current directory ending with .mkv. For each file it has found, it will save the files name into the variable $i, then execute the commands after do, then save the next files name into the variable $i and execute the commands between do and done. It will repeat that cycle until every file which has been found is processed.
As I have no experience with handbrake presets, here a solution with ffmpeg:
for i in *.mkv
do
#put the commands you want to run after "do" and before "done"
noext=${i%.*}
ffmpeg -i "$i" -c:v libx264 -c:a copy -c:s copy "$noext.mp4"
done

Plex DVR File Rename on FFMPEG Encoding

I'm currently using a bash shell script to encode all of my Plex DVR recordings to H.264 using FFMPEG. I'm using this little for loop I found online to encode all of the files in a single directory:
for i in *.ts;
do echo "$i" && ffmpeg -i "$i" -vf yadif -c:v libx264 -preset veryslow -crf 22 -y "/mnt/d/Video/DVR Stash/Seinfeld/${i%.*}.mp4";
done
This has served its purposes well but in the process I would like to rename the file to my preferred naming convention so that the original filename Seinfeld (1989) - S01E01 - Pilot.ts is renamed to Seinfeld S01 E01 Pilot.mp4 in the encoded file. While I am already using a regular expression to change the .ts extension to .mp4, but I'm no expert with regex, especially in the bash shell so any help would be appreciated.
For anyone that's interested in my Plex setup, I'm using an old machine running Linux Mint as my dedicated DVR and actually accessing it over the network with my daily driver which is a gaming machine, so more processing power for video encodes. While that one is a Windows machine, I'm using the Ubuntu bash under WSL2 to run my script, as I prefer it over the Windows command prompt or Powershell (my day job is a web developer on a company issued Mac). So here's a sample of my code for anyone that might consider doing something similar.
if [[ -d "/mnt/sambashare/Seinfeld (1989)" ]]
then
cd "/mnt/sambashare/Seinfeld (1989)"
echo "Seinfeld"
for dir in */; do
echo "$dir/"
cd "$dir"
for i in *.ts;
do echo "$i" && ffmpeg -i "$i" -vf yadif -c:v libx264 -preset veryslow -crf 22 -y "/mnt/d/Video/DVR Stash/Seinfeld/${i%.*}.mp4";
done
cd ..
done
fi
While I've considered doing a for loop for all shows, for now I am doing each show like this individually as there are a few shows I have custom encoding settings for
A small revision from your code, something like this, with extglob
#!/usr/bin/env bash
if [[ -d "/mnt/sambashare/Seinfeld (1989)" ]]; then
cd "/mnt/sambashare/Seinfeld (1989)" || exit
echo "Seinfeld"
for dir in */; do
echo "$dir/"
cd "$dir" || exit
for i in *.ts; do
shopt -s extglob
new_file=${i//#( \(*\)|- )}
new_file=${new_file/E/ E}
new_file=${new_file%.*}
echo "$i" &&
ffmpeg -i "$i" -vf yadif -c:v libx264 -preset veryslow -crf 22 -y "/mnt/d/Video/DVR Stash/Seinfeld/${new_file}.mp4"
shopt -u extglob
done
cd ..
done
fi
The string/glob/pattern slicing might fail if there is/are E's in the file name somewhere besides the episode.
With BASH_REMATCH using the =~ operator for Extended Regular Expression. This will work even if there are more E's in the filename.
#!/usr/bin/env bash
if [[ -d "/mnt/sambashare/Seinfeld (1989)" ]]; then
cd "/mnt/sambashare/Seinfeld (1989)" || exit
echo "Seinfeld"
for dir in */; do
echo "$dir/"
cd "$dir" || exit
for i in *.ts; do
regex='^(.+) (\(.+\)) - (S[[:digit:]]+)(E[[:digit:]]+) - (.+)([.].+)$'
[[ $i =~ $regex ]] &&
new_file="${BASH_REMATCH[1]} ${BASH_REMATCH[3]} ${BASH_REMATCH[4]} ${BASH_REMATCH[5]}"
echo "$i" &&
ffmpeg -i "$i" -vf yadif -c:v libx264 -preset veryslow -crf 22 -y "/mnt/d/Video/DVR Stash/Seinfeld/${new_file}.mp4"
done
cd ..
done
fi
Added a cd ... || exit just to make sure that the script stops/exits if there is/are errors when trying to cd to somewhere and not to continue the script.

Problems with ffmpeg input filenames when using bash script

I'm trying to use ffmpeg to convert some .m4a audio files to .mp3, and have come across something that has me stumped. I'd like to create the .mp3 in the same location and with the same filename as the .m4a, and so I'm using a combination of find/exec and a bash script to do this, as follows:
find /Volumes/Untitled/ -name '[!.]*' -name '*.m4a' -exec ./m4atomp3.sh {} \;
where m4atomp3.sh looks like:
#!/usr/bin/env bash
[[ -f "$1" ]] || { echo "$1 not found" ; exit 1 ; }
P="$1"
echo "$P is the full filename"
filename=${P%.*}
echo "$filename is the stripped filename"
m4afilename=\"$filename.m4a\"
echo "$m4afilename is the input filename"
mp3filename=\"$filename.mp3\"
echo "$mp3filename is the output filename"
mycmd="/Users/nickstyles/Downloads/ffmpeg -i "$m4afilename" -codec:a libmp3lame -qscale:a 2 -nostdin "$mp3filename
echo $mycmd
$mycmd
Whenever I try this, it fails because ffmpeg doesn't find the file, seemingly because of the whitespace in the filename, e.g if the file was called /Volumes/Untitled/My M4As/My M4A.m4a I would see:
ffmpeg version N-99346-g003b5c800f-tessus https://evermeet.cx/ffmpeg/ Copyright (c) 2000-2020 the FFmpeg developers
built with Apple clang version 11.0.0 (clang-1100.0.33.17)
[configuration details]
"/Volumes/Untitled/My: No such file or directory
However, if I just paste what is returned by echo $mycmd into the command line, e.g:
/Users/nickstyles/Downloads/ffmpeg -i "/Volumes/Untitled/My M4As/My M4A.m4a" -codec:a libmp3lame -qscale:a 2 -nostdin "/Volumes/Untitled/My M4As/My M4A.mp3"
then it works absolutely fine. I'm sure I'm missing something very obvious, which hopefully someone can spot!
As Benjamin W. pointed out the problem was that the variable was still getting split by bash, due to WordSplitting, and the quotes I was adding to the content of the variable were not helping against this. The key was to ensure that the quotes were placed around the variable itself like:
m4afilename=$filename.m4a
echo "$m4afilename is the input filename"
mp3filename=$filename.mp3
echo "$mp3filename is the output filename"
/Users/nickstyles/Downloads/ffmpeg -i "$m4afilename" -codec:a libmp3lame -qscale:a 2 -nostdin "$mp3filename"
and now this works!
Try this : mycmd="/Users/nickstyles/Downloads/ffmpeg -i $m4afilename -codec:a libmp3lame -qscale:a 2 -nostdin $mp3filename"
In bash, you can put variable straight into double quotes.

While loop in bash to read a file skips first 2 characters of THIRD Line

#bin/bash
INPUT_DIR="$1"
INPUT_VIDEO="$2"
OUTPUT_PATH="$3"
SOURCE="$4"
DATE="$5"
INPUT="$INPUT_DIR/sorted_result.txt"
COUNT=1
initial=00:00:00
while IFS= read -r line; do
OUT_DIR=$OUTPUT_PATH/$COUNT
mkdir "$OUT_DIR"
ffmpeg -nostdin -i $INPUT_VIDEO -vcodec h264 -vf fps=25 -ss $initial -to $line $OUT_DIR/$COUNT.avi
ffmpeg -i $OUT_DIR/$COUNT.avi -acodec pcm_s16le -ar 16000 -ac 1 $OUT_DIR/$COUNT.wav
python3.6 /home/Video_Audio_Chunks_1.py $OUT_DIR/$COUNT.wav
python /home/transcribe.py --decoder beam --cuda --source $SOURCE --date $DATE --video $OUT_DIR/$COUNT.avi --out_dir "$OUT_DIR"
COUNT=$((COUNT + 1))
echo "--------------------------------------------------"
echo $initial
echo $line
echo "--------------------------------------------------"
initial=$line
done < "$INPUT"
This is the code I am working on.
The contents of file sorted_results.txt are as follows:
00:6:59
00:7:55
00:8:39
00:19:17
00:27:48
00:43:27
While reading the file it skips first two characters of the third line i.e. it takes it as :8:39 which results in the ffmpeg error and the script stops.
However when I only print the variables $INITIAL and $LINE, commenting the ffmpeg command the values are printed correctly i.e. same as the file contents.
I think the ffmpeg command is somehow affecting the file reading process or the variable value. BUT I CAN'T UNDERSTAND HOW?
PLEASE HELP.
Your bash read builtin command and the second ffmpeg command (for the audio) both read from STDIN, that is why they interfere with each other. You can either also specify -nostdin there or use another file descriptor (here number 3 is used) for read:
while IFS= read -r -u 3 line; do
...
done 3< "$INPUT"

How to run ffmpeg in a loop and test it for error?

I have stuttering, seeking, and general playback issues when playing large mkv files through my Plex Media Server setup. I have been looking around for a way to automate scheduled tasks to move everything to mp4. The objective is:
Copy mkv files into mp4 preserving subtitles of every kind. Put the new file in the same subdir, and delete previous mkv version if conversion went successful.
When I tried to run ffmpeg on a loop, I run into the problem described here:
https://unix.stackexchange.com/questions/36310/strange-errors-when-using-ffmpeg-in-a-loop
This is my first adventure on shell scripting and I am pretty much stumbling around and trying to understand the syntax and philosophy of it. What I understand is that they use a file descriptor to redirect ffmpeg output to /dev/null.
The problem with that solution is that I would need to check ffmpeg output for errors to decide whether to delete the previous file or not. Furthermore, there is a common error when converting from picture based subtitles streams, which I circumvent by using a script I found (http://www.computernerdfromhell.com/blog/automatically-extract-subtitles-from-mkv/) to work after some modifications to my needs.
After much frustration I ended modifying the script so much that it does not serve to its purpose. It does not check for errors. Anyways, I will post it here. Mind you that this is my first shell script ever, and almost everything is confusing about it. The problem with this, is that I had to ditch my error checking and I am eliminating files that errored when converting. Losing the original without a valid copy.
#!/bin/bash
FOLDERS=( "/mnt/stg4usb/media0/test/matroska1" "/mnt/stg4usb/media0/test/season1" "/mnt/stg4usb/media0/test/secondtest")
FLAGS="-y -metadata title="" -c:v copy -map 0 -c:a libfdk_aac -ac 2 -movflags +faststart"
COUNTER=0
LOGFILE=batch-$(date +"%Y%m%d-%H%M%S").log
for FOLDER in "${FOLDERS[#]}"
do
echo "---===> STARTING folder: '$FOLDER'"
find $FOLDER -name "*.mkv" | while read line; do
OUTPUT=""
DATE=$(date +"%Y-%m-%d")
TIME=$(date +"%H:%M:%S")
COUNTER=$((COUNTER+1))
FILE=$(basename "$line")
DIR=$(dirname "${line}")
echo $'\n'$'\n'"[$COUNTER][$DATE][$TIME][FILE:'${line%.mkv}.mp4']"$'\n'
echo "#### Transcoding ####"'\n'
ffmpeg -i $line $FLAGS -sn "${line%.mkv}.mp4" < /dev/null
echo "#### Extracting subtitles ###"'\n''\n'
mkvmerge -i "$line" | grep 'subtitles' | while read subline
do
# Grep the number of the subtitle track
tracknumber=`echo $subline | egrep -o "[0-9]{1,2}" | head -1`
# Get base name for subtitle
subtitlename=${line%.*}
# Extract the track to a .tmp file
mkvextract tracks "$line" $tracknumber:"$subtitlename.$tracknumber.srt" < /dev/null
chmod g+rw "$subtitlename.$tracknumber"* < /dev/null
done
rm -frv "$line" < /dev/null
echo "Finished: $(date +"%Y%m%d-%H%M%S")"
done
echo '\n'"<===--- DONE with folder: '$FOLDER'"$'\n'$'\n' >> $LOGFILE
done
exit 0
So, basically, the idea is: run ffmpeg on a loop for all mkv under a directory and subdirectories (I was using find). Check it for all possible errors. If errors, try again without subtitles and extract the subtitles using mkvextract, else everything went ok, and delete the previous file.

Resources