Out of sheer curiosity I would like to know how this quoting dilemma can be fixed.
I already solved the issue by circumnavigating it (I added [vcodec!*=av01] to the -f argument and simply removed the --exec part entirely). Otherwise it only worked, when there were no spaces or minus signs in the --exec argument.
The culprit line is the last and the issue is at the end with the --exec argument. You can ignore the rest.
Thanks for your help on the road to enlightenment! ;-)
#!/bin/bash
trap "exit" INT
avtomp4conv () {
# tests if the given file (in argument) is an AV1 media and if so, converts it to mp4
echo "${1}"
if ($( ffprobe -v error -select_streams v:0 -show_entries stream=codec_name -of default=noprint_wrappers=1:nokey=1 "${1}" | grep -i av > /dev/null )); then
echo "$1 bad codec"
ffmpeg -hide_banner -loglevel error -stats -i "${1}" -movflags faststart -preset ultrafast "${1%.mp4}_fixed.mp4" && mv "${1}" bogus/ && mv -n "${1%.mp4}_fixed.mp4" "${1}"
fi
}
# ... lotsa other stuff ...
export -f avtomp4conv
cat links.txt | parallel -u -I % --retries 3 --max-args 1 --jobs 4 python3 `which youtube-dl` -c -f "'bestvideo[height<=720][ext=mp4]+bestaudio[ext=m4a]/mp4'" --external-downloader aria2c --external-downloader-args "'-x 4 -s 4'" --exec \'bash -c \"export -f avtomp4conv\;avtomp4conv \{\}\"\' %
Use another function to save you from the double indirection in a single command (parallel executes youtube-dl that executes avtomp4conv). GNU parallel uses your current shell to execute its commands, so no need for bash -c here.
avtomp4conv () {
...
}
ytdl() {
youtube-dl ... --exec "bash -c 'avtomp4conv \"$0\"' {}"
}
export -f avtomp4conv ytdl
< links.txt parallel ... ytdl
Without the function ytdl you could try the following. But why bother with these nested quotes?
< links.txt parallel ... -I insteadOf{} \
"youtube-dl ... --exec \"bash -c 'avtomp4conv \\\"\$0\\\"' {}\""
Related
I have some audio recorded in Audacity 3.2.3 that I have exported as an mp3 and a flac. Then I have this file split_by_silence.sh
Which has hardcoded input path values that take an input file, split it by detecting silence, and then finally run an ffmpeg command to split the files. If you save the below code into a file split.sh, you can call it with the command $ ./split_by_silence.sh "value1" "value2"
# ./split_by_silence.sh "full_lowq.flac" %03d_output.flac
#IN=$1
#OUT=$2
OUT="%03d_output.flac"
IN="/mnt/e/martinradio/rips/vinyl/WIP/Dogs On Fire (1983, Vinyl)/dog on fire.flac"
OUTPUT_LOCATION="/mnt/e/martinradio/rips/vinyl/WIP/Dogs On Fire (1983, Vinyl)/"
true ${SD_PARAMS:="-18dB"};
true ${MIN_FRAGMENT_DURATION:="20"};
export MIN_FRAGMENT_DURATION
if [ -z "$OUT" ]; then
echo "Usage: split_by_silence.sh full.mp3 output_template_%03d.mp3"
echo "Depends on FFmpeg, Bash, Awk, Perl 5. Not tested on Mac or Windows."
echo ""
echo "Environment variables (with their current values):"
echo " SD_PARAMS=$SD_PARAMS Parameters for FFmpeg's silencedetect filter: noise tolerance and minimal silence duration"
echo " MIN_FRAGMENT_DURATION=$MIN_FRAGMENT_DURATION Minimal fragment duration"
exit 1
fi
#
# get comma separated list of split points (use ffmpeg to determine points where audio is at SD_PARAMS [-18db] )
#
echo "_______________________"
echo "Determining split points..." >& 2
SPLITS=$(
ffmpeg -v warning -i "$IN" -af silencedetect="$SD_PARAMS",ametadata=mode=print:file=-:key=lavfi.silence_start -vn -sn -f s16le -y /dev/null \
| grep lavfi.silence_start= \
| cut -f 2-2 -d= \
| perl -ne '
our $prev;
INIT { $prev = 0.0; }
chomp;
if (($_ - $prev) >= $ENV{MIN_FRAGMENT_DURATION}) {
print "$_,";
$prev = $_;
}
' \
| sed 's!,$!!'
)
echo "SPLITS= $SPLITS"
#
# Add 5 seconds to each of the comma separated numbers
#
# Convert the comma-separated string into an array
arr=($(echo $SPLITS | tr ',' '\n'))
# Initialize a new array to store the results
new_arr=()
# Iterate through each element and add 5 seconds of padding
for i in "${arr[#]}"; do
result=$(echo "$i + 5" | bc -l)
new_arr+=("$result")
done
# Convert the array back into a comma-separated string
NEW_SPLITS=$(IFS=,; echo "${new_arr[*]}")
# Print the result
echo "NEW_SPLITS= $NEW_SPLITS"
SPLITS=$NEW_SPLITS
#
# Print how many tracks should be exported
#
res="${SPLITS//[^,]}"
CHARCOUNT="${#res}"
num=$((CHARCOUNT + 2))
echo "Exporting $num tracks"
echo "_______________________"
#
# Split audio into individual tracks
#
current_directory=$(pwd)
cd "$OUTPUT_LOCATION"
echo "Running ffmpeg command: "
ffmpeg -i "$IN" -c copy -map 0 -f segment -segment_times "$SPLITS" "$OUT"
#ffmpeg -i "full_lowq.flac" -c copy -map 0 -f segment -segment_times "302.825,552.017" "%03d_output.flac"
echo "Done."
cd $current_directory
echo "running flac command"
# check flac file intrgrity
If I call this code for my flac file:
OUT="%03d_output.flac"
IN="/mnt/e/martinradio/rips/vinyl/WIP/Dogs On Fire (1983, Vinyl)/dog on fire.flac"
The outputted files have an incorrect metadata for the length. They all report as having the same length, but if i import any of them into audacity, the file has a correct length.
but if i run this for my mp3 file, we can see the correct length metadata:
OUT="%03d_output.mp3"
IN="/mnt/e/martinradio/rips/vinyl/WIP/Dogs On Fire (1983, Vinyl)/dogs on fire.mp3"
So there is something with my ffmpeg command that causes it to export flac files with wrong 'length' metadata
ffmpeg -i "$IN" -c copy -map 0 -f segment -segment_times "$SPLITS" "$OUT"
I've tried with the flac example to change -c copy to -c:a flac, but that just gives every output flac file a length of 00:00:00
is it a problem with my ffmpeg command? Or my files? https://file.io/tIFsa1l70076
it works for mp3 files just fine, why does it have this issue with flac?
I have created a script that takes the filename of screenshots from MPV player and grabs the time codes and cuts the video.
I like MPV because it is very fast on big movie files and hitting s (screenshot) for every in and out cut is very easy. I have not found any bash script (I can only do bash or are learning bash) that can do this, only lua and java scripts.
The bash script:
#!/bin/bash
clear
DATE=$(date +"%Y%m%d_%H%M%S")
x-terminal-emulator -geometry 50x20+3100+0 -e "bash -c 'while true; do clear;ls *.jpg;sleep 1;done' &"
rm *.jpg CUT*.mp4 cutLines.* cutMerge.*
mpv --screenshot-template="~/%F-(%P)-%03n" "$1"
echo
read -p "--- Hit ENTER to CUT ---"
echo
ls *.jpg | cut -c 24-35 > cutLines.txt
IFS=$'\n'
while IFS= read -r ONE; do read -r TWO
echo " Making cut for duration: $ONE - $TWO stored as: CUT_${ONE}.mp4"
ffmpeg -nostdin -loglevel quiet -ss "${ONE}" -to "${TWO}" -i "${1}" -c copy CUT_"${ONE}".mp4
echo CUT_"${ONE}".mp4 >> cutMerge.tmp
done < cutLines.txt
cat cutMerge.tmp | sed "s/^/file '/" |sed "s/$/'/" > cutMerge.txt
ffmpeg -f concat -safe 0 -i cutMerge.txt -c copy CUTmerge_"$DATE".mp4
The script works for the clips.
Here is the link where you see what I struggle with.
It looks like read line does not read all the data or something?
Video showing what the problem is
Thanks to Ed Morton's TIPS the script is now working!
It looks like the problem was missing double quotes and the ffmpeg option -nostdin that was the main problem for this script.
#!/bin/bash
clear
rm *.jpg cutLines.txt cutMerge.txt cutMerge.tmp
DATE=$(date +"%Y%m%d_%H%M%S")
x-terminal-emulator -geometry 50x20+3100+0 -e "bash -c 'while true; do clear;ls *.jpg;sleep 1;done' &"
mpv --screenshot-template="~/%F-(%P)-%03n" "$1"
echo
read -p "--- Hit ENTER to CUT ---"
echo
ls *.jpg | cut -c 24-35 > cutLines.txt
IFS=$'\n'
while IFS= read -r ONE; do read -r TWO
echo " Making cut for duration: $ONE - $TWO stored as: CUT_${ONE}.mp4"
ffmpeg -nostdin -loglevel quiet -ss "${ONE}" -to "${TWO}" -i "${1}" -c copy CUT_"${ONE}".mp4
echo CUT_"${ONE}".mp4 >> cutMerge.tmp
done < cutLines.txt
cat cutMerge.tmp | sed "s/^/file '/" |sed "s/$/'/" > cutMerge.txt
ffmpeg -f concat -safe 0 -i cutMerge.txt -c copy VideoMerged_"$DATE".mp4
echo -e "\n--- cutLines"
cat cutLines.txt
echo -e "\n--- cutMerge\n"
cat cutMerge.txt
rm *.jpg cutLines.txt cutMerge.txt cutMerge.tmp
mpv VideoMerged_"$DATE".mp4
A few years ago when I was younger, more carefree, and, uh, less cognisant of good practice in writing shell scripts; I wrote a quick-and-dirty script to assist with a task I was facing:
#!/bin/bash
# autohighlighter which generates highlights from a video clip and file
process_highlight_file() {
n=0
while read -r line; do
begin=$(echo "$line" | awk '{ print $1 }' )
end=$(echo "$line" | awk '{ print $2 }')
hilightname=$(echo "$line" | awk '{ print $3 }')
printf "Begin highlight called %s at %s, to %s\n" "$hilightname" "$begin" "$end"
echo "$begin $end"
sleep 2
echo "ffmpeg -y -ss $begin -i $videofile -to $end -c copy -avoid_negative_ts 1 $hilightname.mkv"
ffmpeg -loglevel quiet -hide_banner -y -ss "$begin" -i "$videofile" -to "$end" -c copy -avoid_negative_ts 1 "$hilightname.mkv"
if [ "$n" -eq 0 ]; then
echo -n "melt $hilightname.mkv " > constructed.melt
else
echo -n "$hilightname.mkv -mix 120 -mixer luma " >> constructed.melt
fi
(( n++ ))
done < $highlightfile
echo -n "-consumer avformat:$videofile-highlights.mkv crf=18" >> constructed.melt
}
highlightfile=$2
videofile=$1
process_highlight_file
exit 0
I call it with the video file name and a highlight file with the following tab-separated contents:
3:55 4:15 tutorialcomplete
10:50 11:15 firstkill
13:30 14:00 pickpocket
If I comment out the actual call to ffmpeg, I get sensible output:
Begin highlight called tutorialcomplete at 3:55, to 4:15
3:55 4:15
ffmpeg -y -ss 3:55 -i 2019-08-27 20-31-27.mkv -to 4:15 -c copy -avoid_negative_ts 1 tutorialcomplete.mkv
Begin highlight called firstkill at 10:50, to 11:15
10:50 11:15
ffmpeg -y -ss 10:50 -i /tmp/footage/gamefootage/pending/sor/2019-08-27 20-31-27.mkv -to 11:15 -c copy -avoid_negative_ts 1 firstkill.mkv
Begin highlight called pickpocket at 13:30, to 14:00
13:30 14:00
ffmpeg -y -ss 13:30 -i /tmp/footage/gamefootage/pending/sor/2019-08-27 20-31-27.mkv -to 14:00 -c copy -avoid_negative_ts 1 pickpocket.mkv
All good, if rather overkill on the debug output.
If I uncomment the call to ffmpeg, I get:
Begin highlight called tutorialcomplete at 3:55, to 4:15
3:55 4:15
ffmpeg -y -ss 3:55 -i /tmp/footage/gamefootage/pending/sor/2019-08-27 20-31-27.mkv -to 4:15 -c copy -avoid_negative_ts 1 tutorialcomplete.mkv
Begin highlight called at firstkill, to
firstkill
ffmpeg -y -ss firstkill -i /tmp/footage/gamefootage/pending/sor/2019-08-27 20-31-27.mkv -to -c copy -avoid_negative_ts 1 .mkv
Begin highlight called pickpocket at 13:30, to 14:00
13:30 14:00
ffmpeg -y -ss 13:30 -i /tmp/footage/gamefootage/pending/sor/2019-08-27 20-31-27.mkv -to 14:00 -c copy -avoid_negative_ts 1 pickpocket.mkv
Naturally, ffmpeg complains that "firstkill" is not a valid time to seek to. If I add further lines to the file I am passing in, it seems to only affect the second pass through the loop. Additional output is also produced:
Enter command: <target>|all <time>|-1 <command>[ <argument>]
The going theory is that a line isn't properly terminated. However, I can't seem to track that down in the script or the input.
I'm aware there are a number of poor practices, foibles and other unexpected behaviour here, for which past-me definitely takes the blame and hangs their head in shame! That said, why does calling ffmpeg in the loop here cause values from a file to be parsed incorrectly, or: why does commenting out the line with the call to ffmpeg give the variables the correct value. Also, why does it only affect the second pass of the loop? Where does Enter command come from?
Additionally, shellcheck.net doesn't complain about anything in the code.
You need to prevent FFMpeg from consuming the stdin characters stream.
See: man ffmpeg.1
-stdin
Enable interaction on standard input. On by default unless standard input is used as an input. To explicitly disable interaction you need to specify -nostdin.
Disabling interaction on standard input is useful, for example, if ffmpeg is in the background process group. Roughly the same result can be achieved with ffmpeg ... < /dev/null but it requires a shell.
Here in your loop, ffmpeg is consuming the input from your while loop commands block.
while read -r line; do
...
# here ffmpeg defaults to consuming the same input
# $highlightfile that is fed to the while loop commands block.
ffmpeg -loglevel quiet -hide_banner -y -ss "$begin" -i "$videofile" -to "$end" -c copy -avoid_negative_ts 1 "$hilightname.mkv"
done < $highlightfile
A fixed version of your code:
#!/usr/bin/env bash
# autohighlighter which generates highlights from a video clip and file
process_highlight_file() {
videofile="$1"
highlightfile="$2"
n=0
melt_caller=(melt)
while read -r begin end hilightname; do
printf 'Begin highlight called %s at %s, to %s\n' "$hilightname" "$begin" "$end"
ffmpeg_caller=(ffmpeg -nostdin -loglevel quiet -hide_banner -y -ss "$begin" -i "$videofile" -to "$end" -c copy -avoid_negative_ts 1 "$hilightname.mkv")
env printf '%q ' "${ffmpeg_caller[#]}" && echo
"${ffmpeg_caller[#]}"
((n++)) && melt_caller+=(-mix 120 -mixer luma)
melt_caller+=("$hilightname.mkv")
done <"$highlightfile"
melt_caller+=(-consumer "avformat:$videofile-highlights.mkv" 'crf=18')
env printf '%q ' "${melt_caller[#]}" && echo
"${melt_caller[#]}"
}
process_highlight_file "$#"
I have 2 wav files wav1 and wav2.I want to play wav2 every 45 seconds
over the wav1 using FFmpeg.
Ok, first create a 45-second file to loop of wav2
ffmpeg -i wav2 -af apad -t 45 wav2-padded.wav
Now, the mix
ffmpeg -i wav1 -f lavfi -i amovie=wav2-padded:loop=9999 -filter_complex [0][1]amix[out] -map [out] -shortest mixed.wav
I think this should accomplish what you want
#!/bin/bash
interval_millis=45000
regex="Duration: ([0-9:.]+)"
[[ $(ffmpeg -i $1 2>&1) =~ $regex ]]
duration_millis=$(echo "${BASH_REMATCH[1]}" | \
awk -F: '{ print 1000 * (($1 * 3600) + ($2 * 60) + $3) }')
duration_counter=$interval_millis
i=1
while [[ $duration_counter -lt $duration_millis ]]; do
inputs+=" -i $2"
delay_params+="[$i]adelay=$duration_counter[del$i];"
amix_params+="[del$i]"
((i++))
((duration_counter += $interval_millis))
done
ffmpeg -i $1 $inputs -filter_complex \
"$delay_params[0]${amix_params}amix=inputs=$i:duration=first" out.wav
I had some problems with adelay on a 2015 build of ffmpeg, so make sure you have an up-to-date version to run this. You call the script as
./script.sh long-file short-file
I have a rmvb file path list, and want to convert this files to mp4 files. So I hope to use bash pipeline to handle it. The code is
Convert() {
ffmpeg -i "$1" -vcodec mpeg4 -sameq -acodec aac -strict experimental "$1.mp4"
}
Convert_loop(){
while read line; do
Convert $line
done
}
cat list.txt | Convert_loop
However, it only handle the first file and the pipe exits.
So, does ffmpeg affect the bash pipe?
Caveat: I've never used ffmpeg, but in working with other questions concerning the program, it appears that, like ssh, ffmpeg reads from standard input without actually using it, so the first call to Convert is consuming the rest of the file list after read gets the first line. Try this
Convert() {
ffmpeg -i "$1" -vcodec mpe4 -sameq -acodec aac \
-strict experimental "$1.mp4" < /dev/null
}
This way, ffmpeg will not "hijack" data from standard input intended for read command.
[...]
for i in `cat list.txt`
Never use this syntax :
for i in $(command); do ...; done # or
for i in `command`; do ...; done
This syntax read the output of a command word by word and not row by row which often creates unexpected problems (like when row contain some spaces and when you want read a row like an item for example).
There always is a smarter solution :
command|while read -r; do ...; done # better general case to read command output in a loop
while read -r; do ...; done <<< "$(command)" # alternative to the previous solution
while read -r; do ...; done < <(command) # another alternative to the previous solution
for i in $DIR/*; do ...; done # instead of "for i in $(ls $DIR); do ...; done
for i in {1..10}; do ...; done # instead of "for i in $(seq 1 10); do ...; done
for (( i=1 ; i<=10 ; i++ )); do ...; done # such that the previous command
while read -r; do ...; done < file # instead of "cat file|while read -r; do ...; done"
while read -r || [[ -n $REPLY ]]; do ...; done < file # Same as before but also deal with files that doesn't have EOF.
# dealing with xargs or find -exec sometimes...
# ...
I wrote a course with more details about this subject and recurring errors, but in French, unfortunately :)
To answer the original question, you could use something like :
Convert() {
ffmpeg -i “$1” -vcodec mpe4 -sameq -acodec aac -strict experimental “$1.mp4”
}
Convert_loop(){
while read -r; do
Convert $REPLY
done < $1
}
Convert_loop list.txt
KISS ! =)
convert() {
ffmpeg -i "$1" \
-vcodec mpe4 \
-sameq -acodec aac \
-strict experimental "${1%.*}.mp4"
}
while read line; do
convert "$line"
done < list.txt