here's a script.. just wanna practise some bash skills and make a quick util for my chinese mp4 player =)
#!/bin/bash
#####################################
# RockChip 4gb Player mencoder preset
#####################################
TOOL='mencoder'
OUTS='./out/'
OPT1='-noodml'
OPT2="-of avi -ofps 22 -vf-add scale=320:-2,expand=320:240 -srate 44100 -ovc xvid -xvidencopts bitrate=400:max_bframes=0:quant_type=s16le -oac lavc -lavcopts acodec=mp2:abitrate=128"
bold=`tput bold`
normal=`tput sgr0`
# check does argument exists
if test -z "$1"; then
echo "There's no file given =)"
fi
# Check is it dir or file
if [ -d $1 ]; then
echo "Directory is given: $1"
# Test if output argument is given
if [ -z $2 ]; then
echo "No output argument given using default: ${bold}${red}$OUTS${normal}"
mkdir out
else
# test is given path a directory
if [ -d $2 ]; then
OUT="$2"
else
echo "Output argument is not a directory"
fi
fi
OLD_IFS=IFS; IFS=$'\n'
for file in `find . -name "*.*" -type f | sed 's!.*/!!'` ; do
file=`printf "%q" "$file"`
echo ${TOOL} ${OPT1} ${file} -o ${OUTS}${file} ${OPT2}
done
IFS=OLD_IFS
fi
Problem is this line:
echo ${TOOL} ${OPT1} ${file} -o ${OUTS}${file} ${OPT2}
When you remove echo , to execute command, command fails, but if you'll copy this echoed script, and execute it manually everything works.
When executing command from shell script output is :
MEncoder 1.0rc4-4.2.1 (C) 2000-2010 MPlayer Team
158 audio & 340 video codecs
-of avi -ofps 22 -vf-add scale=320:-2,expand=320:240 -srate 44100 -ovc xvid -xvidencopts bitrate=400:max_bframes=0:quant_type=s16le -oac lavc -lavcopts acodec=mp2:abitrate=128 is not an MEncoder option
Exiting... (error parsing command line)
as I mentioned before executing command manualy everything works for example:
mencoder -noodml 12\ I\ Love\ You\ 1\ \ I\ Love\ You\ 2\ \ I\ Love\ You\ 3.avi -o ./out/12\ I\ Love\ You\ 1\ \ I\ Love\ You\ 2\ \ I\ Love\ You\ 3.avi -of avi -ofps 22 -vf-add scale=320:-2,expand=320:240 -srate 44100 -ovc xvid -xvidencopts bitrate=400:max_bframes=0:quant_type=s16le -oac lavc -lavcopts acodec=mp2:abitrate=128
now all I can do is to copy paste generated commands.. where's the problem? I tried to google really hard.. with no result... (I know that mencoder have profiles.. it's not the case where I want them)
You have (line 37 i believe):
OUT="$2"
but I think you meant:
OUTS="$2"
I'm not fully sure but maybe it's better to quote the file name with double quotes (") instead of doing printf "%q" "$file".
So replace:
file=`printf "%q" "$file"`
${TOOL} ${OPT1} ${file} -o ${OUTS}${file} ${OPT2}
with
${TOOL} ${OPT1} "${file}" -o "${OUTS}${file}" ${OPT2}
First, use $() instead of back ticks.
bold=$(tput bold)
normal=$(tput sgr0)
OLD_IFS=IFS; IFS=$'\n' should be OLD_IFS=$IFS . you want to get the value of IFS, so put a dollar sign.
You don't need to call sed to get the base name of files
while read -r file
do
filename="${file##*/}"
filename=$(printf "%q" $filename)
echo mencoder "${OPT1}" "${file}" -o "${OUTS}${file}" "${OPT2}"
done < <(find . -name "*.*" -type f)
lastly,
IFS=OLD_IFS should be IFS=$OLD_IFS
You have this statement before your for loop:
IFS=$'\n'
This sets your internal field separator to newlines, instead of the default of matching any whitespace. That changes how substituted parameters are parsed. In the line constructing the command:
${TOOL} ${OPT1} ${file} -o ${OUTS}${file} ${OPT2}
what will happen is that each of those ${variable} expressions will be expanded, and then the shell will try splitting them on \n, not on whitespace as you would normally expect. It will give you a similar result as the following would (unless one of these variables contained a newline):
"${TOOL}" "${OPT1}" "${file}" -o "${OUTS}${file}" "${OPT2}"
Here you can see that you're passing your entire ${OPT2} string in as a single parameter, rather than allowing Bash to split it on spaces and pass each flag in individually. mencoder then gets confused by this one huge parameter that it doesn't know how to deal with. Of course, since the spaces are all still there, they will be printed out by the echo command, and will work fine in a shell in which $IFS has not been reset.
You can demonstrate this effect pretty easily, by defining a simple function that will print each of its arguments on a separate line:
$ print_args() { for arg; do echo $arg; done }
$ foo="1 2"
$ print_args ${foo} ${foo}
1
2
1
2
$ IFS=$'\n'
$ print_args ${foo} ${foo}
1 2
1 2
I would recommend not using the $IFS trick for your for loop. Instead, you can use while read file to iterate over each line in the input. I'd also recommend not using printf "%q" for escaping spaces, but instead just quote the argument to mencoder, which will pass the whole thing in as a single argument. Note that I'm quoting ${file} and ${OUTS}${file} to make sure that they are passed in each as a single argument, but not quoting ${OPT1} and ${OPT2} in order to allow them to be parsed as separate arguments by the shell.
find . -name "*.*" -type f | sed 's!.*/!!' | while read -r file
do
"${TOOL}" ${OPT1} "${file}" -o "${OUTS}${file}" ${OPT2}
done
By the way, I'd recommend that you use $() for command substitution rather than ``; there are many reasons why it's preferable, such as readability, more sane quoting and escaping rules within it, and the ability to nest multiple levels of command substitution. And the problems that Jonathan and Wes point out are good to note, though they aren't what are causing your immediate problem.
Using ... | xargs bash -c '...' you just need to escape embedded single quotes in the file name.
TOOL='mencoder'
OUTS='./out/'
OPT1='-noodml'
OPT2='-of avi -ofps 22 -vf-add scale=320:-2,expand=320:240 -srate 44100 -ovc xvid -xvidencopts bitrate=400:max_bframes=0:quant_type=s16le -oac lavc -lavcopts acodec=mp2:abitrate=128'
# test cases
file='12 I Love You 1 I Love You 2 I Love You 3.avi'
file='12 I Lo"ve You 1 I Love You 2 I Love You 3.avi' # one embedded double quote
file='12 I Lo"ve You 1 I Lo"ve You 2 I Love You 3.avi' # two embedded double quotes
file="12 I Lo've You 1 I Love You 2 I Love You 3.avi" # one embedded single quote
file="12 I Lo've You 1 I Lo've You 2 I Love You 3.avi" # two embedded single quotes
# escape embedded single quotes in $file
escsquote="'\''"
file="${file//\'/${escsquote}}"
# we're passing ${TOOL} as arg0 to bash -c (which then is available as "$0")
# put the variables in the printf line into single quotes to preserve spaces
# remove no-op ':' to execute the command
printf '%s\n' "${file}"
printf '%s' "${OPT1} '${file}' -o '${OUTS}${file}' ${OPT2}" |
xargs bash -c 'set -xv; printf "%s\n" "$0" "$#" | nl' "${TOOL}"
printf '%s' "${OPT1} '${file}' -o '${OUTS}${file}' ${OPT2}" |
xargs bash -c 'set -xv; : "$0" "$#"' "${TOOL}"
Related
I have a lot of videos cut into three parts, and I want to concat them.
I now have the following bash to output videos in the folder to a file named mergedVideo.mp4.
for f in *.mp4 ; do echo file \'$f\' >> fileList.txt;done
ffmpeg -f concat -safe 0 -i fileList.txt -c copy mergedVideo.mp4
All my input files are like Something part1.mp4, Something part2.mp4, Something part3.mp4
What I want is to output a file named Something.mp4
It is possible?
How can I modify my bash to achieve that~?
Would you please try the following:
#!/bin/bash
pat="^([^[:space:]]+)[[:space:]]*part[[:digit:]]+\.mp4$"
for f in *.mp4; do
echo file \'$f\' >> fileList.txt
[[ $f =~ $pat ]] && mergedvideo="${BASH_REMATCH[1]}.mp4"
done
ffmpeg -f concat -safe 0 -i fileList.txt -c copy "$mergedvideo"
Explanation of the regex $pat:
^([^[:space:]]+) matches a non-space substring from the beginning of the
string $f and is assigned to the shell variable BASH_REMATCH[1].
As for the provided example filenames, BASH_REMATCH[1] will be assigned to Something.
[[:space:]]* matches a zero or more whitespace(s).
part[[:digit:]]+ matches a string "part" followed by digit(s).
\.mp4 matches the suffix.
[Alternative]
If you are familiar with sed as well, the following may be more readable and maintainable:
mergedvideo=$(sed -E 's/[[:space:]]*part[[:digit:]]+//' <<< "$f")
I extend tshiono's answer to be able to manipulate all the files in the same folders (just like Marinos An mentioned, the idea is to merge files with the same prefix)
Cases like a folder with the following files:
Monday Blue part1.mp4
Monday Blue part2.mp4
Tuesday.mp4
Wednesday part1.mp4
Wednesday part2.mp4
Wednesday part3.mp4
Expected Output:
Monday Blue.mp4
Tuesday.mp4
Wednesday.mp4
#!/bin/bash
pat="^(.+)[[:space:]]+part[[:digit:]]+\.mp4$"
for f in *.mp4; do
if [[ $f =~ $pat ]]; then
echo file \'$f\' >> "${BASH_REMATCH[1]}.txt"
fi
done
for f in *.txt; do
ffmpeg -f concat -safe 0 -i "${f%%.*}.txt" -c copy "${f%%.*}.mp4"
done
At fisrt,I have write the code,and it run well.
# version1
all_num=10
thread_num=5
a=$(date +%H%M%S)
seq 1 ${all_num} | xargs -n 1 -I {} -P ${thread_num} sh -c 'echo abc{}'
b=$(date +%H%M%S)
echo -e "startTime:\t$a"
echo -e "endTime:\t$b"
Now I want to extract code into a funciton,but it was wrong,how to fix it?
get_file(i){
echo "abc"+i
}
all_num=10
thread_num=5
a=$(date +%H%M%S)
seq 1 ${all_num} | xargs -n 1 -I {} -P ${thread_num} sh -c "$(get_file {})"
b=$(date +%H%M%S)
echo -e "startTime:\t$a"
echo -e "endTime:\t$b"
Because /bin/sh isn't guaranteed to have support for either printing text that when evaluates defines your function, or exporting functions through the environment, we need to do this the hard way, just duplicating the text of the function inside the copy of sh started by xargs.
Other questions already exist in this site describing how to accomplish this with bash, which is quite considerably easier. See f/e How can I use xargs to run a function in a command substitution for each match?
#!/bin/sh
all_num=10
thread_num=5
batch_size=1 # but with a larger all_num, turn this up to start fewer copies of sh
a=$(date +%H%M%S) # warning: this is really inefficient
seq 1 ${all_num} | xargs -n "${batch_size}" -P "${thread_num}" sh -c '
get_file() { i=$1; echo "abc ${i}"; }
for arg do
get_file "$arg"
done
' _
b=$(date +%H%M%S)
printf 'startTime:\t%s\n' "$a"
printf 'endTime:\t%s\n' "$b"
Note:
echo -e is not guaranteed to work with /bin/sh. Moreover, for a shell to be truly compliant, echo -e is required to write -e to its output. See Why is printf better than echo? on UNIX & Linux Stack Exchange, and the APPLICATION USAGE section of the POSIX echo specification.
Putting {} in a sh -c '...{}...' position is a Really Bad Idea. Consider the case where you're passed in a filename that contains $(rm -rf ~)'$(rm -rf ~)' -- it can't be safely inserted in an unquoted context, or a double-quoted context, or a single-quoted context, or a heredoc.
Note that seq is also nonstandard and not guaranteed to be present on all POSIX-compliant systems. i=0; while [ "$i" -lt "$all_num" ]; do echo "$i"; i=$((i + 1)); done is an alternative that will work on all POSIX systems.
hopefully a simple question and the last piece in my puzzle... :-) I have a shell script running in terminal under os x. It contains among other things:
name=$(basename "$file")
printf "%s" "\"$name\";"
... which is fine ... but lets say that the file name contains a double quote - IMA"G09%'27.jpg - then the output would be:
"IMA"G09%'27.jpg;"
... and that would "break" my line intended for putting into a db later (the double quote). So I need to escape it so I get the output:
"IMA\"G09%'27.jpg;"
... but I couldn't figure out how ... anyone? :-)
EDIT - RESULT: With the help of anubhava this is what I use (to get file info incl. type/creator):
#!/bin/bash
find . -type f -name '*' -print0 | while IFS= read -r -d '' file
do
name=$(basename "$file")
path=$(dirname "$file")
# full_path=$(readlink -f "$file") # This only works on Linux
full_path=$(echo "$PWD/${file#./}")
extension=${name##*.}
size_human_readable=$(ls -lh "$file" | awk -F' ' '{print $5}')
size_in_bytes=$(stat -f "%z" "$file")
creation_date=$(stat -f "%SB" "$file")
last_access=$(stat -f "%Sa" "$file")
last_modification=$(stat -f "%Sm" "$file")
last_change=$(stat -f "%Sc" "$file")
creator=$(mdls -name kMDItemFSCreatorCode "$file")
printf "\"%q\";" "$name"
printf "%s" "\"$full_path\";"
printf "%s" "\"$extension\";"
printf "\"$size_human_readable\";"
printf "\"$size_in_bytes\";"
printf "\"$last_modification\";"
printf "%s" "\"$creator\""
printf "\n"
done
Using printf with %q:
name='file"naeme.txt'
printf "\"%q;\"" "$name"
"file\"naeme.txt;"
Here's another approach, using sed to control the escaping:
printquoted() {
printf '"%s";' "$(LC_ALL=C sed 's/["]/\\&/g' <<<"$1")"
}
printquoted "$name"
printquoted "$full_path"
printquoted "$extension"
...etc
If it turns out there are other things you need to escape besides double-quotes (like, say, backslashes themselves), you can add them to the sed [] expression (e.g. ["\] would escape double-quotes and backslashes).
Note that this approach will fail horribly if the string contains any newlines (which is legal in filenames).
I have a bash script that works except it's case sensitive in relation to the extension. So if the extension is MOV or MoV the file gets skipped how can I make the script case insensitive
#!/bin/bash
#/bin/sh
###############################################################################
#execute using bash mkvconv.sh
# Script to recursively search a directory and batch convert all files of a given
# file type into another file type via HandBrake conversion.
#
# To run in your environment set the variables:
# hbcli - Path to your HandBrakeCLI
#
# source_dir - Starting directory for recursive search
#
# input_file_types - Input file types to search for
#
# output_file_type - Output file type to convert into
#
#
# Change log:
# 2014-06-27: Initial release. Tested on ubuntu 13.10. and 14.04
#http://stackoverflow.com/questions/21404059/bash-script-to-select-multiple-file-formats-at-once-for-encode-process/21404530#21404530
###############################################################################
hbcli=HandBrakeCLI
source_dir="/tmp/vidtest"
#source_dir="/tmp/vidtest"
#source_dir="/media/rt/1tera_ext/1_Video_Stuff/1 Nova and bbc/Carbon diamonds"
input_file_types=(avi wmv flv mp4 webm mov mpg rm dv)
output_file_type="mkv"
echo "# Using HandBrakeCLI at "$hbcli
echo "# Using source directory " "$source_dir"
echo "# Converting "$input_file_types" to "$output_file_type
# Convert from one file to another
convert() {
# The beginning part, echo "" | , is really important. Without that, HandBrake exits the while loop.
#echo "" | $hbcli -i "$1" -o "$2" --preset="Universal"; # dont use with preses things are left out
echo "" | $hbcli -i "$1" -t 1 --angle 1 -c 1 -o "$2" -f mkv --decomb --loose-anamorphic --modulus 2 -e x264 -q 20 --cfr -a 1,1 -E faac,copy:ac3 -6 dpl2,auto -R Auto,Auto -B 160,0 -D 0,0 --gain 0,0 --audio-fallback ffac3 --x264-profile=high --h264-level="4.1" --verbose=1
}
# loop over the types and convert
for input_file_types in "${input_file_types[#]}"
do
# Find the files and pipe the results into the read command. The read command properly handles spaces in directories and files names.
#find "$source_dir" -name *.$input_file_type | while read in_file
find "$source_dir" -name "*.$input_file_types" -print0 | while IFS= read -r -d $'\0' in_file
#In order to correctly handle filenames containing whitespace and newline characters, you should use null delimited output. That's what the -print0 and read -d $'\0' is for.
do
echo "Processing…"
echo ">Input "$in_file
# Replace the file type
out_file=$(echo $in_file|sed "s/\(.*\.\)$input_file_types/\1$output_file_type/g")
echo ">Output "$out_file
# Convert the file
convert "$in_file" "$out_file"
if [ $? != 0 ]
then
echo "$in_file had problems" >> handbrake-errors.log
fi
echo ">Finished "$out_file "\n\n"
done
done
echo "DONE CONVERTING FILES"
find: use -iname instead of -name
sed: use that "s/\(.*\.\)$input_file_types/\1$output_file_type/gI") (I at the end of the string)
You need to set nocaseglob for ignore case glob matching in shell:
shopt -s nocaseglob
I want to replace a' into à, e' into è, etc. into a file with a script like this:
#!/bin/sh
if [ -e "$1" ]
then
sed 's/a'/\à/g' -i "$1";
sed 's/e'/\è/g' -i "$1";
sed 's/i'/\ì/g' -i "$1";
sed 's/o'/\ò/g' -i "$1";
sed 's/u'/\ù/g' -i "$1";
else
echo "File not found!"
fi
But I get this error:
Syntax error: Unterminated quoted string
I don't know how to wrote '
Yes there is syntax problem, try your sed command as this:
sed -i "s/a'/à/g" "$1"
sed -i "s/e'/è/g" "$1"
sed -i "s/i'/ì/g" "$1"
sed -i "s/o'/ò/g" "$1"
sed -i "s/u'/ù/g" "$1"
Problem was that you were using nested single quote (quote inside code).
Exit the single quotes, then put an escaped quote, then start another single quoted string:
sed 's/a'\''/\à/g' -i "$1"
awk '{ gsub(/e'\''/,"è"); print}' "$1"
Yet another solution available in bash is $'...' quoting, which does allow escaped single quotes:
sed $'s/a\'/\à/g' -i "$1";