find piped to xargs with complex command - bash

I am trying to process DVD files that are in many different locations on a disk. The thing they have in common is that they (each set of input files) are in a directory named VIDEO_TS. The output in each case will be a single file named for the parent of this directory.
I know I can get a fully qualified path to each directory with:
find /Volumes/VolumeName -type d -name "VIDEO_TS" -print0
and I can get the parent directory by piping to xargs:
find /Volumes/VolumeName -type d -name "VIDEO_TS" -print0 | xargs -0 -I{} dirname {}
and I also know that I can get the parent directory name on its own by appending:
| xargs -o I{} basename {}
What I can't figure out is how do I then pass these parameters to, e.g. HandBrakeCLI:
./HandBrakeCLI -i /path/to/filename/VIDEO_TS -o /path/to/convertedfiles/filename.m4v
I have read here about expansion capability of the shell and suspect that's going to help here (not using dirname or basename for a start), but the more I read the more confused I am getting!

You don't actually need xargs for this at all: You can read a NUL-delimited stream into a shell loop, and run the commands you want directly from there.
#!/bin/bash
source_dir=/Volumes/VolumeName
dest_dir=/Volumes/OtherName
while IFS= read -r -d '' dir; do
name=${dir%/VIDEO_TS} # trim /VIDEO_TS off the end of dir, assign to name
name=${name##*/} # remove everything before last remaining / from name
./HandBrakeCLI -i "$dir" -o "$dest_dir/$name.m4v"
done < <(find "$source_dir" -type d -name "VIDEO_TS" -print0)
See the article Using Find on Greg's wiki, or BashFAQ #001 for general information on processing input streams in bash, or BashFAQ #24 to understand the value of using process substitution (the <(...) construct here) rather than piping from find into the loop.
Also, find contains an -exec action which can be used as follows:
source_dir=/Volumes/VolumeName
dest_dir=/Volumes/OtherName
export dest_dir # export allows use by subprocesses!
find "$source_dir" -type d -name "VIDEO_TS" -exec bash -c '
for dir; do
name=${dir%/VIDEO_TS}
name=${name##*/}
./HandBrakeCLI -i "$dir" -o "$dest_dir/$name.m4v"
done
' _ {} +
This passes the found directory names directly on the argument list to the shell invoked with bash -c. Since the default object for for loop to iterate over is "$#", the argument list, this implicitly iterates over directories found by find.

If I understand what you are trying to do, the simplest solution would be to create a little wrapper which takes a path and invokes your CLI:
File: CLIWrapper
#!/bin/bash
for dir in "$#"; do
./HandBrakeCLI -i "${dir%/*}" -o "/path/to/convertedfiles/${dir##*/}.m4v"
done
Edit: I think I misunderstood the question. It's possible that the above script should read:
./HandBrakeCLI -i "$dir" -o "/path/to/convertedfiles/${dir##*/}.m4v"
or perhaps something slightly different. But the theory is valid. :)
Then you can invoke that script using the -exec option to find. The script loops over its arguments, making it possible for find to send multiple arguments to a single invocation using the + terminator:
find /Volumes/VolumeName -type d -name "VIDEO_TS" -exec ./CLIWrapper {} +

Related

Wildcard within if conditional of Bash fails to execute. Works when literal filename provided

I am trying to execute a command depending on the file type within directory. But am unable to check the content within directory using wildcard. When provided a literal filename I am able to execute.
find ./* -type d -execdir bash -c 'DIR=$(basename {}); if [[ -e {}/*.png ]]; then echo "img2pdf {}/*.png -o $DIR.pdf"; fi ' \;
Instead of going over directories, and then looking for png-s inside, find can find png-s straight away:
find . -name '*.png'
Then you can process it as you do, or using xargs:
find . -name '*.png' | xargs -I '{}' img2pdf '{}' -o '{}.pdf'
The command above will process convert each png to a separate pdf.
If you want to pass all png-s at once, and call img2pdf once:
find . -name '*.png' | xargs img2pdf -o out.pdf

How do I rename files found with the find command

I have a series of music folders. Some of the file names contain an underscore which I would like to get rid of.
With
find /Users/Chris/CDs -type f -name "*_*"
I find all of the files with underscores.
it appears that I can add -execdir mv {} to the command but do not know what to add from there.
I think {} provides the full path and file name as a string of the file with underscores but I do not know how to use something like sed 's/_//g' to remove the _ on the new file name.
Any help would be greatly appreciated.
Try:
find /Users/Chris/CDs -type f -name "*_*" -execdir bash -c 'mv -i -- "$1" "${1//_/}"' Mover {} \;
How it works:
-execdir bash -c '...' Mover {} \;
This starts up bash and tells it to run the command in the single quotes with Mover assigned to $0 and the file name assigned to $1.
mv -i -- "$1" "${1//_/}"
This renames file $1. This uses bash's parameter expansion feature, ${1//_/}, to create the target name from $1 by removing all underlines.
The option -i tells mv to ask interactively before overwriting a file.
The option -- tells mv that there are no more options. This is needed so that files whose names begin with - will be processed correctly.
Example
Let's start with a directory with these files:
$ ls
1_2_3_4 a_b c_d
Next we run our command:
$ find . -type f -name "*_*" -execdir bash -c 'mv -i -- "$1" "${1//_}"' Mover {} \;
After the command completes, the files are:
$ ls
1234 ab cd
The purpose of $0
Observe this command where we have added an error:
$ find . -type f -name "*_*" -execdir bash -c 'foobar -i -- "$1" "${1//_}"' Mover {} \;
Mover: foobar: command not found
Note that Mover appears at the beginning of the error message. This signals that the error comes from within the bash -c command.
If we replace Mover with -, we would see:
$ find . -type f -name "*_*" -execdir bash -c 'foobar -i -- "$1" "${1//_}"' - {} \;
-: foobar: command not found
When running a single command in a terminal, the source of the error may still be obvious anyway. If this find command were buried inside a long script, however, the use of a more descriptive $0, like Mover or whatever, could be a big help.

Rename all files in directory and (deeply nested) sub-directories

What is the shell command for renaming all files in a directory and sub-directory (recursively)?
I would like to add an underscore to all the files ending with *scss from filename.scss to _filename.scss in all the directories and sub-directories.
I have found answers relating to this but most if not all require you to know the filename itself, and I do not want this because the filenames differ and are a lot to know by heart or even type them manually and some of them are deeply nested in directories.
Edit: I was under the impression that the bash -c bit was somehow necessary for multiple expansion of the found element; anubhava's answer proved me wrong. I am leaving that bit in the answer for now as it worked for the OP.
find . -type f -name *scss -exec bash -c 'mv $1 _$1' -- {} \;
find . -- find in current directory (recursively)
-type f -- files
-name *scss -- matching the pattern *scss
-exec -- execute for each element found
bash -c '...' -- execute command in a subshell
-- -- end option parsing
{} -- expands to the name of the element found (which becomes the positional parameter for the bash -c command)
\; -- end the -exec command
You can use -execdir option here:
find ./src/components -iname "*.scss" -execdir mv {} _{} \;
You are close to a solution:
find ./src/components -iname "*.scss" -print0 | xargs -0 -n 1 -I{} mv {} _{}
In this approach, the "loop" is executed by xargs. I prefer this solution overt the usage of the -exec in find. The syntax is clear to me.
Also, if you want to repeat the command and avoid double-adding the underscore to the already processed files, use a regexp to get only the files not yet processed:
find ./src/components -iregex ".*/[^_][^/]*\.scss" -print0 | xargs -0 -n 1 -I{} mv {} _{}
By adding the -print0/-0 options, you also avoid problems with whitespaces.
#!/bin/sh
EXTENSION='.scss'
cd YOURDIR
find . -type f | while read -r LINE; do
FILE="$( basename "$LINE" )"
case "$LINE" in
*"$EXTENSION")
DIRNAME="$( dirname "$LINE" )"
mv -v "$DIRNAME/$FILE" "$DIRNAME/_$FILE"
;;
esac
done

find -exec when used with sed for file rename not working

I've been trying:
find dev-other -name '*.flac' -type f -exec echo $(echo {} | sed 's,^[^/]*/,,') \;
I expect to see a list of paths to .flac files within dev-other, but without a prepended dev-other/, e.g.:
4515/11057/4515-11057-0095.flac
4515/11057/4515-11057-0083.flac
4515/11057/4515-11057-0040.flac
4515/11057/4515-11057-0105.flac
4515/11057/4515-11057-0017.flac
4515/11057/4515-11057-0001.flac
Instead I see
dev-other/4515/11057/4515-11057-0095.flac
dev-other/4515/11057/4515-11057-0083.flac
dev-other/4515/11057/4515-11057-0040.flac
dev-other/4515/11057/4515-11057-0105.flac
dev-other/4515/11057/4515-11057-0017.flac
Why isn't the sed replace working here even though it works on its own
$ echo $(echo dev-other/4515/11057/4515-11057-0047.flac | sed 's,^[^/]*/,,')
4515/11057/4515-11057-0047.flac
I first tried with expansions:
find dev-other -name '*.flac' -type f -exec a={} echo ${a#*/} \;
But got the errors:
find: a=dev-other/700/122866/700-122866-0001.flac: No such file or directory
find: a=dev-other/700/122866/700-122866-0030.flac: No such file or directory
find: a=dev-other/700/122866/700-122866-0026.flac: No such file or directory
find: a=dev-other/700/122866/700-122866-0006.flac: No such file or directory
find: a=dev-other/700/122866/700-122866-0010.flac: No such file or directory
You can just use parameter expansion for your use-case when using find with the -exec option,
find dev-other -name '*.flac' -type f -exec bash -c 'x=$1; y="${x#*/}"; echo "$y"' bash {} \;
I used a separate shell (use bash or sh) using bash -c because to involve separate string operations involving parameter expansion. Think of each output of find result to be passed as argument to this sub-shell where this manipulation takes place.
When bash -c executes a command, the next argument after the command is used as $0 (the script's "name" in the process listing), and subsequent arguments become the positional parameters ($1, $2, etc.). This means that the filename passed by find (in place of the {}) becomes the first parameter of the script -- and is referenced by $1 inside the mini-script
If you don't want to use an extra bash, use _ in-place
find dev-other -name '*.flac' -type f -exec bash -c 'x=$1; y="${x#*/}"; echo "$y"' _ {} \;
where _ i is a bash predefined variable (not defined in dash for instance): "At shell startup, set to the absolute path-name used to invoke the shell or shell script being executed as passed in the environment or argument list" ( See man bash - Special Parameters section)
Worth looking at Using Find - Complex Actions

More elegant use of find for passing files grouped by directory?

This script has taken me too long (!!) to compile, but I finally have a reasonably nice script which does what I want:
find "$#" -type d -print0 | while IFS= read -r -d $'\0' dir; do
find "$dir" -iname '*.flac' -maxdepth 1 ! -exec bash -c '
metaflac --list --block-type=VORBIS_COMMENT "$0" 2>/dev/null | grep -i "REPLAYGAIN_ALBUM_PEAK" &>/dev/null
exit $?
' {} ';' -exec bash -c '
echo Adding ReplayGain tags to "$0"/\*.flac...
metaflac --add-replay-gain "${#:1}"
' "$dir" {} '+'
done
The purpose is to search the file tree for directories containing FLAC files, test whether any are missing the REPLAYGAIN_ALBUM_PEAK tag, and scan all the files in that directory for ReplayGain if they are missing.
The big stumbling block is that all the FLAC files for a given album must be passed to metaflac as one command, otherwise metaflac doesn't know they're all one album. As you can see, I've achieved this using find ... -exec ... +.
What I'm wondering is if there's a more elegant way to do this. In particular, how can I skip the while loop? Surely this should be unnecessary, because find is already iterating over the directories?
You can probably use xargs to achieve it.
For example, if you are looking for text foo in all your files you'll have something like
find . type f | xargs grep foo
xargs passes each result from left-end expression (find) to the right-end invokated command.
Then, if no command exists to achieve what you want to do, you can always create a function, and pass if to xargs
I can't comment on the flac commands themselves, but as for the rest:
find . -name '*.flac' \
! -exec bash -c 'metaflac --list --block-type=VORBIS_COMMENT "$1" | grep -qi "REPLAYGAIN_ALBUM_PEAK"' -- {} \; \
-execdir bash -c 'metaflac --add-replay-gain *.flac' \;
You just find the relevant files, and then treat the directory it's in.

Resources