I often use find to run the same script on a bunch of files. So for example if I want to run process.py on all the .png files in dir, I would do:
find dir -name '*.png' -execdir process.py \{\} \;
The picket fence thing in the end is annoying, any way around it?
Depending on your keyboard layout, " might be more convenient to use than ' or \. The {} does not need escaping, as far as the Unix & Linux Stack Exchange site knows. Nobody could identify a shell that would need {} escaped, and the examples in the man page do not escape the braces.
find dir -name "*.png" -execdir process.py {} ";"
Jonathan Leffler has a solution with + in the end, which is not identical in semantics, but often usable.
Use:
find dir -name '*.png' -execdir process.py {} +
The {} don't need escaping; they only have special meaning to the shell in rather limited circumstances. (In particular, echo {} echoes the braces, whereas echo {a,b,c} echoes a b c.) The + does not need escaping either. It tells find to 'play at being xargs'. That is, it will run the command with as many file names as it reasonably can for each execution.
Note that using -exec or -execdir automatically and comprehensively deals with the problem of spaces (and other awkward characters — newlines, backspaces, form feeds, anyone?) in file names. Piping names with -print into xargs runs foul of problems here. GNU find plus GNU xargs provides the -print0 option to find and the -0 option to xargs to get around issues with odd characters in file names.
If you must execute the script once per file, then you need an escaped semi-colon at the end; there is no easy way around that (unless you count: SC=";"; find ... {} $SC, which I don't).
The only issue I see is the -execdir which runs the script in the sub-directory. You'll have to check that it behaves sanely when there are different files in different directories, and you'll need to be sure that {} translates to 'the file name relative to the directory it is found in' when used with -execdir (as otherwise, the file won't be locatable via the name that is given to the script, in general). All of this should 'just work' as the options wouldn't be meaningfully usable if they didn't.
Personally, I'd rather use just plain -exec, but there's probably a good reason why you chose -execdir.
you can single quote '{}' ';' instead, but you would have to somehow prevent shell itself from interpreting these characters. xargs will work as well if the number of found is small.
Related
I have a ton of screenshots on my desktop (it's the default location where they're saved) with titles of the form "Screen Shot 2020-10-11 at 7.08.12 PM.png" and I'd like to use a bash script with regex on the "Screen Shot" bit to move any such file from my desktop into a folder I created called "screenshots".
Right now, I'm playing around with find . -regex Screen*.*?.png but it's not quite working (it gives find: Screen Shot 2020-10-11 at 7.11.09 PM.png: unknown primary or operator).
I'm also not sure how I'd even use the output once it does find all the correct files to move them to a folder. Could you somehow iterate over all files in a folder using for i in seq 1 100 or something of the like?
You don't actually even need -regex here:
find . -type f -name 'Screen Shot*png' -maxdepth 1 -exec echo mv "{}" screenshots \;
You can run this command safely as it will not do anything but print
what it would do. Remove echo to actually run mv.
All options used are documented in man find but in short:
-type f will make find look only for files, not directories. This
is useful in case you have a directory that matches -name - we don't
want to touch it.
-maxdepth 1 will only look fire files in the same directory level -
it's very useful here because you might already have some files that
match the -name present in screenshots directory - we want to leave
them alone.
-name accepts shell pattern, not regex. We could of course use -regex here but I prefer -name because shell patterns are shorter and easier to use here.
{} is a placeholder that will be replaced will the name of found
file.
\; is a literal semicolon, escaped to prevent it from being
interpreted by shell that ends command specified with -exec.
Taking the regex at face value (probably a mistake), you should use single quotes around the regex:
find . -regex 'Screen*.*?.png'
This prevents the shell from expanding it, leaving that to find. Then to move the files to the ~/screenshots directory (change the name to match the directory you want to use), if you have GNU mv, you can use:
find . -regex 'Screen*.*?.png' -exec mv -t ~/screenshots {} +
This executes a single mv command to move many files to the target directory, reducing the number of times the mv is executed. It might still be executed multiple times, but it will be many fewer times than the alternative.
If you don't have GNU mv with the (very useful) -t option, then you should use:
find . -regex 'Screen*.*?.png' -exec mv {} ~/screenshots ';'
This executes one mv command for each file found, but is more portable.
The primary problem you ran into was that the shell was expanding what you wrote into a list of file names, and then find didn't understand what you meant. Using the quotes prevents the shell from expanding the 'regex'. You can add an echo to the other commands before the mv to see what would be executed.
However, I'm not sure whether you know what your regex matches. It isn't clear that the regex given is a valid regex for find — though it mostly works as a PCRE (Perl-compatible) regular expression. By default, GNU find uses GNU Emacs regular expressions, but you can control the dialect of regular expression it uses. The options available include Emacs, POSIX Awk, POSIX Basic Regular Expressions (BRE), POSIX egrep, and POSIX Extended Regular Expressions (ERE). It doesn't include PCRE. What you supply is more like a shell glob, and the -name operator handles globbing names.
It's quite probable that you should be using the -name operator, using a command along the lines of:
find . -name 'Screen Shot *.png' -exec mv -t ~/screenshots {} +
This code searches and recursively copies the files after the above date.
#!/bin/bash
directory=~/somefolder
DAYSAGO=8
for ((a=0; a <= DAYSAGO ; a++))
do
find $directory -mtime $a -type f | while read file;
do
cp "$file" -t ~/The\ other\ folder/
done
done
Try the following:
#!/usr/bin/env bash
directory=~/'somefolder'
DAYSAGO=8
find "$directory" -mtime -$(( DAYSAGO + 1 )) -type f -exec cp -t ~/'The other folder'/ {} +
Using - to prefix the -mtime argument applies less-than logic to the argument value. All find tests that take numeric arguments support this logic (and its counterpart, +, for more-than logic). Tip of the hat to miracle 173.
Since the desired logic is <= $DAYSAGO, 1 is added using an arithmetic expansion ($(( ... ))), to achieve the desired logic (needless to say, $DAYSAGO could be redefined with less-than logic in mind, to 9, so as to make the arithmetic expansion unnecessary).
Using -exec with the + terminator invokes the specified command with (typically) all matching filenames at once, which is much more efficient than piping to a shell loop.
{} is the placeholder for the list of matching filenames, and note that with + it must be the last argument before the + terminator (by contrast, with the invoke-once-for-each-matching-file terminator \;, the {} can be placed anywhere).
Note that the command above therefore only works with cp implementations that support the -t option, which allows placing the target directory first, notably, GNU cp (BSD/OSX cp and the POSIX specification, by contrast, do NOT support -t).
Also note the changes in quoting:
directory=~/'somefolder': single-quoting literal somefolder - while not strictly necessary in this particular case - ensures that the enclosed name works even if it contains embedded spaces or other shell metacharacters.
Note, however, that the ~/ part must remain unquoted for the ~ to expand to the current user's home dir.
"$directory": double-quoting the variable reference ensures that its value is not interpreted further by the shell, making it safe to use paths with embedded whitespace and other shell metacharacters.
~/'The other folder'/ provides a more legible alternative to ~/The\ other\ folder/ (and is also easier to type), demonstrating the same mix of unquoted and quoted parts as above.
You don't need the while loop at all. Using it as you are exposes you to problems with some corner cases like filenames containing newlines and other whitespace. Just use the -exec primary.
find "$directory" -mtime "$a" -type f -exec cp {} -t ~/The\ other\ folder/ \;
UPDATE: use mklement0's answer, though; it's more efficient.
I have searched looking for the right solution. I have found some close examples.Bash script to replace spaces in file names
But what I'm looking for is how to replace multiple .dots in current DIRECTORY/SUBDIRECTORY names, then replace multiple .dots in FILENAMES excluding the *.extention "recursively".
This example is close but not right:
find . -maxdepth 2 -name "*.*" -execdir rename 's/./-/g' "{}" \;
another example but not right either:
for f in *; do mv "$f" "${f//./-}"; done
So
dir.dir.dir/dir.dir.dir/file.file.file.ext
Would become
dir-dir-dir/dir-dir-dir/file-file-file.ext
You can assign to a variable and pipe like this:
x="dir.dir.dir/dir.dir.dir/file.file.file.ext"
echo "$(echo "Result: ${x%.*}" | sed 's/\./_/g').${x##*.}"
Result: dir_dir_dir/dir_dir_dir/file_file_file.ext
You have to escape . in regular expressions (such as the ones used for rename, because by default it has the special meaning of "any single character". So the replacement statement at least should be s/\./-/g.
You should not quote {} in find commands.
You will need two find commands, since you want to replace all dots in directory names, but keep the last dot in filenames.
You are searching for filenames which contain spaces (* *). Is that intentional?
I'm trying to make a little script to convert some files from a music library.
But If I do something like :
#!/usr/bin/zsh
for x in $(find -name "*.m4a");
do
echo $x;
done
When interpreting in a folder containing :
foo\ bar.m4a
it will return :
foo
bar.m4a
How could I prevent the for loop from interpreting space characters as separators?
I could replace $(find -name "*.m4a") with $(find -name "*.m4a" | sed "s/ /_/g") and then using sed the other way inside the loop, but what if file names/paths already contain underscores (Or other characters I may use instead of underscore)?
Any idea?
You can prevent word splitting of the command substitution by double-quoting it:
#!/usr/bin/zsh
for x in "$(find -name "*.m4a")"
do
echo "$x"
done
Notice that the double quotes inside the command substitution don't conflict with the double quotes outside of it (I'd actually never noticed this before now). You could just as easily use single quotes if you find it more readable, as in "$(find -name '*.m4a')". I usually use single quotes with find primaries anyway.
Quoting inside the loop body is important for the same reason. It will ensure that the value of x is passed as a single argument.
But this is definitely a hybrid, Frankensteinian solution. You'd be better off with either globbing or using find as follows:
find . -name '*.mp3' -exec echo {} \;
but this form is limiting. You can add additional -exec primaries, which will be executed like shell commands that are separated by &&, but you can't pipe from one -exec to another and you can't interact with the shell (e.g. by assigning or expanding parameters).
Use zsh's globbing facilities here instead of find.
for x in **/*.m4a; do
echo "$x"
done
Quoting $x in the body of the loop is optional under the default settings of zsh, but it's not a bad idea to do so anyway.
I found out.
As suggested here : https://askubuntu.com/questions/344407/how-to-read-complete-line-in-for-loop-with-spaces
I may set the IFS (Internal Field Separator) to '\n'.
So this works :
#!/usr/bin/zsh
IFS=$'\n'
for x in $(find -name "*.m4a");
do
echo $x;
done
I hope this could help someone else!
The following command works as expected interactively, in a terminal.
$ find . -name '*.foo' -o -name '*.bar'
./a.foo
./b.bar
$
However, if I do this, I get no results!
$ ftypes="-name '*.foo' -o -name '*.bar'"
$ echo $ftypes
-name '*.foo' -o -name '*.bar'
$ find . $ftypes
$
My understanding was/is that $ftypes would get expanded by bash before find got a chance to run. In which case, the ftypes approach should also have worked.
What is going on here?
Many thanks in advance.
PS: I have a need to dynamically build a list of file types (the ftypes variable above) to be given to find later in a script.
Both answers so far have recommended using eval, but that has a well-deserved reputation for causing bugs. Here's an example of the sort of bizarre behavior you can get with this:
$ touch a.foo b.bar "'wibble.foo'"
$ ftypes="-name '*.foo' -o -name '*.bar'"
$ eval find . $ftypes
./b.bar
Why didn't it find the file ./a.foo? It's because of exactly how that eval command got parsed. bash's parsing goes something like this (with some irrelevant steps left out):
bash looks for quotes first (none found -- yet).
bash substitutes variables (but doesn't go back and look for quotes in the substituted values -- this is what lead to the problem in the first place).
bash does wildcard matching (in this case it looks for files matching '*.foo' and '*.bar' -- note that it hasn't parsed the quotes, so it just treats them as part of the filename to match -- and finds 'wibble.foo' and substitutes it for '*.foo'). After this the command is roughly eval find . -name "'wibble.foo'" -o "'*.bar'". BTW, if it had found multiple matches things would've gotten even sillier by the end.
bash sees that the command on the line is eval, and runs the whole parsing process over on the rest of the line.
bash does quote matching again, this time finding two single-quoted strings (so it'll skip most parsing on those parts of the command).
bash looks for variables to substitute and wildcards to matching, etc, but there aren't any in the unquoted sections of the command.
Finally, bash runs find, passing it the arguments ".", "-name", "wibble.foo", "-o", "-name", and "*.bar".
find finds one match for "*.bar", but no match for "wibble.foo". It never even knows you wanted it to look for "*.foo".
So what can you do about this? Well, in this particular case adding strategic double-quotes (eval "find . $ftypes") would prevent the spurious wildcard substitution, but in general it's best to avoid eval entirely. When you need to build commands, an array is a much better way to go (see BashFAQ #050 for more discussion):
$ ftypes=(-name '*.foo' -o -name '*.bar')
$ find . "${ftypes[#]}"
./a.foo
./b.bar
Note that you can also build the options bit by bit:
$ ftypes=(-name '*.foo')
$ ftypes+=(-o -name '*.bar')
$ ftypes+=(-o -name '*.baz')
Simply prefix the line with eval to force the shell to expand and parse the command:
eval find . $ftypes
Without the eval, the '*.foo' is passed on literally instead of just *.foo (that is, the ' are suddenly considered to be part of the filename, so find is looking for files that start with a single quote and have an extension of foo').
The problem is that since $ftypes a single quoted value, find does see it as a single argument.
One way around it is:
$ eval find . $ftypes