I am running the following bash script to find and execute an ID3 modification command on all the .mp3 files in my folders
find -name "*.mp3" -exec bash -c 'eyeD3 --to-v1.1 "{}"' \;
It fails on filenames that contain dollar signs and backticks
bash: -c: line 0: unexpected EOF while looking for matching ``'
bash: -c: line 1: syntax error: unexpected end of file
filenames that fail are eg.
08 - Sam Smith - I'm Not The Only One (Feat. A$AP Rocky).mp3
02 - The Vaccines - Wreckin` Bar (Ra Ra Ra).mp3
I know these are special characters so need escaping - but how?
You don't need Bash there.
find -name '*.mp3' -exec eyeD3 --to-v1.1 {} +
Most probably you have some characters in file that change character of your command/parameters. Might be that your filename is not recognized as you thought it would be. Remeber: in linux shell problems with spces/special characters never end.
Therefore it would be easier to do it in safer way:
find -name "*.mp3" -print0 | xargs -n 1 -0 eyeD3 --to-v1.1
Related
First I'm not a star with shell-scripting, more used to programming in Python, but have to work with an existing Python script which calls Unix commands via subprocess.
In this script we use 2 find commands to check if 2 certain strings can be found in an xml file / file-name:
FIND_IN_FILE_CMD: find <directory> -name *.xml -exec grep -Hnl STRING1|STRING2 {} +
FIND_IN_FILENAME_CMD: find <directory> ( -name *STRING1*xml -o -name *STRING2*xml )
The problem we saw is that STRING1 and STRING2 are not always written capitalized.
Now I can do something like STRING1|STRING2|String1|String2|string1|string2 and ( -name *STRING1*xml -o -name *STRING2*xml -o -name *String1*xml -o -name *String2*xml -o -name *string1*xml -o -name *string2*xml ), but I was wondering if there was something more efficient to do this check in one go which basically matches all different writing styles.
Can anybody help me with that?
Both of your commands have syntax errors:
$ find -name *.xml -exec grep -Hnl STRING1|STRING2 {} +
bash: STRING2: command not found
find: missing argument to `-exec'
This is because you cannot have an unquoted | in a shell command as that is taken as a pipe symbol. As you can see above, the shell tries to execute STRING2 as a command. In any case, grep cannot understand | unless you use the -E flag or, if your grep supports it, the -P flag. For vanilla grep, you need STRING1\|STRING2.
All implementations of grep should support the POSIX-mandated -i and -E options:
-E
Match using extended regular expressions. Treat each pattern specified as an ERE, as described in XBD Extended Regular Expressions. If any entire ERE pattern matches some part of an input line excluding the terminating <newline>, the line shall be matched. A null ERE shall match every line.
-i
Perform pattern matching in searches without regard to case; see XBD Regular Expression General Requirements.
This means you can use -i for case insensitive matching and -E for extended regular expressions, making your command:
find <directory> -name '*.xml' -exec grep -iEHnl 'STRING1|STRING2' {} +
Note how I also quoted the *.xml since without the quotes, if any xml files
are present in the directory you ran the command in, then *.xml would be expanded by the shell to the list of xml files in that directory.
Your next command also has issues:
$ find ( -name *STRING1*xml -o -name *STRING2*xml )
bash: syntax error near unexpected token `-name'
This is because the ( has a special meaning in the shell (it opens a subshell) so you need to escape it (\(). As for case insensitive matching, GNU find, the default on Linux has an -iname option which is equivalent to -name but case insensitive. If you are using GNU find, then you can do:
find <directory> \( -iname '*STRING1*xml' -o -iname '*STRING2*xml' \)
If your find doesn't have -iname, you are stuck with writing out all possible permutations. In all cases, however, you will need to quote the patterns and escape the parentheses as I have done above.
If you are going to continue using find, just replace -name with the case insensitive version -iname.
I'm trying to run a command on a bunch of files:
find . -name "*.ext" | xargs -0 cmd
The above works but it hangs because one of the folders stupidly has an ' in the file name (others have parens and other nonsense).
How do I safely send escaped output to my command? e.g.:
cmd foo\ bar\(baz\)\'\!
[edit] I know I can run find ... -exec cmd {} \; but the actual command I'm running is more complicated and being piped through sed first
You can use this while loop to process results of find command that uses NUL terminator using process substitution:
while IFS= read -rd '' file; do
# echo "$file"
cmd
done < <(find -iname "*.ext" -print0)
This can handle filenames with all kind of whitespaces, glob characters, newlines or any other special characters.
Note that this requires bash as process substitution is not supported in bourne shell.
If you have GNU find you can use the -print0 option
find -name "*.ext" -print0 | xargs -0 cmd
Otherwise you would have to ditch xargs. If you have Bash you could use
find -name "*.ext" | while read -a list ; do cmd "${list[#]}" ; done
Note that you do not have to specify current directory as the starting point. If no starting is specified, . is assumed.
GNU Parallel was born exactly because of xargs way of dealing with ", ' and space:
find . -name "*.ext" | parallel cmd
When I run this command in Terminal:
find . -type f -name "*.png" -exec sh -c "file {} | egrep -o '^.*\d+,'" \;
I get this error if a filename contains parentheses:
sh: -c: line 0: syntax error near unexpected token `('
sh: -c: line 0: `file ./(terrible filename).png | egrep -o '^.*\d+,''
I know it has something to do with the sh -c, but I don't know how to fix it, thanks.
./(terrible filename).png: PNG image data, 512 x 512,
// trying to get this result
You are basically pasting the file name into sh -c '...' without any quoting. The string inside sh -c after the substitutions made by find needs to be valid sh syntax, which means there can be no unquoted single quotes, parentheses, etc.
A more robust approach is to use -exec file {} and pass all the output from find to egrep.
find . -type f -name "*.png" -exec file {} \; | egrep -o '^.*\d+,'
The placeholder token {} gets replaced by find with the filename currently being processed. When it is a lone token, find can pass in any file name at all; but if you interpolate it into a longer string, such as a shell command, you will need to ensure that any necessary quoting etc. is added somehow. That's messy, so usually you will want to find a solution where you don't need to do that.
(As pointed out in comments to the other answer, -exec sh -c 'file "$1"' _ {} \; is another way to accomplish that; this generalizes to arbitrarily complex shell commands. If your find supports exec {} \+ you want to add a simple loop: -exec sh 'for f; do file "$f"; done' _ {} \+ -- incidentally, the _ is a dummy placeholder for $0.)
Are there parentheses in the file names? This might help:
find . -type f -name "*.png" -exec sh -c "file '{}' | egrep -o '^.*\d+,'" \;
So I have the following command which looks for a series of files and appends three lines to the end of everything found. Works as expected.
find /directory/ -name "file.php" -type f -exec sh -c "echo -e 'string1\string2\nstring3\n' >> {}" \;
What I need to do is also look for any instance of string1, string2, or string3 in the find ouput of file.php prior to echoing/appending the lines so I don't append a file unnecessarily. (This is being run in a crontab)
Using | grep -v "string" after the find breaks the -exec command.
How would I go about accomplishing my goal?
Thanks in advance!
That -exec command isn't safe for strings with spaces.
You want something like this instead (assuming finding any of the strings is reason not to add any of the strings).
find /directory/ -name "file.php" -type f -exec sh -c "grep -q 'string1|string2|string3' \"\$1\" || echo -e 'string1\nstring2\nstring3\n' >> \"\$1\"" - {} \;
To explain the safety issue.
find places {} in the command it runs as a single argument but when you splat that into a double-quoted string you lose that benefit.
So instead of doing that you pass the file as an argument to the shell and then use the positional arguments in the shell command with quotes.
The command above simply chains the echo to a failure from grep to accomplish the goal.
I have the following command
find . -name "*.tiff" -exec echo `basename -s .tiff {}` \;
I expect this to print all my .tiff-files without their file extensions. What I get is
./file1.tiff
./file2.tiff
...
The command,
find . -name "*.tiff" -exec basename -s .tiff {} \;
does yield
file1
file2
...
Is this not supposed to be the input of echo?
The content of the backticks is executed before the find command - yielding just the placeholder {}, which is used in the find command line - hence your result. You can always use set -x to examine what the shell is up to.
Use single-quote characters (') instead of backticks (`) - putting a command in backticks causes it to be executed and replaced by its output in your command.
Also, modify the command to get rid of the echo, like this:
find . -name "*.tiff" -exec 'basename -s .tiff {}' \;
This will execute basename on each found file.