I am trying to build and run a find command from a script. But I get a very cryptic error message from find. The following basically sums up how I build the command line and run it
$ xx="find . -name 'p*' -mmin +10 -exec echo {} \\;"
$ echo "$xx" #.....and I get the same print from echo $xx
find . -name 'p*' -mmin +10 -exec echo {} \;
$ $xx
find: missing argument to `-exec'
$ find . -name 'p*' -mmin +10 -exec echo {} \;
./p2.sh
./p1.sh
$ read xx
find . -name 'p*' -mmin +2 -exec echo {} \\;
$ $xx
find: missing argument to `-exec'
I am stuck and will appreciate your help. I am also wondering what's causing this. I am using bash 3.2.51 on SLES.
The actual command I want to execute is a little bit longer but I used echo here just to illustrate.
Thanks
Dinesh
Trying to store complicated commands in bash variables and then evaluate the variables pretty well never works.
If you need to build a command in pieces, use an array. See this useful Bash FAQ: I'm trying to put a command in a variable, but the complex cases always fail!.
Here's the basic strategy:
# Make an array
declare -a findcmd=(find .)
# Add some arguments
findcmd+=(-name 'p*')
findcmd+=(-mmin +10)
findcmd+=(-exec echo {} \;)
# Run the command
"${findcmd[#]}"
You need to understand how bash quoting works. Remember that the quoting (and de-quoting) only happens once, when you type the command (or when bash reads it from a script file). Quotes which get into the values of variables are just ordinary characters.
If you're experimenting with set -x, remember also that set -x inserts quotes in order to remove ambiguities. These quotes are not part of the variables. While that is clearly essential, it seems to be confusing to programmers who are not familiar with the bash execution model.
Related
I have a mongodump which I want to import apparently I'm looking to do this using the find command. Something like this:
find *.bson -type f -exec echo mongoimport --db=abc --collection=$(echo '{}' | sed s/.bson//g) {} \;
What I'm looking isn't get evaluate what I need is
mongoimport --db=abc --collection=a a.bson
but I'm getting is
mongoimport --db=abc --collection=a.bson a.bson
My version of using sed to strip the .bson suffix from '{}' isn't working. I know its not a blocker but I felt if that is possible.
Any suggestions?
The problem twofold:
Shell expansions: Before a command is executed in a shell environment, the shell (sh/bash/ksh/zsh) will perform a sequence of expansions to build up the actual command that is being executed. There are seven kinds of expansion performed: brace expansion, tilde expansion, parameter and variable expansion, command substitution, arithmetic expansion, word splitting, and pathname expansion. Hence, before the find command will be executed, it will perform all substitutions, including the command substitution located in the exec statement. Ergo, the command is equivalent to:
$ find *.bson -type f -exec echo mongoimport --db=abc --collection={} {} \;
A way forward would be to prohibit the command substitution by using single-quotes, however this leads to problem two.
find's exec statement is limited: The command that -exec can execute is limited to an external utility with optional arguments. Various shell features are therefor not recognized. To use shell built-ins, functions, conditionals, pipelines, redirections etc. directly with -exec is not possible, unless wrapped in something like a sh -c child shell.
Hence the answer would be something in the line of:
$ find *.bson -type f -exec /usr/bin/sh -c 'echo mongoimport --db=abc --collection=$(echo {} | sed s/.bson//g) {}' \;
Suggesting different strategy to this problem.
Use find with option -printf to prepare your commands.
The result will be list of commands to execute (command per line).
After inspecting and testing the commands, save find command output into a file and run the file (as a bash script).
Or just run directly into bash command.
1. find result inspection:
find . -type f -name "*.bson" -printf "mongoimport --db=abc --collection=%f %f\n" | sed s/.bson//
Notice sed replacement only on first .bson match. Do not use g option.
2. Run processed and inspected find output.
bash <<< $(find . -type f -name "*.bson" -printf "mongoimport --db=abc --collection=%f %f\n" | sed s/.bson//)
This command works
find . -name \*.txt -print
and outputs two filenames
This command works
bash -c 'echo . $0 $1 -print' "-name" "\*.txt"
and outputs this result:
. -name *.txt -print
But this command
bash -c 'find . $0 $1 -print' "-name" "\*.txt"
does not give an error but does not output anything either.
Can anyone tell me what is happening here?
It looks like you're trying to use "\*.txt" to forestall glob-expansion so that the find command sees *.txt instead of e.g. foo.txt.
However, what it ends up seeing is \*.txt. No files match that pattern, so you see no output.
To make find see *.txt as its 3rd argument, you could do this:
bash -c 'find . $0 "$1" -print' "-name" "*.txt"
Edit: Are you really getting . -name *.txt -print as the output of the first command where you replaced find with echo? When I run that command, I get . -name \*.txt -print.
Well the suggestions from francesco work. But I am still confused by the behaviour here.
We know that putting unquoted wild cards in a find command will usually result in an error. To wit:
find . -name *.txt -print
find: paths must precede expression: HowTo-Word-Split.txt' find:
possible unquoted pattern after predicate-name'?
However putting the wild card in single quotes (or escaping it if it is only 1 char) will work like so:
find . -name \*.txt -print
which gives this output (on two separate lines)
> ./HowTo-Word-Split.txt
> ./bash-parms.txt
So in the bash -c version what I was thinking was this:
bash -c 'find . $0 $1 -print' "-name" "*.txt"
would result in the *.txt being expanded even before being passed in to the cmd string,
and using single quotes around it would result in trying to execute (after the arg substitution and the -c taking effect)
find . -name *.txt -print
which as I just demonstrated does not work.
However there seems to be some sort of magic associated with the -c switch as demonstrated by setting -x at the bash prompt, like so:
$ set -x
$ bash -c ' find . $0 "$1" -print' "-name" "*.txt"
+ bash -c ' find . $0 "$1" -print' -name '*.txt'
./HowTo-Word-Split.txt
./bash-parms.txt
Note that even though I used double quotes in the -c line, bash actually executed the find with single quotes put around the argument, thus making find work.
Problem solved. :)!
So I have the following command which looks for a series of files and appends three lines to the end of everything found. Works as expected.
find /directory/ -name "file.php" -type f -exec sh -c "echo -e 'string1\string2\nstring3\n' >> {}" \;
What I need to do is also look for any instance of string1, string2, or string3 in the find ouput of file.php prior to echoing/appending the lines so I don't append a file unnecessarily. (This is being run in a crontab)
Using | grep -v "string" after the find breaks the -exec command.
How would I go about accomplishing my goal?
Thanks in advance!
That -exec command isn't safe for strings with spaces.
You want something like this instead (assuming finding any of the strings is reason not to add any of the strings).
find /directory/ -name "file.php" -type f -exec sh -c "grep -q 'string1|string2|string3' \"\$1\" || echo -e 'string1\nstring2\nstring3\n' >> \"\$1\"" - {} \;
To explain the safety issue.
find places {} in the command it runs as a single argument but when you splat that into a double-quoted string you lose that benefit.
So instead of doing that you pass the file as an argument to the shell and then use the positional arguments in the shell command with quotes.
The command above simply chains the echo to a failure from grep to accomplish the goal.
I have written an executable in c++, which is designed to take input from a file, and output to stdout (which I would like to redirect to a single file). The issue is, I want to run this on all of the files in a folder, and the find command that I am using is not cooperating. The command that I am using is:
find -name files/* -exec ./stagger < {} \;
From looking at examples, it is my understanding that {} replaces the file name. However, I am getting the error:
-bash: {}: No such file or directory
I am assuming that once this is ironed out, in order to get all of the results into one file, I could simply use the pattern Command >> outputfile.txt.
Thank you for any help, and let me know if the question can be clarified.
The problem that you are having is that redirection is processed before the find command. You can work around this by spawning another bash process in the -exec call:
find files/* -exec bash -c '/path/to/stagger < "$1"' -- {} \;
The < operator is interpreted as a redirect by the shell prior to running the command. The shell tries redirecting input from a file named {} to find's stdin, and an error occurs if the file doesn't exist.
The argument to -name is unquoted and contains a glob character. The shell applies pathname expansion and gives nonsensical arguments to find.
Filenames can't contain slashes. The argument to -name can't work even if it were quoted. If GNU find is available, -path can be used to specify a glob pattern files/*, but this doesn't mean "files in directories named files", for that you need -regex. Portable solutions are harder.
You need to specify one or more paths for find to start from.
Assuming what you really wanted was to have a shell perform the redirect, Here's a way with GNU find.
find . -type f -regex '.*foo/[^/]*$' -exec sh -c 'for x; do ./stagger <"$x"; done' -- {} +
This is probably the best portable way using find (-depth and -prune won't work for this):
find . -type d -name files -exec sh -c 'for x; do for y in "$x"/*; do [ -f "$y" ] && ./stagger <"$y"; done; done' -- {} +
If you're using Bash, this problem is a very good candidate for just using a globstar pattern instead of find.
#!/usr/bin/env bash
shopt -s extglob globstar nullglob
for x in **/files/*; do
[[ -f "$x" ]] && ./stagger <"$x"
done
Simply escape the less-than symbol, so that redirection is carried out by the find command rather than the shell it is running in:
find files/* -exec ./stagger \< {} \;
I want to do something on the lines of:
find -name *.mk | xargs "for i in $# do mv i i.aside end"
I realize that there might be more than on error in this, but I'd like to specifically know about this sort of inline command definition that I can pass xargs to.
This particular command isn't a great example, but you can use an "inline shell script" by giving sh -c 'here is the script' as a command. And you can give it arguments which will be $# inside the script but there's a catch: the first argument after here is the script goes to $0 inside the script, so you have to put an extra word there or you'll lose the first argument.
find . -name '*.mk' -exec sh -c 'for i; do mv "$i" "$i.aside"; done' fnord '{}' +
Another fun feature I took advantage of there is the fact that for loops iterate over the command line arguments by default: for i; do ... is equivalent to for i in "$#"; do ...
I reiterate, the above command is convoluted and slow compared to the many other methods of doing the bulk mv. I'm posting it only to show some cool syntax.
There's no need for xargs here
find -name *.mk -exec mv {} {}.aside \;
I'm not sure what the semantics of your for loop should be, but blindly coding it would give something like this:
find -name *.mk | while read file
do
for i in $file; do mv $i $i.aside; done
done
If the body is used in multiple places, you can also use bash functions.
In some version of find an argument is needed : . for the current directory
Star * must be escaped
You can try with echo command to be sure what command will do
find . -name '*.mk' -print0 | xargs -0i sh -c "echo mv '{}' '{}.aside'"
man xargs
/-i
man sh
/-c
I'm certain you could do this in a nice manner, but since you requested xargs:
find -name "*.tk" | xargs -I% mv % %.aside
Looping over filenames makes no sense, since you can only rename one at a time. Using inline uglyness is not necessary, but I could not make it work with the pipe and either eval or bash -c.