How can I use command substitution in find … -exec … to avoid using xargs in the following command?
find -L -- /path/to/directory -mindepth 2 -maxdepth 2 -type d -exec dirname '{}' \; | xargs basename -a
I tried the following using command substitution, but it output . for each result instead of the desired output:
find -L -- /path/to/directory -mindepth 2 -maxdepth 2 -type d -exec basename "$(dirname '{}')" \;
Your first command will return strange results if a path contains whitespace.
Use a small shell script:
find -L -- . -mindepth 2 -maxdepth 2 -type d -exec sh -c 'basename "$(dirname "{}")"' \;
Alternative syntax to pass one path argument to the script:
find -L -- . -mindepth 2 -maxdepth 2 -type d -exec sh -c 'basename "$(dirname "$1")"' sh {} \;
Or pass as many arguments to the script as possible:
find -L -- . -mindepth 2 -maxdepth 2 -type d -exec sh -c '
for path do
basename "$(dirname "$path")"
done
' sh {} +
With GNU utilities it's possible to output NUL-terminated strings with dirname passed to xargs -0. The basename command is not run if there are no arguments (-r):
find -L -- . -mindepth 2 -maxdepth 2 -type d -exec dirname -z {} + | xargs -r0 basename -a
Related
I've been using "find -exec" commands as in code 1 and 2:
Code 1
find . -type d -exec chmod 775 {} +
Code 2
find . -type f -exec mv {} ./.. \;
But sometimes I see people use "bash" or "sh" just after "-exec" as in code 3 and 4:
Code 3
alias foo="find . -maxdepth 1 -mindepth 1 -type d -exec sh -c 'echo \"\$(find \"{}\" -type f | wc -l)\" {}' \; | sort -nr
Code 4
alias foo="find . -maxdepth 1 -mindepth 1 -type d -exec bash -c 'echo \"\$(find \"\${1}\" -type f | wc -l) \"\${1}\" \"' -- \"{}\" \; | sort -nr"
Question 1abc: When do we use bash or sh after -exec ? Is there something special about echo as opposed to chmod and mv? Does echo belong to bash and if so, how do I know what other commands need bash in front?
Code 4 was a proposed improvement upon Code 3 because
they said "you should pass the filename to the inner find command as an argument. Otherwise you will run into problems if one of your folders has a name with a " in it:" , which I can understand.
Question 2abc:
I don't understand how ${1} makes it safer than {}? Don't they reference the same thing?
I tried replacing the last \"{}\" part with \"\${1}\" like below but it fails strangely. I don't understand why.
Code 5
alias foo="find . -maxdepth 1 -mindepth 1 -type d -exec bash -c 'echo \"\$(find \"\${1}\" -type f | wc -l) \"\${1}\" \"' -- \"\${1}\" \; | sort -nr"
I have this command to recursively find directories that contain mustExist.js but not cannotExist.js:
comm -13 <(find . -type f -name cannotExist.js -exec dirname {} \; | sort -u) <(find . -type f -name mustExist.js -exec dirname {} \; | sort -u)
It works fine.
Now, I must pass it as a string to a node.js automation script. The script picks up the string and runs it as sh -c <string>. I cannot change that part.
So I pass this string:
'comm -13 <(find . -type f -name cannotExist.js -exec dirname {} \\; | sort -u) <(find . -type f -name mustExist.js -exec dirname {} \\; | sort -u)'
However, I always encounter this error:
Warning: Command failed: /bin/sh -c comm -13 <(find . -type f -name cannotExist.js -exec dirname {} \; | sort -u) <(find . -type f -name mustExist.js -exec dirname {} \; | sort -u)
/bin/sh: 1: Syntax error: "(" unexpected
When I pass this string:
'"comm -13 <(find . -type f -name cannotExist.js -exec dirname {} \\; | sort -u) <(find . -type f -name mustExist.js -exec dirname {} \\; | sort -u)"'
I get:
Warning: Command failed: /bin/sh -c "comm -13 <(find . -type f -name cannotExist.js -exec dirname {} \; | sort -u) <(find . -type f -name mustExist.js -exec dirname {} \; | sort -u)"
/bin/sh: 1: comm -13 <(find . -type f -name cannotExist.js -exec dirname {} \; | sort -u) <(find . -type f -name mustExist.js -exec dirname {} \; | sort -u): not found
When I manually try using sh to immitate the automation script:
sh -c comm -13 <(find . -type f -name cannotExist.js -exec dirname {} \; | sort -u) <(find . -type f -name mustExist.js -exec dirname {} \; | sort -u)
I get a different error:
comm: missing operand
Or with quotes:
sh -c "comm -13 <(find . -type f -name cannotExist.js -exec dirname {} \; | sort -u) <(find . -type f -name mustExist.js -exec dirname {} \; | sort -u)"
I get:
/bin/sh: 1: Syntax error: "(" unexpected
Other quotes:
sh -c 'comm -13 <(find . -type f -name cannotExist.js -exec dirname {} \; | sort -u) <(find . -type f -name mustExist.js -exec dirname {} \; | sort -u)'
/bin/sh: 1: Syntax error: "(" unexpected
Is it possible to do this somehow?
There is a much simpler and more efficient command you can use:
find . -type d -exec sh -c 'test -e "$1"/mustExist.js && ! test -e "$1"/cannotExist.js' _ {} \; -print
This iterates over the directories, and checks each one for the required file as well as making sure the forbidden file is not present.
An example, where only foo/bar3 contains yes.js without also containing no.js.
$ mkdir -p foo/bar1 foo/bar2 foo/bar3
$ touch foo/bar1/no.js foo/bar1/yes.js foo/bar2/no.js foo/bar3/yes.js
$ find foo -type d -exec sh -c 'test -e "$1"/yes.js && ! test -e "$1"/no.js' _ {} \; -print
foo/bar3
Passing this to your script requires some creative quoting, though:
somescript "find foo -type d -exec sh -c 'test -e \"\$1\"/yes.js && ! test -e \"\$1\"/no.js' _ {} \; -print"
If you are using bash, you can simplify it a little:
somescript $'find foo -type d -exec sh -c \'test -e "$1"/yes.js && ! test -e "$1"/no.js\' _ {} \; -print'
If you are willing to use the obsolete and possibly unsupported operator -a, you can reduce this to a single invocation of test.
find . -type d -exec test -e {}/mustExist.js -a ! -e {}/cannotExist.js \; -print
This is also a little simpler to pass to your script, since it does not itself contain any quotes:
somescript 'find . -type d -exec test -e {}/mustExist.js -a ! -e {}/cannotExist.js \; -print'
You can simplify it using multiple -exec primaries as well:
somescript 'find . -type d -exec test -e {}/mustExist.js \; ! -exec test -e {}/cannotExist.js \; -print'
which is a little less efficient (it runs test twice instead of once) but is more portable, while being easier to quote than the version that passes the string to sh -c.
I have a dozen files named
~/DOMAIN1.de/bin/dbdeploy.php
~/DOMAIN2.de/bin/dbdeploy.php
~/DOMAIN3.de/bin/dbdeploy.php
I want to run them all with the same arguments.
My bash script reads:
cd ~
find . -maxdepth 1 -type d -name "*\.de" -exec php56 bin/dbdeploy.php "$1" "$2" \;
However, the path given to exec seems not to be relative to the found subdirectory but rather to my PWD:
$ bash -x ./.dbpush "some argument"
+ cd ~
+ find . -maxdepth 1 -type d -name '*\.de' -exec php56 bin/dbdeploy.php 'some argument' ';'
Could not open input file: bin/dbdeploy.php
Could not open input file: bin/dbdeploy.php
Could not open input file: bin/dbdeploy.php
How can I use the found path in the -exec directive?
Ok, actually I found the answer myself:
The "find"-results are stored in {}, so the line reads
find . -maxdepth 1 -type d -name "*\.de" -exec php56 {}/bin/dbdeploy.php "$1" "$2" \;
Alternativly
find . -type f -wholename "*\.de/bin/dbdeploy.php" -exec php56 {} "$1" "$2" \;
for example:
find /usr/lib -maxdepth 1 -type l -iname "*libblas*"|xargs -I{} echo "{} =>" $(realpath {})
I would like it to output:
/usr/lib/libblas.so.3gf=>/usr/lib/libblas/libblas.so.3gf.0
/usr/lib/libblas.so=>/usr/lib/libblas/libblas.so.3gf.0
/usr/lib/libblas.a=>/usr/lib/libblas/libblas.a
This will not work because the value in $() is expanded before the script actual running.
Is there any way I can achieve this result? without loop in bash?
Alternatively:
find /usr/lib -maxdepth 1 -type l \
-exec echo -n '{} =>' \; \
-exec realpath '{}' \;
Have xargs call the shell:
find /usr/lib -maxdepth 1 -type l -iname "*libblas*"|xargs -I{} sh -c 'echo "{} =>" $(realpath {})'
You need the command substitution to happen after the file name is known. So you need xargs to call a shell and do the substitution there.
Since you're running the command on a single file at a time, using xargs is a useless complication (and it also mangles some file names). Use -exec!
find /usr/lib -maxdepth 1 -type l -iname "*libblas*" -exec sh -c 'echo "$0 => $(realpath "$0")' {} \;
You could make this slightly faster and not less clear by not using a command substitution:
find /usr/lib -maxdepth 1 -type l -iname "*libblas*" -exec sh -c 'echo -n "$0 => "; realpath "$0"' {} \;
To make things a little faster, don't invoke a new shell process for every file:
find /usr/lib -maxdepth 1 -type l -iname "*libblas*" -exec sh -c 'for x; do echo -n "$x => "; realpath "$x"; done' _ {} +
(You can do the same with xargs, but just drop xargs and stick to the simpler, faster, more robust -exec.)
Try to convert each filename separately using line-by-line "while" loop:
find ... | while read f; do echo "$f" '=>' "$(realpath $f)" ; done
The shortest seems to be using GNU Parallel:
find /usr/lib -maxdepth 1 -type l -iname "*libblas*"|parallel echo {} '=\> $(readlink -f {})'
What syntax should I use in a bash script to list files based on 3 dynamic values:
- older than X days
- in a specified directory
- whose name contains a specified string?
FILEAGE=7
FILEDIR='"/home/ecom/tmp"'
FILESTRING='"search-results-*"'
FILES_FOR_REMOVAL=$("/usr/bin/find "${FILEDIR}" -maxdepth 1 -type f -mtime +${FILEAGE} -name "${FILESTRING}" -exec ls -lth {} \;")
echo ${FILES_FOR_REMOVAL}
If I try the above I get:
-bash: /usr/bin/find "/home/ecom/tmp" -maxdepth 1 -type f -mtime +7 -name "search-results-*" -exec ls -lth {} \;: No such file or directory
Remove superfluous quotes:
FILEAGE=7
FILEDIR='/home/ecom/tmp'
FILESTRING='search-results-*'
FILES_FOR_REMOVAL=$(/usr/bin/find "${FILEDIR}" -maxdepth 1 -type f -mtime +${FILEAGE} -name "${FILESTRING}" -exec ls -lth {} \;)
Your syntax for 'find' looks ok. Try removing the quotes around the command string, i.e.
FILES_FOR_REMOVAL=$(/usr/bin/find "${FILEDIR}" -maxdepth 1 -type f -mtime +${FILEAGE} -name "${FILESTRING}" -exec ls -lth {} \;)
FILEAGE=7
FILEDIR='/home/ecom/tmp'
FILESTRING='search-results-*'
/usr/bin/find "${FILEDIR}" -maxdepth 1 -type f -mtime +${FILEAGE} -name "${FILESTRING}" -exec /bin/ls -lth '{}' \;
There were some extra quotes that created the error. Also specify full path to /bin/ls to avoid problems with potential aliasing of ls(1). And to get filenames on a separate line, I dropped the $FILES_FOR_REMOVAL variable. You can also use
/usr/bin/find "${FILEDIR}" -maxdepth 1 -type f -mtime +${FILEAGE} -name "${FILESTRING}" -ls
(I can't add comments, but ... )
To reliably handle file names with spaces, you may want to consider storing the file list in a temp text file instead of a variable and loop through it using a while construct (instead of a for)
For example:
FILEAGE=7
FILEDIR='/home/ecom/tmp'
FILESTRING='search-results-*'
TEMPFILE=".temp${RANDOM}"
CMD="find \"${FILEDIR}\" -maxdepth 1 -type f -mtime +${FILEAGE} -name \"${FILESTRING}\" -exec /bin/ls -lth '{}' \;"
$CMD > $TEMPFILE # write output to file
while read thefile; do
do_somthing_to $thefile
done < $TEMPFILE
rm $TEMPFILE # clean up after
Or, if you're only going to use the list once, pipe the output directly to the while construct:
$CMD | while read thefile; do
do_something_to $thefile
done