The find command seems to differ from other Unix commands.
Why is there the empty curly brackets and a backward flash at the end of the following command?
find * -perm 777 -exec chmod 770 {} \;
I found one reason for the curly brackets but not for the backward flash.
The curly brackets are apparently for the path
Same as -exec, except that ``{}'' is
replaced with as many pathnames as
possible for each invocation of
utility
The -exec command may be followed by any number of arguments that make up the command that is to be executed for each file found. There needs to be some way to identify the last argument. This is what \; does. Note that other things may follow after the -exec switch:
find euler/ -iname "*.c*" -exec echo {} \; -or -iname "*.py" -exec echo {} \;
(This finds all c-files and python files in the euler directory.)
The reason that exec does not require the full command to be inside quotes, is that this would require escaping a lot of quotes inside the command, in most circumstances.
The string {} in find is replaced by the pathname of the current file.
The semicolon is used for terminating the shell command invoked by find utility.
It needs to be escaped, or quoted, so it won't be interpreted by the shell, because ; is one of the special characters used by shell (list operators).
See also: Why are the backslash and semicolon required with the find command's -exec option?
The (escaped) semicolon is needed so that "find" can tell where the arguments to the exec'd program end (if there are any) and additional arguments to "find" begin.
I'd recommend that you instead do that as
find . -perm 777 -print0 | xargs -0 chmod 770
"xargs" says to take the results of the find and feed it 20 at a time to the following command.
Related
I have a mongodump which I want to import apparently I'm looking to do this using the find command. Something like this:
find *.bson -type f -exec echo mongoimport --db=abc --collection=$(echo '{}' | sed s/.bson//g) {} \;
What I'm looking isn't get evaluate what I need is
mongoimport --db=abc --collection=a a.bson
but I'm getting is
mongoimport --db=abc --collection=a.bson a.bson
My version of using sed to strip the .bson suffix from '{}' isn't working. I know its not a blocker but I felt if that is possible.
Any suggestions?
The problem twofold:
Shell expansions: Before a command is executed in a shell environment, the shell (sh/bash/ksh/zsh) will perform a sequence of expansions to build up the actual command that is being executed. There are seven kinds of expansion performed: brace expansion, tilde expansion, parameter and variable expansion, command substitution, arithmetic expansion, word splitting, and pathname expansion. Hence, before the find command will be executed, it will perform all substitutions, including the command substitution located in the exec statement. Ergo, the command is equivalent to:
$ find *.bson -type f -exec echo mongoimport --db=abc --collection={} {} \;
A way forward would be to prohibit the command substitution by using single-quotes, however this leads to problem two.
find's exec statement is limited: The command that -exec can execute is limited to an external utility with optional arguments. Various shell features are therefor not recognized. To use shell built-ins, functions, conditionals, pipelines, redirections etc. directly with -exec is not possible, unless wrapped in something like a sh -c child shell.
Hence the answer would be something in the line of:
$ find *.bson -type f -exec /usr/bin/sh -c 'echo mongoimport --db=abc --collection=$(echo {} | sed s/.bson//g) {}' \;
Suggesting different strategy to this problem.
Use find with option -printf to prepare your commands.
The result will be list of commands to execute (command per line).
After inspecting and testing the commands, save find command output into a file and run the file (as a bash script).
Or just run directly into bash command.
1. find result inspection:
find . -type f -name "*.bson" -printf "mongoimport --db=abc --collection=%f %f\n" | sed s/.bson//
Notice sed replacement only on first .bson match. Do not use g option.
2. Run processed and inspected find output.
bash <<< $(find . -type f -name "*.bson" -printf "mongoimport --db=abc --collection=%f %f\n" | sed s/.bson//)
I want to copy all the xml files which is having current date as file name from all directories. Below is the script i have written.
#!/bin/bash
CURRENT_DATE=`date +'%d%m%Y'`
Temp_Path=/appinfprd/bi/infogix/IA83/InfogixClient/Scripts/IRP/New_Vendors/
FILE_PATH=/bishare/DLSFTP/DLSTREAM/
FILE_DATE=`date -d "-2 days" +"%Y%m%d"`
cd $FILE_PATH
find . -name '*$FILE_DATE*.xml' -exec cp $Temp_Path
But it is not working.
Your find statement is wrong. You should end it with \; to indicate the end of the exec command and put {} where the name of your file found should come in the command. So, you want :
find . -name "*$FILE_DATE*.xml" -exec cp "{}" "$Temp_Path" \;
Edit
As stated in the comments, there were also a problem in your initial post with your single quotes that should be double quotes. You might be interested by this man page. In particular by these sections :
-exec command ;
Execute command; true if 0 status is returned. All following arguments to find are taken to be arguments to the command until an argument consisting of ; is encountered. The string {} is replaced by the current file name being processed everywhere it occurs in the arguments to the command, not just in arguments where it is alone, as in some versions of find. Both of these constructions might need to be escaped (with a \) or quoted to protect them from expansion by the shell. See the EXAMPLES section for examples of the use of the -exec option. The specified command is run once for each matched file. The command is executed in the starting directory. There are unavoidable security problems surrounding use of the -exec action; you should use the -execdir option instead.
-exec command {} +
This variant of the -exec action runs the specified command on the selected files, but the command line is built by appending each selected file name at the end; the total number of invocations of the command will be much less than the number of matched files. The command line is built in much the same way that xargs builds its command lines. Only one instance of {} is allowed within the command, and (when find is being invoked from a shell) it should be quoted (for example, '{}') to protect it from interpretation by shells. The command is executed in the starting directory. If any invocation returns a non-zero value as exit status, then find returns a non-zero exit status. If find encounters an error, this can sometimes cause an immediate exit, so some pending commands may not be run at all. This variant of -exec always returns true.
I have written an executable in c++, which is designed to take input from a file, and output to stdout (which I would like to redirect to a single file). The issue is, I want to run this on all of the files in a folder, and the find command that I am using is not cooperating. The command that I am using is:
find -name files/* -exec ./stagger < {} \;
From looking at examples, it is my understanding that {} replaces the file name. However, I am getting the error:
-bash: {}: No such file or directory
I am assuming that once this is ironed out, in order to get all of the results into one file, I could simply use the pattern Command >> outputfile.txt.
Thank you for any help, and let me know if the question can be clarified.
The problem that you are having is that redirection is processed before the find command. You can work around this by spawning another bash process in the -exec call:
find files/* -exec bash -c '/path/to/stagger < "$1"' -- {} \;
The < operator is interpreted as a redirect by the shell prior to running the command. The shell tries redirecting input from a file named {} to find's stdin, and an error occurs if the file doesn't exist.
The argument to -name is unquoted and contains a glob character. The shell applies pathname expansion and gives nonsensical arguments to find.
Filenames can't contain slashes. The argument to -name can't work even if it were quoted. If GNU find is available, -path can be used to specify a glob pattern files/*, but this doesn't mean "files in directories named files", for that you need -regex. Portable solutions are harder.
You need to specify one or more paths for find to start from.
Assuming what you really wanted was to have a shell perform the redirect, Here's a way with GNU find.
find . -type f -regex '.*foo/[^/]*$' -exec sh -c 'for x; do ./stagger <"$x"; done' -- {} +
This is probably the best portable way using find (-depth and -prune won't work for this):
find . -type d -name files -exec sh -c 'for x; do for y in "$x"/*; do [ -f "$y" ] && ./stagger <"$y"; done; done' -- {} +
If you're using Bash, this problem is a very good candidate for just using a globstar pattern instead of find.
#!/usr/bin/env bash
shopt -s extglob globstar nullglob
for x in **/files/*; do
[[ -f "$x" ]] && ./stagger <"$x"
done
Simply escape the less-than symbol, so that redirection is carried out by the find command rather than the shell it is running in:
find files/* -exec ./stagger \< {} \;
Can someone show me to use xargs properly? Or if not xargs, what unix command should I use?
I basically want to input more than (1) file name for input <localfile>, third input parameter.
For example:
1. use `find` to get list of files
2. use each filename as input to shell script
Usage of shell script:
test.sh <localdir> <localfile> <projectname>
My attempt, but not working:
find /share1/test -name '*.dat' | xargs ./test.sh /staging/data/project/ '{}' projectZ \;
Edit:
After some input from everybody and trying -exec, I am finding that my <localfile> filename input with find is also giving me the full path. /path/filename.dat instead of filename.dat. Is there a way to get the basename from find? I think this will have to be a separate question.
I'd just use find -exec here:
% find /share1/test -name '*.dat' -exec ./test.sh /staging/data/project/ {} projectZ \;
This will invoke ./test.sh with your three arguments once for each .dat file under /share1/test.
xargs would pack up all of these filenames and pass them into one invocation of ./test.sh, which doesn't look like your desired behaviour.
If you want to execute the shell script for each file (as opposed to execute in only once on the whole list of files), you may want to use find -exec:
find /share1/test -name '*.dat' -exec ./test.sh /staging/data/project/ '{}' projectZ \;
Remember:
find -exec is for when you want to run a command on one file, for each file.
xargs instead runs a command only once, using all the files as arguments.
xargs stuffs as many files as it can onto the end of the command line.
Do you want to execute the script on one file at a time or all files? For one at a time, use file's exec, which it looks like you're already using the syntax for, and which xargs doesn't use:
find /share1/test -name '*.dat' -exec ./test.sh /staging/data/project/ '{}' projectZ \;
xargs does not have to combine arguments, it's just the default behavior. this properly uses xargs, to execute the commands, as intended.
find /share1/test -name '*.dat' -print0 | xargs -0 -I'{}' ./test.sh /staging/data/project/ '{}' projectZ
When piping find to xargs, NULL termination is usually preferred, I recommend appending the -print0 option to find. After which you must add -0 to xargs, so it will expect NULL terminated arguments. This ensures proper handling of filenames. It's not POSIX proper, but considered well supported. You can always drop the NULL terminating options, if your commands lack support.
Remeber while find's purpose is finding files, xargs is much more generic. I often use xargs to process non-filename arguments.
I'm currently pawing through a Unix tutorial and I've been met with this:
find ~ -name test3* -ok rm {}\;
I'm curious as to what the {}\; does.
The {}\; string tells find(1) that (a) the file name is to be substituted in place of the {}, and (b) that the commands ends at the ";". The ";" has to be escaped (hence the backslash) because it has special meaning to the shell. For that same reason, you really ought to quote the 'test3*' string. You want find to expand that, not the shell. If there happen to be matching files in the directory where you run find, you're not going to get the results you expect.
Thus, you're telling find(1) to run "rm" on every file it finds.
There's a more efficient solution to that particular problem, though:
find . -name 'test3*' -print | xargs rm -f