scenario: There are multiple files in an folder, I'm trying to find specific set of files and if a given file has specific info then I need to grep the information.
Ex:
find /abc/test \( -type f -name 'tst*.txt' -mtime -1 \) -exec grep -Po '(?<type1).*(?=type1|(?<=type2).*(?=type2)' {} \;
I need to include if condition along with find -exec (if grep is true then print the above)
if grep -q 'case=1' <filename>; then
grep -Po '(?<type1).*(?=type1|(?<=type2).*(?=type2)'
fi
Thanks
You can use -exec in find as a condition -- the file matches if the command returns a successful exit code. So you can write:
find /abc/test -type f -name 'tst*.txt' -mtime -1 -exec grep -q 'case=1' {} \; -exec grep -Po '(?<type1).*(?=type1|(?<=type2).*(?=type2)' {} \;
Tests in find are evaluated left-to-right, so the second grep will only be executed if the first one was successful.
If your conditions are more complicated, you can put the whole shell code into a script, and execute the script with -exec. E.g. put this in myscript.sh:
#!/bin/sh
if grep -q 'case=1' "$1"; then
grep -Po '(?<type1).*(?=type1|(?<=type2).*(?=type2)' "$1";
fi
and then do:
find /abc/test -type f -name 'tst*.txt' -mtime -1 -exec ./myscript.sh {} \;
Since you're using PCRE option -P in grep you can combine both searches into one grep as well using lookahead:
find /abc/test -type f -name 'tst*.txt' -mtime -1 -exec grep -Po '(?=.*case=1).*\K((?<=type1).*(?=type1)|(?<=type2).*(?=type2))' {} +
btw the regex shown in your question is invalid, that I've tried to correct it here.
Related
Here is my current code, my goal is to find every file in a given directory (recursively) and replace "FIND" with "REPLACEWITH" and overwrite the files.
FIND='ALEX'
REPLACEWITH='<strong>ALEX</strong>'
DIRECTORY='/some/directory/'
find $DIRECTORY -type f -name "*.html" -print0 |
LANG=C xargs -0 sed -i "s|$FIND|$REPLACEWITH|g"
The error I am getting is:
sed: 1: "/some/directory ...": command a expects \ followed by text
As given in BashFAQ #21, you can use perl to perform search-and-replace operations with no potential for data being treated as code:
in="$FIND" out="$REPLACEWITH" find "$DIRECTORY" -type f -name '*.html' \
-exec perl -pi -e 's/\Q$ENV{"in"}/$ENV{"out"}/g' '{}' +
If you want to include only files matching the FIND string, find can be told to only pass files which grep flags on to perl:
in="$FIND" out="$REPLACEWITH" find "$DIRECTORY" -type f -name '*.html' \
-exec grep -F -q -e "$FIND" '{}' ';' \
-exec perl -pi -e 's/\Q$ENV{"in"}/$ENV{"out"}/g' '{}' +
Because grep is being used to evaluate individual files, it's necessary to use one grep call per file so its exit status can be evaluated on a per-file basis; thus, the use of the less efficient -exec ... {} ';' action. For perl, it's possible to put multiple files to process on one command, hence the use of -exec ... {} +.
Note that fgrep is line-oriented; if your FIND string contains multiple lines, then files with any one of those lines will be passed to perl for replacements.
You can have find invoke sed directly although I think all the modification times on your files will be affected (which might matter or not):
find $DIRECTORY -type f -name "*.html" -exec sed -i "s|$FIND|$REPLACEWITH|g" '{}' ';'
I am using the following command to grep for string "/local" in all .pl files,can anyone point what is wrong here?
find . *.pl| xargs grep '/local' -sl
Pass -name argument, and quote *.pl:
find . -name "*.pl" | xargs grep '/local' -sl
Why is everyone suggesting "find"? The shell can work out your ".pl" files for you:
grep "/local" *.pl
You could just as easily
find . -type f -name '*.pl' -exec grep '/local/' {} \;
Or a more optimal form if your find supports it, this passes multiple files to grep at a time
find . -type f -name '*.pl' -exec grep '/local/' {} +
-exec tends to be slow.
I'm partial to:
find . -name "*.pl" -print0 | xargs -0 grep -sl '/local'
...because it isn't confused by filenames with newlines in them.
Note however that some versions of GNU grep appear to have a memory leak that is triggered by grep commands with a very long list of filenames to search. Under such circumstances, -exec is more reliable.
In bash I want to return file name (and the path to the file) for every file of type .php|.html|.js containing the case-insensitive string "document.cookie" | "setcookie"
How would I do that?
egrep -ir --include=*.{php,html,js} "(document.cookie|setcookie)" .
The r flag means to search recursively (search subdirectories). The i flag means case insensitive.
If you just want file names add the l (lowercase L) flag:
egrep -lir --include=*.{php,html,js} "(document.cookie|setcookie)" .
Try something like grep -r -n -i --include="*.html *.php *.js" searchstrinhere .
the -i makes it case insensitlve
the . at the end means you want to start from your current directory, this could be substituted with any directory.
the -r means do this recursively, right down the directory tree
the -n prints the line number for matches.
the --include lets you add file names, extensions. Wildcards accepted
For more info see: http://www.gnu.org/software/grep/
find them and grep for the string:
This will find all files of your 3 types in /starting/path and grep for the regular expression '(document\.cookie|setcookie)'. Split over 2 lines with the backslash just for readability...
find /starting/path -type f -name "*.php" -o -name "*.html" -o -name "*.js" | \
xargs egrep -i '(document\.cookie|setcookie)'
Sounds like a perfect job for grep or perhaps ack
Or this wonderful construction:
find . -type f \( -name *.php -o -name *.html -o -name *.js \) -exec grep "document.cookie\|setcookie" /dev/null {} \;
find . -type f -name '*php' -o -name '*js' -o -name '*html' |\
xargs grep -liE 'document\.cookie|setcookie'
Just to include one more alternative, you could also use this:
find "/starting/path" -type f -regextype posix-extended -regex "^.*\.(php|html|js)$" -exec grep -EH '(document\.cookie|setcookie)' {} \;
Where:
-regextype posix-extended tells find what kind of regex to expect
-regex "^.*\.(php|html|js)$" tells find the regex itself filenames must match
-exec grep -EH '(document\.cookie|setcookie)' {} \; tells find to run the command (with its options and arguments) specified between the -exec option and the \; for each file it finds, where {} represents where the file path goes in this command.
while
E option tells grep to use extended regex (to support the parentheses) and...
H option tells grep to print file paths before the matches.
And, given this, if you only want file paths, you may use:
find "/starting/path" -type f -regextype posix-extended -regex "^.*\.(php|html|js)$" -exec grep -EH '(document\.cookie|setcookie)' {} \; | sed -r 's/(^.*):.*$/\1/' | sort -u
Where
| [pipe] send the output of find to the next command after this (which is sed, then sort)
r option tells sed to use extended regex.
s/HI/BYE/ tells sed to replace every First occurrence (per line) of "HI" with "BYE" and...
s/(^.*):.*$/\1/ tells it to replace the regex (^.*):.*$ (meaning a group [stuff enclosed by ()] including everything [.* = one or more of any-character] from the beginning of the line [^] till' the first ':' followed by anything till' the end of line [$]) by the first group [\1] of the replaced regex.
u tells sort to remove duplicate entries (take sort -u as optional).
...FAR from being the most elegant way. As I said, my intention is to increase the range of possibilities (and also to give more complete explanations on some tools you could use).
I have a script that I'm running from the home directory to search for all files called "script.sh" that contain the string "watermelon". It's not finding anything but I can clearly see these scripts in the subdirectories. Could someone please suggest a change to the command I'm using:
find . -name script.sh | grep watermelon
You need to use xargs:
find . -name script.sh | xargs grep watermelon
xargs will modify the behavior to search within the files, rather than just search within the names of the files.
find returns the filename it finds by default. If you want it to search within the files then you need to pipe it to xargs or use the -exec and -print predicates:
find . -name script.sh -exec grep -q watermelon {} \; -print
use -type f to indicate file
find . -type f -name "script.sh" -exec grep "watermelon" "{}" +;
or if you have bash 4
shopt -s globstar
grep -Rl "watermelon" **/script.sh
I'm trying to construct a find command to process a bunch of files in a directory using two different executables. Unfortunately, -exec on find doesn't allow to use pipe or even \| because the shell interprets that character first.
Here is specifically what I'm trying to do (which doesn't work because pipe ends the find command):
find /path/to/jpgs -type f -exec jhead -v {} | grep 123 \; -print
Try this
find /path/to/jpgs -type f -exec sh -c 'jhead -v {} | grep 123' \; -print
Alternatively you could try to embed your exec statement inside a sh script and then do:
find -exec some_script {} \;
A slightly different approach would be to use xargs:
find /path/to/jpgs -type f -print0 | xargs -0 jhead -v | grep 123
which I always found a bit easier to understand and to adapt (the -print0 and -0 arguments are necessary to cope with filenames containing blanks)
This might (not tested) be more effective than using -exec because it will pipe the list of files to xargs and xargs makes sure that the jhead commandline does not get too long.
With -exec you can only run a single executable with some arguments, not arbitrary shell commands. To circumvent this, you can use sh -c '<shell command>'.
Do note that the use of -exec is quite inefficient. For each file that is found, the command has to be executed again. It would be more efficient if you can avoid this. (For example, by moving the grep outside the -exec or piping the results of find to xargs as suggested by Palmin.)
Using find command for this type of a task is maybe not the best alternative. I use the following command frequently to find files that contain the requested information:
for i in dist/*.jar; do echo ">> $i"; jar -tf "$i" | grep BeanException; done
As this outputs a list would you not :
find /path/to/jpgs -type f -exec jhead -v {} \; | grep 123
or
find /path/to/jpgs -type f -print -exec jhead -v {} \; | grep 123
Put your grep on the results of the find -exec.
There is kind of another way you can do it but it is also pretty ghetto.
Using the shell option extquote you can do something similar to this in order to make find exec stuff and then pipe it to sh.
root#ifrit findtest # find -type f -exec echo ls $"|" cat \;|sh
filename
root#ifrit findtest # find -type f -exec echo ls $"|" cat $"|" xargs cat\;|sh
h
I just figured I'd add that because at least the way i visualized it, it was closer to the OP's original question of using pipes within exec.