I have a script that I'm running from the home directory to search for all files called "script.sh" that contain the string "watermelon". It's not finding anything but I can clearly see these scripts in the subdirectories. Could someone please suggest a change to the command I'm using:
find . -name script.sh | grep watermelon
You need to use xargs:
find . -name script.sh | xargs grep watermelon
xargs will modify the behavior to search within the files, rather than just search within the names of the files.
find returns the filename it finds by default. If you want it to search within the files then you need to pipe it to xargs or use the -exec and -print predicates:
find . -name script.sh -exec grep -q watermelon {} \; -print
use -type f to indicate file
find . -type f -name "script.sh" -exec grep "watermelon" "{}" +;
or if you have bash 4
shopt -s globstar
grep -Rl "watermelon" **/script.sh
Related
I want to create shell script that search inside all folders of the actual directory and return all files that satisfy some condition, but without using any print flag.
(Here the condition is to end with .py)
What I have done:
find . -name '*.py'| sed -n 's/\.py$//p'
The output:
./123
./test
./abc/dfe/test3
./testing
./test2
What I would like to achieve:
123
test
test3
testing
test2
Use -exec:
find . -name '*.py' -exec sh -c 'for f; do f=${f%.py}; echo "${f##*/}"; done' sh {} +
If GNU basename is an option, you can simplify this to
find . -name '*.py' -exec basename -s .py {} +
POSIX basename is a little more expensive, as you'll have to call it on every file individually:
find . -name '*.py' -exec basename {} .py \;
Using GNU grep instead of sed:
find . -name '*.py' | grep -oP '[^/]+(?=\.py$)'
If portability is not a concern, this is a very readable option:
find . -name '*.py' | xargs basename -a
This is also differentiated from chepner's answer in that it retains the .py file ending in the output.
I'm not familiar with the -exec flag, and I'm sure his one-liners can be customized to do the same, but I couldn't do so off the top of my head.
Chepner's version achieves the same with the small modification:
find . -name '*.py' -exec basename {} \;
if you want the literal output from find and didn't intend to drop the file endings when you used dummy variables (123,test, etc.) in your question.
find shows entries relative to where you ask it to search, you can simply replace the . with a *:
find * -name '*.py'| sed -n 's/\.py$//p'
(Be aware that this skips top level hidden directories)
This might work for you (GNU parallel):
find . -name '*.py*' 2>/dev/null | parallel echo "{/.}"
scenario: There are multiple files in an folder, I'm trying to find specific set of files and if a given file has specific info then I need to grep the information.
Ex:
find /abc/test \( -type f -name 'tst*.txt' -mtime -1 \) -exec grep -Po '(?<type1).*(?=type1|(?<=type2).*(?=type2)' {} \;
I need to include if condition along with find -exec (if grep is true then print the above)
if grep -q 'case=1' <filename>; then
grep -Po '(?<type1).*(?=type1|(?<=type2).*(?=type2)'
fi
Thanks
You can use -exec in find as a condition -- the file matches if the command returns a successful exit code. So you can write:
find /abc/test -type f -name 'tst*.txt' -mtime -1 -exec grep -q 'case=1' {} \; -exec grep -Po '(?<type1).*(?=type1|(?<=type2).*(?=type2)' {} \;
Tests in find are evaluated left-to-right, so the second grep will only be executed if the first one was successful.
If your conditions are more complicated, you can put the whole shell code into a script, and execute the script with -exec. E.g. put this in myscript.sh:
#!/bin/sh
if grep -q 'case=1' "$1"; then
grep -Po '(?<type1).*(?=type1|(?<=type2).*(?=type2)' "$1";
fi
and then do:
find /abc/test -type f -name 'tst*.txt' -mtime -1 -exec ./myscript.sh {} \;
Since you're using PCRE option -P in grep you can combine both searches into one grep as well using lookahead:
find /abc/test -type f -name 'tst*.txt' -mtime -1 -exec grep -Po '(?=.*case=1).*\K((?<=type1).*(?=type1)|(?<=type2).*(?=type2))' {} +
btw the regex shown in your question is invalid, that I've tried to correct it here.
I am using the following command to grep for string "/local" in all .pl files,can anyone point what is wrong here?
find . *.pl| xargs grep '/local' -sl
Pass -name argument, and quote *.pl:
find . -name "*.pl" | xargs grep '/local' -sl
Why is everyone suggesting "find"? The shell can work out your ".pl" files for you:
grep "/local" *.pl
You could just as easily
find . -type f -name '*.pl' -exec grep '/local/' {} \;
Or a more optimal form if your find supports it, this passes multiple files to grep at a time
find . -type f -name '*.pl' -exec grep '/local/' {} +
-exec tends to be slow.
I'm partial to:
find . -name "*.pl" -print0 | xargs -0 grep -sl '/local'
...because it isn't confused by filenames with newlines in them.
Note however that some versions of GNU grep appear to have a memory leak that is triggered by grep commands with a very long list of filenames to search. Under such circumstances, -exec is more reliable.
What im trying to do is something along the lines of(this is pseudocode):
for txt in $(some fancy command ./*.txt); do
some command here $txt
You can use find:
find /path -type f -name "*.txt" | while read txt; do
echo "$txt"; # Do something else
done
Use the -exec option to find:
find /usr/share/wordlists/*/* -type f -name '*.txt' -exec yourScript {} \;
Try
find . | grep ".txt" | xargs -I script.sh {}
find returns all files in the directory. grep selects only .txt files and xargs sends the file as Parameter to script.sh
I've been using the following command to grep for a string in all the python source files in and below my current directory:
find . -name '*.py' -exec grep -nHr <string> {} \;
I'd like to simplify things so that I can just type something like
findpy <string>
And get the exact same result. Aliases don't seem sufficient since they only do a string expansion, and the argument I need to specify is not the last argument. It sounds like functions are suitable for the task, so I have several questions:
How do I write it?
Where do I put it?
If you don't want to create an entire script for this, you can do it with just a shell function:
findpy() { find . -name '*.py' -exec grep -nHr "$1" {} \; ; }
...but then you may have to define it in both ~/.bashrc and ~/.bash_profile, so it gets defined for both login and interactive shells (see the INVOCATION section of bash's man page).
All the "find ... -exec" solutions above are OK in the sense that they work, but they are horribly inefficient and will be extremely slow for large trees. The reason is that they launch a new process for every single *.py file. Instead, use xargs(1), and run grep only on files (not directories):
#! /bin/sh
find . -name \*.py -type f | xargs grep -nHr "$1"
For example:
$ time sh -c 'find . -name \*.cpp -type f -exec grep foo {} \; >/dev/null'
real 0m3.747s
$ time sh -c 'find . -name \*.cpp -type f | xargs grep foo >/dev/null'
real 0m0.278s
On a side note, you should take a look at Ack for what you are doing. It is designed as a replacement for Grep written in Perl. Filtering files based on the target language or ignoring .svn directories and the like.
Example (snippet from Trac source):
$ ack --python foo ./mysource
ticket/tests/wikisyntax.py
139:milestone:foo
144:<a class="missing milestone" href="/milestone/foo" rel="nofollow">milestone:foo</a>
ticket/tests/conversion.py
34: ticket['foo'] = 'This is a custom field'
ticket/query.py
239: count_sql = 'SELECT COUNT(*) FROM (' + sql + ') AS foo'
I wanted something similar, and the answer by Idelic reminded of one of the nice features of xargs: that it puts the command at the end. You see, my problem was that I wanted to write a shell alias that would "accept parameters" (really, that it would expand in such a way to allow me to pass parameter so grep).
Here's what I added to my bash_aliases:
alias findpy="find . -type f -name '*.py' | xargs grep"
This way, I could write findpy WORD or findpy -e REGEX or findpy -il WORD - the point being that could use any grep command-line option.
Put the following three lines in a file named findpy
#!/bin/bash
find . -name '*.py' -exec grep -nHr $1 {} \;
Then say
chmod u+x findpy
I normally have a directory called bin in my home directory where I put little shell scripts like this. Make sure to add the directory to your PATH.
The script:
#!/bin/bash
find . -name '*.py' -exec grep -nHr "$1" {} ';'
is how I'd do it.
You write it with an editor like vim and put it somewhere on your path. My normal approach is to have a ~/bin directory and make sure my .profile file (or equivalent) contains:
PATH=$PATH:~/bin
Many versions of grep have options to do recursion, specify filename pattern, etc.
grep --perl-regexp --recursive --include='*.py' --regexp="$1" .
This recurses starting from the current directory (.), looks only at files ending in 'py', uses Perl-style regular expressions.
If your version of grep doesn't support --recursive and --include, then you can still use find and xargs, but be sure to allow for pathnames with embedded spaces by using the -print0 argument to find and the --null option to xargs to handle that.
find . -type f -name '*.py' -print0 | xargs --null grep "$1"
should work.
Add the following line to your ~/.bashrc or ~/.bash_profile or ~/.profile
alias findpy='find . -type f -name "*.py" -print0 | xargs -0 grep'
then you can use it like this
findpy def
or with grep options
findpy -i class
the following alias will ignore the version control meta-directory of git and svn
alias findpy='find . -type f -not -path "*/.git/*" -a -not -path "*/.svn/*" -name "*.py" -print0 | xargs -0 grep'
#######################################################################################
#
# Function to search all files (including sub-directories) that match a given file
# extension ($2) looking for an indicated string ($1) - in a case insensitive manner.
#
# For Example:
#
# -> findfile AllowNegativePayments cpp
#
#
#######################################################################################
findfile ()
{
find . -iname "*.$2*" -type f -print0 | xargs -0 grep -i "$1" {} \; 2> /dev/nul
}
alias _ff='findfile'