I have a makefile which does something like this:
define finder
find /usr/include /usr/local/include -name '*.h' \
| xargs grep -in stupid
endef
all:
find:
${finder}
xfind:
find /usr/include /usr/local/include -name '*.h' \
| xargs grep -in stupid
The list of directories is actually much longer, which is why I'd like to break the line when it gets executed. xfind does exactly what I want:
$ make xfind
find /usr/include /usr/local/include -name '*.h' \
| xargs grep -in stupid
However, I'd like to use a canned recipe, to use it for different targets; like the finder used in the find target. Unfortunately, finder is a variable and not a recipe, so different rules apply to it. When the recipe gets executed, everything is in one line.
$ make find
find /usr/include /usr/local/include -name '*.h' | xargs grep -in stupid
Is there a way to embed a backslash-newline into a variable in a way that survives until it is used as a recipe?
Actually, it is possible to embed a backslash-newline into a variable. The tricky thing was the backslash (took me a while to find the answer).
# variable containing a newline
# there must be two blank lines between the define and endef
# (http://stackoverflow.com/a/17055840/2064196)
define nl
endef
# variable containing a backslash
# https://www.netbsd.org/docs/pkgsrc/makefile.html#makefile.variables
# backslash != echo "\\"
# the version below avoids $(shell), as suggested by bobbogo's comment
backslash := \$(strip)
Now the canned recipe can be written as
define find
find /usr/include /usr/local/include -name '*.h' ${backslash}${nl} \
| xargs grep -in stupid
endef
and the output is the desired
find /usr/include /usr/local/include -name '*.h' \
| xargs grep -in stupid
Related
I'm trying to recursively find c and header files in a script, while avoiding globbing out any that exist in the current directory.
FILE_MATCH_LIST='"*.c","*.cc","*.cpp","*.h","*.hh","*.hpp"'
FILE_MATCH_REGEX=$(echo "$FILE_MATCH_LIST" | sed 's/,/ -o -name /g')
FILE_MATCH_REGEX="-name $FILE_MATCH_REGEX"
This does exactly what I want it to:
+ FILE_MATCH_REGEX='-name "*.c" -o -name "*.cc" -o -name "*.cpp" -o -name "*.h" -o -name "*.hh" -o -name "*.hpp"'
Now, if I call find with that string (in quotes), it maintains the leading and trailing quotes and breaks find:
files=$(find $root_dir "$FILE_MATCH_REGEX" | grep -v $GREP_IGNORE_LIST)
+ find [directory] '-name "*.c" -o -name "*.cc" -o -name "*.cpp" -o -name "*.h" -o -name "*.hh" -o -name "*.hpp"'
This results in a "unknown predicate" error from find, because the entire predicate is single quoted.
If I drop the quotes from the variable in the find command, I get a strange behavior:
files=$(find $root_dir $FILE_MATCH_REGEX | grep -v $GREP_IGNORE_LIST)
+ find [directory] -name '"*.c"' -o -name '"*.cc"' -o -name '"*.cpp"' -o -name '"*.h"' -o -name '"*.hh"' -o -name '"*.hpp"'
Where are these single quotes coming from? They exist if I echo that variable as well, but they aren't there in the command when I'm actually setting the $FILE_MATCH_REGEX (As seen at the beginning of the question).
This of course also breaks find, because it's looking for the actual double quoted string, instead of expanding the *.h etc.
How do I get these strings into find without all of these quoting woes?
Fleshing out the array answer:
#!/bin/bash
patterns=( '*.c' '*.cc' '*.h' '*.hh' )
find_args=( "-name" "${patterns[0]}" )
for (( i=1 ; i < "${#patterns[#]}" ; i++ )) ; do
find_args+=( "-o" "-name" "${patterns[i]}" )
done
find [directory] "${find_args[#]}"
Also, to clear up the misconception around quotes, if you echo the last line the output might not be what you expect:
echo find [directory] "${find_args[#]}"
# outputs: find [directory] -name *.c -o -name *.cc -o -name *.h -o -name *.hh
Where are the quotes? Your shell removed them after it was done with them. Quotes are not find syntax, they are shell syntax that tell the shell how to interpret (or perhaps how NOT to interpret) your command line.
The reason for the strange behavior in your debug output is that the quotes in your data are literal quotes, not shell syntax quotes that get removed during command parsing. The debugger is just trying to point out the distinction.
Some useful resources on the Bash wiki:
BashParser explains how your command line gets parsed and executed
BashFAQ/050 explains why embedding quotes in your data isn't sufficient
If you have GNU find - adjust to your liking:
#!/bin/bash
#FILE_MATCH_LIST='"*.c","*.cc","*.cpp","*.h","*.hh","*.hpp"'
FILE_MATCH_LIST='.*/.*\.(c|cc|cpp|h|hh|hpp)'
find . -type f -regextype posix-egrep -regex "${FILE_MATCH_LIST}"
I am having a problems using find and grep together in msys on Windows. However, I also tried the same command on a Linux machine and it behaved the same. Notwithstanding, the syntax below is for windows in that the semicolon on the end of the command is not preceded by a backslash.
I am trying to write a find expression to find *.cpp and *.h files and pass the results to grep. If I run this alone, it successfully finds all the .cpp and .h files:
find . -name '*.cpp' -o -name '*.h'
But if I add in an exec grep expression like this:
find . -name '*.cpp' -o -name '*.h' -exec grep -l 'std::deque' {} ;
It only greps the .h files. If I switch the .h and .cpp order in the command, it only searches the .h. Essentially, it appears to only grep the last file extension in the expression. What do I need to do to grep both .h and .cpp??
Since you're using -o, you will need to use parentheses around it:
find . \( -name '*.cpp' -o -name '*.h' \) -exec grep -l 'std::deque' {} \;
Or.. you can ...
bash$> grep '/bin' `find . -name "*.pl" -o -name "*.sh"`
./a.sh:#!/bin/bash
./pop3.pl:#!/usr/bin/perl
./seek.pl:#!/usr/bin/perl -w
./move.sh:#!/bin/bash
bash$>
Above command greps 'bin' in ".sh" and ".pl" files. And it has found them !!
I have the following function defined in my .bashrc, but for some reason the --exclude-dir option is not excluding the .git directory. Can anyone see what I've done wrong? I'm using Ubuntu 13.10 if that helps.
function fif # find in files
{
pattern=${1?" Usage: fif <word_pattern> [files pattern]"};
files=${2:+"-iname \"$2\""};
grep "$pattern" --color -n -H -s $(find . $files -type f) --exclude-dir=.git --exclude="*.min.*"
return 0;
}
Make sure not to include a trailing slash when you specify the directory to exclude. For example:
Do this:
$ grep -r --exclude-dir=node_modules firebase .
NOT this:
$ grep -r --exclude-dir=node_modules/ firebase .
(This answer not applicable to OP, but may be helpful for others who find --exclude-dir not to be working -- it worked for me.)
Do a man grep on your system, and see what version you have. Your version of grep may not be able to use --exclude-dirs.
You're really better off using find to find the files you want, then use grep to parse them:
$ find . -name '.git' -type d -prune \
-o -name "*.min.*" -prune \
-o -type f -exec grep --color -n -H {} "$pattern" \;
I'm not a fan of the recursive grep. Its syntax has become bloated, and it's really unnecessary. We have a perfectly good tool for finding files that match a particular criteria, thank you.
In the find program, the -o separate out the various clauses. If a file has not been filtered out by a previous -prune clause, it is passed to the next one. Once you've pruned out all of the .git directories and all of the *.min.* files, you pass the results to the -exec clause that executes your grep command on that one file.
Some people prefer it this way:
$ find . -name '.git' -type d -prune \
-o -name "*.min.*" -prune \
-o -type f -print0 | xargs -0 grep --color -n -H "$pattern"
The -print0 prints out all of the found files separated by the NULL character. The xargs -0 will read in that list of files and pass them to the grep command. The -0 tells xargs that the file names are NULL separated and not whitespace separated. Some xargs will take --null instead of the -0 parameter.
I need to take an argument which is a directory of the current directory and search its folders and compile any C files in those folders. I'm just beginning shell scripting in Bash and am a little over my head.
So far things I've tried included using find to search for the files and then pipe it to xargs to compile but kept getting an error saying that testing.c wasn't a directory.
find ~/directory -name *.c | xargs gcc -o testing testing.c
I've also tried ls -R to search folders for .c files but don't know how to then take the paths as arguments to then move to and compile?
find ~/directory -type f -name "*.c" -print0 |
while IFS= read -r -d '' pathname; do
gcc -o "${pathname%.c}" "$pathname"
done
find directory -type f -name "*.c" -exec sh -c \
'cd $(dirname $1);make $(basename $1 .c)' sh {} \;
As #shx2 suggested, using make (or some other build system) would arguably be the best approach. You don't want to go compiling files in some source tree without a proper build system.
I currently use the following command, but it's a little unwieldy to type. What's a shorter alternative?
find . -name '*.txt' -exec grep 'sometext' '{}' \; -print
Here are my requirements:
limit to a file extension (I use SVN and don't want to be searching through all those .svn directories)
can default to the current directory, but it's nice to be able to specify a different directory
must be recursive
UPDATE: Here's my best solution so far:
grep -r 'sometext' * --include='*.txt'
UPDATE #2: After using grep for a bit, I realized that I like the output of my first method better. So, I followed the suggestions of several responders and simply made a shell script and now I call that with two parameters (extension and text to find).
grep has -r (recursive) and --include (to search only in files and directories matching a pattern).
If its too unweildy, write a script that does it and put it in your personal bin directory. I have a 'fif' script which searches source files for text, basically just doing a single find like you have here:
#!/bin/bash
set -f # disable pathname expansion
pattern="-iname *.[chsyl] -o -iname *.[ch]pp -o -iname *.hh -o -iname *.cc
-o -iname *.java -o -iname *.inl"
prune=""
moreargs=true
while $moreargs && [ $# -gt 0 ]; do
case $1 in
-h)
pattern="-iname *.h -o -iname *.hpp -o -iname *.hh"
shift
;;
-prune)
prune="-name $2 -prune -false -o $prune"
shift
shift
;;
*)
moreargs=false;
;;
esac
done
find . $prune $pattern | sed 's/ /\\ /g' | xargs grep "$#"
it started life as a single-line script and got features added over the years as I needed them.
This is much more efficient since it invokes grep many fewer times, though it's hard to say it's more succinct:
find . -name '*.txt' -print0 | xargs -0 grep 'sometext' /dev/null
Notes:
/find -print0 and xargs -0 makes pathnames with embedded blanks work correctly.
The /dev/null argument makes sure grep always prepends a filename.
Install ack and use
ack -aG'\.txt$' 'sometext'
I second ephemient's suggestion of ack. I'm writing this post to highlight a particular issue.
In response to jgormley (in the comments): ack is available as a single file which will work wherever the right Perl version is installed (which is everywhere).
Given that on non-Linux platforms grep regularly does not accept -R, arguably using ack is more portable.
I use zsh, which has recursive globbing. If you needed to look at specific filetypes, the following would be equivalent to your example:
grep 'sometext' **/*.txt
If you don't care about the filetype, the -r option will be better:
grep -r 'sometext' *
Although, A minor tweak to your original example will give you exactly what you want:
find . -name '*.txt' \! -wholename '*/.svn/*' -exec grep 'sometext' '{}' \; -print
If this is something you do frequently, make it a function (put this in your shell config):
function grep_no_svn {
find . -name "${2:-*}" \! -wholename '*/.svn/*' -exec grep "$1" '{}' \; -print
}
Where the first argument to the function is the text you're searching for. So:
$ grep_here_no_svn "sometext"
Or:
$ grep_here_no_svn "sometext" "*.txt"
You could write a script (in bash or whatever -- I have one in Groovy) and place it on the path. E.g.
$ myFind.sh txt targetString
where myFind.sh is:
find . -name "*.$1" -exec grep $2 {} \; -print
I usualy avoid the "man find" by using grep $(find . -name "*,txt")
You say that you like the output of your method (using find) better. The only difference I can see between them is that grepping multiple files will put the filename on the front.
You can always (in GNU grep, but you must be using that or -r and --include wouldn't work) turn the filename off by using -h (--no-filename). The opposite, for anyone who does want filenames but has to use find for some other reason, is -H (--with-filename).