I want to read a file word by word and i want to use each word in that text file as an input to grep.
to read the file word by word i have used the following code
for word in $(<filename)
do
echo "$word"
done
now when I replaced
echo "$word"
with
grep -i "$word"
I'm not getting any output.
The following will read the file word by word and apply grep using the read word as input:
#!/bin/bash
while read line; do
for word in $line; do
grep -i "<REGULAR_EXPRESSION_HERE>" "$word"
done
done < filename
The reason you are not getting any output is that grep expects two arguments. If you leave out the filename argument, it will wait for you to type in the text to grep from; it is reading standard input. (This is what allows you to use it in a pipeline, like command | grep error.)
Anyway, what you are attempting is already built into grep. Just pass it the file of search expressions as an argument to -f.
grep -irf filename .
where -r says to search recursively through all the files in a directory and . is the current directory.
Note, however, that this will search for matches anywhere on a line. If your input file contains dog then grep will find a match on lines which contain dogmatic or endogenous; and if it contains an empty line, it will match all lines in all files. Maybe look at the -w and/or -x options (as well as perhaps -F to disarm any regex specials in the input) to address these issues.
See if this serves your purpose:
$ grep -o "\S*" filename | grep -i "<your regex here>"
The first grep in the pipeline will flatten the file to one word per line. Then second grep will search those word for your regex.
Note: This answer assumes that the individual words in file are the data you want to grep in. If those are supposed to be interpreted as filenames, refer to higuaro's answer.
This is what worked for me
while read line
do
output=`grep -i "$line" /filepath/*`
if [ $? -eq 0 ]; then
echo "$line present in file : $output"
fi
done <filename
Related
I have a requirement to exclude a selected string. I am trying to use grep -v option.
Input:
AM2RGHK
AM2RGHK-JO
AM2RGHK-FN
Output should be:
AM2RGHK-JO
AM2RGHK-FN
From the input list, If I want to exclude only first line, I am using grep -v AM2RGHK
But I am not getting any output. grep -v excludes all the strings in the same sequence. Any clue?
grep is matching all the input lines because it's default behavior is to match line that contains the given pattern. It doesn't have to be exactly equal to it.
You can tell grep that it has to find an exact match by using the option -x (--line-regexp).
grep -v -x AM2RGHK does what you want.
Side notes:
Since you don't seem to use an actual regex but you just need simple text match, you may consider the option -F (--fixed-strings). It tells grep to not give special meaning to any character in the pattern.
Moreover, it's always a good practice to encase shell strings in ''. This ensures that the shell doesn't try to interpret any characters, like whitespaces. It can spare you a lot of headaches.
The resulting command would be:
grep -vxF 'AM2RGHK'
grep -v '^AM2RGHK$' input.txt
input.txt:
AM2RGHK
AM2RGHK-JO
AM2RGHK-FN
standard output:
AM2RGHK-JO
AM2RGHK-FN
Think about what you have control over, in this case you can compare against the string in question but instead of just the string we can add a pad character on each end and look for that.
#!/bin/bash
read_file="/tmp/list.txt"
exclude="AM2RGHK"
while IFS= read -r line ;do
if ! [[ "Z${line}Z" = "Z${exclude}Z" ]] ;then
echo "$line"
fi
done < "${read_file}"
This case we are saying if ZAM2RGHKZ != current then print it, the comparison would be as follows:
ZAM2RGHKZ = ZAM2RGHKZ do nothing
ZAM2RGHKZ != ZAM2RGHK-JOZ print because they don't match
ZAM2RGHKZ != ZAM2RGHK-FNZ print because they don't match
Hence the output becomes:
AM2RGHK-JO
AM2RGHK-FN
Note: there are more succinct ways to do this but this is a good way as well.
grep has the option -x
-x, --line-regexp
Select only those matches that exactly match the whole line. (-x is specified by POSIX.)
for example:
[dachnik#test]$ cat > greptest
AM2RGHK
AM2RGHK-JO
AM2RGHK-FN[dachnik#test]$ cat greptest
AM2RGHK
AM2RGHK-JO
AM2RGHK-FN[dachnik#test]$ grep -v -x AM2RGHK greptest
AM2RGHK-JO
AM2RGHK-FN
I would like to create a bash script that reads a file and then those lines in the file are used as a search string to grep my computer. Also, the strings are embedded within another search string. Normally, I would run the following:
grep -R -oE ".{0,5}findme\(.{0,40}"
which would look for the 5 characters before and 40 characters after the phrase findme(
The following code is what I was thinking but I get the error grep: parentheses not balanced
input="../list.txt"
while IFS= read -r line
do
grep -R -oE ".{0,5}{$line}/(.{0,40}"
echo "$line"
done < "$input"
Any help would be appreciated!
Thanks
I have a folder full of textfiles which i want to grep for a specific String, then add a String (let's say a "+ ") via sed at the beginning of that specific line (at all places where the search-String occurs).
So far i got this:
if grep -q &MyString /path/to/folder/* ; then
echo "String detected"
sed 's/^/+ /g' -i [what to put in here?]
fi
depending on what i put in the brackets it either adds the String in front of every line of every textfile or does nothing because i dont define a correct file. How would i reference the file i detected the String in before?
Thanks
grep -l &MyString /path/to/folder/* | uniq | while read line
do
echo "String detected"
sed '/&MyString/ s/^/+ /g' -i $line
done
Using grep with -l to print just the filename, use uniq to deduplicate and then read each line of the output. Taking each line from the output, use this to represent the filename in sed ($line).
Read line is used as opposed for var in $(grep ...) as this would cause issues for files with spaces in them as each space delimited variable in the output would be used to represent the filename.
I have a file that looks like this:
some random
text
00ab46f891c2emore random
text
234324fc234ba253069
and yet more text
only one line in the file contains only hex characters (234324fc234ba253069), how do I extract that? I tried sed -ne 's/^\([a-f0-9]*\)$/\1/p' file I used line start and line end (^ and &) as delimiters, but I am obviously missing something...
Grep does the job,
$ grep '^[a-f0-9]\+$' file
234324fc234ba253069
Through awk,
$ awk '/^[a-f0-9]+$/{print}' file
234324fc234ba253069
Based on the search pattern given, awk and grep prints the matched line.
^ # start
[a-f0-9]\+ # hex characters without capital A-F one or more times
$ # End
sed can make it:
sed -n '/^[a-f0-9]*$/p' file
234324fc234ba253069
By the way, your command sed -ne 's/^\([a-f0-9]*\)$/\1/p' file is working to me. Note, also, that it is not necessary to use \1 to print back. It is handy in many cases, but now it is too much because you want to print the whole line. Just sed -n '/pattern/p' does the job, as I indicate above.
As there is just one match in the whole file, you may want to exit once it is found (thanks NeronLeVelu!):
sed -n '/^[a-f0-9]*$/{p;q}' file
Another approach is to let printf decide when the line is hexadecimal:
while read line
do
printf "%f\n" "0x"$line >/dev/null 2>&1 && echo "$line"
done < file
Based on Hexadecimal To Decimal in Shell Script, printf "%f" 0xNUMBER executes successfully if the number is indeed hexadecimal. Otherwise, it returns an error.
Hence, using printf ... >/dev/null 2>&1 && echo "$line" does not let printf print anything (redirects to /dev/null) but then prints the line if it was hexadecimal.
For your given file, it returns:
$ while read line; do printf "%f\n" "0x"$line >/dev/null 2>&1 && echo "$line"; done < a
234324fc234ba253069
Using egrep you can restrict your regex to select lines that only match valid hex characters i.e. [a-fA-F0-9]:
egrep '^[a-fA-F0-9]+$' file
234324fc234ba253069
This question already has answers here:
Find lines from a file which are not present in another file [duplicate]
(4 answers)
Closed 8 years ago.
I will fetch the file names from the file say: FILE_A, and will search these file names in another file say: File_B Using the script say: script.sh
I want to print those file names which are not present in a file say: FILE_B.
I use the code but it didn't work.
Code in the script->script.sh is as follows:
#!/bin/bash
while read line
do
grep -v "$line" FILE_B
done<FILE_A
please help me. why it is not working and what is the solution of it?
grep can read its input from a file; no need for a loop.
grep -Fxvf FILE_A FILE_B
The -F option specifies that the input is literal strings, not regular expressions. Otherwise an input which contains regex metacharacters would not match itself; or not only itself. For example, the regular expression a.c matches "aac", "abc", etc.
The -x option requires a full-line match. Otherwise, the input "bc" would match on any line containing it as a substring, such as "abcd".
The -v option says to print non-matching lines instead of matching.
Finally, the lowercase -f option specifies a file name as its argument to use as input for the patterns to match.
comm is good for this, but it requires the input files to be sorted. If that's not a problem:
# lines in FILE_A that are not in FILE_B
comm -23 <(sort FILE_A) <(sort FILE_B)
No extra linefeed between while and do
grep -v expr file will
print all lines of those files, not containing expr. What you want, is just the result whether it's found or not. You need to test the
exit state.
Try:
#!/bin/bash
while read line
do
grep -q "$line" FILE_B || echo "$line"
done<FILE_A
grep returns exit 0 if a line was found. The || concatenation with echo means: execute echo when exit state != 0- i.e. when $line was not found.
This script works but does not print what you want. For each filename in FILE_A it prints all the OTHER filenames in FILE_B. Instead you should print the filename yourself if grep does not find it:
while read line
do
grep "$line" FILE_B >/dev/null || echo "$line"
done <FILE_A
Use this instead
#!/bin/bash
while read line
do
if grep -qw $line "file_B"
then
echo $line
fi
done < file_A