Hi friends,
I need to search for two patterns in a file. Ex. pattern1 and pattern2 and I want exactly that particular word. Based on the success or failure of this command, I'm going to have two actions using "IF".
example.
if (condition)
The condition is evaluating egrep command for searching two patterns in the file.
The condition should be true ($? = 0) only if both patterns are found in the file. else it should fail.
But the actual egrep command is doing OR function between patterns and I want AND logic between patterns. Please help me.
I guess you could just connect two greps in your shell script like so:
#!/bin/bash
if egrep -q "pattern1" filename && egrep -q "pattern2" filename; then
exit 0
else
exit 1
fi
The -q ensures that there is not output from egrep (it just returns true if the pattern was found), and the && ensures that exit 0 is only returned if both patterns were found in the file.
It's logically vague. If you want to check both patterns in file regardless of if they are in the same line or not, you could do
grep 'pattern1' somefile && grep 'pattren2' somefile
If you want both patterns to be present in the same line
grep 'pattern1' somefile | grep 'pattern2'
Hi the following worked out for me :
Requirement: searching two different patterns on different lines and take action based on AND opration of the two pattern search.
Code:
grep -q "pettern1" file1 && grep -q "pettern2" file1
if [ $? -ne 0 ]
then
echo "Both patterns are not found"
else
echo "Both patterns are found"
fi
NOTE : both pattens are on different lines in the same file
egrep -z -q 'pattern1.*pattern2|pattern2.*pattern1'
If both patterns are present in the file it will return true, otherwise false.
Related
I have a requirement to exclude a selected string. I am trying to use grep -v option.
Input:
AM2RGHK
AM2RGHK-JO
AM2RGHK-FN
Output should be:
AM2RGHK-JO
AM2RGHK-FN
From the input list, If I want to exclude only first line, I am using grep -v AM2RGHK
But I am not getting any output. grep -v excludes all the strings in the same sequence. Any clue?
grep is matching all the input lines because it's default behavior is to match line that contains the given pattern. It doesn't have to be exactly equal to it.
You can tell grep that it has to find an exact match by using the option -x (--line-regexp).
grep -v -x AM2RGHK does what you want.
Side notes:
Since you don't seem to use an actual regex but you just need simple text match, you may consider the option -F (--fixed-strings). It tells grep to not give special meaning to any character in the pattern.
Moreover, it's always a good practice to encase shell strings in ''. This ensures that the shell doesn't try to interpret any characters, like whitespaces. It can spare you a lot of headaches.
The resulting command would be:
grep -vxF 'AM2RGHK'
grep -v '^AM2RGHK$' input.txt
input.txt:
AM2RGHK
AM2RGHK-JO
AM2RGHK-FN
standard output:
AM2RGHK-JO
AM2RGHK-FN
Think about what you have control over, in this case you can compare against the string in question but instead of just the string we can add a pad character on each end and look for that.
#!/bin/bash
read_file="/tmp/list.txt"
exclude="AM2RGHK"
while IFS= read -r line ;do
if ! [[ "Z${line}Z" = "Z${exclude}Z" ]] ;then
echo "$line"
fi
done < "${read_file}"
This case we are saying if ZAM2RGHKZ != current then print it, the comparison would be as follows:
ZAM2RGHKZ = ZAM2RGHKZ do nothing
ZAM2RGHKZ != ZAM2RGHK-JOZ print because they don't match
ZAM2RGHKZ != ZAM2RGHK-FNZ print because they don't match
Hence the output becomes:
AM2RGHK-JO
AM2RGHK-FN
Note: there are more succinct ways to do this but this is a good way as well.
grep has the option -x
-x, --line-regexp
Select only those matches that exactly match the whole line. (-x is specified by POSIX.)
for example:
[dachnik#test]$ cat > greptest
AM2RGHK
AM2RGHK-JO
AM2RGHK-FN[dachnik#test]$ cat greptest
AM2RGHK
AM2RGHK-JO
AM2RGHK-FN[dachnik#test]$ grep -v -x AM2RGHK greptest
AM2RGHK-JO
AM2RGHK-FN
I would like to search the last 2 lines of a bunch of similarly named files for a given text pattern and write out the filenames where matches are NOT found.
I tried this:
tail -n 2 slurm-* | grep -L "run complete"
As you can see "slurm" is the file base, and I want to find files where "run complete" is absent. However, this command does not give me any output. I tried the regular (non-inverse) problem:
tail -n 2 slurm-* | grep -H "run complete"
and I get a bunch of output (all the matches that are found but not the filenames):
(standard input):run complete
I figure that I have misunderstood how piping tail output to grep works, any help is greatly appreciated.
This should work -
for file in `ls slurm-*`;
do
res=`tail -n2 $file | grep "run complete" 1>/dev/null 2>&1; echo $?`;
if [ $res -ne 0 ];
then
echo $file ;
fi ;
done ;
Explanation -
"echo $?" gives us the return code of the grep command. If grep finds the pattern in the file, it returns 0. Otherwise the return code is non-zero.
We check for this non-zero return code, and only then, "echo" the file name. Since you have not mentioned whether the output of grep is necessary, I have discarded the STD_OUT and STD_ERR by sending it to /dev/null.
How do I grep without actually filtering, or highlighting?
The goal is to find out if a certain text is in the output, without affecting the output. I could tee to a file and then inspect the file offline, but, if the output is large, that is a waste of time, because it processes the output only after the process is finished:
command | tee file
file=`mktemp`
if grep -q pattern "$file"; then
echo Pattern found.
fi
rm "$file"
I thought I could also use grep's before (-B) and after (-A) flags to achieve live processing, but that won't output anything if there are no matches.
# Won't even work - DON'T USE.
if command | grep -A 1000000 -B 1000000 pattern; then
echo Pattern found.
fi
Is there a better way to achieve this? Something like a "pretend you're grepping and set the exit code, but don't grep anything".
(Really, what I will be doing is to pipe stderr, since I'm looking for a certain error, so instead of command | ... I will use command 2> >(... >&2; result=${PIPESTATUS[*]}), which achieves the same, only it works on stderr.)
If all you want to do is set the exit code if a pattern is found, then this should do the trick:
awk -v rc=1 '/pattern/ { rc=0 } 1; END {exit rc}'
The -v rc=1 creates a variable inside the Awk program called rc (short for "return code") and initializes it to the value 1. The stanza /pattern/ { rc=0 } causes that variable to be set to 0 whenever a line is encountered that matches the regular expression pattern. The 1; is an always-true condition with no action attached, meaning the default action will be taken on every line; that default action is printing the line out, so this filter will copy its input to its output unchanged. Finally, the END {exit rc} runs when there is no more input left to process, and ensures that awk terminates with the value of the rc variable as its process exit status: 0 if a match was found, 1 otherwise.
The shell interprets exit code 0 as true and nonzero as false, so this command is suitable for use as the condition of a shell if or while statement, possibly at the end of a pipeline.
To allow output with search result you can use awk:
command | awk '/pattern/{print "Pattern found"} 1'
This will print "Pattern found" when pattern is matched in any line. (Line will be printed later)
If you want Line to print before then use:
command | awk '{print} /pattern/{print "Pattern found"}'
EDIT: To execute any command on match use:
command | awk '/pattern/{system("some_command")} 1'
EDIT 2: To take care of special characters in keyword use this:
command | awk -v search="abc*foo?bar" 'index($0, search) {system("some_command"); exit} 1'
Try this script. It will not modify anything of output of your-command and sed exit with 0 when pattern is found, 1 otherwise. I think its what you want from my understand of your question and comment.:
if your-command | sed -nr -e '/pattern/h;p' -e '${x;/^.+$/ q0;/^.+$/ !q1}'; then
echo Pattern found.
fi
Below is some test case:
ubuntu-user:~$ if echo patt | sed -nr -e '/pattern/h;p' -e '${x;/^.+$/ q0;/^.+$/ !q1}'; then echo Pattern found.; fi
patt
ubuntu-user:~$ if echo pattern | sed -nr -e '/pattern/h;p' -e '${x;/^.+$/ q0;/^.+$/ !q1}'; then echo Pattern found.; fi
pattern
Pattern found.
Note previous script fails to work when there is no ouput from your-command because then sed will not run sed expression and exit with 0 all the time.
I take it you want to print out each line of your output, but at the same time, track whether or not a particular pattern is found. Simply passing the output to sed or grep would affect the output. You need to do something like this:
pattern=0
command | while read line
do
echo "$line"
if grep -q "$pattern" <<< "$lines"
then
((pattern+=1))
fi
done
if [[ $pattern -gt 0 ]]
then
echo "Pattern was found $pattern times in the output"
else
echo "Didn't find the pattern at all"
fi
ADDENDUM
If the original command has both stdout and stderr output, which come in a specific order, with the two possibly interleaved, then will your solution ensure that the outputs are interleaved as they normally would?
Okay, I think I understand what you're talking about. You want both STDERR and STDOUT to be grepped for this pattern.
STDERR and STDOUT are two different things. They both appear on the terminal window because that's where you put them. The pipe (|) only takes STDOUT. STDERR is left alone. In the above, only the output of STDOUT would be used. If you want both STDOUT and STDERR, you have to redirect STDERR into STDOUT:
pattern=0
command 2>&1 | while read line
do
echo "$line"
if grep -q "$pattern" <<< "$lines"
then
((pattern+=1))
fi
done
if [[ $pattern -gt 0 ]]
then
echo "Pattern was found $pattern times in the output"
else
echo "Didn't find the pattern at all"
fi
Note the 2>&1. This says to take STDERR (which is File Descriptor 2) and redirect it into STDOUT (File Descriptor 1). Now, both will be piped into that while read loop.
The grep -q will prevent grep from printing out its output to STDOUT. It will print to STDERR, but that shouldn't be an issue in this case. Grep only prints out STDERR if it cannot open a file requested, or the pattern is missing.
You can do this:
echo "'search string' appeared $(command |& tee /dev/stderr | grep 'search string' | wc -l) times"
This will print the entire output of command followed by the line:
'search string' appeared xxx times
The trick is, that the tee command is not used to push a copy into a file, but to copy everything in stdout to stderr. The stderr stream is immediately displayed on the screen as it is not connected to the pipe, while the copy on stdout is gobbled up by the grep/wc combination.
Since error messages are usually emitted to stderr, and you said that you want to grep for error messages, the |& operator is used for the first pipe to combine the stderr of command into its stdout, and push both into the tee command.
I have a command that lists the full 8 level deep path of all folders we are backing up.
I also have a command that enumerates all 8 level deep folders on the system.
Both of these are stored as variables in a bash script.
I'm trying to get a loop together that takes file 1 and uses the first line entry as a variable in an if/then/else, and then moves onwards to through the end of the file.
I've tried so many things but its beyond my skillset to provide an example that won't confuse the reader of this post.
TempFile1=/ifs/data/scripts/ConfigMonitor/TempFile1.txt
TempFile2=/ifs/data/scripts/ConfigMonitor/TempFile2.txt
find /ifs/*/*/backup -maxdepth 4 -mindepth 4 -type d > $TempFile1
isi snapshot schedules list -v | grep Path: | awk '{print $2}' > $TempFile2
list line 1 on $TempFile1
Grep for line 1 within $TempFile2
if result yielded then
echo found
else
echo fullpath not being backed up
fi
Use Grep's -f Flag
grep(1) says:
-f FILE, --file=FILE
Obtain patterns from FILE, one per line. The empty file
contains zero patterns, and therefore matches nothing. (-f is
specified by POSIX.)
Therefore, the following should work:
grep -f patterns_to_match.txt file_to_examine.txt
Faster Reporting
Another way to think about this is that you can ask GNU grep to show you all the matches:
echo 'Lines that match a pattern in your pattern file.'
grep -f patterns_to_match.txt file_to_examine.txt
and then show you all the lines that don't match any of the patterns:
echo 'Lines that do not match any patterns in your pattern file.'
grep -f patterns_to_match.txt -v file_to_examine.txt
This is likely to be faster and more efficient than looping through the file one line at a time in Bash. You may or may not get similar results with a grep other than GNU grep; while the -f and -v flags are specified by POSIX, I only tested it against GNU grep 2.16, so your mileage may vary.
This should iterate through Tempfile1.txt and grep for the line in TempFile2.txt.
while read line; do
if grep $line /path/to/TempFile2.txt > /dev/null
then
echo "Found $line"
else
echo "Did not find $line"
fi
done < Tempfile1.txt
Tempfile1.txt:
a
b
c
Tempfile2.txt
b
d
z
Output:
Did not find a
Found b
Did not find c
This question already has answers here:
Find lines from a file which are not present in another file [duplicate]
(4 answers)
Closed 8 years ago.
I will fetch the file names from the file say: FILE_A, and will search these file names in another file say: File_B Using the script say: script.sh
I want to print those file names which are not present in a file say: FILE_B.
I use the code but it didn't work.
Code in the script->script.sh is as follows:
#!/bin/bash
while read line
do
grep -v "$line" FILE_B
done<FILE_A
please help me. why it is not working and what is the solution of it?
grep can read its input from a file; no need for a loop.
grep -Fxvf FILE_A FILE_B
The -F option specifies that the input is literal strings, not regular expressions. Otherwise an input which contains regex metacharacters would not match itself; or not only itself. For example, the regular expression a.c matches "aac", "abc", etc.
The -x option requires a full-line match. Otherwise, the input "bc" would match on any line containing it as a substring, such as "abcd".
The -v option says to print non-matching lines instead of matching.
Finally, the lowercase -f option specifies a file name as its argument to use as input for the patterns to match.
comm is good for this, but it requires the input files to be sorted. If that's not a problem:
# lines in FILE_A that are not in FILE_B
comm -23 <(sort FILE_A) <(sort FILE_B)
No extra linefeed between while and do
grep -v expr file will
print all lines of those files, not containing expr. What you want, is just the result whether it's found or not. You need to test the
exit state.
Try:
#!/bin/bash
while read line
do
grep -q "$line" FILE_B || echo "$line"
done<FILE_A
grep returns exit 0 if a line was found. The || concatenation with echo means: execute echo when exit state != 0- i.e. when $line was not found.
This script works but does not print what you want. For each filename in FILE_A it prints all the OTHER filenames in FILE_B. Instead you should print the filename yourself if grep does not find it:
while read line
do
grep "$line" FILE_B >/dev/null || echo "$line"
done <FILE_A
Use this instead
#!/bin/bash
while read line
do
if grep -qw $line "file_B"
then
echo $line
fi
done < file_A