I am trying to report lines found using grep and while.
I know you can use the following to compare a list of strings from inputs.txt and find them in your target file like so:
grep -f inputs.txt file_to_check
What I want is to read each line of the inputted strings and grep them individual in a loop.
So I have tried the following methods:
cat inputs.txt | while read line; do if grep "$line" filename_to_check; then echo "found"; else echo "not found"; fi; done
This returns nothing when I redirect the output to a file.
while read line
do
if grep "$line" file_to_check
then echo "found"
else
echo "not found"
fi
done < inputs.txt
Same as the first one but from what I found is better to do.
I know it iterates line by line because I can replace grep with echo $line and it prints each line; but either method doesn't return anything like grep -f above, instead it shows:
not found
not found
not found
.
. etc.
So what I'm looking for is something where it will iterate through each line and check it via grep using an if statement to determine if grep has actually found it or not. I know I may not have all proper logic but the output for what I want should look something like:
Found *matching line in file_to_check*
Found *matching line in file_to_check*
Not Found $line *(string that does not match)*
.
. etc.
You can also use && and || operators :
while read line; do
grep -q "$line" file_to_check && echo "$line found in file_to_check" || echo "$line not found in file_to_check"
done < inputfile > result.txt
The -q parameter of the grep just outputs a status code :
if $line is found, it outpouts 0 (True) the command after && will be evaluated
if not found, it outputs 1 (False) the command after || will evaluated
You can rewrite your final solution into
# Do not need this thanks to tr: file=$(dos2unix inputs.txt)
# Use -r so a line with backslashes will be showed like you want
while read -r line
do
# Not empty? Check with test -n
if [ -n "$(grep "${line}" filename)" ]; then
echo "found: ${line}"
else
echo "not found: ${line}"
fi
done < <(tr -d "\r" < "${file}")
Well, your if statement is pretty free form, you might need to clean it up a bit for bash to be able to read it. For example:
if [ "$(grep "$line" file_to_check)" != "" ]; then
echo "found: $line"
else
echo "not found: $line"
fi
This if statement will evaluate true if the grep command finds the line, because if it does it will spit the line out and will not be equal to "", or an empty string.
Here's my final solution:
file=$(dos2unix inputs.txt)
new_file=$file
while read line
do
if [ "$(grep "$line" filename)" != "" ]
then echo "found: $line"
else echo "not found: $line"
fi
done <$new_file
Thanks again!
Related
I am trying to read a parameter file in a shell script and would want to skip the lines which start with "#". Have been trying it on Ubuntu VM (default bash) and for something that I can't understand, it doesn't seem to work.
Following is the pseudo-code that I am using:
while read line
do
if [ grep -q "#" <<< "$line" ]; then
## Do nothing (Commented Out)
echo "$line Line is Commented out"
elif [ "$line" = "" ]; then
## Do nothing (Blank Line)
echo "Blank line"
else
#echo "read line is $line"
...some logic here
fi
done <input_file.ini
This yields the the following error: Syntax error: redirection unexpected
The if [[ $line == *#* ]] construct doesn't seem to work. My earlier experience was on AIX where everything worked fine.
Could someone guide me what I am doing wrong here?
PS: On a related note, how do I handle cases where I don't want to do anything? e.g. when there is no '#' character in the read line, I don't want to do anything. I can't leave my if block blank so I am just using echo 'some random' text. My task works good but just wanted to understand what's a good practice to handle this.
Your code is clearly running with /bin/sh, not bash.
An alternative to [[ $line = *"#"* ]] that works with /bin/sh is case.
Thus, the following will work with /bin/sh, or when invoked with sh yourscript:
#!/bin/sh
while read -r line; do : line="$line"
case $line in
*"#"*) echo "Line is commented out: $line";;
"") echo "Line is empty" ;;
*) key=${line%%=*}
value=${line#*=}
eval "$key="'$line' # unsafe, but works with /bin/sh, which doesn't have better
# indirect assignment approaches.
printf '%s\t\t-\t\t%s\n' "$key" "$value"
;;
esac
done <input_file.ini
Alternately, consider putting in a guard to handle the case when your script is invoked with a non-bash shell:
#!/bin/bash
case $BASH_VERSION in
'')
echo "ERROR: Run with a non-bash shell" >&2
if [ "$tried_reexec" ]; then
echo "ERROR: Already attempted reexec and failed" >&2
exit 1
fi
if [ -s "$0" ]; then
export tried_reexec=1
exec bash "$0" "$#"
fi
;;
esac
while read -r line; do
if [[ $line = *"#"* ]]; then
echo "Line is Commented out: $line"
elif [[ "$line" = "" ]]; then
echo "Blank line"
else
key=${line%%=*}; value=${line#*=}
printf -v "$key" %s "$value"
printf '%s\t\t-\t\t%s\n' "$key" "$value"
fi
done <input_file.ini
I really wasn't able to figure out the exact problem with double [[ ]] and the character search in a string. Thanks to everyone who tried to help me out. However this was acting as a deterrent and I didn't want to continue to fiddle for too long, I used a slightly different approach to handle my situation.
The following code works for me now:
while read line
do
first_char=`echo $line | cut -c 1`
if [ "$first_char" = "#" ]; then
: "do nothing here. Line is commented out"
elif [ "$line" = "" ]; then
: "do nothing here. Blank line"
else
KEY="$(echo $line | cut -d '=' -f1)"
VALUE="$(echo $line | cut -d '=' -f2)"
printf \v "$KEY" %s "$VALUE"
echo "$KEY\t\t-\t\t$VALUE"
fi
done < ${SCHEDULER_LOC}/inputs/script_params.ini
Also I was able to learn few things so incorporated them as well. I did get few negative scores for this question. Understandably so since this might be rudimentary for the experts but it was a genuine problem I was seeking some guidance on. Still, I am thankful that I learnt something new. Kudos to the community.
When I try to match a string and do some conditions; it always fails to do so.
date=`date +%Y%m%d`
kol="/home/user/test_$date"
regex='Terminating the script'
if [ -f $kol ]; then
sudo tail -f $kol | while read line; do
if [[ $line = *"Terminating the"* ]]
then
echo "failed"
else
echo $line >> /home/user/test123_$date
fi
else
echo "File is not yet present"
exit 0
fi
I have also tried with regex and that to failed. So when ever I input the matching string into the file ($path) it wont output "failed"; Is there anything wrong in the code. Help is much appreciated.
You can try this. At least it works for me:
sudo tail -f $1 | while read line; do
if [[ $line = '' ]]
then
break
fi
if [[ $line = *"Terminating the"* ]]
then
echo "failed"
else
echo $line >> /home/nurzhan/test123_$date
fi
done < "$1"
$path is parameter to the running script. Also note that by default the command tail returns only 10 last lines.
This question already has answers here:
A variable modified inside a while loop is not remembered
(8 answers)
Closed 6 years ago.
I'm trying to loop through allURLs.txt and check if every entry in that file exists in PDFtoCheck.pdf. I know of a tool called pdfgrep, but can't seem to apply it to suit my objective.
#!/bin/bash
entriesMissing=0;
cat allURLs.txt | while read line
do
# do something with $line here
if [ ! -z echo `pdfgrep "$line" PDFtoCheck.pdf` ];
then
echo "yay $line";
else
echo "$line not found";
entriesMissing=$[$entriesMissing+1];
fi
done
echo "DONE";
echo "There are $entriesMissing entries missing!";
Despite placing dummy values in allURLs.txt, entires which are present in allURLs.txt but not in PDFtoCheck.pdf are not reflected in the output. Any idea how to make it work as intended?
Please note that a subshell is created when piping: cat file | while. You should use file redirection instead: while ... do; done < file.
As far as I can see pdfgrep supports the -q quiet flag, so you can just use it in the if-statement.
entriesMissing=0
while IFS= read -r line; do
if pdfgrep -q -- "$line" PDFtoCheck.pdf; then
printf "Found '%s'\n" "$line"
else
printf "'%s' not found\n" "$line"
((entriesMissing++))
fi
done < allURLs.txt
printf "There are %d entries missing\n" "%entriesMissing"
I also changed the increment to ((... ++))
Extending my comment as answer. I'm using -c option which is also available in pdfgrep :
entriesMissing=0
while read line
do
# do something with $line here
if [ $(grep -c "$line" b) -eq 0 ]
then
((entriesMissing++))
echo "$line not found"
else
echo "yay $line"
fi
done < allURLs.txt
echo "DONE"
echo "There are $entriesMissing entries missing!";
One thing I want point out in your code that you are incrementing entriesMissing inside a subshell(pipe) which doesn't get reflected at the last line. Hope it helps.
I am trying to create an if statement that performs an action when it reads a blank line.
I would assume it would be something like this : if ($line=='\n');then
where line is the line that it is reading from a text file. But this is not working.
while read line; do
if [ "$line" = "" ]; then
echo BLANK
fi
done < filename.txt
or a slight variation:
while read line; do
if [ "$line" ]; then
echo NOT BLANK
else
echo BLANK
fi
done < filename.txt
try this:
if [[ "x$line" == "x" ]]; then...
or
if [[ "$line" =~ "^$" ]]; ...
Or also:
grep -q '.' <<< $line
Returns 1 if line is empty, 0 if non-empty
I have a shell script like this:
cat file | while read line
do
# run some commands using $line
done
Now I need to check if the line contains any non-whitespace character ([\n\t ]), and if not, skip it.
How can I do this?
Since read reads whitespace-delimited fields by default, a line containing only whitespace should result in the empty string being assigned to the variable, so you should be able to skip empty lines with just:
[ -z "$line" ] && continue
try this
while read line;
do
if [ "$line" != "" ]; then
# Do something here
fi
done < $SOURCE_FILE
bash:
if [[ ! $line =~ [^[:space:]] ]] ; then
continue
fi
And use done < file instead of cat file | while, unless you know why you'd use the latter.
cat i useless in this case if you are using while read loop. I am not sure if you meant you want to skip lines that is empty or if you want to skip lines that also contain at least a white space.
i=0
while read -r line
do
((i++)) # or $(echo $i+1|bc) with sh
case "$line" in
"") echo "blank line at line: $i ";;
*" "*) echo "line with blanks at $i";;
*[[:blank:]]*) echo "line with blanks at $i";;
esac
done <"file"
if ! grep -q '[^[:space:]]' ; then
continue
fi
blank=`tail -1 <file-location>`
if [ -z "$blank" ]
then
echo "end of the line is the blank line"
else
echo "their is something in last line"
fi
awk 'NF' file | while read line
do
# run some commands using $line
done
stole this answer to a similar question:
Delete empty lines using sed