In a bash script, I have a list of lines in a file I wish to grep and then display on standard out, which is easiest done with a while read:
grep "regex" "filepath" | while read line; do
printf "$line\n"
done
However, I would like to inform the user if no lines were matched by the grep. I know that one can do this by updating a variable inside the loop but it seems like a much more elegant approach (if possible) would be to try to read a line in an until loop, and if there were no output, an error message could be displayed.
This was my first attempt:
grep "regex" "filepath" | until [[ -z ${read line} ]]; do
if [[ -z $input ]]; then
printf "No matches found\n"
break
fi
printf "$line\n"
done
But in this instance the read command is malformed, and I wasn't sure of another way the phrase the query. Is this approach possible, and if not, is there a more suitable solution to the problem?
You don't need a loop at all if you simply want to display a message when there's no match. Instead you can use grep's return code. A simple if statement will suffice:
if ! grep "regex" "filepath"; then
echo "no match" >&2
fi
This will display the results of grep matches (since that's grep's default behavior), and will display the error message if it doesn't.
A popular alternative to if ! is to use the || operator. foo || bar can be read as "do foo or else do bar", or "if not foo then bar".
grep "regex" "filepath" || echo "no match" >&2
John Kugelman's answer is the correct and succinct one and you should accept it. I am addressing your question about syntax here just for completeness.
You cannot use ${read line} to execute read -- the brace syntax actually means (vaguely) that you want the value of a variable whose name contains a space. Perhaps you were shooting for $(read line) but really, the proper way to write your until loop would be more along the lines of
grep "regex" "filepath" | until read line; [[ -z "$line" ]]; do
... but of course, when there is no output, the pipeline will receive no lines, so while and until are both wrong here.
It is worth amphasizing that the reason you need a separate do is that you can have multiple commands in there. Even something like
while output=$(grep "regex filepath"); echo "grep done, please wait ...";
count=$(echo "$output" | wc -l); [[ $count -gt 0 ]]
do ...
although again, that is much more arcane than you would ever really need. (And in this particular case, you would want probably actually want if , not while.)
As others already noted, there is no reason to use a loop like that here, but I wanted to sort out the question about how to write a loop like this for whenever you actually do want one.
As mentioned by #jordanm, there is no need for a loop in the use case you mentioned.
output=$(grep "regex" "file")
if [[ -n $output ]]; then
echo "$output"
else
echo "Sorry, no results..."
fi
If you need to iterate over the results for processing (rather than just displaying to stdout) then you can do something like this:
output=$(grep "regex" "file")
if [[ -n $output ]]; then
while IFS= read -r line; do
# do something with $line
done <<< "$output"
else
echo "Sorry, no results..."
fi
This method avoids using a pipeline or subshell so that any variable assignments made within the loop will be available to the rest of the script.
Also, i'm not sure if this relates to what you are trying to do at all, but grep does have the ability to load patterns from a file (one per line). It is invoked as follows:
grep search_target -f pattern_file.txt
Related
The script found error but it always goes to Else condition "No Found Error". Am I missing how to compare two variables?
ERROR="Error:"
for i in `find /logs -mtime -1`
do
CHECK=`cat $i |grep -i "Error"|cut -f 1 -d " "`
if [ "$CHECK" == $ERROR ]
then
echo "Found Error"
else
echo "Not Found Error"
fi
done
Did you tried something like if [[ "$CHECK" == $ERROR ]] ?
To simply detect error without printing the error message, you can use
CHECK=$(cat $i | grep "Error" | wc -l)
if [[ $CHECK -ne 0 ]]
then
echo "Found error"
else
echo "Not found error"
fi
You are using grep -i for case-insensitive matching, but then testing the result for exact equality with the string Error:. If the case-insensitive matching is important then the exact equality test is not an appropriate complement.
You are also capturing a potentially multi-line output and comparing it to a string that can be the result only of a single-line output.
And you are matching "Error:" anywhere on the line, but assuming that it will appear at the beginning.
Overall, you are going about this a very convoluted way, as grep tells you via its exit status whether it found any matches. For example:
#!/bin/bash
for log in `find /logs -mtime -1`; do
if grep -i -q '^Error:' "$log"; then
echo "Found Error"
else
echo "Not Found Error"
fi
done
There is two things that I would advise and may fix your issue:
Add #!/bin/bash on the first line, to make sure it is interpreted as bash and not sh. Many time I had trouble with comparison because of this
When comparing two variables, uses double brackets ([[ and ]]) Also, if it strings, always put quotes "$ERROR" around it. It's missing for the $ERROR variable.
Look at the other answers also, there are many ways to do the same thing in a much simpler way.
Note: When comparing numbers you should use -eq
I have a shell script and require your expertise on this.
SearchAirline() {
echo "Enter Airline Name:"
read airlineName
if [ $? -eq 0 ];then
echo -e "\t\t\E[43;31;1mFlight Information\E[0m"
echo -e "Departure Time Flight Airlines Vacancy"
echo "__________________________________________________________________________"
#cat flightlist.txt | grep $airlineName flightlist.txt
old_IFS=$IFS
IFS=$'\n'
for LINE in `sed -e '$airlineName' flightlist.txt`
do
print_flight $LINE
done
IFS=$old_IFS
fi
}
It does not work to give me the filtered list. Instead, it prints the entire list.
Change the '$airlineName' to "$airlineName". Variables aren't interpolated when they appear in single quotes.
Change the sed expression to only print lines that match:
sed -n "/$airlineName/p"
Edit: Other answers suggest using other tools such as grep, and they might be right. The only reason my answer relates to sed is that your question asks for it specifically. I'm assuming you're expecting to do more significant processing with sed than what you've described in your question.
Add the -n option to your sed invocation:
-n
Suppress the default output (in which each line, after it is examined for editing, is written to standard output). Only lines explicitly selected for output are written.
Edit: You also have to use the correct quotes around $airlineName. Single quotes disable variable substitution. I credit Martin Ellis for this since I didn't notice it the first time.
BTW - I would highly recommend using awk for this sort of report. It can handle the formatting and selection and it will be a lot faster if you have a large data set.
As #D.Shawley pointed out, this is REALLY a job for awk but that would mean rewriting the print_flight function too so here's a fixed shell script given some assumptions about your input file:
SearchAirline() {
echo "Enter Airline Name:"
read airlineName
if [ $? -eq 0 ];then
echo -e "\t\t\E[43;31;1mFlight Information\E[0m"
echo -e "Departure Time Flight Airlines Vacancy"
echo "__________________________________________________________________________"
grep "$airlineName" flightlist.txt |
while IFS= read -r line
do
print_flight "$line"
done
fi
}
I strongly recommend you rewrite your script in awk though. If you'd like help with that, post another question and show us what print_flight looks like.
The command:
value=${value%?}
will remove the last character from a variable.
Is there any logical reason why it would not work from within a script?
In my script it has no effect whatsoever.
if [[ $line =~ "What I want" ]]
then
if [[ $CURRENT -eq 3 ]]
then
echo "line is " $line
value=`echo "$line" | awk '{print $4}'`
echo "value = "$value
value=${value%?}
echo "value = $value "
break
fi
fi
I cant post the whole script, but this is the piece I refer to. The loop is being entered properly, but the 2 echo $value lines return the same thing.
Edit - this question still stands. The code works fine line bu line in a terminal, but all together in a script it fails.
Echo adds an extra line character to $value in this line:
value=`echo "$line" | awk '{print $4}'`
And afaik that extra char is removed with %?, so it seems it does not change anything at all.
Try echo -n instead, which does not add \n to the string:
value=`echo -n "$line" | awk '{print $4}'`
Since you have provided only the relevant part in the code and not the whole file, I'm going to assume that the first line of your file reads `#!/bin/sh'. This is your problem. What you are trying to do (parameter expansion) is specific to bash, so unless /bin/sh points to bash via a symlink, then you are running the script in a shell which does not understand bash parameter expansion.
To see what /bin/sh really is you can do: ls -l /bin/sh. And to remedy the situation, force the script to run in bash by changing the `shebang' at the top to read `#!/bin/bash'
I'm running into an issue where my argument list for echo is too long and would like some ideas on how to get around this issue, or at least test for the condition so I can properly handle it, and it won't kill my script
for file in `cat filelist`; do
PROTOCOLS1=`egrep -i 'rsh|rsync|remsh' "$file" | egrep -v '^[ | ]*#'`
FIELDS=`echo $PROTOCOLS1|wc -l`
if [[ $FIELDS -gt 1024 ]]; then
echo $file >> $debuglog
else
set -A myarray $PROTOCOLS1
do stuff.....
fi
done
So the problem is that when my arg list for echo is too long, $FIELDS is set to null, and thus my test for $FIELDS -gt 1024 always is true and does not get caught.
Problem is when it goes to the array it's obviously too big and I get a subscript out of range error and my script exits.
Any ideas are greatly appreciated.
Edit 9/18
OK so the problem is a little more basic.
myserver-v1> echo $variable
myserver-v1> /usr/bin/echo: too many args
I want to test for this in my script
I tried the following, which works, but I get all this crap to stdout, which fills up my debug log and is annoying
echo $variable
if [[ $? -ne 0 ]]; then
write to error log
fi
Is there a way to test echo $variable....without sending it to stdout?
I tried the following, but neither seemed to work, so I am kind of at a loss here.
[[ ! `echo $variable ]]
[[ `echo $variable ]]
If you keep the unquoted variable $PROTOCOLS1 in the echo, you could simplify life by replacing:
FIELDS=`echo $PROTOCOLS1|wc -l`
with
FIELDS=1
This is because when you echo $PROTOCOLS1 without any quotes around it, you will only have one (possibly very long) line of output. Alternatively, you can use:
FIELDS=$(echo "$PROTOCOLS1" | wc -l)
where the double quotes will preserve the newlines in the value of PROTOCOLS1 (but it gets you back to the 'argument list too long' problem).
So, you need to think about using:
FIELDS=$(egrep -i 'rsh|rsync|remsh' "$file" | egrep -c -v '^[ | ]*#')
which gets the second egrep to do the line counting for you. Obviously, since the later portion of the script uses $PROTOCOLS1, you will need to re-evaluate the egreps to get the data, but you should think about whether your processing scheme is appropriate. If you are running into a string value that is too long, you are probably not doing the processing in the best way. What the alternatives are depends on what you are trying to do, and the question does not reveal that. It might be appropriate to do the extra processing with a scripting language such as Perl, Python or Ruby.
I'm trying what seems like a very simple task: use bash to search a file for strings, and if they exist, output those to another file. It could be jetlag, but this should work:
#!/bin/bash
cnty=CNTRY
for line in $(cat wheatvrice.csv); do
if [[ $line = *$cnty* ]]
then
echo $line >> wr_imp.csv
fi
done
I also tried this for completeness:
#!/bin/bash
cnty=CNTRY
for line in $(cat wheatvrice.csv); do
case $line in
*"$cnty"*) echo $line >> wr_imp.csv;;
*) echo "no";;
esac
done
both output everything, regardless of whether the line contains CNTRY or not, and I'm copy/pasting from seemingly reliable sources, so apparently there's something simple about bash-ness that I'm missing?
Don't use bash, use grep.
grep -F "$cnty" wheatvrice.csv >> wr_imp.csv
While I would suggest to simply use grep too, the question is open, why you approach didn't work. Here a self referential modification of your second approach - with keyword 'bash' to match itself:
#!/bin/bash
cnty=bash
while read -r line
do
case $line in
*${cnty}*)
echo $line " yes" >> bashgrep.log
;;
*)
echo "no"
;;
esac
done < bashgrep.sh
The keypoint is while read -r line ... < FILE. Your command with cat involves String splitting, so every single word is processed in the loop, not every line.
The same problem in example 1.