Bash: Replace part of string with wildcard in search pattern - bash

I have working script that looks for errors in log files :
CONFIGFILE=testfmc.ini
log=log1.log
res=0
while read line ;
do
echo "$line"
if [[ "$line" == PAT[0-9][0-9]* ]] ; then
echo "line --$line--"
patnum=$(echo "$line" | awk -F= '{print $1}');
pattern=$(echo "$line" | awk -F= '{print $2}');
echo "$patnum --- $pattern";
res=$(fgrep -c "$pattern" $log );
if [[ "$res" -gt 0 ]] ; then
echo "Error in log files $res";
fi
echo "End of IF";
fi
echo $res;
done < $CONFIGFILE
testfmc.ini is the file with the patterns :
PAT01=java.net.SocketException: Connection reset
PAT02=javax.mail.StoreClosedException: failed to create new store connection
PAT03=javax.mail.MessagingException: ServerName NO Server Unavailable
PAT04=javax.mail.FolderClosedException: * BYE Connection is closed
The problem is that in PAT03 there is a Server name and this error may show up with several Server names. How can i replace the ServerName in the pattern with WILDCARD. I have tried with * or * or "*" and several more options , but nothing works.

The desired operation (replacing part of a string with a wildcard) is trivially done with a parameter expansion:
s='PAT03=javax.mail.MessagingException: ServerName NO Server Unavailable'
s2="${s//ServerName/'*'}"
echo "$s2"
(There are other issues with your script as a whole -- fgrep matches literal strings, not regular expressions or glob-style patterns, and the other grep variants use regexes while your patterns here are glob-style -- but if your intended question's scope expands beyond what's in the title, you should be asking a new question directly on point rather than slipping what you actually want to know into the details of a different question entirely).

in ini file replace:
PAT03=javax.mail.MessagingException: ServerName NO Server Unavailable
with
PAT03=javax.mail.MessagingException: .* NO Server Unavailable
in script replace:
res=$(fgrep -c "$pattern" $log );
with:
res=$(grep -E "$pattern" $log );
the script is not well optimized but works so :) good luck

I don't think you can use fgrep to do that. I thought fgrep only looks for strings, not patterns.
Perhaps use normal grep or egrep, and put the pattern as .* rather than *.
PS #Charles suggestion regarding IFS is great.

Related

bash script with grep command understanding

I posted earlier with something similar to this. I am trying to check if a user is online with the command $ ./user mburkhar which prints out mburkhar is logged on. My program works correctly but if I just type $ ./user mb is also states mb is logged on. What I have is fine, but is there a way to match what the user typed in exactly instead of slightly matching the first 2 characters..?
Here is my program so you can see what I did:
# Check if a user is logged on
if [ -z $1 ] ; then
read user
else
user=$1
fi
if [ `who | cut -d" " -f1 | grep $1` ] ; then
echo "$1 is logged on"
else
echo "$1 is either not valid or logged on"
fi
To complement the accepted answer:
-w works well and is widely supported (GNU Grep, BSD Grep), but it is not POSIX-compliant.
In your case, given that your output lines contain just a username each and nothing else, using -x - to match entire lines - would make sense too (and it is POSIX-compliant).
Also, since you're searching for a literal username, it's good practice to use grep's -F option to indicate just that.
Using [ $(...) ] (or [ `...` ]) to test for nonempty output from a command is somewhat fragile and inefficient; it's better and simpler to:
use commands directly
base the test on the exit code
and suppress stdout output, if needed
grep's -q option not only suppresses stdout, but also makes the search potentially more efficient by terminating once the first match is found (with exit code 0 to indicate success):
if who | cut -d' ' -f1 | grep -Fxq "$1"; then # ...
Similarly, [ -z $1 ] is fragile in that would break if an argument with embedded whitespace is passed - not likely in this case, but it's better to get in the habit of using [[ -z $1 ]] (or, if you must remain POSIX-compliant, [ -z "$1" ]).
Outside of [[ ... ]], it makes sense to habitually double-quote variable references, such as the $1 in the grep command.
If we put it all together:
# Check if a user is logged on
if [[ -z $1 ]]; then
read user
else
user=$1
fi
if who | cut -d' ' -f1 | grep -Fxq "$1"; then
echo "$1 is logged on"
else
echo "$1 is either not valid or logged on"
fi
You can use grep's -w (--word-regexp) option to match only entire words. Replacing grep $1 with grep -w $1 in your script should fix it.
Specify the start and end of string anchors like this:
if [ -z $1 ] ; then
read user
else
user="^$1$"
fi
if [ $(who | cut -d" " -f1 | grep $user | uniq ) ] ; then
echo "$1 is logged on"
else
echo "$1 is either not valid or logged on"
fi
I also added the uniq to remove duplicate matches, in case an user got more than one tty, or [ will exit with: [: too many arguments

Script to call either from file or user input

I'm trying to write a small script that either takes input from a file or from user, then it gets rid of any blank lines from it.
I'm trying to make it so that if there is no file name specified it will prompt the user for input. Also is the best way to output the manual input to a file then run the code or to store it in a variable?
So far I have this but when I run it with a file it give 1 line of error before returning the output I want. The error says ./deblank: line 1: [blank_lines.txt: command not found
if [$# -eq "$NO_ARGS"]; then
cat > temporary.txt; sed '/^$/d' <temporary.txt
else
sed '/^$/d' <$#
fi
Where am I going wrong?
You need spaces around [ and ]. In bash, [ is a command and you need spaces around it for bash to interpret it so.
You can also check for the presence of arguments by using (( ... )). So your script could be rewritten as:
if ((!$#)); then
cat > temporary.txt; sed '/^$/d' <temporary.txt
else
sed '/^$/d' "$#"
fi
If you want to use only the first argument, then you need to say $1 (and not $#).
Try using this
if [ $# -eq 0 ]; then
cat > temporary.txt; sed '/^$/d' <temporary.txt
else
cat $# | sed '/^$/d'
fi
A space is needed between [ and $# and your usage of $# is not good. $# represents all arguments and -eq is used to compare numeric values.
There are multiple problems here:
You need to leave a space between the square brackets [ ] and the variables.
When using a string type, you cannot use -eq, use == instead.
When using a string comparison you need to use double square brackets.
So the code should look like:
if [[ "$#" == "$NO_ARGS" ]]; then
cat > temporary.txt; sed '/^$/d' <temporary.txt
else
sed '/^$/d' <$#
fi
Or else use $# instead.
Instead of forcing user input to a file, I'd force the given file to stdin:
#!/bin/bash
if [[ $1 && -r $1 ]]; then
# it's a file
exec 0<"$1"
elif ! tty -s; then
: # input is piped from stdin
else
# get input from user
echo "No file specified, please enter your input, ctrl-D to end"
fi
# now, let sed read from stdin
sed '/^$/d'

shell script grep to grep a string

The output is blank fr the below script. What is it missing? I am trying to grep a string
#!/bin/ksh
file=$abc_def_APP_13.4.5.2
if grep -q abc_def_APP $file; then
echo "File Found"
else
echo "File not Found"
fi
In bash, use the <<< redirection from a string (a 'Here string'):
if grep -q abc_def_APP <<< $file
In other shells, you may need to use:
if echo $file | grep -q abc_def_APP
I put my then on the next line; if you want your then on the same line, then add ; then after what I wrote.
Note that this assignment:
file=$abc_def_APP_13.4.5.2
is pretty odd; it takes the value of an environment variable ${abc_def_APP_13} and adds .4.5.2 to the end (it must be an env var since we can see the start of the script). You probably intended to write:
file=abc_def_APP_13.4.5.2
In general, you should enclose references to variables holding file names in double quotes to avoid problems with spaces etc in the file names. It is not critical here, but good practices are good practices:
if grep -q abc_def_APP <<< "$file"
if echo "$file" | grep -q abc_def_APP
Yuck! Use the shell's string matching
if [[ "$file" == *abc_def_APP* ]]; then ...

Command not acting properly in script

The command:
value=${value%?}
will remove the last character from a variable.
Is there any logical reason why it would not work from within a script?
In my script it has no effect whatsoever.
if [[ $line =~ "What I want" ]]
then
if [[ $CURRENT -eq 3 ]]
then
echo "line is " $line
value=`echo "$line" | awk '{print $4}'`
echo "value = "$value
value=${value%?}
echo "value = $value "
break
fi
fi
I cant post the whole script, but this is the piece I refer to. The loop is being entered properly, but the 2 echo $value lines return the same thing.
Edit - this question still stands. The code works fine line bu line in a terminal, but all together in a script it fails.
Echo adds an extra line character to $value in this line:
value=`echo "$line" | awk '{print $4}'`
And afaik that extra char is removed with %?, so it seems it does not change anything at all.
Try echo -n instead, which does not add \n to the string:
value=`echo -n "$line" | awk '{print $4}'`
Since you have provided only the relevant part in the code and not the whole file, I'm going to assume that the first line of your file reads `#!/bin/sh'. This is your problem. What you are trying to do (parameter expansion) is specific to bash, so unless /bin/sh points to bash via a symlink, then you are running the script in a shell which does not understand bash parameter expansion.
To see what /bin/sh really is you can do: ls -l /bin/sh. And to remedy the situation, force the script to run in bash by changing the `shebang' at the top to read `#!/bin/bash'

How would I use grep on a single word in UNIX?

I want to run a script in unix that will look for a specific pattern inside a passed argument. The argument is a single word, in all cases. I cannot use grep, as grep only works on searching through files. Is there a better unix command out there that can help me?
Grep can search though files, or it can work on stdin:
$ echo "this is a test" | grep is
this is a test
Depending on what you're doing you may prefer to use bash pattern matching:
# Look for text in $word
if [[ $word == *text* ]]
then
echo "Match";
fi
or regular expressions:
# Check is $regex matches $word
if [[ $word =~ $regex ]]
then
echo "Match";
fi
you can use case/esac as well. No need to call any external commands (for your case)
case "$argument" in
*text* ) echo "found";;
esac
As of bash 3.0 it has a built in regexp operator =~
Checkout http://aplawrence.com/Linux/bash-regex.html
if echo $argument | grep -q pattern
then
echo "Matched"
fi
my File is:
$ cat > log
loga
hai how are you loga
hai
hello
loga
My command is:
sed -n '/loga/p' log
My answer is:
loga
hai how are you loga
loga

Resources