How to make 'grep' use text and regex at same time? - bash

I have a bash script where I'm using grep to find text in a file. The search-text is stored in a variable.
found=$(grep "^$line$" "$file")
I need grep to use regex while not interpret the variable $line as regex. If for example $line contains a character which is a regex operator, like [, an error is triggered:
grep: Unmatched [
Is it somehow possible to make grep not interpret the content of $line as regex?

You can use the -F flag of grep to make it interpret the patterns as fixed strings instead of regular expressions.
In addition,
if you want the patterns to match entire lines (as implied by your ^$line$ pattern),
you can combine with the -x flag.
So the command in your post can be written as:
found=$(grep -Fx "$line" "$file")

Way to tell grep that the characters provided are ordinary characters is to escape them properly, for example:
$ cat file
this
[that
not[
$ line=\\[ # escaped to mark [ as an ordinary character
$ grep "$line" file
that[
[not
$ line=[is] # [ is part of a proper regex
$ grep "$line" file
this
So the key is to escape the regex chars:
$ line=[
$ echo "${line/[/\\[}"
\[
$ grep "^${line/[/\\[}" file
[that
Or you could use awk:
$ line=[that
$ awk -v s="$line" '$0==s' file
[that

Related

grep the folder using sed command in shell script

I am trying to grep the folder name from full tar file. Below is the example.
example:
TEST-5.3.0.0-build1.x86_64.tar.gz
I want to grep the folder name (TEST-5.3.0.0-build1) in shell script
So i tried below command for grep
$ package_folder=$(echo TEST-5.3.0.0-build1.x86_64.tar.gz | sed -e "s/.[0-9]*[a-z]*[0-9]*.tar.gz$//" | sed -e 's/\/$//')
But I am getting below output:
$ echo $package_folder
TEST-5.3.0.0-build1.x86
Could you please anyone correct me where I am doing mistake. I need folder name as TEST-5.3.0.0-build1
Thanks in Advance!!!
In your command, you do not match _, x, etc. The [0-9]*[a-z]*[0-9]* only matches a sequence of zero or more digits, zero or more (lowercase) letters, and zero or more digits. It is better to use a [^.]* to match any chars other than . between two . chars. Also, literal dots must be escaped, or an unescaped . will match any single char.
You can use
sed 's/\.[^.]*\.tar\.gz$//'
Or, just use string manipulation if x86_64 is also a constant:
s='TEST-5.3.0.0-build1.x86_64.tar.gz'
s="${s/.x86_64.tar.gz/}"
See the online demo:
#!/bin/bash
s='TEST-5.3.0.0-build1.x86_64.tar.gz'
package_folder=$(sed 's/\.[^.]*\.tar\.gz$//' <<< "$s")
echo "${package_folder}"
# => TEST-5.3.0.0-build1
s="${s/.x86_64.tar.gz/}"
echo "$s"
# => TEST-5.3.0.0-build1
You can use uname -m in replacement part of this string:
s='TEST-5.3.0.0-build1.x86_64.tar.gz'
echo "${s%.$(uname -m)*}"
TEST-5.3.0.0-build1
Or using sed:
sed "s/\.$(uname -m).*//" <<< "$s"
TEST-5.3.0.0-build1

what does the at sign before a dollar sign #$VAR do in a SED string in a Shell script?

What does #$VAR mean in Shell? I don't get the use of # in this case.
I encountered the following shell file while working on my dotfiles repo
#!/usr/bin/env bash
KEY="$1"
VALUE="$2"
FILE="$3"
touch "$FILE"
if grep -q "$1=" "$FILE"; then
sed "s#$KEY=.*#$KEY=\"$VALUE\"#" -i "$FILE"
else
echo "export $KEY=\"$VALUE\"" >> "$FILE"
fi
and I'm struggling with understanding the sed "s#$KEY=.*#$KEY=\"$VALUE\"#" -i "$FILE" line, especially the use of #.
When using sed you must not necessarily use a / character as the delimiter for the substitute action.
Thereby, the #, or % characters are also perfectly fine options to be used instead:
echo A | sed s/A/B/
echo A | sed s#A#B#
echo A | sed s%A%B%
In the command
sed "s#$KEY=.*#$KEY=\"$VALUE\"#" -i "$FILE"
the character # is used as a delimiter in the s command of sed. The general form of the s (substitute) command is
s<delim><searchPattern><delim><replaceString><delim>[<flags>]
where the most commonly used <delim> is /, but other characters are sometimes used, especially when either <searchPattern> or <replaceString> contain (or might contain) slashes.

Adding double quotes to beginning, end and around comma's in bash variable

I have a shell script that accepts a parameter that is comma delimited,
-s 1234,1244,1567
That is passed to a curl PUT json field. Json needs the values in a "1234","1244","1567" format.
Currently, I am passing the parameter with the quotes already in it:
-s "\"1234\",\"1244\",\"1567\"", which works, but the users are complaining that its too much typing and hard to do. So I'd like to just take a comma delimited list like I had at the top and programmatically stick the quotes in.
Basically, I want a parameter to be passed in as 1234,2345 and end up as a variable that is "1234","2345"
I've come to read that easiest approach here is to use sed, but I'm really not familiar with it and all of my efforts are failing.
You can do this in BASH:
$> arg='1234,1244,1567'
$> echo "\"${arg//,/\",\"}\""
"1234","1244","1567"
awk to the rescue!
$ awk -F, -v OFS='","' -v q='"' '{$1=$1; print q $0 q}' <<< "1234,1244,1567"
"1234","1244","1567"
or shorter with sed
$ sed -r 's/[^,]+/"&"/g' <<< "1234,1244,1567"
"1234","1244","1567"
translating this back to awk
$ awk '{print gensub(/([^,]+)/,"\"\\1\"","g")}' <<< "1234,1244,1567"
"1234","1244","1567"
you can use this:
echo QV=$(echo 1234,2345,56788 | sed -e 's/^/"/' -e 's/$/"/' -e 's/,/","/g')
result:
echo $QV
"1234","2345","56788"
just add double quotes at start, end, and replace commas with quote/comma/quote globally.
easy to do with sed
$ echo '1234,1244,1567' | sed 's/[0-9]*/"\0"/g'
"1234","1244","1567"
[0-9]* zero more consecutive digits, since * is greedy it will try to match as many as possible
"\0" double quote the matched pattern, entire match is by default saved in \0
g global flag, to replace all such patterns
In case, \0 isn't recognized in some sed versions, use & instead:
$ echo '1234,1244,1567' | sed 's/[0-9]*/"&"/g'
"1234","1244","1567"
Similar solution with perl
$ echo '1234,1244,1567' | perl -pe 's/\d+/"$&"/g'
"1234","1244","1567"
Note: Using * instead of + with perl will give
$ echo '1234,1244,1567' | perl -pe 's/\d*/"$&"/g'
"1234""","1244""","1567"""
""$
I think this difference between sed and perl is similar to this question: GNU sed, ^ and $ with | when first/last character matches
Using sed:
$ echo 1234,1244,1567 | sed 's/\([0-9]\+\)/\"\1\"/g'
"1234","1244","1567"
ie. replace all strings of numbers with the same strings of numbers quoted using backreferencing (\1).

sed or grep to read between a set of parentheses

I'm trying to read a version number from between a set of parentheses, from this output of some command:
Test Application version 1.3.5
card 0: A version 0x1010000 (1.0.0), 20 ch
Total known cards: 1
What I'm looking to get is 1.0.0.
I've tried variations of sed and grep:
command.sh | grep -o -P '(?<="(").*(?=")")'
command.sh | sed -e 's/(\(.*\))/\1/'
and plenty of variations. No luck :-(
Help?
You were almost there! In pgrep, use backslashes to keep literal meaning of parentheses, not double quotes:
grep -o -P '(?<=\().*(?=\))'
Having GNU grep you can also use the \K escape sequence available in perl mode:
grep -oP '\(\K[^)]+'
\K removes what has been matched so far. In this case the starting ( gets removed from match.
Alternatively you could use awk:
awk -F'[()]' 'NF>1{print $2}'
The command splits input lines using parentheses as delimiters. Once a line has been splitted into multiple fields (meaning the parentheses were found) the version number is the second field and gets printed.
Btw, the sed command you've shown should be:
sed -ne 's/.*(\(.*\)).*/\1/p'
There are a couple of variations that will work. First with grep and sed:
grep '(' filename | sed 's/^.*[(]\(.*\)[)].*$/\1/'
or with a short shell script:
#!/bin/sh
while read -r line; do
value=$(expr "$line" : ".*(\(.*\)).*")
if [ "x$value" != "x" ]; then
printf "%s\n" "$value"
fi
done <"$1"
Both return 1.0.0 for your given input file.

How to delete the string which is present in parameter from file in unix

I have redirected some string into one parameter for ex: ab=jyoti,priya, pranit
I have one file : Name.txt which contains -
jyoti
prathmesh
John
Kelvin
pranit
I want to delete the records from the Name.txt file which are contain in ab parameter.
Please suggest if this can be done ?
If ab is a shell variable, you can easily turn it into an extended regular expression, and use it with grep -E:
grep -E -x -v "${ab//,/|}" Name.txt
The string substitution ${ab//,/|} returns the value of $ab with every , substituted with a | which turns it into an extended regular expression, suitable for passing as an argument to grep -E.
The -v option says to remove matching lines.
The -x option specifies that the match needs to cover the whole input line, so that a short substring will not cause an entire longer line to be removed. Without it, ab=prat would cause pratmesh to be removed.
If you really require a sed solution, the transformation should be fairly trivial. grep -E -v -x 'aaa|bbb|ccc' is equivalent to sed '/^\(aaa\|bbb\|ccc)$/d' (with some dialects disliking the backslashes, and others requiring them).
To do an in-place edit (modify Name.txt without a temporary file), try this:
sed -i "/^\(${ab//,/\|}\)\$/d" Name.txt
This is not entirely robust against strings containing whitespace or other shell metacharacters, but if you just need
Try with
sed -e 's/\bjyoti\b//g;s/\bpriya\b//g' < Name.txt
(using \b assuming you need word boundaries)
this will do it:
for param in `echo $ab | sed -e 's/[ ]+//g' -e 's/,/ /g'` ; do res=`sed -e "s/$param//g" < name.txt`; echo $res > name.txt; done
echo $res

Resources