I have a text file, named my_data.txt, with the following contents:
# define var1 and var2
var1=101
var2=202
// display var1 and var2
echo ${var1}
echo ${var2}
I want search all occurrences of var1 but not those in a line starts with '#' or '//'. I can do this:
grep var1 my_data.txt | grep -v '^#' | grep -v '^//'
output:
var1=101
echo ${var1}
The results is correct. The question: is there any way to pass both values '^#' and '^//' to a single -v option?
I suggest:
grep -v -e '^#' -e '^//' file | grep 'var1'
Or with an extended regular expression (-E):
grep -v -E '^(#|//)' file | grep 'var1'
Output:
var1=101
echo ${var1}
If you can make use of grep -P you can use a Perl-compatible regular expression and a single grep command.
grep -P "^(?!#|//).*\bvar1\b" my_data.txt
The pattern matches
^ Start of string
(?!#|//) Negative lookahead, assert not # or //
.*\bvar1\b Match the word var1 in the line
Or using awk skipping the line that starts with # or // and print a line that contains var1
awk '/^(#|\/\/)/{next};index($0, "var1")' my_data.txt
The examples will output:
var1=101
echo ${var1}
Related
I have a variable having following format
bundle="chn-pro-X.Y-Z.el8.x86_64"
X,Y,Z are numbers having any number of digits
Ex:
1.0-2 # X=1 Y=0 Z=2
12.45-9874 # X=12 Y=45 Z=9874
How can I grab X.Y and store it in another variable?
EDIT:
I wasn't right with my wording, but
I want to store X.Y into new variable not individual X & Y's
I'm looking to finally have a variable version which has X.Y grabbed from bundle:
version="X.Y"
I would use awk:
bundle="chn-pro-12.45-9874.el8.x86_64"
echo "$bundle" | awk -F "[.-]" '{print $3,$4,$5}'
12 45 9874
Now if you want to assign to x, y, z use read and process substitution:
read -r x y z < <(echo "$bundle" | awk -F "[.-]" '{print $3,$4,$5}')
echo "x=$x, y=$y, z=$z"
x=12, y=45, z=9874
If you just want the value of X.Y as a single value this is still great use for awk:
bundle="chn-pro-12.45-9874.el8.x86_64"
echo "$bundle" | awk -F "[-]" '{print $3}'
12.45
And if you then want to put that into a variable:
x_y=$(echo "$bundle" | awk -F "[-]" '{print $3}')
echo "x_y=$x_y"
x_y=12.45
Or you can use cut in this case to get the third field:
echo "$bundle" | cut -d- -f3
12.45
Like that:
$ bundle="chn-pro-1.0-2.el8.x86_64"
$ X="$(echo "$bundle" | cut -d . -f1 | cut -d- -f3)"
$ Y="$(echo "$bundle" | cut -d . -f2 | cut -d- -f1)"
$ Z="$(echo "$bundle" | cut -d . -f2 | cut -d- -f2)"
$ echo "$X"
1
$ echo "$Y"
0
$ echo "$Z"
2
You can merge X and Y into a single variable:
$ XY="$X.$Y"
$ echo $XY
1.0
Use regex to separate numbers:
numbers=$(echo $bundle | grep -Eo '([0-9]+\.[0-9]+\-[0-9]+)' | sed 's/\./\t/g;s/\-/\t/g')
Then assign them to variables with using awk or tr or cut, whatever you want:
X=$(echo $numbers| awk '{print $1}')
Y=$(echo $numbers| awk '{print $2}')
Z=$(echo $numbers| awk '{print $3}')
EDIT
For storing x.y into single version variable you can simply ignore pervios commands:
version=$(echo $bundle | grep -Eo '([0-9]+\.[0-9]+\-[0-9]+)' | grep -Eo '([0-9]+\.[0-9]+)')
Given this input:
$ bundle="chn-pro-12.45-9874.el8.x86_64"
using GNU or BSD sed for -E:
$ foo=$(echo "$bundle" | sed -E 's/.*-([0-9]+\.[0-9]+)-[0-9].*/\1/')
$ echo "$foo"
12.45
or with any sed:
$ foo=$(echo "$bundle" | sed 's/.*-\([0-9][0-9]*\.[0-9][0-9]*\)-[0-9].*/\1/')
$ echo "$foo"
12.45
Assumptions:
the input string will always contain (at least) 3 hyphens
the desired version string will always reside between the 2nd and 3rd hyphens of the input string
we need to maintain the input string (ie, don't clobber/overwrite the variable containing the input string)
We can eliminate the subprocess calls (necessary for echo/sed/grep/awk/sed) by using some parameter expansions:
$ bundle="chn-pro-X.Y-Z.el8.x86_64"
$ temp="${bundle#*-}" # strip off 1st hyphen delimited string
$ echo "${temp}"
pro-X.Y-Z.el8.x86_64
$ temp="${temp#*-}" # strip off 2nd hyphen delimited string
$ echo "${temp}"
X.Y-Z.el8.x86_64
$ version="${temp%%-*}" # save 3rd hyphen delimited string (aka our version)
$ echo "${version}"
X.Y
NOTE: We can eliminate the temp variable by replacing all occurrences of temp with version with the understanding version does not contain what we want until after the 3rd parameter expansion has occurred, eg:
$ bundle="chn-pro-X.Y-Z.el8.x86_64"
$ version="${bundle#*-}"
$ version="${version#*-}"
$ version="${version%%-*}"
$ echo "${version}"
X.Y
I have a string:
foo="re-9619-add-selling-office";
I'd like to break up the string on the second - (dash) into variable1 and variable2. I want to end up with variable1=re-9619 and variable2=add-selling-office
I tried it using grep and awk, but now I not sure that's the way to go.
Here is a single sed + read way:
foo="re-9619-add-selling-office"
read var1 var2 < <(sed -E 's/^([^-]*-[^-]*)-/\1 /' <<< "$foo")
# check variables
declare -p var1 var2
declare -- var1="re-9619"
declare -- var2="add-selling-office"
Could you please try following once. Where first variable will have value like re-9619 and second shell variable will have value like add-selling-office
first=$(echo "$foo" | sed 's/\([^-]*-[^-]*\)-.*/\1/')
second=$(echo "$foo" | sed 's/\([^-]*\)-\([^-]*\)-\(.*\)/\3/')
Explanation:
echo "$foo" | sed 's/\([^-]*-[^-]*\)-.*/\1/': Printing value of foo variable and passing its output to sed command. In sed I am using substitute capability to perform substitution, \([^-]*-[^-]*\)-.*(which has everything from starting of value to till 2nd occurrence of - in back reference in it). Then substituting whole value with 1st captured back reference value which will become only re-9619.
echo "$foo" | sed 's/\([^-]*\)-\([^-]*\)-\(.*\)/\3/': Logic is same as above mentioned command. Using sed's capability of substitution with using back reference capability of it. Here we are printing everything after 2nd occurrence of -.
NOTE: second=$(echo "$foo" | sed -E "s/$first-(.*)/\1/") could also help as per #User123's comments.
That can be done using parameter expansions, you don't need an external utility.
$ foo="re-9619-add-selling-office"
$ variable2=${foo#*-*-}
$ variable1=${foo%-"$variable2"}
$
$ echo $variable1
re-9619
$ echo $variable2
add-selling-office
You can use cut:
variable1=$(echo $foo | cut -d '-' -f 1-2)
variable2=$(echo $foo | cut -d '-' -f 3-)
This is the result:
>> echo $variable1
re-9619
>> echo $variable2
add-selling-office
I'm doing the following command in a bash:
echo -e 'UNUSED\nURL: ^/tags/0.0.0/abcd' | sed -rn 's#^URL: \^/tags/([^/]+)/#\1#p'
I think this should output only the matching lines and the content of the capture group. So I'm expecting 0.0.0 as the result. But I'm getting 0.0.0abcd
Why contains the capture group parts from the left and the right side of the /? What I am doing wrong?
echo -e 'UNUSED\nURL: ^/tags/0.0.0/abcd' |
sed -rn 's#^URL: \^/tags/([^/]+)/#\1#p'
echo outputs two lines:
UNUSED
URL: ^/tags/0.0.0/abcd
The regular expression given to sed does not match the first line, so this line is not printed. The regular expression matches the second line, so URL: ^/tags/0.0.0/ is replaced with 0.0.0; only the matched part of the line is replaced, so abcd is passed unchanged.
To obtain the desired output you must also match abcd, for example with
sed -rn 's#^URL: \^/tags/([^/]+)/.*#\1#p'
where the .* eats all characters to the end of the line.
You can use awk:
echo -e 'UNUSED\nURL: ^/tags/0.0.0/abcd' | awk -F/ 'index($0, "^/tags/"){print $3}'
0.0.0
This awk command uses / as field delimiter and prints 3rd column when there ^/tags/ text in input.
Alternatively, you can use gnu grep:
echo -e 'UNUSED\nURL: ^/tags/0.0.0/abcd' | grep -oP '^URL: \^/tags/\K([^/]+)'
0.0.0
Or this sed:
echo -e 'UNUSED\nURL: ^/tags/0.0.0/abcd' | sed -nE 's~^URL: \^/tags/([^/]+).*~\1~p'
0.0.0
This sed catch your desired output.
echo -e 'UNUSED\nURL: ^/tags/0.0.0/abcd' | sed -E '/URL/!d;s#.*/(.*)/[^/]*#\1#'
I have a file VALIDATION_CONFIG_FILE.cfg which contains the records below:
ES_VDF_1|1
DE_VDF_1|2
ES_VDF_1|7
When I am using the grep command below by using variable then the command is returning ES_VDF_1 output. As per my understanding, command should not give any results. When I use the same command without using variables (use values directly) then command is returning no results, which is as expected. So what is the problem with variables which I am using?
FEED_ID_1_7="HU_VDF_1"
FEED_ID_2_7="ES_VDF_1"
FEED_ID_3_7="PT_VDF_2"
awk -F'|' '{ if($2=="7") print $1; }' VALIDATION_CONFIG_FILE.cfg |
grep -E -v '${FEED_ID_1_7}|${FEED_ID_2_7}|${FEED_ID_3_7}'
Output: ES_VDF_1
awk -F'|' '{ if($2=="7") print $1; }' VALIDATION_CONFIG_FILE.cfg |
grep -E -v 'ES_VDF_1|HU_VDF_1|PT_VDF_2'
Output: nothing
The problem you are seeing is that single quotes in Bash do not interpolate variables, whereas double quotes do.
For example with a variable imaginatively called "VARIABLE":
alex#yuzu:~$ export VARIABLE="foo"
If you echo it with double quotes, it is interpolated and the value of the variable is used:
alex#yuzu:~$ echo "$VARIABLE"
foo
But if you use single quotes the literal string '$VARIABLE' is used instead:
alex#yuzu:~$ echo '$VARIABLE'
$VARIABLE
The same goes for your grep.
grep -E -v '${FEED_ID_1_7}|${FEED_ID_2_7}|${FEED_ID_3_7}'
Should be:
grep -E -v "${FEED_ID_1_7}\|${FEED_ID_2_7}\|${FEED_ID_3_7}"
For example:
alex#yuzu:~$ echo "foo" | grep -E "$VARIABLE|$HOME|$USER"
foo
alex#yuzu:~$ echo "foo" | grep -E '$VARIABLE|$HOME|$USER'
[ no output ]
This is happening due to quotes.
Single quotes won't interpolate anything, but double quotes will do. Replace single quotes to double quotes with variables like below :
awk -F'|' '{ if($2=="7") print $1; }' VALIDATION_CONFIG_FILE.cfg |
grep -E -v "${FEED_ID_1_7}|${FEED_ID_2_7}|${FEED_ID_3_7}"
Refer bash manual for more details
Adding to Kaoru/Nishu Tayal's answer, you can make it safer further by using normal text search with fgrep and multiple -e:
fgrep -v -e "${FEED_ID_1_7}" -e "${FEED_ID_2_7}" -e "${FEED_ID_3_7}"
This would help prevent misinterpretations just in case special characters would be added to the values of variables.
If you don't have fgrep try grep -F.
I am using bash script for testing purpose.During my testing I have to find the line number of first occurrence of a string in a file. I have tried "awk" and "grep" both, but non of them return the value.
Awk example
#/!bin/bash
....
VAR=searchstring
...
cpLines=$(awk '/$VAR/{print NR}' $MYDIR/Configuration.xml
this does not expand $VAR. If I use the value of VAR it works, but I want to use VAR
Grep example
#/!bin/bash
...
VAR=searchstring
...
cpLines=grep -n -m 1 $VAR $MYDIR/Configuration.xml |cut -f1 -d:
this gives error line 20: -n: command not found
grep -n -m 1 SEARCH_TERM FILE_PATH |sed 's/\([0-9]*\).*/\1/'
grep switches
-n = include line number
-m 1 = match one
sed options (stream editor):
's/X/Y/' - replace X with Y
\([0-9]*\) - regular expression to match digits zero or multiple times occurred, escaped parentheses, the string matched with regex in parentheses will be the \1 argument in the Y (replacement string)
\([0-9]*\).* - .* will match any character occurring zero or multiple times.
You need $() for variable substitution in grep
cpLines=$(grep -n -m 1 $VAR $MYDIR/Configuration.xml |cut -f1 -d: )
Try something like:
awk -v search="$var" '$0~search{print NR; exit}' inputFile
In awk, / / will interpret awk variable literally. You need to use match (~) operator. What we are doing here is looking for the variable against your input line. If it matches, we print the line number stored in NR and exit.
-v allows you to create an awk variable (search) in above example. You then assign it your bash variable ($var).
grep -n -m 1 SEARCH_TERM FILE_PATH | grep -Po '^[0-9]+'
explanation:
-Po = -P -o
-P use perl regex
-o only print matched string (not the whole line)
Try pipping;
grep -P 'SEARCH TERM' fileName.txt | wc -l