Here's generic command I want to pass variable at DEFINE keyword place but failing
awk '/DEFINE/,/REPLACE/' file
Here's what I have tried
line=DEFINE
awk -v myvar=$line '/myvar/,/REPLACE/' file
The following should work:
line=DEFINE
awk -v myvar="$line" '($0~myvar),/REPLACE/{print}'
Example:
$ seq 1 10 | awk -v x=5 '($0~x),/8/'
5
6
7
8
See Passing variables to range patterns in awk for more details
Related
Is there a way to process 2 different files passed as arguments to a bash script which uses awk.
Script signature:
./statistical_sig.sh path_to_reviews_folder hotel_1 hotel_2
I tried the following but only the first argument got processed.
hotel1="$2";
hotel2="$3";
dos2unix -U $hotel1 | dos2unix -U $hotel2 | echo "$hotel1" "$hotel2" | xargs | awk -v hotel1="$hotel1" -v hotel2="$hotel2" { .. code ..}
You don't need all these pipes to run awk.
Either you will use something like this if you plan to read with awk some other files and use hotel1 and hotel2 somehow inside your awk code:
awk -v hotel1=$(dos2unix -U "$hotel1") -v hotel2=$(dos2unix -U "$hotel2") { awk code ..} file1 file2
Or you will use this if you plan to read and process contents of files hotel1 and hotel2:
awk { awk code ..} <(dos2unix -U "$hotel1") <(dos2unix -U "$hotel2")
Alternativelly you can modify your code like this, but this is less efficient :
hotel1=$(dos2unix "$hotel1") && hotel2=$(dos2unix "$hotel2") && echo "$hotel1 $hotel2" | awk '{your code here}'
If you explain better your question advising what is the awk code and what you are trying to achieve, you will get better advises.
I have an xml file with these lines (among others):
#Env=DEV2,DEV3,DEV5,DEV6
#Enter your required DEV environment after the ENV= in the next line:
Env=DEV6
I need to:
Verify that the text after ENV= is of the pattern DEV{1..99}
extract the number (in this case, 6) from the line ENV=DEV6 to some environment variable
I know a bit of awk and grep, and can use those to get the number, but I'm thinking of Sed, which I'm told matches patterns nicer than awk and takes less time. Also, I'm concerned about long long lines of greps matching the beginning of the line for that particular Env= .
How would I go about doing it with Sed? would I get away with a shorter line?
I'm a sed newbie, read a bunch of tutorials and examples and got my fingers twisted trying to do both things at the same time...
Can use grep also if pcre regex is available
$ cat ip.txt
#Env=DEV2,DEV3,DEV5,DEV6
#Enter your required DEV environment after the ENV= in the next line:
Env=DEV6
foo
Env=DEV65
bar
Env=DEV568
$ grep -xoP 'Env=DEV\K[1-9][0-9]?' ip.txt
6
65
-x match whole line
-o output only matching text
-P use pcre regex
Env=DEV\K match Env=DEV but not part of output
[1-9][0-9]? range of 1 to 99
I suggest with GNU sed:
var=$(sed -nE 's/^Env=DEV([0-9]{1,2})$/\1/p' file)
echo "$var"
Output:
6
awk -F'Env=DEV' '/Env=DEV[0-9]$|Env=DEV[0-9][0-9]$/{print $2}' input
Input:
echo '
Env=DEV6
Env=DEVasd
Env=DEV62
Env=DEV622'
Output:
awk -F'Env=DEV' '/Env=DEV[0-9]$|Env=DEV[0-9][0-9]$/{print $2}' input
6
62
To store it into any variable:
var=$(awk command)
In awk. First some test cases:
$ cat file
foo
Env=DEV0
Env=DEV1
Env=DEV99
Env=DEV100
$ awk 'sub(/^Env=DEV/,"") && /^[1-9][0-9]?$/' file
1
99
You can used sed as
$ sed 's/^Env=DEV\([1-9][0-9]\?\)/\1/' file
6
You can directly use the above command in export command as
export YOUR_EXPORT_VARIABLE=$(sed 's/^Env=DEV\([1-9][0-9]\?\)/\1/' file)
(or) its pretty straight forward with perl
$ perl -nle 'print $1 if /Env=DEV.*?(\d+)/' file
6
This question already has answers here:
How to use awk variables in regular expressions?
(5 answers)
Closed 6 years ago.
I am trying to get data between sequences of numbers, for instance:
100000000
1 2 3 4
5 6 7 8
....
100001000
....
200000000
I match the patterns using awk and I can use it successfully without using variables, for instance:
awk '/10000000 /{flag=1;next}/10001000/{exit}flag' input.dat
However when I try to use shell variables within this command, it gives no output whatsoever:
for i in {1..4}
do
step1=$(($i*10000000))
step2=$(($step1+1000))
awk -v arg1="$step1" -v arg2="$step2" '/arg1 /{flag=1;next}/arg2 /{exit}flag' input.dat
done
Is there something obvious I'm missing?
Regex literal /.../ doesn't allow variables. You can use ~ operator for regex matching in regex with variables:
awk -v arg1="$step1" -v arg2="$step2" '$0 ~ arg1 " "{flag=1; next}
$0 ~ arg2{exit} flag' input.dat
I have a .dat file with | separator and I want to change the value of the column which is defined by a number passed as argument and stored in a var. My code is
awk -v var="$value" -F'|' '{ FS = OFS = "|" } $1=="$id" {$"\{$var}"=8}1'
myfile.dat > tmp && mv tmp myfiletemp.dat
This changes the whole line to 8, obviously doesn't work. I was wondering what is the right way to write this part
{$"\{$var}"=8}1
For example, if I want to change the fourth column to 8 and I have value=4, how do I get {$4=8}?
The other answer is mostly correct, but just wanted to add a couple of notes, in case it wasn't totally clear.
Referring to a variable with a $ in front of it turns it in to a reference to the column. So i=3; print $i; print i will print the third column and then the number 3.
Putting all your variables in the command line will avoid any problems with trying to include bash variables inside your single-quoted awk code, which won't work.
You can let awk do the output to the specific file instead of relying on bash to redirect output and move files.
The -F option on the command line specifies FS for you, so no need to redeclare it in your code.
Here's how I would do this:
#!/bin/bash
column=4
value=8
id=1
awk -v col="$column" -v val="$value" -v id="$id" -F"|" '
BEGIN {OFS="|"}
{$1==id && $col=val; print > "myfiletemp.dat"}
' myfile.dat
you can refer to the awk variable directly by it's name, slight rewrite of your script with correct reference to column number var...
awk -F'|' -v var="$value" 'BEGIN{OFS=FS} $1=="$id"{$var=8}1'
should work as long as $value is a number. If id is another bash variable, pass it the same way as an awk variable
awk -F'|' -v var="$value" -v id="$id" 'BEGIN{OFS=FS} $1==id{$var=8}1'
Not only can you use a number in a variable by putting a $ in front of it, you can also use put a $ in front of an expression!
$ date | tee /dev/stderr | awk '{print $(2+2)}'
Mon Aug 3 12:47:39 CDT 2020
12:47:39
The variable in awk does not return the result.
I am trying to get the next line of the matched value from file by using awk. It works fine without the variable. Thanks.
$ cat file
name=bobk
snm=sahh
emp=bklc
jdate=879
$
$ awk '/name/{getline; print}' file
snm=sahh ---------> Got the result
$
$ export MYVAR=name
$
$ echo $MYVAR
name
$
$ awk -v AVAR=${MYVAR} '/AVAR/{getline; print}' file
$ ---------> No result
You need to use the regexp match operator ~ against the whole line $0 as /AVAR/ is match for the string AVAR not the variable AVAR:
$ awk -v AVAR=${MYVAR} '$0~AVAR{getline; print}' file
snm=sahh
Using AWK variables to pass parameters is almost always cleaner as to intent; however, in this specific case, the resulting script is a bit more complex than it needs to be -- and breaking the rules a bit may actually be easier to understand and communicate to others.
With use of double-quotes to escape the script there is no need to use an additional AWK variable. That, and if we place the shell variable within /'s there is no need for the {}:
$ awk "/$MYVAR/ {getline; print}" file
snm=sahh
$