This is what I'm trying to break my head over... but it still doesn't work
This is what works:
awk '{
if ( $2 ~ /FULL/ )
print "TRUE" $0
}' FILE1
But now I want to have the "/FULL/" part dynamic, so I can change this by linking it to a shell variable.
So this it what I want:
shellfull="/FULL/"
awk -v awkfull=$shellfull '{
if ( $2 ~ awkfull )
print "TRUE" $0
}' FILE1
Where FILE1 has the content:
84738273 RTF-Ref-FULL-Monday
38473847 The-Wed-DLL-DIFF-Fri-FULL
84839489 FULL
83945940 Schedule_FULL
84928049 Schedule_DIFF
It kinda looks like it doesn't let me use awkfull inside the "IF" statment, because I can print awkfull and it shows me indeed the shell variable content.
What I mean:
print awkfull //this works)
if ( $1 ~ awkfull) //this does NOT work... why?
UPDATE 1
Ty all for the responses, some gave really good idea's. But I feel like I need to describe my exact situation. Because I really need the "/" in order to find a word on a reandom place in a string, and I don't know what the string is going to look like in advance.
So here is what I want to use it for:
//Set Variable Content:
FULL="/FULL/||/Full/||/full/||/SQL-Backup_Fri-Last-Of-Month/"
DIFF="/DIFF/||/Diff/||/diff/||/San-Element/"
Now I have a big logfile, that has in one of its strings somewords that I'm going to use to determine if it counts as a FULL, or a DIFF, or maybe both based on if there is a word in $FULL or $DIFF matching a reandom place in that string
So if the stings in the file look like this:
84738273 RTF-Ref-full-Monday
38473847 The-Wed-DLL-DIFF-Fri-FULL
84839489 FULL
83945940 Schedule_Full_backup
84928049 Schedule_DIFF
83940392 2_SQL-Backup_Fri-Last-Of-Month-23049
84828348 Schedule_new-build
I want it to become:
FULL 84738273 RTF-Ref-full-Monday
DIFFFULL 38473847 The-Wed-DLL-DIFF-Fri-FULL
FULL 84839489 FULL
FULL 83945940 Schedule_Full
DIFF 84928049 Schedule_DIFF
FULL 83940392 2_SQL-Backup_Fri-Last-Of-Month-23049
UNKNOWN 84828348 Schedule_new-build
Now it is important to have the list with words that it is going to searche for setup in a dynamic way. So that with changing the variable $DIFF or $FULL it is going to search for different words in the file.
So how I wanted to achieve this was by the sript concept:
//set filter patterns
FULL="/FULL/||/Full/||/full/||/SQL-Backup_Fri-Last-Of-Month/"
DIFF="/DIFF/||/Diff/||/diff/||/San-Element/"
awk -v full=$FULL -v diff=$DIFF '{ //link the shell variable to awk variable, since a direct shell variable in awk didn't work.
if ( $2 ~ diff ) //find all strings with words defined in $DIFF
print "DIFF" $0 //print "DIFF" in front of the line
if ( $2 ~ full ) //find all strings with words defined in $FULL
print "FULL" $0 //print "FULL" in front of the line
if ( $2 !~ full||diff) //if a line does neighter contain words of $DIFF or $FULL.
print "UNKNOWN" $0 //print "UNKNOWN" in front of the line
}' FILE1 //load the file that needs to be filterd
This script needs to make clear to me, and itself, if it finds Schedules and Policies that are not yet defined. This so that the scipt knows when a new unknown schedule or policy is created and can warn me that I needs to adjust its filter. And it gives the script the ability to calculate how much the calculated Kbytes are going to a "unknown schedules and policies pool"
Hope this makes things a bit more clear. I need to search for multiple reandom word [FULL,full,SQL-DB,ect.] in a reandom place inside a string [thats why I wanted to use /$full/||/$diff/, or something like this].
Your string does not need to have the / for the comparison. With the following it works:
shellfull="FULL"
Test
$ shellfull="EMPTY"
$ awk -v patt=$shellfull '$1 ~ patt {print "TRUE" $0}' a
TRUEEMPTY 3928304
$ shellfull="FULL"
$ awk -v patt=$shellfull '$1 ~ patt {print "TRUE" $0}' a
TRUEFULL 2930429
TRUEFULL 3940229
Update
Based on the input file you just posted:
$ shellfull="FULL"
$ awk -v patt=$shellfull '$2 ~ patt {print "TRUE " $0}' a
TRUE 84738273 RTF-Ref-FULL-Monday
TRUE 38473847 The-Wed-DLL-DIFF-Fri-FULL
TRUE 84839489 FULL
TRUE 83945940 Schedule_FULL
In case you want exact matches:
$ awk -v patt=$shellfull '$2==patt {print "TRUE " $0}' a
TRUE 84839489 FULL
Update 2
If you want to match different words that you pass through an string, let's do it like this:
awk -v patt=$shellfull 'BEGIN{split(patt,a,"|")}
{for (i in a) if ($2==a[i]) {print "TRUE " $0; next}}' a
It gets the string, explodes it by the delimiter | (you can define another) and then looks for exact matches of each one of the given strings.
Test
$ shellfull="Schedule_DIFF|FULL"
$ awk -v patt=$shellfull 'BEGIN{split(patt,a,"|")} {for (i in a) if ($2==a[i]) {print "TRUE " $0; next}}' a
TRUE 84839489 FULL
TRUE 84928049 Schedule_DIFF
$ shellfull="FULL"
$ awk -v patt=$shellfull 'BEGIN{split(patt,a,"|")} {for (i in a) if ($2==a[i]) {print "TRUE " $0; next}}' a
TRUE 84839489 FULL
You can simply do it with:
shellfull="/FULL/"
awk '{
if ( $1 ~ '$shellfull' )
print "TRUE" $0
}' FILE1
This looks like it's now quoting the $shellfull variable, but it's actually "unquoting" it so that the shell will look at it.
Based on your latest update, it sounds like all you need is:
$ FULL="FULL|Full|full|SQL-Backup_Fri-Last-Of-Month"
$ DIFF="DIFF|Diff|diff|San-Element"
$ cat file
84738273 RTF-Ref-full-Monday
38473847 The-Wed-DLL-DIFF-Fri-FULL
84839489 FULL
83945940 Schedule_Full_backup
84928049 Schedule_DIFF
83940392 2_SQL-Backup_Fri-Last-Of-Month-23049
84828348 Schedule_new-build
$ awk -v full="$FULL" -v diff="$DIFF" '{
if ( ( $2 ~ diff ) && ( $2 ~ full ) )
print "DIFFFULL", $0
else if ( $2 ~ diff )
print "DIFF", $0
else if ( $2 ~ full )
print "FULL", $0
else
print "UNKNOWN", $0
}' file
FULL 84738273 RTF-Ref-full-Monday
DIFFFULL 38473847 The-Wed-DLL-DIFF-Fri-FULL
FULL 84839489 FULL
FULL 83945940 Schedule_Full_backup
DIFF 84928049 Schedule_DIFF
FULL 83940392 2_SQL-Backup_Fri-Last-Of-Month-23049
UNKNOWN 84828348 Schedule_new-build
but you need to decide if it's OK for words like "Different" to match with "Diff" if they happen to show up.
Getting Error while running below command:
Spacedata_1 is a file
mountpoint is external variable with some value
grep "$mountpoint" Spacedata_1 |awk -v mountpoint=$mountpoint 'if ($5==mountpoint ) { print $4 }'
Related
I have a text file called 'file.txt' with the content like,
test:one
test_test:two
test_test_test:three
If the pattern is test, then the expected output should be one and similarly for the other two lines.
This is what I have tried.
pattern=test && awk '{split($0,i,":"); if (i[1] ~ /'"$pattern"'$/) print i[2]}'
This command gives the output like,
one
two
three
and pattern=test_test && awk '{split($0,i,":"); if (i[1] ~ /'"$pattern"'$/) print i[2]}'
two
three
How can I match the unique pattern being "test" for "test" and not for "test_test" and so on.
How can I match the unique pattern being test for test and not for test_test and so on.
Don't use a regex for comparing the value, just use equality:
awk -F: -v pat='test' '$1 == pat {print $2}' file
one
awk -F: -v pat='test_test' '$1 == pat {print $2}' file
two
If you really want to use regex, then use it like this with anchors:
awk -F: -v pat='test' '$1 ~ "^" pat "$" {print $2}' file
one
If you want to use a regex, you can create it dynamically with pattern and optionally repeating _ followed by pattern until matching a :
If it matches the start of the string, then you can print the second field.
awk -v pattern='test' -F: '
$0 ~ "^"pattern"(_"pattern")*:" {
print $2
}
' file
Output
one
two
three
Or if only matching the part before the first underscore is also ok, then splitting field 1 on _ and printing field 2:
awk -v pattern='test' -F: ' {
split($1, a, "_")
if(a[1] == pattern) print $2
}' file
Using GNU sed with word boundaries
$ sed -n '/\<test\>/s/[^:]*://p' input_file
one
I'm trying to edit 3 columns in a file if the value in column 1 equals a specific string. This is my current attempt:
cp file file.copy
awk -F':' 'OFS=":" { if ($1 == "root1") $2="test"; print}' file.copy>file
rm file.copy
I've only been able to get the awk command working with one column being changed, I want to be able to edit $3 and $8 as well. Is this possible in the same command? Or is it only possible with separate awk commands or with a different command all together?
Edit note: The real command i'll be passing variables to the columns, i.e. $2=$var
It'll be used to edit the /etc/passwd file, sample input/output:
root:$6$fR7Vrjyp$irnF38R/htMSuk0efLSnAten/epf.5v7gfs0q.NcjKcFPeJmB/4TnnmgaAoTUE9.n4p4UyWOgFwB1guJau8AL.:17976::::::
You can create multiple statements for the if condition with a block {}.
awk -F':' 'OFS=":" { if ($1 == "root1") {$2="test"; $3="test2";} print}' file.copy>file
You can also improve your command by using awk's default "workflow": condition{commands}. For this you need to bring the OFS to the input variables (-v flag)
awk -F':' -v OFS=":" '$1=="root1"{$2="test"; $3="test2"; print}' file.copy>file
You may use
# Fake sample values
v1=pass1
v2=pass2
awk -v var1="$v1" -v var2="$v2" 'BEGIN{FS=OFS=":"} $1 == "root1" { $2 = var1; $3 = var2}1' file > tmp && mv tmp file
See the online awk demo:
s="root1:xxxx:yyyy
root11:xxxx:yyyy
root1:zzzz:cccc"
v1=pass1
v2=pass2
awk -v var1="$v1" -v var2="$v2" 'BEGIN{FS=OFS=":"} $1 == "root1" { $2 = var1; $3 = var2}1' <<< "$s"
Output:
root1:pass1:pass2
root11:xxxx:yyyy
root1:pass1:pass2
Note:
-v var1="$v1" -v var2="$v2" pass the variables you need to use in the awk command
BEGIN{FS=OFS=":"} set the field separator
$1 == "root1" check if Field 1 is equal to some value
{ $2 = var1; $3 = var2 } set Field 2 and 3 values
1 calls the default print command
file > tmp && mv tmp file helps you "shrink" the "replace-inplace-like" code.
I've tried the below command:
awk '/search-pattern/ {print $1}'
How do I write the else part for the above command?
Classic way:
awk '{if ($0 ~ /pattern/) {then_actions} else {else_actions}}' file
$0 represents the whole input record.
Another idiomatic way
based on the ternary operator syntax selector ? if-true-exp : if-false-exp
awk '{print ($0 ~ /pattern/)?text_for_true:text_for_false}'
awk '{x == y ? a[i++] : b[i++]}'
awk '{print ($0 ~ /two/)?NR "yes":NR "No"}' <<<$'one two\nthree four\nfive six\nseven two'
1yes
2No
3No
4yes
A straightforward method is,
/REGEX/ {action-if-matches...}
! /REGEX/ {action-if-does-not-match}
Here's a simple example,
$ cat test.txt
123
456
$ awk '/123/{print "O",$0} !/123/{print "X",$0}' test.txt
O 123
X 456
Equivalent to the above, but without violating the DRY principle:
awk '/123/{print "O",$0}{print "X",$0}' test.txt
This is functionally equivalent to awk '/123/{print "O",$0} !/123/{print "X",$0}' test.txt
Depending what you want to do in the else part and other things about your script, choose between these options:
awk '/regexp/{print "true"; next} {print "false"}'
awk '{if (/regexp/) {print "true"} else {print "false"}}'
awk '{print (/regexp/ ? "true" : "false")}'
The default action of awk is to print a line. You're encouraged to use more idiomatic awk
awk '/pattern/' filename
#prints all lines that contain the pattern.
awk '!/pattern/' filename
#prints all lines that do not contain the pattern.
# If you find if(condition){}else{} an overkill to use
awk '/pattern/{print "yes";next}{print "no"}' filename
# Same as if(pattern){print "yes"}else{print "no"}
This command will check whether the values in the $1 $2 and $7-th column are greater than 1, 2, and 5.
!IF! the values do not mach they will be ignored by the filter we declared in awk.
(You can use logical Operators and = "&&"; or= "||".)
awk '($1 > 1) && ($2 > 1) && ($7 > 5)'
You can monitoring your system with the "vmstat 3" command, where "3" means a 3 second delay between the new values
vmstat 3 | awk '($1 > 1) && ($2 > 1) && ($7 > 5)'
I stressed my computer with 13GB copy between USB connected HardDisks, and scrolling youtube video in Chrome browser.
I have a file that has multiple lines that starts with a keyword. I only want to modify one of them and it's easy to distinguish the two. I want the one that is under the [dbinfo] section. The domain name is static so I know that won't change.
awk -F '=' '$1 ~ /^dbhost/ {print $NF};' myfile.txt
myfile.txt
[ual]
path=/web/
dbhost=ez098sf
[dbinfo]
dbhost=ec0001.us-east-1.localdomain
dbname=ez098sf_default
dbpass=XXXXXX
You can use this awk command to first check for presence of [dbinfo] section and then modify dbhost parameter:
awk -v h='newhost' 'BEGIN{FS=OFS="="}
$0 == "[dbinfo]" {sec=1} sec && $1 == "dbhost"{$2 = h; sec=0} 1' file
[ual]
path=/web/
dbhost=ez098sf
[dbinfo]
dbhost=newhost
dbname=ez098sf_default
dbpass=XXXXXX
You want to utilize a little bit of a state machine here:
awk -F '=' '
$0 ~ /^\[.*\]/ {in_db_info=($0=="[dbinfo]"}
$0 ~ /^dbhost/{if (in_db_info) print $2;}' myfile.txt
You can also do it with sed:
sed '/\[dbinfo\]/,/\[/s/\(^dbhost=\).*/\1domain.com/' myfile.txt
I would like to iterate through a list and grep for the items, then use awk to pull out important information from each grep result. (This is the way I thought to do it, but awk and grep aren't necessary if there is a better way).
The input file contains a number of lines that looks similar to this:
chr1 12345 . A G 3e-12 . AB=0;ABP=0;AC=0;AF=0;AN=2;AO=2;CIGAR=1X;
I have a number of locations that should match some part of the second column.
locList="123, 789"
And for each matching location I would like to get the information from columns 4 and 5 and write them to an output file with the corresponding location.
So the output for the above list should be:
123 A G
Something like this is what I'm thinking:
for i in locList; do
grep i inputFile.txt | awk '{print $2,$4,$5}'
done
Invoking grep/awk once per location will be highly inefficient. You want to invoke a single command that will do your parsing. For example, awk:
awk -v locList="12345 789" '
BEGIN {
# parse the location list, and create an array where
# the locations are the array indexes
n = split(locList, a)
for (i=1; i<=n; i++) locations[a[i]] = 1
}
$2 in locations {print $2, $4, $5}
' file
revised requirements
awk -v locList="123 789" '
BEGIN { n = split(locList, patterns) }
{
for (i=1; i<=n; i++) {
if ($2 ~ "^" patterns[i]) {
print $2, $4, $5
break
}
}
}
' file
The ~ operator is the regular expression matching operator.
That will output 12345 A G from your sample input. If you just want to output 123 A G then print patterns[i] instead of $2.
awk -v locList='123|789' '$2~"^("locList")" {print $2,$4,$5}' file
or if you prefer:
locList='123, 789'
awk -v locList="^(${locList//, /|})" '$2~locList {print $2,$4,$5}' file
or whatever other permutation you like. The point is you don't need a loop at all - just create a regexp from the list of numbers in locList and test that regexp once.
What I would do :
locList="123 789"
for i in $locList; do awk -vvar=$i '$2 ~ var{print $4, $5}' file; done