I am using AWX and trying to run this task:
- name: cleanup
shell: awk '{ a[$1 OFS] = a[$1 OFS] ( a[$1 OFS] == "" ? "" : OFS) $2 }END{ for (i in a){print i,a[i]} } OFS="\t" latest.txt "{{ inventory_hostname }}".txt > ./output/"{{ inventory_hostname }}".txt
But I am getting this Error:
> ERROR! Syntax Error while loading YAML. did not find expected key
> The error appears to be in
> '/tmp/awx_8887_nv3efn13/project/Simsek/test/test.yml': line 141,
> column 60, but may be elsewhere in the file depending on the exact
> syntax problem. The offending line appears to be:
> - name: cleanup
> shell: "awk '{ a[$1 OFS] = a[$1 OFS] ( a[$1 OFS] == "" ? "" : OFS) $2 }END{ for (i in a){print i,a[i]} } OFS="\t" latest.txt "{{
> inventory_hostname }}".txt > ./output/"{{ inventory_hostname }}".txt"
> ^ here We could be wrong, but this one looks like it might be an issue with
> missing quotes. Always quote template expression brackets when they
> start a value. For instance:
> with_items:
> - {{ foo }} Should be written as:
> with_items:
> - "{{ foo }}"
try this:
awk '{ a[$1 OFS] = a[$1 OFS] ( a[$1 OFS] == "" ? "" : OFS) $2 }END{ for (i in a){print i,a[i]} } OFS="\t"' latest.txt {{ inventory_hostname }}.txt > ./output/{{ inventory_hostname }}.txt
Related
I'm trying to get the cassandra schema version into variable from output of nodetool command.
Here are some of the output of nodetool commands:
Cluster Information:
Name: Test Cluster
Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch
Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
Schema versions:
65e78f0e-e81e-30d8-a631-a65dff93bf82: [127.0.0.1]
When few nodes are not reachable here's the output.
Cluster Information:
Name: Production Cluster
Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch
Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
Schema versions:
UNREACHABLE: 1176b7ac-8993-395d-85fd-41b89ef49fbb: [10.202.205.203]
Can anyone suggest how to get schema version into variable irrespective of reachable or not?
Tried to use awk and grep commands but didn't work because of unreachable.
Another version of an awk script that will match only the UUID type REGEX can be written to use match() setting the internal RSTART and RLENGTH variables that can then be used with substr().
That would be:
awk '
/Schema versions:/ {
set=1
next
}
set {
match($0,/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/)
print substr($0, RSTART, RLENGTH)
exit
}' file
Example Use/Output
$ awk '
> /Schema versions:/ {
> set=1
> next
> }
> set {
> match($0,/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/)
> print substr($0, RSTART, RLENGTH)
> exit
> }' << 'eof'
> Cluster Information:
> Name: Test Cluster
> Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch
> Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
> Schema versions:
> 65e78f0e-e81e-30d8-a631-a65dff93bf82: [127.0.0.1]
>
> eof
65e78f0e-e81e-30d8-a631-a65dff93bf82
and
$ awk '
> /Schema versions:/ {
> set=1
> next
> }
> set {
> match($0,/[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/)
> print substr($0, RSTART, RLENGTH)
> exit
> }' << 'eof'
> Cluster Information:
> Name: Production Cluster
> Snitch: org.apache.cassandra.locator.DynamicEndpointSnitch
> Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
> Schema versions:
> UNREACHABLE: 1176b7ac-8993-395d-85fd-41b89ef49fbb: [10.202.205.203]
> eof
1176b7ac-8993-395d-85fd-41b89ef49fbb
You can use the command in a command substitution in bash to capture the result in a variable.
Let me know if you have further questions.
Awk will do the job for that:
version=$(awk '/Schema versions:/ {
getline
gsub(/:/,"")
if ($1 == "UNREACHABLE") {
print $2
} else {
print $1
}
}' < <(nodetool_cmd)) # remplace "nodetool_cmd" by the correct command
$ echo "$version" #when reachable
65e78f0e-e81e-30d8-a631-a65dff93bf82
$ echo "$version" # when unreachable
1176b7ac-8993-395d-85fd-41b89ef49fbb
# or in single line:
version=$(awk '/version/ {getline;gsub(/:/,"");if ($1 == "UNREACHABLE") {print $2} else {print $1}}' < <(nodetool_cmd))
I am having an issue with the following command:
awk ‘{if ($1 ~ /^##contig/) {next}else if ($1 ~ /^#/) {print $0; next}else {print $0 | “sort -k1,1V -k2,2n”}’ file.vcf > out.vcf
It gives the following error:
^ unexpected newline or end of string
Your command contains "fancy quotes" instead of normal ones, in addition to a missing }.
awk '{if ($1 ~ /^##contig/) {next} else if ($1 ~ /^#/) {print $0; next} else {print $0 | "sort -k1,1V -k2,2n"} }' file.vcf > out.vcf
Changing your command to the above should work as expected.
I have a YML file with content similar to,
test:
volumes:
- /u01/test-service/conf:/root/config
testmanager:
port:
- "2222:80"
I want to delete test or testmanager block based on some conditions. Here is the awk expression I found here,
awk '{sub(/\r$/, "")}
$1 == "test:"{t=1}
t==1 && $1 != "test:" {t++; next}
t==2 && /:\s*$/{t=0}
t != 2'
This deletes everything under test but keeps the string "test:". Something like this,
reportservice:
reportmanager:
port:
- "2222:80"
How to fix this? Please help.
With the shown input you can use this awk command with empty RS:
awk -v RS= '!/^[[:blank:]]*test:/' file.yml
testmanager:
port:
- "2222:80"
This assumes there is an empty line between each block. If this doesn't work you can modify your existing command you can do:
awk '{sub(/\r$/, "")}
$1 == "test:"{t=1}
t==1 && $1 != "test:" {t++; next}
t==2 && /:\s*$/{t=0}
!t' file.yml
I have an input file with columns seperated by | as follows.
[3yu23yuoi]|$name
!$fjkdjl|[kkklkl]
$hjhj|$mmkj
I want the output as
0 $name
!$fjkdjl 0
$hjhj $mmkj
Whenever the string begins with $ or !$ or "any", i want it to get printed as such else 0.
I have tried the following command.It prints verything same as input file only.
awk -F="|" '{if (($1 ~ /^.*\$/) || ($1 ~ /^.*\!$/) || ($1 ~ /^any/)) {print $1} else if ($1 ~ /^\[.*/){print "0"} else if (($2 ~ /^.*\$/) || ($2 ~ /^.*\!$/) || ($2 ~ /^any/)) {print $2} else if($2 ~ /^\[.*/){print "0"}}' input > output
This should do:
awk -F\| '{$1=$1;for (i=1;i<=NF;i++) if ($i!~/^(\$|!\$|any)/) $i=0}1' file
0 $name
!$fjkdjl 0
$hjhj $mmkj
If data does not start with $ !$ or any, set it to 0
Or if you like tab as separator:
awk -F\| '{$1=$1;for (i=1;i<=NF;i++) if ($i!~/^(\$|!\$|^any)/) $i=0}1' OFS="\t" file
0 $name
!$fjkdjl 0
$hjhj $mmkj
$1=$1 make sure all line have same output, even if no data is changed.
I have code that looks like this:
awk -F'|' '{if($1 in a)print "duplicate found:" $2 " AND "a[$1];else a[$1]=$2 }' dump.txt
I need to set $2 and a[$2] to a variable. How would I go about doing this?
I am taking a file that contains: a value "|" filename and then I want to set the filename and the value to two different variables.
What do you mean Set those variables? Are these environment variables. In Awk, variables that start with a dollar sign and are numeric are reserved for AWK. These are the field values for each line. For example:
test.txt =
this is line one
this is line two
this is line three
The command awk '{print $4}' test.txt will print out the fourth field:
$ awk '{print $4}' test.txt
one
two
three
$ awk '{print $3}' test.txt
line
line
line
As you can see: They don't have to be set. They're automatically set by Awk.
If you want to set environment variables, you can use the -v parameter
awk -v search="foo" '{
if (search = $1) {
print "Found your string in record " NR
}'
In the above, search is an Awk variable that is set equal to foo.
Since Awk is a programming language, it is sometimes easier to see what's going on by correctly formatting your program:
awk -F'|' '{
if($1 in a) {
print "duplicate found:" $2 " AND " a[$1]
}
else {
a[$1] = $2
}
}' dump.txt
The program is taking each line. Each line consists of two parts separated by the |. It appears that the first part in the key and the second part is the data. I've created a text file that looks like this:
This is a|test
That is a|line
who moans for|anchovies
whom moans for|anchovies
This is a|test again
The first and last line should show up as duplicates
I took your program and added a few debug lines. This will help me trace where it is in your program:
awk -F\| '{
if ($1 in a) {
print "DEBUG: In If clause"
print "duplicate found:" $2 " and " a[$1]
} else {
print "DEBUG: In else clause"
a[$1] = $2
print "DEBUG: a[" $1 "] = " a[$1]
}
print "DEBUG: $1 = " $1
print "DEBUG: $2 = " $2 "\n"
}' test.txt
And, this is my output
DEBUG: In else clause
DEBUG: a[This is a] = test
DEBUG: $1 = This is a
DEBUG: $2 = test
DEBUG: In else clause
DEBUG: a[That is a] = line
DEBUG: $1 = That is a
DEBUG: $2 = line
DEBUG: In else clause
DEBUG: a[who moans for] = anchovies
DEBUG: $1 = who moans for
DEBUG: $2 = anchovies
DEBUG: In else clause
DEBUG: a[whom moans for] = anchovies
DEBUG: $1 = whom moans for
DEBUG: $2 = anchovies
DEBUG: In If clause
duplicate found: test again and test
DEBUG: $1 =This is a
DEBUG: $2 = test again
Taking out the debug lines:
awk -F\| '{
if ($1 in a) {
print "duplicate found:" $2 " and " a[$1]
} else {
a[$1] = $2
}
}' test.txt
duplicate found: test again and test
As they say:
IT WORKS ON MY COMPUTER
(rimshot)
Seriously, what is your program suppose to be doing, and what do you see it do? Are there any errors? Your program appears to work as advertised.
Set them to an environment variable like this:
awk -F'|' '{if($1 in a)print "duplicate found:" '"$2"' " AND "a[$1];else a[$1]='"$2"' }' dump.txt
Note that what I did was to "disable" the single-quotes around the $2 parts, and add double-quotes around them in case the environment variable contains spaces (which awk wouldn't want to see splitting its argument into pieces).