How can I pass a shell variable to awk, set it, use it in another awk in same line and print it?
I want to save $0 (all fields) into a variable first, parse $6 (ABC 123456M123000) - get '12300', do a range check on it and if it satisfies, print all fields ($0)
part 1: I am trying to do:
line="hello"
java class .... | awk -F, -v '{line=$0}' | awk 'begin my range check code' | if(p>100) print $line }
part2:
$6="ABC 123456M123000" ( string that I will parse)
Once I store all fields into a variable, I can parse $6 using this:
awk 'begin {FS=" "} { print $2; len=length($2); p=substr($2,8,len)+0 ; print len,p ; if(p>100) print $line }'
But my question is in part1: how to store $0 into a variable so that after my check is done, I can print them?
It's not clear why you need multiple invocations of awk. From your description, it looks like you are just trying to do:
... | awk -F, '{split( $6, f, "M" )} f[2] > min' min=100
or, if you can't split on 'M' but need to use substr (or some other method to extract the desired value):
... | awk -F, '{ split( $6, f, " " )} 0+substr( f[2], 8 ) > min' min=100
With the shell:
java ... | while IFS= read -r line ; do
sixth=$(IFS=,; set -- $line; echo "$6")
val=${sixth:11}
(( $val > 100 )) && echo "$line"
done
Some bash-isms there.
Related
I'm trying to edit 3 columns in a file if the value in column 1 equals a specific string. This is my current attempt:
cp file file.copy
awk -F':' 'OFS=":" { if ($1 == "root1") $2="test"; print}' file.copy>file
rm file.copy
I've only been able to get the awk command working with one column being changed, I want to be able to edit $3 and $8 as well. Is this possible in the same command? Or is it only possible with separate awk commands or with a different command all together?
Edit note: The real command i'll be passing variables to the columns, i.e. $2=$var
It'll be used to edit the /etc/passwd file, sample input/output:
root:$6$fR7Vrjyp$irnF38R/htMSuk0efLSnAten/epf.5v7gfs0q.NcjKcFPeJmB/4TnnmgaAoTUE9.n4p4UyWOgFwB1guJau8AL.:17976::::::
You can create multiple statements for the if condition with a block {}.
awk -F':' 'OFS=":" { if ($1 == "root1") {$2="test"; $3="test2";} print}' file.copy>file
You can also improve your command by using awk's default "workflow": condition{commands}. For this you need to bring the OFS to the input variables (-v flag)
awk -F':' -v OFS=":" '$1=="root1"{$2="test"; $3="test2"; print}' file.copy>file
You may use
# Fake sample values
v1=pass1
v2=pass2
awk -v var1="$v1" -v var2="$v2" 'BEGIN{FS=OFS=":"} $1 == "root1" { $2 = var1; $3 = var2}1' file > tmp && mv tmp file
See the online awk demo:
s="root1:xxxx:yyyy
root11:xxxx:yyyy
root1:zzzz:cccc"
v1=pass1
v2=pass2
awk -v var1="$v1" -v var2="$v2" 'BEGIN{FS=OFS=":"} $1 == "root1" { $2 = var1; $3 = var2}1' <<< "$s"
Output:
root1:pass1:pass2
root11:xxxx:yyyy
root1:pass1:pass2
Note:
-v var1="$v1" -v var2="$v2" pass the variables you need to use in the awk command
BEGIN{FS=OFS=":"} set the field separator
$1 == "root1" check if Field 1 is equal to some value
{ $2 = var1; $3 = var2 } set Field 2 and 3 values
1 calls the default print command
file > tmp && mv tmp file helps you "shrink" the "replace-inplace-like" code.
I have to print out some values in a txt file.
they are of the following format
input="Sno;Name;Field1;Field2"
However the output must be:
Sno-Name
FIELDS ALLOCATED:
Field1
Field2
I do it like so:
echo $input | $(awk -F';' '{print $1"-"$2}') >>$txtfile
echo "FIELDS ALLOCATED:">>$txtfile
echo "$input" | cut -d';' -f 3,4 >>$txtfile
This is easy. However, the problem is that Field1 or Field2 can contain new lines. Whenever this happens, the cut or awk doesn't read the field number 4 and treats it as a new line. Do help how can I print the two fields (with new lines preserved) from the given input format.
If the input is well-formed, you can collect input lines until you have four fields.
awk -F ';' 'r { $0 = r ORS $0 }
NR<4 { next }
{ print $1 "-" $2
print "FIELDS ALLOCATED:"
print $3; print $4
print ""; r="" }' file
Single gnu-awk can do the job with FPAT and empty RS:
input=$'Sno;Name;Field1\nFoo;Field2'
awk -v RS= -v FPAT='[^;]+' '{
printf "%s-%s\nFIELDS ALLOCATED:\n%s\n%s\n", $1, $2, $3, $4}' <<< "$input"
Sno-Name
FIELDS ALLOCATED:
Field1
Foo
Field2
Just change the input record separator in awk - RS. < and > added around each field for clarity.
EDIT: removed extra trailing newline by adding ';' at the end of the here-doc data, plus another condition.
input="Sno;Name;Fie
ld1;Fi
eld2"
awk 'BEGIN{RS=";"} NR==1{f1=$0};
NR==2{print f1 "-" $0; print "FIELDS ALLOCATED:"}
$0=="\n"{next}
NR>2{print "<" $0 ">"}' <<< "$input;"
Gives:
Sno-Name
FIELDS ALLOCATED:
<Fie
ld1>
<Fi
eld2>
input=$'Sno;Name;Field1\nFoo;Field2'
awk 'BEGIN{ RS = "\n\n+" ; FS = ";" } { print $1"-"$2; for(i=3;i<=NF;i++) {print $i}}' <<<"$input"
Since it does not know how many field I can give, i added a for loop until NF and changed the RS to a blank line instead of newline.
I have this simple awk code:
awk -F, 'BEGIN{OFS=FS} {print $2,$1,$3}' $1
Works great, except I've hardcoded how I want to sort the comma-delimited fields of my plaintext file. I want to be able to specify at run time in which order I'd like to sort my fields.
One hacky way I thought about doing this was this:
read first
read second
read third
TOTAL=$first","$second","$third
awk -F, 'BEGIN{OFS=FS} {print $TOTAL}' $1
But this doesn't actually work:
awk: illegal field $(), name "TOTAL"
Also, I know a bit about awk's ability to accept user input:
BEGIN {
getline first < "-"
}
$1 == first {
}
But I wonder whether the variables created can in turn be used as variables in the original print command? Is there a better way?
You have to let bash expand $TOTAL before awk is called, so that awk sees the value of $TOTAL, not the literal string $TOTAL. This means using double, not single, quotes.
read first
read second
read third
# Dynamically construct the awk script to run
TOTAL="\$$first,\$$second,\$$third"
SCRIPT="BEGIN{OFS=FS} {print $TOTAL}"
awk -F, "$SCRIPT" "$1"
A safer method is to pass the field numbers as awk variables.
awk -F, -v c1="$first" -v c2="$second" -v c3="$third" 'BEGIN{OFS=FS} {print $c1, $c2, $c3}' "$1"
All you need is:
awk -v order='3 1 2' 'BEGIN{split(order,o)} {for (i=1;i<=NF;i++) printf "%s%s", $(o[i]), (i<NF?OFS:ORS)}'
e.g.:
$ echo 'a b c' | awk -v order='3 1 2' 'BEGIN{split(order,o)} {for (i=1;i<=NF;i++) printf "%s%s", $(o[i]), (i<NF?OFS:ORS)}'
c a b
$ echo 'a b c' | awk -v order='2 3 1' 'BEGIN{split(order,o)} {for (i=1;i<=NF;i++) printf "%s%s", $(o[i]), (i<NF?OFS:ORS)}'
b c a
I'm trying to take last value in third column of a CSV file and replace then the whole third column with this value.
I've been trying this:
var=$(tail -n 1 math_ready.csv | awk -F"," '{print $3}'); awk -F, '{$3="$var";}1' OFS=, math_ready.csv > math1.csv
But it's not working and I don't understand why...
Please help!
awk '
BEGIN { ARGV[2]=ARGV[1]; ARGC++; FS=OFS="," }
NR==FNR { last = $3; next }
{ $3 = last; print }
' math_ready.csv > math1.csv
The main problem with your script was trying to access a shell variable ($var) inside your awk script. Awk is not shell, it is a completely separate language/tool with it's own namespace and variables. You cannot directly access a shell variable in awk, just like you couldn't access it in C. To access the VALUE of a shell variable you'd do:
shellvar=27
awk -v awkvar="$shellvar" 'BEGIN{ print awkvar }'`
Some additional cleanup:
When FS and OFS have the same value, don't assign them each to that value separately, use BEGIN{ FS=OFS="," } instead for clarity and maintainability.
Do not iniatailize variables AFTER the script that uses those variables unless you have a very specifc reason to do so. Use awk -F... -v OFS=... 'script' to init those variables to separate values, not awk -F... 'script' OFS=... as it's very unnatural to init variables in the code segment AFTER you've used them and variables inited in the args list at the end are not initialized when the BEGIN section is executed which can cause bugs.
A shell variable is not expandable internally in awk. You can do this instead:
awk -F, -v var="$var" '{ $3 = var } 1' OFS=, math_ready.csv > math1.cs
And you probably can simplify your code with this:
awk -F, 'NR == FNR { r = $3; next } { $3 = r } 1' OFS=, math_ready.csv math_ready.csv > math1.csv
Example input:
1,2,1
1,2,2
1,2,3
1,2,4
1,2,5
Output:
1,2,5
1,2,5
1,2,5
1,2,5
1,2,5
Try this one liner. It doesn't depend on the column count
var=`tail -1 sample.csv | perl -ne 'm/([^,]+)$/; print "$1";'`; cat sample.csv | while read line; do echo $line | perl -ne "s/[^,]*$/$var\n/; print $_;"; done
cat sample.csv
24,1,2,30,12
33,4,5,61,3333
66,7,8,91111,1
76,10,11,32,678
Out:
24,1,2,30,678
33,4,5,61,678
66,7,8,91111,678
76,10,11,32,678
I have a string containing this (field separator is the percentage sign), stored in a variable called data
201%jkhjfhn%kfhngjm%mkdfhgjdfg%mkdfhgjdfhg%mkdhfgjdhfg%kdfhgjgh%kdfgjhgfh%mkfgnhmkgfnh%k,gnhjkgfn%jkdfhngjdfng
I'm trying to print out that string replacing the percentage sign with a pipe but it seems harden than i thought:
echo ${data} | awk -F"%" 'BEGIN {OFS="|"} {print $0}'
I know I'm very close to it just not close enough.
I see that code as:
1 echo the variable value into a awk session
2 set field separator as "%"
3 set as output field separator "|"
4 print the line
Try this :
echo "$data" | awk -F"%" 'BEGIN {OFS="|"} {$1=$1; print $0}'
From awk manual
Finally, there are times when it is convenient to force awk to rebuild the entire
record, using the current value of the fields and OFS. To do this, use the seemingly
innocuous assignment:
$1 = $1 # force record to be reconstituted
print $0 # or whatever else with $0
Another lightweight way using only tr if you search an alternative for awk :
tr '%' '|' <<< "$data"
Sputnick gave you the awk solution, but you don't actually need awk at all, just use your shell:
echo ${data//%/|}