I would like to know if it is possible to change the following script, such that "intermediate.tmp" is not generated as output:
To call the script on the command line:
./script.sh file1 file2
script.sh:
#!/bin/bash
FILE_1=$1
FILE_2=$2
awk '{print $1,$2}' $FILE_1 > intermediate.tmp
awk 'NR==FNR {h[$1] = $0; next} {print $0,h[$1]}' intermediate.tmp $FILE_2 > output.file
The awk scripts are not really important per se. I just want to know how to "feed" intermediate.tmp into the second awk command without generating an intermediate.tmp output file in addition to the desired output.file.
Thanks.
awk 'NR==FNR {h[$1] = $1 OFS $2; next} {print $0,h[$1]}' "$FILE_1" "$FILE_2" > output.file
or less sensibly:
awk '{print $1,$2}' "$FILE_1" |
awk 'NR==FNR {h[$1] = $0; next} {print $0,h[$1]}' - "$FILE_2" > output.file
Related
I have a little test file containing:
awk this
and not awk this
but awk this
so do awk this
And I've tried the following awk commands, in bash, but each produces no output:
f=awk; awk '/$f/ && !/not/' test.txt
f=awk; awk '/\$f/ && !/not/' test.txt
f=awk; awk '/"$f"/ && !/not/' test.txt
f=awk; awk -v f="$f" '/f/ && !/not/' gtest.txt
Using double quotes " produces "event not found" error in the shell due to the !.
How can I search on a variable and negate another string in the same command?
Use awk like this:
f='awk'
awk -v f="$f" -v n='not' '$0 ~ f && $0 !~ n' file
awk this
but awk this
so do awk this
Or if you don't want to pass n='not' to awk:
awk -v f="$f" '$0 ~ f && $0 !~ /not/' file
awk this
but awk this
so do awk this
awk points to gawk for me and the following worked just fine:
awk -vf=awk '$0 ~ f && !/not/' file
Info 750: local macro 'ADD_GEN_METHOD' (line 149, file /home/vya3kor/vmshare/vya3kor_rbin_g3g_tas_lcmccatestadapter.vws/di_cfc/components/spm/LcmProject/framework/cca/server/generic/spm_CcaServiceHandlerFiParamConfig.cpp) not referenced
I want to separate above line using awk with field-separator (line
I used this command but it's not working
$ grep Info.* 1.txt |awk -F "(line" '{print $1}'
error : awk: fatal: Unmatched ( or \(: /(line/
output I want:
/di_cfc/components/spm/LcmProject/framework/cca/server/generic/spm_CcaServiceHandlerFiParamConfig.cpp%149%Info 750%local macro 'ADD_GEN_METHOD'%
So I used this command :
$ grep '^[Ii]nfo.*:'|
awk -F ":" '{print $1"%" $2}'|
awk -F ", file.*.vws" '{print $1"%" $2 }'|
awk -F ") not referenced" '{print $1"%" }'|
awk -F '(' '{print $1"%" $2"%" $3}'|
awk -F "line" '{print $1 $2 $3 }' |
awk -F "%" '{print $1$ "\n2" $3 $4 $4 $5}'
You can use this awk:
awk -F '\\(line' '{print $1}'
Info 750: local macro ADD_GEN_METHOD
( is special regex symbol that needs to be escaped.
instead of escaping the (, you can do in this way:
awk -F'[(]line' '... your codes'
personally I think it is easier to read.
You need to use three backslashes if the Field Seperator was set through -v,
$ echo 'Info 750: local macro 'ADD_GEN_METHOD' (line 149, file /home/vya3kor/vmshare/vya3kor_rbin_g3g_tas_lcmccatestadapter.vws/di_cfc/components/spm/LcmProject/framework/cca/server/generic/spm_CcaServiceHandlerFiParamConfig.cpp) not referenced' | awk -v FS="\\\(line" '{print $1}'
Info 750: local macro ADD_GEN_METHOD
With all that chopping, you may be better off with sed:
sed -n '/^[Ii]nfo/s/\(Info.*\): \([^(]*\).*file \([^)]*)\) .*/\3%\1%\2/gp' 1.txt
$ cat file
Info 750: local macro 'ADD_GEN_METHOD' (line 149, file /home/vya3kor/vmshare/vya3kor_rbin_g3g_tas_lcmccatestadapter.vws/di_cfc/components/spm/LcmProject/framework/cca/server/generic/spm_CcaServiceHandlerFiParamConfig.cpp) not referenced
$
$ cat tst.awk
BEGIN{ FS=" *[()] *"; OFS="%" }
/^[Ii]nfo.*:/ {
split($2,a,/[ ,]+/)
sub(/: /,OFS,$1)
sub(/.*\.vws/,"",$2)
print $2, a[2], $1 OFS
}
$
$ awk -f tst.awk file
/di_cfc/components/spm/LcmProject/framework/cca/server/generic/spm_CcaServiceHandlerFiParamConfig.cpp%149%Info 750%local macro 'ADD_GEN_METHOD'%
$
or you could do it all in GNU awk with one gensub() call or just use sed as #chthonicdaemon suggested since this is just a simple substitution on a single line.
May be I'm asking a trivial question, but can't find the right way.
I'm ssh'ing in several servers and comparing expected nfs mounts from /etc/fstab with existing mounts from /proc/mounts.
VAR1=$(awk '!/^#/ && $3 == "nfs" {print $2}' /etc/fstab)
VAR2=$(awk '!/^#/ && $3 ~ /nfs[34]/ && $1 !~ /gfs/ {print $2}' /proc/mounts)
For example, from /etc/fstab :
/data1
/data2
/data3
And from /proc/mounts :
/data1
/data2
If all mounts exist and mounted I have to print that all OK ,and if some missing to print them out and remount.
I tried to work for comparison with :
awk 'FNR==NR {a[$1]; next} $1 in a' file1 file2 ( but not works with $VAR1/$VAR2 ).
With nested loops also didn't success.
Thanks in advance.
You can get the missing mounts like this:
comm -23 <(printf '%s\n' "$VAR1") <(printf '%s\n' "$VAR2")
This prints the lines which are only in $VAR1.
Or as a one-liner:
comm -23 <(awk '!/^#/ && $3 == "nfs" {print $2}' /etc/fstab) <(awk '!/^#/ && $3 ~ /nfs[34]/ && $1 !~ /gfs/ {print $2}' /proc/mounts)
You can get the result from $VAR1 and $VAR2 this way:
for i in $VAR1 $VAR2; do echo $i; done \
| sort | uniq -c | awk '/\s+1/ {print $2}'
Basically, we're looking for the directories that are only mentioned once. There may be a more elegant way to do this, but that should get you what you want.
I got some one liners in awk. How can i figure this three line into a script?
awk -F":|," 'FNR==NR && /INFO - AId:/ {a[$6$8]=$0;next} END {for (i in a) print i "|" a[i]}' log >t1
awk '/<?xml version/ {f=1} /<\/iSig>/ {f=0;print $0 "\n" } f' log >t2
awk -F\| 'FNR==NR {a[$1]=$2;next} FNR==1 {RS="\n\n"} { for (i in a) {if ($0~i) {print a[i] $0 > i".log";close(i".log")}}}' t1 t2
Thanks for helping!
How can I figure this three line into a script?
By learning awk! The best place to start is by reading Effective Awk Programming.
$ cat > myscript.sh <<EOF
#!/bin/sh
awk -F":|," 'FNR==NR && /INFO - AId:/ {a[$6$8]=$0;next} END {for (i in a) print i "|" a[i]}' log > $1
awk '/<?xml version/ {f=1} /<\/iSig>/ {f=0;print $0 "\n" } f' log >$2
awk -F\| 'FNR==NR {a[$1]=$2;next} FNR==1 {RS="\n\n"} { for (i in a) {if ($0~i) {print a[i] $0 > i".log";close(i".log")}}}' $1 $2
EOF
$ chmod +x myscript.sh
$ ./myscript.sh file1 file2
First as bash you could do it like this:
awk -F\| 'FNR==NR {a[$1]=$2;next} FNR==1 {RS="\n\n"} { for (i in a) {if ($0~i) {print a[i] $0 > i".log";close(i".log")}}}' <(awk -F":|," 'FNR==NR && /INFO - AId:/ {a[$6$8]=$0;next} END {for (i in a) print i "|" a[i]}' log) <(awk '/<?xml version/ {f=1} /<\/iSig>/ {f=0;print $0 "\n" } f' log)
I have CSVs like this:
apple,file1.txt
banana,file1.txt
carrot,file2.txt
How can I get it to place all of the items from the left column into files named with the items in the right column? E.g. file.txt would contain this list:
apple
banana
So far, I have this:
while read line
do
firstcolumn=$(echo $line | awk -F ",*" '{print $1}')
secondcolumn=$(echo $line | awk -F ",*" '{print $2}')
done < Text/selection.csv
One way using awk:
awk 'BEGIN { FS = "," } { print $1 >> $2 }' infile
This should work -
awk -F, '{a[$1]=$2} END{for (i in a) print i > a[i]}' file
Test:
[jaypal:~/Temp] cat file
apple,file1.txt
banana,file1.txt
carrot,file2.txt
[jaypal:~/Temp] awk -F, '{a[$1]=$2} END{for (i in a) print i > a[i]}' file
[jaypal:~/Temp] ls file*
file file1.txt file2.txt
[jaypal:~/Temp] cat file1.txt
apple
banana
[jaypal:~/Temp] cat file2.txt
carrot
Update:
You can also do something like this -
awk -F, '{print $1 > $2}' INPUT_FILE
Pure Bash and under the assumption that all target files are empty or non-existing:
while IFS=',' read item file ; do
echo "$item" >> "$file"
done < "$infile"
sed loves this stuff...
sed "s%\(.*\),\(.*\)%echo \1 >> \2 %" inputfile.txt | sh