How to use awk in a `for` loop with two loop variables - shell

Question

I have a zsh shell scripting problem :-(
There is a file containing a list of 4 columns :
NAME SURNAME OLD TOWN
DOE John 30 London
CALAS Maria 50 Athens
...
I want to make a treatment of only some "elements" of each line. For example, I don't know if that's possible but it should be like :
for user,livesIn in `cat MyFile | awk '{print $2 $4}'`
echo "My friend $user lives in $livesIn"
done
Of course this code is wrong and I didn't find how to write it correctly.
Do someone knows if that's possible ?
Thanks in advance for your help.

Answer 1

When you want to loop through the output of awk, you can try
awk '{print $2 $4}' MyFile | while read -r user livesIn; do
echo "${user} was last seen in ${livesIn}."
done
In this case awkis not needed:
while read -r field1 user field3 livesIn; do
echo "${user} was last seen in ${livesIn}."
done < MyFile
The constructions above will fail when some field has a space, like New York.
Take a good look at the specifications of your MyFile, how the fields are seperated. Fixed width? TAB-character? With a TAB you are lucky:
while IFS=$'\t' read -r field1 user field3 livesIn; do
echo "${user} was last seen in ${livesIn}."
done < MyFile

Answer 2

awk processes each line one at a time, so no need for a for loop. Also no need for the cat as awk takes the file name as an argument. Try this:
awk '{print "My friend "$2" lives in "$4}' MyFile

Answer 3

If you are already using awk, why don't you process everything with awk?
For example:
$ awk '{print "My friend", $2, "lives in", $4}' MyFile
That gets the output you are looking for. Unless there is something else not stated in the question.

Answer 4

It is usually simpler and more efficient to do both iterating and processing in AWK rather than trying to divide the task by iterating in the shell and processing in AWK. AWK is well designed for iterating through input, and it also has its own loop structures (e.g. for). If at all possible, do all your processing in AWK.
That said, it seems that your problem also requires access to the input fields in the shell, and so full processing in AWK may not be possible in your case. (It is worth noting that AWK can also execute shell commands, but this may be just another level of complication.)
Other answers use AWK to iterate through the file and print something with columns 2 and 4, like
$ awk '{print "My friend", $2, "lives in", $4}' MyFile
which is fine if you can iterate and process with AWK like this. As an addition to this type of solution, you might want to skip the first line (which seems to have column headers instead of actual data) with
$ awk 'NR>1{print "My friend", $2, "lives in", $4}' MyFile
Your comment
In fact my treatment is a little bit more complicated than a print. I need to make some tests and assignments in the "for" loop.
suggests that what you really want is access to the fields in your shell. You can get this by using AWK to pick out the fields (as before), but piping the values into the shell:
awk 'NR>1{print $2,$4}' MyFile | while read user livesIn; do
echo "My friend $user lives in $livesIn"
done
This gives you $user and $livesIn in the shell, and so you can do more complicated shell processing with it. For example:
awk 'NR>1{print $2,$4}' MyFile | while read user livesIn; do
if [[ "$user" == "John" ]]; then
echo "$user is not my friend, but lives in $livesIn"
else
echo "My friend $user lives in $livesIn"
echo "$user" >> friends.txt
fi
done
Be careful with the format of your input file since AWK is splitting on white space.