Sorting dates by groups - bash

Question

Here is a sample of my data with 4 columns and comma delimiter.
1,A,2009-01-01,2009-07-15
1,A,2009-07-10,2009-07-12
2,B,2009-01-01,2009-07-15
2,B,2009-07-10,2010-12-15
3,C,2009-01-01,2009-07-15
3,C,2009-07-15,2010-12-15
3,C,2010-12-15,2014-07-07
4,D,2009-06-01,2009-07-15
4,D,2009-07-21,2012-12-15
5,E,2011-04-23,2012-10-19
The first 2 columns are grouped. I want the minimum date from the third column, and the maximum date from the fourth column, for each group.
Then I will pick the first line for each first 2 column combination.
Desired output
1,A,2009-01-01,2009-07-15
2,B,2009-01-01,2010-12-15
3,C,2009-01-01,2014-07-07
4,D,2009-06-01,2012-12-15
5,E,2011-04-23,2012-10-19
I have tried the following code, but not working. I get close, but not the max date.
cat exam |sort -t, -nk1 -k2,3 -k4,4r |sort -t, -uk1,2
Would prefer an easy one-liner like above.

Answer 1

sort datafile |
awk -F, -v OFS=, '
{key = $1 FS $2}
key != prev {prev = key; min[key] = $3}
{max[key] = ($4 > max[key]) ? $4 : max[key]}
END {for (key in min) print key, min[key], max[key]}
' |
sort
1,A,2009-01-01,2009-07-15
2,B,2009-01-01,2010-12-15
3,C,2009-01-01,2014-07-07
4,D,2009-06-01,2012-12-15
5,E,2011-04-23,2012-10-19
When you pre-sort, you are guaranteed that the minimum col3 date will occur on the first line of a new group. Then you just need to find the maximum col4 date.
The final sort is required because iterating over the keys of an awk hash is unordered. You can do this sorting in (g)awk with:
END {
n = asorti(min, sortedkeys)
for (i=1; i<=n; i++)
print sortedkeys[i], min[sortedkeys[i]], max[sortedkeys[i]]
}

Answer 2

#!/usr/bin/awk -f
BEGIN { FS = OFS = "," }
{
sub(/[[:blank:]]*<br>$/, "")
key = $1 FS $2
if (!a[key]) {
a[key] = $3
b[key] = $4
keys[++k] = key
} else if ($3 < a[key]) {
a[key] = $3
} else if ($4 > b[key]) {
b[key] = $4
}
}
END {
for (i = 1; i <= k; ++i) {
key = keys[i]
print key, a[key], b[key] " <br>"
}
}
Usage:
awk -f script.awk file
Output:
1,A,2009-01-01,2009-07-15 <br>
2,B,2009-01-01,2010-12-15 <br>
3,C,2009-01-01,2014-07-07 <br>
4,D,2009-06-01,2012-12-15 <br>
5,E,2011-04-23,2012-10-19 <br>
Of course you can add print statements before and after the loop to print the other two <br>'s:
END {
print "<br>"
for (i = 1; i <= k; ++i) {
key = keys[i]
print key, a[key], b[key] " <br>"
}
print "<br>"
}

Answer 3

You want a "one liner" ?
paste -d, \
<(cat exam|sort -t, -nk1,2 -k4 |cut -d, -f1-3) \
<(cat exam|sort -t, -nk1,2 -k4r |cut -d, -f4 ) |
uniq -w4
The key idea is to sort the data once by field 3 asc, and independently by field 4 desc. Then you just have to merge corresponding lines (cut and paste). Finally uniq is used to keep only the first row for each pair of identical first two columns. This is the weak point here, as I assume 4 characters max for comparison. You either have to adjust to your needs, or somehow normalize data for those two columns in order to have a fixed width here when using your actual data.
EDIT: A probably better option is to replace uniq by a simple awk filter:
paste -d, \
<(cat exam|sort -t, -nk1,2 -k4 |cut -d, -f1-3) \
<(cat exam|sort -t, -nk1,2 -k4r |cut -d, -f4 ) |
awk -F , '$1","$2 != last { print; last=$1","$2 }'
On my system (GNU Linux Debian Wheezy), both produce the same result:
1,A,2009-01-01,2009-07-15<br>
2,B,2009-01-01,2010-12-15<br>
3,C,2009-01-01,2014-07-07<br>
4,D,2009-06-01,2012-12-15 <br>
5,E,2011-04-23,2012-10-19<br>