Add line numbers to final nonempty lines using bash - bash

Question

Is there a way to add line numbers to a file, but only have it start from the line after the last blank line? For example
aaa
bbb
ccc
ddd
eee
would become
aaa
bbb
ccc
1. ddd
2. eee
since the line ddd is the first line after the last blank line.
Right now, I'm going through file by file to do this using vim (by selecting the lines and doing a quick command), but I have 1,000 files I need to run through and I'd prefer not having to do it individually by hand, but can't think of how to get around it.

Answer 1

You can do it simply with awk and three-rules, including the END rule to number the final group of lines, e.g.
awk '
NF > 0 { a[++n]=$0 }
NF == 0 { for(i=1; i<=n; i++) print a[i]; print""; n=0 }
END { for(i=1; i<=n; i++) printf "%d. %s\n", i, a[i]}
' file
Explanation
For the first rule NF > 0, if there is at least one field (line non-empty), store the line in the array a and pre-increment counter n (to keep consistent with awk 1 to NF indexing)
For the second rule NF == 0 if the line is blank, output what you have stored in a and then output an empty-line and reset n to zero;
Finally, in the END rule, number and output all lines stored in a.
Example Use/Output
$ awk '
> NF > 0 { a[++n]=$0 }
> NF == 0 { for(i=1; i<=n; i++) print a[i]; print""; n=0 }
> END { for(i=1; i<=n; i++) printf "%d. %s\n", i, a[i]}
> ' file
aaa
bbb
ccc
1. ddd
2. eee

Answer 2

Here is a solution with awk.
awk '!NF{x=NR} {r[NR]=$0}
END {for (i=1;i<=NR;i++) print (i>x? (++n)". "r[i]: r[i])}' file
We store the rows to array. At the END, x will be the last blank number line, so we print the numbering for line numbers greater than x.

Answer 3

Here is a two pass version that does not require reading the entire file into memory. If it is a large file, you might want to consider memory:
awk ' NR==FNR {if (/^[ \t]*$/) ll=FNR; next}
FNR>ll {c++}
{printf "%s%s\n", (c ? c "." OFS : ""), $0}' file file
Prints:
aaa
bbb
ccc
1. ddd
2. eee
With sed and tail you can calculate the last blank line like so:
$ sed -n '/^[[:blank:]]*$/=' file | tail -1
5
Which you can use to set a variable for awk as well:
awk -v ll=$(sed -n '/^[[:blank:]]*$/=' file | tail -1) '
FNR>ll {c++}
{printf "%s%s\n", (c ? c "." OFS : ""), $0}' file
In any case, you can only do one of two things: Hold the entire file in memory and print after reading it all or read it twice. In most cases, it is better to read it twice since it will not impact memory and it is usually just as fast (since most OSs will cache the file.)
Just to compare some timings of read whole file vs read it twice consider a 100 line file (likely too fast to accurate measure):
$ awk 'BEGIN {for (i=1; i<=101; i++) print i%3 ? i : ""}' >/tmp/file
Now time that with the read all vs read twice solutions:
time awk -v ll=$(sed -n '/^[[:blank:]]*$/=' file | tail -1) '
FNR>ll {c++}
{printf "%s%s\n", (c ? c "." OFS : ""), $0}' file > /dev/null
time awk '
NF > 0 { a[++n]=$0 }
NF == 0 { for(i=1; i<=n; i++) print a[i]; print""; n=0 }
END { for(i=1; i<=n; i++) printf "%d. %s\n", i, a[i]}
' file > /dev/null
time awk ' NR==FNR {if (/^[ \t]*$/) ll=FNR; next}
FNR>ll {c++}
{printf "%s%s\n", (c ? c "." OFS : ""), $0}' file file > /dev/null
Prints:
real 0m0.012s
user 0m0.004s
sys 0m0.007s
real 0m0.007s
user 0m0.003s
sys 0m0.003s
real 0m0.007s
user 0m0.003s
sys 0m0.003s
Now try with 100,001 lines:
awk 'BEGIN { for (i=1; i<=100001; i++) print i%3 ? i : ""}' >/tmp/file
time awk -v ll=$(sed -n '/^[[:blank:]]*$/=' file | tail -1) '
FNR>ll {c++}
{printf "%s%s\n", (c ? c "." OFS : ""), $0}' file > /dev/null
time awk '
NF > 0 { a[++n]=$0 }
NF == 0 { for(i=1; i<=n; i++) print a[i]; print""; n=0 }
END { for(i=1; i<=n; i++) printf "%d. %s\n", i, a[i]}
' file > /dev/null
time awk ' NR==FNR {if (/^[ \t]*$/) ll=FNR; next}
FNR>ll {c++}
{printf "%s%s\n", (c ? c "." OFS : ""), $0}' file file > /dev/null
Times:
real 0m0.081s
user 0m0.079s
sys 0m0.007s
real 0m0.047s
user 0m0.042s
sys 0m0.004s
real 0m0.058s
user 0m0.052s
sys 0m0.004s
Now 10,000,001 lines:
awk 'BEGIN { for (i=1; i<=10000001; i++) print i%3 ? i : ""}' >/tmp/file
time awk -v ll=$(sed -n '/^[[:blank:]]*$/=' file | tail -1) '
FNR>ll {c++}
{printf "%s%s\n", (c ? c "." OFS : ""), $0}' file > /dev/null
time awk '
NF > 0 { a[++n]=$0 }
NF == 0 { for(i=1; i<=n; i++) print a[i]; print""; n=0 }
END { for(i=1; i<=n; i++) printf "%d. %s\n", i, a[i]}
' file > /dev/null
time awk ' NR==FNR {if (/^[ \t]*$/) ll=FNR; next}
FNR>ll {c++}
{printf "%s%s\n", (c ? c "." OFS : ""), $0}' file file > /dev/null
Times:
real 0m6.766s
user 0m7.671s
sys 0m0.063s
real 0m3.950s
user 0m3.921s
sys 0m0.026s
real 0m4.801s
user 0m4.754s
sys 0m0.041s
So surprisingly, it is slightly faster for a smaller file to read it twice and slightly faster to hold it in memory if larger. This is on a computer with 64GB RAM and an SSD. Smaller memory or slower hard drives would change this.