how to substite last occurrance of pattern in large file efficently - windows

Question

Given a file with the following contents:
<root>
<a></a>
<b></b>
</root>
The command should output:
<root>
<a></a>
<b></b>
Things I've tried using the GNU Win32 port of sed:
Remove the last two lines.
This is fast, but it assumes </root> is the second to last line and will cause a bug if it's not.
sed -e '$d' test.xml | sed -e '$d'
Substituting all occurrences of </root> with an empty string.
This works, but is slower than the first solution, and will break if there are nested <root> elements (unlikely).
sed -e 's|</root>||' test.xml
The file I'm dealing with can be large so efficiency is important.
Is there a way to limit sed substitution to the last occurrence in the file? Or is there some other utility that would be faster?

Answer 1

Using Perl with File::Backwards should be very fast (relative, I know, but still...). Perlfaq5 has a topic on going through a file backwards and removing lines. You can check for your pattern using this topic's code as a starting point.

Answer 2

With sed:
sed -e ':a;N;$!ba;s|\(.*\)</root>\n\(.*\)|\1\2|'

Answer 3

How about using awk for this.
AWK:
awk '/^<\/root>$/{next}/<\/root>/{sub(/<\/root>/,"");print;next}1' filename
First /pattern/{action} statement looks for lines with only </root>. It pattern finds it, action ignores it.
Second /pattern/{action} statement looks for lines containing </root> anywhere in the line. If pattern finds it, sub function replaces it with nothing and prints rest of the line.
Third action which is 1 is true for all the lines that does not have pattern </root> in them. If it finds it, it prints it.
I did a quick test and this was the result -
Test:
[jaypal:~/Temp] cat tmp
<root>
<a></a>
<b></b>
</root>
<root>
<a></a>
<b></b>
</root><root>
<a></a>
<b></b></root>
[jaypal:~/Temp] awk '/^<\/root>$/{next}/<\/root>/{sub(/<\/root>/,"");print;next}1' tmp
<root>
<a></a>
<b></b>
<root>
<a></a>
<b></b>
<root>
<a></a>
<b></b>
SED:
This should also work. Though it will remove all </root> and not just the last occurrence.
sed '/<\/root>/,$s///' filename

Answer 4

This might work for you:
sed '/<\/root>/,/<root>/{/<\/root>/{h;d};H;//{x;p};${x;s/[^\n]*\n//p};d}' file
This assumes that each <root> tag is matched with a closing </root> tag and that these tags occur on separate lines (as per the example).
Explanation:
Focus on lines between a closing </root> tag and an opening <root> tag or end-of-file.
If it is a closing </root> tag, save it in the hold space (HS) and then delete it and start a new cycle.
For all other lines within focus (see point 1) append them to the HS.
If it is and opening <root> tag, swap to the HS and print out its contents.
If it is the end-of-file i.e. between a </root> tag and last line of the file, swap to the HS, delete the first line i.e. the closing </root> tag and print the remainder.
For all lines within the focus, delete and start a new cycle.
An alternative solution with two passes:
sed -n '/<\/root>/=' file | sed -n '$s/$/d/p' | sed -f - file
Explanation:
Print out the line numbers of closing </root> tags
Generate a sed delete command from the last matched line number.
Pipe the command through to an instance of sed reading the source file.

Answer 5

Use time function to see which one is efficient. sed should be efficient.
$time command
In my opinion, there is nothing which is faster than grep. try it with awk index() to see if it is any faster.