File IO read by delimeter? - ruby

Question

I have the following script that reads a file and then puts it in an array based on line ends with a </h1>. How do I read only the contents between <h1> and </h1>?
This is my script:
out_array = []
open('foo.html') do |f|
f.each('</h1>') do |record|
record.gsub!("\n", ' ')
out_array.push record
end
end
# print array
p out_array
This my html
</h1>
akwotdfg
<h1>
<h1>I am foo</h1>
<h1>
Stubborn quaz
</h1>
<h3>
iThis
is a reas
long one line shit
</h3>
<h1>I am foo</h1>
This is my output:
["</h1>", " akwotdfg <h1> <h1>I am foo</h1>", " <h1> Stubborn quaz </h1>", " <h3> iThis is a reas long one line shit </h3> <h1>I am foo</h1>", " "]

Answer 1

Please take a look of following code:
out_array = open('foo.html') do |f|
f.read.scan(/<h1>(.*)<\/h1>/)
end
puts out_array
execution result:
I am foo
I am foo
updated for multi-line scan:
out_array = open('tempdir/foo.html') do |f|
f.read.scan(/<h1>([^<]*?)<\/h1>/m)
end
out_array.map! {|e| e[0].strip}
p out_array
execution result:
["I am foo", "Stubborn quaz", "I am foo"]

Answer 2

Don't use regular expressions to deal with HTML or XML. For trivial content you manage it's possible, but your code becomes liable to break for anything that can change at someone else's bidding.
Instead use a parser, like Nokogiri:
require 'nokogiri'
html = '
</h1>
akwotdfg
<h1>
<h1>I am foo</h1>
<h1>
Stubborn quaz
</h1>
<h3>
iThis
is a reas
long one line
</h3>
<h1>I am foo</h1>
'
doc = Nokogiri::HTML(html)
h1_contents = doc.search('h1').map(&:text)
puts h1_contents
Which outputs:
# >>
# >> I am foo
# >>
# >> Stubborn quaz
# >>
# >>
# >> iThis
# >> is a reas
# >> long one line
# >>
# >> I am foo
# >> I am foo
# >>
# >> Stubborn quaz
# >>
# >> I am foo
Notice that Nokogiri is returning the content inside the <h3> block. This is correct/expected behavior because the HTML is malformed. Nokogiri fixes malformed HTML in an attempt to help retrieve usable content, but because there are many possible locations for the closing tag, Nokogiri inserts the closing tag at the last location that would be syntactically correct. Humans know to do it earlier, but this is software trying to be helpful.
This situation requires you to preprocess the HTML to make it correct. I'm using a single, simple, sub to fix the first <h1> found:
doc = Nokogiri::HTML(html.sub(/^(<h1>)$/, '\1</h1>'))
h1_contents = doc.search('h1').map(&:text)
puts h1_contents
# >> I am foo
# >>
# >> Stubborn quaz
# >> I am foo