Nokogiri parsing HTML - ruby

Question

I am using Nokogiri to parse my HTML code. My HTML looks like this:
<table>
<tr>
<td>
<p>Important Preferences</p>
To see as much as possible
<br />Relaxation
<br />Quality of accommodation
<br />Quality of activities
<br />Independence & flexibility
<br />Safety & security
</td>
<td>
<p>Budget Preferences</p>
4000 to 5000 USD per person
<br />5000 to 6000 USD per person
<br />Above 6000 USD per person
</td>
</tr>
</table>
I am trying to make a hash from it, which would be like this:
{
"Important Preferences" => "To see as much as possible, Relaxation, Quality of accommodation, Quality of activities, Independence & flexibility, Safety & security",
"Budget Preferences" => "4000 to 5000 USD per person, 5000 to 6000 USD per person, Above 6000 USD per person"
}
I tried:
params = {}
Nokogiri::HTML("my HTML pls see above").css("td p").each do |item|
params.merge!({item.text => item.next.text})
end
But I couldn't collect values inside <BR>.
My result was:
{
"Important Preferences" => "To see as much as possible",
"Budget Preferences" => "4000 to 5000 USD per person"
}

Answer 1

At the first step find out all <td> tags with xpath('//td'). Then, for each, iterate on its children and collect its content, if the child it Nokogiri::XML::Text (you don't want to collect <br> tags):
doc = Nokogiri::HTML.parse(html)
h = {}
doc.xpath('//td').each do |td|
p = td.at_xpath('p')
a = []
td.children.each do |child|
if Nokogiri::XML::Text === child
t = child.text.strip
a << t unless t.empty?
end
end
h[p.text] = a.join(', ')
end
result:
{"Important Preferences"=>"To see as much as possible, Relaxation, Quality of accommodation, Quality of activities, Independence & flexibility, Safety & security",
"Budget Preferences"=>"4000 to 5000 USD per person, 5000 to 6000 USD per person, Above 6000 USD per person"}
or in more compressed form, without using the strict loops:
doc = Nokogiri::HTML.parse(html)
h = {}
doc.xpath('//td').each do |td|
h[td.at_xpath('p').text] = td.children
.select{|x| Nokogiri::XML::Text === x && !x.text.strip.empty?}
.map{|x| x.text.strip}.join(', ')
end

Answer 2

You basically want to get all siblings of td p
You can get list of all siblings and remove p.
item.parent.children.to_a - [item]

Answer 3

I'd do it like this:
require 'nokogiri'
doc = Nokogiri::HTML(<<EOT)
<table>
<tr>
<td>
<p>Important Preferences</p>
To see as much as possible
<br />Relaxation
<br />Quality of accommodation
<br />Quality of activities
<br />Independence & flexibility
<br />Safety & security
</td>
<td>
<p>Budget Preferences</p>
4000 to 5000 USD per person
<br />5000 to 6000 USD per person
<br />Above 6000 USD per person
</td>
</tr>
</table>
EOT
doc.search('td').map { |td|
key = td.at('p').text
[
key,
td.text.sub(/#{key}/, '').lstrip.gsub(/\n +/, ', ')
]
}.to_h
# => {"Important Preferences"=>"To see as much as possible, Relaxation, Quality of accommodation, Quality of activities, Independence & flexibility, Safety & security, ", "Budget Preferences"=>"4000 to 5000 USD per person, 5000 to 6000 USD per person, Above 6000 USD per person, "}
If you're on an older version of Ruby that doesn't have to_h, use:
Hash[
doc.search('td').map { |td|
key = td.at('p').text
[
key,
td.text.sub(/#{key}/, '').lstrip.gsub(/\n +/, ', ')
]
}
]
# => {"Important Preferences"=>"To see as much as possible, Relaxation, Quality of accommodation, Quality of activities, Independence & flexibility, Safety & security, ", "Budget Preferences"=>"4000 to 5000 USD per person, 5000 to 6000 USD per person, Above 6000 USD per person, "}