Shortest match in Regex [duplicate] - ruby

This question already has answers here:
Find shortest matches between two strings
(4 answers)
Closed 3 years ago.
This is my regex:
/<strong>.*ingredients.*<\/ul>/im
Assuming the source code:
<strong>Contest closes on Thursday May 10th 2012 at 9pm PST</strong></div>
<br />
<br />
<br />
* I am not affiliated with Blue Marble Brands or Ines Rosales Tortas in any way. I am not sponsored by them and did not receive any compensation to write this post...I just simply think the Tortas are wonderful!<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<img border="0" height="480" mea="true" src="http://1.bp.blogspot.com/-35J5vNrXkqE/T6htXTafrmI/AAAAAAAAA5E/g2mtiuSpSmw/s640/food+003.JPG" width="640" /></div>
<br />
<strong><span style="font-size: large;">Ingredients:</span></strong><br />
<ul>
<li>Ines Rosales Rosemary and Thyme Tortas</li>
<li>Pizza Sauce (ready made in a jar)</li>
<li>Roma Tomatoes</li>
<li>Roasted Red Peppers </li>
<li>Marinated Artichoke Hearts</li>
<li>Olives (I used Pitted Spanish Manzanilla Olives)</li>
<li>Daiya Vegan Mozzarella Cheese</li>
</ul>
<span style="font-size: large;"><strong>Directions:</strong></span><br />
<br />
Spread small amount of pizza sauce over Torta.
the Regex is greedy and grabs everything from <strong>Contest...</ul> but the shortest match should yield <strong><span style="font-size: large;">Ingredients...</ul>
this is my gist: https://gist.github.com/3660370
::EDIT::
Please allow flexibility inbetween strong tag and ingredients, and ingredients and ul.

Try this:
/<strong><span.*ingredients.*<\/ul>/im
Please refrain from regex-ing html. Use Nokogiri or a similar library instead.

This should work:
/(?!<strong>.*<strong>.*<\/ul>)<strong>.*?ingredients.*?<\/ul>/im
Test it here
Basically, the regex is using the negative lookahead to avoid multiple <strong> before <\ul> like this: (?!<strong>.*<strong>.*<\/ul>)

I think this is what you're looking for:
/<strong>(?:(?!<strong>).)*ingredients.*?<\/ul>/im
Replacing the first .* with (?:(?!<strong>).)* allows it to match anything except another <strong> tag before it finds ingredients. After that, the non-greedy .*? causes it to stop matching at the first instance of </ul> it sees. (Your sample only contains the one <UL> element, but I'm assuming the real data could have more.)
The usual warnings apply: there are many ways this regex can be fooled even in perfectly valid HTML, to say nothing of the dreck we usually see out there.

Related

How to get specific xpath tag value

<div class="container">
<span class="price">
<bdi> 140 </bdi>
</span>
<span class="price">
<del>
<bdi>90</bdi>
</del>
<ins>
<bdi> 120 </bdi>
</ins>
</span>
</div>
I want to scrape a site which html formatting like below. Here I dont want to bdi tag value which is under del tag and want bdi tag value which is under span class and ins tag. Is there any path to figure it out?
Don't pretty much usual //span/ins/bdi/text() work for you?
This is "text of <bdi> which parent is <ins> which parent is <span>"?
CSS variant span>ins>bdi::text should also work I suppose.
Sorry, haven't noticed that you need two values. In that case .xpath('//bdi[not(parent::del)]/text()').extract() will work well.

Make XPath stop at a certain depth?

I have the following HTML
<span class="medium bold day-time-clock">
09:00
<div class="tooltip-box first-free-tip ">
<div class="tooltip-box-inner">
<span class="fa fa-clock-o"></span>
Some more text
</div>
</div>
</span>
I want an XPath that only gets the text 09:00, not Some more text NOT using text()[1] because that causes other problems. My current XPath looks like this
("//span[1][contains(#class, 'day-time-clock')]/text()")
I want one that ignores this whole part of the HTML
<div class="tooltip-box first-free-tip ">
<div class="tooltip-box-inner">
<span class="fa fa-clock-o"></span>
Some more text
</div>
</div>
You can limit the level of descendant:: nodes with position().
So the following expression does work:
span/descendant::node()[2 > position()]
Adjust the number in the predicate to your needs, 2 is only an example. A disadvantage of this approach is that the counting of the descendants is only accurate for the first child in the descending tree.
Another approach is limiting the both: the ancestors and the descendants:
span/descendant::node()[3 > count(ancestor::*) and 1 > count(descendant::*)]
Here, too, you have to adjust the numbers in the predicates to get any useful results.
Use normalize-space() for select all non-whitespace nodes of the document:
//span[contains(#class, 'day-time-clock')]/text()[normalize-space()]
I think (if I understand you correctly) that
"..//div[contains(#class, 'tooltip-box')]/parent::span"
gets you there.

Select all nodes between two elements excluding unnecessary element from the intersection using XPath

There’s a document structured as follows:
<div class="document">
<div class="title">
<AAA/>
</div class="title">
<div class="lead">
<BBB/>
</div class="lead">
<div class="photo">
<CCC/>
</div class="photo">
<div class="text">
<!-- tags in text sections can vary. they can be `div` or `p` or anything. -->
<DDD>
<EEE/>
<DDD/>
<CCC/>
<FFF/>
<FFF>
<GGG/>
</FFF>
</DDD>
</div class="text">
<div class="more_text">
<DDD>
<EEE/>
<DDD/>
<CCC/>
<FFF/>
<FFF>
<GGG/>
</FFF>
</DDD>
</div class="more_text">
<div class="other_stuff">
<DDD/>
</div class="other_stuff">
</div class="document">
The task is to grab all the elements between <div class="lead"> and <div class="other_stuff"> except the <div class="photo"> element.
The Kayessian method for node-set intersection $ns1[count(.|$ns2) = count($ns2)] works perfectly. After substituting $ns1 with //*[#class="lead"]/following::* and $ns2 with //*[#class="other_stuff"]/preceding::*,
the working code looks like this:
//*[#class="lead"]/following::*[count(. | //*[#class="other_stuff"]/preceding::*)
= count(//*[#class="other_stuff"]/preceding::*)]/text()
It selects everything between <div class="lead"> and <div class="other_stuff"> including the <div class="photo"> element. I tried several ways to insert not() selector in the formula itself
//*[#class="lead" and not(#class="photo ")]/following::*
//*[#class="lead"]/following::*[not(#class="photo ")]
//*[#class="lead"]/following::*[not(self::class="photo ")]
(the same things with /preceding::* part) but they don't work. It looks like this not() method is ignored – the <div class="photo"> element remains in the selection.
Question 1: How to exclude the unnecessary element from this intersection?
It’s not an option to select from <div class="photo"> element excluding it automatically because in other documents it can appear in any position or doesn't appear at all.
Question 2 (additional): Is it OK to use * after following:: and preceding:: in this case?
It initially selects everything up to the end and to the beginning of the whole document. Could it be better to specify the exact end point for the following:: and preceding:: ways? I tried //*[#class="lead"]/following::[#class="other_stuff"] but it doesn’t seem to work.
Question 1: How to exclude the unnecessary element from this intersection?
Adding another predicate, [not(self::div[#class='photo'])] in this case, to your working XPath should do. For this particular case, the entire XPath would look like this (formatted for readability) :
//*[#class="lead"]
/following::*[
count(. | //*[#class="other_stuff"]/preceding::*)
=
count(//*[#class="other_stuff"]/preceding::*)
][not(self::div[#class='photo'])]
/text()
Question 2 (additional): Is it OK to use * after following:: and preceding:: in this case?
I'm not sure if it would be 'better', what I can tell is following::[#class="other_stuff"] is invalid expression. You need to mention the element to which the predicate will be applied, for example, 'any element' following::*[#class="other_stuff"], or just 'div' following::div[#class="other_stuff"].

Xpth extract plain email text

I'm trying to extract the email text from a list but without success.
In particular I've used this code
//li/div/p//*[contains(., '#')]
but strangely it doesn't work! Could you help me?
Here's the code exemple
<li class="bgmp_list-item">
<h3 class="bgmp_list-placemark-title">
Name1
</h3>
<div class="bgmp_list-description">
<p class="">
<strong class="">Responsible:</strong> John Doe <br>
<strong class="">Site:</strong> <a title="www.exemple.com" href="http://www.exemple.com" onclick="javascript:_gaq.push(['_trackEvent','outbound-article','www.2ld.it']);" target="_blank" class="">www.2ld.it</a>
<br>
<strong class="">Email:</strong> some_email#email.com
<br><strong class="">Address:</strong> 3, Main Street 00000, London <br>
<strong>Tel:</strong> 00 000000 <strong>Fax:</strong> 0000000
</p>
</div>
You're almost there but not quite. For the sample code the correct xpath would be
//p/text()[contains(.,'#')]
Not to reinvent the wheel here is a very good explanation on it on another answer
By using p//*[contains(., '#')] you apply the predicate on individual child elements of <p>, while there is no such child element because
the target email address text is direct child of <p>. This is one of the reason why the intial XPath didn't work. Applying the predicate on <p> directly should work :
//li/div/p[contains(., '#')]
but that will return the <p> element. If you need to return only the text node that contains email address, then the predicate should be applied on individual text nodes within <p>, as mentioned in the other answer :
//li/div/p/text()[contains(., '#')]

Help with regex / ruby

Hey guys, so I'm making a script to featch words/results off of this site (http://grecni.com/texttwist.php), So I already have the http request post ready, ect.
Only thing I need now is to fetch out the words, So I'm working with an html source that looks like so:
<html>
<head>
<title>Text Twist Unscrambler</title>
<META NAME="keywords" CONTENT="Text,Twist,Text Twist,Unscramble,Free,Source,php">
</head>
<body>
<font face="arial,helvetica" size="3">
<p>
<b>3 letter words</b><br>sae sac ess aas ass sea ace sec <p>
<b>4 letter words</b><br>cess secs seas ceca sacs case asea casa aces caca <p>
<b>5 letter words</b><br>cacas casas caeca cases <p>
<b>6 letter words</b><br>access <br><br>
Found 23 words in 0.22962 seconds
<form action="texttwist.php" method="post">
enter scrambled letters and I'll return all word combinations<br>
<input type="text" name="l" value="asceacas" size="20" maxlength="20">
<input type="submit" name="button" value="unscramble">
<input type="button" name="clear" value="clear" onClick="this.form.l.value='';">
</form><p>
<a href=texttwist.phps>php source</a>
- it's kinda ugly, but it's fast<p>
<a href=/>back to my page</a>
</body>
</html>
I'm trying to fetch the words like "sae", "sav", "secs", "seas", "casas", ect.
Any help?
This is the farthest i've gotten, don't know what to do from here.: link text
Any suggestions? Help?
Use a HTML parser like Nokogiri.
If you want any kind of robustness you really want a parser, as mentioned by Adrian, Nokogiri is most popular solution.
If you insist, aware of the madness that you may be in for as the page becomes more complex the following may help:
Search for a line that matches
/^<b>\d+ letter words/
and then you can dig out the bits like so:
a = line.split(/<br>/)[1] # the second half
a.gsub!('<p>', '') # take out the trailing <p>
res = a.split(' ')# this is your data
That being said, this isn't anything you want in production code. You'll be surprised how learning a parser will change how you see this problem.

Resources