<article class='article-contents'>
<div class='summary'>xxxx</div>
<p>xxxxxx</p>
<table>...</table>
<p>....</p>...
</article>
I have a html structure like above, i'd like to skip pass <div class='summary'> and get the whole content inside article section using Xpath structure.
You could use a query like this:
//article[#class='article-contents']/node()[not(local-name()='div' and #class='summary')]
This should select all child nodes of the article excluding the summary div.
Related
<html>
<body>
Example
SO
<div>
<div class="kekeke">JSAFK</div>
</div>
</body>
</html>
For getting a JSAFK element in this doc, using XPath, can I just write //*div[#class=kekeke] instead full XPath?
// is short for /descendant-or-self::node()/. So...
This XPath,
//div[#class='kekeke']
will select all such div elements in the document:
<div class="kekeke">JSAFK</div>
This XPath,
//div[#class='kekeke']/text()
will select all text nodes under all such div elements in the document:
JSAFK
there is something wrong in "
//*div[#class=kekeke]
you can't use * and div together. if you want to have a shorter path.
you can write like this
//div[#class="kekeke"]/text()
With the help of this SO question I have an almost working xpath:
//div[contains(#class, 'measure-tab') and contains(., 'someText')]
However this gets two divs: in one it's the child td that has someText, the other it's child span.
How do I narrow it down to the one with the span?
<div class="measure-tab">
<!-- table html omitted -->
<td> someText</td>
</div>
<div class="measure-tab"> <-- I want to select this div (and use contains #class)
<div>
<span> someText</span> <-- that contains a deeply nested span with this text
</div>
</div>
To find a div of a certain class that contains a span at any depth containing certain text, try:
//div[contains(#class, 'measure-tab') and contains(.//span, 'someText')]
That said, this solution looks extremely fragile. If the table happens to contain a span with the text you're looking for, the div containing the table will be matched, too. I'd suggest to find a more robust way of filtering the elements. For example by using IDs or top-level document structure.
You can use ancestor. I find that this is easier to read because the element you are actually selecting is at the end of the path.
//span[contains(text(),'someText')]/ancestor::div[contains(#class, 'measure-tab')]
You could use the xpath :
//div[#class="measure-tab" and .//span[contains(., "someText")]]
Input :
<root>
<div class="measure-tab">
<td> someText</td>
</div>
<div class="measure-tab">
<div>
<div2>
<span>someText2</span>
</div2>
</div>
</div>
</root>
Output :
Element='<div class="measure-tab">
<div>
<div2>
<span>someText2</span>
</div2>
</div>
</div>'
You can change your second condition to check only the span element:
...and contains(div/span, 'someText')]
If the span isn't always inside another div you can also use
...and contains(.//span, 'someText')]
This searches for the span anywhere inside the div.
Consider the following html
<div id="relevantID">
<div class="column left">
<h1> Section-Header-1 </h1>
<ul>
<li>item1a</li>
<li>item1b</li>
<li>item1c</li>
<li>item1d</li>
</ul>
</div>
<div class="column">
<ul> <!-- Pay attention here -->
<li>item1e</li>
<li>item1f</li>
</ul>
<h1> Section-Header-2 </h1>
<ul>
<li>item2a</li>
<li>item2b</li>
<li>item2c</li>
<li>item2d</li>
</ul>
</div>
<div class="column right">
<h1> Section-Header-3 </h1>
<ul>
<li>item3a</li>
<li>item3b</li>
<li>item3c</li>
<li>item3d</li>
</ul>
</div>
</div>
My objective is to extract the items for each Section headers. However, inconveniently the designer of the webpage decided to break up the data into three columns, adding an additional div (with classes column right etc).
My current method of extraction was using the xpath
for section headers, I use the xpath (get all h1 elements withing a div with given id)
//div[#id="relevantID"]//h1
above returns a list of h1 elements, looping over each element I apply the additional selector, for each matched h1 element, look up the next ul node and retreive all its li nodes.
following-sibling::ul//li
But thanks to the designer's aesthetics, I am failing in the one particular case I've marked in the HTML file. Where the items are split across two different column divs.
I can probably bypass this problem by stripping out the column divs entirely, but I don't think modifying the html to make a selector match is considered good (I haven't seen it needed anywhere in the examples I've browsed so far).
What would be a good way to extract data that has been formatted like this? Full solutions are not neccessary, hints/tips will do. Thanks!
The columns do frustrate use of following-sibling:: and preceding-sibling::, but you could instead use the following:: and preceding:: axis if the columns at least keep the list items in proper document order. (That is indeed the case in your example.)
The following XPath will select all li items, regardless of column, occurring after the "Section-Header-1" h1 and before the "Section-Header-2" h1 header in document order:
//div[#id='relevantID']//li[normalize-space(preceding::h1) = 'Section-Header-1'
and normalize-space(following::h1) = 'Section-Header-2']
Specifically, it selects the following items from your example HTML:
<li>item1a</li>
<li>item1b</li>
<li>item1c</li>
<li>item1d</li>
<li>item1e</li>
<li>item1f</li>
You can combine following-sibling and preceding-sibling to get possible li elements in a div before the h2 and use the union operator |. As example for the second h2:
((//div[#id="relevantID"]//h1)[2]/preceding-sibling::ul//li) |
((//div[#id="relevantID"]//h1)[2]/following-sibling::ul//li)
Result:
<li>item1e</li>
<li>item1f</li>
<li>item2a</li>
<li>item2b</li>
<li>item2c</li>
<li>item2d</li>
As you're already selecting all h1 using //div[#id="relevantID"]//h1 and retrieving all li items for each h1 using as a second step following-sibling::ul//li, you could combine this to following-sibling::ul//li | preceding-sibling::ul//li.
Take a look at the sample XML below--
<div id="main">
<div id="1">
Some random text
</div>
<div id="2">
Some random text
</div>
<div id="3">
Some random text
</div>
<p> Some more random text</p>
<div id="4">
Some random text
</div>
</div>
Now, how do I find out the number of divs within the main div using Xquery? And how to do this in XPath?
You can use the following XPath:
count(div[#id="main"]/div)
The function count does the counting, the main div is selected by its id.
The XPath expressions below can be used both in XPath and XQuery. This is so, because XPath (2.0) is a proper subset of XQuery.
Use:
count(/*//div)
If "the main div" isn't the top element of the XML document, and this is the only div whose id attribute has string value of "main", use:
count((//div[#id='main'])[1]//div)
If it is guaranteed that the div children of the "main div" dont have div descendents, use:
count((//div[#id='main'])[1]/div)
Do note: The XPath pseudo-operator // can be very inefficient -- this is why, always try to avoid using it, whenever the structure of the XML document is statically known and specific paths can be used.
I want to get to the exact node having this text: 'Company'. Once I get to this node I want to get to the next text node immediately following this node because this contains the company name. How can I do this with Xpath?
Fragment of XML is:
<div id="jobsummary">
<div id="jobsummary_content">
<h2>Job Summary</h2>
<dl>
<dt>Company</dt>
<!-- the following element is the one I'm looking for -->
<dd><span class="wrappable">Pinpoint IT Services, LLC</span></dd>
<dt>Location</dt>
<dd><span class="wrappable">Newport News, VA</span></dd>
<dt>Industries</dt>
<dd><span class="wrappable">All</span></dd>
<dt>Job Type</dt>
<dd class="multipledd"><span class="wrappable">Full Time</span></dd><dd class="multipleddlast"><span class="wrappable"> Employee</span></dd>
</dl>
</div>
</div>
I got to the Company tag with following xpath: //*[text()= 'Company']
Now I want to get to the next text node. My XML is dynamic. So I can't hardcode the node type like <dd> for getting the company value. But this is for sure that the value be in the immediate next text node.
So how can I get to the text node immediately after the node with text as Company?
If you cannot hardcode any part of the following-sibling node your xpath should look like this:
//*[text()='Company']/following::*/*/text()
assuming that the desired text is always enclosed in another element like span.
To test for given dt text, modify your xpath to
//*[text()='Company' or text()='Company:' or text()='Company Name']/following::*/*/text()
use //*[text()='Company']/following-sibling::dd to get the next dd.
You can even insert conditions for that dd and also go further in it.
following-sibling::elementName just looks for the next sibling at the same parent level that meets your requirements.
With no conditions, like above, it will get the next dd after the 'Company'.
The text is in the span so you might try
//*[text()='Company']/following-sibling::dd/span
Another clarifying example would be, let's say that you want to get also the next industries text for the current selected 'Company'.
Having //*[text()='Company',
you can modify it like this: //*[text()='Company']/following-sibling::dt[text()='Industries']/dd/span
Of course, instead of hardcoding the values for text(), you can use variables.
You can Use XPathNavigator and go on to every node type one by one
I think XPathNavigator::MoveToNext is the method you are looking for.
There is the sample code as well at..
http://msdn.microsoft.com/en-us/library/9yxc3x24.aspx
Use this general XPath expression that selects the wanted text node even when it is wrapped in statically unknown markup elements:
(//*[text()='Company']/following-sibling::*[1]//text())[1]
When this XPath expression is evaluated against the provided XML document:
<div id="jobsummary">
<div id="jobsummary_content">
<h2>Job Summary</h2>
<dl>
<dt>Company</dt>
<!-- the following element is the one I'm looking for -->
<dd><span class="wrappable">Pinpoint IT Services, LLC</span></dd>
<dt>Location</dt>
<dd><span class="wrappable">Newport News, VA</span></dd>
<dt>Industries</dt>
<dd><span class="wrappable">All</span></dd>
<dt>Job Type</dt>
<dd class="multipledd"><span class="wrappable">Full Time</span></dd><dd class="multipleddlast"><span class="wrappable"> Employee</span></dd>
</dl>
</div>
</div>
exactly the wanted text node is selected:
Pinpoint IT Services, LLC
Even if we change the XML to this one:
<div id="jobsummary">
<div id="jobsummary_content">
<h2>Job Summary</h2>
<div>
<p>Company</p>
<!-- the following element is the one I'm looking for -->
<dd><span class="wrappable"><b><i><u>Pinpoint IT Services, LLC</u></i></b></span></dd>
<dt>Location</dt>
<dd><span class="wrappable">Newport News, VA</span></dd>
<dt>Industries</dt>
<dd><span class="wrappable">All</span></dd>
<dt>Job Type</dt>
<dd class="multipledd"><span class="wrappable">Full Time</span></dd><dd class="multipleddlast"><span class="wrappable"> Employee</span></dd>
</div>
</div>
</div>
the XPath expression above still selects the wanted text node:
Pinpoint IT Services, LLC