I am trying to back to to previous td but to no avail, can you help
//*[#class='ein' and contains(.,'aaaa')] gets me to td but need to select the previous td-tried below but did not work
//*[#class='ein' and contains(.,'aaaa')][preceding-sibling::td]
Remember /X means "select X", while [X] means "where X". If you want to select preceding siblings, rather than testing whether they exist, use /.
It's impossible to say for certain without seeing the input HTML but I suspect that instead of
//*[#class='ein' and contains(.,'aaaa')][preceding-sibling::td]
you need something like
//*[#class='ein' and contains(.,'aaaa')]/preceding-sibling::td[1]
to navigate from each node selected by the initial expression to its nearest preceding td. Your first attempt will select exactly the same nodes as
//*[#class='ein' and contains(.,'aaaa')]
but only if they have at least one preceding-sibling element named td.
Use // after the element you found
Instead of preceding-sibling, just use preceding
//*[#class='ein' and contains(.,'aaaa')]//preceding::td[1]
Related
I am trying to write an XPath expression which can return the URL associated with the next page of a search.
The URL which leads to the next page of the search is always the href in the a tag following the tag span class="navCurrentPage" I have been trying to use a following-sibling term to pull the next URL. My search in the Chrome console is:
$x('//span[#class="navCurrentPage"][1]/following-sibling::a/#href[1]')
I thought by specifying #href[1] I would only get back one URL (thinking the [1] chooses the first element in list), but instead Chrome (and Scrapy) are returning four URLs. I don't understand why. Please help me to understand how to select the one URL that I am looking for.
Here is the URL where you can find the HTML giving me trouble:
https://www.yachtworld.com/core/listing/cache/searchResults.jsp?cit=true&slim=quick&ybw=&sm=3&searchtype=advancedsearch&Ntk=boatsEN&Ntt=&is=false&man=&hmid=102&ftid=101&enid=0&type=%28Sail%29&fromLength=35&toLength=50&fromYear=1985&toYear=2010&fromPrice=&toPrice=&luom=126¤cyid=100&city=&rid=100&rid=101&rid=104&rid=105&rid=107&rid=108&rid=112&rid=114&rid=115&rid=116&rid=128&rid=130&rid=153&pbsint=&boatsAddedSelected=-1
Thank you for the help.
Operator precedence: //x[1] means /descendant-or-self::node()/child::x[1] which finds every descendant x that is the first child of its parent. You want (//x)[1] which finds the first node among all the descendants named x.
xpath index will apply on all matching records, if you want to get only the first item, get the first instance.
$x('//span[#class="navCurrentPage"][1]/following-sibling::a/#href[1]').extract_first()
just add, .extract_first() or .get() to fetch the first item.
see the scrapy documentation here.
I've found this very helpful to make sure you have the bracket in the right place.
What is the XPath expression to find only the first occurrence?
also, the first occurrence may be [0] not [1]
While finding the relative XPath via Firebug : it creates like
.//*[#id='Passwd']--------- what if we dont use dot at the start what it signifies?
Just add //* in the Xpath --
it highlights --- various page elements ---------- what does it signify?
Below are XPaths for Gmail password fields. What is significance of * ?
.//*[#id='Passwd']
//child::input[#type='password']
There are several distinct, key XPath concepts in play here...
Absolute vs relative XPaths (/ vs .)
/ introduces an absolute location path, starting at the root of the document.
. introduces a relative location path, starting at the context node.
Named element vs any element (ename vs *)
/ename selects an ename root element
./ename selects all ename child elements of the context node.
/* selects the root element, regardless of name.
./* or * selects all child elements of the context node, regardless of name.
descendant-or-self axis (//*)
//ename selects all ename elements in a document.
.//ename selects all ename elements at or beneath the context node.
//* selects all elements in a document, regardless of name.
.//* selects all elements, regardless of name, at or beneath the context node.
With these concepts in mind, here are answers to your specific questions...
.//*[#id='Passwd'] means to select all elements at or beneath the
context node that have an id attribute value equal to
'Passwd'.
//child::input[#type='password'] can be simplified to
//input[#type='password'] and means to select all input elements
in the document that have an type attribute value equal to 'password'.
These expressions all select different nodesets:
.//*[#id='Passwd']
The '.' at the beginning means, that the current processing starts at the current node. The '*' selects all element nodes descending from this current node with the #id-attribute-value equal to 'Passwd'.
What if we don't use dot at the start what it signifies?
Then you'd select all element nodes with an #id-attribute-value equal to 'Passwd' in the whole document.
Just add //* in the XPath -- it highlights --- various page elements
This would select all element nodes in the whole document.
Below mentioned : XPatht's for Gmail Password field are true what is significance of * ?
.//*[#id='Passwd']
This would select all element nodes descending from the current node which #id-attribute-value is equal to 'Passwd'.
//child::input[#type='password']
This would select all child-element nodes named input which #type-attribute-values are equal to 'password'. The child:: axis prefix may be omitted, because it is the default behaviour.
The syntax of choosing the appropriate expression is explained here at w3school.com.
And the Axes(current point in processing) are explained here at another w3school.com page.
The dot in XPath is called a "context item expression". If you put a dot at the beginning of the expression, it would make it context-specific. In other words, it would search the element with id="Passwd" in the context of the node on which you are calling the "find element by XPath" method.
The * in the .//*[#id='Passwd'] helps to match any element with id='Passwd'.
For the first question: It's all about the context. You can see Syntax to know what '.', '..' etc means. Also, I bet you won't find any explanation better than This Link.
Simplified answer for second question: You would generally find nodes using the html tags like td, a, li, div etc. But '*' means, find any tag that match your given property. It's mostly used when you are sure about a given property but not about that tag in which the element might come with, like suppose I want a list of all elements with ID 'xyz' be it in any tag.
Hope it helps :)
I want to select a node based on the text value of a child.
My structure is as follows (sorry for german nodes):
<InspizierteAbwassertechnischeAnlage>
<Objektbezeichnung>10502002</Objektbezeichnung>
<Anlagentyp>1</Anlagentyp>
</InspizierteAbwassertechnischeAnlage>
How can I select the <InspizierteAbwassertechnischeAnlage> node where e.g. <Objektbezeichnung> = 10502002?
Why your solution didn't work
ancestor:://*[text()='10502002'] is syntactically incorrect, it's not valid XPath. I'm not sure what you tried to do with the axes here.
//*[text()='10502002'] itself would just select the Objektbezeichnung itself and not its parent. It would also select any other element with such a value, regardless of its name. In case of this document, nothing redundant would be returned but you have to be careful when using wildcards (*)
The solution
It's quite simple, you have to use a predicate to inspect the content of the child element
//InspizierteAbwassertechnischeAnlage[Objektbezeichnung = '10502002']
Note the double slash (// ), it is the abbreviated syntax for the descendant-or-self axis. The above expression translates to:
/descendant-or-self::InspizierteAbwassertechnischeAnlage[Objektbezeichnung = '10502002']
Or in plain English
In the set of all descendants of the document's root, find InspizierteAbwassertechnischeAnlage elements that contain at least one Objektbezeichnung element with a value of 10502002
As for German element names, at least it's not Hottentottenstottertrottelmutterbeutelrattenlattengitterkofferattentäter or Rhababerbarbarabarbarbarenbartbarbierbierbarbärbel
I have xpath
page.search("//table[#class='campaign']//table")
which returns two tables.
I need to choose only first table. This line doesn't work:
page.search("//table[#class='campaign']//table[1]")
How to choose only first table?
This bugged me, too. I still don't exactly know why your solution does not work. However, this should:
page.search("//table[#class='campaign']/descendant::table[1]")
EDIT: As the docs say,
"The location path //para[1] does not mean the same as the location
path /descendant::para[1]. The latter selects the first descendant
para element; the former selects all descendant para elements that are
the first para children of their parents."
Thanks to your question, I finally understood why this works this way :). So, depending on your structure and needs, this should work.
Instead of using an XPath expression to select the first matching element, you can either find all of them and then pare it down:
first_table = page.search("//table[#class='campaign']//table").first
...or better yet, select only the first by using at:
first_table = page.at("//table[#class='campaign']//table")
Note also that your expression can be found more simply by using the CSS selector syntax:
first_table = page.at("table.campaign table")
I have those two RxPaths which I need to be written in one expresion:
/td[2]/a[1]/tag[1]
and
/td[2]/a[1]
So basically I need to select path with 'tag' element if exists, if not than to select 'a' element.
something like:
if exist /td[2]/a[1]/tag[1] select /td[2]/a[1]/tag[1]
else select /td[2]/a[1]
Those elements need to have innertext attribute with some value in them, so I tried:
/td[2]/descendant::node()[#innertext!='']
but it won't work...
Also those elements are at the bottom of hierarchy so if is there any way to just select first element at lowest level.
I managed to solve this with an regex at the end of my Xpath expression.
/dom/body/div[#id='isc_0']/div/div[#id='isc_B']/div[#id='isc_C']/div[#id='isc_10']/div/div/iframe/body/table/tbody/tr/td[1]/a[#innertext='any uri item']/../../td[2]/*[#innertext~'[^ ]+']
Sorry for misunderstanding with problem...
Regards,
Vajda Vladimir
So basically I need to select path
with 'tag' element if exists, if not
than to select 'a' element. something
like:
if exist
/td[2]/a[1]/tag[1]
select
/td[2]/a[1]/tag[1]
else select
/td[2]/a[1]
I highly doubt that the top element of the document is a td. Don't use /td -- it means you want to select the top element of the document and this top element must be a td .
Also, /td[2] never selects anything, because a (wellformed) XML document has exactly one top element.
Use:
someParentElement/td[2]/a[1]/tag[1]
|
someParentElement/td[2]/a[1][not(someParentElement/td[2]/a[1]/tag[1])]
Those elements need to have innertext
attribute with some value in them
Use:
someParentElement/td[2][.//#innertext[normalize-space()]]/a[1]/tag[1]
|
someParentElement/td[2]
[.//#innertext[normalize-space()]]/a[1]
[not(someParentElement/td[2]
[.//#innertext[normalize-space()]]/a[1]/tag[1])]
Also those elements are at the bottom
of hierarchy so if is there any way to
just select first element at lowest
level.
This is not clear. Please, clarify.
All "leaf" elements can be selected using the following XPath expression:
//*[not(*)]
The elements selected don't have any children-elements, but may have other children (such as text-nodes, PIs, comments) and attributes.
Besides all those good advices from #Dimitre, I want to add that a parent will always come before (in document order) than a child, so you could use this XPath expression:
(/real-path-from-root/td[2]/a[1]
|
/real-path-from-root/td[2]/a[1]/tag[1])[last()]
You could do this without | union set operator in XPath 1.0, but it will end up very unreadable... Of course, in XPath 2.0 you could just do:
(/real-path-from-root/td[2]/a[1]/(.|tag[1]))[last()]