I need to extract http://site.ru/ from this code:
<div class="one">
<dl>
<dt class="two">
<span class="name">Site</span>
</dt>
<dd class="three">
<span class="js-pseudo-link" data-url="rAnDoMlEtTeRsAnDnUmBeRs" style>
<a href="http://site.ru/" class rel="nofollow" target="_blank" style> http://site.ru/ </a>
</span>
</dd>
</dl>
</div>
I use this XPath query: //div//dl//dd//span//a/#href
But it doesn't work. It doesn't return anything.
I'm a newbie in XPath.
Unfortunately, the data source you are looking for is an empty span node (class js-pseudo-link). The data-url attribute has the base64 encoded link you want. This node only gets populated after loading. ImportXML for some reason ignores nodes with no text and there's no way to get it not to do that. To get around this, looks like you'll have to write an apps script that can handle empty nodes or just gets the raw HTML code and parse it.
Related
having the following HTML (snippet grabbed from the web page I wanted to scrape):
<div class="ulListContainer">
<section class="stockUpdater">
<ul class="column4">
<li>
<img src="1.png" alt="">
<strong>
Buy*
</strong>
<strong>
Sell*
</strong>
</li>
<li>
<header>
$USD
</header>
<span class="">
20.90
</span>
<span class="">
23.15
</span>
</li>
</ul>
<ul>...</ul>
</section>
</div>
how do I get the 2nd li 1st span value using XPath? The result should be 20.90.
I have tried the following //div[#class="ulListContainer"]/section/ul[1]/li[2]/span[1] but I am not getting any values. I must said this is being used from a Google Sheet and using the function IMPORTXML (not sure what version of XPath it does uses) can I get some help?
Update
Apparently Google Sheets does not support such "complex" XPath expression since it seems to work fine:
Update 1
As requested I've shared the Google Sheet I am using to test this, here is the link
What you need is :
=IMPORTXML(A1;"//li[contains(text(),'USD')]/span[1]")
Removing section from your original XPath will work too :
=IMPORTXML(A1;"//div[#class='ulListContainer']/ul[1]/li[2]/span[1]")
Try this:
=IMPORTXML("URL","//span[1]")
Change URL to the actual website link/URL
Using Scrapy, I want to extract some data from a HTML well-formed site. With XPath I am able to extract a list of items, but I am not able to extra data from the elements in the list, using XPath
All XPath's have been tested using XPather. I have tested the issue using a local file that contains the webpage, same issue.
Here goes:
# Get the webpage
fetch("https://www.someurl.com")
# The following gives me the expected items from the HTML
products = response.xpath("//*[#id='product-list-146620']/div/div")
The items are like this:
<div data-pageindex="1" data-guid="13157582" class="col ">
<div class="item item-card item-card--static">
<div class="item-card__inner">
<div class="item__image item__image--overlay">
<a href="/www.something.anywhere?ref_gr=9801" class="ratio_custom" style="padding-bottom:100%">
</a>
</div>
<div class="item__text-container">
<div class="item__name">
<a class="item__name-link" href="/c.aspx?ref_gr=9801">The text I want</a>
</div>
</div>
</div>
</div>
</div>
When using the following Xpath to extract "The text I want", i dont get anything:
XPATH_PRODUCT_NAME = "/div/div/div/div/div[contains(#class,'item__name')]/a/text()"
products[0].xpath(XPATH_PRODUCT_NAME).extract()
The output is empty, why?
Try the following code.
XPATH_PRODUCT_NAME = ".//div[#class='item__name']/a[#class='item__name-link']/text()"
products[0].xpath(XPATH_PRODUCT_NAME).extract()
I am trying to scrape "Description" from this HTML structure
<div class="menu-index-page__item-content">
<h6 class="menu-index-page__item-title">
<span> Item title </span>
</h6>
<p class="menu-index-page__item-desc">
<span>
<span>
<span>Description</span>
</span>
</span>
Each tag has an element with it that I don't know how to handle:
data-reactid=".3wrqgx5340.3.5.0.4:$523105.2.$3959254.$menuItemContent.1.0"
Each data-reactid is different. So if I target this attribute I will scrape stuff I don't want.
I've tried .search .xpath, using tags and classes but nothing seems to work.
Is there a way to say: give me the p tag that has a class="menu-index-page__item-desc" and scrape the 3rd span from there?
You can get the required value via xpath
//text()[contains(.,'Description')]
You code and xpath:
How to extract xpath to link of image? I want to extract this specific link:
http://insales.ru/images/bigpic.jpeg
I dont know how I must specify xpath. Do I have to include all parent tags to get it or I just can go directly to tag?
<div class="tango">
<div class="container-horizontal">
<div class="clip-horizontal">
<ul id="carousel" class="pagination jcarousel">
<li class="jcarousel-item">
<a rev=http://insales.ru/images/bigpic.jpeg href="http://insales.ru/images/pic2.jpeg">
<img src="http://insales.ru/images/thumb.jpeg">
</a>
</li>
</ul>
</div>
</div>
</div>
So my xpath would be:
//li[contains(#class, 'jcarousel-item jcarousel-item-horizontal jcarouse')]/a
Or I have to include all parent div tags:
//div[#class="tango"]//div[#class="container-horizontal"]//li[contains(#class, 'jcarousel-item jcarousel-item-horizontal jcarouse')]/a
But anyway both of this xpaths don't work.
How to specify xpath to extract this link: http://insales.ru/images/bigpic.jpeg
There are several options but the shortest one is probably:
.//*[#id='carousel']/li/a/img
I am trying to get the error message off of a page from a site. The list contains several possible errors so i can't check by id; but I do know that the one with display:list-item is the one I want. This is my rule but doesn't seem to work, what is wrong with it? What I want returned is the error text in the element.
//*[#id='errors']/ul/li[contains(#style,'display:list-item')]
Example dom elements:
<div id="errors" class="some class" style="display: block;">
<div class="some other class"></div>
<div class="some other class 2">
<span class="displayError">Please correct the errors listed in red below:</span>
<ul>
<li style="display:none;" id="invalidId">Enter a valid id</li>
<li style="display:list-item;" id="genericError">Something bad happened</li>
<li style="display:none;" id="somethingBlah" ............ </li>
....
</ul>
</div>
The correct XPath should be:
//*[#id='errors']//ul/li[contains(#style,'display:list-item')]
After //*[#id='errors'] you need an extra /, because <ul> is not directly beneath it. Using // again scans all underlying elements for <ul>.
If you are capable to not use // it would be better and faster and less consuming.