The website is : https://www.futbin.com/18/player/2600/Ayhan/
I inspect the element and get the XPath which is: //*[#id="ps-lowest-1"]
Then I use:
=IMPORTXML("https://www.futbin.com/18/player/2600/Ayhan/","//*[#id='ps-lowest-1']")
To get the data which should be 2000
But instead it only shows: - on the sheet. No errors just doesn't show the data that I want it to. Is there anyway to get the data that I need?
Thanks
The Sheets command importXML reads only the HTML source of the page without executing any JavaScript on it. As you can see yourself by using "view source" in the browser, the source indeed has "-" in that span:
<span class="price_big_right">
<span id="ps-lowest-1">-</span>
</span>
The actual numbers are loaded by some JavaScript file which then inserts them in that span. Neither importXML nor other Sheets functions can retrieve dynamically inserted data.
Sometimes, after inspecting the JS files, one can uncover the URL of source of data and try to import that; but this is a tedious reverse engineering exercise for each particular site.
Related
I want to use ImportXML function in google sheets in order to bring some prices into my google sheet, how to find the correct Xpath for this ?
This is the URL: https://super.walmart.com.mx/verduras/aguacate-hass-extra-por-kilo/00000000003354
and im trying to bring the price, in this case is $39.90
I was trying this formula but im getting NA (imported content is empty)
=IMPORTXML("https://super.walmart.com.mx/verduras/aguacate-hass-extra-por-kilo/00000000003354","//*[#id="scrollToTopComponent"]/section/div[1]/div[2]/div[2]/div[3]/h4/text()[1]")
Thanks
IMPORTXML does not work on Javascript content
You can verify that that the URL https://super.walmart.com.mx/verduras/aguacate-hass-extra-por-kilo/00000000003354 content loads via Javascript by
going on your Browser settings
Disable / block Javascript
Refresh your URL
See the following:
So, unfortunately you cannot use IMPORTXML to get the content of this URL.
There is a google sheet containing a list of MPN's (manufacturer part numbers). Trying to scrape a site called wikiarms for the UPC Codes when I have the MPN for an item.
I have the correct formula for doing this on another site.
=IMPORTXML("http://gun.deals/search/apachesolr_search/"&B1,"//dd/a[../../dt[contains(text(),'UPC')]]|//dd/span[../../dt[contains(text(),'UPC')]]")
Trying to figure out what the correct xpath to complete this formula. Some videos I have watch said to open the page in Chrome and use inspector to select and copy the xpath to complete the importxml function. I tried this with no luck.
Sample
Visit https://www.wikiarms.com/guns?q=20071
In the table there is a button "available in 6 stores" click that to reveal the list. The UPC should be listed after the MPN.
If I copy the xpath in Chrome this is the result
/html/body/div[1]/div/div/div[2]/div/div/div[2]/div[2]/table/tbody/tr[2]/td[5]
=IMPORTXML("https://www.wikiarms.com/guns?q="&B2,"xpath here")
What do I have to add at the end of this formula to pull in the UPC code? I will be using this formula to pull in UPC code for about 1000 items.
Thank you for your help.
Using your sample link, try
=IMPORTXML("https://www.wikiarms.com/guns?q=20071","//td[#class='upc']/a/#title")
and see if it works for you.
I am trying to scrape some data from the following website: https://xrpcharts.ripple.com/
The data I am interested in is Total XRP which you can see immediately below or to the side (depending on your browser) of the circle diagram. So what I first did was inspect the element I am interested in. So I see that it is inside <div class="stat" inside span ng-bind="totalXRP | number:2" class="ng-binding">99,993,056,930.18</span>.
The number 99,993,056,930.18 is what I am interested in.
So I started in a scrapy shell and wrote:
fetch("https://xrpcharts.ripple.com")
I then used chrome to copy the Xpath by right clicking on that place of HTML code, the result chrome gave me was:
/html/body/div[5]/div[3]/div/div/div[2]/div[3]/ul/li[1]/div/span
Then I used the Xpath command to extract the text:
response.xpath('/html/body/div[5]/div[3]/div/div/div[2]/div[3]/ul/li[1]/div/span/text()').extract()
but this gave me an empty list []. I really do not understand what I am doing wrong here. I think I am making an obvious mistake but I dont see it. Thanks in advance!
The bottom line is: you cannot expect the page you see in the browser to be the same page Scrapy would download and have available to work with. Scrapy is not a browser.
This page is quite dynamic and complex and is constructed with the help of multiple asynchronous requests bringing in both the logic and the data. There is also JavaScript executed in the browser that plays an important role in forming and supporting the HTML document object tree.
Scrapy does not have all these things, the thing you get when you do fetch() is just the very first initial "bare bones" HTML page without all the "dynamic content".
I encounter the error message imported content is empty when I use the formula below in google spreadsheet.
=IMPORTXML("https://www.moh.gov.sg/content/moh_web/home/pressRoom.html", "//div[#class='article highlight']/h3/a/#title")
I am trying to import the list of press release title on the webpage.
What am I doing wrong?
So this issue is not your formula or XML in this particular case, its that the content is loaded using jQuery, so you need to figure out where or what the url is that actually holds your content.
So How To Use importXML By Google Docs Spreadsheet Plus xPath To Help Me Copy Soundcloud Playlist Track Title?
Today I'm looking for a solution on how to copy the Track Title of tracks inside a Soundcloud Playlist/Compilation.
I searched around and discovered importXML function offered by Google Docs Spreadsheet. While digging further about the importXML function, I discovered about xPath.
Great combination! I thought.
So I quickly get my hands on the tools and tested it, and it's great! I've extracted some data. And so I decided that I'm ready to implement the tool and use it with Soundcloud.
But when I tried to implement the syntax, I got an error saying Import Internal Error
The Syntax Is
=IMPORTXML(A1,"//div[#class='sc-media-content']/a[#title]")
<div class="sc-media-content"> is the div that holds the Track Title of the song, that is enclosed within an anchor tag with title attribute.
Here is the html block for it;
<div class="sc-media-content">
<a class="soundTitle__title sc-link-dark sc-truncate " href="/seven-lions/velvetine-the-great-divide?in=thedubstepgod/sets/melodic-dubstep-chillstep" title="Velvetine - The Great Divide (Seven Lions Remix)">
Velvetine - The Great Divide (Seven Lions Remix)
</a>
What I'm trying to extract is the Velvetine - The Great Divide (Seven Lions Remix). A1 is the cell where the Soundcloud Playlist Link is pasted into.
Other Syntax I've Tried
I've tried other syntax too, like;
=IMPORTXML(A1,"//div[#class='sc-media-content']/title")
As suggested here
=IMPORTXML(A1,"//div[#class='sc-media-content']/#title") From an answer found here, though it wasn't an accepted answer.
So what am I doing wrong? How could I copy those soundcloud playlist title to my Google Docs Spreadsheet using xPath?
UPDATE
Based from the answer given by TGH, this should work //div[#class='sc-media-content']/a/text().
But, the problem is that the div block that we're looking for is not in the source code. I did a view source code on the Playlist's page and the div block is not there. All I'm seeing is JavaScripts. So JS is loading the div blocks/classes.
So another question might be needed to help solve this one
How to use xPath with JavaScript loaded html elements?
Try the following
//div[#class='sc-media-content']/a/text()
Or if you want to grab it from the title do this
//div[#class='sc-media-content']/a/#title
I tested it here and it seems to work. Pasted your html, but had to close the div manually.
http://www.unit-testing.net/Xpath