So How To Use importXML By Google Docs Spreadsheet Plus xPath To Help Me Copy Soundcloud Playlist Track Title?
Today I'm looking for a solution on how to copy the Track Title of tracks inside a Soundcloud Playlist/Compilation.
I searched around and discovered importXML function offered by Google Docs Spreadsheet. While digging further about the importXML function, I discovered about xPath.
Great combination! I thought.
So I quickly get my hands on the tools and tested it, and it's great! I've extracted some data. And so I decided that I'm ready to implement the tool and use it with Soundcloud.
But when I tried to implement the syntax, I got an error saying Import Internal Error
The Syntax Is
=IMPORTXML(A1,"//div[#class='sc-media-content']/a[#title]")
<div class="sc-media-content"> is the div that holds the Track Title of the song, that is enclosed within an anchor tag with title attribute.
Here is the html block for it;
<div class="sc-media-content">
<a class="soundTitle__title sc-link-dark sc-truncate " href="/seven-lions/velvetine-the-great-divide?in=thedubstepgod/sets/melodic-dubstep-chillstep" title="Velvetine - The Great Divide (Seven Lions Remix)">
Velvetine - The Great Divide (Seven Lions Remix)
</a>
What I'm trying to extract is the Velvetine - The Great Divide (Seven Lions Remix). A1 is the cell where the Soundcloud Playlist Link is pasted into.
Other Syntax I've Tried
I've tried other syntax too, like;
=IMPORTXML(A1,"//div[#class='sc-media-content']/title")
As suggested here
=IMPORTXML(A1,"//div[#class='sc-media-content']/#title") From an answer found here, though it wasn't an accepted answer.
So what am I doing wrong? How could I copy those soundcloud playlist title to my Google Docs Spreadsheet using xPath?
UPDATE
Based from the answer given by TGH, this should work //div[#class='sc-media-content']/a/text().
But, the problem is that the div block that we're looking for is not in the source code. I did a view source code on the Playlist's page and the div block is not there. All I'm seeing is JavaScripts. So JS is loading the div blocks/classes.
So another question might be needed to help solve this one
How to use xPath with JavaScript loaded html elements?
Try the following
//div[#class='sc-media-content']/a/text()
Or if you want to grab it from the title do this
//div[#class='sc-media-content']/a/#title
I tested it here and it seems to work. Pasted your html, but had to close the div manually.
http://www.unit-testing.net/Xpath
Related
finally decided to sign up to stackoverflow because of this. So I´d be super grateful about a solution!
I´m trying to get a number of a <span> element. Here is an image of the data box I´m trying to scrape from. It´s on this page: https://de.marketscreener.com/kurs/aktie/SNOWFLAKE-INC-112440376/analystenerwartungen/
The relevant Xpath is //*[#id="highcharts-0oywbsk-200"]/div[2]/div/span/span
I´m trying: =IMPORTXML("https://de.marketscreener.com/kurs/aktie/SNOWFLAKE-INC-112440376/analystenerwartungen/"),"//div[2]/div/span/span")
I´m ignoring the #id-element, this works pretty well with many elements on the same page, but in this case not at all. I ignore the id, because I can´t use it as it changes on every page. Is this ok?
Google Sheets always gives me a #N/A error?! Any idea how to scrape that number?
disabling JavaScript reveals what you can scrape:
I'm trying to set the right xpath for using RSelenium, but I'm not very experienced in this area, so any help would be much appreciated.
Since I'm not allowed to post pictures yet I have tried to add a link to a screenshot of the html:
The html
I need R to scrape the dates (28-10-2020 - 13-11-2020), but so far I have not been able to set the correct xpath when using html.nodes.
I'm trying to scrape from sites like this one: https://www.boligsiden.dk/adresse/topperne-9-3-33-2620-albertslund-01650532___9__3__33
I usually do this on python rather than R
As you can see in this image when you right-click on the element concerned. You get a drop-down menu with an x-path to the element.
Other than that, the site orientation and x-path might change and a full x-path might be a good option in the short-run, so I rather prefer driver.find_element_by_xpath('//button[contains(text(),"Login")]')\ .click()
In your case which would be find_element_by_xpath('//*[contains(#class, 'u-pb-4 u-block')]')
I hope this helps and it is mostly the same across different languages
There is a google sheet containing a list of MPN's (manufacturer part numbers). Trying to scrape a site called wikiarms for the UPC Codes when I have the MPN for an item.
I have the correct formula for doing this on another site.
=IMPORTXML("http://gun.deals/search/apachesolr_search/"&B1,"//dd/a[../../dt[contains(text(),'UPC')]]|//dd/span[../../dt[contains(text(),'UPC')]]")
Trying to figure out what the correct xpath to complete this formula. Some videos I have watch said to open the page in Chrome and use inspector to select and copy the xpath to complete the importxml function. I tried this with no luck.
Sample
Visit https://www.wikiarms.com/guns?q=20071
In the table there is a button "available in 6 stores" click that to reveal the list. The UPC should be listed after the MPN.
If I copy the xpath in Chrome this is the result
/html/body/div[1]/div/div/div[2]/div/div/div[2]/div[2]/table/tbody/tr[2]/td[5]
=IMPORTXML("https://www.wikiarms.com/guns?q="&B2,"xpath here")
What do I have to add at the end of this formula to pull in the UPC code? I will be using this formula to pull in UPC code for about 1000 items.
Thank you for your help.
Using your sample link, try
=IMPORTXML("https://www.wikiarms.com/guns?q=20071","//td[#class='upc']/a/#title")
and see if it works for you.
I can't figure out the correct XPath to use in Google Sheets for extracting the 3 (three) "Yield on Cost" values at the top of this page: https://seekingalpha.com/symbol/GE/dividends/yield-on-cost
Or the 5 (five) "Growth" values at the top of this page: https://seekingalpha.com/symbol/GE/dividends/dividend-growth
I have read many informative webpages on the subject of using both IMPORTXML and IMPORTHTML within Google Sheets, and I have had good success in the past -- but I am coming up empty on these two pages. This is likely due to me missing some fundamental problem that others would probably see and understand immediately.
How can I achieve my goal?
https://seekingalpha.com/ site can't be scraped because it depends on JavaScript and Google Sheets can't work with JavaScript at all. You can see yourself if you disable JS temporarily:
I am trying to scrape some data from the following website: https://xrpcharts.ripple.com/
The data I am interested in is Total XRP which you can see immediately below or to the side (depending on your browser) of the circle diagram. So what I first did was inspect the element I am interested in. So I see that it is inside <div class="stat" inside span ng-bind="totalXRP | number:2" class="ng-binding">99,993,056,930.18</span>.
The number 99,993,056,930.18 is what I am interested in.
So I started in a scrapy shell and wrote:
fetch("https://xrpcharts.ripple.com")
I then used chrome to copy the Xpath by right clicking on that place of HTML code, the result chrome gave me was:
/html/body/div[5]/div[3]/div/div/div[2]/div[3]/ul/li[1]/div/span
Then I used the Xpath command to extract the text:
response.xpath('/html/body/div[5]/div[3]/div/div/div[2]/div[3]/ul/li[1]/div/span/text()').extract()
but this gave me an empty list []. I really do not understand what I am doing wrong here. I think I am making an obvious mistake but I dont see it. Thanks in advance!
The bottom line is: you cannot expect the page you see in the browser to be the same page Scrapy would download and have available to work with. Scrapy is not a browser.
This page is quite dynamic and complex and is constructed with the help of multiple asynchronous requests bringing in both the logic and the data. There is also JavaScript executed in the browser that plays an important role in forming and supporting the HTML document object tree.
Scrapy does not have all these things, the thing you get when you do fetch() is just the very first initial "bare bones" HTML page without all the "dynamic content".