How to Web Scrape the Site in Excel/Google Sheets? - xpath

Question

How should I scrape this webpage https://www.bseindia.com/stock-share-price/asian-paints-ltd/asianpaint/500820/ and specifically need the ROE figure which is mentioned in the table?
I used the following code in Excel. I don't know much about Google Sheets Scraping
Sub FetchData()
With ActiveSheet.QueryTables.Add(Connection:= _
"URL;https://www.bseindia.com/stock-share-price/asian-paints-ltd/asianpaint/500820/", Destination:=Range( _
"$A$1"))
.Name = "www"
.FieldNames = True
.RowNumbers = False
.FillAdjacentFormulas = False
.PreserveFormatting = True
.RefreshOnFileOpen = False
.BackgroundQuery = True
.RefreshStyle = xlInsertDeleteCells
.SavePassword = False
.SaveData = True
.AdjustColumnWidth = True
.RefreshPeriod = 0
.WebSelectionType = xlEntirePage
.WebFormatting = xlWebFormattingNone
.WebPreFormattedTextToColumns = True
.WebConsecutiveDelimitersAsOne = True
.WebSingleBlockTextImport = False
.WebDisableDateRecognition = False
.WebDisableRedirections = False
.Refresh BackgroundQuery:=False
End With
End Sub
I was not able to fetch the data properly.
Any suggestions/help on this? Need the ROE figure rest of it is not required.

Answer 1

Use the API that the page uses it much faster. You can use powerquery to handle json response, a json parser or just use split. Put code in a standard module and link to a button if you want to refresh at a button press.
Option Explicit
Public Sub GetInfo()
Dim s As String, ids(), i As Long
ids = Array(500820, 500312, 500325, 532540)
With CreateObject("MSXML2.XMLHTTP")
For i = LBound(ids) To UBound(ids)
.Open "GET", "https://api.bseindia.com/BseIndiaAPI/api/ComHeader/w?quotetype=EQ&scripcode=" & ids(i) & "&seriesid=", False
.send
s = .responseText
ActiveSheet.Cells(i + 1, 1) = Split(Split(s, """ROE"":""")(1), Chr$(34))(0)
Next
End With
End Sub

Answer 2

Following is the way I find easier to get that particular value. Once the for loop detects ROE, it will go after the required value and exit the loop as they both are within the same parent node.
Sub FetchData()
Dim IE As New InternetExplorer, post As Object
Dim Html As HTMLDocument, elem As Object
With IE
.Visible = False
.navigate "https://www.bseindia.com/stock-share-price/asian-paints-ltd/asianpaint/500820/"
While .Busy Or .readyState < 4: DoEvents: Wend
Set Html = .document
End With
For Each post In Html.getElementsByTagName("td")
If post.innerText = "ROE" Then
Set elem = post.ParentNode.querySelector(".textvalue")
Exit For
End If
Next post
[A1] = elem.innerText
End Sub
References to add:
Microsoft Html Object Library
Microsoft Internet Controls

Answer 3

unfortunately, that won't be possible because the site is controlled by JavaScript and Google Sheets can't understand/import JS. you can test this simply by disabling JS for a given link and you will see a blank page:
all you can get is what you see:
=ARRAY_CONSTRAIN(IMPORTDATA("https://www.bseindia.com/stock-share-price/asian-paints-ltd/asianpaint/500820/"), 5000, 15)