I'm trying to use Execute XPath Function in automation anywhere.
In the response I'm getting the value as "MS.Internal.XML.XPath.XPathSelectionIterator"
XPath looks like :
//GetEmployees/Employee/text()
I'm sure that xpath is correct since I have tested with Java code.
Looks like the control is returning an object instead of text string.
Is there a way we can do it ?
Got the answer to this one by trial and error.
Was trying to parse soap response and I believe it was facing an issue with namespace.
got wit working by using the following expression
string(//soap:Envelope/soap:Body/*[name()='GetEmployees']/*name()='Employee'])
Hope this helps and some one could improve it further.
Related
I am trying Golang for the first time. I am trying to call a GET REST API which has a path variable. I am using net/http for that. I am trying like below but no luck so far. I need to know how I can use the path variable and pass the variable from the code. Any help or code example would be highly appreciated.
This does not seem to work:
http.Get("http://127.19.0.1:8080/student/:id")
what about
http.Get(fmt.Sprintf("http://127.19.0.1:8080/student/%s", id)
?
I have a column of links in Google Sheets. I want to tell if a page is producing an error message using importxml
As an example, this works fine
=importxml("https://en.wikipedia.org/wiki/List_of_postal_codes_of_Canada:_T", "//td/b")
i.e. it looks for td, and pulls out b (which are postcodes in Canada)
But this code that looks for the error message does not work:
=importxml("https://www.awwwards.com/error1/", "//div/h1" )
I want it to pull out the "THE PAGE YOU WERE LOOKING FOR DOESN'T EXIST."
...on this page https://www.awwwards.com/error1/
I'm getting a Resource at URL not found error. What could I be doing wrong? Thanks
after quick trial and error with default formulae:
=IMPORTXML("https://www.awwwards.com/error1/", "//*")
=IMPORTHTML("https://www.awwwards.com/error1/", "table", 1)
=IMPORTHTML("https://www.awwwards.com/error1/", "list", 1)
=IMPORTDATA("https://www.awwwards.com/error1/")
it seems that the website is not possible to be scraped in Google Sheets by any means (regular formulae)
You want to retrieve the value of THE PAGE YOU WERE LOOKING FOR DOESN'T EXIST. from the URL of https://www.awwwards.com/error1/.
If my understanding is correct, how about this answer? Please think of this as just one of several possible answers.
Issue and workaround:
I think that the page of your URL is Error 404 (Not Found). So in this case, the status code of 404 is returned. I thought that by this, the built-in functions like IMPORTXML might not be able to retrieve the HTML data.
So as one workaround, how about using a custom function with UrlFetchApp? When UrlFetchApp is used, the HTML data can be retrieved even when the status code is 404.
Sample script for custom function:
Please copy and paste the following script to the script editor of the Spreadsheet. And please put =SAMPLE("https://www.awwwards.com/error1") to a cell on the Spreadsheet. By this, the script is run.
function SAMPLE(url) {
return UrlFetchApp
.fetch(url, {muteHttpExceptions: true})
.getContentText()
.match(/<h1>([\w\s\S]+)<\/h1>/)[1]
.toUpperCase();
}
Result:
Note:
This custom function is for the URL of https://www.awwwards.com/error1. When you use this for other URL, the expected results might not be able to be retrieved. Please be careful this.
References:
Custom Functions in Google Sheets
fetch(url, params)
muteHttpExceptions: If true the fetch doesn't throw an exception if the response code indicates failure, and instead returns the HTTPResponse. The default is false.
match()
toUpperCase()
If this was not the direction you want, I apologize.
I want extract a number provided by javascript object in site, but I really don't understand that I am doing.
I tried different versions using alike examples and guidelines in import.io site and other tutorial sites, but I got only 1 of two results: extracted all numbers on given page or nothing at all.
I tried e.g. //[contains(.,"Unikālo apmeklējumu skaits:")]#type ; //[contains(.,"Unikālo apmeklējumu skaits:")] . Most likely it's necessary to add there something else, but I just don't know that.
Link I am interested in to extract from is: https://www.ss.lv/msg/lv/clothes-footwear/womens-clothes/trousers/ikcbb.html and information necessary is a number after text "Unikālo apmeklējumu skaits:" which is given by javascript.
Hopefully someone will be able to help me with this problem.
For someone who is new in web-scraping this should be a hard task, I'll ty to explain it. First of all, the xpath to get to that location could be something like this:
'//td[#class="msg_footer" and contains(text(), "Unik")]'
Now you have that tag (and what it contains), but if you check it doesn't contain the number you need, that content is being dynamically loaded with a javascript, and the javascript is this one:
<script type="text/javascript"><!--
var ss_w='rādīt numuru';
document.write( '<scr'+'ipt id="contacts_js" src="/js/2015-10-27/37863/VHoBGkpqSV8bfwkdTX9AXEpZXCVDlASIQ1ZV3kK.js?t='+new Date()+'"></scr'+'ipt>' );
--></script>
which could be gotten from the response with this xpath:
'//script[contains(text(), "contacts_js")]/text()'
from that string, you should replicate the url that comes in src, so this url for example:
/js/2015-10-27/37863/VHoBGkpqSV8bfwkdTX9AXEpZXCVDlASIQ1ZV3kK.js?t=
and add to the end the current date, as javascript creates it with new Date(). Then you should make a request to that url (adding the previous response domain), so something like:
https://www.ss.lv/js/2015-10-27/37863/VHoBGkpqSV8bfwkdTX9AXEpZXCVDlASIQ1ZV3kK.js?t=Wed%20Oct%2028%202015%2020:56:42%20GMT-0500%20(PET)
check that the date is urlencoded. it should return a response like:
var PHONE_CNT=-1;var PHONE_CNT2=-1;var PHONE_CNT3=-1;var EMAIL_CNT=-1;var SHOW_CNT=22;var PH_c="";var PH_1=0;var PH_2=0;var PH_3=0;
pcc_id=0;PH_1=gpzd("JTg3aCU3QyU1QnolN0MlN0JYcWh6JTVCdCU5NSU4QyU5MnV4ayU5QXElN0IlOTQlNUNweiU5MGtvJTdCJThFJTVF","55937369");
where you can check that the value inside SHOW_CNT is the number you want.
If you want to know how I figured out which request and which script was populating that response tag, well that I did using firebug, searching for SHOW_CNT inside all of the responses that involve calling to your URL, which pointed to the request I specified, and then trying to check who was requesting that.
Hope it helped.
support#import.io are the guys to speak to, they give free advice and help trouble shoot problems just like this all the time.
There are all kinds of tips and tricks you can use... for example import.io provide (an undocumented beta) JavaScript Pre-render service that would likely work for you in this scenario. API publish failures are sometimes caused by timeouts while waiting for sites to render JS, this would fix that.
http://support.import.io/knowledgebase/articles/623235-infinite-scroll-and-javascript-prerender-beta
I hope this helps.
How Can i extract value from DWR call using RegEx in jmeter.
In My Script Request Body is something like :
POST data:
`callCount=1`
`page=/abc/xyz.action`
`httpSessionId=`
`scriptSessionId=87F34A5261EFBF481F6D421920EF99F9406`
`c0-scriptName=DWRPostHelper`
`c0-methodName=savePostAsDraft`
`c0-id=0`
`c0-param0=string:447`
`c0-param1=number:933`
`c0-param2=number:0`
`c0-param3=string:Post%20Jmeter`
`c0-param4=string:`
`c0-param5=string:1`
`c0-param6=number:1427199824585`
`batchId=4`
And Response data is as below :
`//#DWR-INSERT`
`//#DWR-REPLY`
`dwr.engine._remoteHandleCallback('4','0',["447","Auto-saved at 17:55"]);`
Need to Extract value 447 from above Response.
is it Possible Using RegEx ? or is there any other way to do the same?
'dwr\.engine\._remoteHandleCallback\('[0-9]','[0-9]',\["([0-9]+)",
This should do it, written from memory, you may need to tweak the syntax a bit. Use $1$ for the template.
Try the following:
dwr\.engine\._remoteHandleCallback\('4','0',\["(.+?)"
Make sure that you set $1$ as "Template"
You can test your regular expressions using View Results Tree listener as follows:
See How to debug your Apache JMeter script guide for more details on how to get to the bottom of failure or unexpected behavior while developing a JMeter Test
I'm running in to an issue in that I need to get one value in a cucumber statement, and then give that value to another statement.
Specifically I am getting a JSON object from one page (where that object gets sent to an api endpoint as a preference) and then using information out of that after I query the api, which happens in a completely separate step.
I am suspecting that I have to write the value somewhere, and then pull that value when the step that needs it comes up, but I'm kind of at a loss for doing that as well.
I can provide any further needed details, thanks for any help!
Definitely a rookie question - to which the answer is to use instance variables - #variable_name = get_method in the helper method your step calls.