Using the recaptcha javascript client ( http://www.google.com/recaptcha/api.js?onload=vcRecaptchaApiLoaded&render=explicit ) the automatic request (GET https://www.google.com/recaptcha/api2/userverify?k=XXXXXXXXX) after you select the correct images returns an invalid json which afterwards is sent to back to google and their json validation fails. This worked as expected for months now. Not anymore :(
Server side library used reCaptcha PHP (1.1)
Response
)]}'
["uvresp","03AHJ_Vuup5SJJ583dSIfezFl80dp2AlJ_rZpz3vqGWlOTmbjZqH8izwjruJASNhQI0tOOnmj2Pzg14xMw7dryeqVfTGhx6dg_x2i0PRA1ZeDyrBNn8DX-w5S262Zb3_ZWKj5JDBiqPVnXtNLGbyBxjd97VHanspJWU_-qKWLSWVKxLK6n3lm9Biw33oUCEiGA39GNa09Z6TSAEtnolQCf_LPPRWKoE_e50f2s5ZpUVG5GNVdX7qGBRwphTgcUhwOjA8uYTzmA9co3Jwk2KR5UQ0zzVRJzRZZTBuK9km3PE1WV05ACAwrJi29niDpVaRmpooAnIkHNgGyGBu7u3W7gU6YAHtwya8PYhdF__G_MMG8XpVFDTBa196hKD6hxw-E2PsxmoIQJrU1K89mmzNIh-xLNQ7KJvrBMzVf8A5FHyUQgL5UNDWVwSkWCdC_3swxBzi7R3p8VIrUtkIqJFA_GSAxy0cBRJ8J55Pfs5rzhfR8j-x1hGCzi_6vJrbrwfNesoLEB7GWJtElcljhBYvcDNzU_B_VJ7Sck-6i1Nd0qdmtSiCRZYNyaZ8uGLoDdNgCY-0Oi4802AlI26H7TjGBcKnr4gmaHXTNRf1W7x_3FV05DxVsTqeAlo8zGqmiVqcgmX64BbLK4fD9Xoait1_Lp5vK26fCaOQmGKF7CJaYPuxnX-zXgSkfZDCG6rs6xv1CfZQnIKD0W3Yz522VD4YdNfATb3FywhFWbZuxoBIt6vslZDlPXh2MYOkAmYfIPIo8WoWazMoLI_8iNBZPiMlRL0PS5aQiLSrvbf-sknMHhfM2MJYsfrQjC52aDRaHYdcZbY6Wxlhw0tQEknX8B47_DAQzCKkpoFsecO1eMHuInIykZ1l7TOdZMytI-NzGg21KeKAE8dK6ZWee0UEqDJvCkj5aH5TQcBA--ygbOS186bAptUP5n6WvORx1Nb2ZU_AF9fB23PJWH1xvB4gZoNDvhLmdpkE1Po9Lyim1P61E2rrgYjWgPRwT4jUo",1,120]
Any ideas?
Thanks
I think it's a duplicate of https://stackoverflow.com/a/36862268/2140017
And here is my answer from that link:
Actually the value returned is not valid json but well parsed by the
Google's API.
Is it a protection ? I don't know, but if you look at the javascript,
you can find that:
var jm=function(a,b,c,d,e,g,h,l,r){this.xl=a;this.$c=c||"GET";this.Ka=d;this.Gg=e||null;this.Td=m(h)?h:1;this.ye=0;this.xh=this.Nh=!1;this.uh=b;this.Mh=g;this.md=l||"";this.Zb=!!r;this.Wf=null};f=jm.prototype;f.getUrl=function(){return this.xl};f.ug=function(){return this.$c};f.Ca=function(){return this.Ka};f.fi=function(){return this.Zb};f.bi=function(){return this.md};var nm=function(){G.call(this);this.nj=new hm(0,mm,1,10,5E3);H(this,this.nj);this.ad=0};x(nm,G);var mm=new Nh;nm.prototype.send=function(a){return new Lc(function(b,c){var d=String(this.ad++);this.nj.send(d,a.Uf.toString(),a.ug(),a.Ca(),mm,void 0,u(function(a,d){var h=d.target;if(Xk(h)){var l=a.ml;h.B?(h=h.B.responseText,0==h.indexOf(")]}'\n")&&(h=h.substring(5)),h=Hg(h)):h=void 0;b(new l(h))}else c(new om(a))},this,a))},this)};var om=function(a){y.call(this);this.request=a};x(om,y);
especially take a look at:
var l=a.ml;h.B?(h=h.B.responseText,0==h.indexOf(")]}'\n")&&(h=h.substring(5)),h=Hg(h)):h=void 0;`
The parser explicitly checks that the value begins by )]} and strips
it.
I suggest you to just apply the same substring on the "json" string
Related
Now I tried to make like a translator through Roblox Studio using Https service by sending a request to the translate.google.com link the thing is that anything I get in return does not give me the translated text.
I put what I received in a google doc and tried to find it by pressing ctrl + f to try to find it but no luck the only thing I could find is that text that was supposed to be translated. Here is the code in case you want to try it for yourself but I do warn you that running this might make Roblox unresponsive for a while since it is a lot of info they gave back.
I don't know if I am doing something wrong or not someone please help! I just want it to give me what 'Hello world' would be in french, there are also no error messages.
local http = game:GetService("HttpService")
local Message = "Hello world"
http:UrlEncode(Message) -- 'Hello world' -> 'Hello%20world'
local response = http:RequestAsync(
{
Url = "https://translate.google.com/?sl=en&tl=fr&text=" .. Message .. "!&op=translate";
Method = "GET"
}
)
if response.Success then
print(response.StatusMessage)
print(response.StatusCode)
print(response.Body)
--print(response.Headers)
else
print("The request failed: ", response.StatusCode, response.StatusMessage)
end
When visiting on your browser (for example) the url https://translate.google.com/?sl=en&tl=fr&text=Hello%20World!&op=translate, the translation you see is fetched using Javascript code executed by the browser after loading the page.
The browser retrieves the html body of the page (like you did in your code) and then executes the javascript in the html body which retrieves the translation and updates the page.
Unless you use a browser driver like Selenium I don't see how you can do what you want in a simple way.
Plus, I'm sure that Google has some protection against automatic bots, so after too many request your program will probably will be blocked by ReCaptcha.
The correct way to translate the text is to use the Google Cloud Translate API which I think is free up to 500k requests per month. There is also Azure Translator from Microsoft which also has a free tier.
Your issue is likely in how you are URL Encoding the string.
http:UrlEncode(Message)
HttpService.UrlEncode returns the encoded string as a new value. It doesn't mutate the existing value. So you just need to store the result of the function call.
Message = http:UrlEncode(Message)
EDIT : Just as #Mohamed AMAZIRH pointed out, hitting this URL will only return HTML.
I have a column of links in Google Sheets. I want to tell if a page is producing an error message using importxml
As an example, this works fine
=importxml("https://en.wikipedia.org/wiki/List_of_postal_codes_of_Canada:_T", "//td/b")
i.e. it looks for td, and pulls out b (which are postcodes in Canada)
But this code that looks for the error message does not work:
=importxml("https://www.awwwards.com/error1/", "//div/h1" )
I want it to pull out the "THE PAGE YOU WERE LOOKING FOR DOESN'T EXIST."
...on this page https://www.awwwards.com/error1/
I'm getting a Resource at URL not found error. What could I be doing wrong? Thanks
after quick trial and error with default formulae:
=IMPORTXML("https://www.awwwards.com/error1/", "//*")
=IMPORTHTML("https://www.awwwards.com/error1/", "table", 1)
=IMPORTHTML("https://www.awwwards.com/error1/", "list", 1)
=IMPORTDATA("https://www.awwwards.com/error1/")
it seems that the website is not possible to be scraped in Google Sheets by any means (regular formulae)
You want to retrieve the value of THE PAGE YOU WERE LOOKING FOR DOESN'T EXIST. from the URL of https://www.awwwards.com/error1/.
If my understanding is correct, how about this answer? Please think of this as just one of several possible answers.
Issue and workaround:
I think that the page of your URL is Error 404 (Not Found). So in this case, the status code of 404 is returned. I thought that by this, the built-in functions like IMPORTXML might not be able to retrieve the HTML data.
So as one workaround, how about using a custom function with UrlFetchApp? When UrlFetchApp is used, the HTML data can be retrieved even when the status code is 404.
Sample script for custom function:
Please copy and paste the following script to the script editor of the Spreadsheet. And please put =SAMPLE("https://www.awwwards.com/error1") to a cell on the Spreadsheet. By this, the script is run.
function SAMPLE(url) {
return UrlFetchApp
.fetch(url, {muteHttpExceptions: true})
.getContentText()
.match(/<h1>([\w\s\S]+)<\/h1>/)[1]
.toUpperCase();
}
Result:
Note:
This custom function is for the URL of https://www.awwwards.com/error1. When you use this for other URL, the expected results might not be able to be retrieved. Please be careful this.
References:
Custom Functions in Google Sheets
fetch(url, params)
muteHttpExceptions: If true the fetch doesn't throw an exception if the response code indicates failure, and instead returns the HTTPResponse. The default is false.
match()
toUpperCase()
If this was not the direction you want, I apologize.
How to place comments inside Postman? Specifically in the JSON request body section?
I want to comment-out a particular key or value from the request body so that it is not sent.
Commenting out a JSON key/value pair with // or /* ... */ appears as a styled comment inside Postman:
But sending this request results in server errors such as the below, and it's clear that the commented-out line is being sent as part of the request body:
Unexpected character ('/' (code 47)): maybe a (non-standard) comment? (not recognized as one since Feature 'ALLOW_COMMENTS' not enabled for parser)
at [Source: (org.glassfish.jersey.message.internal.ReaderInterceptorExecutor$UnCloseableInputStream); line: 2, column: 6
]
The JSON spec does not allow comments: Can comments be used in JSON?
I want Postman to strip the commented lines prior to being sent in the request.
You can write documentation and comments using the description section of the requests, collections or folders.
Finally, since Postman v8.3.0 you can do this in your collections pre-request script:
// Strip JSON Comments
if (pm?.request?.body?.options?.raw?.language === 'json') {
const rawData = pm.request.body.toString();
const strippedData = rawData.replace(
/\\"|"(?:\\"|[^"])*"|(\/\/.*|\/\*[\s\S]*?\*\/)/g,
(m, g) => g ? "" : m
);
pm.request.body.update(JSON.stringify(JSON.parse(strippedData)));
}
This strips all comments from the json and sets the current body to the cleaned one, there are more examples for other body types (GraphQL, URL Encoded, Form Data) in the original github post this code is based from.
A "Comments" option/button is above the send button for every request.
But still, we can't add a comment in the request body, maybe in future, they'll provide this feature.
I checked with GitHub where Postman has their feature requests and bugs tracked, Link here:
Feature request: Raw body editor JSON comment #3358
https://github.com/postmanlabs/postman-app-support/issues/3358
You can go and add comments there so that this feature would be considered for their next release. Also, I found out that you can copy and paste <!-- comment--> to make a comment.
It has been done by Script
https://community.postman.com/t/request-body-should-be-able-to-be-commented/8288
Pre-request Script create an object, convert it to a string, then expose it
object = {
// product: "{{displayName}}",
price : "15.5"
}
pm.environment.set("object", JSON.stringify(object));
Request Body called the variable object
{{object}}
In Postman v9.13.0 you can do so using block comments
Not the actual question's answer, but this addresses the highly upvoted comment below the question:
I just want to disable some code not want to add comments or descriptions..how can I do that? – Mahender Reddy Yasa
Ignoring the comment validation and sending request is now working for me in latest postman 7.7.3 64 bit version of windows.
I'm trying to work with the IUCN Red List web services API (here's an example output). Unfortunately I haven't been able to find any documentation other than this one-off Gist. It looks as though the API is constructing an HTML document rather than returning a data object, which isn't something I've experienced in the past. I also notice that in the example there is no mention of a ?callback=JSON_CALLBACK in the URL, which I would expect when dealing with JSONP.
I've constructed an http request in AngularJS like so:
atRiskApp.controller('IucnController', ['$scope', '$routeParams', '$http', function ($scope, $routeParams, $http) {
$scope.iucn = $routeParams.iucn; // pulling a number from the URL: ex. 22718591
$scope.getIUCN = function () {
var iucnUrl = 'http://api.iucnredlist.org/details/' + $scope.iucn + '/0.js';
$http.jsonp( url )
.success( function (response) {
console.log(response);
})
.error( function (response) {
console.log(response);
});
};
}]);
Although the HTML document is being successfully passed to my app I'm getting the following error message:
Uncaught SyntaxError: Unexpected token <
It seems like the app is expecting to get Javascript, and is instead getting an HTML document, which it apparently can't parse. I've tried adding a config object to the request based on the angular docs: $http.jsonp( {url: iucnUrl, responseType: 'text'} ) without any luck.
My question is, how do I work with the returned HTML document, or am I way off track here?
Response from the API is an HTML document with a javascript extension:
On the page you linked to in your comment , I found some potentially useful information under the heading API Index.
You can actually get JSON for all levels of taxonomy, including your example Aneides aeneus. However, this JSON doesn't include all of the data from the HTML version, so it's not as useful. Hopefully this helps a little.
API Index (excerpt)
It is also possible to retrieve the row(s) of the index corresponding to an individual species:
http://iucn-redlist-api.heroku.com/index/species/panthera-leo.json
You can use dashes for spaces, as a convenient replacement for the standard URL escape, %20.
The HTML format contains direct links to the species account pages. The CSV and JSON formats include a species_id column which can be used to construct species account URLs as follows:
http://iucn-redlist-api.heroku.com/details/species_id/0
To use the index JSON in Web pages directly, you may need JSONP padding; use the “.js” extension and add a “callback” parameter with the name of the function to use.
http://iucn-redlist-api.heroku.com/index/genus/Dioscorea.js?callback=show
I diagonally looked over the website and its sitemap and found no reference to a public API. All the output is HTML, and it makes sense that json parse method jsonp will not be able to make sense of it. First < it encounters, it will fail (as is apparent).
First of all, I would contact the site admin to simply ask if there is an API that will yield you XML or json or some other object notation that's convenient to work with.
Then there's the scenario where his or her answer would be 'no':
Parsing HTML is not something to be taken lightly and certainly not something you would write yourself unless absolutely necessary.
Luckily, there are ways to get data from html using jQuery.parseHTML(), pure ('vanilla') javascript ways you can use from within AngularJS and full-blown HTML parsing libraries such as HTML Agility Pack(for use in .NET), all of which can get you to the heart of the data within the DOM nodes you're trying to poke at.
There are many other libraries that might serve you better, but these examples will give you a good starting point to canvas the landscape of HTML parsing. This will take some looking into, but it will be more than worth it.
In my codeacademy and codeschool lessons, I've been fetching data from google rss and simulated twitter feeds.
My newest exercise, however, involves fetching an array of text data from a REST API.
When I try
$.get('https://api.parse.com/1/classes/chats?order=-createdAt', function(x){$('.messages').append('<li>'+x.responseText+'</li>');});
I get
which has the text and username I need. Sort of...
but when I try to alert or console.log either *x.responseText or x.responseText.results I obviously get undefined instead of an array.
What am I missing?
Study more AJAX and I'll find a technique?
Or do I have to send special instructions to the parse server using some commands found here.
You are not using XMLHttpRequest directly, you are using jQuery and it will read the responseText and handle it for you.
Just use x (or, rather, x.results).