Golang Simulate Page Visit : Surf Not Triggering Javascript - go

I have a Go program that is just simulating a local web page being visited using Surf (gopkg.in/headzoo/surf.v1) but it seems that the Javascript on the page is not being executed. Just to test I have the web page doing an ajax call to another route on the server that will just output a response on the server side.
<html>
<head>
<script src="jquery.min.js"></script>
</head>
<body>
<script>
$.ajax ({
url : "/test",
type : "GET",
success : function(res) {},
error : function(res) {}
});
</script>
</body>
</html>
Then the Go Surf function to visit the page, this route also outputs a response on the server side which is showing up but the one from the ajax request is not showing up.
func simulateVisit() {
bow := surf.NewBrowser()
bow.SetUserAgent("LocalCrawler/1.0")
err := bow.Open("http://127.0.0.1/page")
if err != nil { fmt.Println(err) }
}
To make sure it is only happening from Surf I also tested it with Chromedp (github.com/chromedp/chromedp) which gave both responses correctly.
Is this a limitation with Surf or am I doing something silly?

Surf does not include a JS runtime. It does support the downloading of scripts, but it will never actually execute them.

Related

I can't seem to generate content for dynamic content load

I am using wordpress and php along with ajax to create a random loading of customer reviews on our main page
function loadContent() {
$.ajax({
type: "GET",
url: 'http://skillsetsonline.ssosv.com/contentLoader.php',
data: {
company: 1
},
success: function(data) {
alert(data);
var currReview = document.getElementById('reviewRand');
currReview.innerHTML = data;
}
});
}
setTimeout(loadContent, 10000); // milliseconds, so 10 seconds = 10000ms
<div id="reviewRand" class="elementToFadeInAndOut" style="font-color:#FFF;">Hi how are you</div>
I pasted the ajax command in from a stackoverflow posting that was an accepted answer but may not have it exactly right this does not include the fading CSS code I use but that is working I just need to change the content.
Currently "Hi how are you" fades in every 10 seconds. One thing I have not learned about yet with this ajax command is the
data:{company:1}
I know it simply passes &company=1 to the GET URL but in my case I do not need to send anything and since it should not break anything if it is sent I left it alone not sure if
data:{}
would work and be cleaner
I have verified that the url used does get a random review
formatted like this
I love this program.blah blah.<br>
A USER<br>
A location<br>
June 2016<br>
Each line is formatted in CSS via a class tag
Any ideas would be greatly appreciated
Since the domain you're making the AJAX request to is on a different domain/origin, what you're running in to is a CORS issue. By default, the client will not allow you to update the page with data from an AJAX request that is served on a different origin than the site where the request originated. You can read about making CORS changes here https://enable-cors.org/
A common way around this is to serve the response via JSONP. You can do this in your script at http://skillsetsonline.ssosv.com/contentLoader.php if you have access to change that file. There are also third-party sites that will request that URL for you and create a proxy that serves the response via JSONP, then you can use it on your website.
Here's an example utilizing a JSONP proxy on https://crossorigin.me
function loadContent() {
$.ajax({
type: "GET",
url: 'https://crossorigin.me/http://skillsetsonline.ssosv.com/contentLoader.php',
success: function(data) {
var currReview = document.getElementById('reviewRand');
currReview.classList.add('ready');
currReview.innerHTML = data;
}
});
}
setTimeout(loadContent, 0); /* changed this for the demo */
#reviewRand:not(.ready) {
text-align: center;
}
<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
<div id="reviewRand"><img src="http://thinkfuture.com/wp-content/uploads/2013/10/loading_spinner.gif"></div>

API request from front-end using sails.io.js

I have a basic front-end (html, css, jquery) and I'd like to use sails.io.js to communicate with an API server (developped with sails, with cors enabled). The API is running on localhost:10000 but it will be on an another domain than the one of the webclient later on.
Directly from jquery, I can issue some get request to this API and get the expected results.
When it comes to websocket, I have some problems...
In the index.html (just to test), I put the following:
<script src="js/sails.io.js"></script>
<script type="text/javascript">
io.sails.url('http://localhost:10000');
io.socket.get('/data', function serverResponded (body, sailsResponseObject) {
// body === sailsResponseObject.body
console.log('Sails responded with: ', body);
console.log('with headers: ', sailsResponseObject.headers);
console.log('and with status code: ', sailsResponseObject.statusCode);
});
</script>
But Chrome's developer tools tell me
ReferenceError: io is not defined
Any idea ?
UPDATE
I'm serving index.html with a web server (python -m SimpleHTTPServer)
I've installed sails.io.js using bower.
I've try to make this test as simple as possible:
index.html
<!DOCTYPE html>
<html lang="en">
<head>
</head>
<body>
<script src="bower_components/sails.io.js/dist/sails.io.js"></script>
<script src="index.js"></script>
</body>
</html>
index.js:
window.onload=function(){
io.sails.url = 'http://localhost:10000';
io.socket.get('http://localhost:10000/data', function (body, response) {
console.log('Sails responded with: ', body);
});
};
My sails (0.9.16) API is only returning a json object on the GET /data route.
I have implemented a dummy __getcookie function in the api:
'get /__getcookie': function(req, res, next){
res.json({ok: 123});
}
And commented the line 481 in interpret.js (Scott comments below).
I have also modify config/socket.js with:
authorization: false,
=> I can now get the result from the /data route of my API :)
But... on each request I have the following error:
error: Error: No valid session available from this socket.
First of all, sails.io.js includes the code for socket.io.js, so there is no need to try and include that separately. You should remove this line:
<script src="bower_components/socket.io/lib/socket.js"></script>
Next, if you're just loading index.html from disk (rather than serving it from a web server), you'll need to tell the Sails socket client what URL to connect to:
io.sails.url = 'http://localhost:10000';
Put this anywhere before you start making socket calls; the library is smart enough to wait until its connected before trying to make the calls. So, altogether:
window.onload=function(){
io.sails.url = 'http://localhost:10000';
io.socket.get('http://localhost:10000/data', function (body, sailsResponseObject) {
console.log('Sails responded with: ', body);
console.log('with headers: ', sailsResponseObject.headers);
console.log('and with status code: ', sailsResponseObject.statusCode);
});
};
should work. You should be able to see in the console whether or not the socket connected by looking for the "io.socket connected successfully." message.
did you try with a / in front of the src, like:
< script src="js/sails.io.js">
Do you have the sails.io.js in the /assets/js/ folder (sails 0.10) or in the /assets/linker/js folder (sails 0.9 and below).
Did sails lift copied that js file to .tmp/public/js folder?
Where is your index.html file located?

Loading local content through XHR in a Chrome packaged app

I'm trying to load in a web app that I've built using Backbone and it pulls in JSON and HTML template files that are stored locally. I was wondering with Chrome packaged apps whether it's possible to load these files by using some sort of 'get'/ajax request?
Currently I'm getting this...
OPTIONS chrome-extension://fibpcbellfjkmapljkjdlpgencmekhco/templates/templates.html Cannot make any requests from null. jquery.min.js:2
XMLHttpRequest cannot load chrome-extension://fibpcbellfjkmapljkjdlpgencmekhco/templates/templates.html. Cannot make any requests from null.
I can't find any real information on how to do this so any help would be great thanks!
Yes, it's totally possible, and it's easy. Here's a working sample. Try starting with this, confirm that it works, and then add back in your own code. If you hit a roadblock and come up with a more specific question than whether XHRs work in packaged apps, you might want to ask a new question.
manifest.json:
{
"name": "SO 15977151 for EggCup",
"description": "Demonstrates local XHR",
"manifest_version" : 2,
"version" : "0.1",
"app" : {
"background" : {
"scripts" : ["background.js"]
}
},
"permissions" : []
}
background.js:
chrome.app.runtime.onLaunched.addListener(function() {
chrome.app.window.create("window.html",
{ bounds: { width: 600, height: 400 }});
});
window.html:
<html>
<body>
<div>The content is "<span id="content"/>"</div>
<script src="main.js"></script>
</body>
</html>
main.js:
function requestListener() {
document.querySelector("#content").innerHTML = this.responseText;
};
onload = function() {
var request = new XMLHttpRequest();
request.onload = requestListener;
request.open("GET", "content.txt", true);
request.send();
};
content.txt:
Hello, world!
You are making a request from a sandboxed page, and sandboxed pages have a null origin.
I have posted this issue question on the Google Group.
Unless Chrome decides to changed the sandbox policy, it appears the only workaround is to make XHR requests from a non-sandboxed page and use Chrome's message passing API to give it to your sandboxed page.
I don't know why it has to be so difficult.
EDIT:
The answer from the Chrome Team was to change the CORS header to *.
I believe your problem is on the server side, rather than the client side. The server needs to send the following header for jQuery to deal with the response:
Access-Control-Allow-Origin: *
The problem, with this, however, is that any page can load that content now. Once you know the ID of your extension, you can change that header to something like:
Access-Control-Allow-Origin: chrome-extension://gmelhokjkebpmoejhcelmnopijabmobf/
A short test of something like the following showed these to work:
<h1>Content Below</h1>
<div id="loadme"></div>
<script src="jquery-1.9.1.min.js"></script>
<script src="app.js"></script>
// app.js
$(document).ready(function() {
$.get('http://localhost:8080/content.php', function(data) {
$('#loadme').html(data);
});
});
This would fail with the following message if I didn't add the Access-Control-Allow-Origin header:
XMLHttpRequest cannot load http://localhost:8080/newhope/deleteme.php.
Origin chrome-extension://gmelhokjkebpmoejhcelgkfeijabmobf is not allowed by
Access-Control-Allow-Origin.
Once I added the Access-Control-Allow-Origin header on the php response, it worked fine.
Again, setting this to * may be a security risk as any browser page anywhere is allowed to load it inline.

"Error: Calling parameters do not match signature" when using ajax-jquery in iphone web app

In creating a iphone web app, i used magento XML RPCto call magento web services. With the help of jQuery XML RPC i can access the magento web services.My code,
<script src="js/jquery.mobile-1.2.0.min.js"></script>
<script src="js/jquery-1.8.3.min.js"></script>
<link rel="stylesheet" href="css/jquery.mobile-1.2.0.min.css"></link>
<script src="js/jquery.xmlrpc.js"></script>
<script>
$(function(){
$("button").click(function(){
$.xmlrpc({
url:"link of my magento/xmlrpc",
methodName:'login',
params:['user','pass'],
success: function(response, status, jqXHR) {
var res=response;
alert(res); // getting alert as session id as login response
$.xmlrpc({
url:"link of my magento/xmlrpc",
methodName:'call',
//passing session id from the previous response
params:{sessionId:res,methodName:'customer.info',customerId:'3'},
success: function(response1, status1, jqXHR1) {alert("success:"+response1);},
error: function(jqXHR1, status1, error1) {alert(error1); }
});
},
error: function(jqXHR, status, error) {alert(error); }
});
});
});
</script>
Here my problem is, when i run the app i get the session id and pass the id to next method "call" with the parameters.This code while executing gives me an error stating "Error: Calling parameters do not match signature"
I changed the way of passing parameters too but no hope. Can anyone suggest me how to solve this problem.
I believe this is a bug.
Take a look at this thread: Calling parameters do not match signature

NS_Binding_Aborted error for ajax function

I have a link on click of which a request should go to web server and on successful execution a redirection should happen. I have used ajax for this but I am getting NS_Binding_Aborted error in HTTpFox.
The code:
<a id="lnkredirect" href="javascript:void(0);" onclick="myfunction();">Some text</a>
The ajax code:
function myfunction(){
$.ajax({
url: Web server Url,
type: 'POST',
datatype: 'JSON',
timeout: 20000,
data: null,
success: function{ $("#lnkredirect").attr('href','redirection link...');},
error : function{ $("#lnkredirect").attr('href','redirection link...');}
)};
return true;
}
The redirection is happening but I am getting NS_Binding_Aborted error in Firefox. In both success and error scenario, the redirection should happen but why NS_Binding_Aborted is coming, I am not sure of this. NS_Binding_Aborted error should come only if one event is cancelling some prior running event but I have already suppressed href of the link and redirecting it once the ajax request is executed, so there should be only one server call and NS_Binding_Aborted should not come. Please let me know where am I going wrong?
I got a similar trouble, also while using both a href and a XmlHttpRequest inside a onclick. My XMLHttpRequest was aborted (ns_binding_aborted) and thus never reached status 200. I also could see that my XHR was "blocked by devtools" in Firefox console.
This was because the page was reloaded (by the href) before it could finish its job (what was in the onclick).
I had something like this:
<script type="text/javascript">
function incrementNumberOfDownloads() {
var xhttp = new XMLHttpRequest();
xhttp.onreadystatechange = function() {
if (this.readyState == 4 && this.status == 200) { // 4 = request ended, 200 = success
//update displayed number of downloads
document.getElementById("numberOfDownloads").innerHTML = this.responseText;
}
};
xhttp.open("GET", "incrementNumberOfDownloads.php", true);
xhttp.send();
return true;
}
</script>
<p id="numberOfDownloads">42</p>
Download my file !
I fixed the problem by adding a target="_blank" to my download link, so that the page is no more reloaded when clicking, enabling the XMLHttpRequest to finish with success.
This is caused by another request that abort your request. Generally when your goal is reload data o all page just end request and don'ts is synchronized request, a little novell error.
In this case the "return " sentence is the problem, the return sentence must be in success seccion.
My issue fixed, when I've changed calling native js form submit event to jQuery submit event.
// this code
form[0].dispatchEvent(new Event("submit"));
// changed to
form.submit();

Resources