Where are my links? - casperjs

I am trying to run a basic casperjs script that logs into a website then shows me the links. However my output is not returning anything other then 'Done'
Here is my code
var casper = require('casper').create();
casper.start('http://xxxxxx/Login.aspx', function(){
//Login
this.fill('form#form1', {
'username': 'xxxxx',
'password': 'xxxxx'
}, true);
});
casper.then(function(){
var links = document.getElementsByTagName('a');
for(var i = 0; i < links.length; ++i) {
//These should show something
this.echo(links[i].innerText;
this.echo(this.getHTML());
}
});
casper.run(function(){
this.echo('done').exit();
});
Like I said, the only thing I get back is "done".

You use getElementsByTagName in the CasperJS context, you can't do that here, you have to pass in the page context by using the evaluate function (see evaluate, thenEvaluate).
If you just want to print the text of a link, use that in your casper.then :
this.echo(this.fetchText('a'));
And you forgot a bracket too here : this.echo(links[i].innerText;
When you iterate in casperjs, you should use each (IIFE) : http://docs.casperjs.org/en/latest/modules/casper.html#each

Related

Load raw link contents into terminal

I'm using the raw property to get formatted data from urls into the terminal, like this
$(function() {
var save_state = [];
var terminal = $('#term').terminal(function(command, term) {
term.pause();
url = ...;
$.get(url, function(result) {
term.echo(result, {raw:true}).resume();
});
}, { prompt: '>>', name: 'test', outputLimit: 1000 });
});
I'm wondering, how do I get it so when links in result are clicked, they load their data into the terminal the same way command data is loaded, rather than opening a new browser tab?
Thanks!
If you're using command that include URL or URI (for instance get foo.html or get https://example.com) you can use this:
terminal.on('click', '.terminal-output > div:not(.exception) a', function() {
// if you don't use `true` it will show the command like if you type it
// instead of `get` you can use any command you have that will
// fetch the url and display it on the terminal
terminal.exec('get ' + $(this).attr('href'), true);
return false; // prevent following the link
});
if you have different logic for displaying the urls you may need to dipicate the code from interpreter inside click event handler.
terminal.on('click', '.terminal-output > div:not(.exception) a', function() {
// duplicated code from your interpreter
term.pause();
var url = $(this).attr('href');
$.get(url, function(result) {
term.echo(result, {raw:true}).resume();
});
return false;
});

How to post information in CasperJS

The following is the html I wanna tackle with.
I wanna post a query and see the weather reports for that city. I tried my codes:
var casper = require('casper').create();
var utils = require('utils');
casper.start('http://www.weather.com.cn/weather1d/101210101.shtml');
casper.waitForSelector('input', function()
{
this.fillXPath('div.search clearfix',{'//input[#id="txtZip"]':'shijiazhuang'},true);
});
casper.then(function()
{
utils.dump(this.getTitle());
});
casper.run();
It did not print the web paeg title on the console. I also tried this:
casper.waitForSelector('input', function()
{
this.fillSelectors('div.search clearfix',{'input[id="txtZip"]':'shijiazhuang'},true);
});
}
);
I did not get any web page title also. I got quite confused and did not what I had done wrong. Besides, it seems that this.fill method only tackles with name attributes to post information on CasperJS's official website. I need your help.
CasperJS script:
var casper = require('casper').create(), utils = require('utils');
casper
.start('http://www.weather.com.cn/weather1d/101210101.shtml',function(){
this
.wait(3000,function(){
this.capture('search.png');
utils.dump(this.getTitle());
})
.evaluate(function(){
document.getElementById('txtZip').value='shijiazhuang';
document.querySelector('input[type="button"]').click();
})
})
.run();
Result:
"【石家庄天气】石家庄今天天气预报,今天,今天天气,7天,15天天气预报,天气预报一周,天气预报15天查询"
search.png
You are doing it correct, you only have to modify your script a little bit.
Just modify the selector of the input field the actual don't seems to work, then you should get the correct result.
"this.fillXPath" only fills up the form and don't post it.
...
// fill the form
casper.waitForSelector('input', function() {
this.fillXPath('#txtZip', {
'//input[#id="txtZip"]': 'shijiazhuang'
}, true);
});
// trigger - post the form
casper.then(function() {
this.click('#btnZip');
});
// i can't speak chinese - wait for some specific change on this page
casper.wait(5000);
// take a screenshot of it
casper.then(function() {
this.capture("shijiazhuang_weather.png");
});
...

getjson does not work on production server but works on development machine

I have following javascript that is using a selection changed to fill in a select list.
$(function () {
$("#bedrijvenauto").each(function () {
var target = $(this);
var dest = target.attr("data-autocomplete-destination");
target.autocomplete({
source: target.attr("data-autocomplete-source"),
select: function (event, ui) {
alert('selected bedrijf');
event.preventDefault();
target.val(ui.item.label);
$(dest).val(ui.item.value);
$("#projectenauto").val("");
alert('selected bedrijf');
alert($('#BEDRIJF_ID').val());
$.getJSON("/Project/GetListViaJson", { bedrijf: $('#BEDRIJF_ID').val() }, function (data) {
alert('selected bedrijf');
alert(data);
$("#PROJECT_ID").empty();
$("#PROJECT_ID").append(new Option("Maak een selectie", 0));
for (var i = 0; i < data.length; ++i) {
alert(data[i].value + ' ' + data[i].label);
$("#PROJECT_ID").append(new Option(data[i].label, data[i].value));
}
});
},
focus: function (event, ui) {
event.preventDefault();
target.val(ui.item.label);
}
});
target.val($("#BEDRIJF_NAAM").val());
});
It works like a charm on my development pc. The alert are all coming out even the data is returning results. That is the difference with the development pc that does not give any results after the call to getJSON
I have the feeling I am missing a detail here.
I am not used to debugging on a webserver because I usually create GUI applications in WPF, and this is a student's work for his vacation and I now got to get it working without him being around anymore. Vacation is done :-(
But not for me.
The 404 error indicated in your comments means the url your creating is incorrect. Always make use of the #Url.Action() method to ensure they are correctly generated. In your script
var url = '#Url.Action("GetListViaJson", "Project")';
$.getJSON(url, { bedrijf: $('#BEDRIJF_ID').val() }, function (data) {
....
}
or if this is an external script, then add the var url = '#Url.Action(...)'; in the main view (razor code is not evaluated in external script files), or add it as a data- attribute to the element your handling
data-url = "#Url.Action(...)"
and get it again using var url = $(someElement).data('url');

CasperJS manual example bug?

This example in the CasperJS manual shows how to scrape urls from Google. It shows that the urls will come out nice and clean. However, when I run this example, my output looks like this:
20 links found:
- /url?q=http://casperjs.org/&sa=U&ei=_-TuU-KBC83-yQSu5YKQAg&ved=0CBQQFjAA&usg=AFQjCNH321k0JXrSx5WZp-fH6JwxX-O75Q
- /url?q=http://code4fun.fr/tutoriel-casperjs/&sa=U&ei=_-TuU-KBC83-yQSu5YKQAg&ved=0CBoQFjAB&usg=AFQjCNHreU-9mg7OZxK3TOl94HDPOnA_aQ
- /url?q=http://casperjs.readthedocs.org/&sa=U&ei=_-TuU-KBC83-yQSu5YKQAg&ved=0CCEQFjAC&usg=AFQjCNGzX6V5ZQtmCwHwZerHR3ftK3pHOw
- /url?q=https://github.com/n1k0/casperjs&sa=U&ei=_-TuU-KBC83-yQSu5YKQAg&ved=0CCcQFjAD&usg=AFQjCNEiGMDpYiPm1qXK7ZxDCwWwKjAStg
- /url?q=http://www.technologies-ebusiness.com/enjeux-et-tendances/casperjs-pour-des-tests-d-integration&sa=U&ei=_-TuU-KBC83-yQSu5YKQAg&ved=0CC4QFjAE&usg=AFQjCNFOGl1p6ApqP8TmAxhtQp33DHpbcQ
- /url?q=https://www.lullabot.com/blog/article/testing-front-end-casperjs&sa=U&ei=_-TuU-KBC83-yQSu5YKQAg&ved=0CDQQFjAF&usg=AFQjCNG53ZxHl8yZ0JGdzNbwKuZmPOLqCg
- /url?q=http://blog.newrelic.com/2013/06/04/simpler-ui-testing-with-casperjs-2/&sa=U&ei=_-TuU-KBC83-yQSu5YKQAg&ved=0CDoQFjAG&usg=AFQjCNFzlDb7R4Uv-jj_3S5IbJUpKF-7fA
- /url?q=https://www.npmjs.org/package/grunt-casperjs&sa=U&ei=_-TuU-KBC83-yQSu5YKQAg&ved=0CEEQFjAH&usg=AFQjCNGn-dwJpkX_XTQv8YnFZTClcLosJA
- /url?q=http://www.phase2technology.com/blog/behavorial-test-for-custom-entity-using-casperjs/&sa=U&ei=_-TuU-KBC83-yQSu5YKQAg&ved=0CEcQFjAI&usg=AFQjCNFG0KDAADmocesrDoqQTHW6PPO8KQ
- /url?q=http://blog.codeship.io/2013/03/07/smoke-testing-with-casperjs.html&sa=U&ei=_-TuU-KBC83-yQSu5YKQAg&ved=0CE4QFjAJ&usg=AFQjCNG5AsT2iKCnN-utrCGsthCZCpYKaQ
- /url?q=http://phantomjs.org/&sa=U&ei=_-TuU9yhG4iZyASb8oK4Dw&ved=0CBQQFjAA&usg=AFQjCNGXz7tw-UkfDOpqvYV89KlcJPGfHQ
- /url?q=http://phantomjs.org/download.html&sa=U&ei=_-TuU9yhG4iZyASb8oK4Dw&ved=0CB8QFjAB&usg=AFQjCNG_czKcYiFKskAvoRl1CceXuTJecA
- /url?q=http://www.mathieurobin.com/2013/04/phantomjs-chargez-et-jouez-avec-vos-sites-en-js-sans-quitter-la-console/&sa=U&ei=_-TuU9yhG4iZyASb8oK4Dw&ved=0CCUQFjAC&usg=AFQjCNEAtYz0zcsRVYy-37U9sJL7e9EqYQ
- /url?q=http://svay.com/blog/paris-js-10-introduction-a-phantomjs-un-navigateur-webkit-headless/&sa=U&ei=_-TuU9yhG4iZyASb8oK4Dw&ved=0CCsQFjAD&usg=AFQjCNE9dUuVQmNpK064a9GPJyOIetUWAA
- /url?q=https://github.com/ariya/phantomjs&sa=U&ei=_-TuU9yhG4iZyASb8oK4Dw&ved=0CDIQFjAE&usg=AFQjCNErqnWYxIVwBwXeUjaSd4SFicQqpw
- /url?q=https://github.com/gruntjs/grunt-lib-phantomjs&sa=U&ei=_-TuU9yhG4iZyASb8oK4Dw&ved=0CDkQFjAF&usg=AFQjCNHkRVx926JJkKhdoKxKsKVcQc-QTg
- /url?q=http://blog.octo.com/seo-spa-angular/&sa=U&ei=_-TuU9yhG4iZyASb8oK4Dw&ved=0CD8QFjAG&usg=AFQjCNFcj-ykUo-rSQKlcEZIy1qjSlW-oQ
- /url?q=https://www.npmjs.org/package/phantomjs&sa=U&ei=_-TuU9yhG4iZyASb8oK4Dw&ved=0CEYQFjAH&usg=AFQjCNGweWRdm8qjqxOOybFgtz5B8CnMDQ
- /url?q=http://code.google.com/p/phantomjs/&sa=U&ei=_-TuU9yhG4iZyASb8oK4Dw&ved=0CEwQFjAI&usg=AFQjCNEwvNx7NNMDAaiqHZ_y-3Bbf62W_w
- /url?q=http://casperjs.org/&sa=U&ei=_-TuU9yhG4iZyASb8oK4Dw&ved=0CE4QFjAJ&usg=AFQjCNGNEKkl1eWaFx9Sz6R7ZFVN9r1Bhw
Here's their code I ran:
var links = [];
var casper = require('casper').create();
function getLinks() {
var links = document.querySelectorAll('h3.r a');
return Array.prototype.map.call(links, function(e) {
return e.getAttribute('href');
});
}
casper.start('http://google.fr/', function() {
// search for 'casperjs' from google form
this.fill('form[action="/search"]', { q: 'casperjs' }, true);
});
casper.then(function() {
// aggregate results for the 'casperjs' search
links = this.evaluate(getLinks);
// now search for 'phantomjs' by filling the form again
this.fill('form[action="/search"]', { q: 'phantomjs' }, true);
});
casper.then(function() {
// aggregate results for the 'phantomjs' search
links = links.concat(this.evaluate(getLinks));
});
casper.run(function() {
// echo results in some pretty fashion
this.echo(links.length + ' links found:');
this.echo(' - ' + links.join('\n - ')).exit();
});
Can someone explain what is going on and why my urls are not clean like theirs?
The example is fine since you are getting the urls, but with a little bit of noise. It looks like google changed the hrefs in the mean time. So you can just add
links = links.map(function(link){
return link.substring(0, link.indexOf("&sa=U&ei=")).replace("/url?q=", "");
});
before joining the links in the last step.
I actually found a work around using a function called getLinks(). This works great and should be a bit more versatile for what I need. Using a combination of split() and pop() allows you to get what you want.
Code:
function getLinks() {
var links = document.querySelectorAll('h3.r a');
return Array.prototype.map.call(links, function(e) {
return e.getAttribute('href').split('/').pop();
});
}
casper.start(googleSearch, function() {
links = this.evaluate(getLinks);
});
casper.run(function() {
// echo results in some pretty fashion
this.echo(links.length + ' links found:');
this.echo(links.join('\n')).exit();
});

jQuery wait for an AJAX request to be completed before refreshing the page

I've got the following jQuery:
$("#delete_products").click(function() {
$(":checkbox:checked").each(function() {
var pid = $(this).val();
$.get('delete_product.php',{pid: pid});
});
location.reload();
});
There is a problem with this since the page doesn't wait for the AJAX request to be completed (MULTIPLE AJAX REQUESTS), and refreshes the page immediately and makes the AJAX request to not run and fail.
How can I do that the page will only refresh when it done loading?
I've been given this code:
$("#delete_products").click(function () {
var promises = [];
$(":checkbox:checked").each(function () {
var pid = $(this).val();
promises.push($.get('delete_product.php', {
pid: pid
}));
});
$.when.apply($, promises).done(function () {
location.reload();
});
return false;
});
But this solution just doesn't work.
any suggestions?
your code seems like it should work, but i would recommend to delete all products with one call by passing array of ids.
less work for the browser, less work for the server, faster results.
UPDATED ANSWER
$("#delete_products").click(function () {
var ids = [];
$(":checkbox:checked").each(function () {
ids.push($(this).val());
});
$.post('delete.php', { 'ids': ids } }).done(function() {
alert('hells yeah!');
});
return false;
});
and as for the server side:
$commaSeperatedIds = explode(',', $_POST['ids']);
mysql_query('DELETE FROM products WHERE id IN('.mysql_real_escape_string($commaSeperatedIds).')');
Use "success" parameter of "get"
EDIT: add counter of requests.
total_requests = $(":checkbox:checked").length;
total_success = 0;
...
$.get('delete_product.php',{pid: pid}, function (data, status, xx) {
...
total_success++;
if (total_success >= total_requests) {
location.reload();
}
});
...
Possible Solution #1:
The async parameter could help. Set it to false.
Since you`re using the $.get() function, this is done with:
$.ajaxSetup({
async: false
});
 
With the $.ajax() function, you'd simply set it like:
$.ajax({
url: ...,
async: false,
success: function(data) {}
});
More info on this can be found here.
 
Possible Solution #2:
Use the .success() callback hook OR .complete() if you want to refresh the page no mather if the request failed or not.
$.get('delete_product.php',{pid: pid}).success(function(response)
{
location.reload();
});
 
Happy coding!
 
Edit:
The questioner seems to prever sUP's answer. I'd like to provide an example of how to achieve the desired functionality with jQuery:
$("#delete_products").click(function()
{
var products = [];
$(":checkbox:checked").each(function()
{
var pid = $(this).val();
products.push(pid);
});
$.post('delete_products.php', {'products': products}).done(function()
{
location.reload();
});
});
 
If you prefer to use JSON for the post data, try:
$('#delete_products').click(function()
{
var products = [];
$(':checkbox:checked').each(function()
{
products.push($(this).val());
});
// Convert the products array into JSON
products = JSON.stringify(products);
$.post('delete_products.php', {'products': products}).done(function()
{
location.reload();
});
});
In PHP you need to parse the json string as follows:
<?php
// This creates an associative array from the JSON string
$delete_products = json_decode($_POST['products'], true);
// Use explode to make a comma separated string from the array
// for use in a SQL SELECT query:
$delete_products = explode(',', $delete_products);
Info about json_decode can be found in the PHP Manual.
JSON.stringify is not supported in older browsers. Include JSON-js if you need cross browser support.
I too believe that the desired result can probably be achieved best by collecting the product IDs and then sending a single Ajax call to take care of them all.
But since the OP put the interesting question forward of how to handle multiple Ajax requests and wait for them all to be finished I have looked at the when() method again and it seems to me that the original syntax is still faulty.
According to the jQuery manual when() is a method and therefore requires to be called with one or more argument(s) in parentheses. I have not worked with promises yet and I have not tested anything but I assume that something like the following might bring at least a different result:
$("#delete_products").click(function () {
var promises = [];
$(":checkbox:checked").each(function () {
var pid = $(this).val();
promises.push($.get('delete_product.php', {
pid: pid
}));
});
$.when(promises).done(function () {
location.reload();
});
return false;
});
As I said before, I still have not quite grasped the promises mechanisms/syntax yet ...
In the original version $.when does not have any meaningful context to work on, The apply() method does provide context but only after when has done its (unseccessful) work already.

Resources