Recently working on crawler project.
I noticed when I run the script under adminstrator (elevated console in Windows), the chromium will hangs with a sad face tab... no errors will report until I Ctrl+C to halt the script.
But with no code changes and I run the same script under normal non elevated console, everything works out just fine...
Was wondering if anyone met this issue before ?
Thank you very much for your time.
const browser = await puppeteer.launch({ headless: isHeadless });
const page = await browser.newPage();
var error = undefined;
await page.goto(landingURL,{ waitLoad: true, waitNetworkIdle: true });
await page.addScriptTag({ url: jquerySource });
Related
Is it possible to get background_page target for Firefox Add-on in puppeteer?
For testing chrome extensions, I can easily access the background_page when extension is loaded by doing the following:
// Bring up chrome browser with extension loaded
const browser = await puppeteer.launch({
headless: false,
defaultViewport: null,
ignoreHTTPSErrors: true,
timeout: 0,
args: [
`--disable-extensions-except=${CRX_PATH}`,
`--load-extension=${CRX_PATH}`,
'--ignore-certificate-errors',
'--no-sandbox',
'--disable-setuid-sandbox',
'--window-size=1920,1080',
],
});
// Fetch browser targets
const targets = await browser.targets();
// Filter out background_page for extension from the targets
const backgroundPageTarget = targets.find((target) => {
return target.type() === 'background_page';
});
I was able to load my firefox add-on by following these guides:
https://github.com/puppeteer/puppeteer/issues/4162
https://github.com/puppeteer/puppeteer/blob/f26bb7f4c44d9b7db5cc73c4af32db6fa5bcd3a2/experimental/puppeteer-firefox/README.md#add-ons
However, I've been unable to access the background_page of the add-on. I need that to intercept and validate requests and responses made by the add-on.
I managed to make it work with Selenium by following these steps:
Open up the same Firefox browser application that you're making Puppeteer connect to.
Load the extension the same way that you make Puppeteer load it (there's different ways, I did it by loading an unsigned version with Firefox dev edition).
Activate the popup window for the extension, then right click and select View page source.
In the URL, you should see this pattern view-source:moz-extension://<extension-id>/<extension-page-name>.html
Using this extension ID, you can make your script go to the background page by having the browser navigate to moz-extension://<extension-id>/<background-page-name>.html
I am using Protractor to run tests on a login-based web application on CI with around 1000 tests running daily. It was all going well until all my tests started to fail. The reason is that the web-based app depends upon login, and due to a minor issue with the app, login failed, followed by that all my tests failed and it took 8hrs to complete! This was a big mess. I started to wonder how can I avoid such failures. I found Protractor's fail-fast mode but it did no good because it would stop execution on the first failure. I want something that stops the execution as soon as login fails but runs all the tests (irrespective of any failure) if login passes. Is there a way to work this out?
Tests are run in docker in headless mode.
These two guys are your friends: await browser.close(); & await process.exit(1);
Never really thoughts about it, but seems like I'll need to implement this too in my login method
async login(username, password, quitOnFailure = true) {
// do what you normally do to login
await sendKeys(this.$usernameInput, username);
await sendKeys(this.$passwordInput, password);
await this.$submitButton.click();
await this.waitForLoad();
// lets says if for testing purposes you need to continue on failure
// and check certain scenarios make this optional
if (quitOnFailure) {
// in my case if I'm not logged in I get a red label displayed
if (await this.$errorLabel.isPresent()) {
await browser.close();
await process.exit(1);
}
// this may be your way for checking if you're not logged in
// try to wait until a welcome message is present, if not, quit
try {
await browser.wait(
ExpectedConditions.presenceOf(homePage.$welcomeMessage),
10000,
`Failed at ${__file} -> ${__function}() -> line ${__line}`
)
} catch (e) {
console.log(e);
await browser.close();
await process.exit(1);
}
}
};
``'
I handle this in my CI environment by adding a BVT step. I use jenkins to do so, I cerated a Pipeline in Which before starting the entire Test suite, I just run BVT Job, in which I just run Login test. If BVT is pass next step is to Run full test suite, but if BVT is failed next step will not even execute.
I am trying to use casperjs on a react app and noticed that screenshots were coming back as blank. So, I tried a different app and got the same result.
So, here is a react app I found - http://jaredly.github.io/github-issues-viewer/#rails/rails/0 and here is the casperjs code
var casper = require('casper').create();
casper.start('http://jaredly.github.io/github-issues-viewer/#rails/rails/0', function() {
this.echo(this.getTitle());
this.capture('im.png')
});
The image shows blank info
How do I resolve this?
I'm creating my first Firefox extension using the Addon SDK, but I can't get messages I pass with console.log() to appear in my debugger.
I created a new profile in Firefox 33 and installed the latest version of Firebug. When I launch my addon using cfx run -p <My_Profile_Directory> I can see both Firebug and my addon, and the addon does what it's supposed to do. However, I don't see anywhere messages I wrote to log using the console.log() command
Here's my main.js so far:
function loginToSite(user, password) {
var Request = require("sdk/request").Request;
var doLogin = Request(
{
url: "https://website.com/login/index.php",
contentType: "application/x-www-form-urlencoded",
content: "username=xxxx&password=xxxx&remember=1",
onComplete: function(response) {
console.log(response.text);
}
}
);
doLogin.post();
}
function checkLoginStatus(tab) {
//TODO Actually check if the tab is logged in, currently assume it's not
loginToSite(0,0);
}
// Listens for tabs and checks each loaded tab if it's the website
tabs.on("ready", function(tab) {
var tabUrl = tab.url.toLowerCase();
if(tabUrl.contains("website.com")) {
console.log("Not connected to website.com, running login procedure");
checkLoginStatus(tab);
}
});
Like I said, I'm actually being logged in automatically, but no log messages appear in either Firebug's or the Firefox Developer Tools' console.
What am I doing wrong here? Where are the log messages?
You have to change the default logging level (error) to something more verbose, like info.
The global level is controlled by the preference extensions.sdk.console.logLevel. But you can adjust the logging level of your extension only, with the preference extensions.myExtensionID.sdk.console.logLevel
When you run cfx without passing a profile directory, it takes care of setting up the temporary profile to be developer friendly.
here is what I have done:
- I downloaded, installed and run the firefox extension SDK
- Created a whole new extension with ctx init
- Opened lib/main.js
- This is what I wrote there:
var Request = require("request").Request;
console.log("Here I am!");
Request({
url: "http://www.google.com",
onComplete: function (response) {
console.log(response.responseText);
}
});
Run the extension with ctx run
It perfectly prints out "Here I am!", then nothing happens. No exception is thrown, no errors, nothing. But the url doesn't seem to get loaded.
What happened? Did I do something wrong?
Thank you very much!
Matteo
Silly me. I forgot to actually submit the request using the .get() function.