I'm crawling about 20 Web sites in the background, when a page loads, within the addon script, with the page-worker. Unfortunately the browser freezes, unpredictable during that time.
I tried to use timers.setTimeout(..., 0-400ms) and also tried the example from the wiki
function executeSoon(aFunc) {
var tm = Cc["#mozilla.org/thread-manager;1"]
.getService(Ci.nsIThreadManager);
tm.mainThread.dispatch({
run: function () {
aFunc();
}
}, Ci.nsIThread.DISPATCH_NORMAL);
}
but this also freezes the UI. Is there any other solution?
The crawling code:
...
timer.setTimeout(function () {
let pageWorker = require("sdk/page-worker").Page({
contentScriptFile: self.data.url("js/extractor.js"),
contentURL: url
});
pageWorker.port.on("loaded", function (content) {
if (typeof callback === 'function') {
callback(content);
}
});
}, 200)
...
The extractor.js, even in simpler cases, where it return body.textContent, is blocking.
The page-worker API just creates an invisible page, it does not do so on a background thread since it has to create a complete window/document environment including layout to allow for full dom/styling functionality and layout always happens on the main thread.
If you want to do calculations in the background you should use the Worker or ChromeWorker APIs, in which you won't have access to DOM and many other APIs.
In the SDK you can use
const { ChromeWorker } = require("chrome");
This is because Firefox uses the main thread for page-workers, in Firefox nightly content pages uses separate processes, which means that page-workers will use a separate processes too, so give Firefox Nightly a try, it should work there, and this will be released in a few months.
Related
I'm porting a Chrome extension to a Firefox extension and due to the nature of the website that it runs on, I need to monitor the pushState.
Chrome Extensions has a handy way to handle this: chrome.webNavigation.onHistoryStateUpdated. The way that I use it in the Chrome extension is as follows:
chrome.webNavigation.onHistoryStateUpdated.addListener(function(details) {
var tabUrl = details.url;
if (isTabUrlValid(tabUrl)) {
$.get(tabUrl, function(data) {
var videoUrl = $(data).find('meta[itemprop=contentURL]').prop('content');
videoUrl = validateUrl(videoUrl);
videoUrl5k = make5kUrl(videoUrl);
});
}
});
I need to do the same thing for the Firefox Extension, but I haven't found any good answers. I've tried doing the answer mentioned here: How to get notified about changes of the history via history.pushState?
(function(history) {
var pushState = history.pushState;
history.pushState = function(state) {
if (typeof history.onpushstate == "function") {
history.onpushstate({state: state});
}
var tabUrl = tabs.activeTab.url;
console.log("UPDATED TAB URL: " + tabUrl);
if (isTabUrlValid(tabUrl)) {
$.get(tabUrl, function(data) {
var videoUrl = $(data).find('meta[itemprop=contentURL]').prop('content');
videoUrl = validateUrl(videoUrl);
videoUrl5k = make5kUrl(videoUrl);
});
}
return pushState.apply(history, arguments);
};
})(window.history);
The problem is that when I do cfx run it complains that history/window is undefined and therefore never gets detected. I think this is due to it being within the SDK, but I don't know of a good workaround.
Any thoughts?
Edit: I looked at #willma's answer below and I don't think that would work for me. The issue is that the URL is updated via pushState and the DOM is not... Is there any good way replicate what I do in the chrome extension?
Edit: Here's the pageMod portion
pageMod.PageMod({
attachTo: 'top', // Don't attach to iFrames --> http://goo.gl/b6b1Iv
include: [URLs],
contentScriptFile: [data.url("jquery-2.1.1.min.js"),
data.url("csScript.js")],
onAttach: function(worker) {
worker.port.on('url', function(url) {
var videoUrl = validateUrl(url);
videoUrl5k = make5kUrl(videoUrl);
console.log("--5K URL--: " + videoUrl5k);
});
}
});
That history code needs to get injected into a tab using a content script. Right now your logic says when the history event occurs, check to see if the tab URL is valid.
In Firefox, the logic will be the other way around: when a tab is opened, check if its URL is valid, and if so, then attach a script to it that will monitor for the history event. To do so you'll need to use a Page Mod.
Edit: All the code
One key concept you're missing is the difference between a content script and a main/library script. The library scripts are stored in lib and have access to all the SDK modules, but don't have access to the DOM, window object… The content scripts are stored in data, are injected into a page using the PageMod or tabs modules, can access the dom and window objects, but have no access to any SDK modules. Content scripts are essentially like the page scripts you'd attach your standard HTML page (with <script></script>) with the caveats that they can't share variables other page scripts but they can communicate with the main scripts.
The only reason I bring this up is because your initial problem was trying to access the window object from a main script and the problem in your fiddle is that you're trying to access the tabs module inside a content script. It's worth reading the topmost link in this answer if this is still confusing.
main.js
const { PageMod } = require('sdk/page-mod');
var sendXHR = function(url) {
// Do something with the new URL
// See Request Module docs (below) for sending XHRs from main script.
}
const pageMod = PageMod({
attachTo: 'top',
include: '*',
onAttach: function(worker) {
worker.port.on('newURL', sendXHR);
}
});
content.js
var sendNewUrlToMain = function() {
self.port.emit('newURL', location.href);
}
var pushState = window.history.pushState;
window.history.pushState = function(state) {
if (typeof history.onpushstate == "function") {
history.onpushstate({state: state});
}
sendNewUrlToMain();
return pushState.apply(history, arguments);
}
window.addEventListener('hashchange', sendNewUrlToMain);
Here are the request module docs, for making XHRs.
NB: if you don't want to use the request module (the only reason being that you already have standard XHR code for your chrome extension and don't want to take the time to learn/rewrite that code), you can send a standard XHR from the content script, but in doing so, you risk allowing the user to close the tab and thus destroy the script before your XHR callbacks are executed.
I have an application that uses a combination of the onhashchange event (for new browsers) and the hashchange plugin by Ben Alman (for old browsers) to track the history while making ajax calls or actions. Works like a charm in all browsers, back and forward buttons let the user navigate the actions that get recorded by changing the hash. So far so good. Now our page will be hosted in an iframe on a clients page in a diff domain(cross domain). Chrome kind of works but if you put to many changes in the history it stops working at some point (we can live with that). IE dosen't work at all. When I navigate our application by clicking on links and updating the hash new history items get created in the parent page but when I hit the back button the hash in the nested page is not updated therefore the hashchange event never fires. Anyone solved this problem before? Many thanks
Initialize the hash change event handling
if ("onhashchange" in window && !($j.browser.msie && $j.browser.version == '7.0')) {
window.onhashchange = function() {
var params = parseHash(location.hash)
if (params.tabId) {
if (getSelectedTabId() == params.tabId) return;
reloadPage(params.tabId);
}
};
}
else {// Plugin for older browsers
$j(window).bind('hashchange', function() {
var params = parseHash(location.hash)
if (params.tabId) {
if (getSelectedTabId() == params.tabId) return;
reloadPage(params.tabId);
}
});
}
I just started using the FireFox Builder to build a simple addon. I realised that I cannot get direct access to the window object.
What I want to do is to get the window object and pollute it with some classes and functions so I can call them from the page itself.
Below is the current code:
// This is an active module of the ritcoder Add-on
require("widget").Widget({
id: "widgetID1",
label: "My Mozilla Widget",
contentURL: "http://www.mozilla.org/favicon.ico",
onClick: function(evt){
var tabs = require("tabs");
var activeTab = tabs.activeTab;
var notifications = require("notifications");
notifications.notify({
title: "Jabberwocky",
text: "'Twas brillig, and the slithy toves",
data: "did gyre and gimble in the wabe",
onClick: function (data) {
console.log(data);
// console.log(this.data) would produce the same result.
}
});
activeTab.window.a=20; //this fails
context.alert('yesx');
}
});
How do I do this? Inject some code into the active page so that it can be called.
regards,
You need to use tab.attach() to run a content script in the tab's context, and then use unsafeWindow to add properties the page's script can see. (You should also read the introduction to Content Scripts.)
The Addon SDK doesn't provide a direct access (without the content script) to the page from the add-on's code because it tries to be forward compatible with the plans to make web pages run in processes separate from the browser's and the add-on's process.
I was playing with jQuery and async calls last night and found an unusual issue. I wanted to run multiple Ajax calls inside a loop. I wrote the below (where rand.php just sleeps for a second and returns a random number). Somewhat surprisingly it executes synchronously and takes 20 seconds or so to finish.
$(document).ready(function () {
$([1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20]).each(function() {
var number = this;
$.get("rand.php", function(data) {
$('#'+number).html(data);
});
});
});
The PHP code is as follows,
<?php
sleep(1);
echo rand();
?>
I was thinking this is clearly wrong as the async calls should be no blocking and return almost in parallel. After much playing around (assuming it was a server issue) I discovered that appending anything to the URL to make it look like it was different worked as expected. That is it returned in 3 seconds or so (6 or so calls at a time).
$(document).ready(function () {
$([1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20]).each(function() {
var number = this;
$.get("rand.php?"+number, function(data) {
$('#'+number).html(data);
});
});
});
I don't suppose a jQuery/Javascript Guru can explain this behavior? Is it some browser limitation? Why is it that only when the URL's are different that it runs as I would expect?
EDIT - Rather then reply, this was using Chrome (whatever the latest is) and Firefox 5/6. I did try it in IE which did cache it, so ignored that and focused on Chrome. Interesting the first one works as expected in IE9 on the first page load, but then just displays cached results when reloaded.
You are in ie aren't you. Bad! Bad IE user! IE caches get requests as if it was any other content. Jquery has a built in function to address this:
$.ajaxSetup({ cache : false });
This will add a nifty spoiler to take care of this. But why add the spoiler in other browsers? So usually I do this:
if(!+"\v1"){
$.ajaxSetup({ cache : false });
}
Which is the tests for IE and set it only in that browser.
I would like to write a firefox extension. This extension is not a generic extension but work specifically for a domain where I need to highlight specific html components.
How should I do that? I just want the js loaded when the user is browsing a specific domain.
My current overaly.js is basically empty (generated by the Extension Wizard):
var myextension = {
onLoad: function() {
// initialization code
this.initialized = true;
this.strings = document.getElementById("myextension-strings");
},
onMenuItemCommand: function(e) {
var promptService = Components.classes["#mozilla.org/embedcomp/prompt-service;1"]
.getService(Components.interfaces.nsIPromptService);
promptService.alert(window, this.strings.getString("helloMessageTitle"),
this.strings.getString("helloMessage"));
},
onToolbarButtonCommand: function(e) {
// just reuse the function above. you can change this, obviously!
myextension.onMenuItemCommand(e);
}
};
window.addEventListener("load", myextension.onLoad, false);
And my ff-overlay.xul is:
myextension.onFirefoxLoad = function(event) {
document.getElementById("contentAreaContextMenu")
.addEventListener("popupshowing", function (e){ myextension.showFirefoxContextMenu(e); }, false);
};
myextension.showFirefoxContextMenu = function(event) {
// show or hide the menuitem based on what the context menu is on
document.getElementById("context-myextension").hidden = gContextMenu.onImage;
};
window.addEventListener("load", myextension.onFirefoxLoad, false);
I was thinking to go neanderthal and do a check inside myextension.onFirefoxLoad to see if the currentpage is the one I want but that requires the user to click the proper item on the context menu.
I'm not totally following what you have because both of those look like JS files, not XUL files. But what you probably want to do is listen for the load event coming from the web pages that are loaded. Then, in your event loader, just look at each page that loads and see whether it's coming from the specific domain you want.
A great (though not always quite as easy as it sounds) way to find out how to do something in a Firefox addon is to find another addon that does something similar. DOM Inspector and Inspect Context are your friends! The first such addon that comes to mind in this case is WikiTrust so you could try looking at that one to see if it gives you any inspiration.