I am currently trying to port a chrome extension to firefox (addon-sdk). However I came up with a few problems porting the pac script functionality.
When setting a proxy through chromes extension API, you can set a PAC script as string inside the pac script object (ref: http://developer.chrome.com/extensions/proxy.html#type-PacScript)
Looking in Firefox, there is nothing like that. The only option I see is to pull the script from a url (http://kb.mozillazine.org/Firefox_:FAQs:_About:config_Entries). My problem with this is that the pac script have to change and react when the user adjust addon settings.
Is there a (hacky) way to accomplish something like that in Firefox?
The only solution I came up with is encoding the users options and post them to the pac script server. Server parses them and creates a script matching the needs. I want to avoid using servers at any costs as this results in another dependency!
You can use a data: URI for your PAC file. Generating it dynamically is easy:
var pacScript = "function FindProxyForURL(url, host){return 'DIRECT';}";
var uri = "data:text/javascript," + encodeURIComponent(pacScript);
alert(uri);
Related
I know that there are many threads talking about this but i've tried many of the solutions suggested but nothing seems to work. Im gonna be very specific so you guys could please help me!
Im trying to do web scraping to a website using Selenium in Python 3 on Windows 10. This website blocks me after a certain number of requests so what I've red is that if I use Tor as the Selenium web driver I can just ask Tor for a new identity (which means a different IP) every specific number of requests.
The following code lets me do the scraping I want using Tor firefox profile in the Tor Browser folder. The only thing missing with these code is that I've havent been able to request a new identity (new IP).
profiler = webdriver.FirefoxProfile(r"C:\Users\Samir\Desktop\Tor
Browser\Browser\TorBrowser\Data\Browser\profile.default")
profiler.set_preference("network.proxy.type", 1)
profiler.set_preference("network.proxy.socks",'127.0.0.1')
profiler.set_preference("network.proxy.socks_port",9050)
driver = webdriver.Firefox(firefox_profile=profiler)
driver.implicitly_wait(15)
driver.get("The URL I want to scrape")
#Extract whatever information i want from the URL
I tried to use the Stem library to get new identity but this does not seem to work with the Tor firefox profile of the previous code.However this works fine if I just open the browser double clicking on the Tor Browser shortcut icon that is created when I install Tor.
#This is the code in stem that gets new identity using Stem. As I said,
#this does not work with the selenium firefox profile for Tor.
from stem import Signal
from stem.control import Controller
with Controller.from_port(port = 9051) as controller:
controller.authenticate()
controller.signal(Signal.NEWNYM)
Okay so to wrap up, is there a way to get a new IP with the previous code I showed? Or what can I do to achieve what I want using python 3, selenium and tor on windows 10 plus anyother library or whatever thats necessary.
If you have questions or need more information to help me just let me know.
Thaks a lot!!
Is there a possibility to set Website URL in dynamic way and open it with batch script? (by using e.g. start chrome "-URL here-")
Example.
I have a link with an excel file, which I want to download:
www.example.com/example_15/example_123.xlsx
and each day specific part of URL is changing, e.g. tomorrow it could be:
www.example.com/example_12/example_abc1.xlsx
Is there any way how I can replace changing parts of the URL so it would always download an excel file ?
I am working on device which has embeded OS with limited features and space. I am can't install any scripting language like php curl. I would like to know is there any way to call https url with some header values (e.g. content type) using shell script and once get response that can be displayed on web page or can write into file.
Regds
curl and wget, as already suggested, are usually available in shells. If those are not available you'll have to give more details about your platform to figure out what could be available.
You can probably try wget like this:
wget --http-user=user --http-password=password http://domain.com/dir/file
My Firefox extension needs to replace/delete key3.db in the Firefox profile, is there a way to do that? I tried to nsIFile.copyTo() but the file is not overwritten, nsIFile.remove() but it returns NS_ERROR_FILE_IS_LOCKED.
No, replacing a file while it's being used isn't a good idea. Instead you should be using the XPCOM functionality meant to manipulate this file (meaning the master password). Something like this should work:
var pk11db = Components.classes["#mozilla.org/security/pk11tokendb;1"]
.getService(Components.interfaces.nsIPK11TokenDB);
var token = pk11db.getInternalKeyToken();
token.changePassword("", "foobar");
Using "" instead of "foobar" should remove the master password. However, I'm not entirely sure that changing the master password will work without querying the current password. Firefox Mobile can be used as a relatively simple code example.
The problem: toggle javascript support without restarting firefox (nor resorting to different driver) during cucumber test run.
If Firefox's prefutils were exposed to javascript in a web page, that would make it possible. But it is not the case.
So, is there a plugin that does it? Or is there another way to solve the problem? Or is there a good tutorial (that highlights the exposing bit) on how to make such a plugin?
Edit
On a second thought, how would javascript be of any help once it is disabled? Probably the whole idea is a bit screwed.
I assume that your tests run with normal web content privileges. In that case, they aren't going to be able to affect browser settings such as whether JavaScript is enabled (I assume that's what you mean by "toggle JavaScript support").
I'd implement a simple XPCOM component with a method to turn JS support on and off (by setting the appropriate pref). You can expose it as a JavaScript global property so that your tests can access it. See Expose an XPCOM component to javascript in a web page for more details. Package your component in an extension and make sure it is installed in the Firefox instance where your tests are running.
If you want to access the preferences API directly from your content script, you can add the following prefs to Firefox, either in about:config or by adding the following lines to prefs.js in your profile directory:
user_pref("capability.principal.codebase.p1.granted", "UniversalXPConnect UniversalBrowserRead UniversalBrowserWrite UniversalPreferencesRead UniversalPreferencesWrite UniversalFileRead");
user_pref("capability.principal.codebase.p1.id", "http://www.example.com");
user_pref("capability.principal.codebase.p1.subjectName", "");`
user_pref("signed.applets.codebase_principal_support", true);
Replace www.example.com with the domain that you want to grant the privileges to. Also add this line to your JS code before you call the preferences API:
netscape.security.PrivilegeManager.enablePrivilege('UniversalXPConnect');
A local file (something loaded from file:///) is allowed to request additional privileges. Normally you would get a prompt asking whether you want to allow access - you can "auto-accept" the prompt by adding the following lines to prefs.js in the Firefox profile:
user_pref("capability.principal.codebase.p0.granted", "UniversalXPConnect");
user_pref("capability.principal.codebase.p0.id", "file://");
user_pref("capability.principal.codebase.p0.subjectName", "");
You page can then do:
netscape.security.PrivilegeManager.enablePrivilege("UniversalXPConnect");
var branch = Components.classes["#mozilla.org/preferences-service;1"]
.getService(Components.interfaces.nsIPrefBranch);
branch.setBoolPref("javascript.enabled", false);
This will definitely work if your page is a local file. Judging by the error message however, you are currently running code from about:blank. It might be that changing capability.principal.codebase.p0.id into about:blank or into moz-safe-about:blank will allow that page to get extended privileges as well but I am not sure.
However, none of this will really help if JavaScript is already disabled and you need to enable it. This can only be solved by writing an extension and adding it to the test profile. JavaScript in Firefox extensions works regardless of this setting.
That means you need Javascript to toggle enabling or disabling Javascript.
function setJavascriptPref(bool) {
prefs = Components.classes["#mozilla.org/preferences-service;1"]
.getService(Components.interfaces.nsIPrefBranch);
prefs.setBoolPref("javascript.enabled", bool);
}