I'm currently running a script to pull data from Google Analytics with googleapiclient Python package (that is based on httplib2 client object)
--> My script works perfectly without any proxy.
But I have to put it behind my corporate proxy, so I need to adapt my httplib2.Http() object to embed proxy information.
Following httplib2 doc 1 I tried:
pi = httplib2.proxy_info_from_url('http://user:pwd#someproxy:80')
httplib2.Http(proxy_info=pi).request("http://www.google.com")
But it did not work.
I always get a Time out error, with or without the proxy info (so proxy_info in parameter is not taken into account)
I also downloaded socks in PySocks package (v1.5.6) and tried to "wrapmodule" httplib2 as described in here:
https://github.com/jcgregorio/httplib2/issues/205
socks.setdefaultproxy(socks.PROXY_TYPE_HTTP, "proxyna", port=80, username='p.tisserand', password='Telematics12')
socks.wrapmodule(httplib2)
h = httplib2.Http()
h.request("http://google.com")
But I get an IndexError: (tuple index out of range)
In the meantime,
When I use the requests package, this simple code works perfectly:
os.environ["HTTP_PROXY"] = "http://user:pwd#someproxy:80"
req = requests.get("http://www.google.com")
The problem is that need to fit with googleapiclient requirements and provide a htpplib2.Http() client object.
rather than using Python2, I think you'd better try using httplib2shim
You can have a look at this tutorial on my blog :
https://dinatam.com/fr/python-3-google-api-proxy/
In simple words, just replace this kind of code :
from httplib2 import Http
http_auth = credentials.authorize(Http())
by this one :
import httplib2shim
http_auth = credentials.authorize(httplib2shim.Http())
I decided to recode my web app in Python 2, still using the httplib2 package.
Proxy info are now taken into account. It now works.
Related
I am using a WebAssembly based software that uses multi-threading that requires SharedArrayBuffer. It runs fine both in Chromium local/deployed, and Firefox 89 deployed, but since the best performance is under Firefox, I want to test and tune it on my machine, so I run python -m SimpleHTTPServer. In this situation, when I open 127.0.0.1:8000 or 0.0.0.0:8000 in Firefox, SharedArrayBuffer is undefined. Perhaps this is a security setting, but when using localhost, I'm really not interested in Firefox's interpretation of the situation -- this should just run. How can I make it work? Do I need a different web server, different settings?
As you guessed correctly, it has to do with security restrictions. There have been changes in regards to the use of SharedArrayBuffer that have already been implemented in Firefox 79 and will land in Chrome shortly as well (starting with Chrome 92). (Time of writing this: July 13, 2021.)
The main purpose is to restrict the use of SharedArrayBuffers in postMessage. Any such attempt will throw an error unless certain restrictive COOP/COEP headers are set to prevent cross-origin attacks:
Cross-Origin-Opener-Policy: same-origin
Cross-Origin-Embedder-Policy: require-corp
Unfortunately, without these headers, there is also no global SharedArrayBuffer constructor. Apparently, this restriction may be lifted in the future. The objects themselves still work though (only passing them through postMessage would throw), but you need a different way to instantiate them. You can use WebAssembly.Memory instead:
const memory = new WebAssembly.Memory({ initial: 10, maximum: 100, shared: true })
// memory.buffer is instanceof SharedMemoryBuffer
You could now go one step further and recover the constructor from that. Therefore, with the following code as "shim", your existing code should work as long as it doesn't try to pass the buffer through postMessage:
if (typeof SharedArrayBuffer === 'undefined') {
const dummyMemory = new WebAssembly.Memory({ initial: 0, maximum: 0, shared: true })
globalThis.SharedArrayBuffer = dummyMemory.buffer.constructor
}
// Now, `new SharedArrayBuffer(1024)` works again
Further reading:
Article about this change on MDN
Google blog post about the upcoming change
WebAssembly.Memory documentation
As #CherryDT pointed out in the comment, the problem is missing headers for the local server. Searching the net, there is a blog that walks through the process of developing WebAssembly in Firefox with a python web server. Instead of python -m SimpleHTTPServer, one has to add a file ./wasm-server.py with this contents (for Python 2):
# Python 2
import SimpleHTTPServer
import SocketServer
class WasmHandler(SimpleHTTPServer.SimpleHTTPRequestHandler):
def end_headers(self):
self.send_header("Cross-Origin-Opener-Policy", "same-origin")
self.send_header("Cross-Origin-Embedder-Policy", "require-corp")
SimpleHTTPServer.SimpleHTTPRequestHandler.end_headers(self)
# Python 3.7.5 adds in the WebAssembly Media Type. Version 2.x doesn't
# have this so add it in.
WasmHandler.extensions_map['.wasm'] = 'application/wasm'
if __name__ == '__main__':
PORT = 8080
httpd = SocketServer.TCPServer(("", PORT), WasmHandler)
print("Listening on port {}. Press Ctrl+C to stop.".format(PORT))
httpd.serve_forever()
then it is possible to test the application at 127.0.0.1:8080
I have been following this link to understand how to use HttpClient to call a Web API Method.
https://www.tutorialsteacher.com/webapi/consuming-web-api-in-dotnet-using-httpclient
The code of interest in the article is below with ‘client’ being the HttpClient object:
client.BaseAddress = new Uri("http://localhost:60464/api/"); //HTTP GET
var responseTask = client.GetAsync("student");
responseTask.Wait();
var result = responseTask.Result;
Error results as follows:
System.AggregateException: 'One or more errors occurred. (Failed to connect to localhost/127.0.0.1:443)'
Please understand my background in networking, IIS and the like is very limited.Most of my time is spent in code and SQL Sprocs. This is a personal project so I have to get this setup myself
If I replace localhost with my machines IP I get the following error:
One or more errors occurred. (java.security.cert.CertPathValidatorException: Trust anchor for certification path not found.)'
So two questions:
How do I install this needed certificate or settings or otherwise (again no idea about this configuration network stuff) but I do have IIS up and running with the Web API hosted and working
If using ‘localhost’ is not supposed to work, what might be the reason this article is using it?
This is only for a personal development machine, yes at some point I am going to want it to work in the real world but for now I just need to get some ‘hello world’ stuff going before the end of times.
I'm testing out deploying my own parse server following the steps in the Parse Server Guide. I've got the server up and running and have been able to create and fetch objects via curl. I built a simple iOS app using the Parse SDK (1.14.2). I've initialized the SDK with the app id and server url as described in the Parse Server Guide. When I try to make requests, I get back unauthorized from the server. Digging further, I noticed that the SDK is not sending the application id header to the server. I modified the SDK to send the application id header and everything works. Am I missing a configuration step somewhere?
This is because you are not passing the ClientKey. In swift 3 you would pass it like this in the didFinishLaunchingWithOptions.
// Init Parse
let configuration = ParseClientConfiguration {
$0.applicationId = PARSE_APP_KEY
$0.clientKey = PARSE_CLIENT_KEY
$0.server = PARSE_SERVER_URL
$0.isLocalDatastoreEnabled = true
}
Parse.initialize(with: configuration)
If you are falling when trying to test CloudCode, then its because your parse-server is not passing the Javascript key. So just make sure you initialize the server to do so if this issue is related to Parse.Cloud function.
Netflix recently updated their API methods for obtaining the full Netflix catalog. I'm curious if anyone has had any success accessing these new xml documents and downloading them via API v1.5 (9/2012). Previously, you could download the entire Netflix catalog via one API call (which I had working perfectly). Now, there are supposedly two calls to make: one for dvd's and one for streaming movies.
I cannot make these calls return anything except for an empty array. Please don't offer an answer unless you have personally downloaded the entire catalog via these new API's.
Bonus points if you can tell me how to do it in Ruby.
http://developer.netflix.com/blog/read/Update_Changes_for_the_Public_API
This did it for me (download the netflix instant cat)...it's in php but can prob be easily rewritten in ruby..this is using JR Collings OAuthsimple
args = Array(
max_results=> 20,
start_index=>0
);
//args don't matter, netflix doesn't listen here
// this is the URL path (note the lack of arguments.)
$rpath = "http://api-public.netflix.com/catalog/titles/streaming";
// Create the Signature object.
$roauth = new OAuthSimple();
$rsigned = $roauth->sign(Array(path=>$rpath,
parameters=>$args,
signatures=> Array('consumer_key'=>YOURKEY,
'shared_secret'=>YOURSECRET,
)));
$getxml = file_get_contents($rsigned['signed_url']);
file_put_contents("streaming.xml", $getxml);
Is it possible to programmatically change settings on a Netgear wireless router using C#? I have settings that I change often and I would like to create my own interface for making those changes. Currently I navigate to the admin web page (10.0.0.1) and it prompts me for a username and password. After I authenticate I can use the web interface to change the router's configuration.
If this isn't possible with Netgear, do any outher wireless routers have an API for developers?
There aren't any APIs out there to do this, but you can write something to make HTTP requests to the router to simulate the webUI being used.
I'm guessing most consumer routers are probably pretty simple to talk to. Authentication is probably nothing more than basic realm.
Selenium offers a firefox plugin that lets you record manual interactions with your browser. And then you can export the steps to python, ruby, java or c#. It worked for me to programmatically adjust my router settings to turn off wifi. Clicking on the elements while recording identifies everything you need.
This code works on an Actiontec MI424WR (FIOS)
Edit the code to add your username, password, and router address.
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import Select
from selenium.common.exceptions import NoSuchElementException
from selenium.common.exceptions import NoAlertPresentException
import unittest, time, re
class Routr(unittest.TestCase):
def setUp(self):
self.driver = webdriver.Firefox()
self.driver.implicitly_wait(30)
self.base_url = "http://routerip_or_address"
self.verificationErrors = []
self.accept_next_alert = True
def test_routr(self):
driver = self.driver
driver.get(self.base_url + "/")
driver.find_element_by_name("user_name").clear()
driver.find_element_by_name("user_name").send_keys("your_username")
driver.find_element_by_id("pass2").clear()
driver.find_element_by_id("pass2").send_keys("enter_your_password_here")
driver.find_element_by_link_text("OK").click()
driver.find_element_by_link_text("Change Wireless Settings").click()
driver.find_element_by_id("ws_off").click()
driver.find_element_by_link_text("Apply").click()
def is_element_present(self, how, what):
try: self.driver.find_element(by=how, value=what)
except NoSuchElementException, e: return False
return True
def is_alert_present(self):
try: self.driver.switch_to_alert()
except NoAlertPresentException, e: return False
return True
def close_alert_and_get_its_text(self):
try:
alert = self.driver.switch_to_alert()
alert_text = alert.text
if self.accept_next_alert:
alert.accept()
else:
alert.dismiss()
return alert_text
finally: self.accept_next_alert = True
def tearDown(self):
self.driver.quit()
self.assertEqual([], self.verificationErrors)
if __name__ == "__main__":
unittest.main()
If this is just a few things you want to change programmatically, simulating HTTP requests should be simple enough. Another option would be to install DD-WRT in your router, basically transforming it into a small Linux installation that allows full programmatic configuration through SSH using standard Linux commands.
I'm unaware of any consumer-grade routers that have an API like that, but you could always build something that (ab)uses the Web interface to do what you want, using something like selenium-rc or watir
MiktoTik sells customer grade routers that allow ssh configuration (mind that they use ssh, but not bash inside ssh). You can even roll your own PHP REST API for router (not that I like PHP, but people are doing it).
I'm not familiar with this router, but I have done similar stuff programmatically via a telnet connection the router with Python.
There's a cood telnet lib for C#:
http://www.codeproject.com/KB/IP/MinimalisticTelnet.aspx
There is a python based Github repo here that describes a SOAP based API. I've used it to program a device schedule for my kids devices. Not willing to pay Disney for Circle. Works great. There's also a js version here.