Im trying to make a simple http server in python 3.x running on Windows 7 64-bit. I had this working fine on python 2.7 on my Mac, and made a few changes to upgrade to python 3.2 running on Windows.
When queried, the headers are all returned correctly, but there is no content showing in browser inspector - however telnet reveals that the response is being fully received!
Why does it not show up in the browser? HELP!
Code is:
import sys
import http.server
from http.server import HTTPServer
from http.server import SimpleHTTPRequestHandler
import usb.core
class MyHandler(SimpleHTTPRequestHandler):
def __init__(self,req,client_addr,server):
SimpleHTTPRequestHandler.__init__(self,req,client_addr,server)
def do_GET(self):
r="<h1>Hello World</h1>"
self.send_response(200)
self.send_header("Content-type", "application/json;charset=utf-8")
self.send_header("Content-length", len(r))
self.end_headers()
self.wfile.write(r.encode("utf-8"))
self.wfile.flush()
print(r)
HandlerClass = MyHandler
Protocol = "HTTP/1.1"
port = 80
server_address = ('localhost', port)
HandlerClass.protocol_version = Protocol
try:
httpd = HTTPServer(server_address, MyHandler)
print ("Server Started")
httpd.serve_forever()
except KeyboardInterrupt:
print('Shutting down server')
httpd.socket.close()
I fixed it - it needs "Access-Control-Allow-Origin" header adding:
self.send_header("Access-Control-Allow-Origin","*")
-- did the trick!
Related
I use aiohttp to send a request like so:
async with ClientSession() as session:
res = await session.get("http://0.0.0.0:8000/./")
When I start up a http server using python like so:
python3 -m http.server
I see that the path is normalized, i.e., the server gets the following request:
GET / HTTP/1.1" 200
How do I disable this normalization, to enforce a behavior like in urrlib, for example, where urllib.request.urlopen("http://0.0.0.0:8000/./") results in the following request:
GET /./ HTTP/1.1.
aiohttp uses yarl for URL processing.
session.get('http://example.com') works as well as session.get(yarl.URL('http://example.com'))
You can disable URL encoding for yarl.URL with encoded=True, but you have to take care of URL correctness.
e.g.
import asyncio
import yarl
import aiohttp
async def test():
url = yarl.URL('https://stackoverflow.com/./', encoded=True)
async with aiohttp.ClientSession() as session:
async with session.get(url, allow_redirects=False) as resp:
print(resp.url)
asyncio.run(test())
I am trying to talk to an ElectrumX server using JSON-RPC over TLS, but through Tor (SOCKS proxy on localhost).
When running the following code, drain() in asyncio.streams is calling _drain_helper in SSLProtocol, which I don't know how to implement.
If I just make it a no-op, it seems to not work.
I know that the JSON-RPC server is working because I have tested with
echo -ne '{"id":0,"args":["3.0.2","1.1"],"method":"server.version"}\n' | socat stdio openssl-connect:songbird.bauerj.eu:50002,verify=0
My attempt at using TLS through SOCKS in Python with asyncio:
from asyncio.sslproto import SSLProtocol
import aiosocks
import asyncio
loop = None
class MySSLProtocol(SSLProtocol):
def __init__(otherself):
super().__init__(loop, None, False , None)
# app_proto context waiter
async def l(fut):
try:
socks4addr = aiosocks.Socks4Addr("127.0.0.1", 9050)
transport, protocol = await aiosocks.create_connection(MySSLProtocol, proxy=socks4addr, proxy_auth=None, dst=("songbird.bauerj.eu", 50002))
reader = asyncio.StreamReader()
reader.set_transport(transport)
writer = asyncio.StreamWriter(transport, protocol, reader, loop)
writer.write(b'{"id":0,"method":"server.version","args":["3.0.2", "1.1"]}\n')
await writer.drain()
print(await reader.readuntil(b"\n"))
fut.set_result("finished")
except BaseException as e:
fut.set_exception(e)
def f():
global loop
loop = asyncio.get_event_loop()
fut = asyncio.Future()
asyncio.ensure_future(l(fut))
loop.run_until_complete(fut)
print(fut.result())
loop.close()
f()
I am using aiosocks from master. Commit 932374c
asyncio.sslproto and SSLProtocol are part of asyncio private API.
You should never use the class directly or derive from it.
For working with SSL please pass normal protocol (derived from asyncio.Protocol) and ssl.SSLContext as ssl param into loop.create_connection() / loop.create_server().
I'm using it like so:
class HTTP(asyncio.Protocol):
def __init__(self, config: Config):
self.config = config
def data_received(self, data) -> None:
print(data)
class HTTPS:
def __new__(cls, config: Config):
ssl_context = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
ssl_context.load_cert_chain(config.rsa_cert, config.rsa_key)
return asyncio.sslproto.SSLProtocol(
loop=asyncio.get_running_loop(),
app_protocol=HTTP(config),
sslcontext=ssl_context,
waiter=None,
server_side=True,
)
See mitm for working example. Not the way I would've chosen to use it, but the only way I could figure out. From my knowledge there is no way to upgrade an asyncio.Protocol to use SSL - you must create a new one.
I have some periodic tasks which I execute with Celery (parse pages).
Also I established a websocket with tornado.
I want to pass data from periodic tasks to tornado, then write this data to websocket and use this data on my html page.
How can I do this?
I tried to import module with tornado websocket from my module with celery tasks, but ofcourse, that didn't work.
I know only how to return some data, if I get a message from my client-side. Here is how I cope with it:
import tornado.httpserver
import tornado.websocket
import tornado.ioloop
import tornado.web
import socket
'''
This is a simple Websocket Echo server that uses the Tornado websocket handler.
Please run `pip install tornado` with python of version 2.7.9 or greater to install tornado.
This program will echo back the reverse of whatever it recieves.
Messages are output to the terminal for debuggin purposes.
'''
class handler():
wss = []
class WSHandler(tornado.websocket.WebSocketHandler):
def open(self):
print ('new connection')
if self not in handler.wss:
handler.wss.append(self)
def on_message(self, message):
print ('message received: ' + message)
wssend('Ihaaaa')
def on_close(self):
print ('connection closed')
if self in handler.wss:
handler.wss.remove(self)
def check_origin(self, origin):
return True
def wssend(message):
print(handler.wss)
for ws in handler.wss:
if not ws.ws_connection.stream.socket:
print ("Web socket does not exist anymore!!!")
handler.wss.remove(ws)
else:
print('I am trying!')
ws.write_message(message)
print('tried')
application = tornado.web.Application([
(r'/ws', WSHandler),
])
if __name__ == "__main__":
http_server = tornado.httpserver.HTTPServer(application)
http_server.listen(8888)
myIP = socket.gethostbyname(socket.gethostname())
print ('*** Websocket Server Started at %s***' % myIP)
main_loop = tornado.ioloop.IOLoop.instance()
main_loop.start()
The option is to make a handle in tornado and then post results of celery task to this handle.
After that, there will be an opportunity to pass this data to websocket.
Inspired by ipython-notebook-proxy, and based on ipydra, and extending the latter to support more complex user authentication as well as a proxy, because in my use case, only port 80 can be exposed.
I am using flask-sockets for the gunicorn worker, but I am having troubles to proxy WebSockets. IPython uses three different WebSockets connections, /shell, /stdin, and /iopub, but I am only able to get the 101 Switching Protocols for the first two. And /stdin receives a Connection Close Frame as soon as is created.
This is the excerpt code in question:
# Flask imports...
from werkzeug import LocalProxy
from ws4py.client.geventclient import WebSocketClient
# I use my own LocalProxy because flask-sockets does not support Werkzeug Rules
websocket = LocalProxy(lambda: request.environ.get('wsgi.websocket', None))
websockets = {}
PROXY_DOMAIN = "127.0.0.1:8888" # IPython host and port
methods = ["GET", "POST", "PUT", "DELETE", "HEAD", "OPTIONS", "PATCH",
"CONNECT"]
#app.route('/', defaults={'url': ''}, methods=methods)
#app.route('/<path:url>', methods=methods)
def proxy(url):
with app.test_request_context():
if websocket:
while True:
data = websocket.receive()
websocket_url = 'ws://{}/{}'.format(PROXY_DOMAIN, url)
if websocket_url not in websockets:
client = WebSocketClient(websocket_url,
protocols=['http-only', 'chat'])
websockets[websocket_url] = client
else:
client = websockets[websocket_url]
client.connect()
if data:
client.send(data)
client_data = client.receive()
if client_data:
websocket.send(client_data)
return Response()
I also tried to create my own WebSocket proxy class, but it doesn't work either.
class WebSocketProxy(WebSocketClient):
def __init__(self, to, *args, **kwargs):
self.to = to
print(("Proxy to", self.to))
super(WebSocketProxy, self).__init__(*args, **kwargs)
def opened(self):
m = self.to.receive()
print("<= %d %s" % (len(m), str(m)))
self.send(m)
def closed(self, code, reason):
print(("Closed down", code, reason))
def received_message(self, m):
print("=> %d %s" % (len(m), str(m)))
self.to.send(m)
Regular request-response cycle works like a charm, so I removed that code. If interested, the complete code is hosted in hidra.
I run the server with
$ gunicorn -k flask_sockets.worker hidra:app
Here is my solution(ish). It is crude, but should serve as a starting point for building websocket proxy. The full code is available in unreleased project, pyramid_notebook.
This uses ws4py and uWSGI instead of gunicorn
We use uWSGI's internal mechanism to receive downstream websocket message loop. There is nothing like WSGI for websockets in Python world (yet?), but looks like every web server implements its own mechanism.
A custom ws4py ProxyConnection is created which can combine ws4py event loop with uWSGI event loop
The thing is started and messages start fly around
This uses Pyramid request (based on WebOb), but this really shouldn't matter and code should be fine for any Python WSGI app with little modifications
As you can see, this does not really take advantage of asynchronicity, but just sleep() if there is nothing coming in from the socket
Code goes here:
"""UWSGI websocket proxy."""
from urllib.parse import urlparse, urlunparse
import logging
import time
import uwsgi
from ws4py import WS_VERSION
from ws4py.client import WebSocketBaseClient
#: HTTP headers we need to proxy to upstream websocket server when the Connect: upgrade is performed
CAPTURE_CONNECT_HEADERS = ["sec-websocket-extensions", "sec-websocket-key", "origin"]
logger = logging.getLogger(__name__)
class ProxyClient(WebSocketBaseClient):
"""Proxy between upstream WebSocket server and downstream UWSGI."""
#property
def handshake_headers(self):
"""
List of headers appropriate for the upgrade
handshake.
"""
headers = [
('Host', self.host),
('Connection', 'Upgrade'),
('Upgrade', 'websocket'),
('Sec-WebSocket-Key', self.key.decode('utf-8')),
# Origin is proxyed from the downstream server, don't set it twice
# ('Origin', self.url),
('Sec-WebSocket-Version', str(max(WS_VERSION)))
]
if self.protocols:
headers.append(('Sec-WebSocket-Protocol', ','.join(self.protocols)))
if self.extra_headers:
headers.extend(self.extra_headers)
logger.info("Handshake headers: %s", headers)
return headers
def received_message(self, m):
"""Push upstream messages to downstream."""
# TODO: No support for binary messages
m = str(m)
logger.debug("Incoming upstream WS: %s", m)
uwsgi.websocket_send(m)
logger.debug("Send ok")
def handshake_ok(self):
"""
Called when the upgrade handshake has completed
successfully.
Starts the client's thread.
"""
self.run()
def terminate(self):
raise RuntimeError("NO!")
super(ProxyClient, self).terminate()
def run(self):
"""Combine async uwsgi message loop with ws4py message loop.
TODO: This could do some serious optimizations and behave asynchronously correct instead of just sleep().
"""
self.sock.setblocking(False)
try:
while not self.terminated:
logger.debug("Doing nothing")
time.sleep(0.050)
logger.debug("Asking for downstream msg")
msg = uwsgi.websocket_recv_nb()
if msg:
logger.debug("Incoming downstream WS: %s", msg)
self.send(msg)
s = self.stream
self.opened()
logger.debug("Asking for upstream msg")
try:
bytes = self.sock.recv(self.reading_buffer_size)
if bytes:
self.process(bytes)
except BlockingIOError:
pass
except Exception as e:
logger.exception(e)
finally:
logger.info("Terminating WS proxy loop")
self.terminate()
def serve_websocket(request, port):
"""Start UWSGI websocket loop and proxy."""
env = request.environ
# Send HTTP response 101 Switch Protocol downstream
uwsgi.websocket_handshake(env['HTTP_SEC_WEBSOCKET_KEY'], env.get('HTTP_ORIGIN', ''))
# Map the websocket URL to the upstream localhost:4000x Notebook instance
parts = urlparse(request.url)
parts = parts._replace(scheme="ws", netloc="localhost:{}".format(port))
url = urlunparse(parts)
# Proxy initial connection headers
headers = [(header, value) for header, value in request.headers.items() if header.lower() in CAPTURE_CONNECT_HEADERS]
logger.info("Connecting to upstream websockets: %s, headers: %s", url, headers)
ws = ProxyClient(url, headers=headers)
ws.connect()
# Happens only if exceptions fly around
return ""
How do I use Django with the Tornado web server?
it's very simple ( especially with django 1.4) .
1 - just build your django project( and apps ) and make sure it works fine.
2- create a new python file at the root folder ( same dir where you used django-admin.py startproject)
3- then copy the code below , edit the os.environ['DJANGO_SETTINGS_MODULE'] line, and paste it in that new .py file.
import os
import tornado.httpserver
import tornado.ioloop
import tornado.wsgi
import sys
import django.core.handlers.wsgi
#sys.path.append('/home/lawgon/') # path to your project ( if you have it in another dir).
def main():
os.environ['DJANGO_SETTINGS_MODULE'] = 'myProject.settings' # path to your settings module
application = django.core.handlers.wsgi.WSGIHandler()
container = tornado.wsgi.WSGIContainer(application)
http_server = tornado.httpserver.HTTPServer(container)
http_server.listen(8888)
tornado.ioloop.IOLoop.instance().start()
if __name__ == "__main__":
main()
Django 1.6+ it should be like this:
import os
import tornado.httpserver
import tornado.ioloop
import tornado.wsgi
from django.core.wsgi import get_wsgi_application
def main():
os.environ['DJANGO_SETTINGS_MODULE'] = 'myproject.settings' # path to your settings module
application = get_wsgi_application()
container = tornado.wsgi.WSGIContainer(application)
http_server = tornado.httpserver.HTTPServer(container)
http_server.listen(8888)
tornado.ioloop.IOLoop.instance().start()
if __name__ == "__main__":
main()
UPDATE:
I created a minimal working demo which shows how to use the Tornado web server to run nicely with django:
https://github.com/tamasgal/django-tornado
ORIGINAL POST:
Just a remark: The WSGI application workflow has been changed from 1.6 to 1.7. You have to replace the import
import django.core.handlers.wsgi
with
from django.core.wsgi import get_wsgi_application
and change the application initialisation from
application = django.core.handlers.wsgi.WSGIHandler()
to
application = get_wsgi_application()
This is the modified code from the Moayyad Yaghi's answer:
import os
import tornado.httpserver
import tornado.ioloop
import tornado.wsgi
import sys
import django.core.handlers.wsgi
from django.core.wsgi import get_wsgi_application
#sys.path.append('/home/lawgon/') # path to your project ( if you have it in another dir).
def main():
os.environ['DJANGO_SETTINGS_MODULE'] = 'myProject.settings' # path to your settings module
application = django.core.handlers.wsgi.WSGIHandler()
application = get_wsgi_application()
container = tornado.wsgi.WSGIContainer(application)
http_server = tornado.httpserver.HTTPServer(container)
http_server.listen(8888)
tornado.ioloop.IOLoop.instance().start()
if __name__ == "__main__":
main()
There's a project called tornado-proxy that would help you. But I would like to recommend that you use Nginx. In the Nginx config you could now use proxy_pass to direct your calls like this:
location /comet {
proxy_pass http://localhost:8081;
}
location / {
proxy_pass http://localhost:8080;
}
In real world you would connect Django and some production-ready webserver with WSGI. This demo shows how you can run Tornado (and it's webserver) and Django side by side from one python module serving different URL prefixes: https://github.com/bdarnell/django-tornado-demo.
Tornado would block on serving any request directed to Django, though.