How to check for a websocket bottleneck in a python project - websocket

Question

I currently have a script that connects to a server, makes a websocket connection and receives high frequency messages.
I am quite sure that the processing on my client end cannot keep up with the messages and thus i am getting behind after small periods of time.
My understanding is the messages are queued in both the servers sending buffer and in my clients receive buffer too, and if i do not process them quick enough evenutally the buffer will fill up and i will lose messages which will cause an out of sequence issue, is my assumption correct?
My question is, what is the best way (tools) to go about tracing possible bottle necks and track down if the issue is the server or the client? I am working with python in Visual Studio and have the single process running for now using PM2.
I am looking for advice on way to trace low level bottlenecks even if it means using tools like wireshark etc.
thanks.

Answer 1

My advice is to use gevent and gevent-websocket so that all the connections are async. Then you can do multiple connections asynchonously.
With GIPC, you could launch an instance per cpu core and load balance between ports.
example:
from gevent import monkey, socket, Timeout, sleep
monkey.patch_all()
import sys
pyver = sys.version_info[0]
if pyver == 3:
import signal
from gevent import signal_handler as sig
else:
from gevent import signal
import bottle
from bottle import route, request, response, abort
import ujson as json
from gevent.pywsgi import WSGIServer
from geventwebsocket.handler import WebSocketHandler
from geventwebsocket import WebSocketError
import traceback
#route('/ws/app')
def handle_websocket():
global ws_users
ws = request.environ.get('wsgi.websocket')
if not ws:
abort(400, 'Expected WebSocket request.')
while 1:
message = None
try:
with Timeout(2, False) as timeout:
message = ws.receive()
if message:
message = json.loads(message)
# process message, report back with ws.send()
except WebSocketError:
break
except Exception as exc:
traceback.print_exc()
sleep(1)
if __name__ == '__main__':
print(socket.gethostname())
print('Started...')
botapp = bottle.app()
server = WSGIServer(("0.0.0.0", int(80)), botapp , handler_class=WebSocketHandler)
def shutdown():
print('Shutting down ...')
server.stop(timeout=60)
exit(signal.SIGTERM)
if pyver == 3:
sig(signal.SIGTERM, shutdown)
sig(signal.SIGINT, shutdown)
else:
signal(signal.SIGTERM, shutdown)
signal(signal.SIGINT, shutdown) #CTRL C
server.serve_forever()