Websocket running in asyncio loop inside a thread disconnects immediately - websocket

I'm developing an application that uses multiple threads for various communication channels. One of them is websocket for which I'm using asyncio loop to run:
class Rotator():
def __init__(self):
# some irrelevant stuff here...
self._readThread = threading.Thread(target=self._ReadThread, daemon=True)
self._usbThread = threading.Thread(target=self._UsbThread, daemon=True)
self._webThread = threading.Thread(target=self._WebThread, daemon=True)
self._socketThread = threading.Thread(target=self._SocketThread, daemon=True)
self.InitializeRotator()
def InitializeRotator(self):
# some more irrelevant stuff here
self._readThread.start()
self._usbThread.start()
self._webThread.start()
self._socketThread.start()
# not including _ReadThread() and _UsbThread() as they are irrelevant in this context
def _WebThread(self): # including _WebThread() as illustration
handler_object = server.WebHttpRequestHandler
_httpServer = socketserver.TCPServer(("",self.webPort), handler_object)
_httpServer.serve_forever()
async def ws_handler(websocket, path):
data = await websocket.recv()
reply = f"Data recieved as: {data}!"
await websocket.send(reply)
def _SocketThread(self):
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
start_server = websockets.serve(self.ws_handler, str(self.DHCPIP), 8000)
loop.run_until_complete(start_server)
loop.run_forever()
The frontend JS code that communicates with this is:
var ws;
function startWebsocket() {
ws = new WebSocket('ws://192.168.200.76:8000')
ws.onopen = function(e){
console.log('Connection established')
ws.send('Connection established')
}
ws.onmessage = function(e){
console.log('websocket message event:', e)
}
ws.onclose = function(){
// connection closed, discard old websocket and create a new one in 5s
console.log('Connection lost, will reconnect in 5s...')
ws = null
setTimeout(startWebsocket, 5000)
}
}
startWebsocket();
The web page that contains the frontend is served by the server run by _WebThread(). The issue is that the connection is being closed immediately after being established. I'm assuming the issue is caused by some conflict in the asyncio/thread combination, because I tested the server side code on its own and it worked like a charm. What am I doing wrong?

I recreated your example and managed to fix it using two changes:
ws_handler is missing the self parameter. Since it's an instance method, the instance itself will be passed as the first parameter. The actual websocket object will be the second one.
in ws_handler I added a while True loop. I followed the example in the docs here https://websockets.readthedocs.io/en/stable/intro/tutorial1.html#bootstrap-the-server
final result:
async def ws_handler(self, websocket, path):
while True:
data = await websocket.recv()
reply = f"Data recieved as: {data}!"
await websocket.send(reply)
Other changed that I made were to use 8000 as a port for the http server, 8001 for the websocket server and 'localhost' instead of 192.168.200.76. But I'm pretty sure that's not relevant

Related

How to test LiteCable broadcasting from irb console?

Actually my question is not about console only. I haven't found any libraries for testing LiteCable, and I try to implement some. The problem I faced now, is when I call in console something like (exactly same as official example):
LiteCable.broadcast "chat_#{chat_id}", user: user, message: data["message"], sid: sid
It returns nil. I've tried to put byebug in the code, and when I call this method with absolutely same parameters there, I got pretty big sheet with some unreadable parameters. And the main problem is I don't see messages I try to send from console.
Can someone explain why is that and how can I implement sending requests from console? The point is I working on api-only application, and try to cover this functionality with proper tests. Thank you very much in advance!
P.S: the project is on pure ruby, without rails
Since you don't have the entire code, I am unable to get a clear sense of your setup. The default broadcast_adapter for LiteCable is memory, so if you were to broadcast from your console, processes outside your console will not receive such streams. If you use the anycable adapter which I believe queues broadcasts in redis, then you should be able to broadcast from your console provided you have the correct connection.
Here is a simple implementation of a WS client and a server using LiteCable without Rails
# ~/app/server.rb
# A simple WS server that evaluates math expression literally using ruby eval
# Run this server with ruby server.rb
require "rack"
require "rack/handler/puma"
require "lite_cable"
require "lite_cable/server"
class CalculatorConnection < LiteCable::Connection::Base
identified_by :token
def connect
self.token = request.params["token"]
$stdout.puts "Session #{token} is connected to the server"
end
def disconnect
$stdout.puts "Session #{token} disconnected from the server"
end
end
class CalculatorChannel < LiteCable::Channel::Base
identifier :calculator
def subscribed
stream_from "calculator"
end
def evaluate(payload)
$stdout.puts "Data received from the client"
LiteCable.broadcast "calculator", result: eval(payload['expression'])
end
end
app = Rack::Builder.new
app.map "/cable" do
use LiteCable::Server::Middleware, connection_class: CalculatorConnection
run(proc { |_| [200, {"Content-Type" => "text/plain"}, ["OK"]] })
end
Rack::Handler::Puma.run app
// ~/app/client.js
// A client that talks to the WS server.
// To run, Open chrome console and paste this code
let socket = null;
(function init() {
socket = new WebSocket("ws://0.0.0.0:9292/cable?token=abc123");
socket.onopen = function (e) {
console.info("[connected] : Connection established to the server");
};
socket.onmessage = function (e) {
console.info("[message] : Message from the server", e.data);
};
socket.onerror = function (e) {
console.error("[error] : Error establishing connection", e.message);
};
})();
// subscribe to the calculator channel
socket.send(
JSON.stringify({
command: "subscribe",
identifier: JSON.stringify({
channel: "calculator",
}),
})
);
// call the evaluate method on the Calculator channel
socket.send(
JSON.stringify({
command: "message",
identifier: JSON.stringify({
channel: "calculator",
}),
data: JSON.stringify({
expression: "150 * 4 - 3 / (2 + 2)",
action: "evaluate",
}),
})
);
There's nothing special going on here just a simple rack app that uses Puma which adds support for rack socket hijacking. This should guide you in the right direction. Make sure you keep both your server and chrome console open to see the output. Try sending additional message to the server to see the output.
If you are looking at implementing a WebSocket client in ruby instead of js using LiteCable, check the client socket class in the LiteCable repo

aiohttp websocket and redis pub/sub

i created a simple websocket server using aiohttp . my server reads message from redis pub/sub and sends it to client .
this is my websocket code:
import aiohttp
from aiohttp import web
import aioredis
router = web.RouteTableDef()
#router.get("/ws")
async def websocket_handler(request):
ws = web.WebSocketResponse()
await ws.prepare(request)
sub = request.config_dict["REDIS"]
ch, *_ = await sub.subscribe('hi')
async for msg in ch.iter(encoding='utf-8'):
await ws.send_str('{}: {}'.format(ch.name, msg))
async def init_redis(app):
redis_pool = await aioredis.create_redis_pool('redis://localhost')
app["REDIS"] = redis_pool
yield
redis_pool.close()
await redis_pool.wait_closed()
async def init_app():
app = web.Application()
app.add_routes(router)
app.cleanup_ctx.append(init_redis)
return app
web.run_app(init_app())
my first client can connect to server and receive messages but when i create another client to connect to this endpoint it receive no messages !
what is the problem ? how can i fix this problem?
You need to call create_redis for each client and publish the message to the channel. Otherwise, only the first client will receive the subscribed message.
So, you can edit your code as follows.
import aiohttp
from aiohttp import web
import aioredis
import asyncio
router = web.RouteTableDef()
async def reader(ws, ch):
while (await ch.wait_message()):
await ws.send_str('{}: {}'.format(ch.name, msg))
#router.get("/ws")
async def websocket_handler(request):
ws = web.WebSocketResponse()
await ws.prepare(request)
sub = await aioredis.create_redis_pool('redis://localhost')
pub = await aioredis.create_redis_pool('redis://localhost')
ch, *_ = await sub.subscribe('hi')
asyncio.ensure_future(reader(ws, ch))
async for msg in ws:
await pub.publish('hi', msg)
sub.close()
pub.close()
async def init_app():
app = web.Application()
app.add_routes(router)
return app
web.run_app(init_app())
Please note there may be minor syntax error (e.g. format of the message), but this is the structure that you should follow as it worked for me. I got it to work with my application.

websocket handshake fails with django channels

EDIT: correction after #Ken4scholars comment below
I have the following consumer which fails right after connecting
consumers.py
from channels.generic.websocket import AsyncJsonWebsocketConsumer
#...
class ListGeneratedTokensByFileConsumer(AsyncJsonWebsocketConsumer):
stop = False
async def websocket_connect(self,event):
await self.accept()
self.stop = False
async def websocket_receive(self,event):
await self.send_json({"text":"received","accept": True})
await self.send_tokens_list()
async def websocket_disconnect(self,event):
self.stop = True
async def send_tokens_list(self):
some_path = "..."
while self.stop == False:
await asyncio.sleep(2)
the_message = {}
if os.path.isfile("some_file.json")):
with open(os.path.join(some_path ,"some_file.json"),'r') as new_tok:
the_message = json.load(new_tok)
if not the_message:
print("waiting...")
else:
await self.send_json(the_message)
await self.close()
It always throws the error: ERR_CONNECTION:RESEST and the websocket disconnects with code 1006. This might seem familiar to recent changes in django-channels but since I am sending a text once the websocket opens and send a message back from the consumer it should do the trick. Or is there something wrong?
routing.py
url(r'^myapp/sub_path/(?P<pk>\d+)/sub_sub_path/',ListGeneratedTokensByFileConsumer)
and the websocket endpoint in js is:
.js
var loc = window.location;
var wsStart = "ws://";
if (loc.protocol == "https:") {
wsStart = "wss://";
}
var endpoint = wsStart + loc.host + loc.pathname + "sub_sub_path" + "/";
for info, with channels-redis==2.3.2, channels==2.3.0, asgiref==3.2.2, daphne==2.3.0, django==2.0.8
If you see something like in django logs:
WebSocket HANDSHAKING /ws/your_route
WebSocket REJECT /ws/your_route
WebSocket DISCONNECT /ws/your_route
And you wrapped websocket router with AllowedHostsOriginValidator in the asgi.py like:
application = ProtocolTypeRouter({
"http": django_asgi_app,
"websocket": AllowedHostsOriginValidator(
URLRouter(
chat_websocket_urlpatterns
))
})
Then you definetely should check the ALLOWED_HOSTS variable in the settings.py, maybe you forgot to put something in there. In my case it was just because I didn't specify my IP address, only localhost.
This is most likely not that the asker looked for, but I thought it may come handy to someone else, as there is not much info about the django channels in this site.

MassTransit MessageRequestClient timeout issue

We have two services which exchange messages via MassTransit on top of RabbitMQ.
The goal is to send a message in a request/response way. Here's the code of the service which listens for a message, let's call it Service1:
Bus.Factory.CreateUsingRabbitMq(
sbc =>
{
var host = sbc.Host(new Uri($"rabbitmq://{RabbitMqHost}"), h =>
{
h.ConfigureRabbitMq();//Custom extension to specify credentials
});
sbc.ReceiveEndpoint(host, "CUSTOM_QUEUE_NAME", ep =>
{
ep.Exclusive = false;
ep.AutoDelete = true;
ep.Durable = false;
ep.PrefetchCount = 1;
ep.Handler<EngineStartingMessage>(async context =>
{
//SourceAddress and ResponseAddress are auto generated queues
//Message processing is done here
context.Respond(response);
});
});
});
The code of the service which sends a message and process the result, let's call it Service2:
var requestClient =
new MessageRequestClient<EngineStartingMessage, EngineStartingResponse>(
EntityServiceBus,
new Uri("CUSTOM_QUEUE_NAME?durable=false&autodelete=true&exclusive=false"),
new TimeSpan(0, 0, 30));
var engineStartResponse = requestClient.Request(new EngineStartingMessage() { Version = SystemVersion }).Result;
When I run the above code I can see Service1 gets a request and calls context.Respond(response); but on the Service2 side I always get a Timeout exception. Since, a message can make it from Service2 to Service1 I assume there are no network related issues. The timeout is pretty high as well. The message processing on Service1 end takes less than a second. So I think a response message is just not routed properly and I don't understand why. What is suspicious to me is that SourceAddress and ResponseAddress contain auto generated values and not "CUSTOM_QUEUE_NAME?durable=false&autodelete=true&exclusive=false". Any ideas what I'm doing wrong?
You should start Bus, on your service start, like it shown here
await busControl.StartAsync(source.Token);

myWebSocketSubject.multiplex(..).subscribe().unsubscribe() closes connection, event further observers exists

The following code will close the connection, event further observers exists on the myWebSocketSubject:
myWebSocketSubject.Observable.webSocket('ws://mysocket');
myWebSocketSubject.subscribe();
myWebSocketSubject.multiplex(..).subscribe().unsubscribe()
// the connection closed now
My expectation was, that the connection gets closed with the last unsubscribe() call (and not with the first one).
Use Case
If I get it right, with the multiplex(..) operator, on create and complete a message is send to the socket, which e.g. allows to un-/subscribe on server side to specific event.
My preferred Web Socket service could therefore look like as below. There exists only one connection, and this single connection provides several streams. On first subscription to the web socket the connection gets created; and with the last unsubscribe call the connection gets closed. For each data-stream a un-/subscribe message is sent once.
I haven't found a solution to use the WebSocketSubject.multiplex(..) method...
Preferred Example Web Socket Service
export class WebSocketService {
connection: WebSocketSubject<any>;
constructor() {
this.connection = Observable.webSocket<any>(_createConfig())
}
dataStream(eventType: string): Observable<WebSocketMessage> {
return connection.multiplex(
() => new WebSocketMessage("WebSocket.Subscribe." + eventType),
() => new WebSocketMessage("WebSocket.Unsubscribe." + eventType),
message => (message.type == eventType)
)
.retry() // reconnect on error and send subscription messages again
.share(); // send messages on last/fist un-/subscribe on this stream
}
// ...
}
export class WebSocketMessage {
type: string;
data: any;
constructor(command: string, data?:any) {
this.type = command;
this.data = data || undefined;
}
}
I have written the following test case which fails...
it('should able to handle multiple subscriptions', () => {
const subject = Observable.webSocket(<any>{url: 'ws://mysocket'});
const sub1 = subject.subscribe();
const sub2 = subject.subscribe();
const socket = MockWebSocket.lastSocket;
socket.open();
sinon.spy(socket, 'close');
sub1.unsubscribe();
// Fails, because the socket gets closed on first unsubscribe
expect(socket.close).have.not.been.called;
sub2.unsubscribe();
expect(socket.close).have.been.called;
});
If I get it right the share operator would do the trick. But after using the operator, the multiplex method is not available.
Thanks for any feedback, input, ...!

Resources