I'm trying to build a video-chat with a WebRTC. The problem which I have, is that the video-offer is going in both sides like:
PC1 -> send video-offer
PC2 <- handle video-offer message
PC2 -> send video-answer
PC1 <- handle video-answer (when there is debugger breakpoint, the connection is working)
PC2 -> send video-offer
...
I'm sending video-offer message only in negotiationEventHandler:
function handleNegotiationNeededEvent() {
logMessage('HandleNegotiationNeededEvent fired!');
myPeerConnection.createOffer() //tworzymy SDP offer dla drugiego uzytkownika
.then(function (offer) {
return myPeerConnection.setLocalDescription(offer);
})
.then(function () {
sendToWebSocket(msgTypeVideoOffer, senderId, receiverId, JSON.stringify(myPeerConnection.localDescription));
})
.catch(reportError);
}
and handling video-offer / video-answer in two functions
function handleVideoOfferMsg(msg){
logMessage("handle video-offer message call");
var localStream = null;
createPeerConnection();
var desc = new RTCSessionDescription(JSON.parse(msg.messageContent));
debugger;
myPeerConnection.setRemoteDescription(desc)
.then(function () {
return navigator.mediaDevices.getUserMedia(mediaConstraints);
})
.then(function (stream) {
localStream = stream;
document.getElementById("local_video").srcObject = localStream;
myPeerConnection.addStream(localStream);
})
.then(function () { //Utworz odpowiedz
return myPeerConnection.createAnswer();
})
.then(function (answer) { //Ustaw ja sobie jako lokalna
return myPeerConnection.setLocalDescription(answer);
})
.then(function () { //I wyslij SDP do peera
sendToWebSocket(msgTypeVideoAnswer, senderId, receiverId, JSON.stringify(myPeerConnection.localDescription));
})
.catch(handleGetUserMediaError);
}
function handleVideoAnswerMessage(message) {
logMessage("handle video answer message" + message);
var desc = message.messageContent;
myPeerConnection.setRemoteDescription(JSON.parse(desc));
}
The problem is, that handling video-offer is setting a new objects, what probably is a reason of losing a connection.
This is a bug in the Chrome browser. MDN is right, and this works fine in Firefox.
For a basic sendrecv audio+video offer, the answerer should be allowed to add one video and one audio stream and call setLocalDescription without this firing a negotiationneeded event.
Chrome fails to clear negotiationneeded on the answerer side after the expected audio and video have been added, and in fact will keep firing negotiationneeded back and forth forever.
The answer is, that negotiationneeded event is fired every time, when RTCPeerConnection.addStream() function is called with success. So my solution is to check if only inviter has such event handler set:
if(isInviter){
myPeerConnection.onnegotiationneeded = handleNegotiationNeededEvent;
}
My question code is based on mozilla WebRTC Signalling sample, which is just wrong implemented, so be careful using it: https://developer.mozilla.org/en-US/docs/Web/API/WebRTC_API/Signaling_and_video_calling
Related
I have a client which takes audio from mic then sends it to a remote server via websocket stream.
Server side I get the audio stream from websocket
const WebSocket = require('websocket-stream');
const wss = WebSocket.createServer({host: '192.168.254.161', port: 8090},handle);
function handle(stream) {
stream.pipe(recognizeStream);
stream.on('close', function (){
console.log("stream closed")
}).on('error', function() {
console.log("stream error")
})
}
wss.on('connection', function connection(ws) {
ws.on('message', function incoming(message) {
console.log('%s bytes received', message.length);
ws.send("some mesage")
}).on('close', function () {
ws.send("End of audio data")
}).on('error',function (err) {
console.log("error:",err)
});
});
and send it to Google Speech API using SpeechClient with streamingRecognize
const recognizeStream = client
.streamingRecognize(request)
.on('error', console.error)
.on('data', (data) => {
process.stdout.write(
data.results[0] && data.results[0].alternatives[0]
? `Transcription: ${data.results[0].alternatives[0].transcript}\n`
: `\n\nReached transcription time limit, press Ctrl+C\n`
);
});
All works and I get the transcript written to che server console..
Now I need to send the transcript back via websocket, but I cannot find how to do it. The only place where I can send something back is in the ws event handler, but here I do not have access to the transcript data
You create the stream when connection is available and use the connection variable to send back the data:
wss.on('connection', function connection(ws) {
ws.on('message', function incoming(message) {
console.log('%s bytes received', message.length);
ws.send("some mesage")
}).on('close', function () {
ws.send("End of audio data")
}).on('error',function (err) {
console.log("error:",err)
});
const recognizeStream = client
.streamingRecognize(request)
.on('error', console.error)
.on('data', (data) => {
// We send it back here, ws is still accessible
ws.send(
data.results[0] && data.results[0].alternatives[0]
? `Transcription: ${data.results[0].alternatives[0].transcript}\n`
: `\n\nReached transcription time limit, press Ctrl+C\n`
);
});
});
There are different methods too - you can use global variable to pass a connection or wrap it within some prototype.
I'm trying to set up a server that can dynamically create many rooms for many namespaces. I'm currently just trying to broadcast to sockets of a room, when a new socket has joined that room.
So far I have been able to broadcast to a specific namespace and my event listeners on the client receives the message. However when I try to broadcast to a room, of a specific namespace, my event listener doesn't receive that message.
I've turned on the Debugger mode and see the socket.io-client:socket emitting the event with the right payload and event type. So I am not sure what I am missing since the documentation also seems fairly straightforward. Any help would be much appreciated. Below is my code.
Server
const colorNs = io.of('/color');
colorNs.on('connection', (socket) => {
const { id } = socket.handshake.query;
const { id:connId } = socket.conn;
if(id) {
socket.join(id);
socket.broadcast.to(id).emit('user:connect', { id: connId });
}
socket.on('disconnect', () => {
const { id } = socket.handshake.query;
const { id:connId } = socket.conn;
socket.broadcast.to(id).emit('user:disconnect', { id: connId });
});
});
Client
const socket = io('/color?id="123"');
socket.on('user:connect', () => console.log('data', data));
Client - Debug Trace
socket.io-parser decoded 2/color,["user:connect",{"id":"IZTTPidF121JCzf9AAAO"}] as {"type":2,"nsp":"/color","data":["user:connect",{"id":"IZTTPidF121JCzf9AAAO"}]} +1ms
browser.js:133
socket.io-client:socket emitting event ["user:connect",{"id":"IZTTPidF121JCzf9AAAO"}] +3ms
I have a websocket stream being listened:
widget.channel.stream.listen((data) {
print("!!!!new msg: $data");
var dataJson = json.decode(data);
print(dataJson["content"]);
// do my job
setState(() {
_allAnimateMessages.insert(0, newMsg);
});
newMsg.animationController.forward();
});
But, when enter that page again, there was an error says:
Bad state: Stream has already been listened to.
How to make it as broadcast and other pages can receive that broadcast?
Solution for package web_socket_channel:
final channel = IOWebSocketChannel.connect(socketUrl);
final streamController = StreamController.broadcast();
streamController.addStream(channel.stream);
After that simply use streamController.stream to listen web socket events.
You can use broadcasts.
//Here is the solution
StreamController<String> streamController = new StreamController.broadcast(); //Add .broadcast here
//Now you can listen from various places
#override
void initState() {
super.initState();
print("Creating a StreamController...");
//First subscription
streamController.stream.listen((data) {
print("DataReceived1: " + data);
}, onDone: () {
print("Task Done1");
}, onError: (error) {
print("Some Error1");
});
//Second subscription
streamController.stream.listen((data) {
print("DataReceived2: " + data);
}, onDone: () {
print("Task Done2");
}, onError: (error) {
print("Some Error2");
});
streamController.add("This a test data");
print("code controller is here");
}
Font: https://medium.com/#ayushpguptaapg/using-streams-in-flutter-62fed41662e4
When using broadcasts you can have multiple listeners in the same stream.
If you simply use a stream without ".broadcast ()" you can only have one listener
As there are no useful answer, I update my answer here for other reference.
duplicate subscribe stream is a desired behaviour in flutter.
if you are just using a StreamBuilder, the stream can be listen only once. think about it, if your stream can be listen to many other pages or widgets, then data would be repeated.
But if you want using one single stream, and update all widgets
this is really can be occured when develop a complicated app, for example, you are building a chat app, new message comes, you should update many pages UI (your dialog chat ui, your session list ui....), then you should subscribe this streams in many pages, I still not found a proper way to do this, except make this stream to be broadcast, and do your work.
How can you detect that you received a message on a socket.io connection that you do not have a handler for?
example:
// client
socket.emit('test', 'message');
// server
io.on('connection', function (socket) {
console.log('connection received...');
// logs all messages
socket.conn.on('message', function(data) {
console.log('this gets every message.');
console.log('how do I get just the ones without explicit handlers?');
});
socket.on('other' function(data) {
console.log('expected message');
});
}
By accessing the internals of the socket object you can determine what events it is currently listening for. You can use this server-side code to see if the current message is being handled.
io.on('connection', (socket) => {
console.log('A user connected.');
socket.on('disconnect', () => {
console.log('A user disconnected.');
});
socket.on('chat', (msg) => {
console.log('message: ' + msg);
io.emit('chat', msg);
});
socket.conn.on('message', (msg) => {
if(!Object.keys(socket._events).includes(msg.split('"')[1])) {
console.log(`WARNING: Unhandled Event: ${msg}`);
}
});
}
Once connected I am handling two events, 'disconnect' and 'chat'. After that I define a handler that catches all messages with socket.conn.on(...).
The message it receives is going to be a string that looks something like this: '2["myEventName","param1","param2"]'. By splitting it along the double quotes we can get the event name.
We then peek into the internals of socket to find all the keys of socket._events, which happen to be the event name strings. If this collection of strings includes our event name, then another handler will take care of it, and we don't have to.
You can test it from the console in the browser. Run socket.emit('some other event') there and you should see your warning come up in the server console.
IMPORTANT NOTE: Normally you should not attempt to externally modify any object member starting with an underscore. Also, expect that any data in it is unstable. The underscore indicates it is for internal use in that object, class or function. Though this object is not stable, it should be up to date enough for us to use it, and we aren't modifying it directly.
Tested with SocketIO version 2.2.0 on Chrome.
I didn't find a way to do it like socket.io, but using a simple js function to transform message into json it's doing the same job. Here you can try this:
function formatMessage(packetType, data) {
var message = {'packetType': packetType, 'data': data}
return JSON.stringify(message)
}
With:
socket.on('message', function(packet){
packet = JSON.parse(packet)
switch (packet.packetType) {
case 'init':
...
and
socket.send(formatMessage('init', {message}));
I would do so, of course it is the abstract code ... you would have to implement all the listeners and the logic to get the ids of the users to work
Client
var currentUser = {
id: ? // The id of current user
};
var socketMessage = {
idFrom: currentUser.id,
idTo: ?, // Some user id value
message: 'Hello there'
};
socket.emit('message', socketMessage);
socket.on('emitedMessage' + currentUser.id, function(message) {
// TODO: handle message
});
Server
io.on('connection', function (socket) {
// Handle emit messages
socket.on('message', function(socketMessage) {
// With this line you send the message to a specific user
socket.emit('emitedMessage-' + socketMessage.idTo, {
from: socketMessage.idFrom,
message: socketMessage.message
});
});
});
More info: http://socket.io/docs/
So i've built a simple websocket client implementation using Haxe NME (HTML5 target ofc).
It connects to
ws://echo.websocket.org (sorry no link, SO sees this as an invalid domain)
which works perfectly!
(i'm using xirsys_stdjs haxelib to use the HTML5 websocket stuff.)
I want to have a local (on my own machine) running websocket server.
I'm using Socket.io at the moment, because i cannot find an easier / simpler solution to go with.
I'm currently trying to use socket.io as socket server, but a 'standard' javascript socket implementation as client (Haxe HTML5), without using the socket.io library clientside.
Does anyone know if this should be possible? because i cannot get it working.
Here's my socket.io code:
var app = require('http').createServer(handler)
, io = require('socket.io').listen(app)
, fs = require('fs')
app.listen(1337);
function handler (req, res) {
fs.readFile(__dirname + '/client.html',
function (err, data) {
if (err) {
res.writeHead(500);
return res.end('Error loading index.html');
}
res.writeHead(200);
res.end(data);
});
}
// WEBSOCKET IMPLEMENTATION
io.sockets.on('connection', function (socket) {
console.log("webSocket connected...");
socket.on('message', function () {
console.log("server recieved something");
// TODO: find out how to access data recieved.
// probably 'msg' parameter, omitted in example?
});
socket.on('disconnect', function () {
console.log("webSocket disconnected.");
});
});
And here's my Haxe (client) code:
static var webSocketEndPoint:String = "ws://echo.websocket.org";
//static var webSocketEndPoint:String = "ws://localhost:1337";
...
private function initializeWebSocket ():Void {
if (untyped __js__('"MozWebSocket" in window') ) {
websocket = new MozWebSocket(webSocketEndPoint);
trace("websocket endpoint: " + webSocketEndPoint);
} else {
websocket = new WebSocket(webSocketEndPoint);
}
// add websocket JS events
websocket.onopen = function (event:Dynamic):Void {
jeash.Lib.trace("websocket opened...");
websocket.send("hello HaXe WebSocket!");
}
websocket.onerror = function (event:Dynamic):Void {
jeash.Lib.trace("websocket erred... " + event.data);
}
websocket.onmessage = function (event:Dynamic):Void {
jeash.Lib.trace("recieved message: " + event.data);
switchDataRecieved(event.data);
}
websocket.onclose = function (event:Dynamic):Void {
jeash.Lib.trace("websocket closed.");
}
}
In case the Haxe code is unclear: it's using 2 extern classes for the webSocket implementation: MozWebSocket and WebSocket. These are just typed 'interfaces' for the corresponding JavaScript classes.
websocket.io! from the same guys. sample shows exact same thing that you are asking about... and something that I spent past 20 hours searching for (and finally found!)
https://github.com/LearnBoost/websocket.io
Update: Jan 2014
The websocket.io repository has not seen any activity for about 2 years. It could be because it is stable, or it could be because it is abandoned.
The same people have another repository called engine.io. In the readme they say that this is isomorphic with websocket.io... It seems that engine.io is where all the action is these days.
https://github.com/LearnBoost/engine.io
While searching for the same thing I just found https://github.com/einaros/ws/ and its server example worked for me with my pre-existing plain javascript client.
http://socket.io/#how-to-use
At the mentioned link, down towards the bottom of the page,
the socket.io documentation demonstrates as it's last
example, how to use their module as a plain
old xbrowser webSocket server.
SERVER
var io = require('socket.io').listen(80);
io.sockets.on('connection', function (socket)
{
socket.on('message', function () { });
socket.on('disconnect', function () { });
});
BROWSER
<script>
var socket= io.connect('http://localhost/');
socket.on('connect', function ()
{
socket.send('hi');
socket.on('message', function (msg)
{ // my msg
});
});
</script>
Hope that's what your looking for
--Doc