I try to create a Screen Sharing application with the opentok JS client that shares the publishers audio as well.
Screen Sharing works fine. But the audio is never shared.
Now, I noticed a warning in the console (Firefox) saying Invalid audioSource passed to Publisher - when using screen sharing no audioSource may be used. Does that mean it is not possible at all, or that the audio source is invalid?
With v2.13.0 it is now possible to pass a MediaStreamTrack as a custom audioSource and videoSource to initPublisher. This means you are able to add your microphone audio to the screen sharing stream. This will only work in Chrome or Firefox. IE does not support MediaStreamTrack's and Safari does not currently support screensharing.
const publish = Promise.all([
OT.getUserMedia({
videoSource: 'screen'
}),
OT.getUserMedia({
videoSource: null
})
])
.then(([screenStream, micStream]) => {
return OT.initPublisher(null, {
videoSource: screenStream.getVideoTracks()[0],
audioSource: micStream.getAudioTracks()[0]
});
});
Here is a sample of it all working https://output.jsbin.com/wozuhuc This sample will only work in Firefox because Chrome needs an extension.
I contacted the tokbox support and they confirmed, that the audio has to be published in an additional stream.
I had a go at making this work in Chrome, which is possible by using getDisplayMedia({video: true, audio: true}), which now shows a tickbox to allow the user to share device audio:
You can then use this to create a normal publisher which uses the video and audio streams from this call like so:
let prom = navigator.mediaDevices.getDisplayMedia({ video:true, audio: true });
prom.then(function(result) {
console.log("Collected permission. Going to start publishing.");
desktopStream = result;
var audioStream = desktopStream.getAudioTracks()[0];
var videoStream = desktopStream.getVideoTracks()[0];
console.log(audioStream);
// Screen sharing is available. Publish the screen.
screenPublisher = OT.initPublisher('ownScreen',
{
videoSource: videoStream,
audioSource: audioStream,
name: 'Sharing Screen',
maxResolution: { width: 1920, height: 1920 },
mirror: false,
fitMode: "contain",
},
function(error) {
if (error) {
console.log(error);
// Look at error.message to see what went wrong.
} else {
session.publish(screenPublisher, function(error) {
if (error) {
handleError();
}
$('#shareScreen').fadeOut(150, function(){
$('#stopShare').fadeIn(150);
});
$('#stopShare').addClass('blob blue');
});
}
}
);
You can also add a name to the screen share publisher to allow subscribers to distinguish between a regular video publisher and this new custom screen share publisher.
If you create a subscriber, and connect it to the session, it will receive audio and video from all publishers. As far as I know, there is no audio in screen sharing, that's why you cannot publish it. That should solve it. I hope this helps.
Related
I'm working on a Flex plugin and trying to get a notification sound to play when a new SMS is accepted, very similar to how a sound is played when a call is accepted. This is where I'm at:
manager.workerClient.on("reservationCreated", (reservation) => {
if(reservation.task.taskChannelUniqueName === 'sms') {
flex.AudioPlayerManager.play(
{
url: "twiml link",
repeatable: false
});
}
});
Not entirely sure where to go from here.
I am using FCM for web push notifications.
All worked fine until suddenly Firefox stopped delivering push notifications except the page is open in the browser.
If the page is loaded and not in focus, notifications come in. I console.log()'d the onBackgroundMessage payload and I can confirm that the service worker receives it. But if I close that tab and send a push notification, it does not receive it.
The same setup works on Chrome, Opera, and Edge just fine no errors. Is there something with firefox?
Firefox version: 88.0.1 (64-bit)
in my Service Worker I have:
messaging.onBackgroundMessage(function (payload) {
// Customize notification here
var action_label;
if('action_label' in payload.data){
action_label = payload.data.action_label;
} else{
action_label = "Details";
}
const notificationTitle = payload.data.title;
const notificationOptions = {
body: payload.data.body,
icon: payload.data.icon,
requireInteraction: true,
data: {
click_action: payload.data.click_action,
url:payload.data.click_action
},
actions: [
{
action: "open_url", title: action_label
}
]
};
self.registration.showNotification(notificationTitle, notificationOptions);
});
This had to do with Firefox quitting instead closing, obtainable in MacBooks.
Not a real technical issue.
I have a websocket stream being listened:
widget.channel.stream.listen((data) {
print("!!!!new msg: $data");
var dataJson = json.decode(data);
print(dataJson["content"]);
// do my job
setState(() {
_allAnimateMessages.insert(0, newMsg);
});
newMsg.animationController.forward();
});
But, when enter that page again, there was an error says:
Bad state: Stream has already been listened to.
How to make it as broadcast and other pages can receive that broadcast?
Solution for package web_socket_channel:
final channel = IOWebSocketChannel.connect(socketUrl);
final streamController = StreamController.broadcast();
streamController.addStream(channel.stream);
After that simply use streamController.stream to listen web socket events.
You can use broadcasts.
//Here is the solution
StreamController<String> streamController = new StreamController.broadcast(); //Add .broadcast here
//Now you can listen from various places
#override
void initState() {
super.initState();
print("Creating a StreamController...");
//First subscription
streamController.stream.listen((data) {
print("DataReceived1: " + data);
}, onDone: () {
print("Task Done1");
}, onError: (error) {
print("Some Error1");
});
//Second subscription
streamController.stream.listen((data) {
print("DataReceived2: " + data);
}, onDone: () {
print("Task Done2");
}, onError: (error) {
print("Some Error2");
});
streamController.add("This a test data");
print("code controller is here");
}
Font: https://medium.com/#ayushpguptaapg/using-streams-in-flutter-62fed41662e4
When using broadcasts you can have multiple listeners in the same stream.
If you simply use a stream without ".broadcast ()" you can only have one listener
As there are no useful answer, I update my answer here for other reference.
duplicate subscribe stream is a desired behaviour in flutter.
if you are just using a StreamBuilder, the stream can be listen only once. think about it, if your stream can be listen to many other pages or widgets, then data would be repeated.
But if you want using one single stream, and update all widgets
this is really can be occured when develop a complicated app, for example, you are building a chat app, new message comes, you should update many pages UI (your dialog chat ui, your session list ui....), then you should subscribe this streams in many pages, I still not found a proper way to do this, except make this stream to be broadcast, and do your work.
I have been using Botkit Facebook Messenger and I can receive text messages from Facebook perfectly, however I can not capture audio messages, images or attachments.
Has anyone been able to capture these types of messages?
var Botkit = require('botkit');
var controller = Botkit.facebookbot({
access_token: process.env.access_token,
verify_token: process.env.verify_token,
})
var bot = controller.spawn({
});
// if you are already using Express, you can use your own server instance...
// see "Use BotKit with an Express web server"
controller.setupWebserver(process.env.port,function(err,webserver) {
controller.createWebhookEndpoints(controller.webserver, bot, function() {
console.log('This bot is online!!!');
});
});
// this is triggered when a user clicks the send-to-messenger plugin
controller.on('facebook_optin', function(bot, message) {
bot.reply(message, 'Welcome to my app!');
});
// user said hello
controller.hears(['hello'], 'message_received', function(bot, message) {
bot.reply(message, 'Hey there.');
});
controller.hears(['cookies'], 'message_received', function(bot, message) {
bot.startConversation(message, function(err, convo) {
convo.say('Did someone say cookies!?!!');
convo.ask('What is your favorite type of cookie?', function(response, convo) {
convo.say('Golly, I love ' + response.text + ' too!!!');
convo.next();
});
});
});
there is an example for stickers, images, and audio replies in the facebook starter project: https://github.com/howdyai/botkit-starter-facebook/blob/master/skills/sample_events.js
If you have trouble using them, feel free to create an issues on the github!
I know it may sound shady, but I'm developing a window recording program (right now using ffmpeg + gdigrab to grab windows) that records
The question is, can I pass the window to be recorded without the user having to choose it?
Thanks!
If you're using Chrome browser, you can open Google Chrome Properties dialog, find the "Target" box and put --enable-usermedia-screen-capturing at the end. E.g.
"C:\Program Files (x86)\Google\Chrome\Application\chrome.exe" --enable-usermedia-screen-capturing
Now relaunch the Chrome browser.
Screenshot for chrome properties box:
Now try following code on any HTTPs page (or on localhost page):
var screen_constraints = {
mandatory: {
chromeMediaSource: 'screen'
},
optional: []
};
var hints = {
audio: false,
video: screen_constraints
};
navigator.webkitGetUserMedia(hints, function(screen) {
// this is your screen; record it using MediaRecorder or RecordRTC
}, function(error) {
console.error(error);
});
Want to try on HTTP pages? Following flag may work:
--allow-http-screen-capture
Chrome flags reference: http://peter.sh/experiments/chromium-command-line-switches/