I have a bot client and in which I have enabled speech functionality.
Everything works fine when I open the bot client in new window.
Recently we have added this bot client site url in a page as IFrame.
In this all the functionalities are working fine except speech functionality.
When we are clicking on microphone icon nothing is happening.
Have tried in multiple browsers and devices, same issue in all the devices.
Windows - Chrome, IE
IPad,
IPhone
Is it the expected behavior? or Do i need to update anything?
Please let me know if you want me to share any code.
Code.
speechOptions = {
speechRecognizer: new CognitiveServices.SpeechRecognizer({
subscriptionKey:
'xxx'
}),
speechSynthesizer: new CognitiveServices.SpeechSynthesizer({
gender: CognitiveServices.SynthesisGender.Female,
subscriptionKey: 'xxx'
voiceName: 'Microsoft Server Speech Text to Speech Voice (en-US, ZiraRUS)'
})
};
BotChat.App({
botConnection: botConnection,
user: { id: model.userId, Username: LoginUser,},
bot: { id: model.botId, name: model.botName },
resize: 'window',
speechOptions: speechOptions,
locale: 'en',
sendTypingIndicator: true
}, document.getElementById('BotChatElement'));
botConnection
.postActivity({
from: user,
name: 'requestWelcomeDialog',
type: 'event',
value: ''
})
.subscribe(function (id) {
console.log('"trigger requestWelcomeDialog" sent');
});
There's a few things you need to do in order to get speech working with WebChat.
Ensure that you are not using the version of WebChat with the embed code from the Azure Portal. You must use one of the WebChat bundles (use one of the es5 bundles if you want it to work in IE)
Speech Services are not supported on many version of major browsers. Ensure that you and your users are only using supported browsers or use Bing Speech
Chrome (and likely in the future, other browsers) blocks microphone access until certain interactions occur first on the page/iFrame. You should be able to get around this with <iframe allow="microphone *">
Related
I have an add-in that opens a dialog box using Office.context.ui.displayDialogAsync() with displayInIframe: true
I compose an email and open the add-in. Using the add-in I select the files I wish to attach to the email. On the desktop Outlook app this works perfectly.
On the web version nothing happens. I can't see any communication between the dialog and host page when using Office.context.ui.messageParent() or Office.Dialog.messageChild()
The frontend in the dialog sends a message with the file blob to the parent function in the functionfile. The functionfile should then call addFileAttachmentFromBase64Async() when it receives this message but on the web version I can't see any evidence of this.
I've read it could be to the urls in the in the manifest.xml. Here is ours:
<AppDomains>
<AppDomain>https://outlook.office.com</AppDomain>
<AppDomain>https://outlook.live.com</AppDomain>
<AppDomain>https://localhost:44312</AppDomain>
<AppDomain>https://*.{company}.cloud</AppDomain>
<AppDomain>https://localhost:3001</AppDomain>
</AppDomains>
And here is the simplified section of code that opens the dialog:
Office.context.ui.displayDialogAsync(url, { height: 60, width: 60, displayInIframe: true }, asyncResult => {
if (handleError(asyncResult)) return;
dialog = asyncResult.value;
dialog.addEventHandler(Office.EventType.DialogMessageReceived, processMessage);
const promises = [];
{working business logic}
Promise.all(promises).then(values => {
{working business logic}
event.completed();
});
});
Other issues we have in the web version that might be releavent:
The 'X' close button in the top right of the dialog box does not work. Neither will calls to Office.context.ui.closeContainer() or Office.Dialog.close()
If we set displayInIframe: false then the add-in will immediately close itself upon opening unless we throttle our internet connect. But even when throttling the attaching email process does not work. If I remove the <script type="text/javascript" src="https://appsforoffice.microsoft.com/lib/1.1/hosted/office.js" crossorigin></script> script in the .html then it no longer crashes (but naturally the app needs that)
I have tested this on both Edge and Chrome and they both have this issue. I have not tested on any other browsers
Again, the app works correctly and as expected on the Outlook desktop app. Any help or suggests are appreciated. Thanks.
Just remove the event.completed() call from the callback.
I am trying to send a file (calendar ics file) in the Teams channel. I am using the below code which works fine in webchat channel:
const reminder = fs.readFileSync(`temp/reminder.ics`);
const base64reminder = Buffer.from(reminder).toString('base64');
await dc.context.sendActivity({'text':'Click here to set up a reminder on your calendar', 'attachments': [{
name: 'Leader X Activities',
contentType: 'text/plain',
contentUrl: `data:text/calendar;base64,${ base64reminder }`}]
});
I have used both text/plain and text/calendar, the type there didn't make a difference. But teams is throwing an Unknown attachment type error. Here is what I captured in App Insights.
{"error":"Unknown attachment type","callStack":"Error: Unknown attachment type\n at new RestError (D:\\home\\site\\wwwroot\\node_modules\\#azure\\ms-rest-js\\dist\\msRest.node.js:1399:28)\n at D:\\home\\site\\wwwroot\\node_modules\\#azure\\ms-rest-js\\dist\\msRest.node.js:2495:37\n at <anonymous>\n at process._tickDomainCallback (internal/process/next_tick.js:228:7)","botName":"itInnovationBot"}
I've tried to review the documentation here, but I can't make any sense out of what it's doing with the consent and then send files. It's also using different syntax than I am, probably because it is extending TeamsActivityHandler. I'm doing this inside a waterfall dialog extending ComponentDialog, so I'm not even sure I could utilize that class if I wanted to.
It seems like I need to ask for some sort of consent to send a file, but I can't find any documentation on how to do this. What do I ened to do in order to get this ics file attached in Teams?
I am somewhat interested in doing this directly via Graph API, but I believe we have some internal corporate roadblocks to this, which is why I am trying an initial implementation with ics file attachment. And besides, that would solve only ics files and I know we will need to have the bot send other file types in the future.
EDIT: I've tried a few other things where I've perhaps made a bit of progress, but it's still not working. The below gets rid of the unknown attachment error but it is replaced with empty content error:
await dc.context.sendActivity({'text':'Click here to set up a reminder on your calendar', 'attachments': [{
name: 'Leader X Activities',
contentType: 'application/vnd.microsoft.teams.card.file.info',
contentUrl: `data:text/calendar;base64,${ base64reminder }`}]
});
Adding the content object I just get a "ServiceError/Unknown" error, which isn't very helpful troubleshooting further. I've tried all kinds of different information in the content object all with the same result.
await dc.context.sendActivity({'text':'Click here to set up a reminder on your calendar', 'attachments': [{
name: 'Leader X Activities',
contentType: 'application/vnd.microsoft.teams.card.file.info',
contentUrl: `data:text/calendar;base64,${ base64reminder }`,
content: {
uniqueId: '12345whatever'
contentType: 'text/plain'
}}]
});
EDIT 2: One further update, I've tried setting a consent card, with the assumption that I can then look for an "invoke" activity, but the consent buttons don't seem to actually generate any activity. Here's the code:
await dc.context.sendActivity({'attachments': [{
content: {
description: 'Here is the file I want to send you',
fileSizeInBytes: fileSizeInBytes,
accetpContext: fileContext,
declineContext: fileContext
},
contentType: 'application/vnd.microsoft.teams.card.file.consent',
name: 'reminder.ics'}]
})
EDIT 3: I didn't get any message or indication of activity from desktop app, but when I tried to click on the consent buttons from mobile app it was giving me a "BOTNAME does not support files" error message, even though in the teams app manifest I do have it set to support files. So it seems I'm not getting an activity because that button press is getting blocked. Not sure if this is because I am not using TeamsActivityHandler (and if that is required, then I will need to know how to incorporate that into a waterfall dialog that is using ComponentDialog).
We have a bot running in Azure (Web App Bot) that I'm trying to embed on a website. The bot is based of the Bot Builder V4 SDK Tamplate CoreBot v4.9.2. At first I used the iframe to embed the bod. This worked but didn't provide the features we need, so now im changing it to use DirectLine.
My code on the webpage looks like this:
<script crossorigin="anonymous"
src="https://cdn.botframework.com/botframework-webchat/latest/webchat.js"></script>
<div id="webchat" role="main"></div>
<script>
(async function () {
const res = await fetch('https://[my bot name here].azurewebsites.net/.bot/v3/directline/tokens/generate',
{
method: 'POST',
headers: new Headers({
'Authorization': "Bearer [my token here]"
})
});
const { token } = await res.json();
window.WebChat.renderWebChat(
{
directLine: await window.WebChat.createDirectLineAppServiceExtension({
domain: 'https://[my bot name here].azurewebsites.net/.bot/v3/directline',
token
})
},
document.getElementById('webchat')
);
document.querySelector('#webchat > *').focus();
})().catch(err => console.error(err));
</script>
After some struggles I managed to fetch a token from https://[my bot name here].azurewebsites.net/.bot/v3/directline.
And I can see the chat window on my webpage, but is says connecting for a while then it changes to Taking longer than usual to connect, like this:
In the Chrome console there is an error saying Failed to connect Error: Connection response code 500. When I check Chrome's Network tab I can see that the token generated completed with status 200 and that the websocket connection is open, like this:
----------EDIT---------
I just noticed that when go to https://[my bot name here].azurewebsites.net/.bot using a webbrowser, the resulting json is
{"v":"1.0.0.0.55fa54091a[some key?]","k":true,"ib":false,"ob":false,"initialized":true}
ib and ob should be true but are false, maybe this is part of the problem.
----------EDIT 2---------
OK so now I'm starting to go crazy.
Ashish helped me and at some point the ib and ob were true. They were true for most of yesterday. At some point yesterday they turned false for a short while (no more than 2 hours). I checked if someone had triggered the release pipeline but no recent releases. After that ib and ob magically turned true again and connecting to the direct line worked again.
Now this morning ib and ob were false again. And again no recent releases. I don't know what is causing this.
Does anybody know what's going on here or how to fix this? How do I find what causes ib and ob to be false?
Any help is appreciated! Thanks in advance. If you need more information, just ask and I'll post it.
If the ib and ob values displayed by the *.bot endpoint are false this means the bot and the Direct Line app service extension are unable to connect to each other.
Make sure you verify below things:
Double check the code for using named pipes has been added to the
bot.
Confirm the bot is able to start up and run at all. Useful
tools are Test in WebChat, connecting an additional channel, remote
debugging, or logging.
Restart the entire Azure App Service the bot
is hosted within, to ensure a clean start up of all processes.
Please check troubleshooting guide, it seems updated today. (still old date reflected some how, not sure why)
I am working in bot framework and enabled speech in it.
When testing the speech functionality it is not recognizing few words like "HR".
I tried with different region's subscription key (Westus, eastus, common). In all the cases it is not recognizing word "HR".
How to make it recognize that word?
Is there any option?
speechOptions = {
speechRecognizer: new CognitiveServices.SpeechRecognizer({
subscriptionKey:
'*******************'
}),
speechSynthesizer: new CognitiveServices.SpeechSynthesizer({
gender: CognitiveServices.SynthesisGender.Female,
subscriptionKey: '***********************',
voiceName: 'Microsoft Server Speech Text to Speech Voice (en-US, JessaRUS)'
})
};
Unfortunately, I don't believe this is possible at the moment. There is currently an issue on the Web Chat repo to add support for Biased Word Lists for Speech. Hopefully, it should be prioritized in the 4.6 release. I know the development is beginning to focus on creating a better user experience for speech in Web Chat.
In the meantime, you could prompt your use to phrases like "Human Resources" instead of "HR."
Hope this helps!
I am creating a firefox OS application, I want the user to be able to share a link through any application installed and eligible (facebook, twitter etc). I saw android has this kind of feature and firefox OS has it as well, as I saw it in one of it's built in applications. Went through Web API, didn't find a suitable match,
Any ideas how to do it?
This is the intended use of the Web Activity API . The idea is that applications register to handle activities like share. The Firefox OS Boiler Plate app has several examples of using Web Activities. In that example a user could share a url using code like:
var share = document.querySelector("#share");
if (share) {
share.onclick = function () {
new MozActivity({
name: "share",
data: {
number: 1,
url: "http://robertnyman.com"
}
});
};
}
Any app that handles the share activity will be shown allowing the user to pick the proper app to handle the share.