I am using Bot Framework Web Chat and I correctly setup a front-end for the user to chat with my bot. I am trying to enable speech for it, I try following the tutorial here: https://learn.microsoft.com/en-us/azure/bot-service/bot-service-channel-connect-webchat-speech?view=azure-bot-service-3.0
The problem I have is I try to use Azure Speech Service, I setup my service correctly, and I set the key. But I am not sure where to get the CognitiveServices? The tutorial doesn't specify where to get it.
Here is my code:
<div id="bot"/>
<script src="https://cdn.botframework.com/botframework-webchat/latest/botchat.js"></script>
<script>
const speechOptionsRemote = {
speechRecognizer: new CognitiveServices.SpeechRecognizer({ subscriptionKey: '...' }),
speechSynthesizer: new CognitiveServices.SpeechSynthesizer({
gender: CognitiveServices.SynthesisGender.Female,
subscriptionKey: '...',
voiceName: 'Microsoft Server Speech Text to Speech Voice (en-US, JessaRUS)'
})
};
BotChat.App({
directLine: { secret: '...' },
user: { id: 'WebChat' },
bot: { id: '...' },
resize: 'detect',
speechOptions: speechOptionsRemote,
showUploadButton: false
}, document.getElementById("bot"));
var header = document.getElementsByClassName("wc-header");
header[0].innerHTML = "<span ><p align='center' >My Bot</p></span>"
</script>
It complain that CogntiveService is not found when I navigate to the page. Where do I get it?
Your code sample is using v3 of Webchat, which is now deprecated, see here. There is a v4 of BotFramework-WebChat on the GitHub repository, the update has been a few days ago.
So when in your code your are downloading cdn.botframework.com/botframework-webchat/latest/botchat.js, it is the v4: that explains why it can't found CognitiveServices: it has been refactored.
For using Cognitive Services Speech in the v4, have a look to the dedicated sample: https://github.com/Microsoft/BotFramework-WebChat/tree/master/samples/speech-cognitive-services-bing-speech
Related
Migrating from BotFramework V3 to V4, not able to set the chatTitle of chat window.
This is my old code
BotChat.App({
user: { id: userID, name: 'test' },
botConnection: botConnection,
locale: 'fr-fr',
showUploadButton: false,
chatTitle: 'Assistant virtuel',
resize: 'detect'
})
Now with new version, 'chatTitle' is no longer working.
window.WebChat.renderWebChat(
{
directLine: webChatConnection,
userID: userID,
username: 'test',
locale: 'fr-fr',
chatTitle: 'Assistant virtuel',
},
document.getElementById('botDiv')
);
According to this issue Here
We omitted the chat title in Web Chat v4 (compare to v3) because:
It is one of the most popular customization
It can be easily done in HTML by appending HTML code just before Web Chat DOM element
It is more intuitive to do it in plain HTML than "customizing" in Web Chat, feel over-engineered if we do it that way
In this way, the user can customize the title as much as they want, for example, adding buttons to the chat title easily.
Check this Answer here if it has what you need using Javascript
I built a QnA Maker and integrated it via Direct Line in my Website using BotFramework-WebChat for styling.
Messages of the user and the bot are appearing at the same side of the chat container. I can't figure why.
This is how it currently looks like:
This is the code I'm using:
<script>
const styleSet = window.WebChat.createStyleSet({
bubbleFromUserBackground: 'rgba(227, 227, 227, .1)',
hideUploadButton: true,
botAvatarInitials: 'WD',
sendTypingIndicator: true,
userAvatarInitials: 'you'
});
styleSet.textContent = Object.assign(
{},
styleSet.textContent,
{
fontFamily: '\'Lato\', sans-serif'
}
);
window.WebChat.renderWebChat(
{
directLine: window.WebChat.createDirectLine({
token: 'xxxxxx'
}),
styleSet,
userID: 'qna-homepage-bot',
username: 'Web Chat User',
locale: 'en-US',
},
document.getElementById('webchat')
);
document.querySelector('#webchat > *').focus();
</script>
I wasn't able to reproduce this, but I suspect you are setting your user ID to the same value as the bot ID. When Web Chat receives an activity, it sets the role property in the activity's from attribute based on the ID (you can take a look at the source code here). Web Chat then uses the role to determine how the activity is styled. If the bot id equals the user id, Web Chat will confuse the role attribute and apply the wrong CSS stylings. Try changing the userID value in the render Web Chat options to something else.
Note, the userID value should be unique for each user; otherwise, every conversation will share the same user state.
Hope this helps!
I have created few scenarios in health bot designer. I am trying to integrate with my front end. However, I don't see any complete documentation around integrate process. I have already referred https://github.com/Microsoft/HealthBot-WebChat without any luck. How do I get directline link for healthbot. I have tried with web bot and able to generate directline but not sure how to link web bot channel to health bot scenario. Any help?
You can integrate the Healthcare bot service into a web application using WebChat. First, you need to get your WebChat Secret from the Healthcare Bot Service Manager. In the pane on the left, click on the integrations blade, select secrets in the drop down options, and copy the webchat_secret.
Once you have the secret, you can request a token from DirectLine and render a WebChat component on your web app. Take a look at the example below.
<!DOCTYPE html>
<html lang="en-US">
<head>
<title>Healthcare bot</title>
<script src="https://cdn.botframework.com/botframework-webchat/master/webchat.js"></script>
<style>
html, body { height: 100% }
body { margin: 0 }
#webchat,
#webchat > * {
height: 100%;
width: 100%;
}
</style>
</head>
<body>
<div id="webchat" role="main"></div>
<script>
(async function() {
// Note, for the simplicity of this example, we are fetching the DirectLine token here;
// however, it is recommended that you create a backend REST API to generate and manage
// your tokens.
const res = await fetch('https://directline.botframework.com/v3/directline/tokens/generate',
{
method: 'POST',
headers: {
'Authorization': `Bearer <WEBCHAT_SECRET>`,
'Content-Type': 'application/json'
},
body: {
// The user id must start with `dl` and should be unique for each user.
User: { Id: 'dl_user_id' }
}
});
const { token } = await res.json();
window.WebChat.renderWebChat({
directLine: window.WebChat.createDirectLine({ token }),
}, document.getElementById('webchat'));
})().catch(err => console.log(err));
</script>
</body>
Note, for the simplicity of this example, we are fetching the DirectLine token here; however, it is recommended that you create a backend REST API to generate and manage your tokens.
Hope this helps!
Found a way to do it. We need to add a model and enable the trigger through Health Bot Management Portal
There is a direct way to trigger a scenario from the front end. If you want to completely rely on your own responses, then you have to turn off the built-in scenarios and call a scenario name in the event post javascript code. Look at the "trigger" element below:
botConnection
.postActivity({
type: "event",
value: {
trigger: "your_scenario_name_here", args: {}
},
from: your_user_name,
name: "BeginDebugScenario"
});
I'm trying to get web chat with my bot (V4 bot and web chat) to work with the speech cognitive service, using a specific voice. I've got it almost working as per this sample and others in the same folder (https://github.com/Microsoft/BotFramework-WebChat/blob/master/samples/06.c.cognitive-services-speech-services-js/index.html)
The only part of the equation I'm missing is whether I can specify the voice. I can't find how to specify voice in the samples and the web chat source code.
This page linked to from the speech cog service docco (https://learn.microsoft.com/en-gb/azure/cognitive-services/speech-service/speech-synthesis-markup) mentions specifying voice inside the SSML but I don't want to have to somehow crack open and modify the SSML being generated by the bot if I can avoid it.
Does anyone have any idea if this is possible, and if so how?
Thanks
Lee
OK I came to an answer on this myself after looking at the pony fill code. Partial snippet below. Update the list of voice to locale mappings to match the voice you want to use for the specified locale.
const speechServicesPonyfillFactory = await window.WebChat.createCognitiveServicesSpeechServicesPonyfillFactory({ authorizationToken, region });
return options => {
const ponyfill = speechServicesPonyfillFactory(options);
var speechSynthesisUtterance = ponyfill.SpeechSynthesisUtterance;
var speechSynthesis = ponyfill.speechSynthesis;
speechSynthesis.getVoices = function () {
return [
{ lang: 'en-US', gender: 'Male', voiceURI: 'Microsoft Server Speech Text to Speech Voice (en-US, BenjaminRUS)' }
];
}
return {
SpeechGrammarList: ponyfill.SpeechGrammarList,
SpeechRecognition: ponyfill.SpeechRecognition,
speechSynthesis: speechSynthesis,
SpeechSynthesisUtterance: speechSynthesisUtterance
}
};
};
...
var ponyfillFactory = await createSpeechPonyfillFactory({ authorizationToken, region });
...
// Do the usual stuff from the sample to get auth token and region...
window.WebChat.renderWebChat({
directLine: directLine,
webSpeechPonyfillFactory: ponyfillFactory,
store
}, document.getElementById('webchat'));
All the tutorial are connecting to direct line, what about using my own bot server? How do I connect in javascript?
I already setup server side:
// Setup BotFramework
var connector = new Builder.ChatConnector({
appId: "a8...",
appPassword: "ko...",
openIdMetadata: process.env.BotOpenIdMetadata
});
// Setup Bot
var bot = new Builder.UniversalBot(connector);
bot.set('storage', new Builder.MemoryBotStorage());
// Setup Server
var server = Restify.createServer();
server.listen(process.env.port || process.env.PORT || 4000, function () {
console.log("Listening on port "+(process.env.port || process.env.PORT || 4000));
});
server.post('/api/messages', connector.listen());
On the client side, here is what I currently got using direct line:
<script src="https://cdn.botframework.com/botframework-webchat/latest/botchat.js"></script>
<script src="https://cdn.botframework.com/botframework-webchat/latest/CognitiveServices.js"></script>
<script>
const params = BotChat.queryParams(location.search);
const speechOptions = {
speechRecognizer: new BotChat.Speech.BrowserSpeechRecognizer(),
speechSynthesizer: new BotChat.Speech.BrowserSpeechSynthesizer()
};
BotChat.App({
showUploadButton: false,
directLine: { secret: '_bm...' },
bot: { id: 'a8...' },
locale: params['locale'],
resize: 'detect',
speechOptions: speechOptions,
user: {
id: 'WebChatDemoUser',
name: 'You'
},
}, document.getElementById('ChatBot'));
var header = document.getElementsByClassName("wc-header");
header[0].innerHTML = "<span>Chat</span>"
</script>
So this work, but it connect directly to the bot framework, I need to connect to my server like this:
http://localhost:4000/api/messages
Or on production would be like this:
http://myserver.com:4000/api/messages
Basically, similar to how Bot Emulator connect:
DirectLineJs will call the Direct Line Connector Service by default. The connector service then calls your bot, and the bot calls back to the connector service with responses (or proactive messages). DirectLineJs does provide a domain parameter you can supply to override the default:
const dl = new DirectLine({
secret: /* put your Direct Line secret here */,
token: /* or put your Direct Line token here (supply secret OR token, not both) */,
domain: /* optional: if you are not using the default Direct Line endpoint, e.g. if you are using a region-specific endpoint, put its full URL here */
webSocket: /* optional: false if you want to use polling GET to receive messages. Defaults to true (use WebSocket). */,
pollingInterval: /* optional: set polling interval in milliseconds. Default to 1000 */,
});
However, there is a lot more involved in hosting your own Direct Line Connector Service. The SDK code will use the activity's serviceUrl property to send messages back to the connector service. The bot is expecting endpoints such as those in this MockChannelController: https://github.com/Microsoft/BotFramework-Samples/blob/master/blog-samples/CSharp/MockChannel/Controllers/MockChannelController.cs
More details are explained in this Load Testing blog post: https://blog.botframework.com/2017/06/19/load-testing-a-bot/
There's also this limited node example: offline-directline