I'm having a go with speech for a BOT. I have been running through the Microsoft Tutorial found here. I've taken the example Echo BOT example from there, found here, as a basis so I can use it as a basis going forward. This has been successfully deployed to my Azure environment. Also in the tutorial, you run your Bot through the Direct Line Speech Client v1, everything works as expected when doing this.
I have looked at the Bot Framework Web Chat speech notes to get the Bot working using this as my channel. Here is my code for this:
<!DOCTYPE html>
<html lang="en-US">
<head>
<title>Web Chat: Browser-supported speech</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<script src="https://cdn.botframework.com/botframework-webchat/latest/webchat.js"></script>
<style>
html, body { height: 100% }
body { margin: 0 }
#webchat {
height: 100%;
width: 100%;
}
</style>
</head>
<body>
<div id="webchat" role="main"></div>
<script>
(async function () {
window.WebChat.renderWebChat({
directLine: createDirectLine({
secret: '<My Direct Line secret>'
}),
language: 'en-US',
webSpeechPonyfillFactory: await createCognitiveServicesSpeechServicesPonyfillFactory({
region: '<Speech cognitive service region>',
subscriptionKey: '<Speech cognitive service key>'
})
}, document.getElementById('webchat'));
document.querySelector('#webchat > *').focus();
})().catch(err => console.error(err));
</script>
</body>
</html>
I'm able to get this working and I can do speech to text and text inputs and the contents is written back, but when it tries to do text to speech back I'm getting the following errors in the browser console:
POST https://<region>.tts.speech.microsoft.com/cognitiveservices/v1 net::ERR_ABORTED 400 (Speak node can only be the root.)
webchat.js:1 Error: Failed to syntheis speech, server returned 400
at webchat.js:1
at c (webchat.js:1)
at Generator._invoke (webchat.js:1)
at Generator.e.<computed> [as next] (webchat.js:1)
at n (webchat.js:1)
at s (webchat.js:1)
I'm not quite sure if it is something in the script code or the Bot, let me know if you need any more detail. Thanks in advance!
It is weird... I hosted a echo bot on Azure and just copy and paste your html code and did some config ,I tested on edge firefox and chrome but everything works well including tts calling :
You can try it here and check the config . Hope it helps .
Related
I have created few scenarios in health bot designer. I am trying to integrate with my front end. However, I don't see any complete documentation around integrate process. I have already referred https://github.com/Microsoft/HealthBot-WebChat without any luck. How do I get directline link for healthbot. I have tried with web bot and able to generate directline but not sure how to link web bot channel to health bot scenario. Any help?
You can integrate the Healthcare bot service into a web application using WebChat. First, you need to get your WebChat Secret from the Healthcare Bot Service Manager. In the pane on the left, click on the integrations blade, select secrets in the drop down options, and copy the webchat_secret.
Once you have the secret, you can request a token from DirectLine and render a WebChat component on your web app. Take a look at the example below.
<!DOCTYPE html>
<html lang="en-US">
<head>
<title>Healthcare bot</title>
<script src="https://cdn.botframework.com/botframework-webchat/master/webchat.js"></script>
<style>
html, body { height: 100% }
body { margin: 0 }
#webchat,
#webchat > * {
height: 100%;
width: 100%;
}
</style>
</head>
<body>
<div id="webchat" role="main"></div>
<script>
(async function() {
// Note, for the simplicity of this example, we are fetching the DirectLine token here;
// however, it is recommended that you create a backend REST API to generate and manage
// your tokens.
const res = await fetch('https://directline.botframework.com/v3/directline/tokens/generate',
{
method: 'POST',
headers: {
'Authorization': `Bearer <WEBCHAT_SECRET>`,
'Content-Type': 'application/json'
},
body: {
// The user id must start with `dl` and should be unique for each user.
User: { Id: 'dl_user_id' }
}
});
const { token } = await res.json();
window.WebChat.renderWebChat({
directLine: window.WebChat.createDirectLine({ token }),
}, document.getElementById('webchat'));
})().catch(err => console.log(err));
</script>
</body>
Note, for the simplicity of this example, we are fetching the DirectLine token here; however, it is recommended that you create a backend REST API to generate and manage your tokens.
Hope this helps!
Found a way to do it. We need to add a model and enable the trigger through Health Bot Management Portal
There is a direct way to trigger a scenario from the front end. If you want to completely rely on your own responses, then you have to turn off the built-in scenarios and call a scenario name in the event post javascript code. Look at the "trigger" element below:
botConnection
.postActivity({
type: "event",
value: {
trigger: "your_scenario_name_here", args: {}
},
from: your_user_name,
name: "BeginDebugScenario"
});
We have created a bot using ms bot framework and wanted to pass some data to bot which user will not input. Like I was thinking if we can pass query string to below web channel.
https://webchat.botframework.com/embed/myformbot?s=XXXXXX&mycustomfield=text
And from bot api code, I can somehow read this querystring parameter...Ideally I know we don't have control over above webchat link, we are only giving access to bot api. But is this possible or any other way to pass data to bot which is not entered by user ?
We are planning to display web channel url to different sites in iframe and wanted to pass currently browsed url to identify from where user has started conversation.
Thanks
Siddharth
This is easily done using the BotFramework-WebChat library. There are just a few steps to perform to get yourself setup.
First, include a div with an id for the webchat instance of your bot to anchor to. In this example, the id is "webchat".
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no" />
<title>Bot Chat</title>
<style>
#webchat,
#webchat>* {
border: 1px solid #333;
float: left;
min-height: 600px;
height: 600px;
position: relative;
width: 460px;
}
</style>
</head>
<body>
<div class="example">
<h2>Back Channel Bot</h2>
<div id="webchat"></div>
Next, you will want to include the following scripts in your html page. Please keep in mind that you should NOT store your secret in the browser or client app. It is included here for simplicity.
Here's what is happening. You use the Direct Line secret (from the channel settings in your Azure bot) to generate a token. The token is used to instantiate the bot webchat control. In this example, the user's location is saved in the store's payload as the activity object. This is sent to the bot on any POST_ACTIVITY event (i.e. the user interacts with the bot).
<script src='https://code.jquery.com/jquery-3.3.1.min.js'></script>
<script src='https://cdn.botframework.com/botframework-webchat/master/webchat.js'></script>
<script src='https://unpkg.com/simple-update-in/dist/simple-update-in.production.min.js'></script>
<script>
(async function () {
// To talk to your bot, you should use the token exchanged using your Direct Line secret.
// You should never put the Direct Line secret in the browser or client app.
const res = await fetch('https://directline.botframework.com/v3/directline/tokens/generate', {
method: 'POST',
headers: {
'Authorization': 'Bearer ' + secret
},
json: true
});
const { token } = await res.json();
let location = window.location.href;
let store = window.WebChat.createStore(
{},
({ dispatch }) => next => action => {
if (action.type === 'DIRECT_LINE/POST_ACTIVITY') {
// simple-update-in is used to update the "action"
action = window.simpleUpdateIn(action, ['payload', 'activity', 'channelData', 'location'], () => location);
}
return next(action);
}
);
window.WebChat.renderWebChat({
directLine: window.WebChat.createDirectLine({ token }),
store,
styleOptions: {
botAvatarInitials: 'BF',
userAvatarInitials: 'WC'
}
}, document.getElementById('webchat'));
document.querySelector('#webchat > *').focus();
})().catch(err => console.error(err));;
</script>
</body>
</html>
On the bot side, the bot is listening for the incoming message. The activity object will contain the data you sent and will look something like this:
channelData: { clientActivityID: '15469824216440.emdrovn0f5h', location: 'http://localhost:3000/' }
Hope of help!
In our app, we have a simple Google-Sign-In flow where a popup opens, users log in and grant us offline permission for accessing Google Analytics.
We just got emailed that we're using a Google+ API (plus.people.getOpenIdConnect method) that is about to get deprecated, but we don't use it in our code.
I can't seem to figure out where we are using Google+ API so I could replace it.
Here is our simple code:
prepareGoogleClient() {
$.ajax({
url: "//apis.google.com/js/client:platform.js",
dataType: "script"
}).done(() => {
gapi.load("auth2", () => {
let auth = gapi.auth2.init({
client_id: ENV.googleClientId,
scope:
"https://www.googleapis.com/auth/analytics.readonly https://www.googleapis.com/auth/webmasters.readonly"
});
this.auth = auth;
});
if (gapi.auth2 && !this.auth) {
this.auth = gapi.auth2.getAuthInstance();
}
});
}
Later on we call this.auth.grantOfflineAccess(params), which returns the token that we save for later.
If I disable Google+ API in our Google Platform dashboard, the Sign-In stops working and the popup responds with a sign-in error. I was also able to confirm that Google+ API (from its metrics panel) is indeed used in the process of our users signing in the popup and granting scope permissions.
How do I need to rewrite this so it won't use the deprecated plus.people.getOpenIdConnect method?
The issue was in Rails back-end code which handles OAuth2. The outdated omniauth-google-oauth2 gem was using the deprecated Google+ endpoint.
I think everyone using Google+ API's in their app, have got that mail.
Don't know if this helps but got this is from google API's site.
The Google+ Sign-in feature has been fully depreciated and will also
be shut down on March 7, 2019. Developers should migrate to the more
comprehensive Google Sign-in authentication system.
https://developers.google.com/+/web/api/javascript
https://developers.google.com/+/integrations-shutdown
Other References:
List of API's to be removed https://developers.google.com/+/api-shutdown
New Sign(identity) https://developers.google.com/identity/
Identity for web app https://developers.google.com/identity/sign-in/web/
Add Google Sign-In to Your Web App
function onSignIn(googleUser) {
// Useful data for your client-side scripts:
var profile=googleUser.getBasicProfile();
console.log("ID: " + profile.getId()); // Don't send this directly to your server!
console.log('Full Name: ' + profile.getName());
console.log('Given Name: ' + profile.getGivenName());
console.log('Family Name: ' + profile.getFamilyName());
console.log("Image URL: " + profile.getImageUrl());
console.log("Email: "+profile.getEmail());
// The ID token you need to pass to your backend:
var id_token=googleUser.getAuthResponse().id_token;
console.log("ID Token: "+id_token);
}
<html lang="en">
<head>
<meta name="google-signin-scope" content="profile email">
<meta name="google-signin-client_id" content="YOUR_CLIENT_ID.apps.googleusercontent.com">
<script src="https://apis.google.com/js/platform.js" async defer></script>
</head>
<body>
<div class="g-signin2" data-onsuccess="onSignIn" data-theme="dark"></div>
</body>
</html>
code from index.html:
<!DOCTYPE html>
<html>
<head>
<meta charset="UTF-8">
<title>Hello World!</title>
</head>
<body>
<h1>Hello World!</h1>
<!-- All of the Node.js APIs are available in this renderer process. -->
We are using Node.js <script>document.write(process.versions.node)</script>,
Chromium <script>document.write(process.versions.chrome)</script>,
and Electron <script>document.write(process.versions.electron)</script>.
<script src="./renderer.js"></script>
</body>
<form><input></form><div></div>
</html>
code in renderer.js:
var io = require('socket.io')();
var os = require('os');
var fs = require('fs');
io.listen(6000);
io.on('connection', function (socket) {
socket.on('my socketId', function (data) {
socket.emit('client connect', {
nickname: os.hostname()
});
});
});
Problem: It works fine using electron . running in browser. But client got no respond after packaging by electron-packager.
Thank you very much for any help!!
I finally solved this problem. I was copying the app to build first. But node_modules (and those packages) didn't exist in that directory. Either copy it or run npm install in build before running Electron Packager will make it work.
I am very new to flowplayer using wowza in order to have secure streaming. Below is the code I am using, but video not playing at all. I am pretty sure all files are loading properly without any 404 or 403 errors.
Here is the code:
<html>
<head>
<title>Wow! This is video</title>
<script src="js/flowplayer-3.1.4.min.js"></script>
</head>
<body>
<a href="videos/MyVideo.mp4"
style="display:block;width:425px;height:300px;"
id="wowza" class="player">
<!-- splash image inside the container -->
<img src="./flow_eye.jpg"
alt="Search engine friendly content" /></a>
<script language="JavaScript">
flowplayer("wowza", "swf/flowplayer-3.1.5.swf", {
log: { level: 'debug', filter: 'org.flowplayer.rtmp.,org.flowplayer.securestreaming.' },
clip: {
url: 'mp4:videos/MyVideo.mp4',
// use RTMP streaming
provider: 'rtmp',
// with a secured connection
connectionProvider: 'secure'
},
plugins: {
// set up the RTMP streaming plugin
rtmp: {
url: "swf/flowplayer.rtmp-3.2.13.swf",
// The net connection URL with HDDN looks like this
netConnectionUrl: 'rtmpte://d.securevod.flowplayervod.netdna-cdn.com:1935/securevod.flowplayervod'
},
// set up the secure streaming plugin
secure: {
url: "swf/flowplayer.securestreaming-3.2.9.swf",
// the token value (shared secret).
token: 'bky9p52t'
}
}
});
</script>
</body>
</html>
Please test it from your end and tell me what still needs to be added in above code.
Your are using wrong script.
what I see, that you are trying to make a live stream using RTMP(Real Time Messaging Protocol). and you want to play just video.
well incase of flow-player you don't need to re-invent the wheel.
just use any sample script.
Embed Videos in your Web Pages with Flowplayer