TLDR: How can I initialise my conversations with data sent from the backchannel and use that information throughout the conversation with a user?
Using Microsoft botbuilder, a "backchannel" mechanism is provided with which I can send data to and from the bot.
Similar to this answer I am using the backchannel method to send a conversationStarted event to the bot in order to start a conversation as you might expect.
Upon receiving the event, I start a dialog which fires off my proactive messages to the user.
bot.on('event', event => {
if (event.name === 'conversationStarted') {
bot.beginDialog(event.address, determineWhichDialogToStart(event))
}
})
I can then use middleware or event listeners to intercept this event and see its contents. Sure enough I can see the event
{ type: 'event',
name: 'conversationStarted',
text: '',
myMetaProperty: 'foobar',
attachments: [],
entities: [],
address:
{ id: '8f5d3952-df3b-4340-89b4-97360f8d4965',
channelId: 'emulator',
user: { id: '33abc508-86d7-49c1-aa68-00a3491fda12' },
conversation: { id: '4e8a943d-6a45-41f2-aa11-6671cc1ca4f3' },
bot: { id: 'bot' },
serviceUrl: 'http://localhost:3010/' },
source: 'emulator',
agent: 'botbuilder',
user: { id: '33abc508-86d7-49c1-aa68-00a3491fda12' } }
But the events I can listen to to see this event, don't have access to the session because the message hasn't yet been dispatched to one it seems.
If I use the botbuilder middleware, or the routing event, I then see that it's been turned into a message rather than an event, and has lost my meta data passed to the event. (see myMetaProperty)
{ type: 'message',
agent: 'botbuilder',
source: 'emulator',
sourceEvent: {},
address:
{ id: '8f5d3952-df3b-4340-89b4-97360f8d4965',
channelId: 'emulator',
user: { id: '33abc508-86d7-49c1-aa68-00a3491fda12' },
conversation: { id: '4e8a943d-6a45-41f2-aa11-6671cc1ca4f3' },
bot: { id: 'bot' },
serviceUrl: 'http://localhost:3010/' },
text: '',
user: { id: '33abc508-86d7-49c1-aa68-00a3491fda12' } }
I tried getting access to the session in my receive event/middleware per the comment here on Github but whilst I get access to a session, it's not the session.
bot.on('receive', event => {
bot.loadSessionWithoutDispatching(event.address, (error, session) => {
session.conversationData.myMetaProperty = event.myMetaProperty
})
})
This actually ends up starting a new session - having looked into the loadSession/loadSessionWithoutDispatching, they both lead to startSession and therefore me adding data to the session is lost when i try to use it in a dialog
bot.dialog('example', [
(session, args, next) => {
console.log(session.conversationData.myMetaProperty) // nope
console.log(session.cantUseThisEither) // nope
session.send('foo')
next()
}
])
Just to reiterate the question now that there's been a bit of background, how can I initialise my conversations with data sent from the backchannel and use that information throughout the conversation with a user?
I ended up starting a generic dialog which sets up my session and then moves onto the root dialog.
bot.on('event', event => {
if (event.name === 'conversationStarted') {
bot.beginDialog(event.address, 'initialise-conversation', {
myMetaProperty: event.myMetaProperty
});
}
});
bot.dialog('initialise-conversation', (session, args, next) => {
session.conversationData.myMetaProperty = args.myMetaProperty
session.beginDialog('/');
});
Then whenever I need myMetaProperty throughout my conversation I can get it from session.conversationData
--
In response to the comment below, here's some additional information which may help someone.
The shape of the event you receive on the bot must contain an address obviously. For example here's the one I send to my bot.
{ myMetaProperty: 'foo',
name: 'conversationStarted',
type: 'event',
address:
{ id: '4q5aaBw77aNGGvrpkeji8A',
channelId: 'custom-directline',
user: { id: 'c214imy6PCJZY4G1x2AXSD' },
conversation: { id: 'bGvMBjEmFTg3DMxsbvJGjH' },
bot: { id: 'bot' },
serviceUrl: 'https://localhost:3010/' },
source: 'custom-directline',
agent: 'botbuilder',
user: { id: 'c214imy6PCJZY4G1x2AXSD' } }
Start by ensuring that you're sending the address. The key bits of information for routing messages back to an individual are the user and conversation objects.
For example, if you're sending information using the backchannel of the botframework webchat you'd do it as follows.
var user = { id: uuidv4() };
var botConnection = new BotChat.DirectLine({...});
botConnection.postActivity({
type: 'event',
from: user,
name: 'conversationStarted',
myMetaProperty: 'foo',
}).subscribe();
BotChat.App({
user: user,
botConnection: botConnection,
}, document.getElementById("bot"));
This will be picked up by the event listener described above.
Another thing to note is that the emulator (and possibly other clients/channels send a conversationUpdate event which I just passed onto my conversationStarted event when I was using it. Here's the listener to do that for reference in case it's useful.
bot.on('conversationUpdate', message => {
message.membersAdded &&
message.membersAdded
.filter(identity => identity.id === message.address.bot.id)
.forEach(identity => bot.emit('conversationStarted', message));
});
You can pass data via beginDialog(address: IAddress, dialogId: string, dialogArgs?: any, done?: (err: Error) => void): void;, you can pass the data you want to send as the third parameter in this function.
You can refer to the source code at https://github.com/Microsoft/BotBuilder/blob/master/Node/core/src/bots/UniversalBot.ts#L243 for more details.
Related
We have welcome examples using OnMembersAddedAsync method but no examples showing how to handle user leaving conversation. I tried to override OnMembersRemovedAsync but it does not seem to be invoked (at least when I use bot framework emulator).
I need to do some cleanup at the event of user leaving/left conversation.
An example or any tips would be appreciated.
Update: I'm using C# and Bot framework v4
This is going to be channel specific as it is dependent on the channel providing a feature that sends an update when the user leaves a conversation. Any other channels, you will need to research.
For Facebook, I was unable to find a scope that covers such an action. These are the available scopes which you can reference more closely here:
messages
message_deliveries
message_echoes
message_reads
messaging_account_linking
messaging_checkout_updates (beta)
messaging_game_plays
messaging_handovers
messaging_optins
messaging_payments(beta)
messaging_policy_enforcement
messaging_postbacks
messaging_pre_checkouts (beta)
messaging_referrals
standby
Web Chat, as a feature, also does not include this. However, given this is a web page, you can utilize the onbeforeunload() window function to dispatch an event. The event listener will make use of Web Chat's store to dispatch either a message or event to the bot. For the sake of clarity, I'm sending different types of data via SEND_MESSAGE and SEND_EVENT.
const store = window.WebChat.createStore( {}, ( { dispatch } ) => next => async action => {
return next( action );
};
window.addEventListener( 'sendEventActivity', ( { data } ) => {
store.dispatch({
type: 'WEB_CHAT/SEND_MESSAGE',
payload: {
text: data
}
} )
,
store.dispatch( {
type: 'WEB_CHAT/SEND_EVENT',
payload: {
name: 'user_event',
value: {
name: 'end_conversation',
value: 'user ended conversation'
},
text: 'The user has left the conversation.'
}
} )
} );
window.onbeforeunload = function() {
const eventSendActivity = new Event( 'sendEventActivity' );
eventSendActivity.data = 'User left conversation';
window.dispatchEvent( eventSendActivity );
}
{ type: 'message',
id: '4uPdpZhlTFfBMziBE7EmEI-f|0000004',
timestamp: 2020-01-10T18:21:26.767Z,
serviceUrl: 'https://directline.botframework.com/',
channelId: 'directline',
from: { id: 'dl_123', name: 'johndoe', role: 'user' },
conversation: { id: '4uPdpZhlTFfBMziBE7EmEI-f' },
recipient: { id: 'botberg#QaeuoeEamLg', name: 'Dungeon Runner' },
textFormat: 'plain',
locale: 'en-US',
text: 'User left conversation',
channelData:
{ clientActivityID: '15786804807910gegwkp2kai',
clientTimestamp: '2020-01-10T18:21:20.792Z' }
}
{ type: 'event',
id: '4uPdpZhlTFfBMziBE7EmEI-f|0000005',
timestamp: 2020-01-10T18:21:26.780Z,
serviceUrl: 'https://directline.botframework.com/',
channelId: 'directline',
from: { id: 'dl_123', name: 'johndoe', role: 'user' },
conversation: { id: '4uPdpZhlTFfBMziBE7EmEI-f' },
recipient: { id: 'botberg#QaeuoeEamLg', name: 'Dungeon Runner' },
locale: 'en-US',
channelData:
{ clientActivityID: '1578680480821h7kgfm9cyz',
clientTimestamp: '2020-01-10T18:21:20.821Z' },
value:
{ name: 'end_conversation', value: 'user ended conversation' },
name: 'user_event'
}
Hope of help!
Update (Aug. 6th, 2021):
As Chrome, and other browsers, disallow blocking / delaying the closing of a window during onbeforeunload(), sending an event using an event handler or listener is usually unreliable, at best. At worst, it just doesn't work.
However, there is another method that does appear work by using 'window.navigator.sendBeacon()' (providing it's supported).
sendBeacon is a low-level, simplified version of fetch that “asynchronously sends a small amount of data over HTTP to a web server”. It only sends as a POST, only takes the URL or URL + data as properties, and doesn’t wait for a response. As the docs state:
The data is sent reliably
It's sent asynchronously
It doesn't impact the loading of the next page
In testing, I have coupled it with a proactive messaging endpoint in my bot and it appears to work perfectly. (My code sample below is in JS, pulled from a project - for reference, here is the proactive messaging C# sample, available in other languages, as well).
When I navigate to another page or close the browser, sendBeacon posts the message to the endpoint creating the proactive message, which, in turn, sends an activity to the bot. When I return to the Web Chat page, the message is now visible in the chat window. Keep in mind, I also have persistence set up in Web Chat allowing me to return to a conversation previously started.
In the below image, I demo loading Web Chat, navigating to my browser’s home page, and then return – the proactive message is now visible in the chat.
(Web Chat) hosting page:
window.onbeforeunload = () => {
let body = { user: { userName: 'McUser', userId: 'abc123' } };
const headers = { type: 'application/json', 'Access-Control-Allow-Origin': '*' };
const blob = new Blob( [ JSON.stringify( body ) ], headers )
navigator.sendBeacon( 'http://localhost:3978/api/notify', blob )
}
Bot's proactive messaging endpoint:
server.post('/api/notify', async (req, res) => {
const userName = req.body.user.userName;
const userId = req.body.user.userId;
for (const conversationReference of Object.values(conversationReferences)) {
await adapter.continueConversation(conversationReference, async (turnContext) => {
await turnContext.sendActivity(`${ userName } (userId: ${ userId }) exited chat.`);
});
}
});
We are using botframework-webchat v4. Is there any way to provide history that will be shown up in the chat?
This is currently what I have, but its not working, not sure what format should be for activities in store.
const store = window.WebChat.createStore(
{
activities: ['{"type":"message",...}']
},
({ dispatch }: { dispatch: any }) => (next: any) => (action: any) => {
if (action.type === 'DIRECT_LINE/INCOMING_ACTIVITY') {
const { activity } = action.payload;
if (activity.type === 'event' && activity.name === 'sample:backchannel') {
alert(JSON.stringify(activity, null, 2));
}
}
return next(action);
}
)
window.WebChat.renderWebChat(
{
directLine: this.directLine,
userID: this.userId,
styleOptions,
store
},
this.botWindowElement.nativeElement
);
Thanks in advance!!
You're solution above will, technically, work. Although, it's not very scalable in the long run. I would recommend you look over this BotFramework-WebChat experimental sample, Conversation History. It utilizes the sendConversationHistory API. This sample is a bit on the complex side, but will do precisely what you are wanting, i.e. load a previous user's conversation when a new session is started.
If you are wanting to reconnect a previous conversation (meaning continue a conversation using the same conversationId) then you should be aware that the Direct Line service has certain limitations. Reconnecting will only work for up to 14 days after the last activity to that conversation and only 24 hours if activities are present.
Hope of help!
#StevenKanberg Thanks for the help!
I found the answer in source code of BotFramework-WebChat.
Here is the sample,
test('absolute timestamp', async () => {
const activities = [
{
type: 'message',
id: '6266x5ZXhXkBfuIH0fNx0h-o|0000000',
timestamp: '2019-08-08T16:41:12.9397263Z',
from: {
id: 'dl_654b35e09ab4149595a70aa6f1af6f50',
name: '',
role: 'user'
},
textFormat: 'plain',
text: 'echo "Hello, World!"'
},
{
type: 'message',
id: '6266x5ZXhXkBfuIH0fNx0h-o|0000001',
timestamp: '2019-08-08T16:41:13.1835518Z',
from: {
id: 'webchat-mockbot',
name: 'webchat-mockbot',
role: 'bot'
},
text: 'Echoing back in a separate activity.'
},
{
type: 'message',
id: '6266x5ZXhXkBfuIH0fNx0h-o|0000002',
timestamp: '2019-08-08T16:41:13.3963019Z',
from: {
id: 'webchat-mockbot',
name: 'webchat-mockbot',
role: 'bot'
},
text: 'Hello, World!'
}
];
const styleOptions = { timestampFormat: 'absolute' };
const { driver } = await setupWebDriver({ storeInitialState: { activities }, props: { styleOptions } });
I am using Bot Framework Webchat. There are few user related data which I am posting using back channel post activity through the store option to greet the user.
<ReactWebChat
activityMiddleware={ activityMiddleware }
directLine={ window.WebChat.createDirectLine( this.state.token ) }
store = {this.handleGetStore()}
styleOptions={styleOptions}
/>
handleGetStore returns the store data:
handleGetStore(){
const store = window.WebChat.createStore({}, ({ dispatch }) => next => action => {
if (action.type === 'DIRECT_LINE/CONNECT_FULFILLED') {
dispatch({
type: 'WEB_CHAT/SEND_EVENT',
payload: {
name: 'userDetail',
value: this.state.userDetail
}
});
}
return next(action);
});
return store;
}
When the connection initiates the loader appears.
After that there is delay of about 3-5 seconds before the welcome message appears and in the meantime the Webchat seems ready for the user.
A slight delay of 3 seconds is acceptable but quite often the delay is upto 10 seconds or more. I understand that this can be slightly improved by using the Always On feature of the App Service and scaling up the plan. Is there a way I can wait for the back channel welcome message to appear and show the loader until then?
Reference: https://github.com/microsoft/BotFramework-WebChat/pull/1866
Unfortunately, the connection status display relies on events received from DirectLineJs and Web Chat does not support customizing its behavior at the moment. That being said, there is a hacky way to accomplish what you're trying to do by dispatching pseudo DirectLine events.
Here are the steps below:
Create a flag that will indicate whether or not the bot has sent a welcome message - received_welcome_message.
When Web Chat dispatches a connection fulfilled event, check the flag
to ensure a welcome message has been received. If the bot has not
sent a welcome message, dispatch the welcome event to the bot and reset the
connection status to fulfilling.
When Web Chat receives an activity
from the bot, check if it is a welcome message. I would recommend
adding a name attribute to message on the bot side to check - await
context.sendActivity({ text: 'Welcome', name: 'welcome'}). If the
activity is a welcome message, dispatch a connection fulfilled event and set the flag to true.
For more details take a look at the code snippets below.
let received_welcome_message = false;
const store = createStore(
{},
({ dispatch}) => next => action => {
if (action.type === 'DIRECT_LINE/CONNECT_FULFILLED') {
if (!received_welcome_message) {
dispatch({
type: 'DIRECT_LINE/CONNECT_FULFILLING'
});
dispatch({
type: 'WEB_CHAT/SEND_EVENT',
payload: { name: 'webchat/join' }
});
return
}
} else if (action.type === 'DIRECT_LINE/INCOMING_ACTIVITY' && action.payload.activity.name === 'welcome') {
received_welcome_message = true;
dispatch({
type: 'DIRECT_LINE/CONNECT_FULFILLED',
});
}
return next(action);
}
);
Edit
A less hacky approach is to dispatch a post activity pending event when the connection to the bot is fulfilled to mimic the bot sending a welcome message. Note, that the bot is unaware of the mimicked activity. See the code snippet below.
const store = createStore(
{},
({ dispatch}) => next => action => {
console.log(action)
if (action.type === 'DIRECT_LINE/CONNECT_FULFILLED') {
dispatch({
type: 'DIRECT_LINE/POST_ACTIVITY_PENDING',
meta: { method: 'keyboard' },
payload: {
activity: {
from: { role: "bot" },
text: "Welcome Message",
textFormat: "plain",
timestamp: new Date().toString(),
type: "message"
}
}
})
}
return next(action);
}
Hope this helps!
I'm using a directline inside my website and I was wondering if there is anyway to get the URL of the website inside my bot code. Previously, in v3, I was initializing the chat with:
BotChat.App({
directLine: { secret: "{directline_secret}" },
user: { id: 'You', referrer: window.location.href},
bot: { id: '{bot_id}' },
resize: 'detect'
}, document.getElementById("bot"));
and I was able to get the referrer with this line of code activity.From.Properties["referrer"].ToString(), but in v4 I can't find a way to get the referrer inside the bot.
Can someone help me?
Thanks in advance.
In v4 the value is part of the turnContext.activity (in Node) or turnContext.Activity (in C#) object. Passing the url value as you have done in your question (i.e., as part of the user object) you would access it like so (Node example):
async onTurn(turnContext) {
if (
turnContext.activity.type === "event" && turnContext.activity.name === "eventName"
) {
this.userProfile.location = turnContext.activity.from.referrer;
await console.log(this.userProfile.location);
}
I included a name as well as specified a type in my BotChat.App post to match this event to in the turnContext.activity:
function testMethod(someValue) {
botConnection
.postActivity({
from: { id: 'me', referrer: window.location.href },
name: 'eventName',
type: 'event',
value: someValue
})
.subscribe(function (id) {
console.log('"eventName" sent');
});
};
In this example, the method is tied to a button being pressed on the page.
Hope of help!
I read that you can share location with the backchannel.
I was thinking to use this code to talk directly to the bot:
function postButtonMessage() {
botConnection.postActivity({
entities:{type: "ClientCapabilities", requiresBotState: true, supportsTts: true, supportsListening: true},
from: { id: 'userid', name: 'username' },
name: 'botname',
type: 'message',
value: 'Hi',
textFormat: 'plain'
})
.subscribe(function (id) {
console.log('"buttonClicked" sent');
});
};
But I get an error saying "bad gateway 502", but when I talk through the web channel windows it works perfectly so I know that the direct line key is configured correctly. And when I use the type: event instead of message it works fine and I don't have problems, so I am confused about that.
(Question originally asked at https://github.com/Microsoft/BotFramework-WebChat/issues/778#)
There are two approaches here.
One is to use the backchannel to send an event activity to the bot with whatever data you like. You could do this when a button is clicked, at regular intervals, or whenever the location changes.
var dl = new BotChat.DirectLine({secret});
BotChat.App({
botConnection: dl,
// other Chat props go here
});
function postButtonMessage() {
dl.postActivity({
from: { id: 'userid', name: 'username' },
type: 'event',
name: 'location',
value: { /* location goes here */ }
})
.subscribe(id => {
console.log('"buttonClicked" sent');
});
};
The other is to use client middleware to send that data with every message, by intercepting and modifying each message as it goes out. The advantage to this approach is that each message is 'tagged' with its location. The disadvantage is that you only get location updates when the user sends a message.
var dl = new BotChat.DirectLine({secret});
BotChat.App({
botConnection: {
... dl,
postActivity: activity => dl.postActivity({
... activity,
channelData: { location: /* location goes here */ }
})
},
// other Chat props go here
});
And of course you could do both!