I am using the virtual assistant template provided via Microsoft's botframework-solutions Github repository, and I am having trouble maintaining the active dialog between our main entry point -- a bot developed that implements the dispatch model to determine which skill to send the user's utterance for further handling -- and the individual skill processes the user's input.
I have our dispatch model bot listening to POSTs made to localhost:3979/api/messages using the route method in mainDialog.ts to determine the relevant skill to pass the users' utterance to, which takes that utterance and determines which dialog within the skill should be used to handle the users' utterance. When we begin a dialog that implements a multi-step waterfall dialog, the dispatch model bot is not keeping track of the dialog that is active within the skill bot, and therefore the skill bot doesn't know how to route the users' utterance to the already active dialog. When we are POSTing directly to the skill bot and foregoing the dispatch model bot, the skill bot is able to keep track of the active dialog, which leads me to believe that we aren't properly managing state between the dispatch bot and the skill bot.
I've noticed that bot the virtual assistant and skill templates we're using from the botframework-solutions templates instantiate a new instance of AutoSaveStateMiddleWare within their defaultAdapter.ts, so it appears that reading and writing of conversation and user state should already be automatically managed.
mainDialog.ts within the dispatch bot, which routes utterances to the appropriate skill
protected async route(dc: DialogContext): Promise<void> {
// Get cognitive models for locale
const locale: string = i18next.language.substring(0, 2);
const cognitiveModels: ICognitiveModelSet | undefined = this.services.cognitiveModelSets.get(locale);
if (cognitiveModels === undefined) {
throw new Error('There is no value in cognitiveModels');
}
// Check dispatch result
const dispatchResult: RecognizerResult = await cognitiveModels.dispatchService.recognize(dc.context);
const intent: string = LuisRecognizer.topIntent(dispatchResult);
if (this.settings.skills === undefined) {
throw new Error('There is no skills in settings value');
}
// Identify if the dispatch intent matches any Action within a Skill if so, we pass to the appropriate SkillDialog to hand-off
const identifiedSkill: ISkillManifest | undefined = SkillRouter.isSkill(this.settings.skills, intent);
if (identifiedSkill !== undefined) {
// We have identified a skill so initialize the skill connection with the target skill
await dc.beginDialog(identifiedSkill.id);
// Pass the activity we have
const result: DialogTurnResult = await dc.continueDialog();
if (result.status === DialogTurnStatus.complete) {
await this.complete(dc);
}
} else if (intent === 'l_NOVA_general') {
// If dispatch result is general luis model
const luisService: LuisRecognizerTelemetryClient | undefined = cognitiveModels.luisServices.get(this.luisServiceGeneral);
if (luisService === undefined) {
throw new Error('The specified LUIS Model could not be found in your Bot Services configuration.');
} else {
const result: RecognizerResult = await luisService.recognize(dc.context);
if (result !== undefined) {
const generalIntent: string = LuisRecognizer.topIntent(result);
// switch on general intents
switch (generalIntent) {
case 'Escalate': {
// start escalate dialog
await dc.beginDialog(EscalateDialog.name);
break;
}
case 'None':
default: {
// No intent was identified, send confused message
await this.responder.replyWith(dc.context, MainResponses.responseIds.confused);
}
}
}
}
} else {
// If dispatch intent does not map to configured models, send 'confused' response.
await this.responder.replyWith(dc.context, MainResponses.responseIds.confused);
}
}
The following code from the skill bot's dialogBot.ts which listens to all turn events
public async turn(turnContext: TurnContext, next: () => Promise<void>): Promise<any> {
// Client notifying this bot took to long to respond (timed out)
if (turnContext.activity.code === EndOfConversationCodes.BotTimedOut) {
this.telemetryClient.trackTrace({
message: `Timeout in ${ turnContext.activity.channelId } channel: Bot took too long to respond`,
severityLevel: Severity.Information
});
return;
}
const dc: DialogContext = await this.dialogs.createContext(turnContext);
if (dc.activeDialog !== undefined) {
const result: DialogTurnResult = await dc.continueDialog();
} else {
await dc.beginDialog(this.rootDialogId);
}
await next();
}
The mainDialog.ts within the skill bot that routes to the claims_claimsStatus waterfall dialog
protected async route(dc: DialogContext): Promise<void> {
// get current activity locale
const locale: string = i18next.language.substring(0, 2);
const localeConfig: Partial<ICognitiveModelSet> | undefined = this.services.cognitiveModelSets.get(locale);
// Populate state from SkillContext slots as required
await this.populateStateFromSkillContext(dc.context);
if (localeConfig === undefined) {
throw new Error('There is no cognitiveModels for the locale');
}
// Get skill LUIS model from configuration
if (localeConfig.luisServices !== undefined) {
const luisService: LuisRecognizerTelemetryClient | undefined = localeConfig.luisServices.get(this.solutionName);
if (luisService === undefined) {
throw new Error('The specified LUIS Model could not be found in your Bot Services configuration.');
} else {
let turnResult: DialogTurnResult = Dialog.EndOfTurn;
const result: RecognizerResult = await luisService.recognize(dc.context);
const intent: string = LuisRecognizer.topIntent(result);
switch (intent) {
case 'claims_claimStatus': {
turnResult = await dc.beginDialog(StatusDialog.name);
break;
}
.
.
.
Expected results: When using the dispatch bot to route to the skill bot, after beginning a waterfall dialog within the claims_claimStatus dialog, the dialogContext.activeDialog should be statusDialog for further steps of the waterfall dialog
Actual results: When using the dispatch bot to route to the skill bot, after beginning a waterfall dialog within the claims_claimStatus dialog, the dialogContext.activeDialog should is undefined for further steps of the waterfall dialog
Related
I'm working on bot framework virtual assistant template. For getting QnA responses, bot uses following method
protected virtual QnAMakerDialog TryCreateQnADialog(string knowledgebaseId, CognitiveModelSet cognitiveModels, Metadata[] metadata)
{
if (!cognitiveModels.QnAConfiguration.TryGetValue(knowledgebaseId, out QnAMakerEndpoint qnaEndpoint)
|| qnaEndpoint == null)
{
throw new Exception($"Could not find QnA Maker knowledge base configuration with id: {knowledgebaseId}.");
}
// QnAMaker dialog already present on the stack?
if (Dialogs.Find(knowledgebaseId) == null)
{
return new QnAMakerDialog(
//dialogId: knowledgebaseId,
knowledgeBaseId: qnaEndpoint.KnowledgeBaseId,
endpointKey: qnaEndpoint.EndpointKey,
hostName: qnaEndpoint.Host,
strictFilters: metadata,
noAnswer: _templateManager.GenerateActivityForLocale("UnsupportedMessage"),
activeLearningCardTitle: _templateManager.GenerateActivityForLocale("QnaMakerAdaptiveLearningCardTitle").Text,
cardNoMatchText: _templateManager.GenerateActivityForLocale("QnaMakerNoMatchText").Text)
{
Id = knowledgebaseId,
LogPersonalInformation = true
};
}
else
{
return null;
}
}
var qnaDialog = TryCreateQnADialog(knowledgebaseId, localizedServices, metadata);
if (qnaDialog != null)
{
Dialogs.Add(qnaDialog);
}
return await stepContext.BeginDialogAsync(knowledgebaseId, cancellationToken: cancellationToken);
I want to check the result text in the QnA response and perform a set of operations before displaying the QnA response in chat window. Where can I get the response value for this QnA dialog ?
I have already gone through this question - how to get the qna response inside QnAMakerDialogbase: QnAMakerDialog, but I did not understand how can I override DisplayQnAResultAsync.
My bot right now is using local memory, the goal is whenever a conversation end. I want to delete everything from local memory about this user. So I tried this onEndOfConversation.
Apparently this error shows up saying that onEndOfConversation is not a function.
This is my code :
const { CardFactory } = require('botbuilder');
const { DialogBot } = require('./dialogBot');
const WelcomeCard = require('./resources/welcomeCard.json');
class DialogAndWelcomeBot extends DialogBot {
constructor(conversationState, userState, dialog) {
super(conversationState, userState, dialog);
this.onMembersAdded(async (context, next) => {
const membersAdded = context.activity.membersAdded;
for (let cnt = 0; cnt < membersAdded.length; cnt++) {
if (membersAdded[cnt].id !== context.activity.recipient.id) {
//const welcomeCard = CardFactory.adaptiveCard(WelcomeCard);
//await context.sendActivity({ attachments: [welcomeCard] });
await dialog.run(context, conversationState.createProperty('DialogState'));
}
}
// By calling next() you ensure that the next BotHandler is run.
await next();
});
this.onEndOfConversation(async (context, next) => {
console.log("END!");
await conversationState.delete(context);
await userState.delete(context);
});
}
}
module.exports.DialogAndWelcomeBot = DialogAndWelcomeBot;
So how should I do this? If onEndOfConversation isn't recognize, what alternatives I can do to clear user and conversation from the memory after a dialogue ends.
The endOfConversation activity handler is used internally when a bot is also coupled with a skill. When a conversation is ended by the user, the bot then sends this activity type to the skill notifying it that the conversation has ended with the user.
There are different ways you could attack this. The method I use is component dialogs. Modeled after the cancelAndHelpDialog design, when a user types "cancel" or "quit", the user is brought to an exit dialog where feedback is gathered, etc.
As part of the exiting process, you could call conversationState.delete() within the dialog followed by cancelAllDialogs(true).
Hope of help!
I have a SMS / Twilio Channel that I'm using to send out a Proactive message to the user. To send the Proactive message I'm calling a API method passing in MySpecialId which is used later in the conversation.
I want to save this MySpecialId into the conversation but at the point I have the MySpecialId the conversation doesn't exist yet, and I don't have a turnContext, so I can't really save it yet.
Is there a way to pass this Id from my API method into the BotCallback? I created a quick example. (Here is the original example I'm using https://github.com/microsoft/BotBuilder-Samples/tree/master/samples/csharp_dotnetcore/16.proactive-messages)
Thanks
[HttpGet("{number}")]
public async Task<IActionResult> Get(string MySpecialId)
{
//For Twillio Channel
MicrosoftAppCredentials.TrustServiceUrl("https://sms.botframework.com/");
var NewConversation = new ConversationReference
{
User = new ChannelAccount { Id = $"+1{PHONENUMBERTOTEXTHERE}" },
Bot = new ChannelAccount { Id = "+1MYPHONENUMBERHERE" },
Conversation = new ConversationAccount { Id = $"+1{PHONENUMBERTOTEXTHERE}" },
ChannelId = "sms",
ServiceUrl = "https://sms.botframework.com/"
};
BotAdapter ba = (BotAdapter)_HttpAdapter;
await ((BotAdapter)_HttpAdapter).ContinueConversationAsync(_AppId, NewConversation, BotCallback, default(CancellationToken));
return new ContentResult()
{
Content = "<html><body><h1>Proactive messages have been sent.</h1></body></html>",
ContentType = "text/html",
StatusCode = (int)HttpStatusCode.OK,
};
}
private async Task BotCallback(ITurnContext turnContext, CancellationToken cancellationToken)
{
try
{
var MyConversationState = _ConversationState.CreateProperty<MyConversationData>(nameof(MyConversationData));
var ConversationState = await MyConversationState.GetAsync(turnContext, () => new MyConversationData(), cancellationToken);
//********************************************************************************************************************
ConversationState.MySpecialId = //HOW DO I GET MySpecialId FROM THE GET METHOD ABOVE HERE?
//********************************************************************************************************************
await _ConversationState.SaveChangesAsync((turnContext, false, cancellationToken);
await turnContext.SendActivityAsync("Starting proactive message bot call back");
}
catch (Exception ex)
{
this._Logger.LogError(ex.Message);
}
}
I don't believe that you can. Normally, you would do something like this by passing values through Activity.ChannelData, but ChannelData doesn't exist on ConversationReference.
Per comments below, #Ryan pointed out that you can pass data on ConversationAccount.Properties. Note, however, this is currently only available in the C# SDK. I've opened an issue to bring this into the Node SDK, but ETA is unknown at this point.
Instead, I'd suggest using a something more native to C#, like:
Create a ConcurrentDictionary
private ConcurrentDictionary<string, string> _idMap;
Map MySpecialId to Conversation.Id (in your Get function)
_idMap.AddOrUpdate(conversationReference.Conversation.Id, MySpecialId, (key, newValue) => MySpecialId);
Access the MySpecialId from the Activity.Conversation.Id (in BotCallback)
var ConversationState = await MyConversationState.GetAsync(turnContext, () => new MyConversationData(), cancellationToken);
ConversationState.MySpecialId = _idMap.GetValueOrDefault(turnContext.Activity.Conversation.Id);
Save ConversationState
await ConversationState.SaveChangesAsync(turnContext, false, cancellationToken);
There's other ways you could do this and some validation checks you'll need to add, but this should get you started.
I'm trying to send an event activity from the bot to the client when I'm in a certain dialog, but I just can't get it to work... Any suggestions or code samples I can look at?
Also I already tried to make a back channel and it works but on the bot side as far as I could tell it only works in the message controller.
EDIT*
I'm sorry, for not providing any details, I was in a hurry last week.
So i'm making a bot that will fill a "report" for a user, the bot asks questions, and the user gives the answers. For the first question i have to call a function in my angular app when i'm in the first dialog, that is after the root dialog, that will check the user input and return an "Account" object to the bot if the account exists, if it doesn't then it returns null... (and i know it would be easier to just make an API and connect directly to the bot, but i have to use angular)
I'm using the back channel like so:
botConnection.activity$
.filter( activity => {
// tslint:disable-next-line:max-line-length
return (activity.type === 'message' && activity.from['id'] === 'VisitReportV3' && activity.text === 'Please select an account...') ||
(activity.type === 'message' && activity.from['id'] === 'user' && this.accountFlag)
})
.subscribe(activity => {
if (activity.from['id'] === 'VisitReportV3' && activity.text === 'Please select an account...') {
console.log('"account" received');
this.accountFlag = true;
postAccountInfo();
} else if (activity.from['id'] === 'user' && this.accountFlag) {
console.log('"account" flag recieved');
this.accountFlag = false;
}
});
It's probably completely wrong but i didn't know how else to do it..
tl;dr:
So in short, i need to check if i'm in the first dialog (or the one that asks me for the account) and if I am then wait for the next user input and call the angular function to check the input and see if there are any accounts that match, if not return null if they do return the serialized object to the bot for some more processing... I hope this explains some more.
In your messages controller you are probably only forwarding activities with a type of Message to your dialogs. If you used the default bot template, you probably have something like
public async Task<HttpResponseMessage> Post([FromBody]Activity activity)
{
if (activity.GetActivityType() == ActivityTypes.Message)
{
await Conversation.SendAsync(activity, () => new Dialogs.RootDialog());
}
else
{
HandleSystemMessage(activity);
}
var response = Request.CreateResponse(HttpStatusCode.OK);
return response;
}
that you need to change to
public async Task<HttpResponseMessage> Post([FromBody]Activity activity)
{
if (activity.GetActivityType() == ActivityTypes.Message || activity.GetActivityType() == ActivityTypes.Event)
{
await Conversation.SendAsync(activity, () => new Dialogs.RootDialog());
}
else
{
HandleSystemMessage(activity);
}
var response = Request.CreateResponse(HttpStatusCode.OK);
return response;
}
That will send both messages and events to your dialogs. Keep in mind though that you need to check for both of them in each of you dialog methods now especially when you are expecting events as the user can type and send a message before your event gets sent.
EDIT
You can't get the dialog name from activity.from['id']. However, in your dialog you can create an Activity reply and set the name to be something else. For example in your dialog,
private async Task MessageReceivedAsync(IDialogContext context, IAwaitable<object> result)
{
Activity activity = await result as Activity;
Activity reply = activity.CreateReply();
reply.Text = "Please select an account...";
reply.Name = "VisitReportV3";
await context.PostAsync(reply);
}
then in your back channel JavaScript you would change from checking if activity.from['id'] === "VisitReportV3" to activity.name === "VisitReportV3".
How to integrate FormFlow and QnA dialogs in a simple bot. I'm not able to call FormFlow context once QnA is completed. If there are any samples for the same, then please share.
If you want to use QnA and FormFlow, create a dialog QnADialog and you can send all your messages first to the root dialog from there you can call your QnA Dialog like
var qnadialog = new QnADialog();
var messageToForward = await message;
await context.Forward(qnadialog, ResumeAfterQnA, messageToForward, CancellationToken.None);
Once th QnADilalog is executed, it will call the ResumeAfterQnA and there you can call your FormFlow Dialog.
private async Task ResumeAfterQnA(IDialogContext context, IAwaitable<object> results)
{
SampleForm form = new SampleForm();
var sampleForm = new FormDialog<SampleForm>(form, SampleForm.BuildForm, FormOptions.PromptInStart);
context.Call(sampleForm, RootDialog.SampleFormSubmitted);
}
You need to have a SampleFormSubmitted method that will be called after you form is submitted.
private async Task SampleFormSubmitted(IDialogContext context, IAwaitable<SampleForm> result)
{
try
{
var query = await result;
context.Done(true);
}
catch (FormCanceledException<SampleForm> e)
{
string reply;
if (e.InnerException == null)
{
reply = $"You quit. Maybe you can fill some other time.";
}
else
{
reply = $"Something went wrong. Please try again.";
}
context.Done(true);
await context.PostAsync(reply);
}
}
One approach is to start of from a Luis template.
Then make a specific Intent to start the Form.
Then you can have an empty Luis Intent of ”” and even ”None” and you put your QnA there.
That way the Qna will be on the background LUIS will give you great flexibility to trigger a specific dialogue with intents
Here is an example
http://www.garypretty.co.uk/2017/03/26/forwarding-activities-messages-to-other-dialogs-in-microsoft-bot-framework/