How to call Dialog Class From a LUIS Intent - botframework

I am using Bot Framework V4 dispatch model to invoke LUIS and QnA services.
I can call the code inside of the top scoring intents class.
However, I couldn't find a way to call an external dialog from it. How can I do that?
// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License.
namespace Microsoft.BotBuilderSamples
{
public class DispatchBot : ActivityHandler
{
private ILogger<DispatchBot> _logger;
private IBotServices _botServices;
public DispatchBot(IBotServices botServices, ILogger<DispatchBot> logger)
{
_logger = logger;
_botServices = botServices;
}
protected override async Task OnMessageActivityAsync(ITurnContext<IMessageActivity> turnContext, CancellationToken cancellationToken)
{
// First, we use the dispatch model to determine which cognitive service (LUIS or QnA) to use.
var recognizerResult = await _botServices.Dispatch.RecognizeAsync(turnContext, cancellationToken);
// Top intent tell us which cognitive service to use.
var topIntent = recognizerResult.GetTopScoringIntent();
// Next, we call the dispatcher with the top intent.
await DispatchToTopIntentAsync(turnContext, topIntent.intent, recognizerResult, cancellationToken);
}
protected override async Task OnMembersAddedAsync(IList<ChannelAccount> membersAdded, ITurnContext<IConversationUpdateActivity> turnContext, CancellationToken cancellationToken)
{
const string WelcomeText = "Type a greeting, or a question about the weather to get started.";
foreach (var member in membersAdded)
{
if (member.Id != turnContext.Activity.Recipient.Id)
{
await turnContext.SendActivityAsync(MessageFactory.Text($"Welcome to Dispatch bot {member.Name}. {WelcomeText}"), cancellationToken);
}
}
}
private async Task DispatchToTopIntentAsync(ITurnContext<IMessageActivity> turnContext, string intent, RecognizerResult recognizerResult, CancellationToken cancellationToken)
{
switch (intent)
{
case "l_HomeAutomation":
//WANT TO CALL THE EXTERNAL DIALOG CLASS FROM HERE
//await ProcessHomeAutomationAsync(turnContext, recognizerResult.Properties["luisResult"] as LuisResult, cancellationToken);
break;
case "l_Weather":
await ProcessWeatherAsync(turnContext, recognizerResult.Properties["luisResult"] as LuisResult, cancellationToken);
break;
case "q_sample-qna":
await ProcessSampleQnAAsync(turnContext, cancellationToken);
break;
default:
_logger.LogInformation($"Dispatch unrecognized intent: {intent}.");
await turnContext.SendActivityAsync(MessageFactory.Text($"Dispatch unrecognized intent: {intent}."), cancellationToken);
break;
}
}
private async Task ProcessHomeAutomationAsync(ITurnContext<IMessageActivity> turnContext, LuisResult luisResult, CancellationToken cancellationToken)
{
_logger.LogInformation("ProcessHomeAutomationAsync");
// Retrieve LUIS result for Process Automation.
var result = luisResult.ConnectedServiceResult;
var topIntent = result.TopScoringIntent.Intent;
await turnContext.SendActivityAsync(MessageFactory.Text($"HomeAutomation top intent {topIntent}."), cancellationToken);
await turnContext.SendActivityAsync(MessageFactory.Text($"HomeAutomation intents detected:\n\n{string.Join("\n\n", result.Intents.Select(i => i.Intent))}"), cancellationToken);
if (luisResult.Entities.Count > 0)
{
await turnContext.SendActivityAsync(MessageFactory.Text($"HomeAutomation entities were found in the message:\n\n{string.Join("\n\n", result.Entities.Select(i => i.Entity))}"), cancellationToken);
}
}
private async Task ProcessWeatherAsync(ITurnContext<IMessageActivity> turnContext, LuisResult luisResult, CancellationToken cancellationToken)
{
_logger.LogInformation("ProcessWeatherAsync");
// Retrieve LUIS results for Weather.
var result = luisResult.ConnectedServiceResult;
var topIntent = result.TopScoringIntent.Intent;
await turnContext.SendActivityAsync(MessageFactory.Text($"ProcessWeather top intent {topIntent}."), cancellationToken);
await turnContext.SendActivityAsync(MessageFactory.Text($"ProcessWeather Intents detected::\n\n{string.Join("\n\n", result.Intents.Select(i => i.Intent))}"), cancellationToken);
if (luisResult.Entities.Count > 0)
{
await turnContext.SendActivityAsync(MessageFactory.Text($"ProcessWeather entities were found in the message:\n\n{string.Join("\n\n", result.Entities.Select(i => i.Entity))}"), cancellationToken);
}
}
private async Task ProcessSampleQnAAsync(ITurnContext<IMessageActivity> turnContext, CancellationToken cancellationToken)
{
_logger.LogInformation("ProcessSampleQnAAsync");
var results = await _botServices.SampleQnA.GetAnswersAsync(turnContext);
if (results.Any())
{
await turnContext.SendActivityAsync(MessageFactory.Text(results.First().Answer), cancellationToken);
}
else
{
await turnContext.SendActivityAsync(MessageFactory.Text("Sorry, could not find an answer in the Q and A system."), cancellationToken);
}
}
}
}

You'll need to initialize instances of the required dialogs in your DispatchBot, and then invoke them like this:
await this.weatherDialog.RunAsync(turnContext, dialogStateAccessor, cancellationToken)
Here is some sample code. It's slightly different from your design since I am delegating intent detection to a parent dialog and then do the logic branching from there (to keep the bot class logic simple), but how to invoke dialogs is the same.
Update based on discussion below:
If you are using multi turn dialogs, I would suggest you follow the pattern in my code. Otherwise you'll have to deal dialog continue by yourself and good luck with it.

Related

I am using Bot framework V4.3, I want to retrieve adaptive card submit values

I'm using Bot framework V4.3, I have been using adaptive card in waterfall dialog, to get user information, I would want to get values once user clicks submit button and also I would like to go back to previous step if user click back button.
Here is how my adaptive card looks like
I have tried the solution given by #mdrichardson in Stack Overflow
But the adaptive card re-prompts again.
And the below code help us to go back to previous step but how to implement it to back button of adaptive card.
stepContext.ActiveDialog.State["stepIndex"] =(int)stepContext.ActiveDialog.State["stepIndex"] - 2;
Adding adaptive card to dialog. I had even used TextPrompt instead of ChoicePrompt
AddDialog(new ChoicePrompt("AdaptiveCardPrompt") { Style = ListStyle.None });
This is how I'm displaying adaptive card. My adaptive card is in Json format
cardAttachment = CreateAdaptiveCardAttachment();
return await stepContext.PromptAsync("AdaptiveCardPrompt",
new PromptOptions
{
Prompt = (Activity)MessageFactory.Attachment(new Attachment
{
ContentType = AdaptiveCard.ContentType,
Content = cardAttachment.Content
}),
}, cancellationToken);
Kindly help me in solving this issue. Thank you in advance
Edit from Botframework Support: Please do not use the code block below. It only works in Emulator. Instead, use:
if (string.IsNullOrWhiteSpace(activity.Text) && activity.Value != null)
{
activity.Text = JsonConvert.SerializeObject(activity.Value);
}
Edit 1: #mdrichardson Here is how I have setup the dialog call
public static async Task Run(this Dialog dialog, ITurnContext turnContext,IStatePropertyAccessor<DialogState> accessor, CancellationToken cancellationToken = default(CancellationToken))
{
var dialogSet = new DialogSet(accessor);
dialogSet.Add(dialog);
var dialogContext = await dialogSet.CreateContextAsync(turnContext, cancellationToken);
// Ensure that message is a postBack (like a submission from Adaptive Cards)
if (dialogContext.Context.Activity.GetType().GetProperty("ChannelData") != null)
{
var channelData = JObject.Parse(dialogContext.Context.Activity.ChannelData.ToString());
if (channelData.ContainsKey("postBack"))
{
var postbackActivity = dialogContext.Context.Activity;
// Convert the user's Adaptive Card input into the input of a Text Prompt
// Must be sent as a string
postbackActivity.Text = postbackActivity.Value.ToString();
await dialogContext.Context.SendActivityAsync(postbackActivity);
}
}
var results = await dialogContext.ContinueDialogAsync(cancellationToken);
if (results.Status == DialogTurnStatus.Empty)
{
await dialogContext.BeginDialogAsync(dialog.Id, null, cancellationToken);
}
}
And in OnTurnAsync method
if (turnContext.Activity.Type == ActivityTypes.Message)
{
await Dialog.Run(turnContext, ConversationState.CreateProperty<DialogState>(nameof(DialogState)), cancellationToken);
}
Edit 2 : I modified the code and I was able to go to next waterfall step. But I'm facing another issue here.
Next prompt is not getting displayed but I can see it in Log
This is how it shows in Emulator
Emulator View
Once user clicks the button control lands in MoreInfoAsync method
private async Task<DialogTurnResult> MoreInfoAsync(WaterfallStepContext stepContext, CancellationToken cancellationToken)
{
var goback = JObject.Parse(stepContext.Result.ToString());
stepContext.Values["AdaptiveCardDetails"] = stepContext.Result.ToString();
if (goback.ContainsKey("goBack"))
{
return await stepContext.ReplaceDialogAsync(InitialDialogId);
}
// stepContext.ActiveDialog.State["stepIndex"] = (int)stepContext.ActiveDialog.State["stepIndex"] - 2;
else
return await stepContext.PromptAsync("MoreInfo", new PromptOptions { Prompt = MessageFactory.Text("Tell Me more.") }, cancellationToken);
}
I would like to go to initial dialog so I'm using ReplaceDialogAsync.
MoreInfo dialog is not displayed in emulator but its shown in log
Edit 3: Here is the complete code of waterfall steps
// This array defines how the Waterfall will execute.
var waterfallSteps = new WaterfallStep[]
{
ChoiceAsync,
CardAsync,
MoreInfoAsync,
ConfirmAsync
};
AddDialog(new WaterfallDialog(nameof(WaterfallDialog), waterfallSteps));
AddDialog(new ChoicePrompt("ChoiceType"));
AddDialog(new TextPrompt("AdaptiveCardPrompt"));
AddDialog(new TextPrompt("MoreInfo"));
InitialDialogId = nameof(WaterfallDialog);
private async Task<DialogTurnResult> ChoiceAsync(WaterfallStepContext stepContext, CancellationToken cancellationToken)
{
options = new PromptOptions()
{
Prompt = MessageFactory.Text("Select the Choice"),
RetryPrompt = MessageFactory.Text("That was not a valid choice."),
Choices = GetChoices(),
Style = ListStyle.HeroCard
};
return await stepContext.PromptAsync("ChoiceType", options, cancellationToken);
}
private async Task<DialogTurnResult> CardAsync(WaterfallStepContext stepContext, CancellationToken cancellationToken)
{
var cardAttachment = new Attachment();
stepContext.Values["leaveType"] = stepContext.Result.ToString();
cardAttachment = CreateAdaptiveCardAttachment();
return await stepContext.PromptAsync("AdaptiveCardPrompt",
new PromptOptions
{
Prompt = (Activity)MessageFactory.Attachment(new Attachment
{
ContentType = AdaptiveCard.ContentType,
Content = cardAttachment.Content,
}),
}, cancellationToken);
}
private async Task<DialogTurnResult> MoreInfoAsync(WaterfallStepContext stepContext, CancellationToken cancellationToken)
{
var goback = JObject.Parse(stepContext.Result.ToString());
stepContext.Values["AdaptiveCardDetails"] = stepContext.Result.ToString();
if (goback.ContainsKey("goBack"))
{
return await stepContext.ReplaceDialogAsync(InitialDialogId);
}
else return await stepContext.PromptAsync("MoreInfo", new PromptOptions { Prompt = MessageFactory.Text("Tell Me more.") }, cancellationToken);
}
private async Task<DialogTurnResult> ConfirmAsync(WaterfallStepContext stepContext, CancellationToken cancellationToken)
{
stepContext.Values["MoreInfo"] = stepContext.Result;
//As of now I wouldn't perform any task here so I'll end
return await stepContext.EndDialogAsync();
}
Dealing with the Re-Prompt
The issue is with your OnTurnAsync() method:
if (turnContext.Activity.Type == ActivityTypes.Message)
{
await Dialog.Run(turnContext, ConversationState.CreateProperty<DialogState>(nameof(DialogState)), cancellationToken);
}
Every time a user sends a message, it causes a new instance of your dialog to be run. Since Adaptive Card Input gets sent as a PostBack message (which is still a message), it causes the Dialog to run again, re-prompting the user.
If you're going to run dialogs from OnTurnAsync() or OnMessageAsync(), there's a couple of different things you should do, either:
Use if/switch statements. For example, if the message contains "help", run the HelpDialog, or
Start a dialog that saves user responses and skips steps as necessary. You can see an example of this in Core Bot's Booking Dialog. Notice how it's saving the user response in each step with something like bookingDetails.TravelDate = (string)stepContext.Result; and checks to see if it exists in the previous step before prompting with something like if (bookingDetails.TravelDate == null). For yours, you might store something like userProfile.AdaptiveCardDetails or something.
Back Button
To get the back button working, let's say it looks like this in your Adaptive Card:
{
"type": "Action.Submit",
"title": "Back",
"data": {
"goBack": "true",
}
},
When the user clicks "Back", the bot will receive an activity with:
Since the user wants to go back and you don't need the data, you could do something like:
var activity = turnContext.Activity;
if (string.IsNullOrWhiteSpace(activity.Text) && activity.Value.GetType().GetProperty("goBack"))
{
dc.Context.Activity.Text = "Back";
}
and then in your Dialog step:
if (stepContext.Result == "Back")
{
stepContext.ActiveDialog.State["stepIndex"] = (int)stepContext.ActiveDialog.State["stepIndex"] - 2;
}

Next step after PromptAsync is not called

I have this simple dialog, with 2 simple waterfall steps.
The user sees "How may I help you today?" and when it answers, nothing happens. I can't get Validate to work.
Am I missing something? I'm using SDK 4.1.5.
public ComplaintsDialog() : base(nameof(ComplaintsDialog))
{
var steps = new WaterfallStep[]
{
Ask,
Validate
};
AddDialog(new WaterfallDialog("flow", steps));
AddDialog(new TextPrompt("asking"));
}
private static async Task<DialogTurnResult> Ask(WaterfallStepContext sc, CancellationToken cancellationToken)
{
return await sc.PromptAsync("asking", new PromptOptions { Prompt = new Activity { Text = "How may I help you today?", Type= ActivityTypes.Message} }, cancellationToken);
}
private static async Task<DialogTurnResult> Validate(WaterfallStepContext sc, CancellationToken cancellationToken)
{
var answer = sc.Result;
await sc.Context.SendActivityAsync(answer.ToString());
return await sc.EndDialogAsync();
}
}
UPDATE
I tried to simplify the code, and this is how I currently call ComplaintsDialog directly from the main bot.
It looks like the stack is always empty when it gets to await dc.ContinueDialogAsync();, so it's going into a loop and start ComplaintsDialog over and over again
public async Task OnTurnAsync(ITurnContext turnContext, CancellationToken cancellationToken)
{
// Create dialog context.
var dc = await _dialogs.CreateContextAsync(turnContext);
switch (turnContext.Activity.Type)
{
case ActivityTypes.Message:
// Continue outstanding dialogs.
await dc.ContinueDialogAsync();
// Begin main dialog if no outstanding dialogs/ no one responded.
if (!dc.Context.Responded)
{
await dc.BeginDialogAsync(nameof(ComplaintsDialog));
}
break;
case ActivityTypes.ConversationUpdate:
if (dc.Context.Activity.MembersAdded != null && dc.Context.Activity.MembersAdded.Any())
{
foreach (var newMember in dc.Context.Activity.MembersAdded)
{
if (newMember.Id != dc.Context.Activity.Recipient.Id)
{
await dc.BeginDialogAsync(nameof(WelcomeDialog));
}
}
}
break;
}
}
The code example you provided looks like it should be working so the problem is probably elsewhere.
My guess is that you are starting the ComplaintsDialog inside of a WaterfallStep (from another dialog) so make sure that you are calling the BeginDialogAsync method like this:
return await stepContext.BeginDialogAsync(nameof(ComplaintsDialog));
instead of:
await stepContext.BeginDialogAsync(nameof(ComplaintsDialog));
If this is not the error probably more information is necessary
Update
Your problem is on the OnTurnAsync method. You're not saving the new turn into the conversation state. The Message case on your switch should look like this:
case ActivityTypes.Message:
if (dc.ActiveDialog == null)
{
await dc.BeginDialogAsync(nameof(ComplaintsDialog), cancellationToken);
}
else
{
await dc.ContinueDialogAsync(cancellationToken);
}
await _accessors.ConversationState.SaveChangesAsync(turnContext);
break;
And your constructor:
private readonly MyBotAccessors _accessors;
public MyBot(MyBotAccessors accessors, ILoggerFactory loggerFactory)
{
...
_accessors = accessors ?? throw new System.ArgumentNullException(nameof(accessors));
...
}
SaveChangesAsync documentation

Return values from proactive dialog

My scenario is something like this, there are 2 users -> U1 and U2.
U1 wants the bot to ping U2 and get some information back
This is where i use proactive dialog based messages.
I am currently using the v3 Bot Builder SDK sample
using (var scope = DialogModule.BeginLifetimeScope(Microsoft.Bot.Builder.Dialogs.Conversation.Container, message))
{
var botData = scope.Resolve<IBotData>();
await botData.LoadAsync(CancellationToken.None);
//This is our dialog stack
var task = scope.Resolve<IDialogTask>();
// Interrupt the stack. This means that we're stopping whatever conversation that is currently happening with the user
// Then adding this stack to run and once it's finished, we will be back to the original conversation
var dialog = new GetFeedbackDialog(this.UserData);
task.Call(dialog.Void<object, IMessageActivity>(), null);
await task.PollAsync(CancellationToken.None);
// Flush the dialog stack back to its state store.
await botData.FlushAsync(CancellationToken.None);
}
class GetFeedbackDialog : IDialog<object>
{
private RequestFeedbackData userData;
public GetFeedbackDialog()
{
}
public GetFeedbackDialog(RequestFeedbackData userData)
{
this.userData = userData;
}
public async Task StartAsync(IDialogContext context)
{
await context.PostAsync($"Hello, Please provide feedback for "+ userData.OrganizerName+" on "+userData.FeedbackSubject);
context.Wait(this.MessageReceivedAsync);
}
public virtual async Task MessageReceivedAsync(IDialogContext context, IAwaitable<IMessageActivity> result)
{
var message = await result;
if (message.Text.ToLower().Equals("quit"))
{
await context.PostAsync("Quitting.. Thank you for using FeedbackBot!");
context.Done(String.Empty); // Finish this dialog.
}
await context.PostAsync("Thank you for your feedback!");
context.Done(message.Text);
}
}
How do can i access the response typed by U2 in GetFeedbackDialog from where i initiated the proactive message ?
Thanks in advance for your help

A very long message for LUIS dialog resets dialog state

Using bot framework emulator v.3.5.36, if a user sends long text (about 1K characters), emulator silently resets dialog stack back to root dialog, without any errors or warnings. (see the screenshot below.)
Is there a declared message limit for bot framework?
Is there a way for bot to handle such situations and warn user instead of this silent something?
There is nothing really specific about the code at all:
[LuisModel("{GUID}", "{CODE}", LuisApiVersion.V2, domain: "westeurope.api.cognitive.microsoft.com", threshold: 0.5)]
[Serializable]
public class LuisSearchDialog2 : LuisDialog<object>
{
[LuisIntent("")]
[LuisIntent("None")]
public async Task None(IDialogContext context, LuisResult result)
{
await context.PostAsync(JsonConvert.SerializeObject(result));
context.Wait(this.MessageReceived);
}
}
A simple approach would be to check the length of your message in the MessageController and decide whether you want to process it or not.
public async Task<HttpResponseMessage> Post([FromBody]Activity activity)
{
if (activity.Type == ActivityTypes.Message)
{
MicrosoftAppCredentials.TrustServiceUrl(activity.ServiceUrl);
var connector = new ConnectorClient(new Uri(activity.ServiceUrl));
if (activity.Text != null && activity.Text.Length > 200)
{
var errorReply = activity.CreateReply();
errorReply.Text = "Well well, that is too much of data. How about keeping it simple? How can I help you?";
await connector.Conversations.ReplyToActivityAsync(errorReply);
}
else
{
await Conversation.SendAsync(activity, () => new Dialogs.RootDialog());
}
}
}
The reason is that base LuisDialog doens't handle failed API requests (in case if query is too long, it returns 414 code). So the simplest way to handle such errors is to override MessageReceived as follows:
[Serializable]
public class LuisSearchDialog2 : LuisDialog<object>
{
protected override async Task MessageReceived(IDialogContext context, IAwaitable<IMessageActivity> activity)
{
try
{
await base.MessageReceived(context, activity);
}
catch(HttpRequestException e)
{
// Handle error here
//await context.PostAsync("Error: " + e.ToString());
context.Wait(this.MessageReceived);
}
}
}

ResumeAfter method is already called without calling context.done in the next dialog

I have implemented a structure where a QnA dialog is first started. If the QnA Dialog cannot solve the problem then it starts a Luis Dialog which has some main functionalities defined. Based on those main functionalities I start specific dialogs that can solve the problem.
My problem is that when I try to start LuisDialog from QnAMaker, it starts another LuisDialog to for conversation, That dialog doesn't stop on with wait method and automatically calls ResumeAfter method immediately after executing.
QnADialog:
protected override async Task RespondFromQnAMakerResultAsync(IDialogContext context, IMessageActivity message, QnAMakerResults results)
{
if (results == null || results.Answers.Count==0 || !IsConfidentAnswer(results) || results.Answers.FirstOrDefault().Score<0.75) {
await context.Forward(new MainLuisDialog(), MessageReceived, context.Activity.AsMessageActivity(), CancellationToken.None);
}
}
First Luis Dialog:
[LuisIntent(ErrorFileLink)]
public async Task ErrorFileLinkIntentHandler(IDialogContext context, LuisResult result) {
await context.Forward(new ErrorFileLinkDialog(), CallBackHandler, context.Activity.AsMessageActivity(), CancellationToken.None);
}
private async Task CallBackHandler(IDialogContext context, IAwaitable<object> result)
{
try {
var returnedResult = await result;
if (returnedResult as string == "done")
context.Done(false);
}
catch (Exception e) {
}
}
2nd Luis Dialog:
[LuisIntent(MainAppIntent)]
public async Task MainAppIntentHandler(IDialogContext context, LuisResult result)
{
if(context.GetPrivateConversationData<SyncIssueStates>(CurrentDialogState) == SyncIssueStates.ExpectingSyncCompleteMessage)
{
await context.PostAsync(Utility.GetResourceString("SYNC_ISSUE_PLEASE_WAIT_SYNC_COMPELTE"));
context.Wait(MessageReceived);
return;
}
await context.PostAsync(Utility.GetResourceString("SYNC_ISSUE_GET_ERROR_MESSAGE"));
context.SetPrivateConversationData(CurrentDialogState, SyncIssueStates.ExpectingErrorMessage);
context.Wait(MessageReceived);
}
CallBackHandler Method in First Luis Dialog is called right after Forward is executed.
I think this behavior is due to the fact that the QnAMakerDialog calls context.Done(true); inside the DefaultWaitNextMessageAsync method. ref: https://github.com/Microsoft/BotBuilder-CognitiveServices/blob/master/CSharp/Library/QnAMaker/QnAMaker/QnAMakerDialog.cs#L203
Try overriding the DefaultWaitNextMessageAsync method instead:
protected override async Task DefaultWaitNextMessageAsync(IDialogContext context, IMessageActivity message, QnAMakerResults results)
{
if (results == null || results.Answers.Count == 0 || !IsConfidentAnswer(results) || results.Answers.FirstOrDefault().Score < 0.75)
{
await context.Forward(new FirstDialog(), AfterForward, context.Activity.AsMessageActivity(), CancellationToken.None);
context.Wait(base.MessageReceivedAsync);
}
else
{
await base.DefaultWaitNextMessageAsync(context, message, results);
}
}

Resources