Teams Bot Adaptive Card action.Submit returns undefined but works in Bot Emulator - botframework

I am having an issue with Teams Bot adaptive card action submit, when testing Bot Emulator it works as expected, but when it's published and performing the same action in Teams conversation, the action submit returns undefined.
I have attempted with both adaptive card version 1.0, and version 1.3, the issue is the same in both cases.
Anyone know a solution for this?

The other answer provides some useful background, but I think it's only applicable to specific scenarios - I think the issue you're having here is that you're sending a raw string value in the data property ("My Action") which Teams doesn't like, but the emulator doesn't mind at all. You can see more about that described in this Microsoft blog post. You should instead be sending an actual object. I describe it in more detail in this answer: QnA Maker Bot AdaptiveCards: how to add Data object in C#.
What #billoverton is referring to is a specific use case where you want the button behaviour to be, for instance, actually putting a message into the text stream as a response, as well as sending it to your bot. There are various options for these specific use cases, as described here, but they are optional if you want this specific behaviour. If you're happy for the user to click the button and that to simply invoke the message to your bot, you don't need the "msteams" section in the data payload.

A "standard" action.submit won't work in Teams. You need to add an msteams object under the data attribute. I'm guessing you are using a standard definition like
{
"type": "ActionSet",
"actions": [
{
"type": "Action.Submit",
"title": "My Action",
"data": "My Action"
}
]
}
For Teams, it instead needs to look like this:
{
"type": "ActionSet",
"actions": [
{
"type": "Action.Submit",
"title": "My Action",
"data": {
"msteams": {
"type": "imBack",
"value": "My Action"
}
}
}
]
}
Of course when you do this, it won't work in the emulator or any non-teams channel. I've seen some people update their onMessage handler to account for this by extracting the value so a single card definition can work for both channels, but it made the selection a backchannel event for one of the channels (can't remember if it was web or Teams) which was not the experience I was looking for. So instead, I just conditionally display a Teams or non-Teams card based on channel. For example:
if (context.activity.channelId == 'msteams') {
var welcomeCard = CardHelper.GetMenuCardTeams();
} else {
var welcomeCard = CardHelper.GetMenuCard();
}
If you are not using a helper to generate your cards you can define them explicitly here, though I do recommend using a helper to keep things clean. This does mean that you need to maintain two versions of your cards, but for me it was worth it to ensure a consistent experience across channels.

Related

Starting a new conversation in a Teams channel via REST

As far as I know, there are two APIs that are required to interact with bots
Graph - Access to Microsoft Graph endpoints, which do not provide access to conversations in Teams.
Bot Framework - The endpoints in this API provide access to conversations.
Moving past the difficulty of finding bot-user IDs, the problem becomes the creation of a new conversation in a channel by the bot. This link says it should be possible, but I have not been successful after hours of attempts.
The service URL in the documentation does not match my requests. It says https://smba.trafficmanager.net/teams/, but I see https://smba.trafficmanager.net/amer/ and https://smba.trafficmanager.net/apis/ in other places. Which one is it?
The crux of the issue is the required fields in the JSON data. This link is meant to help with the creation of channel conversations, but it does not explain the data. In that section is the following comment:
Alternatively, you can use the REST API and issue a POST request to /conversations resource.
which leads to here. Unfortunately, that page does not explain how this works because the sample JSON data is incomplete and not targeted at channels. It appears to be related only to new group chats.
Ultimately, I am requesting an example JSON request to the POST /v3/conversations endpoint, which will create a new conversation in a Team (conversationType = channel, to be clear). This is how it is supposed to look for a new group chat, which does not work with a channel:
{
"bot": {
"id": "12345678",
"name": "bot's name"
},
"isGroup": false,
"members": [
{
"id": "1234abcd",
"name": "recipient's name"
}
],
"topicName": "News Alert"
}
What you're trying to do is called "Proactive Messaging" - basically, it's having your bot create the first message in a series, rather than being the 'recipient' in the usual case. You definitely will need a bot for this, but it sounds like you have one already so that's good. In essence, in a Teams context, you're never really starting a 'new' conversation with the bot, it's just 'continuing' an existing one, so you're wanting to post to the endpoint of that converation. I've covered (I hope well) in some other answers, so please refer here for more: Sending proactive messages from an outside process to organizational users via Teams chat bot . If you're still stuck, let me know and I'll try help further.
I managed to get a JSON object to create a conversation. I have been at this for days, by the way.
The following JSON object will create a new conversation in the General channel of a Team:
{
"activity": {
"type": "message",
"text": "got here"
},
"bot": {
"id": "{{botID}}",
"name": "{{botName}}"
},
"channelData":{
"teamsChannelId":"{{teamID}}",
"teamsTeamId":"{{teamID}}",
// The channel ID of the General channel is the team ID
// Use another channel ID here if you want the message to go elsewhere
"channel":{"id":"{{teamID}}"},
"team":{"id":"{{teamID}}"},
"tenant":{"id": "{{tenantID}}"}
},
"isGroup": true,
"tenantId": "{{tenantID}}"
}
Here is the way to reply:
POST {{urlBase}}/v3/conversations/{{conversationId}}/activities
{
"bot": {
"id": "{{botID}}",
"name": "{{botName}}"
},
"text": "reply test",
"type": "message"
}
where urlBase is something like https://smba.trafficmanager.net/amer and conversationId is something like 19:c25ae532345659b67313c54e1310fdb6#thread.tacv2;messageid=2456564642569 .
A ton of helpful information can be found here. That link is buried and completely obscure, but it has most of what you'll want for low-level calls.
Unfortunately, adding a title/subject to the conversation seems to be impossible, which completely breaks my use case. All that work yielded nothing for me because the easiest part is missing, but at least I figured it out. I hope this helps someone.

Laravel JSON-API, includes are not consistent when a model is included multiple times

We have stumbled across a bug that I can only assume is a bug with the JSON API code. This is for the old laravel JSON API (in my composer.json file its "cloudcreativity/laravel-json-api": "^2.0")
The issue is when a resource is included multiple times (in different ways), it is possible to not get all the include information you asked for.
In my example, I am dealing with timesheets. Timesheets belong to a user. They are also approved by a user. Those users are usually different users, but not always. If I want to include both, I would add include=user,approved-by, and this works great.
The front end also sometimes needs to know the employeeType of the user, so we instead use include=user.employee-type,approved-by, and again this works, we get the employee type info for the user. The issue arises when the user is the same as the approver. It appears that JSON-API gets the approved-by user (without the employee type include data), then when it tries to get the timesheet user, it sees that is has already grabbed that user, and just stops there.
The difference in the output is:
The include with the related in
"employeeType": {
"data": {
"type": "employee-types",
"id": "1"
},
"links": {
"self": "link url",
"related": "link url"
}
},
Vs the include without all the related info
"employeeType": {
"links": {
"self": "link url",
"related": "link url"
}
},
We have a work around, where we need to include the employee type of the user and a the approver, but that seems cumbersome and annoying.
I was wondering if anyone knows if there is any good fix for this? Or if this has been fixed in the more recent version (could be the kick in the pants we need to actually migrate to the most recent version of the library)

Get teamId in message extension handler

When developing a message extension for Microsoft Teams, is it possible to retrieve the ID of a team where the user is invoking the message extension command without first adding the bot to that team?
I can do this when the bot is added to the team manually based on TeamsInfo.getTeamDetails(), however, I don't really need (or want) to add the bot to the team for my goal. All I need is the channel ID (which is available from the context/conversation) and the ID of the underlying team. Retrieving the team details without the bot being added beforehand errors with "The bot is not part of the conversation roster".
Have a look at the ChannelData property on the Activity class, that should give what you need. You can read more about it here.
Here's an example of the underlying payload, for interest:
"channelData": { "eventType": "channelCreated", "tenant": { "id": "72f988bf-86f1-41af-91ab-2d7cd011db47" }, "channel": { "id": "19:693ecdb923ac4458a5c23661b505fc84#thread.skype", "name": "My New Channel" }, "team": { "id": "19:693ecdb923ac4458a5c23661b505fc84#thread.skype" } }
we had the same trouble with the Team documentation and APIs.
However for that specific case, we found a solution that may work for you. I will say is more a hack than a solution. But it worked on my use case. It will only work on messages with attachments.
When the context is received in a message, the message contains an attachments array. Each attachment object has a contentUrl. Inside that url is the mailNickname for the group. That mailNickname field represents a readeable unique id. The format is something like sites/{mailNickname}/General.
from there you can retrieve the field and use the Groups Graph API.
With a query like this one:
https://graph.microsoft.com/v1.0/groups?$filter=startswith(mailNickname, 'themailNicknameFromcontenturl')
You will get the group full information, including the aadGroupId
In general, is a nightmare to work with Teams documentation. Hope this hack helps you.

Setting a flag in LUIS for BING spell check

I've connected bing spell check to my LUIS app, but I need to change the mode flag from 'proof' to 'spell'. Is there a possible way to do so while keeping the process automated.
The bing spell check API uses the default mode of "Proof" if you have integrated the API from your application/bot directly, Changing the mode value to "spell" should set the search to spell.
If you are adding the spell check endpoint URL directly with the LUIS API each and every call must include the required information for spelling corrections to work correctly.
The endpoint URL needs the spellCheck parameter to "true" along with the value of bing-spell-check-subscription-key.
https://{region}.api.cognitive.microsoft.com/luis/v2.0/apps/{appID}?subscription-key={luisKey}&spellCheck=true&bing-spell-check-subscription-key={bingKey}&verbose=true&timezoneOffset=0&q={utterance}
The response from this endpoint for a misspelled word should suggest "alteredQuery" for the utterance passed. For example:
{
"query": "How far is the mountainn?",
"alteredQuery": "How far is the mountain?",
"topScoringIntent": {
"intent": "Concierge",
"score": 0.183866
},
"entities": []
}

Dispatch CLI not passing Entities from Luis App

When generating a Dispatch model using the CLI, it doesn't pass the Entities from the Luis app in reference. This drastically affects the accuracy of the dispatch app.
For example, for the utterance "My [iPhone] isn't working", iPhone is attached to an entity list name CellPhoneType. There are three items in the list iPhone, Samsung, Smartphone.
In the bot emulator, using the Dispatch, if I write "my iPhone isn't working", the dispatch model passes it to Luis, as it should. However, if I write "my smartphone isn't working", the dispatch tool sends it over to QnA Maker.
I checked the model, and the entities are not passed in reference. I also tested with simple entities, they do not work as well.
I have the most recent version of the CLI installed.
Is this normal, is this a bug? Is there a work around to fix this?
So a couple things to address here with how you've built your LUIS model and what to expect from dispatch. Skip down to 2.) if you're a user who's reading this post and already has entities working in child LUIS models beautifully. #AlexandreViegas, read point 1.) to help properly build your LUIS model to detect intent properly in dispatch.
1. Use a Simple Entity + Phrase List to take Advantage of LUIS's machine learning--not List Entity
Right now it seems like your choice of using a list entity is not the best way to go here, and not how it's intended to be used. Instead list entities are used for terms that might have multiple ways of referring to the same thing.
Examples of When You Would Want to Use a List Entity
For example, California, Cali, CA, and The Golden State are all terms that refer to the same thing (a state). You can create a "States" list entity, include all 50 U.S. states and their nicknames. Now since this is a closed, explicit list, there is no machine learning when you use a list entity--LUIS will only detect "States" list entity if there's an exact text match.
Another example of when you would want to use list entities would be say with "Departments" for a school. You could have "chemistry", "CHEM142", "chem", etc. all meant to refer to that specific department, and do so with the rest of the departments in the school.
Why you want to use a Simple Entity and add a Phrase List
You can refer to this other StackOverflow answer I wrote, regarding how to create a simple entity and boost the signal of the entity using a phrase list.
To not completely duplicate the answer given in the link above, in essence, you want to use a simple entity, so LUIS can properly predict terms as CellPhoneType entity, even though you did not explicitly include it in your model.
For example you could have a Phone intent with utterances labeling various words as CellPhoneIntententity.
When I go to the Test panel, I type in "sunflower" and "moonstone" as made up mobile phones (maybe some phone company in the future creates phones with these names as their models):
Above you can see LUIS correctly predicts Phone intent and correctly extracts sunflower and moonstone as CellPhoneType entities.
However if I enter in brand names of mobiles that don't exist in the English language--for example Blackberry's "Z3" or T-Mobile's "G2X", LUIS cannot detect this with our model as is right now. (See 2 most recent utterances).
Above you can see utterances "i'd like to order a z3" and "my g2x is broken" do not properly predict as Phone intent, nor do z3 or g2x get detected as CellPhoneType entity. This is where phrase lists come in. As specified in the docs, phrase lists are good for boosting the signal of what a cell phone type may look like, as well as adding proprietary or foreign words to your LUIS model, such as the "made-up" words of many cell phone models. Again, refer to the StackOverflow answer I linked to, if you need guidance on how to create a phrase list.
After adding different names of cell phone models to phrase list
2. Query the endpoint of the LUIS model that was created by dispatch directly
Clarification:
When you add a child LUIS model to dispatch, even if that child LUIS model has entities in it, it will not show up in the model of the parent LUIS model created by dispatch.
the exception to the above bullet would be if you labelled an entity in a pattern
Why entities do not need to be labelled in the parent LUIS model, is because when you call the endpoint of the parent LUIS model, it does sort of a shared call, under the hood, so it doesn't have to ping LUIS twice.
You see the entities labelled from the child LUIS model in the connectedServiceResult property
How to extract entities from child LUIS model, using your parent dispatch LUIS app
Make sure to publish both the child LUIS app and the parent dispatch app.
Going to your parent dispatch-created LUIS app, go to Manage > Keys and Endpoints > click "Endpoint" to open a browser tab where your can query the parent app in the URL after q=
type in your utterances in the URL, after q= to see the entities and intents extracted from the child LUIS model under connectedServiceResult
https://westus.api.cognitive.microsoft.com/luis/v2.0/apps/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx?verbose=true&timezoneOffset=-360&subscription-key=b7xxxxxxxxxxxxxxxxxxxxxxxxxxxx67&q=my%20iphone%20is%20broken
{
"query": "my iphone is broken",
"topScoringIntent": {
"intent": "l_Reminders",
"score": 0.99594605
},
"intents": [
{
"intent": "l_Reminders",
"score": 0.99594605
},
{
"intent": "None",
"score": 0.002990469
}
],
"entities": [],
"connectedServiceResult": {
"query": "my iphone is broken",
"topScoringIntent": {
"intent": "Phone",
"score": 0.9658808
},
"intents": [
{
"intent": "Phone",
"score": 0.9658808
},
{
"intent": "Calendar.Add",
"score": 0.0142210266
},
{
"intent": "Calendar.Find",
"score": 0.0112086516
},
{
"intent": "None",
"score": 0.009813501
},
{
"intent": "Email",
"score": 0.0025855056
}
],
"entities": [
{
"entity": "iphone",
"type": "CellPhoneType",
"startIndex": 3,
"endIndex": 8,
"score": 0.998970151
}
]
}
}
Above you can see that the parent LUIS app created from dispatch properly identifies iphone from the utterance my iphone is broken as a CellphoneType entity.
Note: you will not see results from the child LUIS model in the Test panel of the parent dispatch, because the UI does not show connectedServiceResult

Resources