Responding to Conversations async: Graph or Bot? - botframework

I have a Teams Message extension that returns a Task response which is a medium sized embedded web view iFrame
This is working successfully; including added a custom Tab within the channel and other nice magic calls to Microsoft Graph.
What I am confused about is how to do (and this is probably my not understanding the naming of things)
insert "something" Back into the Message/Post stream which is a link to newly created Tab ... like the what you get when you have a "configureTabs" style Tab created -- there is a friendly Message (Post) in the chat pointing to this new Tab.
do I do this with Microsoft Graph or back through the Bot?
the code that does the communication may be a different service elsewhere that is acting async ... so it needs to communicate with something somewhere with context. Confused if this is the Bot with some params or Microsoft Graph with params.
how to insert an image (rather than a link to the tab) into the Message/Post stream -- but showing the image not a link off to some random URL (ie: )
could not find any samples that do this; again, will be async as per above; but the format of the message will be a Card or something custom?

So just to be clear, a Task Response is NOT the same as a Tab, albeit that they might end up hosted in the same backend web application (and also albeit that your TAB can actual bring up your Task Response popup/iframe using the Teams javascript library).
Aside from that, in order to post something back to the channel, like when the Tab is created, there are two ways to do so:
First is to use Graph Api's Create ChatMessage option (this link is just for a channel though - not sure if your tab/task apply to group chats and/or 1-1 chats as well).
2nd Option is to have a Bot be part of your application as well. Then, when you're ready to send something to the channel, you'd effectively be sending something called a "pro-active messaging". You need to have certain reference data to do this, which you would get when the bot is installed into the channel ("conversation reference", "ServiceUrl", and so on). I describe this more in my answer at Programmatically sending a message to a bot in Microsoft Teams
With regards sending the image, either of the above would work here too, in terms of how to send the image. As to the sending of an image, you'd need to make use of one of the kinds of "Cards" (basically "richer" messages than just raw text). You can learn more about this at Introducing cards and about the types of cards for Teams at Card reference. There are a few that can be used to send an image, depending on perhaps what else you want the card to do. For instance, an Adaptive Card can send an image, some text, and an action button of some sort.
Hope that helps

To close the loop for future readers.
I used the following Microsoft Graph API docs, and the posting above, and this is working: Create chatMessage in a channel and Creating a Custom Microsoft Graph call from the SDK
The custom graph call (as it is not implemented in the .NET SDK at the time of this response) looks something like:
var convoReq = $"https://graph.microsoft.com/beta/teams/{groupId}/channels/{channelId}/messages";
var body = this.TeamsMessageFactory(newCreatedTabUrl, anotherstring).ToJson();
var postMessage = new HttpRequestMessage(HttpMethod.Post, convoReq)
{
Content = new StringContent(body, System.Text.Encoding.UTF8, "application/json")
};
await _graphClient.CurrentGraphClient.AuthenticationProvider.AuthenticateRequestAsync(postMessage);
var response = await _graphClient.CurrentGraphClient.HttpProvider.SendAsync(postMessage);
if (response.IsSuccessStatusCode)
{
var content = await response.Content.ReadAsStringAsync();
return true;
}
The groupId and channelId are found elsewhere; and the TeamsMessageFactory is just some boilerplate that serialized the C# object graph for the POST request, as detailed in Create chatMessage in a channel

Related

Microsoft Teams Bot - Is it possible to display an Adaptive Card to get user input when a user enters a command?

I want to develop a Microsoft Teams Bot that when a user enters a keyword, such as 'pto' it will show an Adaptive Card form that the user can populate with information which will then get sent to a web service for processing.
Is it possible to display an Adaptive Card based on a user entering a text command?
Yes, absolutely possible. A "Command" is really just a regular message, and it's how you interpret / respond to the message. You can either detect these messages yourself by looking at the content, or you can use a conversational AI engine like LUIS.ai (part of Azure) which can more easily be configured to check for variants (e.g. "pto", "PTO", "POT", "[whatever 'pto' stands for']" etc. In this case, LUIS is basically building a ton of "if" statements to find a match, and the returning you an "intent" (e.g. "User is requesting whatever "PTO" means"). Because it's using AI to generate and maintain this "if" list, language conversational engines like LUIS are widely used in bots, but they're not -required-. It's why you see them in most samples though.
With regards sending an Adaptive Card, that's also pretty standard stuff in Teams bots, and they can be sent in response to a user's message, like you're trying here, as well as other ways to invoke them.
I actually cover both of these topics one after the other in a conference session earlier this year - see the video here: https://www.youtube.com/watch?v=mM7-fYdcJhw&t=1398s (the earlier parts might actually be of interest too).
Bot Framework can't handle the card prompts natively, so the best solution I've come up with is to
Display the card as a standard activity (i.e. before the prompt)
Provide a message indicating you are waiting for prompt input (e.g. waiting for selection...)
Validate that the prompt input is an object (e.g. input.match(/{.+/g))
Use the values from the object in your next step to call the webservice
So in code it ends up like this:
await step.context.sendActivity(CardHelper.datePicker());
return await step.prompt(DATE_PROMPT, `*waiting for selection...*`);
with a validator to make sure you are receiving an object like this:
async validateDateCard(prompt) {
prompt.activeDialog = await this.userDialogStateAccessor;
prompt.context = await prompt.context;
const input = prompt.recognized.value;
if (input.match(/{.+/g)) {
return true;
} else {
await prompt.context.sendActivity(`Please use the calendar widget above to enter the date.`);
return false;
}
}
In my case my widget just has a single field for date, but it works the same regardless of how many fields you have. They will all be in the submitted object. If you try to type something in manually, you are reprompted to use the widget. In general I find prompting for plain text values in sequence instead of using cards works fine and it not too cumbersome for users. But if you have a ton of inputs or need specialized controls like the above date widget, sometimes cards are the only way.

Return initial data on subscribe event in django graphene subscriptions

I'm trying to response to user on subscribe. By example, in a chatroom when an user connect to subscription, the subscription responses him with data (like a welcome message), but only to same user who just connect (no broadcast).
How can I do that? :(
Update: We resolve to use channels. DjangoChannelsGraphqlWs does not allow direct back messages.
Take a look at this DjangoChannelsGraphQL example. Link points to the part which is there to avoid "user self-notifications" (avoid user being notified about his own actions). You can use the same trick to send notification only to the user who made the action, e.g. who just subscribed.
Modified publish handler could look like the following:
def publish(self, info, chatroom=None):
new_msg_chatroom = self["chatroom"]
new_msg_text = self["text"]
new_msg_sender = self["sender"]
new_msg_is_greetings = self["is_greetings"]
# Send greetings message only to the user who caused it.
if new_msg_is_greetings:
if (
not info.context.user.is_authenticated
or new_msg_sender != info.context.user.username
):
return OnNewChatMessage.SKIP
return OnNewChatMessage(
chatroom=chatroom, text=new_msg_text, sender=new_msg_sender
)
I did not test the code above, so there could be issues, but I think it illustrates the idea quite well.

Notifying clients from (boxed) Syncfusion Ajax call

I'm trying to integrate the schedule component from Syncfusion. The component has a URL adaptor to connect to the controller; GetData() and Batch() for Crud Operations. Batch has a payload indicating what actions to perform. At the end, the Batch method would requery the database and send data identical to GetData() back.
Unfortunately, there is no built-in method to notify clients of anything going wrong - whether there is an exception, server-side validation kicks in or similar.
What I'd like to do is to add a placeholder outside the compentent to receive and display server messages (be it a notification popup, a or whatever.
Since I can't influence the Ajax call itself, I was wondering if I had to get started with SignalR (still in beta for .Net Core 2 as far as I know), or if I may have missed something more obvious? I have read a lot about push notifications etc - but these are not quite what I'm after, it'd be slightly over the top I think.
To summarise, let's say I have
<div id="messages"></div>
<div id="component">HereGoesTheScheduleWhichICantDoMuchWith</div>
Now in the Batch() method, it would be great to call a SendMessage("Sorry,you can't do this") - the text of which would ideally then appear in the messages-div.
How would you go about this?
I have now solved this, using SignalR (currently 1.0.0-alpha2-final) and for a nice view on the Client, PNotify.
Presently, it only works if the client is authenticated, if it needs to work anonymously you'd need to figure out a way to track SignalR's connection id.
On the page with the Syncfusion Schedule component, I connect to SignalR.
let connection = new signalR.HubConnection("/signalr", { transport: signalR.TransportType.ServerSentEvents });
connection.on("Notify",
(title, message) => {
new PNotify({
title: title,
text: message
});
});
connection.start();
The Hub (SignalRHub : Hub) creates a notification group for the user connecting:
public override Task OnConnectedAsync()
{
Groups.AddAsync(Context.ConnectionId, Context.User.Identity.Name);
return base.OnConnectedAsync();
}
The associated controller gets IHubContext<SignalRHub> signalRHub injected.
Now in the Batch-Method for the Syncfusion component, which returns Json and can't itself carry messages or notifications, you can notify the user:
_signalRHub.Clients.Group(User.Identity.Name).InvokeAsync("Notify", "A title", "A message");
In my particular case, I'm sending over an object to control layout, animation and popup duration for PNotify (e.g. longer for an exception to allow copy/paste etc) - as you please. Returning an object could be done using:
_signalRHub.Clients.Group(User.Identity.Name).InvokeAsync("Notify", JsonConvert.SerializeObject(new { title = "Some Title", message = "notification", type = "notice"}););
Obviously, connection.on("Notify"... needs to be changed accordingly.
I hope this is clear enough and might help someone else.

Pubnub chat application with storage

I'm looking to develop a chat application with Pubnub where I want to make sure all the chat messages that are send is been stored in the database and also want to send messages in chat.
I found out that I can use the Parse with pubnub to provide storage options, But I'm not sure how to setup those two in a way where the messages and images send in the chat are been stored in the database.
Anyone have done this before with pubnub and parse? Are there any other easy options available to use with pubnub instead of using parse?
Sutha,
What you are seeking is not a trivial solution unless you are talking about a limited number of end users. So I wouldn't say there are no "easy" solutions, but there are solutions.
The reason is your server would need to listen (subscribe) to every chat channel that is active and store the messages being sent into your database. Imagine your app scaling to 1 million users (doesn't even need to get that big, but that number should help you realize how this can get tricky to scale where several server instances are listening to channels in a non-overlapping manner or with overlap but using a server queue implementation and de-duping messages).
That said, yes, there are PubNub customers that have implemented such a solution - Parse not being the key to making this happen, by the way.
You have three basic options for implementing this:
Implement a solution that will allow many instances of your server to subscribe to all of the channels as they become active and store the messages as they come in. There are a lot of details to making this happen so if you are not up to this then this is not likely where you want to go.
There is a way to monitor all channels that become active or inactive with PubNub Presence webhooks (enable Presence on your keys). You would use this to keep a list of all channels that your server would use to pull history (enable Storage & Playback on your keys) from in an on-demand (not completely realtime) fashion.
For every channel that goes active or inactive, your server will receive these events via the REST call (and endpoint that you implement on your server - your Parse server in this case):
channel active: record "start chat" timetoken in your Parse db
channel inactive: record "end chat" timetoken in your Parse db
the inactive event is the kickoff for a process that uses start/end timetokens that you recorded for that channel to get history from for channel from PubNub: pubnub.history({channel: channelName, start:startTT, end:endTT})
you will need to iterate on this history call until you receive < 100 messages (100 is the max number of messages you can retrieve at a time)
as you retrieve these messages you will save them to your Parse db
New Presence Webhooks have been added:
We now have webhooks for all presence events: join, leave, timeout, state-change.
Finally, you could just save each message to Parse db on success of every pubnub.publish call. I am not a Parse expert and barely know all of its capabilities but I believe they have some sort or store local then sync to cloud db option (like StackMob when that was a product), but even if not, you will save msg to Parse cloud db directly.
The code would look something like this (not complete, likely errors, figure it out or ask PubNub support for details) in your JavaScript client (on the browser).
var pubnub = PUBNUB({
publish_key : your_pub_key,
subscribe_key : your_sub_key
});
var msg = ... // get the message form your UI text box or whatever
pubnub.publish({
// this is some variable you set up when you enter a chat room
channel: chat_channel,
message: msg
callback: function(event){
// DISCLAIMER: code pulled from [Parse example][4]
// but there are some object creation details
// left out here and msg object is not
// fully fleshed out in this sample code
var ChatMessage = Parse.Object.extend("ChatMessage");
var chatMsg = new ChatMessage();
chatMsg.set("message", msg);
chatMsg.set("user", uuid);
chatMsg.set("channel", chat_channel);
chatMsg.set("timetoken", event[2]);
// this ChatMessage object can be
// whatever you want it to be
chatMsg.save();
}
error: function (error) {
// Handle error here, like retry until success, for example
console.log(JSON.stringify(error));
}
});
You might even just store the entire set of publishes (on both ends of the conversation) based on time interval, number of publishes or size of total data but be careful because either user could exit the chat and the browser without notice and you will fail to save. So the per publish save is probably best practice if a bit noisy.
I hope you find one of these techniques as a means to get started in the right direction. There are details left out so I expect you will have follow up questions.
Just some other links that might be helpful:
http://blog.parse.com/learn/building-a-killer-webrtc-video-chat-app-using-pubnub-parse/
http://www.pubnub.com/blog/realtime-collaboration-sync-parse-api-pubnub/
https://www.pubnub.com/knowledge-base/discussion/293/how-do-i-publish-a-message-from-parse
And we have a PubNub Parse SDK, too. :)

Identify http request / response by id

I am building an extension with Firefox's ADD ON SDK (v1.9) that will be able to read all HTTP requests / responses and calculate the time they took to load. This includes not only the main frame but any other loading file (sub frame, script, css, image, etc.).
So far, I am able to use the "observer-service" module to listen for:
"http-on-modify-request" when a HTTP request is created.
"http-on-examine-response" when a HTTP response is received
"http-on-examine-cached-response" when a HTTP response is received entirely from cache
"http-on-examine-merged-response" when a HTTP response is received partially from cache
My application follows the following sequence:
A request is created and registered through the observer.
I save the current time and mark it as start_time of the request load.
A response for a request is received and registered through one of the observers.
I save the current time and use the previously saved time to calculate load time of the request.
Problem:
I am not able to link the start and end times of the load since I cannot find a request ID (or other unique value) that will tie the request with the response.
I am currently using the URL of the request / response to tie them together but this is not correct since it will raise a "race condition" if two or more equal urls are loading at the same time. Google Chrome solves this issue by providing unique requestIds, but I have not been able to find a similar functionality on Firefox.
I am aware of two ways to recognize a channel that you receive in this observer. The "old" solution is to use nsIWritablePropertyBag interface to attach data to the channel:
var {Ci} = require("chrome");
var channelId = 0;
...
// Attach channel ID to a channel
if (channel instanceof Ci.nsIWritablePropertyBag)
channel.setProperty("myExtension-channelId", ++channelId);
...
// Read out channel ID for a channel
if (channel instanceof Ci.nsIPropertyBag)
console.log(channel.getProperty("myExtension-channelId"));
The other solution would be using WeakMap API (only works properly starting with Firefox 13):
var channelMap = new WeakMap();
var channelId = 0;
...
// Attach channel ID to a channel
channelMap.set(channel, ++channelId);
...
// Read out channel ID for a channel
console.log(channelMap.get(channel));
I'm not sure whether WeakMap is available in the context of Add-on SDK modules, you might have to "steal" it from a regular JavaScript module:
var {Cu} = require("chrome");
var {WeakMap} = Cu.import("resource://gre/modules/FileUtils.jsm", null);
Obviously, in both cases you can attach more data to the channel than a simple number.
Firebug does what you're thinking of by implementing a central observer for these events:
https://github.com/firebug/firebug/blob/master/extension/modules/firebug-http-observer.js
This might be a good place to start, although eventually Firefox will ship a more complete network monitor / debugger by default. I think I read somewhere that it will be based on Firebug's.

Resources