Automating Deployment in Bot Framework (Bot + LUIS+ QnA + Table Storage) - asp.net-web-api

We have a bot deployed on Azure but we want to give it to a client so he can deploy it run it using his own resources. We need to give them a Powershell script that magically create and deploy all the resources needed for the bot to work. My bot architecture consists on the following parts:
Bot's Logic (ASP.NET Web API Project deployed over an Azure App Service)
LUIS Model (published over a Cognitive Services Account)
QnA Service Knowledge Base done with QnA Maker (published directly from the QnaMaker Portal (have no idea where it's deployed)
Azure Table Storage
My Questions are:
1) How to configure bots web api to connection strings parameters? (table storage, luis and qna service will be different when they re redeployed) Currently I am defining the conn. strings and api keys on the web.config, but as I said, this needs to be dynamic.
2) How to automate deployment for LUIS? Luis needs to have the Key of the Cognitive Services Account that should be created first. And I assume I have the exported model json file. I was thinking of using the LUIS API to do the app export and the publishing part. Would that be enough?
3) How to deploy qna services? I think currently is deployed somewhere magically so maybe I won't need to do anything with it.
Thanks!

Maybe a little bit late, but I just had to implement the very same thing, so here are the answers to your questions in hope they could be useful to others:
1) As JoyrexJ9 mentioned above, you can do this via an ARM template by setting the Application Settings of your App Service which will override the values in your Web.config. More about this here.
You can put together and set the connection string of a storage account in the ARM template like this:
{
"type": "Microsoft.Web/sites",
"kind": "app",
"name": "MyWebApp",
"apiVersion": "2015-08-01",
"location": "westeurope",
"properties": {
"name": "MyWebApp",
"serverFarmId": "[resourceId('Microsoft.Web/serverfarms', 'MyAppServicePlanName')]",
"siteConfig": {
"appSettings": [{
"name": "StorageConnectionString",
"value": "[concat('DefaultEndpointsProtocol=https;AccountName=','MyStorageAccountName',';AccountKey=',listKeys(resourceId('Microsoft.Storage/storageAccounts', 'MyStorageAccountName'), '2017-10-01').keys[0].value,';EndpointSuffix=core.windows.net')]"
}],
"cors": {
"allowedOrigins": [
"*"
]
}
}
},
"dependsOn": [
"[resourceId('Microsoft.Storage/storageAccounts/', 'MyStorageAccountName')]",
"[resourceId('Microsoft.Web/serverfarms', 'MyAppServicePlanName')]"
]
}
For LUIS and QnA maker you will need to get the values from the respective portals manually and either update the App Settings after the ARM deployment manually, or re-run the ARM deployment with the manually acquired values passed to it as parameters. The latter works because for the first time you can leave those values empty in your ARM template, and when you deploy it the second time with the parameter values in place, ARM will simply update those App Settings values. More on this topic here.
(Hint: if you provision the QnA Maker and LUIS apps programmatically through their API-s, then you'll only need to get the subscription key for LUIS manually, because you'll get the knowledge base's credentials from QnA Maker's API.)
2) Unfortunately you cannot automate the provisioning of a LUIS app completely at the moment. You can create the resource in Azure via an ARM template and you can do the bulk of the rest of the work through the LUIS API, but for example you cannot assign the subscription key created by the ARM template to a LUIS app programmatically, because that API method is deprecated.
3) The QnA Maker service and it's hosting model changed significantly since you've submitted your questions. I wrote a full blog post about how to do the provisioning of it in the new system.
As JoyrexJ9 mentioned above, it's very important to point out that you won't be able to automate the bot registration fully even with a script, because there's no API for registering an application at https://apps.dev.microsoft.com/. You'll have to do that manually as well. Everything else (besides the things I mentioned above) can be fully automated either via ARM templates or scripts.

Some of this you can automate with ARM templates, you can use functions like listKeys() to query the connection keys of one resource and use it as parameters for another resource.
If you use Azure App Services you can override settings in web.config with what are called App Settings, and these can be set inside an ARM template
Check out this ARM template which deploys a bot and dynamically links it to a newly deployed cognitive service
The bot and app-id registration I don't think you can currently automate, there's no API or CLI

Related

Connect an Existing QnA Knowledge base in Bot Composer

Sorry if this is a stupid question. I just installed Bot Composer v2 and want to connect the bot to a QnAMaker knowledgebase. This is what I have done:
Provision the necessary resources on Azure.
In the QnAMaker website, (www.qnamaker.ai), create a knowledge base
Upload a File and have it populates the QnA pairs. Save and publish (not sure if publish is necessary).
Create a new bot with the "Core bot with QnA" template.
Configure the bot to use the existing resource.
However, after doing the five steps, it doesn't give me the option to connect to an existing knowledgebase. The only option I have is to create a new knowledge base, and the only option I have to populate the QnA pair is via a public URL - which is not a preferable option.
Since it doesn't work, I was trying to work the other direction by
Create a new bot with the "Core bot with QnA" template.
Configure the bot to use the existing resource.
Create a new Knowledge base with nothing in it.
Test the bot.
I checked QnAMaker website and realized the new empty knowledge base has been added. I try to upload files to it - but it won't let me (Oops something went wrong)
The use case I am trying to do is to manage my knowledge (add new files etc) through the QnAMaker website, so content owner can manage their own content. As a developer I will simply connect to the knowledge base. However it seems that I can only manage the knowledgebase directly in the Bot Composer with reduced functionality (e.g. no direct import of a file).
Follow this https://learn.microsoft.com/en-us/composer/how-to-add-qna-to-bot?tabs=v2x
to Add Existing QnA Maker KB to Bot Composer

LUIS: Edit project downloaded from Azure

I have subscription on Azure and I dowloaded chat bot with my AppId and password. On page luis.ai I trained new model and exported it to downloaded azure project (with flight booking). I replaced their cognitive model with my model from luis.ai, but after that - azure project is always working with old data. I don´t understand why, because their model is removed from my PC. What should I do to working with my own model? Thanks.
You code is based on the core-bot sample. First of all, make sure that your LUIS configuration is set up correctly in your .env file (or in App Settings if running from Azure). The recognizer is created in index.js and passed to MainDialog.js. From the core-bot sample code, I'm actually not seeing where it is importing the local code. I think that is maybe just to give you the model to import to your own LUIS app? If you have the proper LUIS keys and app ID, it should respond to whatever you have in there. My guess is maybe that you replaced the FlightBooking.json LUIS model file, but didn't actually point the bot to your LUIS app with the new intents.
I would suggest, though, that this isn't the best sample to use if you are trying to just tweak it. There are a lot of things here that are set up specifically for booking flights that don't really make sense if that's not what your bot is doing. Personally I like the Dispatch Bot sample better as a starting point (even if you are not using Dispatch CLI tool), though it has the intent actions within the bot file instead of separate dialogs. Maybe that will give you a better starting point though?

How to add a Prediction Resource to LUIS app programmatically

I am using C# SDK to create the LUIS apps with required Intents, Utterances and Entities, and then training and Publishing the apps using an Authoring key created in Azure portal. After this, I use the REST Prediction endpoint for Intent matching, and till now I used the free Starter Prediction Key. After a month since the Starter key gets expired, I created a Cognitive service in Azure to be used as the Prediction resource. Now, how to add this new Prediction resource(Cognitive service) to my LUIS apps via SDK or REST? I am able to do so from LUIS Portal though, but don't want manual intervention.
Once you create the prediction endpoint resource, the resource should be only used for endpoint prediction queries and not for authoring changes to the app. If you want to add the prediction resource to your app by not going through the portal and automate the assignment of the resource to a LUIS app for purposes like CI/CD pipeline, then you can follow the steps below:
Get an Azure Resource Manager token from this website. This token does expire so use it immediately. The request returns an Azure Resource Manager token.
Use the token to request the LUIS runtime resources across subscriptions, from the Get LUIS azure accounts API, which your user account has access to.
This POST API requires the following settings:
This API returns an array of JSON objects of your LUIS subscriptions including subscription ID, resource group, and resource name, returned as account name. Find the one item in the array that is the LUIS resource to assign to the LUIS app.
Assign the token to the LUIS resource with the Assign a LUIS azure accounts to an application API.
This POST API requires the following settings:
When this API is successful, it returns a 201 - created status.
Hope this helps.
Thanks, accepted. The only issue being, since I am doing every thing using SDK, it takes a lot of API calls to get things done. I mean to solve this, first I am generating a Token and then calling this API. A simple add default "Prediction Resource" for all apps on LUIS portal would have helped a lot. Anyways, thanks for this answer. – user3868541 Feb 3 at 9:45
Agreed that an Add default Prediction Resource would have made everything much better. I'm still not sure how to do this programmatically. I'm trying to do it in my service using the SDK

Cosu app development through Android Management Api or device owner locktask

i have to implement single use application for my Enterprise's devices where there is only one major android app that itself can use 2-3 other apps like call, sms and google maps, other than that user must not be able to use or access other apps and settings,
1) I have considered Corporate owned Single use through Enterprise Management Api which is more sophisticated and big, but more complete solution.
2) I have implemented and tested the sample
Lock task with an android device owner app
that looks more like being my solution, but there is a problem
How do i provision device owner for production level devices? for my test i was able to provision with adb shell commmand. i know that device must be new/Reset and unprovisioned thats not a problem.
I am seeking some suggestion here, if anybody has implemented it it could help.
Update
going with the android management Api QuickStartGuide, suggested by Fred seems like correct way.
where my policy is Multiple app from custom launcher
now i am stuck in a situation, i want to publish my launcher app or other app to play store only for my enterprise.
I have followed Upload your own app to the Google Play Store,which led me to publish private app, but i am not able to do so as i am not getting Restrict Distribution option.
I don't know how to perfectly achieve this.
for my current policy and enterprise, i have 2 active email ids,
first email_1 is the one which is connected to the initial project
and
all the google api call is done under email_1 .
and the other is email_2 the one is admin for the enterprise
and connected to managed Google Play Store.
both email_1 and email_2 has admin access to my company developer account
.
Now i need to figure out to publish app only for my enterprise, i think there is a issue with correct permission or otherwise,
need Help.
Thanks
It's easy, once you understand the missing links.
The documentation should definitely clarify these steps.
1 Setup the accounts
We created a Google Suite Account for our client and uploaded his app in the Google Play Console account created using a Google Account under their organization, and limited the app distribution to their organization.
We also added Android for Work to our organization here: https://admin.google.com/AdminHome?pli=1&fral=1#SelectServices
And added a role to our organization to manage Google Play Private Uploads:
https://admin.google.com/AdminHome#DomainSettings/notab=1&role=new-role&subtab=roles
2 Find the organization ID
This is the key part. You should find your organization ID here: https://play.google.com/work/adminsettings
3 Enroll the organization
When following the steps for the Android Management API, you need to enroll the organization ID found in your Google For Work account.
Follow the step detailed here using your enterpriseID:
enterprises/{enterpriseId}/enrollmentTokens/{enrollmentTokenId}
4 Add your app
Add your app using its PackageName in a ApplicationPolicy. Your devices will now be able to find it and install it.
You no longer need to implement a Device Policy Controller to manage Android devices, Google has recently released the Android Management API which allows you to set up a COSU device with just a few Cloud API calls.
If you have one main app and want to allow to open a few other apps, you can set the main app as a custom launcher and mark the other apps as lockTaskAllowed. You can simply do so by defining an ApplicationPolicy such as the one below (copied from Create a policy):
"applications": [
{
"packageName": "com.example.custom_launcher",
"installType": "FORCE_INSTALLED",
"lockTaskAllowed": true,
"defaultPermissionPolicy": "GRANT",
},
{
"packageName": "com.example.app1",
"installType": "FORCE_INSTALLED",
"lockTaskAllowed": true,
"defaultPermissionPolicy": "GRANT",
}
],
"persistentPreferredActivities": [
{
"receiverActivity": "com.example.custom_launcher",
"actions": [
"android.intent.action.MAIN"
],
"categories": [
"android.intent.category.HOME",
"android.intent.category.DEFAULT"
]
}
]
Google provides a good feature list here:
https://developers.google.com/android/work/requirements/features
If your devices have NFC, I would provision using NFC. You can look at the code here for your own implementation or you might be able to use the app with very few modifications.
https://github.com/googlesamples/android-NfcProvisioning
By tapping the welcome screen in the Google Setup Wizard 6 times will allow you to use QR code provisioning. It is a bit more cumbersome in my opinion and requires Android 7.0+.
Those are your only two options unless you become a Google EMM Partner or again, partner with them to support Zero Touch Enrollment on Android 8.0+ devices for your own homebaked solution.
You might want to also look at existing open source EMM/MDM implementations that already exist such as WSO2.

Using MSAL in a machine-to-machine scenario as a CSP

I am trying to use the GraphAPI using the official nuget library (https://github.com/microsoftgraph/msgraph-sdk-dotnet). However, the authentication process is not trivial and not very well documented.
I am in a CSP partner in a machine-to-machine scenario, so there is no UI application with a redirect url where a user manually enter it's credentials to consent access. But it seems to be the unique scenario well documented or available in the samples I found... Even the official CSP documentation is not clear (https://developer.microsoft.com/en-us/graph/docs/concepts/auth_cloudsolutionprovider).
I previously used many other APIs in production (and TIP environment) as a CSP like MSOL, CrestAPI and AzureGraph, so all the configuration and registration of the application is not a problem.
Is it possible to use MSAL as a CSP in a machine-to-machine scenario? Is there any documentation or sample available for my use case?
UPDATE
I think I am on the good path.
First, I had to register my app on apps.dev.microsoft.com. My apps were previously registered on portal.azure.com. Looks like AzureAD and Graph had different requirements related to where applications must be registered.
Second, I use the sample here: https://github.com/Azure-Samples/active-directory-dotnet-daemon-v2 .
Third, I am able to call the Graph API when I provide my CSP TenantId. However, when I try to call the Graph API using the TenantId of one of the tenant I manage, I got a:
{ "error": {
"code": "Authorization_IdentityNotFound",
"message": "The identity of the calling application could not be established.",
"innerError": {
"request-id": "7cab3137-b3e7-4622-9123-e47f2c018c56",
"date": "2017-09-04T14:48:34"
} } }
(I cannot have the consent page in my use case so I tried to pre-consent my app for all consumers using: https://developer.microsoft.com/en-us/graph/docs/concepts/auth_cloudsolutionprovider)
You can use MSAL to perform the auth required for the Graph SDK. There are some code snippets of using the .NET Graph SDK with the MSAL, but your best bet is the Connect Sample that shows this in a working app.
For a complete guided sample of how to use MSAL and what the library does, checkout the MSAL .NET Desktop Sample.

Resources