Azure Service Bus - ARM Template (Update existing Topic parameters) - visual-studio

I need help with the Azure service bus service. I need to create a new topic in an existing service bus service by using Visual Studio, is there any way to achieve this. Because when I try to do this, I’m getting the following error:
Template deployment returned the following errors:
15:28:14 - 15:28:13 - Resource Microsoft.ServiceBus/namespaces “#######” failed with message '{
15:28:14 - "error": {
15:28:14 - "message": "Namespace update failed with conflict in backend. CorrelationId: 3c155444-2c1e-525d-943f-8b25d0a1da7e",
15:28:14 - "code": "Conflict"
15:28:14 - }
Any help would be very helpful to me.

The Microsoft.Resources/deployments template allows you to update an existing resource.
Modifiying the quickstart template to allow updates looks something like this (I did remove the subscription deployment to make it a bit shorter). Keep in mind that some parameters like partitioning cannot be modified- you must delete the resource and redeploy to change them.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"service_BusNamespace_Name": {
"type": "String",
"metadata": {
"description": "Name of the Service Bus namespace"
}
},
"serviceBusTopicName": {
"type": "String",
"metadata": {
"description": "Name of the Topic"
}
},
"serviceBusSubscriptionName": {
"type": "String",
"metadata": {
"description": "Name of the Subscription"
}
},
"location": {
"defaultValue": "[resourceGroup().location]",
"type": "String",
"metadata": {
"description": "Location for all resources."
}
}
},
"variables": {
"defaultSASKey_Name": "RootManageSharedAccessKey",
"authRuleResource_Id": "[resourceId('Microsoft.ServiceBus/namespaces/authorizationRules', parameters('service_BusNamespace_Name'), variables('defaultSASKey_Name'))]",
"sbVersion": "2017-04-01"
},
"resources": [
{
"type": "Microsoft.Resources/deployments",
"apiVersion": "2015-01-01",
"name": "updateTopic",
"properties": {
"mode": "Incremental",
"parameters": {},
"template": {
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {},
"variables": {},
"resources": [
{
"apiVersion": "2017-04-01",
"name": "[parameters('service_BusNamespace_Name')]",
"type": "Microsoft.ServiceBus/namespaces",
"location": "[parameters('location')]",
"sku": {
"name": "Standard"
},
"resources": [
{
"apiVersion": "2017-04-01",
"name": "[parameters('serviceBusTopicName')]",
"type": "Topics",
"dependsOn": [
"[concat('Microsoft.ServiceBus/namespaces/', parameters('service_BusNamespace_Name'))]"
],
"properties": {
"defaultMessageTimeToLive": "P10675199DT2H48M5.4775807S",
"maxSizeInMegabytes": "1024",
"requiresDuplicateDetection": "false",
"duplicateDetectionHistoryTimeWindow": "PT10M",
"enableBatchedOperations": "false",
"supportOrdering": "false",
"autoDeleteOnIdle": "P10675199DT2H48M5.4775807S",
"enablePartitioning": "false",
"enableExpress": "false"
}
}
]
}
]
}
}
}
],
"outputs": {
"NamespaceConnectionString": {
"type": "String",
"value": "[listkeys(variables('authRuleResource_Id'), variables('sbVersion')).primaryConnectionString]"
},
"SharedAccessPolicyPrimaryKey": {
"type": "String",
"value": "[listkeys(variables('authRuleResource_Id'), variables('sbVersion')).primaryKey]"
}
}
}

Usually you get "conflict in backend" errors when you request an operation that isn't allowed in the resource's current state.
For example, once you upgrade Service Buses to Standard, deployments will fail if it tries to downgrade to Basic.
Effectively, the only solution for those cases is re-provision the resource from scratch.

Related

VSC validation with $schema and usage of additionalProperties = false

i have a dataobject.json and a corresponding example.json. I like to compare both if everything what is in the example is the same notation as in the dataobject.
I added in the exampe the dataobject file as a schema to validate against. This works, but only for the required field, not for the optional properties. - there the validation doesn't find a problem, even if there are some.
To validate these I added the "additionalProperties": false line. This works in general, so I find all the deviations, but I also get a problem that property §schema is not allowed.
How can I solve this?
the dataobject
{
"$schema": "http://json-schema.org/draft-07/schema",
"type": "object",
"title": "GroupDO",
"required": [
"id",
"name"
],
"additionalProperties": false,
"properties": {
"id": {
"type": "string",
"format": "uuid",
"example": "5dd6c80a-3376-4bce-bc47-8t41b3565325",
"description": "Unique id ."
},
"name": {
"type": "string",
"example": "ABD",
"description": "The name."
},
"GroupSort": {
"type": "integer",
"format": "int32",
"example": 1,
"description": "Defines in which order the groups should appear."
},
"GroupTextList": {
"type": "array",
"description": "A descriptoin in multiple languages.",
"items": {
"$ref": "../../common/dataobjects/Description_1000_DO.json"
}
},
"parentGroupId": {
"type": "string",
"format": "uuid",
"example": "8e590f93-1ab6-40e4-a5f4-aa1eeb2b6a80",
"description": "Unique id for the parent group."
}
},
"description": "DO representing a group object. "}
the example
{ "$schema": "../dataobjects/GroupDO.json",
"id": "18694b46-0833-4790-b780-c7897ap08500",
"version": 1,
"lastChange": "2020-05-12T13:57:39.935305",
"sort": 3,
"name": "STR",
"parentGroupId": "b504273e-61fb-48d1-aef8-c289jk779709",
"GroupTexts": [
{
"id": "7598b668-d9b7-4d27-a489-19e45h2bdad0",
"version": 0,
"lastChange": "2020-03-09T14:14:25.491787",
"languageIsoCode": "de_DE",
"description": "Tasche"
},
{
"id": "376e82f8-837d-4bb2-a21f-a9e0ebd59e23",
"version": 0,
"lastChange": "2020-03-09T14:14:25.491787",
"languageIsoCode": "en_GB",
"description": "Bag"
}
]
}
the problem messages:
property $schema is not allowed
Thanks in advance for your help.

Add images via Shopware 6 API

I have a Shopware 6.3 shop and need to migrate images to it using the integration API.
How should I construct a body for a media upload? Do I need to put a file somewhere or just pass in the link?
I have managed to push new products into Shopware via guide here: https://docs.shopware.com/en/shopware-platform-dev-en/admin-api-guide/writing-entities?category=shopware-platform-dev-en/admin-api-guide#creating-entities but I am not sure how to handle media. In this guide it is only explained how to create links between already uploaded media files to products in here https://docs.shopware.com/en/shopware-platform-dev-en/admin-api-guide/writing-entities?category=shopware-platform-dev-en/admin-api-guide#media-handling but no examples as to how to actually push the media files.
I have URL's for each image I need (in the database, along with produc id's and image positions).
The entity schema describes media as:
"media": {
"name": "media",
"translatable": [
"alt",
"title",
"customFields"
],
"properties": {
"id": {
"type": "string",
"format": "uuid"
},
"userId": {
"type": "string",
"format": "uuid"
},
"mediaFolderId": {
"type": "string",
"format": "uuid"
},
"mimeType": {
"type": "string",
"readOnly": true
},
"fileExtension": {
"type": "string",
"readOnly": true
},
"uploadedAt": {
"type": "string",
"format": "date-time",
"readOnly": true
},
"fileName": {
"type": "string",
"readOnly": true
},
"fileSize": {
"type": "integer",
"format": "int64",
"readOnly": true
},
"metaData": {
"type": "object",
"readOnly": true
},
"mediaType": {
"type": "object",
"readOnly": true
},
"alt": {
"type": "string"
},
"title": {
"type": "string"
},
"url": {
"type": "string"
},
"hasFile": {
"type": "boolean"
},
"private": {
"type": "boolean"
},
"customFields": {
"type": "object"
},
"createdAt": {
"type": "string",
"format": "date-time",
"readOnly": true
},
"updatedAt": {
"type": "string",
"format": "date-time",
"readOnly": true
},
"translated": {
"type": "object"
},
"tags": {
"type": "array",
"entity": "tag"
},
"thumbnails": {
"type": "array",
"entity": "media_thumbnail"
},
"user": {
"type": "object",
"entity": "user"
},
"categories": {
"type": "array",
"entity": "category"
},
"productManufacturers": {
"type": "array",
"entity": "product_manufacturer"
},
"productMedia": {
"type": "array",
"entity": "product_media"
},
"avatarUser": {
"type": "object",
"entity": "user"
},
"mediaFolder": {
"type": "object",
"entity": "media_folder"
},
"propertyGroupOptions": {
"type": "array",
"entity": "property_group_option"
},
"mailTemplateMedia": {
"type": "array",
"entity": "mail_template_media"
},
"documentBaseConfigs": {
"type": "array",
"entity": "document_base_config"
},
"shippingMethods": {
"type": "array",
"entity": "shipping_method"
},
"paymentMethods": {
"type": "array",
"entity": "payment_method"
},
"productConfiguratorSettings": {
"type": "array",
"entity": "product_configurator_setting"
},
"orderLineItems": {
"type": "array",
"entity": "order_line_item"
},
"cmsBlocks": {
"type": "array",
"entity": "cms_block"
},
"cmsSections": {
"type": "array",
"entity": "cms_section"
},
"cmsPages": {
"type": "array",
"entity": "cms_page"
},
"documents": {
"type": "array",
"entity": "document"
}
}
},
but it is not clear what fields are crucial. Do I need to create product-media folder first and then use it's id when making a POST request to media endpoint? Can I just specify the URL and will Shopware download the image itself to a folder or keep pointing to the URL I have used. I need to house the images inside the Shopware.
There is no problem for me to download the images from the URL and push them to Shopware but I am not sure how to use the API for it (there is a lot of images and they need to be done in bulk).
One possible solution:
FIRST: create a new media POST /api/{apiVersion}/media?_response=true
SECOND: "Upload Image" /api/{apiVersion}/_action/media/{mediaId}/upload?extension={extension}&fileName={imgName}&_response=true
more information can be found here: https://forum.shopware.com/discussion/comment/278603/#Comment_278603
In CASE images are for products use the endpoint POST /api/{apiVersion}/product-media and set the coverId
A complete listing of all routes is available via the OpenAPI schema: [your-domain/localhost]/api/v3/_info/openapi3.json
It's also possible to set all the media and the cover & coverId during product creation by one request. Therefore, set the product Cover and product Media
{
"coverId":"3d5ebde8c31243aea9ecebb1cbf7ef7b",
"productNumber":"SW10002","active":true,"name":"Test",
"description":"fasdf",
"media":[{
"productId":"94786d894e864783b546fbf7c60a3640",
"mediaId":"084f6aa36b074130912f476da1770504",
"position":0,
"id":"3d5ebde8c31243aea9ecebb1cbf7ef7b"
},
{
"productId":"94786d894e864783b546fbf7c60a3640",
"mediaId":"4923a2e38a544dc5a7ff3e26a37ab2ae",
"position":1,
"id":"600999c4df8b40a5bead55b75efe688c"
}],
"id":"94786d894e864783b546fbf7c60a3640"
}
Keep in mind to check if the bearer token is valid by checking for example like this:
if (JwtToken.ValidTo >= DateTime.Now.ToUniversalTime() - new TimeSpan(0, 5, 0))
{
return Client.Get(request);
}
else
{
// refresh the token by new authentication
IntegrationAuthenticator(this.key, this.secret);
}
return Client.Get(request);
This will work for Shopware 6.4
As a general advice, it depends. The APIs changed a little bit since 6.4 and there is also an official documentation available at https://shopware.stoplight.io/docs/admin-api/docs/guides/media-handling.md.
However, i think that it is always a little easier to have a real life example. What i do in our production environment is basically these steps.
(Optional) Check, if the media object exists
Create an media-file object using the endpoint GET /media-files/
If it exist then upload an image using the new media-id reference.
Let us assume the filename is yourfilename.jpg. What you also will need is a media-folder-id, which will reference the image-folder within Shopware. This can be obtained in Shopware via Admin > Content > Media > Product Media.
Step 0
Before uploading an image to Shopware, you want to ensure that the image does not exists, so that you can skip it.
This step is optional, as it is not mandatory to create an image. However you want to have some sort of validation mechanism in a production environment.
Request-Body
POST api/search/media
This will run a request against the Shopware-API with a response.
{
"filter":[
{
"type":"equals",
"field":"fileName",
"value":"yourfilename"
},
{
"type":"equals",
"field":"fileExtension",
"value":"jpg"
},
{
"type":"equals",
"field":"mediaFolderId",
"value":"d798f70b69f047c68810c45744b43d6f"
}
],
"includes":{
"media":[
"id"
]
}
}
Step 1
Create a new media-file
Request-Body
POST api/_action/sync
This request will create a new media-object in Shopware.
The value for media_id must be any UUID. I will use this value: 94f83a75669647288d4258f670a53e69
The customFields property is optional. I just use it to keep a reference of hash value which i could use to validate changed values.
The value for the media folder id is the one you will get from your Shopware-Backend.
{
"create-media": {
"entity": "media",
"action": "upsert",
"payload": [
{
"id": "{{media_id}}",
"customFields": {"hash": "{{file.hash}}"},
"mediaFolderId": "{{mediaFolderId}}"
}
]
}
}
Response
The response will tell you that everything works as expected.
{
"success":true,
"data":{
"create-media":{
"result":[
{
"entities":{
"media":[
"94f83a75669647288d4258f670a53e69"
],
"media_translation":[
{
"mediaId":"94f83a75669647288d4258f670a53e69",
"languageId":"2fbb5fe2e29a4d70aa5854ce7ce3e20b"
}
]
},
"errors":[
]
}
],
"extensions":[
]
}
},
"extensions":[
]
}
Step 2
This is the step where we will upload an image to Shopware. We will use a variant with the content-type image/jpg. However, a payload with an URL-Attribute would also work. See the details in the official documentation.
Request-Body
POST api/_action/media/94f83a75669647288d4258f670a53e69/upload?extension=jpg&fileName=yourfilename
Note that the media-id is part of the URL. And also the filename but without the file-extension JPG!
This body is pretty straightforward an in our case there is no payload, as we use an upload with Content-Type: "image/jpeg".
This would be a payload if you want to use an URL as resource:
{
"url": "<url-to-your-image>"
}

ARM Template unable to set $connections

I have created an ARM template in Visual Studio project with connection to slack. When I connect to slack in Azure portal - everything is fine. My slack API connection is authorized and works. My problem is with deployment and setting $connections.
Here is my template
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
<template-params>
},
"variables": {
"slack": "[concat(parameters('appPrefix'), '-slack-', parameters('environment'))]"
},
"resources": [
{
"type": "Microsoft.Web/connections",
"apiVersion": "2016-06-01",
"location": "[resourceGroup().location]",
"name": "[variables('slack')]",
"properties": {
"api": {
"id": "[concat('/subscriptions/', subscription().subscriptionId, '/providers/Microsoft.Web/locations/', resourceGroup().location, '/managedApis/slack')]"
},
"displayName": "Slack",
"parameterValues": {}
}
},
{
"name": "[parameters('logicAppName')]",
"type": "Microsoft.Logic/workflows",
"location": "[parameters('logicAppLocation')]",
"tags": {
"displayName": "LogicApp"
},
"apiVersion": "2016-06-01",
"properties": {
"definition": {
"$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
"actions": {
"Collecting_went_wrong": {
"inputs": {
"host": {
"connection": {
"name": "#parameters('$connections')['slack']['connectionId']"
}
},
"method": "post",
"path": "/chat.postMessage",
"queries": {
"channel": "<channel>",
"text": "<message>",
"username": "<user>"
}
},
"runAfter": {},
"type": "ApiConnection"
}
},
"parameters": {
"$connections": {
"type": "object",
"defaultValue": {
}
}
},
"triggers": {
<trigger>
}
},
"parameters": {
"$connections": {
"value": {
"slack": {
"id": "[resourceId('Microsoft.Web/connections', variables('slack'))]",
"connectionId": "[resourceId('Microsoft.Web/connections', variables('slack'))]",
"connectionName": "slack"
}
}
}
}
},
"dependsOn": [
"[resourceId('Microsoft.Web/connections', variables('slack'))]"
]
}
],
"outputs": {}
}
The 'fun' part is validation of template during the deployment:
New-AzureRmResourceGroupDeployment : 17:08:51 - Resource Microsoft.Logic/workflows 'reporting-reminder-logic-app-dev'
failed with message '{
"error": {
"code": "ConnectionsParameterInvalid",
"message": "The provided API connection parameter 'slack' is missing the required property 'id'."
}
}'
I'm really confused here.
What I tried to deploy it as connection and added $connection parameter definition as well, deployment went through. Nevertheless, when I changed connection to $connections in template parameters, Azure portal throw at me the same validation error.
Any idea what am I doing wrong here?
Thanks
The issue was about different ID in resources -> Microsoft/Web.connections -> properties -> id and slack -> connection. Once this connection ids were the same, the validation passed.
So it's only confusing validation message.

Logic App deployment from Visual Studio 2017 fails

I have created a Logic App in Azure which works fine in Azure itself. So I first recreated the Logic App within VS2017 (with "Azure Logic Apps for Visual Studio" extension installed). The logic app contains a trigger to read from the Azure Servicebus and calls a custom logic app connector action. This custom connector calls a SOAP webservice via an Azure Data Gateway, which runs fine.
Now the issue is that when I try to deploy it from VS2017, it complains in the Output window. The message I get is:
New-AzureRmResourceGroupDeployment : 13:41:34 - Resource MICROSOFT.WEB/CONNECTIONS 'MyCustomConnector' failed with message '{"error": {"code": "ConnectionGatewayFailure","message": "Establishing connection with the service failed with code 'BadRequest'."}}'
My Logic App works great because I receive the SOAP request in my on-premise application.
What I have tried so far:
Checked my permissions, I am now owner of the resource groups (one where the Logic App and custom connector resides in, and one where the Gateway resides in)
Copied content in: "My resource group > Settings > Automation script > Json Template" and pasted it into Visual Studio
Run Visual Studio as Administrator
Added "-DeploymentDebugLogLevel All" to my Deploy-AzureResourceGroup.ps1 file, doesn't show me anything more useful.
Troubleshoot common Azure deployment errors which says for BadRequest: "You sent deployment values that do not match what is expected by Resource Manager. Check the inner status message for help with troubleshooting." I see no inner status message.
Checked Google for a useful answer
But up until now, without success. So hopefully some of you can help me.
Edit: Here's my ARM template, I renamed a few things, hopefully not too much.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"connections_servicebus_name": {
"defaultValue": "servicebus",
"type": "String"
},
"customApis_MyCustomConnector_name": {
"defaultValue": "My-Custom-Connector",
"type": "String"
},
"connections_MyCustomConnector_name": {
"defaultValue": "My-Custom-Connector",
"type": "String"
},
"workflows_MyLogicApp_name": {
"defaultValue": "My-LogicApp",
"type": "String"
},
"workflows_MyLogicApp_id": {
"defaultValue": "/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/MyResourceGroup/providers/Microsoft.Logic/integrationAccounts/My-Integration-Account",
"type": "String"
}
},
"variables": {},
"resources": [
{
"comments": "Generalized from resource: '/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/MyResourceGroup-B/providers/Microsoft.Logic/workflows/My-LogicApp'.",
"type": "Microsoft.Logic/workflows",
"name": "[parameters('workflows_MyLogicApp_name')]",
"apiVersion": "2017-07-01",
"location": "westeurope",
"tags": {},
"scale": null,
"properties": {
"state": "Enabled",
"integrationAccount": {
"id": "[parameters('workflows_MyLogicApp_id')]"
},
"definition": {
"$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"$connections": {
"defaultValue": {},
"type": "Object"
}
},
"triggers": {
},
"actions": {
},
"outputs": {}
},
"parameters": {
"$connections": {
"value": {
"My-Custom-Connector": {
"connectionId": "[resourceId('Microsoft.Web/connections', parameters('connections_MyCustomConnector_name'))]",
"connectionName": "My-Custom-Connector",
"id": "[resourceId('Microsoft.Web/customApis', parameters('customApis_MyCustomConnector_name'))]"
},
"servicebus": {
"connectionId": "[resourceId('Microsoft.Web/connections', parameters('connections_servicebus_name'))]",
"connectionName": "servicebus",
"id": "/subscriptions/00000000-0000-0000-0000-000000000000/providers/Microsoft.Web/locations/westeurope/managedApis/servicebus"
}
}
}
}
},
"dependsOn": [
"[resourceId('Microsoft.Web/connections', parameters('connections_MyCustomConnector_name'))]",
"[resourceId('Microsoft.Web/customApis', parameters('customApis_MyCustomConnector_name'))]",
"[resourceId('Microsoft.Web/connections', parameters('connections_servicebus_name'))]"
]
},
{
"comments": "Generalized from resource: '/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/MyResourceGroup-B/providers/Microsoft.Web/connections/My-Custom-Connector'.",
"type": "Microsoft.Web/connections",
"name": "[parameters('connections_MyCustomConnector_name')]",
"apiVersion": "2016-06-01",
"location": "westeurope",
"scale": null,
"properties": {
"displayName": "LogicAppsCustomConnector-Connection",
"customParameterValues": {},
"api": {
"id": "[resourceId('Microsoft.Web/customApis', parameters('customApis_MyCustomConnector_name'))]"
}
},
"dependsOn": [
"[resourceId('Microsoft.Web/customApis', parameters('customApis_MyCustomConnector_name'))]"
]
},
{
"comments": "Generalized from resource: '/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/MyResourceGroup-B/providers/Microsoft.Web/connections/servicebus'.",
"type": "Microsoft.Web/connections",
"name": "[parameters('connections_servicebus_name')]",
"apiVersion": "2016-06-01",
"location": "westeurope",
"scale": null,
"properties": {
"displayName": "worker_outbound",
"customParameterValues": {},
"api": {
"id": "[concat('/subscriptions/00000000-0000-0000-0000-000000000000/providers/Microsoft.Web/locations/westeurope/managedApis/', parameters('connections_servicebus_name'))]"
}
},
"dependsOn": []
},
{
"comments": "Generalized from resource: '/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/MyResourceGroup-B/providers/Microsoft.Web/customApis/My-Custom-Connector'.",
"type": "Microsoft.Web/customApis",
"name": "[parameters('customApis_MyCustomConnector_name')]",
"apiVersion": "2016-06-01",
"location": "westeurope",
"scale": null,
"properties": {
"connectionParameters": {
"authType": {
"type": "string",
"allowedValues": [
{
"value": "none"
}
],
"uiDefinition": {
"displayName": "Authentication Type",
"description": "Authentication type to connect to your API",
"tooltip": "Authentication type to connect to your API",
"constraints": {
"tabIndex": 1,
"required": "true",
"allowedValues": [
{
"text": "none",
"value": "anonymous"
}
],
"capability": [
"gateway"
]
}
}
},
"gateway": {
"type": "gatewaySetting",
"gatewaySettings": {
"dataSourceType": "CustomConnector",
"connectionDetails": []
},
"uiDefinition": {
"constraints": {
"tabIndex": 4,
"required": "true",
"capability": [
"gateway"
]
}
}
}
},
"brandColor": "#ffffff",
"description": "Calls My SOAP test webservice.",
"displayName": "[parameters('customApis_MyCustomConnector_name')]",
"iconUri":
},
"dependsOn": []
}
]
}

Azure DF CopyFromBlob Failing

I am trying to copy files from an Azure Blob to an Azure Data Lake using a data factory. I keep running into this error and am not finding any information on what the parameter 'baseURI' maps to:
"errorCode": "2200",
"message": "ErrorCode=InvalidParameter,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=The value of the property &apos;baseUri&apos; is invalid: &apos;Value cannot be null.\r\nParameter name: baseUri&apos;.,Source=,''Type=System.ArgumentNullException,Message=Value cannot be null.\r\nParameter name: baseUri,Source=Microsoft.DataTransfer.MsiStoreServiceClient,'",
"failureType": "UserError",
"target": "CopyFromBlob"
I am using Powershell with json files. Anything obvious that I am missing here?
Azure Data Lake Linked Service
{
"name": "<redacted>",
"properties": {
"type": "AzureDataLakeStore",
"typeProperties": {
"dataLakeStoreUri": "<redacted>",
"tenant": "<redacted>",
"subscriptionId": "<redacted>",
"resourceGroupName": "<redacted>"
},
"connectVia": {
"referenceName": "<redacted>",
"type": "IntegrationRuntimeReference"
}
}
}
Azure Blob Linked Service:
{
"name": "<redacted>",
"properties": {
"type": "AzureStorage",
"typeProperties": {
"connectionString": {
"type": "SecureString",
"value": "DefaultEndpointsProtocol=https;AccountName=<redacted>;AccountKey=<redacted>"
}
},
"connectVia": {
"referenceName": "<redacted>",
"type": "IntegrationRuntimeReference"
}
}
}
Data Lake Dataset
{
"name": "<redacted>",
"properties": {
"type": "AzureDataLakeStoreFile",
"linkedServiceName":{
"referenceName": "<redacted>",
"type": "LinkedServiceReference"
},
"typeProperties": {
"folderPath": "<redacted>"
}
}
}
Blob DataSet
{
"name": "<redacted>",
"properties": {
"type": "AzureBlob",
"linkedServiceName": {
"referenceName": "<redacted>",
"type": "LinkedServiceReference"
},
"typeProperties": {
"folderPath": "<redacted>",
}
}
}
Pipeline
{
"name": "<redacted>",
"properties": {
"activities":[
{
"name": "CopyFromBlob",
"type": "Copy",
"inputs": [
{
"referenceName": "<redacted>",
"type": "DatasetReference"
}
],
"outputs": [
{
"referenceName": "<redacted>",
"type": "DatasetReference"
}
],
"typeProperties": {
"source": {
"type": "BlobSource"
},
"sink": {
"type": "AzureDataLakeStoreSink"
}
}
}]
}}
Powershell does the following:
1. Create Data Factory
2. Create Azure Integration Runtime
3. Create Azure Data Lake Linked Service
4. Create Azure Blob Linked Service
5. Create Azure Blob Dataset
6. Create Azure Data Lake Dataset
7. Create pipeline
8. Invoke pipeline
Check your Azure Integration Runtime powershell command - are you defining a location and type? I ran into a very similar error (my null parameter was 'dictionary') when I forgot to define a location for my integration runtime.
Hope this helps!

Resources