Is it possible to assign public static IPs to individual VMs in Azure VMSS? - azure-vm-scale-set

There is a document describing how to allocate a public IP per VM in VMSS: https://learn.microsoft.com/en-us/azure/virtual-machine-scale-sets/virtual-machine-scale-sets-networking#public-ipv4-per-virtual-machine
But it is not clear how to assign public static IP per VM. Is it possible?

Unfortunately, Azure does not provide control of the Public IP allocation method per instance in VMSS. You can see the all supported Properties of
VirtualMachineScaleSetPublicIPAddressConfigurationProperties object in the latest ARM API version.
However, after my validation, when you restart the instance or VMSS scale-in or scale-out, the public IP address of existing instances is not changed. The public IP address of instances will be updated unless you stop the instance of VMSS.
Update
Currently, you can manage it with IpPublicPrefix. Note that IpPublicPrefix requires a standard SKU load balancer and public IP address. Here is a working sample. You can check the public IP address of the instances in VMSS with the REST API.
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"vmSku": {
"type": "string",
"defaultValue": "Standard_A1_v2",
"metadata": {
"description": "Size of VMs in the VM Scale Set."
}
},
"windowsOSVersion": {
"type": "string",
"defaultValue": "2019-Datacenter",
"allowedValues": [
"2008-R2-SP1",
"2012-Datacenter",
"2012-R2-Datacenter",
"2016-Datacenter",
"2019-Datacenter"
],
"metadata": {
"description": "The Windows version for the VM. This will pick a fully patched image of this given Windows version. Allowed values: 2008-R2-SP1, 2012-Datacenter, 2012-R2-Datacenter & 2016-Datacenter, 2019-Datacenter."
}
},
"vmssName": {
"type": "string",
"minLength": 3,
"maxLength": 61,
"metadata": {
"description": "String used as a base for naming resources. Must be 3-61 characters in length and globally unique across Azure. A hash is prepended to this string for some resources, and resource-specific information is appended."
}
},
"instanceCount": {
"type": "int",
"defaultValue": 3,
"minValue": 1,
"maxValue": 100,
"metadata": {
"description": "Number of VM instances (100 or less)."
}
},
"singlePlacementGroup": {
"type": "bool",
"defaultValue": true,
"metadata": {
"description": "When true this limits the scale set to a single placement group, of max size 100 virtual machines. NOTE: If singlePlacementGroup is true, it may be modified to false. However, if singlePlacementGroup is false, it may not be modified to true."
}
},
"adminUsername": {
"type": "string",
"defaultValue": "vmssadmin",
"metadata": {
"description": "Admin username on all VMs."
}
},
"adminPassword": {
"type": "securestring",
"metadata": {
"description": "Admin password on all VMs."
}
},
"location": {
"type": "string",
"defaultValue": "[resourceGroup().location]",
"metadata": {
"description": "Location for all resources."
}
},
"platformFaultDomainCount": {
"type": "int",
"defaultValue": 1,
"metadata": {
"description": "Fault Domain count for each placement group."
}
},
"publicIPPrefixes_pubprefix_name": {
"defaultValue": "vmsspublicprefix",
"type": "string"
}
},
"variables": {
"namingInfix": "[toLower(substring(concat(parameters('vmssName'), uniqueString(resourceGroup().id)), 0, 9))]",
"longNamingInfix": "[toLower(parameters('vmssName'))]",
"addressPrefix": "10.0.0.0/16",
"subnetPrefix": "10.0.0.0/24",
"virtualNetworkName": "[concat(variables('namingInfix'), 'vnet')]",
"publicIPAddressName": "[concat(variables('namingInfix'), 'pip')]",
"subnetName": "[concat(variables('namingInfix'), 'subnet')]",
"loadBalancerName": "[concat(variables('namingInfix'), 'lb')]",
"publicIPAddressID": "[resourceId('Microsoft.Network/publicIPAddresses',variables('publicIPAddressName'))]",
"lbProbeID": "[resourceId('Microsoft.Network/loadBalancers/probes',variables('loadBalancerName'), 'tcpProbe')]",
"natPoolName": "[concat(variables('namingInfix'), 'natpool')]",
"bePoolName": "[concat(variables('namingInfix'), 'bepool')]",
"lbPoolID": "[resourceId('Microsoft.Network/loadBalancers/backendAddressPools',variables('loadBalancerName'),variables('bePoolName'))]",
"natStartPort": 50000,
"natEndPort": 50119,
"natBackendPort": 3389,
"nicName": "[concat(variables('namingInfix'), 'nic')]",
"ipConfigName": "[concat(variables('namingInfix'), 'ipconfig')]",
"frontEndIPConfigID": "[resourceId('Microsoft.Network/loadBalancers/frontendIPConfigurations',variables('loadBalancerName'),'loadBalancerFrontEnd')]",
"osType": {
"publisher": "MicrosoftWindowsServer",
"offer": "WindowsServer",
"sku": "[parameters('windowsOSVersion')]",
"version": "latest"
},
"imageReference": "[variables('osType')]"
},
"resources": [
{
"type": "Microsoft.Network/loadBalancers",
"apiVersion": "2020-06-01",
"name": "[variables('loadBalancerName')]",
"location": "[parameters('location')]",
"dependsOn": [
"[resourceId('Microsoft.Network/publicIPAddresses', variables('publicIPAddressName'))]"
],
"sku": {
"name": "Standard"
},
"properties": {
"frontendIPConfigurations": [
{
"name": "LoadBalancerFrontEnd",
"properties": {
"publicIPAddress": {
"id": "[variables('publicIPAddressID')]",
"name": "Standard"
}
}
}
],
"backendAddressPools": [
{
"name": "[variables('bePoolName')]"
}
],
"inboundNatPools": [
{
"name": "[variables('natPoolName')]",
"properties": {
"frontendIPConfiguration": {
"id": "[variables('frontEndIPConfigID')]"
},
"protocol": "Tcp",
"frontendPortRangeStart": "[variables('natStartPort')]",
"frontendPortRangeEnd": "[variables('natEndPort')]",
"backendPort": "[variables('natBackendPort')]"
}
}
],
"loadBalancingRules": [
{
"name": "LBRule",
"properties": {
"frontendIPConfiguration": {
"id": "[variables('frontEndIPConfigID')]"
},
"backendAddressPool": {
"id": "[variables('lbPoolID')]"
},
"protocol": "Tcp",
"frontendPort": 80,
"backendPort": 80,
"enableFloatingIP": false,
"idleTimeoutInMinutes": 5,
"probe": {
"id": "[variables('lbProbeID')]"
}
}
}
],
"probes": [
{
"name": "tcpProbe",
"properties": {
"protocol": "Tcp",
"port": 80,
"intervalInSeconds": 5,
"numberOfProbes": 2
}
}
]
}
},
{
"type": "Microsoft.Network/publicIPPrefixes",
"apiVersion": "2020-11-01",
"name": "[parameters('publicIPPrefixes_pubprefix_name')]",
"location": "[parameters('location')]",
"sku": {
"name": "Standard",
"tier": "Regional"
},
"properties": {
"prefixLength": 28,
"publicIPAddressVersion": "IPv4",
"ipTags": []
}
},
{
"type": "Microsoft.Compute/virtualMachineScaleSets",
"apiVersion": "2020-06-01",
"name": "[variables('namingInfix')]",
"location": "[parameters('location')]",
"sku": {
"name": "[parameters('vmSku')]",
"tier": "Standard",
"capacity": "[parameters('instanceCount')]"
},
"dependsOn": [
"[resourceId('Microsoft.Network/loadBalancers', variables('loadBalancerName'))]",
"[resourceId('Microsoft.Network/virtualNetworks', variables('virtualNetworkName'))]",
"[resourceId('Microsoft.Network/publicIPPrefixes',parameters('publicIPPrefixes_pubprefix_name'))]"
],
"properties": {
"overprovision": true,
"upgradePolicy": {
"mode": "Automatic"
},
"singlePlacementGroup": "[parameters('singlePlacementGroup')]",
"platformFaultDomainCount": "[parameters('platformFaultDomainCount')]",
"virtualMachineProfile": {
"storageProfile": {
"osDisk": {
"caching": "ReadWrite",
"createOption": "FromImage"
},
"imageReference": "[variables('imageReference')]"
},
"osProfile": {
"computerNamePrefix": "[variables('namingInfix')]",
"adminUsername": "[parameters('adminUsername')]",
"adminPassword": "[parameters('adminPassword')]"
},
"networkProfile": {
"networkInterfaceConfigurations": [
{
"name": "[variables('nicName')]",
"properties": {
"primary": true,
"ipConfigurations": [
{
"name": "[variables('ipConfigName')]",
"properties": {
"subnet": {
"id": "[resourceId('Microsoft.Network/virtualNetworks/subnets', variables('virtualNetworkName'), variables('subnetName'))]"
},
"loadBalancerBackendAddressPools": [
{
"id": "[variables('lbPoolID')]"
}
],
"loadBalancerInboundNatPools": [
{
"id": "[resourceId('Microsoft.Network/loadBalancers/inboundNatPools', variables('loadBalancerName'), variables('natPoolName'))]"
}
],
"publicipaddressconfiguration": {
"name": "pub1",
"properties": {
"idleTimeoutInMinutes": 15,
"publicIPAddressVersion": "IPv4",
"publicIPPrefix":{
"id": "[resourceId('Microsoft.Network/publicIPPrefixes',parameters('publicIPPrefixes_pubprefix_name'))]"
}
}
}
}
}
]
}
}
]
}
}
}
},
{
"type": "Microsoft.Network/publicIPAddresses",
"apiVersion": "2020-06-01",
"name": "[variables('publicIPAddressName')]",
"location": "[parameters('location')]",
"sku": {
"name": "Standard"
},
"properties": {
"publicIPAllocationMethod": "Static",
"dnsSettings": {
"domainNameLabel": "[variables('longNamingInfix')]"
}
}
},
{
"type": "Microsoft.Network/virtualNetworks",
"apiVersion": "2020-06-01",
"name": "[variables('virtualNetworkName')]",
"location": "[parameters('location')]",
"properties": {
"addressSpace": {
"addressPrefixes": [
"[variables('addressPrefix')]"
]
},
"subnets": [
{
"name": "[variables('subnetName')]",
"properties": {
"addressPrefix": "[variables('subnetPrefix')]"
}
}
]
}
},
{
"type": "Microsoft.Insights/autoscaleSettings",
"apiVersion": "2015-04-01",
"name": "autoscalehost",
"location": "[parameters('location')]",
"dependsOn": [
"[resourceId('Microsoft.Compute/virtualMachineScaleSets/', variables('namingInfix'))]"
],
"properties": {
"name": "autoscalehost",
"targetResourceUri": "[resourceId('Microsoft.Compute/virtualMachineScaleSets', variables('namingInfix'))]",
"enabled": true,
"profiles": [
{
"name": "Profile1",
"capacity": {
"minimum": "1",
"maximum": "10",
"default": "1"
},
"rules": [
{
"metricTrigger": {
"metricName": "Percentage CPU",
"metricResourceUri": "[resourceId('Microsoft.Compute/virtualMachineScaleSets', variables('namingInfix'))]",
"timeGrain": "PT1M",
"statistic": "Average",
"timeWindow": "PT5M",
"timeAggregation": "Average",
"operator": "GreaterThan",
"threshold": 50
},
"scaleAction": {
"direction": "Increase",
"type": "ChangeCount",
"value": "1",
"cooldown": "PT5M"
}
},
{
"metricTrigger": {
"metricName": "Percentage CPU",
"metricResourceUri": "[resourceId('Microsoft.Compute/virtualMachineScaleSets', variables('namingInfix'))]",
"timeGrain": "PT1M",
"statistic": "Average",
"timeWindow": "PT5M",
"timeAggregation": "Average",
"operator": "LessThan",
"threshold": 30
},
"scaleAction": {
"direction": "Decrease",
"type": "ChangeCount",
"value": "1",
"cooldown": "PT5M"
}
}
]
}
]
}
}
]
}

Related

Creating/Manipulating Elasticsearch Data View from filebeat.yml

I created some kibana dashboards and they are referencing to a kibana data view. This data view is currently default generated by Filebeat when i'm uploading the filebeat setup, etc. Therefore the data view gets a random Id and my dashboard gets a mising reference since those id's are changing. Now i want to configure somewhere the default created data view since my index- and template-pattern have fixed names and i could create a fixed data view. Then my Dashboards wouldn't go into "missing references". I searched the following sites:
https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-installation-configuration.html
https://github.com/elastic/kibana/issues
I created the Dashboards within the kibana UI and exported them for easier integration within my filebeat project.
My filebeat.yml file
# ============================== Filebeat modules ==============================
filebeat.config.modules:
# Glob pattern for configuration loading
path: ${path.config}/modules.d/*.yml
# Set to true to enable config reloading
reload.enabled: false
# ======================= Elasticsearch template setting =======================
setup.template.settings:
index.number_of_shards: 1
setup.template.name: pac-filebeat-%{[agent.version]}
setup.template.pattern: pac-filebeat-%{[agent.version]}
setup.template.fields: ${path.config}/fields.yml
setup.dashboards.enabled: false
setup.dashboards.directory: ${path.config}\kibana\custom
# =================================== Kibana ===================================
# Starting with Beats version 6.0.0, the dashboards are loaded via the Kibana API.
# This requires a Kibana endpoint configuration.
setup.kibana:
# Kibana Host
# Scheme and port can be left out and will be set to the default (http and 5601)
# In case you specify and additional path, the scheme is required: http://localhost:5601/path
# IPv6 addresses should always be defined as: https://[2001:db8::1]:5601
host: "localhost:5601"
# ---------------------------- Elasticsearch Output ----------------------------
output.elasticsearch:
# Array of hosts to connect to.
hosts: ["localhost:9200"]
# Protocol - either `http` (default) or `https`.
protocol: "http"
index: pac-filebeat-%{[agent.version]}
# ================================= Processors =================================
processors:
- add_host_metadata:
when.not.contains.tags: forwarded
- add_cloud_metadata: ~
- add_docker_metadata: ~
- add_kubernetes_metadata: ~
# ================================== Logging ===================================
This is one example Dashboard.
{
"attributes": {
"description": "",
"hits": 0,
"kibanaSavedObjectMeta": {
"searchSourceJSON": {
"query": {
"query": "",
"language": "kuery"
},
"filter": []
}
},
"optionsJSON": {
"useMargins": true,
"syncColors": false,
"syncTooltips": false,
"hidePanelTitles": false
},
"panelsJSON": [
{
"version": "8.5.3",
"type": "lens",
"gridData": {
"x": 0,
"y": 0,
"w": 9,
"h": 10,
"i": "70779a3d-7880-416d-9736-8a374af8ae41"
},
"panelIndex": "70779a3d-7880-416d-9736-8a374af8ae41",
"embeddableConfig": {
"attributes": {
"title": "",
"visualizationType": "lnsLegacyMetric",
"type": "lens",
"references": [
{
"type": "index-pattern",
"id": "454f0a8e-d5b8-4c8d-9dbe-a1a61782d24d",
"name": "indexpattern-datasource-layer-2a27a515-43d0-4aae-8764-467983de9faf"
},
{
"type": "index-pattern",
"name": "99c2853a-48e3-4042-badf-9c0829cec119",
"id": "454f0a8e-d5b8-4c8d-9dbe-a1a61782d24d"
}
],
"state": {
"visualization": {
"layerId": "2a27a515-43d0-4aae-8764-467983de9faf",
"accessor": "10b01daf-1c3d-46d1-a014-b381ac70019c",
"layerType": "data"
},
"query": {
"query": "",
"language": "kuery"
},
"filters": [
{
"meta": {
"index": "99c2853a-48e3-4042-badf-9c0829cec119",
"alias": "pac.log.level : ERROR",
"type": "custom",
"key": "query",
"value": {
"bool": {
"must": [],
"filter": [
{
"bool": {
"should": [
{
"match": {
"pac.log.level": "ERROR"
}
}
],
"minimum_should_match": 1
}
}
],
"should": [],
"must_not": []
}
},
"disabled": false,
"negate": false
},
"query": {
"bool": {
"must": [],
"filter": [
{
"bool": {
"should": [
{
"match": {
"pac.log.level": "ERROR"
}
}
],
"minimum_should_match": 1
}
}
],
"should": [],
"must_not": []
}
},
"$state": {
"store": "appState"
}
}
],
"datasourceStates": {
"indexpattern": {
"layers": {
"2a27a515-43d0-4aae-8764-467983de9faf": {
"columns": {
"10b01daf-1c3d-46d1-a014-b381ac70019c": {
"label": "Errors",
"customLabel": true,
"dataType": "number",
"operationType": "count",
"isBucketed": false,
"scale": "ratio",
"sourceField": "pac.log.level",
"filter": {
"query": "pac.log.level : ERROR",
"language": "kuery"
},
"params": {
"emptyAsNull": true
}
}
},
"columnOrder": [
"10b01daf-1c3d-46d1-a014-b381ac70019c"
],
"incompleteColumns": {}
}
}
}
},
"internalReferences": [],
"adHocDataViews": {}
}
},
"enhancements": {}
}
},
{
"version": "8.5.3",
"type": "lens",
"gridData": {
"x": 9,
"y": 0,
"w": 12,
"h": 10,
"i": "2bbc2f60-dfb2-4545-a725-9f9bec323e4b"
},
"panelIndex": "2bbc2f60-dfb2-4545-a725-9f9bec323e4b",
"embeddableConfig": {
"attributes": {
"title": "",
"visualizationType": "lnsXY",
"type": "lens",
"references": [
{
"type": "index-pattern",
"id": "454f0a8e-d5b8-4c8d-9dbe-a1a61782d24d",
"name": "indexpattern-datasource-layer-a0f69cd9-6726-4551-9a91-cb330cf0b16d"
}
],
"state": {
"visualization": {
"legend": {
"isVisible": true,
"position": "right"
},
"valueLabels": "hide",
"fittingFunction": "None",
"xTitle": "",
"yTitle": "",
"axisTitlesVisibilitySettings": {
"x": false,
"yLeft": false,
"yRight": true
},
"tickLabelsVisibilitySettings": {
"x": true,
"yLeft": false,
"yRight": true
},
"labelsOrientation": {
"x": 0,
"yLeft": 0,
"yRight": 0
},
"gridlinesVisibilitySettings": {
"x": true,
"yLeft": false,
"yRight": true
},
"preferredSeriesType": "bar_horizontal_percentage_stacked",
"layers": [
{
"layerId": "a0f69cd9-6726-4551-9a91-cb330cf0b16d",
"seriesType": "bar_horizontal_percentage_stacked",
"accessors": [
"cbf5323b-0d81-4411-afa3-91087f062874"
],
"yConfig": [],
"layerType": "data",
"splitAccessor": "a98ba577-d539-4ab3-b798-2bddcb2efd3f",
"palette": {
"type": "palette",
"name": "status"
}
}
]
},
"query": {
"query": "",
"language": "kuery"
},
"filters": [],
"datasourceStates": {
"indexpattern": {
"layers": {
"a0f69cd9-6726-4551-9a91-cb330cf0b16d": {
"columns": {
"a98ba577-d539-4ab3-b798-2bddcb2efd3f": {
"label": "ESA states",
"dataType": "string",
"operationType": "terms",
"scale": "ordinal",
"sourceField": "pac.log.esa.connection",
"isBucketed": true,
"params": {
"size": 3,
"orderBy": {
"type": "column",
"columnId": "cbf5323b-0d81-4411-afa3-91087f062874"
},
"orderDirection": "desc",
"otherBucket": true,
"missingBucket": false,
"parentFormat": {
"id": "terms"
}
},
"customLabel": true
},
"cbf5323b-0d81-4411-afa3-91087f062874": {
"label": "Connected",
"dataType": "number",
"operationType": "unique_count",
"scale": "ratio",
"sourceField": "pac.log.esa.connection",
"isBucketed": false,
"filter": {
"query": "pac.log.esa.connection : Connected",
"language": "kuery"
},
"params": {
"emptyAsNull": true
},
"customLabel": true
}
},
"columnOrder": [
"a98ba577-d539-4ab3-b798-2bddcb2efd3f",
"cbf5323b-0d81-4411-afa3-91087f062874"
],
"incompleteColumns": {}
}
}
}
},
"internalReferences": [],
"adHocDataViews": {}
}
},
"enhancements": {}
}
},
{
"version": "8.5.3",
"type": "lens",
"gridData": {
"x": 21,
"y": 0,
"w": 7,
"h": 10,
"i": "ef851b6b-222a-4b70-a7c3-ec6a85bdf4d1"
},
"panelIndex": "ef851b6b-222a-4b70-a7c3-ec6a85bdf4d1",
"embeddableConfig": {
"attributes": {
"title": "",
"visualizationType": "lnsLegacyMetric",
"type": "lens",
"references": [
{
"type": "index-pattern",
"id": "454f0a8e-d5b8-4c8d-9dbe-a1a61782d24d",
"name": "indexpattern-datasource-layer-d8719702-b241-4931-8bd3-d706f1403c25"
}
],
"state": {
"visualization": {
"layerId": "d8719702-b241-4931-8bd3-d706f1403c25",
"accessor": "312ded46-d0e9-4dcf-9779-b0ca10249a90",
"layerType": "data"
},
"query": {
"query": "",
"language": "kuery"
},
"filters": [],
"datasourceStates": {
"indexpattern": {
"layers": {
"d8719702-b241-4931-8bd3-d706f1403c25": {
"columns": {
"312ded46-d0e9-4dcf-9779-b0ca10249a90": {
"label": "Failed DEO Executions",
"customLabel": true,
"dataType": "number",
"operationType": "count",
"isBucketed": false,
"scale": "ratio",
"sourceField": "pac.log.deo.status",
"filter": {
"query": "pac.log.deo.status : failed",
"language": "kuery"
},
"params": {
"emptyAsNull": true
}
}
},
"columnOrder": [
"312ded46-d0e9-4dcf-9779-b0ca10249a90"
],
"incompleteColumns": {}
}
}
}
},
"internalReferences": [],
"adHocDataViews": {}
}
},
"enhancements": {}
}
},
{
"version": "8.5.3",
"type": "lens",
"gridData": {
"x": 0,
"y": 10,
"w": 14,
"h": 11,
"i": "7bcd2a3e-54c1-4b09-a0b6-d41c339c2ae6"
},
"panelIndex": "7bcd2a3e-54c1-4b09-a0b6-d41c339c2ae6",
"embeddableConfig": {
"attributes": {
"title": "",
"visualizationType": "lnsXY",
"type": "lens",
"references": [
{
"type": "index-pattern",
"id": "454f0a8e-d5b8-4c8d-9dbe-a1a61782d24d",
"name": "indexpattern-datasource-layer-cd91feb0-0927-4465-b0fd-b5bac50af734"
}
],
"state": {
"visualization": {
"title": "Empty XY chart",
"legend": {
"isVisible": true,
"position": "right",
"showSingleSeries": true
},
"valueLabels": "hide",
"preferredSeriesType": "line",
"layers": [
{
"layerId": "cd91feb0-0927-4465-b0fd-b5bac50af734",
"accessors": [
"8ccc17fc-529e-4460-9925-149e77e37dd1"
],
"position": "top",
"seriesType": "line",
"showGridlines": false,
"layerType": "data",
"xAccessor": "43aa69d1-f806-4f87-a2b8-57718e85400a"
}
],
"yTitle": "",
"axisTitlesVisibilitySettings": {
"x": true,
"yLeft": false,
"yRight": true
}
},
"query": {
"query": "",
"language": "kuery"
},
"filters": [],
"datasourceStates": {
"indexpattern": {
"layers": {
"cd91feb0-0927-4465-b0fd-b5bac50af734": {
"columns": {
"43aa69d1-f806-4f87-a2b8-57718e85400a": {
"label": "#timestamp",
"dataType": "date",
"operationType": "date_histogram",
"sourceField": "#timestamp",
"isBucketed": true,
"scale": "interval",
"params": {
"interval": "auto",
"includeEmptyRows": true,
"dropPartials": false
}
},
"8ccc17fc-529e-4460-9925-149e77e37dd1": {
"label": " System cpu usage",
"dataType": "number",
"operationType": "median",
"sourceField": "pac.log.system.monitor.cpu.pct",
"isBucketed": false,
"scale": "ratio",
"params": {
"emptyAsNull": true,
"format": {
"id": "percent",
"params": {
"decimals": 2
}
}
},
"customLabel": true
}
},
"columnOrder": [
"43aa69d1-f806-4f87-a2b8-57718e85400a",
"8ccc17fc-529e-4460-9925-149e77e37dd1"
],
"incompleteColumns": {}
}
}
}
},
"internalReferences": [],
"adHocDataViews": {}
}
},
"enhancements": {}
}
}
],
"timeRestore": false,
"title": "ESA overview",
"version": 1
},
"coreMigrationVersion": "8.5.3",
"id": "d29c4ac0-ab68-11ed-bfcf-939f8111002e",
"migrationVersion": {
"dashboard": "8.5.0"
},
"references": [
{
"id": "454f0a8e-d5b8-4c8d-9dbe-a1a61782d24d",
"name": "70779a3d-7880-416d-9736-8a374af8ae41:indexpattern-datasource-layer-2a27a515-43d0-4aae-8764-467983de9faf",
"type": "index-pattern"
},
{
"id": "454f0a8e-d5b8-4c8d-9dbe-a1a61782d24d",
"name": "70779a3d-7880-416d-9736-8a374af8ae41:99c2853a-48e3-4042-badf-9c0829cec119",
"type": "index-pattern"
},
{
"id": "454f0a8e-d5b8-4c8d-9dbe-a1a61782d24d",
"name": "2bbc2f60-dfb2-4545-a725-9f9bec323e4b:indexpattern-datasource-layer-a0f69cd9-6726-4551-9a91-cb330cf0b16d",
"type": "index-pattern"
},
{
"id": "454f0a8e-d5b8-4c8d-9dbe-a1a61782d24d",
"name": "ef851b6b-222a-4b70-a7c3-ec6a85bdf4d1:indexpattern-datasource-layer-d8719702-b241-4931-8bd3-d706f1403c25",
"type": "index-pattern"
},
{
"id": "454f0a8e-d5b8-4c8d-9dbe-a1a61782d24d",
"name": "7bcd2a3e-54c1-4b09-a0b6-d41c339c2ae6:indexpattern-datasource-layer-cd91feb0-0927-4465-b0fd-b5bac50af734",
"type": "index-pattern"
},
{
"id": "c6a11f70-ab68-11ed-bfcf-939f8111002e",
"name": "tag-c6a11f70-ab68-11ed-bfcf-939f8111002e",
"type": "tag"
}
],
"type": "dashboard",
"updated_at": "2023-02-13T07:14:45.551Z",
"version": "WzE1NTI1NiwzMl0="
}
This would be the used tag inside the above dashboard
{
"attributes": {
"color": "#52bc22",
"description": "",
"name": "custom"
},
"coreMigrationVersion": "8.5.3",
"id": "c6a11f70-ab68-11ed-bfcf-939f8111002e",
"migrationVersion": {
"tag": "8.0.0"
},
"references": [],
"type": "tag",
"updated_at": "2023-02-13T07:39:07.543Z",
"version": "WzE1NTM0NywzMl0="
}
The error message:
error dashboard asset: returned 200 to import file: 1 error: error: missing_references, asset ID=d29c4ac0-ab68-11ed-bfcf-939f8111002e; asset type=dashboard; references=[{Type:index-pattern ID:454f0a8e-d5b8-4c8d-9dbe-a1a61782d24d} {Type:index-pattern ID:454f0a8e-d5b8-4c8d-9dbe-a1a61782d24d} {Type:index-pattern ID:454f0a8e-d5b8-4c8d-9dbe-a1a61782d24d} {Type:index-pattern ID:454f0a8e-d5b8-4c8d-9dbe-a1a61782d24d} {Type:index-pattern ID:454f0a8e-d5b8-4c8d-9dbe-a1a61782d24d}]. Response: {"successCount":1,"success":false,"warnings":[],"successResults":[{"type":"tag","id":"c6a11f70-ab68-11ed-bfcf-939f8111002e","meta":{"title":"custom","icon":"tag"},"overwrite":true}],"errors":[{"id":"d29c4ac0-ab68-11ed-bfcf-939f8111002e","type":"dashboard","meta":{"title":"ESA overview","icon":"dashboardApp"},"error":{"type":"missing_references","references":[{"type":"index-pattern","id":"454f0a8e-d5b8-4c8d-9dbe-a1a61782d24d"},{"type":"index-pattern","id":"454f0a8e-d5b8-4c8d-9dbe-a1a61782d24d"},{"type":"index-pattern","id":"454f0a8e-d5b8-4c8d-9dbe-a1a61782d24d"},{"type":"index-pattern","id":"454f0a8e-d5b8-4c8d-9dbe-a1a61782d24d"},{"type":"index-pattern","id":"454f0a8e-d5b8-4c8d-9dbe-a1a61782d24d"}]},"overwrite":true}]}
kind regardes
You can configure Filebeat to use your own dashboard using the following settings:
setup.dashboards.url
setup.dashboards.file
setup.dashboards.index
setup.dashboards.string_replacements (<-- probably the one you're looking for)

ECS Service stuck on creation

Trying to deploy a CloudFormation template gets stuck with 'AWS::ECS::Service' stuck on 'CREATE_IN_PROGRESS' with status 'Resource creation Initiated'.
Can anyone please tell me if there's a configuration problem with my CloudFormation template?
How would i troubleshoot this?
CloudFormation Template
Thank you!
"FargateTasksCluster": {
"Type": "AWS::ECS::Cluster"
},
"FargateTasksLogGroup": {
"Type": "AWS::Logs::LogGroup"
},
"MytaskTask": {
"Type": "AWS::ECS::TaskDefinition",
"Properties": {
"ContainerDefinitions": [{
"Name": "my-task",
"Image": "mountainpass/infrastructure:helloworld",
"Environment": [],
"LogConfiguration": {
"LogDriver": "awslogs",
"Options": {
"awslogs-region": {
"Fn::Sub": "${AWS::Region}"
},
"awslogs-group": {
"Fn::Sub": "${FargateTasksLogGroup}"
},
"awslogs-stream-prefix": "fargate"
}
}
}],
"Family": "my-task",
"NetworkMode": "awsvpc",
"ExecutionRoleArn": {
"Fn::Sub": "arn:aws:iam::${AWS::AccountId}:role/ecsTaskExecutionRole"
},
"TaskRoleArn": {
"Fn::Sub": "${IamRoleLambdaExecution}"
},
"RequiresCompatibilities": ["FARGATE"],
"Memory": "0.5GB",
"Cpu": 256
}
},
"MytaskService": {
"Type": "AWS::ECS::Service",
"Properties": {
"Cluster": {
"Fn::Sub": "${FargateTasksCluster}"
},
"LaunchType": "FARGATE",
"ServiceName": "my-task",
"DesiredCount": 1,
"TaskDefinition": {
"Fn::Sub": "${MytaskTask}"
},
"NetworkConfiguration": {
"AwsvpcConfiguration": {
"AssignPublicIp": "DISABLED",
"SecurityGroups": [],
"Subnets": ["subnet-2cc37974", "subnet-3c959d5b", "subnet-49d7d200"]
}
}
}
}

switch from WebSocket to RawSocket Transport

I am new to crossbar/autobahn framework and to network programming in general, recently at work we would like to experiment to move from WebSocket transport to RawSocket Transport with using MessagePackSerializer and do performance comparison. I am requesting to the framework developers to please guide me to convert following config to use raw socket. will using auth settings from ws section should work ?
{
"version": 2,
"workers": [
{
"type": "router",
"options": {
"pythonpath": ["../components"]
},
"realms": [
{
"name": "realm1",
"roles": [
{
"name": "system",
"authorizer": "com.XXXX.auth.authorize"
},
{
"name": "user",
"authorizer": "com.XXXX.auth.authorize"
},
{
"name": "service",
"authorizer": "com.XXXX.auth.authorize"
}
]
}
],
"transports": [
{
"type": "rawsocket",
"endpoint": {
"type": "unix",
"path": "/tmp/crossbar-socket"
}
},
{
"type": "web",
"endpoint": {
"type": "tcp",
"port": 8080
},
"paths": {
"status": {
"type": "resource",
"classname": "health_web_indicator.run",
"extra": {
"DVMT_VERSION": "2.0.1",
"HEALTHY_INDICATOR": ":) DVMT UP",
"UNHEALTHY_INDICATOR": ":( DVMT DOWN",
"MSSQL_DRIVER": "FreeTDS",
"MSSQL_SERVER": "XXXX.dv.local",
"MSSQL_PORT": "1515",
"MSSQL_DATABASE": "XXXX",
"MSSQL_USER": "XXXX",
"MSSQL_PASSWORD": "XXXX",
"SUPERVISOR_PROCESSES": ["XXXX", "XXXX", "XXXX", "XXXX", "XXXX"],
"SUPERVISOR_SERVER_SOCKET": "/crossbar/supervisord.sock"
}
},
"ws": {
"type": "websocket",
"auth": {
"anonymous": {
"type": "dynamic",
"authenticator": "com.XXXX.auth.authenticate"
},
"ticket": {
"type": "dynamic",
"authenticator": "com.XXXX.auth.authenticate"
}
},
"debug": true,
"options": {
"auto_ping_interval": 0,
"auto_ping_timeout": 60000,
"auto_ping_size": 4,
"compression": {
"deflate": {
"request_no_context_takeover": false,
"request_max_window_bits": 11,
"no_context_takeover": false,
"max_window_bits": 11,
"memory_level": 4
}
}
}
}
}
}
],
"components": [
{
"id": "auth",
"type": "class",
"classname": "auth.AuthComponent",
"realm": "realm1",
"role": "trusted",
"extra": {
"prefix": "com.XXXX.auth",
"system_role": "system",
"anonymous_role": "anonymous",
"mssql": {
"driver": "FreeTDS",
"server": "XXXX.dv.local",
"port": "1515",
"database": "XXXX",
"uid": "XXXX",
"pwd": "XXXX",
"autocommit": true
},
"heartbeat_rate": 30.0,
"ldap_api_url": "http://XXXX:80/rssapi/dvmsapi/admin/GetUserLdapGroups/{username}",
"enforce_cert_match": false,
"cert_sub_pattern": "[\\s_-]"
}
}
]
}
]
}
also will it be easy enough for you to please give me an example for using autobahnpython as a client for rawsocket transport. https://crossbar.io/docs/RawSocket-Transport/#example---connecting-transport

SAM template for API Gateway has errors within Visual Studio (as part of Serverless Application solution)

I am trying to create a SAM template (serverless.template) in visual studio to publish my API Gateway. I have a couple of errors being produced within the template validation (in Visual Studio) that I'm unable to resolve. When published, the stack deploys, but there is no usage plan(s) or api key(s) created (or rather, they are created but not viewable in the console, they come up as 'invalid reference' if you try to view them via Cloudformation > Resources, and they don't show up at all in the API Gateway console).
The errors are:
ServerlessRestApiDeployment26aad1646f is an unknown reference
"ServerlessRestApiProdStage": {
"Type": "AWS::ApiGateway::Stage",
"Properties": {
"DeploymentId": {
"Ref": "ServerlessRestApiDeployment26aad1646f"
},
"RestApiId": {
"Ref": "ServerlessRestApi"
},
"StageName": "Prod"
}
},
And
ServerlessRestApiProdStage is an invalid type for this reference
"APIGatewayHeartInHandKey": {
"Type": "AWS::ApiGateway::ApiKey",
"DependsOn": [
"ServerlessRestApi",
"ServerlessRestApiProdStage"
],
"Properties": {
"Name": "HeartInHandApiKey",
"Description": "Api Key for Heart In Hand",
"Enabled": true,
"GenerateDistinctId": true,
"StageKeys": [
{
"RestApiId": {
"Ref": "ServerlessRestApi"
},
"StageName": {
"Ref": "ServerlessRestApiProdStage"
}
}
]
}
},
The full SAM template shown below.
{
"AWSTemplateFormatVersion": "2010-09-09",
"Transform": "AWS::Serverless-2016-10-31",
"Description": "API Gateway to access InSite data-store",
"Resources": {
"Get": {
"Type": "AWS::Serverless::Function",
"Properties": {
"VpcConfig": {
"SecurityGroupIds": [
"sg-111a1476"
],
"SubnetIds": [
"subnet-3029a769",
"subnet-5ec0b928"
]
},
"Handler": "AWSServerlessInSiteDataGw::AWSServerlessInSiteDataGw.Functions::Get",
"Runtime": "dotnetcore2.0",
"CodeUri": "",
"MemorySize": 256,
"Timeout": 30,
"Role": null,
"Policies": [
"AWSLambdaBasicExecutionRole",
"AWSLambdaVPCAccessExecutionRole",
"AmazonSSMFullAccess"
],
"Events": {
"PutResource": {
"Type": "Api",
"Properties": {
"Path": "/",
"Method": "GET"
}
}
}
}
},
"GetTableBasic": {
"Type": "AWS::Serverless::Function",
"Properties": {
"VpcConfig": {
"SecurityGroupIds": [
"sg-111a1476"
],
"SubnetIds": [
"subnet-3029a769",
"subnet-5ec0b928"
]
},
"Handler": "AWSServerlessInSiteDataGw::AWSServerlessInSiteDataGw.Functions::GetTableBasic",
"Runtime": "dotnetcore2.0",
"CodeUri": "",
"MemorySize": 256,
"Timeout": 30,
"Role": null,
"Policies": [
"AWSLambdaBasicExecutionRole",
"AWSLambdaVPCAccessExecutionRole",
"AmazonSSMFullAccess"
],
"Events": {
"PutResource": {
"Type": "Api",
"Properties": {
"Path": "/tables/{tableid}/{columnid}",
"Method": "GET"
}
}
}
}
},
"GetColumnList": {
"Type": "AWS::Serverless::Function",
"Properties": {
"VpcConfig": {
"SecurityGroupIds": [
"sg-111a1476"
],
"SubnetIds": [
"subnet-3029a769",
"subnet-5ec0b928"
]
},
"Handler": "AWSServerlessInSiteDataGw::AWSServerlessInSiteDataGw.Functions::GetColumnList",
"Runtime": "dotnetcore2.0",
"CodeUri": "",
"MemorySize": 256,
"Timeout": 30,
"Role": null,
"Policies": [
"AWSLambdaBasicExecutionRole",
"AWSLambdaVPCAccessExecutionRole",
"AmazonSSMFullAccess"
],
"Events": {
"PutResource": {
"Type": "Api",
"Properties": {
"Path": "/list/columns/{tableid}",
"Method": "GET"
}
}
}
}
},
"GetTableList": {
"Type": "AWS::Serverless::Function",
"Properties": {
"VpcConfig": {
"SecurityGroupIds": [
"sg-111a1476"
],
"SubnetIds": [
"subnet-3029a769",
"subnet-5ec0b928"
]
},
"Handler": "AWSServerlessInSiteDataGw::AWSServerlessInSiteDataGw.Functions::GetTableList",
"Runtime": "dotnetcore2.0",
"CodeUri": "",
"MemorySize": 256,
"Timeout": 30,
"Role": null,
"Policies": [
"AWSLambdaBasicExecutionRole",
"AWSLambdaVPCAccessExecutionRole",
"AmazonSSMFullAccess"
],
"Events": {
"PutResource": {
"Type": "Api",
"Properties": {
"Path": "/list/tables",
"Method": "GET"
}
}
}
}
},
"PostClickCollectNotification": {
"Type": "AWS::Serverless::Function",
"Properties": {
"VpcConfig": {
"SecurityGroupIds": [
"sg-111a1476"
],
"SubnetIds": [
"subnet-3029a769",
"subnet-5ec0b928"
]
},
"Handler": "AWSServerlessInSiteDataGw::AWSServerlessInSiteDataGw.Functions::PostClickCollectNotification",
"Runtime": "dotnetcore2.0",
"CodeUri": "",
"MemorySize": 256,
"Timeout": 30,
"Role": null,
"Policies": [
"AWSLambdaBasicExecutionRole",
"AWSLambdaVPCAccessExecutionRole",
"AmazonSSMFullAccess"
],
"Events": {
"PutResource": {
"Type": "Api",
"Properties": {
"Path": "/datagw/general/webhook/ccnotify",
"Method": "POST"
}
}
}
}
},
"PostClickCollectStockUpdate": {
"Type": "AWS::Serverless::Function",
"Properties": {
"VpcConfig": {
"SecurityGroupIds": [
"sg-111a1476"
],
"SubnetIds": [
"subnet-3029a769",
"subnet-5ec0b928"
]
},
"Handler": "AWSServerlessInSiteDataGw::AWSServerlessInSiteDataGw.Functions::PostClickCollectStockUpdate",
"Runtime": "dotnetcore2.0",
"CodeUri": "",
"MemorySize": 256,
"Timeout": 30,
"Role": null,
"Policies": [
"AWSLambdaBasicExecutionRole",
"AWSLambdaVPCAccessExecutionRole",
"AmazonSSMFullAccess"
],
"Events": {
"PutResource": {
"Type": "Api",
"Properties": {
"Path": "/datagw/general/post/sohupdate",
"Method": "POST"
}
}
}
}
},
"GetTableResponse": {
"Type": "AWS::Serverless::Function",
"Properties": {
"VpcConfig": {
"SecurityGroupIds": [
"sg-111a1476"
],
"SubnetIds": [
"subnet-3029a769",
"subnet-5ec0b928"
]
},
"Handler": "AWSServerlessInSiteDataGw::AWSServerlessInSiteDataGw.Functions::GetTableResponse",
"Runtime": "dotnetcore2.0",
"CodeUri": "",
"MemorySize": 256,
"Timeout": 30,
"Role": null,
"Policies": [
"AWSLambdaBasicExecutionRole",
"AWSLambdaVPCAccessExecutionRole",
"AmazonSSMFullAccess"
],
"Events": {
"PutResource": {
"Type": "Api",
"Properties": {
"Path": "datagw/general/table/get/{tableid}",
"Method": "GET"
}
}
}
}
},
"ServerlessRestApi": {
"Type": "AWS::ApiGateway::RestApi",
"Properties": {
"Description":"This is a placeholder for the description of this web api",
"ApiKeySourceType":"HEADER",
"Body": {
"info": {
"version": "1.0",
"title": {
"Ref": "AWS::StackName"
}
},
"paths": {
"/list/tables": {
"get": {
"x-amazon-apigateway-integration": {
"httpMethod": "POST",
"type": "aws_proxy",
"uri": {
"Fn::Sub": "arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${GetTableList.Arn}/invocations"
}
},
"responses": {}
}
},
"/list/columns/{tableid}": {
"get": {
"x-amazon-apigateway-integration": {
"httpMethod": "POST",
"type": "aws_proxy",
"uri": {
"Fn::Sub": "arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${GetColumnList.Arn}/invocations"
}
},
"responses": {}
}
},
"datagw/general/table/get/{tableid}": {
"get": {
"x-amazon-apigateway-integration": {
"httpMethod": "POST",
"type": "aws_proxy",
"uri": {
"Fn::Sub": "arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${GetTableResponse.Arn}/invocations"
}
},
"responses": {}
}
},
"/": {
"get": {
"x-amazon-apigateway-integration": {
"httpMethod": "POST",
"type": "aws_proxy",
"uri": {
"Fn::Sub": "arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${Get.Arn}/invocations"
}
},
"responses": {}
}
},
"/tables/{tableid}/{columnid}": {
"get": {
"x-amazon-apigateway-integration": {
"httpMethod": "POST",
"type": "aws_proxy",
"uri": {
"Fn::Sub": "arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${GetTableBasic.Arn}/invocations"
}
},
"responses": {}
}
}
},
"swagger": "2.0"
}
}
},
"ServerlessRestApiProdStage": {
"Type": "AWS::ApiGateway::Stage",
"Properties": {
"DeploymentId": {
"Ref": "ServerlessRestApiDeployment26aad1646f"
},
"RestApiId": {
"Ref": "ServerlessRestApi"
},
"StageName": "Prod"
}
},
"CustomLambdaExecutionRole": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"lambda.amazonaws.com"
]
},
"Action": [
"sts:AssumeRole"
]
}
]
},
"Policies": [
{
"PolicyName": "lambdaAccessApiKeys",
"PolicyDocument": {
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"apigateway:GET"
],
"Resource": {
"Fn::Sub": [
"arn:aws:apigateway:ap-southeast-2::/apikeys/${__keyId__}",
{
"__keyId__": {
"Ref": "APIGatewayHeartInHandKey"
}
}
]
}
}
]
}
}
]
}
},
"GetApiKeyValueLambdaFunction": {
"Type": "AWS::Lambda::Function",
"Properties": {
"Code": {
"ZipFile": {
"Fn::Join": [
"\n",
[
"import json",
"import boto3",
"client = boto3.client('apigateway')",
"def lambda_handler(event, context):",
" response = client.get_api_key(",
" apiKey= event['apiKeyId'],",
" includeValue = True",
")",
" return {",
" 'statusCode': 200,",
" 'body': response['value']}"
]
]
}
},
"Handler": "index.lambda_handler",
"Runtime": "python3.6",
"Timeout": 30,
"Role": {
"Fn::GetAtt": [
"CustomLambdaExecutionRole",
"Arn"
]
}
}
},
"APIGatewayUsagePlanInternal": {
"Type": "AWS::ApiGateway::UsagePlan",
"Properties": {
"ApiStages": [
{
"ApiId": {
"Ref": "ServerlessRestApi"
},
"Stage": {
"Ref": "ServerlessRestApiProdStage"
}
}
],
"Description": "Internal Apps Usage Plan",
"UsagePlanName": "Insite-datagw-InternalAppPlan"
}
},
"APIGatewayUsagePlanExternal": {
"Type": "AWS::ApiGateway::UsagePlan",
"Properties": {
"ApiStages": [
{
"ApiId": {
"Ref": "ServerlessRestApi"
},
"Stage": {
"Ref": "ServerlessRestApiProdStage"
}
}
],
"Description": "External Apps Usage Plan",
"UsagePlanName": "InSite-datagw-ExternalAppPlan"
}
},
"APIGatewayHeartInHandKey": {
"Type": "AWS::ApiGateway::ApiKey",
"DependsOn": [
"ServerlessRestApi",
"ServerlessRestApiProdStage"
],
"Properties": {
"Name": "HeartInHandApiKey",
"Description": "Api Key for Heart In Hand",
"Enabled": true,
"GenerateDistinctId": true,
"StageKeys": [
{
"RestApiId": {
"Ref": "ServerlessRestApi"
},
"StageName": {
"Ref": "ServerlessRestApiProdStage"
}
}
]
}
},
"LinkHeartInHandKey": {
"Type": "AWS::ApiGateway::UsagePlanKey",
"Properties": {
"KeyId": {
"Ref": "APIGatewayHeartInHandKey"
},
"KeyType": "API_KEY",
"UsagePlanId": {
"Ref": "APIGatewayUsagePlanInternal"
}
}
},
"APIGatewayPricelineSiteKey": {
"Type": "AWS::ApiGateway::ApiKey",
"DependsOn": [
"ServerlessRestApi",
"ServerlessRestApiProdStage"
],
"Properties": {
"Name": "PricelineSiteApiKey",
"Description": "Api Key for Priceline Website",
"Enabled": true,
"GenerateDistinctId": true,
"StageKeys": [
{
"RestApiId": {
"Ref": "ServerlessRestApi"
},
"StageName": {
"Ref": "ServerlessRestApiProdStage"
}
}
]
}
},
"LinkPricelineSiteKey": {
"Type": "AWS::ApiGateway::UsagePlanKey",
"Properties": {
"KeyId": {
"Ref": "APIGatewayPricelineSiteKey"
},
"KeyType": "API_KEY",
"UsagePlanId": {
"Ref": "APIGatewayUsagePlanInternal"
}
}
}
},
"Outputs": {
"ApiURL": {
"Description": "API endpoint URL for Prod environment",
"Value": {
"Fn::Sub": "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/"
}
}
}
}
it looks like you're trying to use features of the AWS::Serverless::Api resource from SAM without defining an AWS::Serverless::Api in your template.
In order to fix the issues you've brought up, it looks like you need to:
Remove the ServerlessRestApiProdStage resource
Sam will generate this resource (and the deployment resource) for you if you use an AWS::Serverless::Api resource.
Convert your AWS::ApiGateway::RestApi resource into an AWS::Serverless::Api resource:
Remove ApiKeySourceType property and add "x-amazon-apigateway-api-key-source" : "HEADER", to the swagger,
Change the Type from AWS::ApiGateway::RestApi to AWS::Serverless::Api
"ServerlessRestApi": {
"Type": "AWS::Serverless::RestApi",
"Properties": {
"Description":"This is a placeholder for the description of this web api",
"Body": {
"info": {
"version": "1.0",
"title": {
"Ref": "AWS::StackName"
}
},
"x-amazon-apigateway-api-key-source" : "HEADER",
"paths": {
...

JSONata, pulling data from nested objects

I'm new to JSONata and so still getting my head around it. I need to pull data out of a nested object, for example from :
{
"transaction": {
"id": "de112b4b-82e2-4172-a89f-68724c90b692"
},
"domain": {
"id": "realworld"
},
"listing": {
"spanner": {
"information": {
"type": {
"VENDOR": "Charlie"
},
"variables": {
},
"uid": "08_spanner",
"mode": {
"store": "bob"
},
"version": "1",
"name": "Harrold"
}
},
"hammer": {
"information": {
"type": {
"VENDOR": "Cliff"
},
"variables": {
},
"uid": "08_hammer",
"mode": {
"store": "steve"
},
"version": "1",
"name": "Mike"
}
},
"wrench": {
"information": {
"type": {
"VENDOR": "Dave"
},
"variables": {
},
"uid": "08_wrench",
"mode": {
"store": "bob"
},
"version": "1",
"name": "Kent"
}
}
}
}
... I need to pull out the listing data with it's "information", but without the information key. So the result would look like:
{
"spanner": {
"type": {
"VENDOR": "Charlie"
},
"variables": {
},
"uid": "08_spanner",
"mode": {
"store": "bob"
},
"version": "1",
"name": "Harrold"
},
"hammer": {
"type": {
"VENDOR": "Cliff"
},
"variables": {
},
"uid": "08_hammer",
"mode": {
"store": "steve"
},
"version": "1",
"name": "Mike"
},
"wrench": {
"type": {
"VENDOR": "Dave"
},
"variables": {
},
"uid": "08_wrench",
"mode": {
"store": "bob"
},
"version": "1",
"name": "Kent"
}
}
I have been messing around on http://try.jsonata.org/ and can see how powerful the library is but so far no success at achieving this goal. Any and all help is appreciated :).
The following expression will do this:
listing.$each(function($value, $key) {
{ $key: $value.information }
}) ~> $merge()
See http://try.jsonata.org/S17Erm85z

Resources