Jolt transformation for spatial data (from geojson to arcGIS json) - transformation

I'm new to Jolt transformation and am trying to write a spec for spatial data. I'm unable to get the
expected output especially with the data for the coordinates. Any help would greatly be appreciated.
Below is the input I get:
{
"type": "Feature Collection",
"c rs": {
"type": "name",
"properties": {
"name": "4326"
}
},
"features": [
{
"type": "Feature",
"id": 1,
"geometry": {
"type": "Point",
"coordinates": [
-122.682207049,
45.526159497
]
},
"properties": {
"F ID": 1,
"Place Name": "Boyd Coffee",
"Place Address": "404 NW 11th Ave Portland Oregon",
"Score": 100,
"latitude": 45.526159497,
"longitude": -122.682207049
}
},
{
"type": "Feature",
"id": 2,
"geometry": {
"type": "Point",
"coordinates": [
-122.677518466,
45.525246078
]
},
"properties": {
"F ID": 2,
"Place Name": "John's Coffee Shop",
"Place Address": "301 NW Broadway St Portland Oregon",
"Score": 100,
"lat": 45.525246078,
"l on": -122.677518466
}
},
{
"type": "Feature",
"id": 3,
"geometry": {
"type": "Point",
"coordinates": [
-122.68287749,
45.526496967
]
},
"properties": {
"F ID": 3,
"Place Name": "Starbucks",
"Place Address": "1134 NW St Portland Oregon",
"Score": 100,
"latitude": 45.526496967,
"longitude": -122.68287749
}
}
]
}
The expected output is -
"features": [
{
"attributes": {
"F ID": 1,
"Place Name": "Boyd Coffee",
"Place Address": "404 NW 11th Ave Portland Oregon",
"Score": 100,
"latitude": 45.526159497,
"longitude": -122.682207049
},
"geometry": {
"x": -122.68220704900002,
"y": 45.526159496999998
}
},
{
"attributes": {
"FID": 2,
"Place Name": "John's Coffee Shop",
"Place Address": "301 NW Broadway St Portland Oregon",
"Score": 100,
"latitude": 45.525246078,
"longitude": -122.677518466
},
"geometry": {
"x": -122.677518466,
"y": 45.52524607799999
}
}
]
I've come up with spec as below, and not getting the expected out. Can anyone help please?
The spec is:
[
{
"operation": "shift",
"spec": {
"features": {
"*": {
"properties": "features[&1].attributes",
"geometry": {
"coordinates": {
"0": "features[&].geometry.x",
"1": "features[&].geometry.y"
}
}
}
}
}
}
]

Change the below in your spec
"0": "features[&].geometry.x", -> "0": "features[&3].geometry.x",
"1": "features[&].geometry.y", -> "1": "features[&3].geometry.y",
[
{
"operation": "shift",
"spec": {
"features": {
"*": {
"properties": "features[&1].attributes",
"geometry": {
"coordinates": {
"0": "features[&3].geometry.x",
"1": "features[&3].geometry.y"
}
}
}
}
}
}
]

Related

Draw tree Layout chart in vega

I want to have a tree chart of my data using vega in kibana 7.9.0, but I don't know how to write the query for that.
the below code is an example of tree chart from github website. I want such layout for my own index which I have it in kibanan.
Help me how to do that.
tree chart example
Sample tree chart code:
{
"$schema": "https://vega.github.io/schema/vega/v5.json",
"description": "An example of Cartesian layouts for a node-link diagram of hierarchical data.",
"width": 1000,
"height": 1600,
"padding": 5,
"signals": [
{
"name": "labels", "value": true,
"bind": {"input": "checkbox"}
},
{
"name": "layout", "value": "tidy",
"bind": {"input": "radio", "options": ["tidy", "cluster"]}
},
{
"name": "links", "value": "diagonal",
"bind": {
"input": "select",
"options": ["line", "curve", "diagonal", "orthogonal"]
}
},
{
"name": "separation", "value": false,
"bind": {"input": "checkbox"}
}
],
"data": [
{
"name": "tree",
"url": "data/flare.json",
"transform": [
{
"type": "stratify",
"key": "id",
"parentKey": "parent"
},
{
"type": "tree",
"method": {"signal": "layout"},
"size": [{"signal": "height"}, {"signal": "width - 100"}],
"separation": {"signal": "separation"},
"as": ["y", "x", "depth", "children"]
}
]
},
{
"name": "links",
"source": "tree",
"transform": [
{ "type": "treelinks" },
{
"type": "linkpath",
"orient": "horizontal",
"shape": {"signal": "links"}
}
]
}
],
"scales": [
{
"name": "color",
"type": "linear",
"range": {"scheme": "magma"},
"domain": {"data": "tree", "field": "depth"},
"zero": true
}
],
"marks": [
{
"type": "path",
"from": {"data": "links"},
"encode": {
"update": {
"path": {"field": "path"},
"stroke": {"value": "#ccc"}
}
}
},
{
"type": "symbol",
"from": {"data": "tree"},
"encode": {
"enter": {
"size": {"value": 100},
"stroke": {"value": "#fff"}
},
"update": {
"x": {"field": "x"},
"y": {"field": "y"},
"fill": {"scale": "color", "field": "depth"}
}
}
},
{
"type": "text",
"from": {"data": "tree"},
"encode": {
"enter": {
"text": {"field": "name"},
"fontSize": {"value": 9},
"baseline": {"value": "middle"}
},
"update": {
"x": {"field": "x"},
"y": {"field": "y"},
"dx": {"signal": "datum.children ? -7 : 7"},
"align": {"signal": "datum.children ? 'right' : 'left'"},
"opacity": {"signal": "labels ? 1 : 0"}
}
}
}
]
}
I recommend going and learning the Kibana Vega interaction, Vega spec and elasticsearch search api. This is too broad of a question, you are basically asking someone to do the work for you which is easily deducible from the the documentation.
https://www.elastic.co/guide/en/kibana/current/vega.html
https://vega.github.io/vega/docs/
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html

Add extension to R4 Patient

How do I add the below to an R4 Patient?
"search": {
"extension": [
{
"url": "http://hl7.org/fhir/StructureDefinition/match-grade",
"valueCode": "certain"
}
],
"mode": "match",
"score": 1.0
}
The objects that were under stu2 and stu3 (I think) have been deprecated.
DSTU3 and Later
// Create an example patient
Patient patient = new Patient();
patient.addIdentifier().setUse(Identifier.IdentifierUse.OFFICIAL).setSystem("urn:example").setValue("7000135");
// Create an extension
Extension ext = new Extension();
ext.setUrl("http://example.com/extensions#someext");
ext.setValue(new DateTimeType("2011-01-02T11:13:15"));
// Add the extension to the resource
patient.addExtension(ext);
Full "response" json from patient match below (sample data, not real data !! )
{
"resourceType": "Bundle",
"id": "94897c77-4120-482f-bdb4-4c8b9a71ade1",
"meta": {
"versionId": "1",
"lastUpdated": "2020-11-24T08:45:49.445-05:00"
},
"type": "searchset",
"total": 1,
"link": [
{
"relation": "self",
"url": "http://wildfhir4.aegis.net/fhir4-0-0/Patient/$match?"
}
],
"entry": [
{
"fullUrl": "http://wildfhir4.aegis.net/fhir4-0-0/Patient/9126db824dcf4d5ab3f68e63bd5eaced",
"resource": {
"resourceType": "Patient",
"id": "9126db824dcf4d5ab3f68e63bd5eaced",
"meta": {
"versionId": "1",
"lastUpdated": "2020-09-03T09:03:28.193-04:00",
"tag": [
{
"system": "http://terminology.hl7.org/CodeSystem/v3-ActReason",
"code": "HTEST",
"display": "test health data"
}
]
},
"text": {
"status": "generated",
"div": "<div xmlns=\"http://www.w3.org/1999/xhtml\"><table><tbody><tr><td>Name</td><td>Peter James \n <b>Chalmers</b> ("Jim") \n </td></tr><tr><td>Address</td><td>534 Erewhon, Pleasantville, Vic, 3999</td></tr><tr><td>Contacts</td><td>Home: unknown. Work: (03) 5555 6473</td></tr><tr><td>Id</td><td>MRN: 12345 (Acme Healthcare)</td></tr></tbody></table></div>"
},
"identifier": [
{
"use": "usual",
"type": {
"coding": [
{
"system": "http://terminology.hl7.org/CodeSystem/v2-0203",
"code": "MR"
}
]
},
"system": "urn:oid:1.2.36.146.595.217.0.1",
"value": "12345",
"period": {
"start": "2001-05-06"
},
"assigner": {
"display": "Acme Healthcare"
}
}
],
"active": true,
"name": [
{
"use": "official",
"family": "Chalmers",
"given": [
"Peter",
"James"
]
},
{
"use": "usual",
"given": [
"Jim"
]
},
{
"use": "maiden",
"family": "Windsor",
"given": [
"Peter",
"James"
],
"period": {
"end": "2002"
}
}
],
"telecom": [
{
"use": "home"
},
{
"system": "phone",
"value": "(03) 5555 6473",
"use": "work",
"rank": 1
},
{
"system": "phone",
"value": "(03) 3410 5613",
"use": "mobile",
"rank": 2
},
{
"system": "phone",
"value": "(03) 5555 8834",
"use": "old",
"period": {
"end": "2014"
}
}
],
"gender": "male",
"birthDate": "1974-12-25",
"_birthDate": {
"extension": [
{
"url": "http://hl7.org/fhir/StructureDefinition/patient-birthTime",
"valueDateTime": "1974-12-25T14:35:45-05:00"
}
]
},
"deceasedBoolean": false,
"address": [
{
"use": "home",
"type": "both",
"text": "534 Erewhon St PeasantVille, Rainbow, Vic 3999",
"line": [
"534 Erewhon St"
],
"city": "PleasantVille",
"district": "Rainbow",
"state": "Vic",
"postalCode": "3999",
"period": {
"start": "1974-12-25"
}
}
],
"contact": [
{
"relationship": [
{
"coding": [
{
"system": "http://terminology.hl7.org/CodeSystem/v2-0131",
"code": "N"
}
]
}
],
"name": {
"family": "du Marché",
"_family": {
"extension": [
{
"url": "http://hl7.org/fhir/StructureDefinition/humanname-own-prefix",
"valueString": "VV"
}
]
},
"given": [
"Bénédicte"
]
},
"telecom": [
{
"system": "phone",
"value": "+33 (237) 998327"
}
],
"address": {
"use": "home",
"type": "both",
"line": [
"534 Erewhon St"
],
"city": "PleasantVille",
"district": "Rainbow",
"state": "Vic",
"postalCode": "3999",
"period": {
"start": "1974-12-25"
}
},
"gender": "female",
"period": {
"start": "2012"
}
}
],
"managingOrganization": {
"reference": "Organization/1"
}
},
"search": {
"extension": [
{
"url": "http://hl7.org/fhir/StructureDefinition/match-grade",
"valueCode": "certain"
}
],
"mode": "match",
"score": 1.0
}
}
]
}
APPEND: I found this:
https://hapifhir.io/hapi-fhir/apidocs/hapi-fhir-structures-r4/org/hl7/fhir/r4/model/codesystems/MatchGrade.html
That extension isn't allowed on Patient. It's defined with a context of Bundle.entry.search. It's not a characteristic of the patient themselves, it's a characteristic of a particular entry in a search result based on a specific set of search criteria. The same Patient instance might well be a good result on one search and a poor result on a different search. So to add the extension, you'd have to add it on a 'search' object, not patient.
Ok.
Here it is. The primary objects were inner/nested objects under Bundle....thus why I didn't see them early. (example : Bundle.BundleEntrySearchComponent)
// a java example with the import statements. #shocking!
//import org.hl7.fhir.r4.model.Bundle;
//import org.hl7.fhir.r4.model.CodeType;
//import org.hl7.fhir.r4.model.Extension;
//import org.hl7.fhir.r4.model.Patient;
//import org.hl7.fhir.r4.model.codesystems.MatchGrade;
//import java.util.Collections;
Bundle innerBundle = new Bundle();
innerBundle.setType(Bundle.BundleType.SEARCHSET);
innerBundle.setTotal(1);
Patient pat = new Patient();
HumanName hn = new HumanName();
hn.setFamily("Smith");
p.setName(Collections.singletonList(hn));
Bundle.BundleEntrySearchComponent besc = new Bundle.BundleEntrySearchComponent();
besc.setScore(0.88);
besc.setMode(Bundle.SearchEntryMode.MATCH);
Bundle.BundleEntryComponent bec = new Bundle.BundleEntryComponent();
Extension matchGradeExtension = besc.addExtension();
matchGradeExtension.setUrl("http://hl7.org/fhir/StructureDefinition/match-grade");
matchGradeExtension.setValue(new CodeType(MatchGrade.PROBABLE.toCode()));
bec.setResource(pat);
bec.setSearch(besc);
innerBundle.setEntry(Collections.singletonList(bec));
Json looks like this:
{
"resourceType": "Bundle",
"type": "searchset",
"total": 1,
"entry": [
{
"resource": {
"resourceType": "Patient",
"name": [
{
"family": "Smith"
}
]
},
"search": {
"extension": [
{
"url": "http://hl7.org/fhir/StructureDefinition/match-grade",
"valueCode": "certain"
}
],
"mode": "match",
"score": 0.88
}
}
]
}

Is it possible to assign public static IPs to individual VMs in Azure VMSS?

There is a document describing how to allocate a public IP per VM in VMSS: https://learn.microsoft.com/en-us/azure/virtual-machine-scale-sets/virtual-machine-scale-sets-networking#public-ipv4-per-virtual-machine
But it is not clear how to assign public static IP per VM. Is it possible?
Unfortunately, Azure does not provide control of the Public IP allocation method per instance in VMSS. You can see the all supported Properties of
VirtualMachineScaleSetPublicIPAddressConfigurationProperties object in the latest ARM API version.
However, after my validation, when you restart the instance or VMSS scale-in or scale-out, the public IP address of existing instances is not changed. The public IP address of instances will be updated unless you stop the instance of VMSS.
Update
Currently, you can manage it with IpPublicPrefix. Note that IpPublicPrefix requires a standard SKU load balancer and public IP address. Here is a working sample. You can check the public IP address of the instances in VMSS with the REST API.
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"vmSku": {
"type": "string",
"defaultValue": "Standard_A1_v2",
"metadata": {
"description": "Size of VMs in the VM Scale Set."
}
},
"windowsOSVersion": {
"type": "string",
"defaultValue": "2019-Datacenter",
"allowedValues": [
"2008-R2-SP1",
"2012-Datacenter",
"2012-R2-Datacenter",
"2016-Datacenter",
"2019-Datacenter"
],
"metadata": {
"description": "The Windows version for the VM. This will pick a fully patched image of this given Windows version. Allowed values: 2008-R2-SP1, 2012-Datacenter, 2012-R2-Datacenter & 2016-Datacenter, 2019-Datacenter."
}
},
"vmssName": {
"type": "string",
"minLength": 3,
"maxLength": 61,
"metadata": {
"description": "String used as a base for naming resources. Must be 3-61 characters in length and globally unique across Azure. A hash is prepended to this string for some resources, and resource-specific information is appended."
}
},
"instanceCount": {
"type": "int",
"defaultValue": 3,
"minValue": 1,
"maxValue": 100,
"metadata": {
"description": "Number of VM instances (100 or less)."
}
},
"singlePlacementGroup": {
"type": "bool",
"defaultValue": true,
"metadata": {
"description": "When true this limits the scale set to a single placement group, of max size 100 virtual machines. NOTE: If singlePlacementGroup is true, it may be modified to false. However, if singlePlacementGroup is false, it may not be modified to true."
}
},
"adminUsername": {
"type": "string",
"defaultValue": "vmssadmin",
"metadata": {
"description": "Admin username on all VMs."
}
},
"adminPassword": {
"type": "securestring",
"metadata": {
"description": "Admin password on all VMs."
}
},
"location": {
"type": "string",
"defaultValue": "[resourceGroup().location]",
"metadata": {
"description": "Location for all resources."
}
},
"platformFaultDomainCount": {
"type": "int",
"defaultValue": 1,
"metadata": {
"description": "Fault Domain count for each placement group."
}
},
"publicIPPrefixes_pubprefix_name": {
"defaultValue": "vmsspublicprefix",
"type": "string"
}
},
"variables": {
"namingInfix": "[toLower(substring(concat(parameters('vmssName'), uniqueString(resourceGroup().id)), 0, 9))]",
"longNamingInfix": "[toLower(parameters('vmssName'))]",
"addressPrefix": "10.0.0.0/16",
"subnetPrefix": "10.0.0.0/24",
"virtualNetworkName": "[concat(variables('namingInfix'), 'vnet')]",
"publicIPAddressName": "[concat(variables('namingInfix'), 'pip')]",
"subnetName": "[concat(variables('namingInfix'), 'subnet')]",
"loadBalancerName": "[concat(variables('namingInfix'), 'lb')]",
"publicIPAddressID": "[resourceId('Microsoft.Network/publicIPAddresses',variables('publicIPAddressName'))]",
"lbProbeID": "[resourceId('Microsoft.Network/loadBalancers/probes',variables('loadBalancerName'), 'tcpProbe')]",
"natPoolName": "[concat(variables('namingInfix'), 'natpool')]",
"bePoolName": "[concat(variables('namingInfix'), 'bepool')]",
"lbPoolID": "[resourceId('Microsoft.Network/loadBalancers/backendAddressPools',variables('loadBalancerName'),variables('bePoolName'))]",
"natStartPort": 50000,
"natEndPort": 50119,
"natBackendPort": 3389,
"nicName": "[concat(variables('namingInfix'), 'nic')]",
"ipConfigName": "[concat(variables('namingInfix'), 'ipconfig')]",
"frontEndIPConfigID": "[resourceId('Microsoft.Network/loadBalancers/frontendIPConfigurations',variables('loadBalancerName'),'loadBalancerFrontEnd')]",
"osType": {
"publisher": "MicrosoftWindowsServer",
"offer": "WindowsServer",
"sku": "[parameters('windowsOSVersion')]",
"version": "latest"
},
"imageReference": "[variables('osType')]"
},
"resources": [
{
"type": "Microsoft.Network/loadBalancers",
"apiVersion": "2020-06-01",
"name": "[variables('loadBalancerName')]",
"location": "[parameters('location')]",
"dependsOn": [
"[resourceId('Microsoft.Network/publicIPAddresses', variables('publicIPAddressName'))]"
],
"sku": {
"name": "Standard"
},
"properties": {
"frontendIPConfigurations": [
{
"name": "LoadBalancerFrontEnd",
"properties": {
"publicIPAddress": {
"id": "[variables('publicIPAddressID')]",
"name": "Standard"
}
}
}
],
"backendAddressPools": [
{
"name": "[variables('bePoolName')]"
}
],
"inboundNatPools": [
{
"name": "[variables('natPoolName')]",
"properties": {
"frontendIPConfiguration": {
"id": "[variables('frontEndIPConfigID')]"
},
"protocol": "Tcp",
"frontendPortRangeStart": "[variables('natStartPort')]",
"frontendPortRangeEnd": "[variables('natEndPort')]",
"backendPort": "[variables('natBackendPort')]"
}
}
],
"loadBalancingRules": [
{
"name": "LBRule",
"properties": {
"frontendIPConfiguration": {
"id": "[variables('frontEndIPConfigID')]"
},
"backendAddressPool": {
"id": "[variables('lbPoolID')]"
},
"protocol": "Tcp",
"frontendPort": 80,
"backendPort": 80,
"enableFloatingIP": false,
"idleTimeoutInMinutes": 5,
"probe": {
"id": "[variables('lbProbeID')]"
}
}
}
],
"probes": [
{
"name": "tcpProbe",
"properties": {
"protocol": "Tcp",
"port": 80,
"intervalInSeconds": 5,
"numberOfProbes": 2
}
}
]
}
},
{
"type": "Microsoft.Network/publicIPPrefixes",
"apiVersion": "2020-11-01",
"name": "[parameters('publicIPPrefixes_pubprefix_name')]",
"location": "[parameters('location')]",
"sku": {
"name": "Standard",
"tier": "Regional"
},
"properties": {
"prefixLength": 28,
"publicIPAddressVersion": "IPv4",
"ipTags": []
}
},
{
"type": "Microsoft.Compute/virtualMachineScaleSets",
"apiVersion": "2020-06-01",
"name": "[variables('namingInfix')]",
"location": "[parameters('location')]",
"sku": {
"name": "[parameters('vmSku')]",
"tier": "Standard",
"capacity": "[parameters('instanceCount')]"
},
"dependsOn": [
"[resourceId('Microsoft.Network/loadBalancers', variables('loadBalancerName'))]",
"[resourceId('Microsoft.Network/virtualNetworks', variables('virtualNetworkName'))]",
"[resourceId('Microsoft.Network/publicIPPrefixes',parameters('publicIPPrefixes_pubprefix_name'))]"
],
"properties": {
"overprovision": true,
"upgradePolicy": {
"mode": "Automatic"
},
"singlePlacementGroup": "[parameters('singlePlacementGroup')]",
"platformFaultDomainCount": "[parameters('platformFaultDomainCount')]",
"virtualMachineProfile": {
"storageProfile": {
"osDisk": {
"caching": "ReadWrite",
"createOption": "FromImage"
},
"imageReference": "[variables('imageReference')]"
},
"osProfile": {
"computerNamePrefix": "[variables('namingInfix')]",
"adminUsername": "[parameters('adminUsername')]",
"adminPassword": "[parameters('adminPassword')]"
},
"networkProfile": {
"networkInterfaceConfigurations": [
{
"name": "[variables('nicName')]",
"properties": {
"primary": true,
"ipConfigurations": [
{
"name": "[variables('ipConfigName')]",
"properties": {
"subnet": {
"id": "[resourceId('Microsoft.Network/virtualNetworks/subnets', variables('virtualNetworkName'), variables('subnetName'))]"
},
"loadBalancerBackendAddressPools": [
{
"id": "[variables('lbPoolID')]"
}
],
"loadBalancerInboundNatPools": [
{
"id": "[resourceId('Microsoft.Network/loadBalancers/inboundNatPools', variables('loadBalancerName'), variables('natPoolName'))]"
}
],
"publicipaddressconfiguration": {
"name": "pub1",
"properties": {
"idleTimeoutInMinutes": 15,
"publicIPAddressVersion": "IPv4",
"publicIPPrefix":{
"id": "[resourceId('Microsoft.Network/publicIPPrefixes',parameters('publicIPPrefixes_pubprefix_name'))]"
}
}
}
}
}
]
}
}
]
}
}
}
},
{
"type": "Microsoft.Network/publicIPAddresses",
"apiVersion": "2020-06-01",
"name": "[variables('publicIPAddressName')]",
"location": "[parameters('location')]",
"sku": {
"name": "Standard"
},
"properties": {
"publicIPAllocationMethod": "Static",
"dnsSettings": {
"domainNameLabel": "[variables('longNamingInfix')]"
}
}
},
{
"type": "Microsoft.Network/virtualNetworks",
"apiVersion": "2020-06-01",
"name": "[variables('virtualNetworkName')]",
"location": "[parameters('location')]",
"properties": {
"addressSpace": {
"addressPrefixes": [
"[variables('addressPrefix')]"
]
},
"subnets": [
{
"name": "[variables('subnetName')]",
"properties": {
"addressPrefix": "[variables('subnetPrefix')]"
}
}
]
}
},
{
"type": "Microsoft.Insights/autoscaleSettings",
"apiVersion": "2015-04-01",
"name": "autoscalehost",
"location": "[parameters('location')]",
"dependsOn": [
"[resourceId('Microsoft.Compute/virtualMachineScaleSets/', variables('namingInfix'))]"
],
"properties": {
"name": "autoscalehost",
"targetResourceUri": "[resourceId('Microsoft.Compute/virtualMachineScaleSets', variables('namingInfix'))]",
"enabled": true,
"profiles": [
{
"name": "Profile1",
"capacity": {
"minimum": "1",
"maximum": "10",
"default": "1"
},
"rules": [
{
"metricTrigger": {
"metricName": "Percentage CPU",
"metricResourceUri": "[resourceId('Microsoft.Compute/virtualMachineScaleSets', variables('namingInfix'))]",
"timeGrain": "PT1M",
"statistic": "Average",
"timeWindow": "PT5M",
"timeAggregation": "Average",
"operator": "GreaterThan",
"threshold": 50
},
"scaleAction": {
"direction": "Increase",
"type": "ChangeCount",
"value": "1",
"cooldown": "PT5M"
}
},
{
"metricTrigger": {
"metricName": "Percentage CPU",
"metricResourceUri": "[resourceId('Microsoft.Compute/virtualMachineScaleSets', variables('namingInfix'))]",
"timeGrain": "PT1M",
"statistic": "Average",
"timeWindow": "PT5M",
"timeAggregation": "Average",
"operator": "LessThan",
"threshold": 30
},
"scaleAction": {
"direction": "Decrease",
"type": "ChangeCount",
"value": "1",
"cooldown": "PT5M"
}
}
]
}
]
}
}
]
}

Ranking results based on field in nested array

Suppose I have an elasticsearch index called 'neighborhood' and in there I am storing documents ( example far below) with the following settings and mappings.
There are many nested fields in the model and so I am using copy_to on many of them to make searching easier - e.g. I just do a match query on the _nayberz field.
The search works quite well, however I would like for matches where the ratedTags[n].rating (it can be 1-5) is higher to rank higher
Mappings:
{
"properties": {
"_nayberz": {
"type": "text",
"analyzer": "autocomplete",
"store" : true
},
"describedTagCombos": {
"type": "nested",
"properties": {
"tags": {
"type": "nested",
"properties": {
"keywords": {
"type": "text",
"analyzer": "autocomplete",
"boost": 5,
"copy_to": "_nayberz"
},
"name": {
"type": "text",
"analyzer": "autocomplete",
"boost": 5,
"copy_to": "_nayberz"
},
"synonym": {
"type": "text",
"analyzer": "autocomplete",
"boost": 5,
"copy_to": "_nayberz"
}
}
}
}
},
"name": {
"type": "text",
"fielddata" : true,
"copy_to": "_nayberz",
"boost": 1
},
"ratedTags": {
"type": "nested",
"properties": {
"tags": {
"type": "nested",
"properties": {
"keywords": {
"type": "text",
"boost": 5,
"copy_to": "_nayberz"
},
"name": {
"type": "text",
"boost": 5,
"copy_to": "_nayberz"
},
"synonym": {
"type": "text",
"boost": 5,
"copy_to": "_nayberz"
}
}
}
}
}
}
}
{
"id": 6475,
"neighborhoodId": 2495,
"name": "Some neighborhood name",
"xMin": -87.7351229157221,
"xMax": -87.687849915678,
"xAvg": -87.7114864157001,
"yMin": 41.9316410223988,
"yMax": 41.9466920224128,
"yAvg": 41.9391665224058,
"city": {
"id": 539,
"name": "SomeCity",
"county": "SomeCounty",
"state": {
"id": 174,
"name": "Illinois",
"abbreviation": "IL",
"country": "USA"
}
},
"ratedTags": [
{
"id": 11572,
"rating": 2,
"tags": [
{
"id": 2323,
"tagId": 36,
"name": "shopping options",
"synonym": "",
"keywords": "shopping options, shopping, shop, shopper, shoppers, shops"
}
]
},
{
"id": 11418,
"rating": 3,
"tags": [
{
"id": 2292,
"tagId": 5,
"name": "public transportation options",
"synonym": "",
"keywords": "public transport, public transportation"
}
]
},
{
"id": 11434,
"rating": 4,
"tags": [
{
"id": 2295,
"tagId": 8,
"name": "quiet",
"synonym": "",
"keywords": "quiet, chill, peaceful, not noisy, not loud, not too noisy, not too loud, relaxed"
}
]
},
{
"id": 11458,
"rating": 3,
"tags": [
{
"id": 2300,
"tagId": 13,
"name": "expensive relative to other neighborhoods",
"synonym": "costly",
"keywords": "upscale, chic"
}
]
},
{
"id": 11469,
"rating": 4,
"tags": [
{
"id": 2302,
"tagId": 15,
"name": "restaurants",
"synonym": "",
"keywords": "restaurants, restaurant, eatery, eateries"
}
]
},
{
"id": 11477,
"rating": 2,
"tags": [
{
"id": 2304,
"tagId": 17,
"name": "clean",
"synonym": "",
"keywords": "clean, cleanest, not dirty"
}
]
},
{
"id": 11603,
"rating": 3,
"tags": [
{
"id": 2329,
"tagId": 42,
"name": "safe compared to other neighborhoods",
"synonym": "",
"keywords": "safe, safety, safest, not dangerous"
}
]
},
{
"id": 11557,
"rating": 2,
"tags": [
{
"id": 2320,
"tagId": 33,
"name": "green space and parks",
"synonym": "",
"keywords": "green space, green, parks, open space, nature"
}
]
},
{
"id": 11577,
"rating": 2,
"tags": [
{
"id": 2324,
"tagId": 37,
"name": "tourist attractions",
"synonym": "sightseeing options",
"keywords": "tourist attractions, sightseeing, attractions, sightsee, tourist"
}
]
},
{
"id": 11582,
"rating": 2,
"tags": [
{
"id": 2325,
"tagId": 38,
"name": "entertainment options",
"synonym": "fun options",
"keywords": "fun, entertainment, good time, enjoyment, enjoy, pleasure"
}
]
},
{
"id": 11588,
"rating": 3,
"tags": [
{
"id": 2326,
"tagId": 39,
"name": "cafes",
"synonym": "coffee shops",
"keywords": "cafes, coffee shops, cafe, coffee shop, coffee houses, coffee house, café, cafés"
}
]
},
{
"id": 11594,
"rating": 4,
"tags": [
{
"id": 2327,
"tagId": 40,
"name": "dining options",
"synonym": "",
"keywords": "dining places, dining spots, dining options, restaurants, food options, food places, places to eat, eateries, dining, fast food, chain restaurant, chain restaurants, eatery"
}
]
}
],
"describedTagCombos": [
{
"id": 26842,
"descriptor": "Foodies: must try a polish sausage in this neighborhood",
"tags": [
{
"id": 2302,
"tagId": 15,
"name": "restaurants",
"synonym": "",
"keywords": "restaurants, restaurant, eatery, eateries"
},
{
"id": 2327,
"tagId": 40,
"name": "dining options",
"synonym": "",
"keywords": "dining places, dining spots, dining options, restaurants, food options, food places, places to eat, eateries, dining, fast food, chain restaurant, chain restaurants, eatery"
}
]
},
{
"id": 26843,
"descriptor": "Addison Mall in area with a Target, Starbucks, etc.",
"tags": [
{
"id": 2323,
"tagId": 36,
"name": "shopping options",
"synonym": "",
"keywords": "shopping options, shopping, shop, shopper, shoppers, shops"
}
]
}
]
}
Settings:
{
"analysis": {
"filter": {
"autocomplete_filter": {
"type": "edge_ngram",
"min_gram": 3,
"max_gram": 20
},
"english_stop": {
"type": "stop",
"stopwords": [
"a",
"an",
"and",
"are",
"as",
"at",
"be",
"but",
"by",
"for",
"if",
"in",
"into",
"is",
"it",
"no",
"not",
"of",
"on",
"or",
"such",
"that",
"the",
"their",
"then",
"there",
"these",
"they",
"this",
"to",
"was",
"will",
"with",
"have"
]
}
},
"analyzer": {
"autocomplete": {
"type": "custom",
"tokenizer": "standard",
"filter": [
"lowercase",
"autocomplete_filter",
"english_stop"
]
}
}
}
}
I think the best candidate here is the field_value_factor function:
GET neighborhoods/_search
{
"query": {
"bool": {
"should": [
{
"query_string": {
"default_field": "_nayberz",
"query": "enjoy"
}
},
{
"nested": {
"path": "ratedTags",
"query": {
"function_score": {
"functions": [
{
"field_value_factor": { <--
"field": "ratedTags.rating",
"factor": 1
}
}
],
"boost_mode": "sum"
}
}
}
}
]
}
}
}

REQL to match string expression

I have the following json:
{
"release": {
"genres": {
"genre": "Electronic"
},
"identifiers": {
"identifier": [
{
"description": "Text",
"value": "5 709498 101026",
"type": "Barcode"
},
{
"description": "String",
"value": 5709498101026,
"type": "Barcode"
}
]
},
"status": "Accepted",
"videos": {
"video": [
{
"title": "Future 3 - Renaldo",
"duration": 446,
"description": "Future 3 - Renaldo",
"src": "http://www.youtube.com/watch?v=hpc9aQpnUjc",
"embed": true
},
{
"title": "Future 3 - Silver M from album We are the Future / 1995 Denmark / Archivos de Kraftwerkmusik",
"duration": 461,
"description": "Future 3 - Silver M from album We are the Future / 1995 Denmark / Archivos de Kraftwerkmusik",
"src": "http://www.youtube.com/watch?v=nlcHRI8iV4g",
"embed": true
},
{
"title": "Future 3 - Bubbles At Dawn",
"duration": 710,
"description": "Future 3 - Bubbles At Dawn",
"src": "http://www.youtube.com/watch?v=ABBCyvGMOFw",
"embed": true
}
]
},
"labels": {
"label": {
"catno": "APR 010CD",
"name": "April Records"
}
},
"companies": {
"company": {
"id": 26184,
"catno": "",
"name": "Voices Of Wonder",
"entity_type_name": "Published By",
"resource_url": "http://api.discogs.com/labels/26184",
"entity_type": 21
}
},
"styles": {
"style": [
"Abstract",
"IDM",
"Downtempo"
]
},
"formats": {
"format": {
"text": "",
"name": "CD",
"qty": 1,
"descriptions": {
"description": "Album"
}
}
},
"country": "Denmark",
"id": 5375,
"released": "1995-00-00",
"artists": {
"artist": {
"id": 5139,
"anv": "",
"name": "Future 3",
"role": "",
"tracks": "",
"join": ""
}
},
"title": "We Are The Future 3",
"master_id": 638422,
"tracklist": {
"track": [
{
"position": 1,
"duration": "8:04",
"title": "Future 3"
},
{
"position": 2,
"duration": "7:38",
"title": "Silver M"
},
{
"position": 3,
"duration": "7:27",
"title": "Renaldo"
},
{
"position": 4,
"duration": "6:04",
"title": "B.O.Y.D."
},
{
"position": 5,
"duration": "6:12",
"title": "Fumble"
},
{
"position": 6,
"duration": "6:12",
"title": "Dawn"
},
{
"position": 7,
"duration": "11:54",
"title": "Bubbles At Dawn"
},
{
"position": 8,
"duration": "6:03",
"title": "D.A.W.N. At 6"
},
{
"position": 9,
"duration": "8:50",
"title": 4684351684651
}
]
},
"data_quality": "Needs Vote",
"extraartists": {
"artist": [
{
"id": 2647642,
"anv": "",
"name": "Danesadwork",
"role": "Cover",
"tracks": "",
"join": ""
},
{
"id": 2647647,
"anv": "",
"name": "Djon Edvard Petersen",
"role": "Photography By",
"tracks": "",
"join": ""
},
{
"id": 114164,
"anv": "",
"name": "Anders Remmer",
"role": "Written-By",
"tracks": "",
"join": ""
},
{
"id": 435979,
"anv": "",
"name": "Jesper Skaaning",
"role": "Written-By",
"tracks": "",
"join": ""
},
{
"id": 15691,
"anv": "",
"name": "Thomas Knak",
"role": "Written-By",
"tracks": "",
"join": ""
}
]
},
"notes": "© 1995 April Records APS ℗ 1995 April Records APS"
}
}
I am trying to get those titles which end with 'At Dawn'.
I am using the following command
r.db("discogs1").table("releases").filter(function(doc){ return doc('release')('title').match('At Dawn$')})
But I get errors as follows:
RqlRuntimeError: Expected type STRING but found NUMBER in:r.db("discogs1").table("releases").filter(function(var_24) { return var_24("release")("title").match("At Dawn$"); })
I tried different combinations but I can't seem to get it to work
It seems that some of your documents don't have a row('release')('title') property that is a string. Some of them are numbers, so when you try to call .match on them, they throw an error because .match only works on strings.
To see if this is true, try the following:
r.db("discogs1").table("releases")
.filter(r.row('release')('title').typeOf().ne('STRING'))
.count()
Ideally, the result of this should be 0, since no document should have a title property that's not a string. If it's higher than 0, that's why you're getting an error.
If you want to only get documents where the title is a string, you can do the following:
r.db("discogs1").table("releases")
.filter(r.row('release')('title').typeOf().eq('STRING'))
.filter(function(doc){ return doc('release')('title').match('At Dawn$')})
This query will work, because it will filter our all documents where the title is not a string.
If you want to coerce all title into strings, you can do the following:
r.db("discogs1").table("releases")
.filter(r.row('release')('title').typeOf().ne('STRING'))
.merge(function (row) {
return {
'title': row('title').coerceTo('string')
}
})
If you want to delete all documents where the title is not a string, you can do this:
r.db("discogs1").table("releases")
.filter(r.row('release')('title').typeOf().ne('STRING'))
.delete()

Resources