Related
Using Elastic version 7.15.1
{
"mappings": {
"properties": {
"Activity": {
"type": "nested",
"properties": {
"Data": {
"type": "text"
},
"Type": {
"type": "keyword"
},
"created_at": {
"type": "date"
},
"updated_at": {
"type": "date"
}
}
},
"FirstName": {
"type": "text",
"analyzer": "standard_autocomplete",
"search_analyzer": "standard_autocomplete_search"
}
}
}
}
Example Data
{
"Activity": [
{
"Type": "type1",
"Data": "data",
"created_at": "2022-08-08T15:23:58.000000Z"
},
{
"Type": "type1",
"Data": "data",
"created_at": "2022-08-08T15:25:45.000000Z"
},
{
"Type": "type2",
"Data": "data",
"created_at": "2022-08-08T15:26:03.000000Z"
}
],
"FirstName": "Testtt"
}
Want this document to return only if "Activity.Type" is "type1" and the count of the "type1" is greater than 1.
Also how can we use created_at in nested array with above constraint
There is a document describing how to allocate a public IP per VM in VMSS: https://learn.microsoft.com/en-us/azure/virtual-machine-scale-sets/virtual-machine-scale-sets-networking#public-ipv4-per-virtual-machine
But it is not clear how to assign public static IP per VM. Is it possible?
Unfortunately, Azure does not provide control of the Public IP allocation method per instance in VMSS. You can see the all supported Properties of
VirtualMachineScaleSetPublicIPAddressConfigurationProperties object in the latest ARM API version.
However, after my validation, when you restart the instance or VMSS scale-in or scale-out, the public IP address of existing instances is not changed. The public IP address of instances will be updated unless you stop the instance of VMSS.
Update
Currently, you can manage it with IpPublicPrefix. Note that IpPublicPrefix requires a standard SKU load balancer and public IP address. Here is a working sample. You can check the public IP address of the instances in VMSS with the REST API.
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"vmSku": {
"type": "string",
"defaultValue": "Standard_A1_v2",
"metadata": {
"description": "Size of VMs in the VM Scale Set."
}
},
"windowsOSVersion": {
"type": "string",
"defaultValue": "2019-Datacenter",
"allowedValues": [
"2008-R2-SP1",
"2012-Datacenter",
"2012-R2-Datacenter",
"2016-Datacenter",
"2019-Datacenter"
],
"metadata": {
"description": "The Windows version for the VM. This will pick a fully patched image of this given Windows version. Allowed values: 2008-R2-SP1, 2012-Datacenter, 2012-R2-Datacenter & 2016-Datacenter, 2019-Datacenter."
}
},
"vmssName": {
"type": "string",
"minLength": 3,
"maxLength": 61,
"metadata": {
"description": "String used as a base for naming resources. Must be 3-61 characters in length and globally unique across Azure. A hash is prepended to this string for some resources, and resource-specific information is appended."
}
},
"instanceCount": {
"type": "int",
"defaultValue": 3,
"minValue": 1,
"maxValue": 100,
"metadata": {
"description": "Number of VM instances (100 or less)."
}
},
"singlePlacementGroup": {
"type": "bool",
"defaultValue": true,
"metadata": {
"description": "When true this limits the scale set to a single placement group, of max size 100 virtual machines. NOTE: If singlePlacementGroup is true, it may be modified to false. However, if singlePlacementGroup is false, it may not be modified to true."
}
},
"adminUsername": {
"type": "string",
"defaultValue": "vmssadmin",
"metadata": {
"description": "Admin username on all VMs."
}
},
"adminPassword": {
"type": "securestring",
"metadata": {
"description": "Admin password on all VMs."
}
},
"location": {
"type": "string",
"defaultValue": "[resourceGroup().location]",
"metadata": {
"description": "Location for all resources."
}
},
"platformFaultDomainCount": {
"type": "int",
"defaultValue": 1,
"metadata": {
"description": "Fault Domain count for each placement group."
}
},
"publicIPPrefixes_pubprefix_name": {
"defaultValue": "vmsspublicprefix",
"type": "string"
}
},
"variables": {
"namingInfix": "[toLower(substring(concat(parameters('vmssName'), uniqueString(resourceGroup().id)), 0, 9))]",
"longNamingInfix": "[toLower(parameters('vmssName'))]",
"addressPrefix": "10.0.0.0/16",
"subnetPrefix": "10.0.0.0/24",
"virtualNetworkName": "[concat(variables('namingInfix'), 'vnet')]",
"publicIPAddressName": "[concat(variables('namingInfix'), 'pip')]",
"subnetName": "[concat(variables('namingInfix'), 'subnet')]",
"loadBalancerName": "[concat(variables('namingInfix'), 'lb')]",
"publicIPAddressID": "[resourceId('Microsoft.Network/publicIPAddresses',variables('publicIPAddressName'))]",
"lbProbeID": "[resourceId('Microsoft.Network/loadBalancers/probes',variables('loadBalancerName'), 'tcpProbe')]",
"natPoolName": "[concat(variables('namingInfix'), 'natpool')]",
"bePoolName": "[concat(variables('namingInfix'), 'bepool')]",
"lbPoolID": "[resourceId('Microsoft.Network/loadBalancers/backendAddressPools',variables('loadBalancerName'),variables('bePoolName'))]",
"natStartPort": 50000,
"natEndPort": 50119,
"natBackendPort": 3389,
"nicName": "[concat(variables('namingInfix'), 'nic')]",
"ipConfigName": "[concat(variables('namingInfix'), 'ipconfig')]",
"frontEndIPConfigID": "[resourceId('Microsoft.Network/loadBalancers/frontendIPConfigurations',variables('loadBalancerName'),'loadBalancerFrontEnd')]",
"osType": {
"publisher": "MicrosoftWindowsServer",
"offer": "WindowsServer",
"sku": "[parameters('windowsOSVersion')]",
"version": "latest"
},
"imageReference": "[variables('osType')]"
},
"resources": [
{
"type": "Microsoft.Network/loadBalancers",
"apiVersion": "2020-06-01",
"name": "[variables('loadBalancerName')]",
"location": "[parameters('location')]",
"dependsOn": [
"[resourceId('Microsoft.Network/publicIPAddresses', variables('publicIPAddressName'))]"
],
"sku": {
"name": "Standard"
},
"properties": {
"frontendIPConfigurations": [
{
"name": "LoadBalancerFrontEnd",
"properties": {
"publicIPAddress": {
"id": "[variables('publicIPAddressID')]",
"name": "Standard"
}
}
}
],
"backendAddressPools": [
{
"name": "[variables('bePoolName')]"
}
],
"inboundNatPools": [
{
"name": "[variables('natPoolName')]",
"properties": {
"frontendIPConfiguration": {
"id": "[variables('frontEndIPConfigID')]"
},
"protocol": "Tcp",
"frontendPortRangeStart": "[variables('natStartPort')]",
"frontendPortRangeEnd": "[variables('natEndPort')]",
"backendPort": "[variables('natBackendPort')]"
}
}
],
"loadBalancingRules": [
{
"name": "LBRule",
"properties": {
"frontendIPConfiguration": {
"id": "[variables('frontEndIPConfigID')]"
},
"backendAddressPool": {
"id": "[variables('lbPoolID')]"
},
"protocol": "Tcp",
"frontendPort": 80,
"backendPort": 80,
"enableFloatingIP": false,
"idleTimeoutInMinutes": 5,
"probe": {
"id": "[variables('lbProbeID')]"
}
}
}
],
"probes": [
{
"name": "tcpProbe",
"properties": {
"protocol": "Tcp",
"port": 80,
"intervalInSeconds": 5,
"numberOfProbes": 2
}
}
]
}
},
{
"type": "Microsoft.Network/publicIPPrefixes",
"apiVersion": "2020-11-01",
"name": "[parameters('publicIPPrefixes_pubprefix_name')]",
"location": "[parameters('location')]",
"sku": {
"name": "Standard",
"tier": "Regional"
},
"properties": {
"prefixLength": 28,
"publicIPAddressVersion": "IPv4",
"ipTags": []
}
},
{
"type": "Microsoft.Compute/virtualMachineScaleSets",
"apiVersion": "2020-06-01",
"name": "[variables('namingInfix')]",
"location": "[parameters('location')]",
"sku": {
"name": "[parameters('vmSku')]",
"tier": "Standard",
"capacity": "[parameters('instanceCount')]"
},
"dependsOn": [
"[resourceId('Microsoft.Network/loadBalancers', variables('loadBalancerName'))]",
"[resourceId('Microsoft.Network/virtualNetworks', variables('virtualNetworkName'))]",
"[resourceId('Microsoft.Network/publicIPPrefixes',parameters('publicIPPrefixes_pubprefix_name'))]"
],
"properties": {
"overprovision": true,
"upgradePolicy": {
"mode": "Automatic"
},
"singlePlacementGroup": "[parameters('singlePlacementGroup')]",
"platformFaultDomainCount": "[parameters('platformFaultDomainCount')]",
"virtualMachineProfile": {
"storageProfile": {
"osDisk": {
"caching": "ReadWrite",
"createOption": "FromImage"
},
"imageReference": "[variables('imageReference')]"
},
"osProfile": {
"computerNamePrefix": "[variables('namingInfix')]",
"adminUsername": "[parameters('adminUsername')]",
"adminPassword": "[parameters('adminPassword')]"
},
"networkProfile": {
"networkInterfaceConfigurations": [
{
"name": "[variables('nicName')]",
"properties": {
"primary": true,
"ipConfigurations": [
{
"name": "[variables('ipConfigName')]",
"properties": {
"subnet": {
"id": "[resourceId('Microsoft.Network/virtualNetworks/subnets', variables('virtualNetworkName'), variables('subnetName'))]"
},
"loadBalancerBackendAddressPools": [
{
"id": "[variables('lbPoolID')]"
}
],
"loadBalancerInboundNatPools": [
{
"id": "[resourceId('Microsoft.Network/loadBalancers/inboundNatPools', variables('loadBalancerName'), variables('natPoolName'))]"
}
],
"publicipaddressconfiguration": {
"name": "pub1",
"properties": {
"idleTimeoutInMinutes": 15,
"publicIPAddressVersion": "IPv4",
"publicIPPrefix":{
"id": "[resourceId('Microsoft.Network/publicIPPrefixes',parameters('publicIPPrefixes_pubprefix_name'))]"
}
}
}
}
}
]
}
}
]
}
}
}
},
{
"type": "Microsoft.Network/publicIPAddresses",
"apiVersion": "2020-06-01",
"name": "[variables('publicIPAddressName')]",
"location": "[parameters('location')]",
"sku": {
"name": "Standard"
},
"properties": {
"publicIPAllocationMethod": "Static",
"dnsSettings": {
"domainNameLabel": "[variables('longNamingInfix')]"
}
}
},
{
"type": "Microsoft.Network/virtualNetworks",
"apiVersion": "2020-06-01",
"name": "[variables('virtualNetworkName')]",
"location": "[parameters('location')]",
"properties": {
"addressSpace": {
"addressPrefixes": [
"[variables('addressPrefix')]"
]
},
"subnets": [
{
"name": "[variables('subnetName')]",
"properties": {
"addressPrefix": "[variables('subnetPrefix')]"
}
}
]
}
},
{
"type": "Microsoft.Insights/autoscaleSettings",
"apiVersion": "2015-04-01",
"name": "autoscalehost",
"location": "[parameters('location')]",
"dependsOn": [
"[resourceId('Microsoft.Compute/virtualMachineScaleSets/', variables('namingInfix'))]"
],
"properties": {
"name": "autoscalehost",
"targetResourceUri": "[resourceId('Microsoft.Compute/virtualMachineScaleSets', variables('namingInfix'))]",
"enabled": true,
"profiles": [
{
"name": "Profile1",
"capacity": {
"minimum": "1",
"maximum": "10",
"default": "1"
},
"rules": [
{
"metricTrigger": {
"metricName": "Percentage CPU",
"metricResourceUri": "[resourceId('Microsoft.Compute/virtualMachineScaleSets', variables('namingInfix'))]",
"timeGrain": "PT1M",
"statistic": "Average",
"timeWindow": "PT5M",
"timeAggregation": "Average",
"operator": "GreaterThan",
"threshold": 50
},
"scaleAction": {
"direction": "Increase",
"type": "ChangeCount",
"value": "1",
"cooldown": "PT5M"
}
},
{
"metricTrigger": {
"metricName": "Percentage CPU",
"metricResourceUri": "[resourceId('Microsoft.Compute/virtualMachineScaleSets', variables('namingInfix'))]",
"timeGrain": "PT1M",
"statistic": "Average",
"timeWindow": "PT5M",
"timeAggregation": "Average",
"operator": "LessThan",
"threshold": 30
},
"scaleAction": {
"direction": "Decrease",
"type": "ChangeCount",
"value": "1",
"cooldown": "PT5M"
}
}
]
}
]
}
}
]
}
Im using ES 1.7, trying to look for documents using match_phrase to search using exact match string. The filter works fine when used alone, however when I combine the filters, I get an error.
example: people document
q=aaron&address=scarborough - searching a person by name and address, works fine.
{
"query": {
"match_phrase": {
"name": "aaron"
}
},
"filter": {
"bool": {
"must": {
"nested": {
"path": "addresses",
"query": {
"match_phrase": {
"address": "scarborough"
}
}
}
}
}
},
q=aaron&phone=813-689-6889 - searching a person by name and phone number works fine as well.
{
"query": {
"match_phrase": {
"name": "aaron"
}
},
"filter": {
"bool": {
"must": {
"query": {
"match_phrase": {
"phone": "813-689-6889"
}
}
}
}
}
However, When I try to use both filters, address and phone I get a No filter registered for [match_phrase] error
for example: q=aaron&address=scarborough&phone=813-689-6889
{
"query": {
"match_phrase": {
"name": "aaron"
}
},
"filter": {
"bool": {
"must": {
"nested": {
"path": "addresses",
"query": {
"match_phrase": {
"address": "scarborough"
}
}
},
"query": {
"match_phrase": {
"phone": "813-689-6889"
}
}
}
}
}
the error, when using address and phone filters together:
nested: QueryParsingException[[pl_people] No filter registered for [match_phrase]]; }]","status":400}):
index mapping (person):
{
"pl_people": {
"mappings": {
"person": {
"properties": {
"ac_name": {
"type": "string",
"analyzer": "autocomplete"
},
"addresses": {
"type": "nested",
"properties": {
"address": {
"type": "string"
},
"city": {
"type": "string",
"index": "not_analyzed"
},
"city_id": {
"type": "long"
},
"country": {
"type": "string",
"index": "not_analyzed"
},
"county": {
"type": "string",
"index": "not_analyzed"
},
"county_id": {
"type": "long"
},
"id": {
"type": "long"
},
"location": {
"type": "geo_point"
},
"parameterized": {
"type": "string"
},
"state": {
"type": "string",
"index": "not_analyzed"
},
"state_id": {
"type": "long"
},
"zip": {
"type": "string",
"index": "not_analyzed"
}
}
},
"author": {
"type": "string",
"index": "not_analyzed"
},
"body": {
"type": "string",
"analyzer": "remove_html",
"fields": {
"ns_body": {
"type": "string",
"analyzer": "remove_html_stopwords"
}
}
},
"charities": {
"type": "nested",
"properties": {
"email": {
"type": "string",
"index": "not_analyzed"
},
"id": {
"type": "long"
}
}
},
"community": {
"properties": {
"name": {
"type": "string",
"index": "not_analyzed"
},
"parameterized": {
"type": "string",
"index": "not_analyzed"
},
"slug": {
"type": "string",
"index": "not_analyzed"
}
}
},
"created_at": {
"type": "date",
"format": "dateOptionalTime"
},
"date_of_birth": {
"type": "date",
"format": "dateOptionalTime"
},
"delimiters": {
"type": "nested",
"properties": {
"delimiter_type": {
"type": "string",
"index": "not_analyzed"
},
"id": {
"type": "long"
}
}
},
"description": {
"type": "string"
},
"employments": {
"type": "nested",
"properties": {
"email": {
"type": "string",
"index": "not_analyzed"
},
"employment_status": {
"type": "string",
"index": "not_analyzed"
},
"foia_contact": {
"type": "boolean"
},
"id": {
"type": "long"
},
"phone": {
"type": "string",
"index": "not_analyzed"
},
"phone_extension": {
"type": "string",
"index": "not_analyzed"
}
}
},
"first_name": {
"type": "string",
"fields": {
"na_first_name": {
"type": "string",
"index": "not_analyzed"
}
}
},
"last_name": {
"type": "string",
"fields": {
"na_last_name": {
"type": "string",
"index": "not_analyzed"
}
}
},
"market": {
"properties": {
"name": {
"type": "string",
"index": "not_analyzed"
},
"parameterized": {
"type": "string",
"index": "not_analyzed"
},
"slug": {
"type": "string",
"index": "not_analyzed"
}
}
},
"middle_name": {
"type": "string",
"fields": {
"na_middle_name": {
"type": "string",
"index": "not_analyzed"
}
}
},
"most_recent_organization": {
"properties": {
"description": {
"type": "string"
},
"id": {
"type": "long"
},
"name": {
"type": "string"
},
"parameterized": {
"type": "string"
},
"phone": {
"type": "string"
}
}
},
"name": {
"type": "string",
"fields": {
"na_name": {
"type": "string",
"index": "not_analyzed"
},
"ngram_name": {
"type": "string",
"analyzer": "my_start"
},
"ns_name": {
"type": "string",
"analyzer": "no_stopwords"
}
}
},
"organizations": {
"properties": {
"name": {
"type": "string"
},
"parameterized": {
"type": "string",
"index": "not_analyzed"
}
}
},
"package": {
"properties": {
"name": {
"type": "string",
"index": "not_analyzed"
},
"parameterized": {
"type": "string",
"index": "not_analyzed"
},
"slug": {
"type": "string",
"index": "not_analyzed"
}
}
},
"parameterized": {
"type": "string",
"index": "not_analyzed"
},
"phone": {
"type": "string"
},
"photo": {
"properties": {
"large": {
"type": "string"
},
"medium": {
"type": "string"
},
"teaser": {
"type": "string"
},
"thumb": {
"type": "string"
},
"url": {
"type": "string"
}
}
},
"projects": {
"properties": {
"id": {
"type": "long"
},
"name": {
"type": "string",
"index": "not_analyzed"
},
"parameterized": {
"type": "string",
"index": "not_analyzed"
},
"slug": {
"type": "string",
"index": "not_analyzed"
}
}
},
"public_offices": {
"type": "nested",
"properties": {
"email": {
"type": "string",
"index": "not_analyzed"
},
"employment_status": {
"type": "string",
"index": "not_analyzed"
},
"id": {
"type": "long"
}
}
},
"published": {
"type": "string",
"index": "not_analyzed"
},
"region": {
"properties": {
"name": {
"type": "string",
"index": "not_analyzed"
},
"parameterized": {
"type": "string",
"index": "not_analyzed"
},
"slug": {
"type": "string",
"index": "not_analyzed"
}
}
},
"resource": {
"type": "string"
},
"short_description": {
"type": "string"
},
"show_path": {
"type": "string"
},
"time": {
"type": "date",
"format": "dateOptionalTime"
},
"updated_at": {
"type": "date",
"format": "dateOptionalTime"
}
}
}
}
}
}
Document I am using to test
[
{
"_index": "pl_people",
"_type": "person",
"_id": "813106",
"_score": null,
"sort": [
-9223372036854775808
],
"resource": "Person",
"parameterized": "813106-aaron-mcguire",
"created_at": "2011-10-29T19:51:24.000-05:00",
"updated_at": "2014-12-11T07:21:08.000-06:00",
"name": "Aaron McGuire",
"title": null,
"photo": {
"url": "/assets/140x140.gif"
},
"description": null,
"short_description": null,
"time": "2014-12-11",
"show_path": "/people/813106-aaron-mcguire",
"published": true,
"aliases": [],
"phone": "813-689-6889",
"date_of_birth": "1991-03-01",
"first_name": "Aaron",
"middle_name": "",
"last_name": "McGuire",
"delimiters": [],
"market": null,
"region": null,
"most_recent_organization": null,
"households": [],
"court_cases": [],
"addresses": [
{
"id": 1,
"parameterized": "1",
"address": "123 Scarborough road",
"zip": "L5A2A9",
"city": "Ontario",
"country": "USA",
"state": "California",
"location": null,
"state_id": null,
"county_id": null,
"city_id": null
}
],
"projects": [],
"voter_ids": [],
"id": "813106"
}
]
Finally, I was able to reproduce the issue, Looks like "include_in_parent": true, missed in your mapping, due to which I was getting the error mentioned In my comment..
To fix, the issue I used the same mapping provided you but add "include_in_parent": true, at the top level of address nested field. For brevity providing only the address part of modified mapping.
"addresses": {
"type": "nested",
"include_in_parent": true, --> added only this in your mapping.
"properties": {
"address": {
"type": "string"
},
"city": {
"type": "string",
"index": "not_analyzed"
},
"city_id": {
"type": "long"
},
"country": {
"type": "string",
"index": "not_analyzed"
},
"county": {
"type": "string",
"index": "not_analyzed"
},
"county_id": {
"type": "long"
},
"id": {
"type": "long"
},
"location": {
"type": "geo_point"
},
"parameterized": {
"type": "string"
},
"state": {
"type": "string",
"index": "not_analyzed"
},
"state_id": {
"type": "long"
},
"zip": {
"type": "string",
"index": "not_analyzed"
}
}
}
After that indexed some sample docs, which have different values of address and name param, to verify the right outcome of my search query.
And final search query which includes both the filter, please note, I removed the nested part on address field as it's included at the parent document, so you can directly query now on it. This query looks much simple and now gives the expected result.
{
"query": {
"match_phrase": {
"name": "aaron"
}
},
"filter": {
"bool": {
"must": [{
"query": {
"match_phrase": {
"address": "scarborough"
}
}
},
{
"query": {
"match_phrase": {
"phone": "813-689-6889"
}
}
}]
}
}
}
Output on sample docs
"hits": [
{
"_index": "so-match-phrase",
"_type": "pl_people",
"_id": "4",
"_score": 0.19178301,
"_source": {
"resource": "Person",
"parameterized": "813106-aaron-mcguire",
"created_at": "2011-10-29T19:51:24.000-05:00",
"updated_at": "2014-12-11T07:21:08.000-06:00",
"name": "aaron McGuire",
"title": null,
"photo": {
"url": "/assets/140x140.gif"
},
"description": null,
"short_description": null,
"time": "2014-12-11",
"show_path": "/people/813106-aaron-mcguire",
"published": true,
"aliases": [],
"phone": "813-689-6889",
"date_of_birth": "1991-03-01",
"first_name": "Aaron",
"middle_name": "",
"last_name": "McGuire",
"delimiters": [],
"market": null,
"region": null,
"most_recent_organization": null,
"households": [],
"court_cases": [],
"addresses": [
{
"id": 1,
"parameterized": "1",
"address": "Scarborough road",
"zip": "L5A2A9",
"city": "Ontario",
"country": "USA",
"state": "California",
"location": null,
"state_id": null,
"county_id": null,
"city_id": null
}
],
"id": "813106"
}
},
{
"_index": "so-match-phrase",
"_type": "pl_people",
"_id": "1",
"_score": 0.19178301,
"_source": {
"resource": "Person",
"parameterized": "813106-aaron-mcguire",
"created_at": "2011-10-29T19:51:24.000-05:00",
"updated_at": "2014-12-11T07:21:08.000-06:00",
"name": "Aaron McGuire",
"title": null,
"photo": {
"url": "/assets/140x140.gif"
},
"description": null,
"short_description": null,
"time": "2014-12-11",
"show_path": "/people/813106-aaron-mcguire",
"published": true,
"aliases": [],
"phone": "813-689-6889",
"date_of_birth": "1991-03-01",
"first_name": "Aaron",
"middle_name": "",
"last_name": "McGuire",
"delimiters": [],
"market": null,
"region": null,
"most_recent_organization": null,
"households": [],
"court_cases": [],
"addresses": [
{
"id": 1,
"parameterized": "1",
"address": "123 Scarborough road",
"zip": "L5A2A9",
"city": "Ontario",
"country": "USA",
"state": "California",
"location": null,
"state_id": null,
"county_id": null,
"city_id": null
}
],
"id": "813106"
}
}
]
Similar to hive querying records for a specific uniontype
I have data on s3 in avro format and following is the avro structure:
{
"type": "record",
"name": "Event",
"namespace": "com.company.avro.event",
"fields": [
{
"name": "content",
"type": [
{
"type": "record",
"name": "Follow",
"fields": [
{
"name": "content",
"type": [
{
"type": "record",
"name": "UserFollowBrand",
"fields": [
{
"name": "id",
"type": "string"
},
{
"name": "actor",
"type": "com.company.avro.entity.User"
},
{
"name": "verb",
"type": "string",
"default": "UserFollowBrand"
},
{
"name": "direct_object",
"type": "com.company.avro.entity.Brand"
},
{
"name": "on",
"type": [
"com.company.avro.type.IoSScreen",
"com.company.avro.type.AndroidScreen",
"null"
]
},
{
"name": "using",
"type": "com.company.avro.entity.App"
},
{
"name": "from",
"type": "string"
},
{
"name": "at",
"type": "long"
}
]
},
{
"type": "record",
"name": "UserFollowUser",
"fields": [
{
"name": "id",
"type": "string"
},
{
"name": "actor",
"type": "com.company.avro.entity.User"
},
{
"name": "verb",
"type": "string",
"default": "UserFollowUser"
},
{
"name": "direct_object",
"type": "com.company.avro.entity.User"
},
{
"name": "on",
"type": [
"com.company.avro.type.IoSScreen",
"com.company.avro.type.AndroidScreen",
"null"
]
},
{
"name": "using",
"type": "com.company.avro.entity.App"
},
{
"name": "from",
"type": "string"
},
{
"name": "at",
"type": "long"
}
]
}
]
}
]
},
{
"type": "record",
"name": "Like",
"fields": [
{
"name": "content",
"type": [
{
"type": "record",
"name": "UserLikeListing",
"fields": [
{
"name": "id",
"type": "string"
},
{
"name": "actor",
"type": "com.company.avro.entity.User"
},
{
"name": "verb",
"type": "string",
"default": "UserLikeListing"
},
{
"name": "direct_object",
"type": "com.company.avro.entity.Listing"
},
{
"name": "on",
"type": [
"com.company.avro.type.IoSScreen",
"com.company.avro.type.AndroidScreen",
"com.company.avro.type.WebScreen",
"null"
]
},
{
"name": "using",
"type": "com.company.avro.entity.App"
},
{
"name": "from",
"type": "string"
},
{
"name": "at",
"type": "long"
}
]
}
]
}
]
}
]
}
]
}
I am not sure how can I query for specific field within the uniontype.
For ex: select * from events where content.verb = "a" and content.actor.id = 34
Earlier hive did not support union types but now it seems it does support https://issues.apache.org/jira/browse/HIVE-2390
Unable to figure out how to use create_union function to query this.
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+Types#LanguageManualTypes-UnionTypes
Running the command:
curl -XGET http://127.0.0.1:9200/30556/_search -d '{
"query": {
"constant_score" : {
"filter" : {
"term" : { "portal_type" : "Folder"}
}
}
}
}'
yields 0 results. The output is:
{"took":1,"timed_out":false,"_shards":{"total":5,"successful":5,"failed":0},"hits":{"total":0,"max_score":null,"hits":[]}}
If fact, I can't get any queries to yield results.
However, when I run the same query using the head plugin, it works fine.
I'm on elasticsearch 0.20.2 on Mac OS X. I'm starting elastic search using the command:
bin/elasticsearch -f
Is there something obvious I'm missing? Seems like I have the correct syntax and I don't get any errors.
Mapping:
{
"30556": {
"portal_catalog": {
"properties": {
"CreationDate": {
"type": "date",
"format": "dateOptionalTime"
},
"Creator": {
"type": "string"
},
"Date": {
"type": "date",
"format": "dateOptionalTime"
},
"Description": {
"type": "string"
},
"ModificationDate": {
"type": "date",
"format": "dateOptionalTime"
},
"SearchableText": {
"type": "string"
},
"Title": {
"type": "string"
},
"Type": {
"type": "string"
},
"UID": {
"type": "string"
},
"allowedRolesAndUsers": {
"type": "string"
},
"created": {
"type": "date",
"format": "dateOptionalTime"
},
"effective": {
"type": "date",
"format": "dateOptionalTime"
},
"effectiveRange": {
"dynamic": "true",
"properties": {
"effectiveRange1": {
"type": "date",
"format": "dateOptionalTime"
},
"effectiveRange2": {
"type": "date",
"format": "dateOptionalTime"
}
}
},
"exclude_from_nav": {
"type": "boolean"
},
"expires": {
"type": "date",
"format": "dateOptionalTime"
},
"getId": {
"type": "string"
},
"getObjPositionInParent": {
"type": "long"
},
"getObjSize": {
"type": "string"
},
"id": {
"type": "string"
},
"is_default_page": {
"type": "boolean"
},
"is_folderish": {
"type": "boolean"
},
"listCreators": {
"type": "string"
},
"meta_type": {
"type": "string"
},
"modified": {
"type": "date",
"format": "dateOptionalTime"
},
"object_provides": {
"type": "string"
},
"path": {
"dynamic": "true",
"properties": {
"depth": {
"type": "long"
},
"path": {
"type": "string"
}
}
},
"portal_type": {
"type": "string"
},
"review_state": {
"type": "string"
},
"sortable_title": {
"type": "string"
},
"total_comments": {
"type": "long"
}
}
}
}
}
Example Indexed Document:
{
"_index": "30556",
"_type": "portal_catalog",
"_id": "30613",
"_score": 1,
"_source": {
"sortable_title": "news",
"exclude_from_nav": false,
"meta_type": "ATFolder",
"Date": "2013-01-14T09:24:56-06:00",
"CreationDate": "2013-01-14T09:24:56-06:00",
"path": {
"depth": 2,
"path": "/el/news"
},
"allowedRolesAndUsers": [
"Anonymous"
],
"portal_type": "Folder",
"id": "news",
"UID": "3116b6c7ec384a9393f238fdde778612",
"expires": "2499-12-31T00:00:00-06:00",
"Subject": [],
"is_folderish": true,
"is_default_page": false,
"effectiveRange": {
"effectiveRange1": "1000-01-01T00:00:00-06:00",
"effectiveRange2": "2499-12-31T00:00:00-06:00"
},
"commentators": [],
"created": "2013-01-14T09:24:56-06:00",
"getRawRelatedItems": [],
"cmf_uid": [],
"Creator": "admin",
"end": [],
"modified": "2013-01-14T09:24:56-06:00",
"Description": "Site News",
"ModificationDate": "2013-01-14T09:24:56-06:00",
"total_comments": 0,
"in_reply_to": [],
"getIcon": "",
"effective": "1000-01-01T00:00:00-06:00",
"SearchableText": "news News Site News ",
"getObjPositionInParent": 61,
"object_provides": [
"collective.syndication.interfaces.ISyndicatable",
"Products.ATContentTypes.interfaces.folder.IATFolder",
"Products.CMFCore.interfaces._content.IContentish",
"z3c.relationfield.interfaces.IHasIncomingRelations",
"webdav.interfaces.IWriteLock"
],
"last_comment_date": null,
"review_state": "published",
"start": [],
"Type": "Folder",
"listCreators": [
"admin"
],
"getId": "news",
"getObjSize": "1 kB",
"Title": "News"
}
Try to use lower case index names.
Does it work?
If not, can you provide your indexed document and mapping if any?
UPDATE:
You use default analyzer so your field is broken into tokens which are lowercased.
A TermFilter is not analyzed So it does not match.
You can lowercase you TermFilter or use a MatchQuery which is analyzed or change your mapping and set the field to not_analyzed.