I'm using Elasticsearch to query a document type, that has an optional location field. When searching, if that field does not exist, those results should be returned, as well as filtering on the results that do.
It seems like the OR filter in Elasticsearch does not short circuit, as this:
"query": {
"filtered": {
"query": {
"match_phrase_prefix": {
"display_name": "SearchQuery"
}
},
"filter": {
"or": [
{
"missing": {
"field": "location"
}
},
{
"geo_distance" : {
"distance" : "20mi",
"location" : {
"lat" : 33.47,
"lon" : -112.07
}
}
}
]
}
Fails with "failed to find geo_point field [location]".
Is there any way to perform this (or something along the same vein) in ES?
I don't know why yours isn't working but I've used the bool filter with great success in the past. The should option is essentially an or and makes sure at least one is true. Give it a try and comment on my answer if it still doesn't work. Also double check I copied your query terms properly :)
{
"filtered" : {
"query" : {
"match_phrase_prefix": {
"display_name": "SearchQuery"
}
},
"filter" : {
"bool" : {
"should" : [
{
"missing": { "field": "location" }
},
{
"geo_distance" : {
"distance" : "20mi",
"location" : {
"lat" : 33.47,
"lon" : -112.07
}
}
}
]
}
}
}
}
For anyone with the same issue, I kind of just hacked around it. For any documents that were missing a "location", I added one with a lat/lon of 0/0. Then I altered my query to be:
"filter": {
"or": [
{
"geo_distance": {
"distance": "0.1mi",
"location": {
"lat": 0,
"lon": 0
}
}
},
{
"geo_distance": {
"distance": "30mi",
"location": {
"lat": [lat variable],
"lon": [lon variable]
}
}
}
]
}
Related
I'm trying to implement the Multi-Term Auto Completion that's presented here.
Filtering down to the correct documents works, but when aggregating the completion_terms they are not filtered to those that match the current partial query, but instead include all completion_terms from any matched documents.
Here are the mappings:
{
"mappings": {
"dynamic" : "false",
"properties" : {
"completion_ngrams" : {
"type" : "text",
"analyzer" : "completion_ngram_analyzer",
"search_analyzer" : "completion_ngram_search_analyzer"
},
"completion_terms" : {
"type" : "keyword",
"normalizer" : "completion_normalizer"
}
}
}
}
Here are the settings:
{
"settings" : {
"index" : {
"analysis" : {
"filter" : {
"edge_ngram" : {
"type" : "edge_ngram",
"min_gram" : "1",
"max_gram" : "10"
}
},
"normalizer" : {
"completion_normalizer" : {
"filter" : [
"lowercase",
"german_normalization"
],
"type" : "custom"
}
},
"analyzer" : {
"completion_ngram_search_analyzer" : {
"filter" : [
"lowercase"
],
"tokenizer" : "whitespace"
},
"completion_ngram_analyzer" : {
"filter" : [
"lowercase",
"edge_ngram"
],
"tokenizer" : "whitespace"
}
}
}
}
}
}
}
I'm then indexing data like this:
{
"completion_terms" : ["Hammer", "Fortis", "Tool", "2000"],
"completion_ngrams": "Hammer Fortis Tool 2000"
}
Finally, the autocomplete search looks like this:
{
"query": {
"bool": {
"must": [
{
"term": {
"completion_terms": "fortis"
}
},
{
"term": {
"completion_terms": "hammer"
}
},
{
"match": {
"completion_ngrams": "too"
}
}
]
}
},
"aggs": {
"autocomplete": {
"terms": {
"field": "completion_terms",
"size": 100
}
}
}
}
This correctly returns documents matching the search string "fortis hammer too", but the aggregations include ALL completion terms that are included in any of the matched documents, e.g. for the query above:
"buckets": [
{ "key": "fortis" },
{ "key": "hammer" },
{ "key": "tool" },
{ "key": "2000" },
]
Ideally, I'd expect
"buckets": [
{ "key": "tool" }
]
I could filter out the terms that are already covered by the search query ("fortis" and "hammer" in this case) in the app, but the "2000" doesn't make any sense from a user's perspective, because it doesn't partially match any of the provided search terms.
I understand why this is happening, but I can't think of a solution. Can anyone help?
try filters agg please
{
"query": {
"bool": {
"must": [
{
"term": {
"completion_terms": "fortis"
}
},
{
"term": {
"completion_terms": "hammer"
}
},
{
"match": {
"completion_ngrams": "too"
}
}
]
}
},
"aggs": {
"findOuthammerAndfortis": {
"filters": {
"filters": {
"fortis": {
"term": {
"completion_terms": "fortis"
}
},
"hammer": {
"term": {
"completion_terms": "hammer"
}
}
}
}
}
}
}
I have got a DOC 'company' and a TYPE 'user'. Inside 'user' there is a field called 'locations' and this is an array. Every location has a field called 'point' which is a GeoPoint as a string "40.748770,-73.985487". How can I get the points nearby this point?
This example I'm showing bellow is not working:
GET /company/user/_search
{
"query": {
"bool" : {
"must" : {
"match_all" : {}
},
"filter" : {
"geo_distance" : {
"distance" : "500m",
"locations.point" : "40.748770, -73.985487"
}
}
}
}
}
Below example might help you,
GET /company/user/_search
{
"query": {
"filtered": {
"filter": {
"geo_distance": {
"distance": "1km",
"location": {
"lat": 40.715,
"lon": -73.988
}
}
}
}
}
}
Looking at the query below, I've added a cached geo_bounding_box filter in front of my geo_shape filter. My expectation after reading https://www.elastic.co/guide/en/elasticsearch/guide/current/geo-caching.html was that this query should be faster. However, in my benchmarking the query with both filters turns out to be slightly slower on average, and MUCH slower in the worst case. Am I doing something wrong, or misinterpreting the doc?
{
"query": {
"filtered": {
"filter": {
"bool" : {
"must" : [
{"geo_bounding_box" : {
"_cache": True,
"properties.center" : {
"top_left" : {
"lat" : math.ceil(float(lat)),
"lon" : math.floor(float(lon))
},
"bottom_right" : {
"lat" : math.floor(float(lat)),
"lon" : math.ceil(float(lon))
}
}
}},
{"geo_shape": {
"geometry": {
"relation": "intersects",
"shape": {
"coordinates": [lon,lat],
"type": "point"
}
}
}}
]
}
}
}
}
}
Use lowercase JSON boolean values:
"_cache": true
My mapping contains a nested field like this:
"Locations": {
"type": "nested",
"properties": {
"Name": {
"type": "string"
},
"GeoPoint": {
"type": "geo_point"
}
}
}
So basically what I'm trying to do is store some additional attributes with every location of the document.
Unfortunately, it looks like this won't work with a geo distance filter:
GET /myIndex/myType/_search
{
"filter": {
"geo_distance": {
"distance": "100 mi",
"Locations.GeoPoint": {
lat: 40.70,
lon: -74.00
}
}
}
}
won't return the any results, whereas the filter works flawlessly if the GeoPoint is directly on the document itself instead of on the nested field:
GET /myIndex/myType/_search
{
"filter": {
"geo_distance": {
"distance": "100 mi",
"GeoPoint": {
lat: 40.70,
lon: -74.00
}
}
}
}
Is there any way to make geo distance filter work with geo_point on the nested field?
A nested filter does the job:
GET /myIndex/myType/_search
{
"filter" : {
"nested" : {
"path" : "Locations",
"filter" : {
"geo_distance": {
"distance": "100 mi",
"GeoPoint": {
"lat": 40.70,
"lon": -74.00
}
}
}
}
}
}
An update for ElasticSearch 5.5 users, #Max's answer has been deprecated for the Nested Filter and Nested Query feature using the dot notation instead of explicitly wrapping a "nested" field around your query.
For example:
{
"query": {
"nested" : {
"path" : "obj1",
"score_mode" : "avg",
"query" : {
"bool" : {
"must" : [
{ "match" : {"obj1.name" : "blue"} },
{ "range" : {"obj1.count" : {"gt" : 5}} }
]
}
}
}
}
}
More information: https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-nested-filter.html
I'm trying to setup a geo_point object on ES 1.0.0 and run a simple proof of concept query against it but the query is failing to return any hits. Here are my setup steps:
1) Create the mapping:
PUT jay/geotest/_mapping
{
"geotest" : {
"properties" : {
"name" : {
"type" : "string"
},
"pin" : {
"type": "geo_point"
}
}
}
}
2) verify the mapping:
GET jay/geotest/_mapping
3) Add a piece of data
put jay/geotest/1
{
"name": "test1",
"pin": {
"lat": 0,
"lon": 0
}
}
4) query for that data:
GET jay/geotest/_search?search_type=count
{
"filtered" : {
"filter" : {
"geo_distance" : {
"distance" : "100km",
"pin" : {
"lat" : 0,
"lon" : 0
}
}
}
}
}
My expected result is that I will get one hit returned but instead nothing is returned by the query.
Thanks in advance!
I think you're missing the "query" part of the request.
POST jay/geotest/_search
{
"query": {
"filtered": {
"filter": {
"geo_distance": {
"distance": "100km",
"pin": {
"lat": 0,
"lon": 0
}
}
}
}
}
}
I've just tested your steps, and making that change returns the document.