I currently have an ES query to find the nearest location to a lat/long:
GET /geo/_search
{
"sort": [
{
"_geo_distance": {
"geometry": {
"lat": 64,
"lon": 34
},
"order": "asc",
"unit": "mi",
"distance_type": "plane"
}
}
],
"size": 1
}
I want to be able to run this in 1 query for multiple lat/longs, which would return each lat/long related to their nearest location. Is there some way to do this?
GET /geo/_search
{
"sort": [
{
"_geo_distance": {
"geometry": [
{
"lat": 64,
"lon": 34
},
{
"lat": 0,
"lon": 0
}
],
"order": "asc",
"unit": "mi",
"distance_type": "plane"
}
}
],
"size": 1
}
This answer gives more than one point per geo_point. I don't think you can retrieve only the top one per geo_point in one query. You might need to filter the results or to use a loop per each geo_point
Related
today I would like to add geo based sorting to my query.
I use Elastic (8.6) Enterprise Search / App Search.
My request body:
{
"query": "",
"filters": {
"location": {
"center": "51.071646,6.3195429",
"distance": 500,
"unit": "km"
}
},
"sort": [
{
"_geo_distance": {
"location": [
51.071646,
6.3195429
],
"order": "asc",
"mode": "min",
"distance_type": "plane",
"ignore_unmapped": true
}
}
],
"page": {
"size": 20,
"current": 1
}
}
... and I get the following response body:
{
"errors": [
"Sort contains invalid field: _geo_distance"
]
}
My document field location is set to geolocation in schema.
Can anyone give me a hint about what I fundamentally do wrong here?
Without that 'sort' property the search performs as intended, but I would like to have the distances in relation to the requested location in the response, too.
Thanks a lot!
For those facing similar situation, I applied the wrong syntax!
As I am in Elastic App Search, it has to be
{
"query": "",
"filters": {
"location": {
"center": "51.071646,6.3195429",
"distance": 500,
"unit": "km"
}
},
"sort": [
{
"location": {
"center": "51.071646,6.3195429",
"order": "asc"
}
}
],
"page": {
"size": 20,
"current": 1
}
}
I have the following query - which works fine (this might not be the actual query):
{
"query": {
"bool": {
"must": [
{
"nested": {
"path": "location",
"query": {
"geo_distance": {
"distance": "16090km",
"distance_type": "arc",
"location.point": {
"lat": "51.794177",
"lon": "-0.063055"
}
}
}
}
},
{
"geo_distance": {
"distance": "16090km",
"distance_type": "arc",
"location.point": {
"lat": "51.794177",
"lon": "-0.063055"
}
}
}
]
}
}
}
Although I want to do the following (as part of the query but not affecting the existing query):
Find all documents that have field_name = 1
On all documents that have field_name = 1 run ordering by geo_distance
Remove duplicates that have field_name = 1 and the same value under field_name_2 = 2 and leave the closest item in the documents result, but remove the rest
Update (further explanation):
Aggregations can't be used as we want to manipulate the documents in the result.
Whilst also maintaining the order within the documents; meaning:
If I have 20 documents, sorted by a field; and I have 5 of which have field_name = 1, I would like to sort the 5 by distance, and eliminate 4 of them; whilst still maintaining the first sort. (possibly doing the geodistance sort and elimination before the actual query?)
Not too sure how to do this, any help is appreciated - I'm currently using ElasticSearch DSL DRF - but I can easily convert the query to ElasticSearch DSL.
Example documents (before manipulation):
[{
"field_name": 1,
"field_name_2": 2,
"location": ....
},
{
"field_name": 1,
"field_name_2": 2,
"location": ....
},
{
"field_name": 55,
"field_name_5": 22,
"location": ....
}]
Output (Desired):
[{
"field_name": 1,
"field_name_2": 2,
"location": .... <- closest
},
{
"field_name": 55,
"field_name_5": 22,
"location": ....
}]
One way to achieve what you want is to keep the query part as you have it now (so you still get the hits you need) and add an aggregation part in order to get the closest document with an additional condition on filed_name. The aggregation part would be made of:
a filter aggregation to only consider the documents with field_name = 1
a geo_distance aggregation with a very small distance
a top_hits aggregation to return the document with the closest distance
The aggregation part would look like this:
{
"query": {
...same as you have now...
},
"aggs": {
"field_name": {
"filter": {
"term": {
"field_name": 1 <--- only select desired documents
}
},
"aggs": {
"geo_distance": {
"field": "location.point",
"unit": "km",
"distance_type": "arc",
"origin": {
"lat": "51.794177",
"lon": "-0.063055"
},
"ranges": [
{
"to": 1 <---- single bucket for docs < 1km (change as needed)
}
]
},
"aggs": {
"closest": {
"top_hits": {
"size": 1, <---- closest document
"sort": [
{
"_geo_distance": {
"location.point": {
"lat": "51.794177",
"lon": "-0.063055"
},
"order": "asc",
"unit": "km",
"mode": "min",
"distance_type": "arc",
"ignore_unmapped": true
}
}
]
}
}
}
}
}
}
}
This can be done using Field Collapsing - which is the equivalent of grouping. - Below is an example of how this can be achieved:
{"collapse": {"field": "vin",
"inner_hits": {
"name": "closest_dealer",
"size": 1,
"sort": [
{
"_geo_distance": {
"location.point": {
"lat": "latitude",
"lon": "longitude"
},
"order": "desc",
"unit": "km",
"distance_type": "arc",
"nested_path": "location"
}
}
]
}
}
}
The collapsing is done on the field vin - and the inner_hits is used to sort the grouped items and get the closest one. (size = 1)
I have data in this format generated from a random walk (to simulate people walking around). It is set up in this manner { location : { lat: someLat, lon: someLong }, id: uniqueId, date:date }. I am trying to write a query given a users unique ID, find how many other unique IDs came within X distance of the given ID between a certain time range. Any hints on how to accomplish this?
My idea is to have a top level filter aggregration, with a nested geo-query of some sort. I think the geo-distance query is the way to go, but I am not sure how to include it into the below query to get all of unique IDs that come within X distance of the ID I am filtering on. The query below is where I am starting from, I am filtering all documents from now - 1 day to now, where the documents user Id is the provided value. How would I check all other documents for their distances against documents that match this query?
{
"aggs" : {
"range": {
"date_range": {
"field": "date",
"format": "MM-yyyy",
"ranges": [
{ "to": "now" },
{ "from": "now-1d" }
]
}
},
"locations" : {
"filter" : {
"term": { "id.keyword": "7a50ab18-886b-42a2-80ad-3d45112e3cfd" }
}
}
}
}
Your hunch is correct. All of this can be done using range & geo_distance filtering and _geo_distance sorting. You wanna filter on the query-level, not in the aggs though:
GET walking/_search
{
"size": 0,
"query": {
"bool": {
"must": [
{
"range": {
"date": {
"gte": "now-1d"
}
}
}
],
"filter": [
{
"geo_distance": {
"distance": "20m",
"location": {
"lat": 48.20150179951008,
"lon": 16.39111876487732
}
}
}
]
}
},
"aggs": {
"rings_around_loc": {
"geo_distance": {
"field": "location",
"origin": {
"lat": 48.20150179951008,
"lon": 16.39111876487732
},
"unit": "m",
"keyed": true,
"ranges": [
{
"to": 10
},
{
"from": 10,
"to": 50
},
{
"from": 50
}
]
}
},
"locations": {
"value_count": {
"field": "id.keyword"
}
}
},
"sort": [
{
"_geo_distance": {
"location": {
"lat": 48.20150179951008,
"lon": 16.39111876487732
},
"order": "asc",
"unit": "m",
"mode": "min",
"distance_type": "arc",
"ignore_unmapped": true
}
}
]
}
Not sure what you need the range buckets for so I left them out.
Full steps to replicate:
PUT walking
{
"mappings": {
"properties": {
"date": {
"type": "date"
},
"id": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
},
"location": {
"type": "geo_point"
}
}
}
}
And then POST _bulk this random walk data
I'm creating a completion suggester with a geo context (Elastic 5.x).
mapping...
"suggest": {
"type": "completion",
...
"contexts": [
{
"name": "geoloc",
"type": "geo",
"precision": 3,
"path": "geolocation"
}
]
When I query this, I'd like to have it not filter by the geo context, only boost results that are within the geohash. It works great to filter by a single geohash, or filter by a lower precision, and then boost a higher precision within that original filter like this:
GET /my-index/_search
{
"suggest": {
...
"completion": {
"field": "suggest",
"size": "10",
"contexts": {
"geoloc": [
{
"lat": 44.8214564,
"lon": -93.475399,
"precision": 1
},
{
"lat": 44.8214564,
"lon": -93.475399,
"boost": 2
}
]
}
}
}
}
However, I can't get it to only boost on a single geo context without filtering.
When I submit the following query, it filters and boosts:
GET /my-index/_search
{
"suggest": {
...
"completion": {
"field": "suggest",
"size": "10",
"contexts": {
"geoloc": [
{
"lat": 44.8214564,
"lon": -93.475399,
"boost": 2
}
]
}
}
}
}
Is what I'm trying to do just not supported, or am I missing something?
Thanks!
Jason
Just ran into this issue as well.
The solution I came up with through trial and error was to use the category context to filter first to all my documents. Say you had added a category to your documents named "all" you could do this:
GET /my-index/_search
{
"suggest": {
...
"completion": {
"field": "suggest",
"size": "10",
"contexts": {
"category": ["all"],
"geoloc": [
{
"lat": 44.8214564,
"lon": -93.475399,
"precision": 2,
"boost": 2
}
]
}
}
}
}
When this is done, it seems to be selecting everything with the "all" category and then boosts the ones within the precision level specified to the top.
Using Elastic 6.*
It doesn't seem possible based on https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html , but I'd like confirmation.
In plain English, I'm asking to score the results (with geo-point locations) by how close they are to 500km from some latitude, longitude origin.
It's confusing because there is a parameter called "offset" but according to the documentation it doesn't seem to be an offset from origin (eg. distance) but instead seems to mean "threshold" instead.
I see a few ways to accomplish this:
A. One way would be to simply sort by distance in reverse order from the origin. You'd use a geo_distance query and then sort by distance. In the following query, the most distant documents will come up first, i.e. the sort value is the distance from the origin and we're sorting in decreasing order.
{
"query": {
"filtered": {
"filter": {
"geo_distance": {
"from" : "100km",
"to" : "200km",
"location": {
"lat": 10,
"lon": 20
}
}
}
}
},
"sort": [
{
"_geo_distance": {
"location": {
"lat": 10,
"lon": 20
},
"order": "desc",
"unit": "km",
"distance_type": "plane"
}
}
]
}
B. The second way involves using a geo_distance_range query in order to define a "ring" around the origin. The width of that ring could somehow symbolize the offset + scale you'd use in a gauss function (although there would be no decay). Here we define a ring that is 10km wide at 500km distance from the origin point and sort the documents by distance in that ring.
{
"query": {
"filtered": {
"filter": {
"geo_distance_range": {
"from": "495km",
"to": "505km",
"location": {
"lat": 10,
"lon": 20
}
}
}
}
},
"sort": [
{
"_geo_distance": {
"location": {
"lat": 10,
"lon": 20
},
"order": "desc",
"unit": "km",
"distance_type": "plane"
}
}
]
}
C. The last way is a bit more involved. We're basically after an "inverse gauss" shape, basically this figure (33), but upside-down, or this one which better represents the donut shape we're after. We can combine solution B above with a gauss function that would only score within that ring. In the query below, we're basically saying that we're only interested in the locations around 500km from the origin and we let a gauss function kick in only for those documents. It's not perfect, though, but might be close enough to what you need.
{
"query": {
"filtered": {
"filter": {
"geo_distance_range": {
"from": "495km",
"to": "505km",
"location": {
"lat": 10,
"lon": 20
}
}
},
"query": {
"function_score": {
"functions": [
{
"gauss": {
"location": {
"origin": {
"lat": 10,
"lon": 20
},
"offset": "500km",
"scale": "5km"
}
}
}
]
}
}
}
},
"sort": {
"_score": "desc"
}
}