Elasticsearch custom geo distance filter - elasticsearch

From an Elasticsearch query I'd like to retrieve all the points within a variable distance.
Let say I have 2 shops, one is willing to deliver at maximum 3 km and the other one at maximum 5 km:
PUT /my_shops/_doc/1
{
"location": {
"lat": 40.12,
"lon": -71.34
},
"max_delivery_distance": 3000
}
PUT /my_shops/_doc/2
{
"location": {
"lat": 41.12,
"lon": -72.34
},
"max_delivery_distance": 5000
}
For a given location I'd like to know which shops are able to deliver. IE query should return shop1 if given location is within 3km and shop2 if given location is within 5km
GET /my_shops/_search
{
"query": {
"bool": {
"must": {
"match_all": {}
},
"filter": {
"geo_distance": {
"distance": max_delivery_distance,
"location": {
"lat": 40,
"lon": -70
}
}
}
}
}
}

There's another way to solve this without scripting (big performance hogger !!) and let ES sort it out using native Geo shapes.
I would model each document as a circle, with a center location and a (delivery) radius. First, your index mapping should look like this:
PUT /my_shops
{
"mappings": {
"properties": {
"delivery_area": {
"type": "geo_shape",
"strategy": "recursive"
}
}
}
}
Then, your documents then need to have the following form:
PUT /my_shops/_doc/1
{
"delivery_area" : {
"type" : "circle",
"coordinates" : [-71.34, 40.12],
"radius" : "3000m"
}
}
PUT /my_shops/_doc/2
{
"delivery_area" : {
"type" : "circle",
"coordinates" : [-72.34, 41.12],
"radius" : "5000m"
}
}
And finally the query simply becomes a geo_shape query looking at intersections between a delivery point and the delivery area of each shop.
GET /my_shops/_search
{
"query": {
"bool": {
"filter": {
"geo_shape": {
"delivery_area": {
"shape": {
"type": "point",
"coordinates": [ -70, 40 ]
},
"relation": "contains"
}
}
}
}
}
}
That's it! No scripting, just geo operations.

I think that you need to work with a script to use another field as parameter. After some research I come to this answer:
GET my_shops/_search
{
"query": {
"script": {
"script": {
"params": {
"location": {
"lat": 40,
"lon": -70
}
},
"source": """
return doc['location'].arcDistance(params.location.lat, params.location.lon)/1000 <= doc['max_delivery_distance'].value"""
}
}
}
}
Basically, we exploit the fact that the classes related to the GEO points are whitelisted in painless https://github.com/elastic/elasticsearch/pull/40180/ and that scripts accepts additional parameters (your fixed location).
According to the documentation of arcDistance we retrieve the size in meters, so you need to convert this value into km by dividing by 1000.
Additional Note
I assume that location and max_delivery_distance are always (for each document) defined. If it is not the case, you need to cover this case.
Reference
Another related question
https://github.com/elastic/elasticsearch/pull/40180/

Related

Elasticsearch query documents where two locations have relative distance

I can't achieve to only get documents with a maximum distance of 1km between two geopoints of the same document.
I have simple documents like this :
{
start_location: {
lat: 34.0583,
lon: -118.2476
},
end_location: {
lat: 33.989521,
lon: -117.531614
}
}
I want to get all the documents where start_location is located less than 1km from end_location.
Im stuck with this for a while, thank you in advance !
You can use the arcDistance function within a script:
PUT geo
{
"mappings": {
"properties": {
"start_location": {
"type": "geo_point"
},
"end_location": {
"type": "geo_point"
}
}
}
}
POST geo/_doc
{
"start_location": {
"lat": 34.0583,
"lon": -118.2476
},
"end_location": {
"lat": 33.989521,
"lon": -117.531614
}
}
GET geo/_search
{
"query": {
"script": {
"script": """
def distance_in_m = doc['start_location'].arcDistance(
doc['end_location'].getLat(),
doc['end_location'].getLon()
);
return distance_in_m < 1000"""
}
}
}

ElasticSearch - Increase distance if no results

Would it be possible to create query that would increase the distance field when no results are found?
Here's an example of a query that will return whether there's a match or not:
{
"query": {
"bool": {
"filter": {
"geo_distance": {
"distance": "100km",
"location": {
"lat": 40,
"lon": -7
}
}
}
}
}
}
But I would like that Elasticsearch could keep looking until at least 1 item is found.
I could increase the value programmatically and make a new query with "distance": "200km" and so on... until I find something, but I would like to make a query that would do that automatically.
Elasticsearch version: 6.2

Looking for someone to help me with ElasticSearch

I'm beginner in ElasticSearch. I'm trying to test if a list of geopoint (lat / long ) is existing in a list of geopoints.
For example I give this geopoint :
"lat": 49.01536940596998
"lon": 2.4967825412750244
and I want to test if this point exist in the list below. Thanks.
"positions": [
{
"millis": 12959023,
"lat": 49.01525113731623,
"lon": 2.4971945118159056,
"rawX": -3754,
"rawY": 605,
"rawVx": 0,
"rawVy": 0,
"speed": 9.801029291617944,
"accel": 0.09442740907572084,
"grounded": true
},
{
"millis": 12959914,
"lat": 49.01536940596998,
"lon": 2.4967825412750244,
"rawX": -3784,
"rawY": 619,
"rawVx": -15,
"rawVy": 7,
"speed": 10.841861737855924,
"accel": -0.09534648619563282,
"grounded": true
}
...
}
To be able to search in an array of objects, you need to use the nested data type. As the linked page explains, to keep the internal elements of the array as independent, you cannot use the default mapping. First, you will have to update the mapping.
Note: Mappings only take effect on new indexes. Reference.
PUT YOUR_INDEX
{
"mappings": {
"YOUR_TYPE": {
"properties": {
"positions": {
"type": "nested"
}
}
}
}
}
Now we can query the data. You're looking for a bool query, which combines other queries (in your case, term queries).
POST _search
{
"query": {
"nested": {
"path": "positions",
"query": {
"bool" : {
"must" : [
{ "term" : { "lat": 49.01536940596998 } },
{ "term" : { "lon": 2.4967825412750244 } }
]
}
}
}
}
}

Find matching locations / distances without using scripting in Elasticsearch?

I'm using Elasticsearch to store user locations and their distance preference when finding other users. This is stored in a location geo_point and a distance integer.
For example, the index contains these documents:
Alice, located at [0,100] and looking for users within 100 meters;
Bob, located at [100,0] and looking for users within 50 meters.
When Carlos, located at [0,0], searches within 100 meters I need my query to return Alice, but not Bob (since Bob only wants users within 50m, and Carlos is 100m away).
In other words, I want to return all documents D such that D.reach contains Carlos.location and Carlos.reach contains D.location.
As far as I can see, the only way to do this is by comparing the distances with scripting like so:
{
"filter": {
"script": {
"script": "min(doc['distance'].value, distance) >= doc['location'].arcDistance(lat, lon)",
"params": {
"distance": 100,
"lat": 0,
"lon": 0
}
}
}
}
However, I'd rather avoid scripting if at all possible. Is there an alternative method to achieve this?
Another way worth investigating would be using a geo_shape circle. So instead of (or in addition to) storing discrete values for location and distance, you could store a combination of those two values as a circle representing the reach of the user. In your mapping, it would look like this:
{
"properties": {
"reach": {
"type": "geo_shape",
"tree": "quadtree",
"precision": "10cm"
}
}
}
Then when you index your document, you'd specify the reach circle like this:
{
"name": "Alice",
"reach" : {
"type" : "circle",
"coordinates" : [0.0, 100.0], <---- Alice's current location field
"radius" : "100m" <---- Alice's current distance field
}
}
{
"name": "Bob",
"reach" : {
"type" : "circle",
"coordinates" : [100.0, 0.0], <---- Bob's current location field
"radius" : "50m" <---- Bob's current distance field
}
}
At this point, all your users will have a geo_shape associated to them representing their reach. Now you can unleash the power of ES geo queries and filters in order to find intersections or what have you, for instance by using the geo_shape filter. The idea is to filter on another geo_shape representing the reach of the user who is searching other users (e.g. Carlos above)
{
"query":{
"filtered": {
"filter": {
"geo_shape": {
"location": {
"shape": {
"type": "circle",
"coordinates" : [0.0, 0.0] <--- Carlos location
"radius": "100m" <--- Carlos reach
}
}
}
}
}
}
}
The above query will find all documents (i.e. users) whose reach intersects the Carlos' reach specified in the filter. Give it a shot.
Thanks to Val's answer pointing me in the right direction, I used the following solution.
Documents look like this, containing users' location as geo_point and reach as a geo_shape.
{
"name": "Alice",
"location" : [1,0],
"reach" : {
"type": "shape",
"coordinates": [1,0],
"radius": 100
}
}
The query then contains two filters; one for matching Carlos' location inside the users' reach, and another for matching the user's location inside the Carlos' reach.
{
"filter": {
"and" : [
{
"geo_shape": {
"preferences.reach": {
"shape": {
"type": "Point",
"coordinates": Carlos.location
}
}
}
},
{
"geo_distance": {
"distance": Carlos.distance,
"user.location" : Carlos.location
}
}
]
}
}
This could be done with two geo_shapes but geo_points are more performant.

Accessing nested property in Elasticsearch distance script.

My index in elastic search has the following mapping:
"couchbaseDocument": {
"properties": {
"doc": {
"properties": {
"properties": {
"properties": {
"location": {
"type": "geo_point"
The source document is as follows:
{"properties" : {"location":"43.706596,-79.4030464"}}
I am trying to use the distance script to calculate the distance based on geo-points. I found this post Return distance in elasticsearch results? to help me out. I am trying to get all results,filter by radius 1km, get the distance, and sort on geo_point. The query is constructed as follows:
{
"query": {
"match_all": {}
},
"filter": {
"geo_distance": {
"distance": "1km",
"doc.properties.location": {
"lat": 43.710323,
"lon": -79.395284
}
}
},
"script_fields": {
"distancePLANE": {
"params": {
"lat": 43.710323,
"lon": -79.395284
},
"script": "doc[properties]['location'].distanceInKm(lat, lon)"
},
"distanceARC" :{
"params": {
"lat": 43.710323,
"lon": -79.395284
},
"script": "doc[properties]['location'].arcDistanceInKm(lat,lon)"
}
},
"sort": [
{
"_geo_distance":{
"doc.properties.location": [-79.395284,43.710323],
"order": "desc",
"unit": "km"
}
}
],
"track_scores": true
}
I get the following error with status 500:
"PropertyAccessException[[Error: could not access: properties; in class: org.elasticsearch.search.lookup.DocLookup]\n[Near : {... doc[properties]['location'].distan ....}]\n ^\n[Line: 1, Column: 5]]"
I tried rewriting the query in this way:
..."script": "doc['properties']['location'].arcDistanceInKm(lat,lon)"...
Then I get this error:
"CompileException[[Error: No field found for [properties] in mapping with types [couchbaseDocument]]\n[Near : {... doc['properties']['location']. ....}]\n ^\n[Line: 1, Column: 1]]; nested: ElasticSearchIllegalArgumentException[No field found for [properties] in mapping with types [couchbaseDocument]]; "
When I remove the script part from the query all together, the sorting and filtering works just fine. Is there a different way to access nested fields when using scripts? Any insights would be really appreciated!
Thank you!
Managed to get it done with
"script" : "doc.last_location.distance(41.12, -71.34)"
Don't know why but doc['last_location'] does not seem to work at all!
As mentioned in my comment when you sort by _geo_distance the "_sort" field that is returned, is the actual distance. So there is no need to do a separate computation. Details here: http://elasticsearch-users.115913.n3.nabble.com/search-by-distance-and-getting-the-actual-distance-td3317140.html#a3936224

Resources